As companies continue to ramp up vaccine mandates to combat the contagious delta variant, some institutions are giving employees a chance to opt-out of getting the vaccine if they have a medical exemption.
"Other than age, there are no major exemptions that cover large groups of people," he told ABC News.
The current guidance from the U.S. Centers for Disease Control and Prevention, said the two-dose mRNA vaccines and the one dose Johnson & Johnson vaccine are safe for almost all patients.
The only major contraindication to the vaccines listed by the CDC is a severe allergic reaction to the first dose. In those cases, the person is advised to consult a physician and hold off on their second dose, according to Dowdy.
"We're not talking about some people who had pain at the site of injection or a rash, we're talking about anaphylactic shock," he said.
Dowdy said the data so far shows this severe allergy is rare, and less than one in 1 million people experience it.
Dr. Jeff Linder, the chief of general internal medicine and geriatrics at Northwestern University Feinberg School of Medicine, told ABC News that research so far shows that those who have a severe allergic reaction are likely triggered by polyethylene glycol (PEG), a component in the vaccines.
"An allergy to that is pretty rare," he told ABC News. "It would have to be documented, as a moderate or severe allergy, before I would consider giving a medical exemption."
Overall, the COVID-19 vaccines are safe for people with "moderate to severe immune compromise," underlying conditions, pregnant women, women trying to get pregnant and breastfeeding mothers, according to the CDC.
Linder said these populations are most vulnerable to severe illness and death from the coronavirus and it is important that they get their shot.
"Anyone who says, 'I have a medical condition,' that is more of a reason to get vaccinated," he said.
The CDC has some extra precautions in place for people with certain medical conditions. For example, people with a history of heparin-induced thrombocytopenia (HIT), are advised to get an mRNA vaccine if they are within 90 days of illness, the CDC said. Women over 50 are also warned about a potential risk of thrombosis with thrombocytopenia syndrome (TTS) if they chose the Johnson & Johnson vaccine, according to the CDC.
The agency currently advises holding off on immediately getting the vaccine under two circumstances.
If a person is currently diagnosed with COVID-19 or under quarantine for a suspected case, they are advised from getting their shots until the quarantine period is over, according to the agency's guidelines. If a patient is receiving monoclonal antibodies or convalescent plasma for COVID-19 treatment, they are advised to hold off on their vaccine appointment for 90 days, the CDC said.
Dowdy said neither scenario should stop someone from getting any of the vaccines once they're eligible.
"People ask, 'If I've gotten COVID in the past can I get the vaccine?' The answer is yes, getting the vaccine adds additional protection," he said.
Dr. Jay Bhatt, an internal medicine physician, an instructor at the University of Illinois School of Public Health and an ABC News contributor, added that special care should be taken for patients who are awaiting an organ transplant, recently received an organ transplant or are receiving metastatic cancer treatment. Those patients should talk with their physicians and set up a timetable for the earliest and safest time to get their shots.
"It's less about not being vaccinated, it's more about when they want to do it," he said. "If they're in the midst of treatment…you want to make sure they're situated appropriately."
Researchers say it's highly unlikely that that list of medical exemptions will change in the near future. Over 178 million Americans over 12 have been fully vaccinated since December and there have been no reports so far of any adverse effects to patients who have medical conditions, according to Linder.
"The idea that we're missing something that's even rare or severe seems very unlikely to me," he said.
Linder recommended that anyone who is still hesitant about getting the vaccine over a medical issue should consult their doctor and review the data that has overwhelmingly shown the vaccines are safe.
"The risk for COVID is still high," he said. "At the end of the day, we know the COVID vaccines are highly effective at preventing hospitalization and death."
Anyone who needs help scheduling a free vaccine appointment can go to vaccines.gov.