The incident is the latest in a string of embarrassing hiccups in the county's vaccine rollout.
Lisa Piercey, Tennessee's health commissioner, said that the state has alerted the FBI about the alleged theft.
The latest disclosures come on the heels of an announcement made earlier in the week that more than 2,400 vaccine doses in Shelby County had been wasted over the course of a month, and that the state was transferring control over the storage and distribution of the county's vaccine inventory to the city of Memphis.
To culminate the tumultuous week, the county's health director submitted her resignation Friday.
According to Piercey, a volunteer is believed to have stolen multiple doses from a vaccine event on Feb. 3, apparently by drawing the vaccine into syringes before leaving the site with the syringes.
Piercey's office contacted the FBI Thursday evening about the alleged theft.
An FBI spokesman told the Associated Press Friday that the bureau had been made aware of the situation, but the spokesman did not say whether an investigation had begun.
Piercey said the kids arrived with their mother at the site, apparently with appointments, and a volunteer gave them the vaccines.
Neither the theft nor the administration of the vaccines to the children were reported to the state until Thursday, weeks after they occurred.
Piercey said she brought up rumors of stolen doses to Shelby County Mayor Lee Harris, who told her he was aware of the reports. He later sent the state more information about the theft, along with details about the children who were mistakenly vaccinated.
Harris, in a press conference Friday, said he knew of "suspicious activity" at the site of the theft within a day of the incident, and that the issue was taken up with law enforcement but was not reported to the state.
Harris also announced that Alisa Haushalter, the Shelby County Health Director, had resigned earlier in the day.
Gov. Bill Lee, in a trip to a Shelby County vaccine site Friday "to get eyes on the ground on what's happening," told reporters there have been some "very serious challenges" in the county's vaccine rollout and "very disappointing circumstances that have unfolded over the last several days."
Of the move to transfer control over vaccines to the city of Memphis, Lee said, "That speaks to the great deal of concern we have."