Earlier Wednesday, a Brazilian judge ordered the seizure of Lochte's passport after an investigation into the claims produced little evidence. But Lochte had already left Brazil and arrived in the U.S., according to American officials.
Feigen's passport has also been requested. As of Wednesday, he was still in Brazil.
Conger and Bentz were seen by Brazilian television entering a police station inside the airport to speak with police officers after being removed from their flight. The two declined to speak with reporters at the entrance to the station. A few hours later, they were spotted being escorted out of the station amid a media scrum and then whisked away in a black car.
U.S. Olympic Committee spokesman Patrick Sandusky confirmed in a statement that Conger and Bentz "were detained Wednesday night shortly before their flight was scheduled to depart from Rio. They were released by local authorities with the understanding that they would continue their discussions about the incident on Thursday."
"James Feigen is also communicating with local authorities and intends to make further statements regarding the incident on Thursday as well," Sandusky added. "We will continue to provide updated information as it is appropriate."
The U.S. State Department is monitoring the situation, said spokesman John Kirby. "We have seen media reports that two U.S. citizen athletes were detained," he said. "We stand ready to provide all appropriate consular assistance. Due to privacy considerations, we do not have any further information to offer. We refer you to Brazilian authorities for more information about this case."
Representatives from the U.S. Consulate arrived at the airport shortly after the swimmers were held, according to The Associated Press.
Sergio Riera, a well-known Brazilian criminalist who is the attorney for Conger and Bentz, spoke in Portuguese with reporters about his clients at the airport.
"They didn't understand why they couldn't board the plane," Riera said. "They were very scared. It's confusing because the police chief is saying one thing and the judge is saying something else."
When asked by reporters if his clients were indeed robbed, he said, "I haven't managed to talk to them yet. Until we talk to them, we won't be able to understand what happened. We haven't talked to them yet because they're very scared."
He added, "They are ready to cooperate with the police chief."
Lochte, a gold medalist and reality TV show star, claimed on Sunday that he and the three other swimmers were robbed at about 4 a.m. in a taxi headed for the Olympic Village.
The threat of violent crime in Rio was widely reported before the games.
ABC News has learned that Rio police have searched the rooms of the four swimmers in the Olympic Village.
A police official with knowledge of the investigation told AP that witnesses to the alleged robbery, along with the taxi driver, cannot be located. The official spoke to the AP on condition of anonymity.
There were some contradictions in the statements of the four swimmers, including the number of assailants and the time the events supposedly took place, according to the judge's order to search the rooms and seize the passports.
Olympic security footage showed the four swimmers arriving at the Olympic Village shortly before 7 a.m., an arrival time that leaves an unexplained time gap. The swimmers said that they left a party at 4 a.m. and were robbed shortly thereafter.
A statement from the U.S. swim team acknowledged the Brazilian authorities' request and said that it was cooperating with authorities.
"Local police arrived at the Olympic Village this a.m. and asked to meet with Ryan Lochte and James Feigen and collect their passports in order to secure further testimony from the athletes. The swim team moved out of the village after their competition ended, so we were not able to make the athletes available," the statement said.
ABC News' Drew Millhon and Robert Zepeda contributed to this report.