April 15, 2012— -- President Obama said today that he expects a "thorough" and "rigorous" investigation of allegations of misconduct involving members of the Secret Service and military assigned to a security detail ahead of his visit to Cartagena, Colombia.
"If it turns out that some of the allegations that have been made in the press are confirmed, then of course I'll be angry," Obama said during a press conference with Colombian President Juan Manuel Santos following a bi-lateral meeting.
The comments were the first public statement by the president on the scandal, which has cast a shadow over his trip to the Summit of the Americas.
Obama said he holds the Secret Service to the same high standards as the rest of his staff, expecting them to conduct themselves with "dignity and probity."
"We're representing the people of the United States, and when we travel to another country I expect us to observe the highest standards because we're not just representing ourselves," he said.
While Obama and the White House have praised the "extraordinary service" of the agents and sought to downplay the scandal's significance, Secret Service and military officials and members of Congress have called the alleged behavior outrageous and launched a methodical fact-finding investigation.
Since adult prostitution is legal in Colombia in designated "tolerance zones," officials said the investigation would center less on moral or legal aspects of the alleged behavior and more on whether Secret Service and military protocols were violated -- and whether the security of the president could have been compromised.
"If all this happened, this compromised the agents themselves," House Homeland Security Committee chairman Rep. Peter King, R-N.Y., told ABC News. "It left [the agents] open to be threatened and blackmailed in the future. ... They could have been threatened or blackmailed secondly to bring prostitutes in an area that's a secured zone. It just violates a basic code of conduct."
Rep. Darrell Issa, R-Calif., who chairs the House Oversight and Government Affairs Committee, said Congress will also scrutinize the Secret Service as a whole to determine whether and how often similar situations may have happened before.
"In this particular case, the president may not have been in danger. But that begs the question -- what happens if somebody six months ago, six years ago became the victim of their own misconduct and is now being blackmailed?" Issa said on CBS' "Face the Nation."
"The question is, is the whole organization in need of some soul searching, some changes before the president, the vice president, members of the cabinet are in danger?" Issa said.
The Secret Service members were interviewed Saturday in Washington and have been placed on administrative leave. If the allegations are proven true, they could face reprimands and could be fired.
The Defense Department restricted to their quarters five personnel who were assigned to assist the Secret Service for alleged participation in the inappropriate conduct. They will return to the United States for questioning at the conclusion of the mission, officials said.
The House Homeland Security Committee, which oversees the Secret Service, has also launched an inquiry.
A heated argument between at least one of the alleged prostitutes and at least one of the Secret Service agents on Thursday first alerted local authorities to the alleged misbehavior, officials told ABC News.
One of the women believed to have spent the night with the agents would not leave the room until she was paid for her services. Hotel authorities who then got involved also checked to see who else of the American guests may have signed in female guests -- alleged prostitutes -- for the evening, a senior Obama administration official told ABC News.
Initially, that inspection led the hotel authorities to have questions about 22 Americans -- 17 Secret Service agents and five special operations soldiers who were there to assist the Secret Service, the official said. Their names were reported to the lead U.S. military official on the ground.
U.S. officials have stressed that some of those about whom the hotel raised questions may merely have been attending a party and violating curfew. But Secret Service Director Mark Sullivan made a prompt decision to relieve the entire detail of duty and have them return to Washington for questioning.
"I don't think inspector Sullivan would have all of them leave if there was not evidence of prostitution," King said.
Sources familiar with the situation also said the allegations against the agents include excessive drinking.
The group included special agents, Uniformed Division officers and two supervisors, sources said. None were assigned to the Presidential Protective Division.
"The nature of the allegations, coupled with a zero tolerance policy on personal misconduct, resulted in the Secret Service taking the decisive action to relieve these individuals of their assignment, return them to their place of duty and replace them with additional Secret Service personnel," Assistant Secret Service Director Paul S. Morrissey said in a statement.
"These actions have had no impact on the Secret Service's ability to execute a comprehensive security plan for the President's visit to Cartagena," he said.
Commander Gen. Douglas Fraser, who heads the U.S. Southern Command, said he is "disappointed by the entire incident and that this behavior is not in keeping with the professional standards expected of members of the United States military."
He also pledged a thorough investigation of the military members who may have participated in the incident, and punishment, if appropriate, in accordance with established policies and the Uniform Code of Military Justice.
White House press secretary Jay Carney said Obama was made aware of the allegations but said it "would not be appropriate for the president to characterize something that's being looked into by the Secret Service at this time."
Carney insisted, however, that the incident has not been a distraction for Obama, who is participating in the two-day Summit of the Americas with other Western Hemisphere leaders.
The Secret Service most recently faced public embarrassment and intense scrutiny in November 2009 when several agents allowed two uninvited guests onto White House grounds for a state dinner and photo line with the president. The so-called "Gate-crasher" incident resulted in three agents later being placed on administrative leave.
ABC News' Mary Bruce and Devin Dwyer contributed to this report.