CARTAGENA, COLOMBIA - U.S. Secret Service officials were sent home from President Obama's trip to Colombia because of allegations of misconduct, apparently involving prostitution.

While the Secret Service would not confirm the number of personnel involved, the allegations make this an acutely embarrassing incident for the elite security tasked with protecting the president.

"There have been allegations of misconduct made against Secret Service personnel in Cartagena, Colombia, prior to the president's trip," spokesman Ed Donovan, a special agent, told ABC News this evening. "Because of this, those personnel are being relieved of their assignments, returned to their place of duty, and are being replaced by other Secret Service personnel.

"The Secret Service takes all allegations of misconduct seriously. This entire matter has been turned over to our Office of Professional Responsibility, which serves as the agency's internal affairs component. These personnel changes will not affect the comprehensive security plan that has been prepared in advance of the President's trip," Donovan added.

The story was first reported by the Associated Press, which received an anonymous tip that a dozen Secret Service officials were relieved of duty for misconduct involving prostitutes in Cartagena.

The officials in question, some of whom are married, arrived at the beachfront Hotel Caribe a week ago, the same hotel where the White House staff and press are currently staying, according to the Associated Press. The officials were reportedly drinking heavily during their stay.

The beach-resort city is hosting the Summit of the Americas this weekend. The president left Washington today to attend the series of meetings with his Latin American counterparts. He is scheduled to return home Sunday.