Chairman of the Joint Chiefs  Gen. Martin Dempsey said today he is embarrassed by the conduct of the military service members involved in the Secret Service scandal in Colombia and, referring to President Obama, said: "We let the boss down."

At a Pentagon briefing this afternoon with Defense Secretary Leon Panetta, Dempsey said the incident has proven to be a distraction from what was a very important regional story.

Earlier today, Pentagon spokesmen acknowledged that more than five military service members were under investigation for the incident in which 11 members of the Secret Service were involved in a scandal involving prostitutes in Colombia.    President Obama was in Colombia to attend the Summit of the Americas.

Pentagon spokesman George Little told reporters that "these personnel were in the hotel where the alleged misconduct occurred. " He said they were not directly involved in providing security to the president, but provided support to the Secret Service.

"We are embarrassed," said Dempsey, who said he was speaking for himself and his fellow Joint Chiefs of Staff in saying they "were embarrassed by what occurred in Colombia, though we're not sure exactly what it is."

"What we do know is that we distracted - that several of our members distracted the issue from what was a very important regional engagement for our president, so we let the boss down, because nobody's talking about what went on in Colombia other than this incident," he said. "So to that extent, we let him down."

Dempsey said an ongoing investigation will hold military members accountable if  "it turns out that they violated order or policies or laws."

Panetta stressed the importance of the military behaving to the highest standards anywhere.

"Whether our forces are in Colombia or any other country, or here in this country, we expect them to abide by the highest standard of behavior," he said. "That's a requirement."

He said he did not want to prejudge the ongoing investigation, but "obviously if violations are determined to have been the case, then these individuals will be held accountable, and that's as it should be."

The military service members involved in the incident were to leave Colombia today.   Unlike the members of the Secret Service, who were returned to the United States immediately, the military service members continued with their duties though they were confined to quarters.

U.S. Officials tell ABC News that the military service members being investigated are from various military services and that they were working in explosives ordnance, communications and as dog handlers.