Deals for Guantanamo Detainee Transfers Held Up by Political Fear

June 8, 2014— -- There are deals in place to transfer dozens of the remaining 149 men being detained in Guantanamo Bay, an administration official speaking in anonymity tells ABC News.

But the release of these men -- described as low-risk cooks, drivers, and bodyguards -- are backlogged in the system and stalled by "fear" of political blowback, heightened this week with the swap of five Taliban prisoners for Sgt. Bowe Bergdahl.

Although 78 detainees have already been cleared for transfer back to their homeland or a third country, a transfer can only be made after Congress is given 30-day notice, a requirement skipped by President Obama in the controversial prisoner swap.

Despite resistance by several Congressional lawmakers, the president is continuing to push to close the prison. But the administration faces two big dilemmas.

The first is determining which current detainees may still pose a threat. The second challenge is finding a suitable destination for these men that won't draw the ire of Congress.

The Remarkable, Top-Secret Deal With the Taliban to Free US Soldier

Congressional Trust Is Collateral Damage in Bergdahl Swap

Clinton Won't Second Guess Obama's 'Hard Choice' On Bergdahl

Cliff Sloan, one of two envoys the president tasked with closing down the facility says the administration will have to work with Congress to change U.S. law that prohibits detainees from entering U.S. soil.

"For detention and trial and prosecution, we think people should be allowed to be brought to the United States, our super max [prison] facilities are very secure and we have hundreds of people convicted of terrorist offenses in our super max prisons," Sloan said. "In addition to other issues with Guantanamo, it is enormously expensive. ... [It's] 2.7 million [dollars] per detainee each year compared to in our super-max prisons at the high end around 78,000 [dollars] each year."

33 men in Gitmo are either serving sentences or in most cases have been referred for prosecution. Among these men, described as the worst of the worst are detainees like 9/11 architect Khalid Sheik Muhammad. But that leaves the remaining 149 detainees facing an uncertain future.

Sloan says the State Department is talking with more than 25 nations regarding the detainees who may not pose as a future security risk and have been cleared for transfer.

"We don't need to have Guantanamo open. It is hurting us," Sloan said.

This year a new Periodic Review Board, comprised of a member from several government agencies began taking another look at the detainees not cleared for transfer.