But a number of Americans who went to Syria to fight have made it back into the United States in recent years. Some have been arrested and prosecuted for providing support to a terrorist group, while the FBI tries to "maintain coverage" on those in the United States not currently behind bars, the outgoing head of the National Counterterrorism Center, Matt Olsen, said at the Aspen Institute's forum on Friday.
FBI Director James Comey recently said the government is spending "a tremendous amount of time and effort trying to identify" anyone who's gone to Syria, but "the challenge" is how to not miss anyone.
Over four days at the forum in Aspen, current and former U.S. officials warned of a potential homeland threat from the Islamic State, a brutal terrorist group distanced from al Qaeda that is now wreaking havoc in Iraq and Syria. But many officials suggested the group is currently focused on efforts in the region with only "aspirational" hopes to ultimately attack the West.
In Aspen today, Lisa Monaco, President Obama's homeland security and counterterrorism adviser, said she is worried about the threat from the Islamic State, but "not on the most immediate or imminent level."
ABC News' Pierre Thomas, Jack Date and Jack Cloherty contributed to this report.