Putting potentially dangerous returnees like that behind bars, however, has been a slow and painstaking process.
“People aren’t saying, ‘Hey, I just got back from fighting with ISIL, here’s my ticket [proving it],’” a federal source quipped about the challenges in bringing cases against returnees.
In fact, U.S. law sets a “high bar” to prosecute an American for joining a group like ISIS, especially given the “complicated dynamic” and “limited visibility” on the ground in Syria and Iraq, and the reluctance to present classified sources and methods in open court, according to current and former federal officials.
“The problem is some of the guys we … know traveled, but we didn’t know about it until they came back,” one federal source said. “So how do we find out what they did?”
The FBI has spent much of the past two years trying to figure that out.
Over that time, at least 40 Americans have returned from Syria or Iraq, and at one recent point about half of them were under “full investigation,” indicating the FBI had come across some bit of information – even “single-source” information – suggesting those suspects posed a possible threat, ABC News was told.
FBI agents across the country have conducted electronic surveillance, scrutinized travel records and passenger databases, reviewed messages and posts on social media, interviewed family and friends, and in some cases approached the suspects directly.
“We worked very hard to sort out who are the ones” to worry about, FBI Director James Comey said last month.
Through that work, the FBI has cataloged a recent "shift" in returnees and other so-called "travelers," with an increasingly younger crop of American jihadists replacing those focused on providing humanitarian assistance or “nationalistic support," according to federal sources.
Many of the investigations into the "early travelers" – who the FBI determined never fought with or supported a terrorist group – have since been “closed out,” one federal source said.
So the FBI is now focusing its efforts on that small group of returnees it “assesses” pose an “actual” and “significant threat to the homeland,” as the federal source put it.
“There are several cases in the pipeline” at “various stages,” the federal source said.
Arresting suspects for lower-level offenses would take them off the streets at least for a short time. But to put returnees behind bars for longer, the Justice Department “relies” on a law that prohibits someone from providing “material support” to terrorist organizations or even trying to do so, Holder recently said.
And under that law, federal investigators need to prove suspects linked up with a group officially designated a terrorist organization by the U.S. government, and that they did it “knowingly” – meaning they didn’t end up there through chance or misfortune.
“Traveling to Syria and engaging in combat there is not enough,” one federal source said.
Syria and Iraq are home to several U.S.-designated terrorist organizations, such as ISIS and the Al Nusrah Front. However, there are also countless rag-tag rebel groups there that have not been outlawed by the U.S. State Department.
“Once [Americans] get into that melting pot, sorting out who belongs to which group… who they’re exposed to … [and] what skills they gathered … is where the complicated dynamic comes into this,” one federal source said.
That complicated dynamic can undermine a federal prosecution, as illustrated last year when FBI agents in Virginia arrested a former U.S. Army soldier for fighting with militants in Syria.
Eric Harroun, 30, had appeared in online videos with many of those militants, and he repeatedly told FBI agents he was fighting with the Al Nusrah Front as part of its “RPG Team.” He even posted photos and messages about it on his Facebook page.
Federal prosecutors indicted him for providing material support to a terrorist organization, calling their case “extremely strong.” He faced life in prison.
But within months the case dramatically changed course, with further investigation revealing Harroun had not been fighting with the Al Nusrah Front after all. He wanted to fight with them and thought he had found them, but he actually fought “with a different violent extremist group” not designated a terrorist organization by the U.S. government, one federal law enforcement official said.
In a deal with prosecutors, Harroun pleaded guilty to an obscure weapons-related violation. He was released from prison six months after his arrest, sentenced to time already served.
“Until we have more of an ability to collect and gather evidence and support these prosecutions, they’re going to present challenges,” said John Cohen, a former Los Angeles-area police officer and Naval Intelligence investigator who until recently was a top counterterrorism coordinator at the U.S. Department of Homeland Security. “We’re going to have to look at different ways to mitigate the threat or to neutralize the threat.”
Cohen predicted the FBI will now be looking to make cases against returnees “based on what they do in this country” rather than what they did previously overseas.
Federal prosecutors in Los Angeles built such a case last year after a 25-year-old California man returned from Syria, where he attained what he described as his “first confirmed kill” and spent five months fighting with the Al Nusrah Front.
To put Sinh Vinh Ngo Nguyen behind bars back on U.S. soil, the FBI launched a four-month undercover operation, ultimately ensnaring him in a fake plan to leave the United States again to train al Qaeda fighters in Pakistan. In June, he was sentenced to 13 years in prison.
In Europe, the threat from radical returnees “became real” months ago, when a former ISIS fighter opened fire in a Jewish museum in Brussels and killed four people, the then-director of the National Counterterrorism Center, Matt Olsen, said at a forum in Washington last month before he left office.
One federal source said the FBI’s “priority” is stopping a radical returnee from taking an action like that, and the "prevention piece” is more important to the FBI than proving any criminal case.
The FBI is undertaking that effort even as it tries to identify others who may have left for Syria or Iraq and then slipped back into the United States.
“There is no doubt that there are people that have traveled and returned that [we] don’t know about,” the federal source said, adding such anonymity makes stopping any potential threat from them even harder.
Of course, there are likely also so-called "lone wolves” across the United States that have never stepped foot in Syria or Iraq and are being radicalized online “in basements [and] in pajamas” by groups like ISIS, as Comey and Homeland Security Secretary Jeh Johnson recently described them.
“In many respects, that’s the terrorist threat that I worry most about because it’s the hardest to detect, and it could happen on very little notice,” Johnson said earlier this month.
ABC News’ Pierre Thomas contributed to this report.