-- One of the reforms designed to rein in the surveillance authorities of the National Security Agency has perhaps inadvertently solved a technical problem for the spy outfit and granted it potential access to much more data than before, a former top official told ABC News.
The problem for the NSA was that the haystack was only about 30 percent as big as it should've been; the NSA database was missing a lot of data. As The Washington Post reported in 2014, the agency was not getting information from all wireless carriers and it also couldn't handle the deluge of data that was coming in.
On the technical side, Chris Inglis, who served as the NSA's deputy director until January 2014, recently told ABC News that when major telecommunications companies previously handed over customer records, the NSA "just didn't ingest all of it."
"[NSA officials] were trying to make sure they were doing it exactly right," he said, meaning making sure that the data was being pulled in according to existing privacy policies. The metadata also came in various forms from the different companies, so the NSA had to reformat much of it before loading it into a searchable database.
Both hurdles meant that the NSA couldn't keep up, and of all the metadata the agency wanted to be available for specific searches internally, only about a third of it actually was.
But then the USA Freedom Act was signed into law, and now Inglis said, all that is "somebody else's problem."
The USA Freedom Act ended the NSA's bulk collection of metadata but charged the telecommunications companies with keeping the data on hand. The NSA and other U.S. government agencies now must request information about specific phone numbers or other identifying elements from the telecommunications companies after going through the Foreign Intelligence Surveillance Act (FISA) court and arguing that there is a "reasonable, articulable suspicion" that the number is associated with international terrorism.
As a result, the NSA no longer has to worry about keeping up its own database and, according to Inglis, the percentage of available records has shot up from 30 percent to virtually 100. Rather than one internal, incomplete database, the NSA can now query any of several complete ones.
The new system "guarantees that the NSA can have access to all of it," Inglis said.
NSA general counsel Glenn Gerstell made a brief reference to the increased capacity in a post for the Lawfare blog in January after terrorist attacks at home and abroad.
"Largely overlooked in the debate that has ensued in the wake of recent attacks is the fact that under the new arrangement, our national security professionals will have access to a greater volume of call records subject to query in a way that is consistent with our regard for civil liberties," he wrote.
Mark Rumold, a senior staff attorney at the Electronic Frontier Foundation, told ABC News he doesn't have much of a problem with the NSA's wider access to telephone data, since now the agency has to go through a "legitimate" system with "procedural protections" before jumping into the databases.
"Their ability to obtain records has broadened, but by all accounts, they're collecting a far narrower pool of data than they were initially," he said, referring to returns on specific searches. "They can use a type of legal process with a broader spectrum of providers than earlier. To me, that isn't like a strike against it. That's almost something in favor of it, because we've gone through this public process, we've had this debate, and this is where we settled on the scope of the authority we were going to give them."
Rumold said he's still concerned about the NSA's ability to get information on phone numbers linked to a number in question — up to two "hops" away — but he said the USA Freedom Act "remains a step in the right direction."
The trade-off of the new system, according to Inglis, is in the efficiency of the searches. Whereas in the past the NSA could instantaneously run approved searches of its database, now the agency must approach each telecommunications company to ask about a number and then wait for a response.
In his January post Gerstell acknowledged concerns that the new approach could be "too cumbersome to be effective" and said the NSA will report to Congress on how the arrangement is working. A representative for the NSA declined to tell ABC News if any problems have been encountered so far, and Rumold noted there has been no public evidence of any issues.
Inglis said he isn't terribly concerned if the searches are a little slower. It's a small price to pay, he said, for what he called an "additional safeguard" that could increase the public's confidence in what the NSA is and how it operates.