Government Resources Called Too Tough to Find Online

— -- Five years after the U.S. Congress passed legislation to improve electronic access to government information, 2,000 government Web sites contain public information that cannot be accessed through outside search engines, e-government experts said Tuesday.

The U.S. government has improved e-government services since 2002, but it still has a lot of work to do, representatives of Google, Wikipedia and the Center for Democracy and Technology (CDT) told a congressional committee.

"Today, too much public information is effectively unavailable to the average American," said John Lewis Needham, Google's manager of public sector content partnerships. "It can't even be found in the federal government's own search engine, USA.gov."

Many government Web sites use technologies, including search forms, that cannot be indexed by search engines, Needham told the U.S. Senate Homeland Security and Governmental Affairs Committee. About 80 percent of the time, U.S. residents don't first look for information on government Web sites, but through search engines, he said.

CDT and government watchdog group OMB Watch released a report Tuesday, detailing how government Web sites prevent search engines from crawling through their information. For example, a search for "small farm loans" turns up commercial offers for loans and statistics about government loans, but not most of the major U.S. government programs, the groups said.

A search for "New York radiation" doesn't find basic information from the U.S. Department of Homeland Security or the Federal Emergency Management Agency, the groups said.

CDT and OMB Watch called on Congress to pass an extension of the E-Government Act that would require the White House Office of Management and Budget (OMB) to create best practices that encourage searchability of federal Web sites.

The problem, Needham said, is that many government Web sites use search forms linked to dynamic databases, requiring users to input several fields of information. But Web sites could fix the problem by inputting information using the Sitemap Protocol embraced by most major search engines.

Senator Joe Lieberman, a Connecticut independent, questioned why many government Web sites don't make it easier for outside search engines. "Is it accidental?" he said. "Is it that they're not going the extra mile to make this happen?"

Many government agencies are still working on ways to present vast amounts of information, said Karen Evans, administrator of e-government and IT at OMB. "It's a lot of information, and, therefore, we try to figure out the best way to position things," she said.

Another problem is that many agencies want to present information in context, Evans added. In some cases, incomplete information could cause confusion among users, she said. "We want to make sure we're providing it in context, so we don't create more frustration," he said.

But Ari Schwartz, CDT's deputy director, said most U.S. residents now understand how to find additional information. "The American people are smart, and know how to use search engines," he said. "You don't want to block information from the vast majority of people who can figure out the context and use that information for the minority of people who may be frustrated."

Lieberman has cosponsored a bill that would expand the E-Government Act, and on Tuesday, he and two other senators introduced a bill that would make information from the Congressional Research Service, which provides research for congressional members, more accessible to the U.S. public. Right now, there's a "knowledge gap" between the general public and those willing to pay fees to access the service, he said.

While several government Web sites have drawn millions of page views, Regulations.gov has seen relatively little traffic from people wanting to comment on proposed regulations, Lieberman added. The government is still working out how to best allow comments, and not everyone is convinced that allowing everyone to comment is the best plan, Evans said.

Jimmy Wales, founder of Wikipedia, agreed, saying unmoderated comments would open up the site to "spammers and crazy people."

"I think these things take careful study," he said.

But the U.S. government could embrace wiki-type collaboration tools internally and in public-facing Web sites, Wales said.

"As with any large enterprise, internal communications problems are the cause of many inefficiencies and failures," he said. "Just as top corporations are finding wiki usage exploding, because the tools bring about new efficiencies, government agencies should be exploring these tools as well."