Even before Defense Secretary Leon Panetta contradicted the initial story about the attack on the U.S. consulate in Benghazi, Libya, today, Obama administration officials told ABC News they were concerned after the White House began pushing the line that the attack was spontaneous and not the work of terrorists.

Events were too uncertain, and suspicions had been aroused, officials said.

Panetta today said that the attack that killed four Americans on the anniversary of 9/11 was not only carried out by terrorists - it was pre-meditated.

"As we determined the details of what took place there and how that attack took place," Panetta told reporters, "it became clear that there were terrorists who had planned that attack."

The White House first suggested the attack was spontaneous - the result of an anti-Muslim video that incited mobs throughout the region.

"Let's be clear, these protests were in reaction to a video that had spread to the region," White House press secretary Jay Carney said on September 14.

When ABC News pressed Carney on whether that included the Benghazi attack, in which U.S. Ambassador Chris Stevens and three other American men were killed, Carney said, "we certainly don't know. We don't know otherwise. We have no information to suggest that it was a preplanned attack."

On THIS WEEK on September 16, U.S. Ambassador to the United Nations Susan Rice said, "our current best assessment, based on the information that we have at present, is that, in fact, what this began as, it was a spontaneous - not a premeditated - response to what had transpired in Cairo. In Cairo, as you know, a few hours earlier, there was a violent protest that was undertaken in reaction to this very offensive video that was disseminated. We believe that folks in Benghazi, a small number of people came to the embassy to - or to the consulate, rather, to replicate the sort of challenge that was posed in Cairo. And then as that unfolded, it seems to have been hijacked, let us say, by some individual clusters of extremists who came with heavier weapons… And it then evolved from there."

White House officials acknowledge that assessments have changed over time as intelligence has been confirmed, but they insist that no information was given in bad faith and there was no attempt to downplay the attack.

But sources told ABC News that intelligence officials on the ground immediately suspected the attack was not tied to the movie at all. The attackers knew Ambassador Stevens had been trying to flee - to a so-called safe house half a mile away. That building was hit with insurgent mortars - suggesting the terrorists knew what they were doing.

As of Thursday afternoon, officials from the Obama administration were not even 100 percent certain that the protest of the anti-Muslim film in Benghazi occurred outside the U.S. diplomatic post.

In a closed-door briefing with top officials, Director of National Intelligence James Clapper described the mortar attack on the safe house as suggesting that the terrorist attack was one of opportunity, not pre-meditation, since the mortars were not used to attack the consulate earlier in the day.

Campaigning in Virginia Beach today, President Obama seemed eager to paint the terrorist threat as waning. "Al Qaeda's on the path to defeat," he said. "Bin Laden is dead."

But the Daily Beast's Eli Lake on Wednesday reported that intelligence officials said "the early information was enough to show that the attack was planned and the work of al Qaeda affiliates operating in Eastern Libya." "There was very good information on this in the first 24 hours," one of the officials told Lake. "These guys have a return address. There are camps of people and a wide variety of things we could do."

It's certainly possible that intelligence officials wouldn't want the terrorists to know that the U.S. knew about them, but that does beg the question as to why White House officials seemed to strongly suggest the attack was merely the work of an unruly mob.

President Obama has repeatedly said the investigation is on to find the killers and bring them to justice. But as first reported by CNN, ABC news has learned that the FBI - which has been dispatched to Libya to take the lead in the investigation - has not even reached Benghazi yet.

This is largely due to safety concerns. Indeed, as of Thursday, senior State Department officials said that the diplomatic presence in Libya - which was already down to emergency-level staffing - would be further reduced.

A spokeswoman for Ambassador Rice, Erin Pelton, issued a statement to ABC News regarding her appearances on THIS WEEK and other Sunday shows on September 16, saying Ambassador Rice's comments in those interviews "were prefaced at every turn with a clear statement that an FBI investigation was underway that would provide the definitive accounting of the events that took place in Benghazi. At every turn Ambassador Rice provided - and said she was providing - the best information and the best assessment that the Administration had at the time, based on what was provided to Ambassador Rice and other senior U.S. officials by the U.S. intelligence community."

*This post has been updated