Broadwell, a married mother of two, had access to Petraeus while she was with him in Afghanistan as his official biographer. People close to the general had previously suspected Broadwell's feelings for him had crossed a professional line.
They found Broadwell, who spent a year embedded with Petraeus in Afghanistan, to be embarrassing and far too "gushy" about him. They said to one another they thought Broadwell "was in love with him," sources told ABC News.
Petraeus is said to have been the one to have broken off the extramarital affair.
His storied career, first as the public face of two wars in Iraq and Afghanistan, and later as director of the CIA, came crashing down Friday when he announced his resignation from the intelligence agency, citing the indiscretion.
"After being married for over 37 years, I showed extremely poor judgment by engaging in an extramarital affair. Such behavior is unacceptable, both as a husband and as the leader of an organization such as ours," Petraeus said in a statement Friday.
Director of National Intelligence James Clapper was made aware of the Petraeus situation Tuesday evening around 5 p.m. by the FBI, according to a senior intelligence source.
After having several conversations with Petraeus that evening and the next day, Clapper advised Petraeus that the best thing to do would be for him to resign, the source said.
Clapper notified the White House the next afternoon that Petraeus was considering resigning, according to the source. Petraeus then went to the White House Thursday and told the president he thought he should resign, and Obama accepted his resignation the next day, the source said.
Despite the lengthy investigation into Broadwell by the FBI, the White House says it was not made aware of it until Wednesday, the day after the election, a revelation that surprised many.
"It just doesn't add up. That the FBI would be carrying on this type of investigation without, again, bringing it to the president or the highest levels of the White House," Rep. Peter King, R-N.Y., said.
Petraeus and his wife, Holly, who have been married for 38 years, are said to be staying in their Arlington Home and are doing "OK."
"Knowing the family, I suspect it will be hard work, but given the effort, they will get through it," Boylan, the former Petraeus spokesman, said.
Numerous questions still remain about the investigation, and some on Capitol Hill are also frustrated because Petraeus was schedule to testify to the House and Senate intelligence committees about the attack on the U.S. consulate in Benghazi, Libya, in September.
The timing of Petraeus' resignation "was what it was," an official told ABC News, adding that the time had come to tie up any loose ends in the investigation and confront the general.