Describing the moment when Petraeus told her who her stalker was, Kelley said: "I saw him break down, and he immediately looked at me in the eyes and said, 'It's Paula Broadwell.'"
In her first ever broadcast interview since the scandal erupted, Kelley talked about receiving the threatening emails.
Asked whether she thought she was in danger, she replied: “Oh, absolutely.”
The emails in question “did not go to us at first,” Kelley said. “They initially went to the generals and General Petraeus, or Director Petraeus and another ambassador and then they came to my husband on my birthday.”
In the book, Kelley explains how she got access to two of the nation's top military officers and denies allegations of an affair.
"I was very close with their wives," she told Chang. "They were my best friends."
Kelley, who told Chang she "never had an affair with anybody in my life," also dismissed claims that she inappropriately touched Petraeus at an Easter dinner in 2012.
"It never happened," Kelley said. "Sadly nothing was the same after that and that's when the stalking started to happen."
Last year, Petraeus -- a four-star general who had served nearly four decade in the U.S. Army -- was sentenced to two years’ probation and a $100,000 fine after he admitted leaking classified information to Broadwell.
"I actually felt sorry for David," Kelley said. "In his defense, he was in a difficult situation where he had a very unhappy ex-girlfriend and the most amazing job in the world, and friends who were now being stalked."
Petraeus and Allen both declined to comment. Broadwell did not reply to ABC News.
Kelley told Chang she hopes Broadwell -- who was never charged with stalking or any other crime in the case -- is "in a better place today."
"I really have nothing to say to her," Kelley said. "I wish she never sent those emails."
Kelley recently dropped a long-standing lawsuit filed against the federal government. She told Chang she is speaking out now about her experience because the right to privacy is something she is passionate about.