"I said, 'I'm coming home,'" recalled Bush. "There was a lot of uncertainty about whether there were other planes, but I was not going to give a speech to the nation from a bunker in Omaha, Neb."
On the night of Sept.11, Bush declined to sleep in the bunker four floors underneath the White House, telling Winfrey that he took one look at the pullout bed the secret service wanted him to sleep on and headed straight to the presidential quarters.
But in the middle of the night, a false alarm about an attack on the White House forced the president and the first lady to retreat downstairs.
When he woke up the next morning, Bush said he was determined to "find out who the enemy was and what the enemy was thinking."
Asked whether he ever personally feared for his safety, Bush said, "No, never afraid," but added that he was "afraid for the country."
Bush visited ground zero in New York a few days after the attack, a visit he likened to "walking into hell."
"I got out there with the firefighters and the first responders and police and there was a palpable feeling of anger and revenge and raw emotion," said Bush. "I did the best I could do to comfort them and tell them how proud we were of them."
Questioned about whether he made a mistake in his quest to find Iraqi President Saddam Hussein's weapons of mass destruction -- weapons that ended up not actually existing -- Bush insisted that while he feels bad about it, bad intelligence had steered him astray.
"Everyone felt that Saddam had weapons of mass destruction," said Bush. "And that ... after 9/11, that when you saw a threat you couldn't let it fully materialize."
"When we didn't find weapons, I felt terrible about it and sick about it, and I still do because a lot of the case to remove Saddam Hussein was based on these weapons of mass destruction," he said. "Oftentimes history [judges] you on the decisions you make and not on the decisions you don't make."
"Saddam in power today would mean the world would be less stable and more dangerous. The world is better with him gone," he said.
Bush said the day he left the White House for good in January 2009, he was "comfortable" and "ready to go home.
"Well, I had just come from watching my successor be sworn in, and that in itself is an amazing moment, and it was a historic moment," he said. "I was reflective upon the peaceful transfer of power which makes America a special place."
"I had a great sense of gratitude that I was honored to serve our country for eight years," he said. "I was pretty relaxed, and unsure of what my life was going to be like post-presidency."
He declined to comment on President Obama's presidency, saying he didn't like criticism when he was in office.
Today, Bush and former first lady Laura Bush spend most of their time at their home in suburban Dallas, away from the limelight.
"I really do have zero desire [to go back into politics]," said Bush. "It's hard for people to believe that, but I had my moment."
"Eight years as president and six as governor of Texas, I've used politics as a chapter in my life and not my life."