"I don't see how I can get back home in Texas and look in the mirror and be proud of what I see if I allowed the loud voices, the loud critics, to prevent me from doing what I thought was necessary to protect this country," he told reporters.
In an interview with ABC News' Charles Gibson last month, Bush stressed that he was unprepared for war and that he never compromised his principles.
"I think when the history of this period is written, people will realize a lot of the decisions that were made on Wall Street took place over a decade or so, before I arrived," he said.
As the Bushes moved out, a crew of 95 cleared the White House of their belongings and readied it for the Obama family.
Early this morning -- as the crowds flooded into the National Mall to secure spots for Obama's swearing in -- paint rollers and ladders were bustled aroundPerino's former White House office in preparation for incoming Press Secretary Robert Gibbs. Contractors were also moving full sheets of drywall into the front entrance of the West Wing.
Looks like Obama won't have to do a paint job himself after all, even though he told a group while volunteering at a Northeast Washington, D.C. neighborhood Monday that "It's [Painting] good practice because I'm moving into a new house tomorrow."
Even though he is now officially retired, Bush -- who will be residing with his wife in a more-than-$2 million home in an affluent Dallas neighborhood does not plan on lounging around any time soon. He has expressed his desire to write a book, and will likely also work on his presidential library to be built at Southern Methodist University in Dallas.
ABC News' Martha Raddatz and The Associated Press contributed to the report.