She said when her husband called his Argentine mistress his "soul mate," "it hurt."
Sanford, a former investment banker and Georgetown alumna, was 27 when she married the governor 20 years ago. Many called her the woman behind the man, running his campaigns while raising their sons.
Sanford told Walters that she has no political ambitions of her own. "I have no interest. I never have," she said.
While she has forgiven her husband for his transgressions, she told Walters she has not forgotten.
"No, we never forget," she said. "That's a different story."
Sanford's memoir, "Staying True," will be published in April 2010.
A special Barbara Walters hour interview with Sanford will air early next year, in conjunction with the book's release.