Bernie Madoff 'Can Live With' Fraud Victims' Anger, But Not Family Scorn, He Tells Barbara Walters Exclusively

Oct. 27, 2011 — -- Convicted Ponzi schemer Bernard Madoff was forced to "let ... go" of his wife almost a year ago and is wracked by "horrible nightmares" as he sits in a North Carolina prison, he told ABC News' Barbara Walters in an exclusive interview.

Though he "can live with" the anger of people he defrauded out of billions of dollars and he is adjusting to the rhythms of life in prison, even at 73 years old, he is troubled by anger and turmoil within his own family.

"Not seeing my family and knowing they hate me" is the worst thing about being in prison, he said. "I betrayed them."

Asked what he'd like to say to his grandchildren, he said, without apparent emotion, "I am sorry to have caused them pain."

As he sat across from Walters during a two-hour conversation at the Federal Correction Complex at Butler, N.C., on Oct. 14, Madoff often seemed to be trying as much as possible to feel no pain, himself.

There were no pictures taken of the session because cameras are not allowed into the prison for interviews with Madoff.

Madoff said he passes the time by reading, recently finishing a book about Wall Street robber barons.

The man who ran a Ponzi scheme of more than $60 billion has held six or seven different jobs in prison, he said, and he makes $170 a month.

He said he is relieved to be free from years of fear he'd be discovered as a fraud and finally has overcome thoughts of suicide.

"I feel safer here than outside," Madoff said. "Days go by. I have people to talk to and no decisions to make. ... I know that I will die in prison. I lived the last 20 years of my life in fear. Now I have no fear -- nothing to think about because I'm no longer in control of my own life."

Repeatedly throughout the interview he told Walters that he was guilty of the crimes that put him in prison, saying "I deserved to be punished. I deserved to go to jail."

Bernard Madoff Estranged From Wife

Though Madoff has people to talk to in prison -- the young prisoners look up to him for all the wrong reasons, he said -- his family situation is far more complicated.

He has not spoken to his wife, Ruth Madoff, since after the suicide of their son, Mark Madoff, on Dec. 11, 2010. And Mark Madoff's widow, Stephanie Madoff Mack, has told ABC News she holds Bernie Madoff responsible for her husband's death and, "I'd spit in his face," if she ever saw him again.

Madoff told Walters that his wife used to visit him at the prison weekly and they spoke on the phone daily. In order to visit Butner, N.C., Ruth Madoff would drive 12 hours alone, stay at a motel overnight and drive 12 hours back to Florida, which was hard on her.

But after their son's suicide, the couple had an emotional final meeting at the prison at which Ruth Madoff "asked me to let her go, which I understood," Madoff said.

"Ruth not communicating is the hardest thing," he added. "Ruth doesn't hate me. She has no-one. It's not fair to her. She lost her first son. ... She is a devoted wife and didn't care about the money."

Madoff told Walters he has not reached out to his wife since that final meeting.

Bernard Madoff on Clients' Anger: 'I Can Live With That'

Madoff seems to have fewer worries about the victims of his massive fraud.

"I understand why clients hate me," he said. "The gravy train is over. I can live with that.

"The average person thinks I robbed widows and orphans," he added. "I made wealthy people wealthier."

Bernard Madoff Mulled Suicide, Is 'Lucky to Still Be Sane'

Madoff said all he has left in prison is his sense of humor, and sometimes he is horrified to find himself smiling.

He is "lucky to still be sane," he added.

During four months he spent in a New York jail, Madoff said, he was on suicide watch. He considered killing himself, but he "didn't have enough courage to do it" at the time.

Earlier, on Christmas Eve 2008, Madoff and his wife apparently botched a suicide bid soon after Madoff's multibillion-dollar fraud was exposed, Ruth Madoff told CBS News' "60 Minutes" in a separate interview. The couple woke up the morning after trying to kill themselves with pills, she said.

"I took what we had, he took more," Ruth Madoff said. "It was very impulsive and I am glad we woke up."

Bernard Madoff told Walters he no longer has suicidal thoughts, although prison officials still ask him about it every day, he said.

A prison psychologist, with whom Madoff spends significant amounts of time, has told him he has been able to cope with his problems over the years by "compartmentalizing" them, and he still does, he added.

"This young woman has kept me alive," he said.

Even so, he finds it hard to sleep and some of his dreams are "horrible."