Jesse James Says He Doesn't Talk to Sandra Bullock and Not Seeing Their Adopted Son Has Been 'Painful'

May 4, 2011— -- Although he has been trying to make a fresh start, Jesse James said his family has never been the same since his affair led to a nasty, public divorce from actress Sandra Bullock and cost him the baby the couple was in the process of adopting.

"I think, you know, a marriage ending, all this crazy stuff ... that was like a lesson to work on myself and fix what's wrong with me," James told ABC News in an exclusive network interview. "So I don't transfer it and keep it going, like my parents did."

After closing up the business that made him famous, West Coast Choppers, in Long Beach, the 42-year-old bought a home on 15 acres in the countryside near Austin, Texas, where Bullock also has a home. But he said he and Bullock don't speak to each other.

James said he had hoped that by moving to Austin, their youngest daughter Sunny, who was close with Bullock, would get to see the film star, and he would be able to have a relationship with Louie, the baby the couple was adopting. But this hasn't been the case. Bullock later adopted Louie as a single parent.

"I've never seen Louie since everything happened, so a year," he said, adding that Sunny had only seen Bullock "a couple of times," but that there has been "no contact at all for several months."

James said not seeing the son he thought he would have in his life has been painful for him, but he had to accept that he would no longer be in his life.

"I could only cry so much about [Louie] until I have to suck it up and keep a stiff upper lip and realize, Hey, [there are] three kids that I do have," James said. "I need to take care of them and not worry about the one that I don't, you know, and I think that's the lesson."

His new home was an escape "to get out of California," James said, and since he's no longer putting in 15-hour days at West Coast Chopper, he said he has been able to develop a new, better relationship with his three children, Chandler, Jesse Jr. and Sunny.

"I think I was just dad again," he said. "And like, eliminated all of the things that distracted me from being a dad. ... I think we're all connecting better. I think for the first time in their lives, they can depend on me to be there and I think the life here is better, you know, for the family."

That family also includes his first wife, Karla, who lives in a guest house on the property.

Once known as the star of Discovery Channel's "Monster Garage" and the king of motorcycles, James became infamous as the husband who cheated on "America's Sweetheart."

Stalked incessantly by paparazzi at his home in Long Beach last year, James entered a 30-day stint at the Sierra Tucson treatment center in Arizona where he said he dealt with childhood and anger management issues. He talked about those issues and more in his new memoir, "American Outlaw," out in stores now.

James sat down for an exclusive interview with "Nightline" shortly after his release from rehab in May 2010 to talk about why he threw away what seemed like the perfect life.

"I don't know," he said at the time. "I knew it was horrible. It made me feel horrible. [I] think I wanted to get caught, I mean I was trying to self-sabotage my life."

Now, a year later, James' focus has turned more inward as he ponders what's next professionally, and prepares for his soon-to-be fourth marriage to Kat Von D.

The star of TLC's reality series, "L.A. Ink," she and James have been friends for years, but it wasn't romance until she texted him after his breakup with Bullock and took him to dinner to cheer him up. Now, he says, the couple plans to marry this summer.

Because of his turbulent childhood, James says in the past he's had trust issues. But with Kat von D he said, "She's like 100 percent the first woman that I've ever trusted ... 100 percent, I've met my match with her."

Jesse James on Sandra Bullock: 'I Can't Worry About Her Anymore'

When asked if he thought that jumping into another marriage so soon would be hurtful to Bullock, James said though it has been painful, he realized he has to move on.

"I can't worry about her anymore," James said. "I think I've spent a good chunk of the last five or six years worrying only about her, and what she thinks, and what I should do ... controlling all my movements and everything else ... I think it's time to worry about Jesse and making sure Jesse's happy."

After such a tabloid-filled year, James' life seems simpler. Though he shuttered West Coast Choppers, he's still co-owner of the Austin Speed Shop, restoring vintage cars and mentoring new mechanics. The full-time dad also spends his days practicing his blacksmithing, a skill he went to Israel to learn, but he's taking his time deciding what his next professional venture will be.

"I like to make stuff and hand it to the person and then they pay me," he said. "When everything was big and Wal-Mart deals and licensing and all this craziness, it just took me away from what I was supposed to be doing."

He continued, "I don't need billions of dollars and I don't need licensing deals. All I need is my welding helmet, a box full of welding rods, and something to make and I'm in heaven."

In his memoir -- and in person -- James seems somber and more at peace, ready to move on and put the past behind him.

"I need to come out of the shadows and regain what I do best," he said. "Just because I cheated on my wife and got busted for it, and it became a whole media s--tstorm, it doesn't mean that I'm not valuable. I'm sorry it happened. I'm sorry it went down the way it did, but I forgive myself and now I can move on."