"This was a decision that LeBron made in his heart," Carter told ESPN.com's Jason Whitlock on the Real Talk podcast. "We didn't push him to do it. We don't push him to do anything. If he asks our opinion or what did we think about the pros and the cons, we help him think through it. We don't push him."
James decided to return to the Cavaliers on July 11 after spending four years with the Miami Heat. He opted out of his contract with Miami and signed a two-year deal to return to Cleveland, where he spent his first seven seasons in the NBA.
"Listen," Carter said, "LeBron's a 29-year-old man with lots of money, got a wife, two kids, one on the way. He makes his own decisions. He doesn't need anyone pushing him any way, and a guy like that, you're not going to push him either way. We'll help him think through things and help him see through things, but he makes his own decision in his heart. Because, ultimately, he has to live with it. I'm not the one who has to show up and play games for any team."
Carter said he wasn't sure if James would have made a different decision if the Heat had beaten San Antonio for the NBA title in June. It would have been Miami's third championship in four years with LeBron on the team.
"Logic says how could he not," Carter said. "You've got to go for four. But who knows. The decision about Cleveland was bigger than basketball. It was in his heart."
He also said that James and Dwyane Wade will be "friends for life" and that James told Wade before the Sports Illustrated article was released of his decision. Carter did not know exactly when James told Wade.
"Their friendship goes beyond basketball," Carter said. "It's bigger than basketball."
He also said that he, as well as James, has nothing but good things to say about the Heat or their management.
"The Heat run a first-class organization," Carter said. "They have one of the best organizations in all of sports. Just being around that organization, I think LeBron learned a ton about what it takes to be a championship organization and how it works."
Later, he added: "We had a fantastic time with the Heat and really enjoyed it."
Carter was asked about his budding relationship with Cleveland Browns quarterback Johnny Manziel, who recently signed on to have Carter manage some of his business affairs. Carter said he and Manziel talk at least every other day.
He also defended Manziel as a 21-year-old man who's "got to live his life."
"He's not hurting anyone," Carter said. "He's not beating anyone up or doing anything that's criminal. Have fun. People will adjust."