"She'll call me like, almost like a lame fish. Like I wasn't going anywhere," Gosselin told "Good Morning America's" Chris Cuomo in his first television interview since the divorce papers were filed.
Gosselin, 32, said he was at home with the eight children -- one set of twins and one of sextuplets -- while his wife was traveling.
"Well, excuse me, I'm taking care of the kids. She's on book tours, she's doing all these things. You know, she's gone a week. You know, comes back. Unpacks, repacks, does laundry, packs up and leaves again," Gosselin said. "I'm standing there like, 'Oh, OK. Your mom's gone again.'"
The couple, stars of the show "Jon & Kate Plus Eight," has had a public and sometimes nasty breakup.
Tune in to "GMA" Tuesday, Sept. 8, for more of the Jon Gosselin interview.
There were allegations of cheating and an ongoing custody battle that resulted in the police being called to the couple's Pennsylvania home.
"I wouldn't let her in the gate," Gosselin said. "I said I'm spending time with my kids ... and she's going to have to leave and she tried to cry it up with the cops and it didn't work and they just basically said [to her] you have to leave."
Now the former stay-at-home dad is acting more like a playboy. He hosted a recent party in Las Vegas and went sailing this summer on a yacht in Saint Tropez with his girlfriend, Hailey Glassman, 22, who is the daughter of a plastic surgeon who performed Kate's tummy tuck in 2006.
Gosselin spoke to People magazine in July about his relationship with Glassman.
"What began as friendship has grown into something more. I know that my decision to appear publicly with Hailey this weekend will be scrutinized, but I hope that people can see I'm a regular guy who is going through a very difficult time in life and wants to move forward," he told the magazine.
Kate Gosselin has publicly expressed concern about Jon's dating and how it could affect their children, including on a recent appearance on "Larry King Live."
"Suppose he brought a date over to the house when he has the children?" Larry King asked.
"These are all things that everybody who has been or will be or is in the middle of a divorce goes through," Kate Gosselin replied.
Since the split, she has tried to explain his behavior by joking that her husband was abducted by aliens.
Now the father of eight says he is fed up and ready to move on.
"I took a lot of abuse from her. I was put down," Gosselin said.
Watch the full Jon Gosselin interview on "Primetime: Family Secrets," Tuesday Sept. 8 at 10 p.m. ET.