"He's being psychologically brainwashed," he said. "You cannot hug your father, you cannot love your father. Your father abandoned you."
"He needs to come home, he needs to be freed from this," Goldman said. "He needs to be a little boy. He can't live with this pressure."
Goldman said he has seen Sean four times so far this year. The beginning of the visits, he said, are typically marred by the presence of the "big, hairy guy" who accompanies Sean. But once the two are alone, Goldman said the bond begins to return.
"And we would play, we would love, we would be father and son," he said.
Sean has been in Brazil since 2004, when he went on vacation with his mother, Bruna Bianchi, and never returned. Instead Bianchi remarried a politically connected lawyer and died giving birth last year.
The Hague convention requires children such as Sean who are unlawfully taken to other countries to be returned. But Bianchi's husband refused to send him back and a custody battle ensued.
"Everyone knows what is going on, everyone knows the abuse my son is being afflicted by and it is very, very sad. And I honestly believe that the justice system doesn't want to be looked at as a country that is a safe haven for kidnappers," Goldman said.
Sean's Brazilian family insists that the issue should remain between the two families, and that Sean wants to stay in his mother's native country.
"This is not a fight between two countries. There should not be any political issue involved. This is just the pursuit of the truth and the pursuit of what is in the best interest of the boy," Sergio Tostes, a lawyer representing Sean Goldman's stepfather, said.
Goldman spoke about a visit with his son on "Good Morning America" in June.
"It was so joyous," Goldman said. "And the only one question he asked me is, after we were talking, 'How come you haven't been here to see me in this time?' ... I just said, 'Sean, I've been here many times.'"
Goldman said the visits with his son were so emotionally difficult that he worried the stress from the excessively supervised visits could be damaging to Sean.
He said a court-ordered psychiatric evaluation has proved that his son has been subjected to what he called parental alienation.
Goldman said that he had no inkling of what the future held on the day he drove wife and son to airport five years ago.
"We all gave each other hugs and kisses," he said. "I waved goodbye with the blessings of a safe trip."
Katie Escherich, Brandy Zadrozny and the Associated Press contributed to this report.