Goldberg, 58, praised Crystal, 66, on his tribute to Williams at the Emmy Awards late last month.
Crystal added that it helped him get through the pain all of Williams' close friends must have been going through.
"He was such an amazing person. He was the most brilliant performer you could ever imagine. His physical presence on a stage was amazing. Many times, we'd be together on stage, the three of us, and Whoopi and I were like his parents," he joked. "It just became so sort of magical to work with him and it was just the greatest."
Crystal, Goldberg and Williams raised more than $70 million for the homeless in America through Comic Relief, but "for us, it was a chance to get to know each other," he said.
"We started in 1986 and we became the closest of friends," he added. "Robin and I just bonded in the quiet moments."
He talked about the great times, the times the three would bring checks for hundreds of thousands of dollars to shelters.
"Robin and I as we got older, it became this amazing bond," he said. "Very fiercely protective of each other, in great ways. The phone calls -- we talked about doing an album, like let's take these phone calls, we were like two jazz musicians, late at night I'd get these calls and we'd go for hours and we never spoke as ourselves."
He continued, "When it was announced I was going to Broadway, I had 50 phone messages from some person named Gary, who wanted to be my backstage dresser [which was Williams]."
Then Crystal added the duo were very protective of their dear friend Goldberg.
"You would host the Oscars," Crystal said to Goldberg. "Robin lives in San Francisco and I'd be in LA. She'd do her opening, phone would ring, 'How do you think she's doing boss?' I'd say, 'Thought it was good, really strong opening, great.' Five minutes later, you'd done your first intro, 'What do you think, i think it was really great.' The whole show would be the two of us on the phone."
Goldberg got noticeably choked up when hearing all about the love that Williams and Crystal shared for the iconic actress and comedian.
"The two of them," Goldberg said of her time on Comic Relief. "I would just sit [and watch], you didn't want to go do anything, but watch these fools."
Williams took his own life at the age of 63 last month on Aug. 11.