"They are one of the nicest families I know," he said. "I have done family law for 30 years. In terms of families who have troubles and problems, they have less than others."
But Brown said Joe Jackson's warm pledge to provide his son's children with a stable home while announcing his new record company, was "the lowest I'd ever seen him or anyone connected and certainly related to Michael ever sink."
"His son isn't in the ground and he's promoting himself," he said. "Katherine Jackson deserves a purple heart for putting up with Joe."
In the years that Brown spent at Hayvenhurst when the grandchildren were having fun, Joe Jackson would call and asked to be picked up.
"The kids would scatter like bugs when the lights came on," said Brown. "They did not want any part of being in their grandfather's presence. Michael's children will be traumatized all the more if they have to spend any length of time around Joseph Jackson."
Journalist Dimond worries about the elder parents taking custody.
"Bless Katharine Jackson, but she's 79 years old and separated from a man accused by all of his children as being a brutal dictator," she said.
"I don't think these children are doomed," said Dimond. "But I do think their lives forever changed the day their Dad died."
Celebrity psychologist Judy Kuriansky agrees the three children have a struggle ahead.
"In my view, they will need psychological help," she told ABCNews.com. "The grandparents also need guidance and counseling, to be prepared to be the children's custodians, especially Joe."