"Because of the lifestyle the children have lived already and their celebrity status, living with their white mother might be more of a shock than living with their black grandmother, who has dealt with fame and celebrity all of her children's lives," he said.
Pertman said the Jacksons' class and privilege could buffer them from some of the issues that other African-Americans raising white children might face.
"Madonna has black kids and Michael Jackson has white kids," Pertman said. "People of significant means, and not just celebrities, do not live in the same day-to-day world. They can provide buffers for their kids."
White parents who have adopted children of color also deal with less recrimination. They may draw stares and nosy questions, but it's probably less likely that someone will follow them through a grocery store and ask their children if they are "OK."
That's what happened to Mark Riding, an African-American living in Baltimore who co-parents a pale, freckled-faced red-haired girl named Katie. Riding's mother-in-law Phyllis Smith, also black, took in Katie, now 9, when she was 3 and living in foster care.
A child of a local prostitute, Katie had tantrums so awful that she was turned away from 12 homes. Smith eventually became her legal guardian and Riding, a private school admissions director, and his wife Teri, a human resources executive, became "backup" parents, keeping Katie on weekends and summers and taking her along on family vacations.
"No one ever assumes that Katie is our daughter, even though we're looking after her in the same way (as a white family who has adopted a non-white child)," Riding said. "We've been followed through grocery stores. And it's happened to my wife where someone came up to Katie and said, 'Are you OK?' It's completely disrespectful."
"It's frustrating," Riding added. "I understand it's unique, that white people aren't used to seeing a white girl in a black person's care unless it's a nanny situation, and especially a black man's care. But if I were going to steal someone's little white child why would I take her to the park or the grocery store?"
Riding believes prejudices that people are not even aware of make it difficult for some to grasp his family's situation. But Riding said he's never received a negative response from other blacks. In fact, in the black social group they belong to, others have assumed that Katie is mixed despite her overwhelmingly Irish appearance.
Riding believes such reaction harkens back to a time when the "one-drop rule" was commonplace, in which even a drop of black blood made one black. As a result blacks are accustomed to having family members of all varying shades.
Still, Riding and his wife are conscious of finding ways to help Katie embrace her Irish heritage -- by playing up St. Patrick's Day and buying her a cup emblazoned with her family crest -- and also connect with other whites and navigate the sometimes tricky waters of race and identity.
"She avoids race. She doesn't like to talk about it, but we're trying to take the time and be thoughtful about it now," Riding said.
While African-American parents may face more scrutiny, O'Connor believes their adopted white children will have an easier time adjusting than children of color raised by white parents.