Jennifer Jones, a psychologist and author of the book "The Three P's of Parenting," watched our social experiment from a control vehicle nearby, and said people are often hesitant to interfere with others' parenting practices.
"In society now, there are taboos for people about interfering with someone's parenting, but if they feel that it is violent enough or aggressive enough, that it crosses a certain line, then they have a social permission, then they can get involved," said Jones.
If bystanders can connect at all to the situation, Jones said their reaction will be stronger.
"A lot of what drives what we do with our kids and what we interpret when we see other parents, is how we were raised...What did our parents do? Did they leave us at a critical moment?" Jones said. "Did our mom not show up one afternoon unexpectedly? Did we get left on the school bus one day? All these little things kind of register in our brain."
Sometimes our actress mom drove off before someone stopped her, but our abandoned child actors drew plenty of sympathy.
A concerned passerby, Adelle Rothman, stopped when she saw Brooke and Lauren. Brooke asked Rothman if she could call the girls' mom on a cell phone.
"Your two children are here and they are scared to death," Rothman told our mom-actor, Carla. "I don't know what you think you are doing."
"I told them to walk home," Carla said.
"Well, it doesn't matter, they are sitting here scared to death," Rothman said. "I think you made a big mistake."
Rothman reminded Carla of the Westchester incident that we used as a basis for our scenario.
"This is very reminiscent of what happened in Westchester, if you remember that. She told them to walk home too," Rothman said.
Rothman later said that the mom had to teach kids a lesson some other way.
As a twist in our experiment, we decided to see if bystanders' reactions would be different depending on the mother's perceived social status.
"She's smoking in the car with kids. I think I had a little bit of prejudice there," said Laura Wendle.
We gave the family in crisis a slightly different appearance, dressing them in nicer clothes and trading their scratched sedan with a missing hubcap for a luxury SUV.
Other than that, the scenario remained unchanged. This time, when people on the street saw the mom abandon her kids, the reaction was completely different.
Find out what happens on "What Would You Do?" Friday, May 14, at 9 p.m. ET.