"My life was crazy for a long time," said Harvey, who has been married three times. "I love the idea of being married. Now have I blown it? Yeah."
Now living with his third wife Marjorie and with seven children between them, Harvey said it took him time to "learn that my wife was an individual."
"And because I'm the main breadwinner in the house, I can't come in and treat her like a child. She's an adult. She's as big a part of this family as I have," he said.
But the stress Harvey felt between love and money is a common one.
Toni Langdon, 32, and Wayne, 38, from West Chester, Pa., feel like they don't communicate well enough about their money. Toni is a stay-at-home mom for their six-month-old child and says that since Wayne handles all the finances, she's out of the loop.
According to Harvey, couples should separate some money to give each individual independence, but spend and save jointly on necessary things.
"Every couple should have four bank accounts," Harvey said, two individual spending money accounts and two joint accounts for essential spending and saving. The savings account, he said, should require two signatures to move the money.
"I think that sounds like a great idea," Wayne said.
Leon Thomas, 32 and Cameo, 28, of Detroit, Mich., have three kids under five years old.
Leon has been unemployed for three months and Cameo is still in nursing school. The couple is struggling to cut back and sometimes argue about how to spend the money they have. Cameo doesn't mind cutting back on herself, but she refuses to change the way they spend on the kids.
"We got the kids accustomed to getting what they asked for," Cameo said.
Harvey's advice on this one is simple: "Everybody's gotta participate in the cut back," he said.
"Ever heard the slogan save money, live better? The severance pay is not going to last forever... come on now, you don't see that the kids need to pitch in?" he said.
Don, 60, and Cyd Evanowski, 54, from Grosse Pointe, Mich., are dealing with a shuffle of power just as their 17 year-old daughter starts college.
Don retired from Chrysler a year and a half ago. He has a pension but now works with Cyd at her store "Brides To Be."
"I've had my company for 30 years... my company was doing beautifully," Cyd said. But then just days after Don retired, "he started working for me.
"He, in this female, estrogen environment, is the only man," Cyd said. But the real problem is that they're both having trouble adjusting to the idea that after more than 30 years, Cyd is now the couple's breadwinner.
"What you've got to understand about a man is who he is, what he does and how much he makes defines him," Harvey said. "The who he is now is the employee of his wife. The what he does is he works at a girls' store for brides company... This is an ego problem here more than a finance problem. He didn't sign up to be the lesser. He signed up to be your husband and your man."
And while Cyd said she expected Don to do more in the role reversal, such as help out more around the house, Harvey said that wasn't part of the deal.
"Well I didn't sign up to be the bread winner," Cyd said.