The average 30-year fixed-rate mortgage is close to 3.6 percent, a near rock-bottom low that's more than a full percentage point lower than this time last year.

And for many, refinancing could mean huge savings.

In the words of "Shark Tank's" Barbara Corcoran: "Money's on sale."

ABC News

Watch: Falling mortgage rates may mean it's a good time to refinance.

If a home buyer has a typical $200,000 30-year fixed rate mortgage at 4.6 percent and they refinance at 3.6 percent, they stand to save $118 a month - more than $40,000 over the life of that loan.

Too many homeowners across the U.S., however, are paying more than they need to - 20 percent are still paying rates of more than 8 percent, according to the Consumer Finance Protection Bureau.

"Anyone who has equity in their home - and a rate higher than today's current rate - should take a look," said Erin Lantz, a mortgage expert at real estate website Zillow.

Lantz said it was about all about shopping around - something that 47 percent of home buyers never do, according to the Consumer Finance Protection Bureau.

First-time home buyers Scott and Maya Kohn of Brooklyn, New York, said they had no idea where to start so Lantz shared the following tips to help them save as they look for their new home:

1. Don't jump at the first offer.

"I always encourage people to look at at least three lenders and talk to them and let them know that you're talking to other lenders," Lantz said.

2. Negotiate until you get the best possible deal.

"So the difference between a 3.5 percent rate and a 4.5 percent rate on your purchase price could be savings of $500 a month," Lantz said.

Experts do caution homeowners though to look at the fees and if they plan to move in two years or less, refinancing likely does not make sense because the costs would outweigh the savings.

ABC News' Jared Wiener contributed to this story.