Ben Bernanke Refinanced His Mortgage: Should You?

And once the bank comes back with a number, don't just take it at face value. "If they tell you don't qualify--that you have too much debt, that your credit is bad-- make sure you go through it line by line," said Gross. "I have seen credit reports when things aren't accurate, so go through it line by line and try to make sure your information matches theirs. Try to be as precise as you can when you apply for the loan, because that will give you a much better chance and the bank will have much more accurate information."

Website likes Zillow.com can help you determine the values of the property in your neighborhood. You should also talk to local realtors.

The final problem, said Gross, is that it can take 60 to 90 days for a refinance, because the banks are very particular and also a bit overwhelmed because they under staffed. So you have to be willing to be patient.

Here's a calculator to help get you started.

Keep in mind that even if your credit is tarnished, you can still get a loan but the lender may require a larger downpayment and or a higher interest rate. Mortgage brokers are good sources to consult if you have an unusual situation.

How soon should you act to take advantage of the low mortgage rates? Here's where Chairman Ben has already tipped his hand: Earlier this year the Federal Reserve vowed to keep rates low though at least the end of 2014.