Points to consider:

• Do I have the funds that refinancing may require to cover up-front costs and fees?
  Refinancing your mortgage may require you to pay a large amount of money to cover up-front costs and fees. If you do not have enough money to pay the up-front costs completely it may be possible to finance some of the costs by including them into the new mortgage.
   
• How long until I recover the costs of refinancing?
  The rule of thumb is the refinancing costs are recovered within 2-3 years. So, if you plan to sell the house or pay it off shortly, you may not want to refinance because you will not recover the costs. Obviously, this depends on the up-front costs and the savings with the new mortgage.
   
• Has my income increased substantially?
  If your income has increased substantially, you may be able to afford higher monthly payments. This may allow you to shorten the term of your mortgage. If the available interest rate is lower for the shorter term mortgage, refinancing is a good option. Otherwise, simply make larger principal payments against your current mortgage.
   
• Is the current loan an Adjustable Rate Mortgage (ARM)
  If the current rates for a fixed rate mortgage are the same or slightly higher than your ARM, refinancing may make sense. If the fixed rate is lower than they are expected to be when your ARM converts to a fixed rate, it may make sense to refinance.

Obviously, much thought needs to go into the refinancing decision. Also, you should evaluate this decision regularly to account for changes both in your financial situation and the economy. Perhaps, the decision is not to refinance now, but a few years from now it may save you thousands of dollars.