The Federal Reserve said last week that they plan to reduce the amount of mortgage-backed securities they buy, basing their purchases on their confidence about jobs and inflation.

"The housing market is no different from the bond market and equity market," said Mark Grant, managing director at Southwest Securities. "It’s artificially inflated."

To be sure, some say an early housing recovery will weather a phasing out of QE3, the Fed’s economic stimulus also known as quantitative easing. The housing affordability index is still near record highs of 200, according to the National Association of Realtors. That is, the average household has 200% of the qualifying income needed to purchase the median-priced home, financed with a 30-year fixed-rate mortgage.