The next couple of quarters may be rough going for the housing and finance industry.

Housing prices and mortgage activity will stay highly sensitive to the Federal Reserve interest rate policy and guidance because of a weak job market, affordability challenges and the declining pool of first-time homebuyers.

Worse still, home price appreciation may level off and even dip into negative territory by the third quarter of 2014.

"Housing price appreciation (is) already on the decline, with only six cities in the Case-Shiller index showing strength in recent indexing – Dallas, Las Vegas, Miami, San Francisco, Tampa, and Washington,” says Tom Showalter, chief analytics officer at Digital Risk, which handles $8 billion in loan volume monthly. “Moreover, while home prices have increased, at least 25% of all homes are still under water."

December home sales showed that 40% of sales were all cash – suggesting strong investor participation. Originations are also at a 14-year low.

"As investors leave market, there is little evidence that typical retail buyer will take up the slack," Showalter said.

Ron D’Vari, CEO of NewOak, said he is pleased the Federal Reserve has shifted guidance and appear to be saying that economic conditions justify keeping rates below 4-4.25% for some time due to headwinds from recession.

“We agree with the new policy because of the many structural changes that have occurred domestically and globally. These include demographic shifts, decreasing competitiveness of the U.S. economy and the high correlation of large emerging markets to U.S. economic health,” D’Vari said.

“This has both positive and negative implications for U.S. home prices. On one hand, low, stable long-term interest rates should continue fueling the recovery in the housing and mortgage finance sectors,” he said. “On the other hand, if the Fed is correct, the U.S. economy is not even close to where most people believe it to be and their high expectations will not be realized.”

This, D’Vari said, will likely mean a modest stock market correction and a dampening of home price rises.

“As a large portion of home purchases have been driven by speculating institutional and private investors the last two years, there could be a pull back. Let's hope the Fed is not right in its long-term outlook,” D’vari said.

Showalter said the writing is on the wall.

These issues have been compounded by the fact that “many banks and lenders are exiting mortgage lending as application rates hit ten year lows, complemented by increasing regulatory burdens and penalties, and increasing capital requirements.  New Basel III regs now require 5% capital ratios for mortgage lending, a requirement that is largely punitive," Showalter said. “The ultimate causes have nothing to do with the weather. Rather the issues are weak job market, flat consumer income and excessive regulation.”