The head of the New York Federal Reserve Bank told the New York Association for Business Economics on Tuesday that while the housing market has passed the worst of the effects of the housing crash, he was surprised by how much it slowed in the past year and worries that it can’t take rising rates.

“Turning first to economic activity, the trajectory of economic growth continues to disappoint.  Since the downturn ended in mid-2009, real GDP growth has averaged only 2.2% per year despite a very accommodative monetary policy,” said New York Federal Reserve Bank president and CEO William C. Dudley. “This performance reflects three major factors—the significant headwinds resulting from the bursting of the housing bubble, the shift of fiscal policy from expansion toward restraint, especially in 2012 and 2013, and a series of shocks from abroad—most notably the European crisis.”

The good news, he said, is that all three of these factors have abated. Dudley is considered a dove. (Compare this Tuesday speech to the one given at the same time by Philadelphia Federal Reserve Bank president and CEO David Plosser, a hawk, who spoke in Washington to Women in Housing & Finance Inc. on Tuesday morning.)

“As housing prices have climbed, the number of homes moving into the foreclosure process and the number of households with mortgages underwater has fallen sharply.  Moreover, households have deleveraged their balance sheets.  Debt levels have declined and lower interest rates have cut financing costs,” Dudley said. 

But if, as Dudley says, his take on the economic outlook is that it is improving, then how does one explain the sharp slowdown in growth in the first quarter? 

“My own view is that the principal factors behind the slowdown were transitory so we should not be overly concerned at this point. Some slowing in the first quarter was nearly inevitable,” Dudley said. “With the fundamentals of the economy improving and fiscal drag abating, I expect the economy to get back on to a roughly 3% growth trajectory over the remainder of this year, with some further strengthening likely in 2015.  But, there remains considerable uncertainty about that forecast and, given the persistent over-optimism about the growth outlook by Federal Reserve officials and others in recent years, we shouldn’t count our chickens before they hatch.” 

On the housing side, residential investment has stalled out over the past few quarters, Dudley conceded, and it was a surprise. 

“Although I expected some slowing due to the rise in mortgage rates in the middle of 213, the extent of the slowdown has surprised me given that the recent pace of housing starts—roughly 1 million per year—is far below what is consistent with the economy’s underlying demographic trends,” he said. “I think housing has been weaker than anticipated because several significant headwinds persist for this sector.” 

He cited three reasons for the weakness: 1) mortgage credit is still not readily available to households with lower credit scores, 2) some are coping with higher student loan debt burdens, and 3) there may be some ongoing difficulties increasing housing supply. 

“The housing downturn was very deep and protracted. It takes time to shift resources back into this area,” Dudley said. “Also, in some markets house prices still appear to be below the cost of building a new home. Thus, in those markets, it remains uneconomic to undertake new home construction.  Although I expect that the housing recovery will resume, the pace will likely be slow, especially relative to past economic recoveries.” 

Ultimately, it’s going to take time.

“We need an economy that is strong enough to more fully utilize the nation’s labor resources and to begin to push inflation back towards the Federal Reserve’s long-term objective. Only then can the monetary policy normalization process proceed. Although we are making progress towards our goals, we still have a considerable way to go,” Dudley said.

The full text of Dudley’s remarks can be read here.