Mortgage rates have spent the past month in a one-way elevator going up, and this may spell bad news for refinancings, which have been driving a large portion of originations these past few years.

Furthermore, with rates shooting up, more borrowers may steer clear of refinancing their debt, creating a possible headwind for banks.

Wells Fargo (WFC) recently announced it's laying off approximately 350 workers nationally because higher home loan rates are impacting activity levels.

"The recent cuts at Wells Fargo are driven by market demand and our evaluation of the needs of the market," Vickee Adams, a spokesperson for Wells Fargo Home Mortgage, said.

Meanwhile, the rest of the market is also dealing with growing mortgage rates.

"Refinancing is going to come down, and it will reduce bank earnings as to that activity," Dick Bove, vice president and bank analyst with Rafferty Capital Markets, said. "But new housing activity and housing sales will continue to increase and ultimately those two lines cross and new housing will continue to grow."

Looking at the market as a whole, Bove said the issue is far too complex to sum up by just looking at rates.

"The new hosing market is going to continue to be very strong the next 5 to 6 years," he explained. Refinancings may fall, but "ultimately mortgage revenue will increase," he suggested. 

Bove pointed to profits posted by the banks this quarter and last.

"The banking industry made more money then it has ever made, so to get worked up about refinancings is missing the big pictures," Bove explained.