Renters and owners are more likely to be concerned about housing affordability when they have to find a new place to live compared to the affordability of where they already live, according to a report from Fannie Mae.

Basically, people feel more comfortable paying what they pay now and are concerned that their payment (whether it's rent or a mortgage) will be less affordable than what they currently have. This may explain why people are moving less frequently and, by extension, explain some of the shortage of homes on the market.

According to a new survey from Fannie Mae, just 8% of owners with a mortgage said their mortgage was not affordable, whereas as a whopping 45% of owners overall feel that it is difficult to find affordable housing in their area.

The same trend is evident in renters. They are much more likely to feel okay about their current dwelling’s rent level than they are about the prospect of finding a new affordable place to live in their area.

Additionally, two thirds of renters say that owning a home makes more financial sense, but these same renters also express a fear that getting a mortgage would be difficult, citing low credit scores or inability to afford a down payment.

High-income renters are the exception to this rule, and their dissatisfaction with paying high rents may have led to the bump in homeownership over the past two years.

But aside from the high-rollers, thanks to affordability issues, more than half of the survey respondents, especially the renters, said they have put a chill on non-housing expenditures, skipping vacations, taking it easy on daily spending and living with roommates to help with the cost of living.

According to Fannie Mae, some of these issues may take "years" to address.