With a stalling economy, wage stagnation and a weak jobs recovery, the majority of homeowners and renters are finding it hard to meet rising rents and mortgage payments, MarketWatch reported Tuesday.

Over half of Americans (52%) have had to make at least one major sacrifice in order to cover their rent or mortgage over the last three years, according to the “How Housing Matters Survey,” which was commissioned by the nonprofit John D. and Catherine T. MacArthur Foundation and carried out by Hart Research Associates. These sacrifices include getting a second job, deferring saving for retirement, cutting back on health care, running up credit card debt, or even moving to a less safe neighborhood or one with worse schools.

“Affordability issues are real and a major hurdle,” says Lawrence Yun, chief economist at the National Association of Realtors, an industry group. Home prices have increased 20% over the past two years while wages have barely gone up, he says. “Only by adding more new supply, via housing starts, can home prices be tamed,” Yun adds. In fact, construction of housing units has averaged around 1.5 million a year for the past five decades, he says, but it’s likely to be less than 1 million in 2014.