Live in certain cities, and it doesn’t really matter whether you rent or buy: It’s going to be hard to make your monthly payment.

Need an explanation? Let’s say you’re a mortgage-holding U.S. homeowner of moderate income. Then you likelyspend around two-fifths of your income on housing. That’s the average calculated by the National Center for Housing in a recent study, which also found moderate-income renters spend around one-third of their incomes in housing. And it’s getting worse, as housing costs have been growing twice as quickly as incomes, according to the study.