The housing market improved significantly since the recession including rising home prices and a drop in foreclosures, however many Americans are still scarred from their last real estate choices.

About 44% of Americans say they regret their current home choice, or the process they went through to get it, according to the newest report from Trulia. This is down from 46% who regretted their decisions five year ago.

Trulia’s survey was conducted online in June by Harris Poll among 2,264 Americans age 18 and older.

Among those with regrets, more renters wished they had bought instead of rented, while more homeowners regret not buying a bigger or smaller home. While these regrets largely remained unchanged from 2013, fewer people expressed regrets today.

Millennials were the most likely generation to regret their home-buying decisions as 71% they regretted their decisions, Trulia’s study showed. That's a far higher level of regret when compared to Baby Boomers at 28%.

The greatest regret among Millennials is that more than one third, 34%, said they wished they picked a different-sized home. About 29% of those wished they picked a larger home, while the remaining 5% said they wished it were smaller.

And these decisions continues to hold many Americans back today. More than one out of every five Americans, about 21%, say the housing purchase they made is holding them back from changing their current home.

Housing continues to grow more unaffordable, and 26% of those with an annual household income of $100,000 or more, which is about 50% more than the median U.S. household income of $73,298, said they didn’t think they could afford to buy a home in the current market.