Fears of not having a good enough credit score plagues all levels of the housing market. Yes, even folks who have a lot of money, according to an article in The Wall Street Journal by Anya Martin.

From the article:

In a survey released last month, 40% of respondents who earn $100,000 a year or more worry that their credit score will hurt their homebuying ability this year, according to credit-reporting agency Experian. And 29% of the affluent respondents surveyed said they were working to improve their credit scores to qualify for better terms on a home loan.

“It doesn’t matter how much money you have, it really is about how you manage your credit,” says Sandra Bernardo, manager of public relations and consumer education at Experian.

As it stands, it’s tough across the market to get a mortgage. Although, it did get slightly easier at the end of 2015, according to The Housing Finance Policy Center’s credit availability index.

After a year of declines, mortgage credit availability increased by 5.6% during the fourth quarter of 2015, according to the report. Despite that, lenders could still afford to risk more. While still remaining in the cautious standards of 2001 to 2003, lenders could take twice the default risk that they are taking now.

For added help, here’s the fastest way to improve your credit score. 

For those borrowers that are qualified but their credit score doesn’t reflect it, San Francisco-based SoFi, which operates like a young tech garage in Silicon Valley, recently pushed the limits by choosing to not use FICO scores when evaluating applicants.

SoFi noted that while it has chosen to not use FICO scores when evaluating the financial wherewithal of applicants, it will still consider a borrower’s track record of meeting financial obligations. It will also look at a more complete picture of a borrower’s financial situation than what a credit score can provide.