Americans are growing less confident in housing, and experts wonder if, after years of climbing, their confidence is hitting a plateau.

Fannie Mae’s Home Purchase Sentiment Index dropped 1.6 points in June to 90.7, just after reaching new survey highs in both April and May.

“After several years of steadily climbing, HPSI’s slowing upward trend suggests the index may be reaching a plateau,” said Doug Duncan, Fannie Mae senior vice president and chief economist. “Tight supply and lackluster income growth continue to weigh on housing activity, and consumer expectations for home price growth over the next 12 months have moderated.”

Four of the six HPSI components decreased in June, but the net share of respondents who said now is a good time to buy a home remained unchanged at 28%, and those who said now is a good time to sell a home increased one percentage point to 47%. This represents a new survey high for the third straight month.

The four components that decreased included Americans expressing a decreased sense of job security. Those who said they are not concerned about losing their job fell two percentage points in June to 76%, while the share reporting their income is significantly higher than it was 12 months ago decreased two percentage points to 19%.

The share of Americans who answered that mortgage rates will go down over the next 12 months dropped four percentage points to -53%, and the net share who said home prices will go up over the next 12 months decreased three percentage points to 46%.

But despite this decrease in confidence, Fannie Mae pointed out many Americans are still confident in the current direction of the economy.

“Consumers expressed increased optimism about the direction of the economy and their personal financial situations over the next 12 months, with both measures matching previous survey highs this month,” Duncan said.