U.S. Department of Housing and Urban Development Secretary Ben Carson addressed HUD employees Monday, but one of his speech’s comments quickly drew fire on social media sites.

In his speech, Carson spoke about the U.S.’s foundations, and referred to slaves as some of America’s first immigrants. This quickly set the Internet on fire as everyone took to Twitter to vent their frustration with the secretary’s comparison.

Immigrants??? https://t.co/f0RH7iXBrn

— NAACP (@NAACP) March 6, 2017

Later Monday night, Carson appeared on The Armstrong Williams Show, which airs on SiriusXM on weekday evenings, to clarify his statement. During the show, Carson expressed he was disappointed that the nation did not focus on the positive elements of his speech, according to an article by Michael Walsh for Yahoo News.

From the article:

“When you use the term ‘immigrant,’ what you’re saying is that anybody who’s come from a foreign place is an immigrant,” suggested conservative radio host Williams, who advised Carson during his presidential campaign.

“Of course they are,” Carson replied. “And everybody in that auditorium was with me. They knew exactly what I was saying. It’s only those people who are always trying to stir up controversy.”

He also posted to his Facebook page that he shares with his wife, saying an immigrant is defined as anyone who comes to a new country to live permanently.

In other posts, he commented that “You can be an involuntary immigrant” and “Slaves didn’t just give up and die, our ancestors made something of themselves!”

And later, the HUD secretary posted again, explaining there is a very clear difference between slaves who were forcefully brought to the U.S. and immigrants who came voluntarily.

“The two experiences should never be intertwined, nor forgotten, as we demand the necessary progress towards an America that's inclusive and provides access to equal opportunity for all,” he said in his post.

After the speech Monday, HUD was quick to come to its new secretary’s defense, saying no one sitting in that room honestly believes Carson thinks slavery is the same as voluntary immigration.

@HuffingtonPost pic.twitter.com/iDjowndyIE

— HUDgov (@HUDgov) March 6, 2017