A group of investors warned a House subcommittee Thursday that foreign investors crucial to funding the U.S. mortgage market will turn elsewhere if government backing is removed.

According to Freddie Mac, foreign investors are the third largest single holder of agency mortgage-backed securities. Richard Dorfman, head of the Securities Industry Financial Markets Association securitization group, said these investors, especially central foreign banks, hold vast sums of U.S. dollars, which must be invested in a low-risk, high-quality debt, and they must be able to trade securities on a large scale and on short notice.

"Therefore, foreign investors hold large sums of very liquid, low risk agency MBS and debt especially Ginnie Mae MBS," Dorfman explained. "We note, however, foreign agency MBS investments dwarf foreign non-agency investments, as many foreign investors simply will not invest in products with credit risk."

Lawmakers are considering how to structure the future housing finance system. In a white paper released in February, the Obama administration proposed three different paths, each including the wind down of Fannie Mae and Freddie Mac. However, the future plans could range between a full-scale public backing of home loans, a completely private one, or some hybrid of the two extremes with the government backing the securities and not the individual institutions.

Treasury Secretary Timothy Geithner told another congressional committee last week that the administration would be submitting a plan in the near future. Lawmakers in either chamber have yet to introduce any legislation on the future housing finance system beyond some stalled reforms of Fannie and Freddie themselves.

Michael Farrell, CEO of the real estate investment trust Annaly Capital Management (NLY: 16.94 +0.36%), which has been investing heavily in agency MBS since the crisis, told the House subcommittee Thursday the private market would unable to fill the gap should the government-sponsored enterprises are removed.

"Many, if not most, investors in agency MBS won't invest in private-label MBS at any price or only in reduced amounts because of their need for liquidity or the restrictions of their investment guidelines," Farrell said.

He cited Credit Suisse analysts, who recently estimated the U.S. housing market would lose between $3 trillion and $4 trillion in funding both domestically and globally if agency MBS was replaced by products such as Alt-A mortgages or other credit sensitive instruments.

"I believe that a housing finance system that does not include the homogeneity and liquidity made possible by government involvement will be smaller and more expensive, with potentially negative consequences for home prices and homeowner flexibility," Farrell said.

Write to Jon Prior.

Follow him on Twitter @JonAPrior.