The Inspector General for the Federal Housing Finance Agency concludes that Fannie Mae and Freddie Mac would benefit from getting some help in pursuing those who walk away from their mortgage, despite their ability to pay.

In 2011, the recovery rate for the GSEs stood at 0.22%, which is equal to $4.7 million in clawed back losses. This is out of a potential pool of 35,321 deficiency accounts worth $2.1 billion.

"The Enterprises have room for improvement in how they manage deficiencies," the report concludes.

The report states that the government-sponsored enterprises deficiency management is not overseen by the FHFA. As a result, there is no uniform way in which the GSEs go about recovering losses from strategic defaulters.

Fannie Mae, for example, is much more aggressive about pursing strategic defaulters. However, with oversight from the FHFA, the report states, the GSEs could see more money coming back. Two years ago, strategic defaulters became locked out of securing a Fannie Mae-backed mortgage for seven years after foreclosure. Freddie does not use a similar policy.

"Further, with 1.1 million seriously delinquent mortgages looming on the foreclosure horizon FHFA's timely guidance on deficiency management process may help the Enterprises recoup future losses and protect taxpayers' investment in their financial health," the report states.

The Inspector General suggests the FHFA begin to routinely gather info related to deficiencies at the GSEs. After analyzing the data, the FHFA should then manage the deficiency collection process.