The outgoing CEO of Freddie Mac, Charles 'Ed' Haldeman, is in a state of reflection. He predicts he will leave by mid-year and is deeply dismayed at some of the things that happened in the short time he held the reins at Freddie Mac.

At this point, his main focus is to seek a smooth transition until the next CEO comes aboard. But he would have liked to have led Freddie out of conservatorship.

And it's a job he said he would have done for $1.

Haldeman and I had a discussion at HousingWire's REthink symposium last week in West Palm Beach. He revealed deep levels of frustration at the way things turned out. We agreed to title the chat "Setting the record straight," and he pulled no punches.

Haldeman, who took the job in July 2009, said he's been called before Congress twice. The politicians, he said, missed two opportunities to try to solve some of the mortgage financing issues facing the country.

"I have been asked to testify two times before Congress, and that doesn't seem wrong or unusual. They could find lots of questions on how we can help people and if we are doing enough about the future state of Freddie Mac," Haldeman said, barely hiding his disappointment. "There is a huge number of housing finance questions that I could have been asked to testify about."

"Two times (before Congress)," he said. "One subject was executive compensation and the second was on the MBA convention spending. The second was particularly frustrating."

Haldeman said he interviewed for the chief executive spot long before any compensation level was declared. He was willing to take the job for $1. Of course, when offered more, he had a change of heart. But not simply for selfish reasons.

With millions on the table, Freddie Mac wanted the message to be clear. It wasn't a charity case.

"If I didn't take it, all I'm doing is setting up lower numbers for the next guy. What sort of favor is that? It was important to the whole group that we're not seen as a charity," he said.

However, his replacement will definitely earn less. No more than $500,000 a year, according to the new cap. Haldeman himself turned down more than half ($2.7 million) in added bonus money on top of his base salary of $900,000 in 2011.

But the issue of pay and expenses is not tops on his list of major frustrations while working at Freddie Mac. The GSE was never meant to be in conservatorship for so long, he said, and being a private-market guy, Haldeman believed he would be the perfect candidate to begin transitioning Freddie Mac back to the private markets.

"The largest frustration by far that dwarfs any other frustration is the inability of policymakers to make a decision as to the future of Freddie Mac. That is a huge frustration and huge problem and a huge mistake," he said. "Freddie conservatorship for four years."

Haldeman said that has meant four years of people not knowing whether they would have a job or whether the terms would change, along with four years of Freddie's best people "having attractive offers dangled in front of them."

His entire tenure is not characterized by these troubles alone, he said. He said working with the Federal Housing Finance Agency and its acting director, Ed DeMarco, is an example of an excellent business collaboration. It's not the one-way street most people see it as, he said.

"It sounds like there's a higher authority sending down a lightening bolt, but we work very closely with FHFA to talk about how we could make more effective programs," Haldeman said.

"Ed DeMarco makes ultimate decisions in some cases," he said, adding "it is extremely rare that I had a different point of view than Ed DeMarco."

Follow Jacob on Twitter: @jacobgaffney