Was it the potential for backlash from Republicans and Democrats alike? Was it Larry Summers' history in the Clinton era of advocating for excessive homeownership or was it Summers perceived failure to rein in some of the excesses in the financial system that many analysts have blamed for the eventual 2008 housing crisis?

It’s hard to say, but news reports from various outlets, including CNNMoney, report that as of Sunday, potential Federal Reserve Chairman nominee Larry Summers has pulled his name out of the hat, leaving Vice Chair of the Federal Reserve Board of Governors Janet Yellen the presumed front-runner once again to replace Ben Bernanke.

Whoever is picked will play a key role in the mortgage finance space since the Fed sets interest rate policy and is directly responsible for quantitative easing — a policy that allows the Fed to acquire mortgage-backed securities and Treasurys on a monthly basis to keep interest rates low, providing a stimulus for housing and the economy. Who gets the Fed chair could very well determine how quickly Bernanke's accommodative policies are carved back.

Although it was likely Summers would face pushback from both parties – and from Wall Street – which in many ways prefers the predictability of a Yellen pick, it's likely the president felt more pushback from his own coalition than from across the aisle or Wall Street. 

Not long after news broke that Summers backed out, Jim Dean, Democracy for America’s chair, had this to say:

"Larry Summers did the right thing to avoid a contentions confirmation process that would have pitted progressives and many Democratic Senators against the White House. Now, it's up to President Obama to do the right thing and nominate the best person for the job: Janet Yellen."

And that is someone from an Obama-friendly camp.

It seems no one was keen on Summers, who was perceived to be more of a fiscal hawk than Bernanke and a potential break from the Fed status quo of the past five years.  And for the president, Summers was perceived as someone carrying alot of baggage. Over the weekend, Seeking Alpha reported that Summers suspended his ties to Citigroup (C) as Obama continued to ponder his pick to lead the Fed. How quickly times have changed? As of last week, a Japanese paper had Summers in the lead.

So what spooked him? Not only does the above quote show progressives – tied to Obama – disliked Summers as a potential Fed chair, but Seeking Alpha noted that three democrats on the Senate Banking Committee confessed they would not back his nomination, making it a difficult confirmation process unless Republican votes could fill the void.

In the end, Summers backed out. While that leaves Yellen presumably in the lead, HousingWire has noted in the past, that she is not the only name on Obama's list. Yellen's the obvious top contender for now, unless you ask US economist Paul Ashworth with Capital Economics. He says Yellen is far from a shoe-in:

"The decision by Larry Summers to withdraw from the race to be the next Fed Chair will persuade many that the current Fed Vice Chair Janet Yellen is now a near-certainty to win the nomination," he wrote. "As Yellen is perceived to be the most dovish candidate, we would expect the news to be positive for both bonds and equities on Monday morning. The Obama administration has shown little, if any, enthusiasm for Yellen, however, so we're not convinced she will necessarily get the nod."

Okay then, back to square one.