Lee Farkas will not be back in the mortgage game, despite his best efforts.

A Los Angeles Fourth Circuit panel rejected former Taylor, Bean & Whitaker Chairman Lee Farkas’ attempts to overturn a Securities and Exchange Commission injunction.

The former CEO of the once-huge failed mortgage lender was convicted of all 14 counts of bank, wire and securities fraud back in 2011. Farkas orchestrated $3 billion in fraud schemes for more than a decade.

As part of his conviction, Farkas is banned from managing mortgage companies.

Farkas' lawyers argued that his jury didn't fully understand the mortgage business and were therefore unqualified to rule that he should be banned.

According to evidence presented at Farkas' trial back in 2011, Colonial Bank submitted false information in its financial data and filings related to mortgages and securities held by the bank as a result of the scheme.

Senior executives at both Colonial and Ocala Funding, organization connected to Farkas and TBW, concealed TBW's overdrawn account "through a pattern of sweeping overnight money from one TBW account to another," according to the report.

Conspirators also made fictitious sales of mortgages to Colonial, knowing the loans either didn't exist or that TBW had already sold them to someone else. SIGTARP said, in the end, because the program dealt with already toxic mortgage products, it was unsurprising to find some executives who approved the loans in the first place to be found committing fraud in unloading them – even onto taxpayers.

"This is the most significant criminal prosecution to date rising out of the financial crisis," SIGTARP said in its report at the time. "The convictions in this case are a result of the dedicated and selfless work of the staff of SIGTARP and its law enforcement partners."