A crisis could soon emerge for the roughly 400 community banks that still owe the Treasury Department nearly $2 billion combined in bailouts, according to the Special Inspector General for the Troubled Asset Relief Program.

"A common misconception is that most of the 707 TARP banks have paid back TARP, when really only the largest banks have exited TARP," the watchdog said in a report released Thursday.

According to the Federal Deposit Insurance Corp., 326 of the 392 bank failures in the U.S. since 2008 have been community banks.

Beginning in the fall of 2013, the dividend rate these smaller institutions must start paying to the Treasury increases to 9% from 5%. Many of these banks are relying on the Treasury's willingness to restructure the TARP investments at a steep discount.

"This impression could create moral hazard concerns by taking away incentives for banks to find capital on their own — a necessary step to exit TARP," SIGTARP said.

The Treasury invested $4 billion into community banks through the Small Business Lending Fund, but $2.2 billion went to 137 banks that have yet to repay the TARP bailout. The banks simply swapped it out for a dividend rate below 5%, removing executive compensation requirements and the dreaded TARP stigma.

About 320 of the more than 500 banks still left in TARP applied for the SBLF program. For many, SIGTARP said, this was their TARP exit strategy.

"Despite the dramatic efforts to expedite the largest banks exit from TARP, there appears to be no corresponding concrete plan for community banks’ exit from TARP," the watchdog said.

SIGTARP referred to the regulatory bending the Treasury and other supervisors committed to let the larger firms out of TARP possibly too soon.

The watchdog recommended the Treasury create a clear exit plan by providing new access to capital larger lenders enjoy over these smaller institutions. The criteria at a minimum, SIGTARP said, should include a more uniform approach to restructuring the bailouts, rather than continue on the case-by-case basis SIGTARP called "ad hoc and inconsistent."

"Something needs to happen. There needs to be some action taken right now," said Acting SIGTARP Christy Romero in an interview with HousingWire. "Otherwise these small banks have no ability to raise capital. When the dividend increases, they will scramble to raise capital in the same time frame. That can flood the markets, and that will put more of these banks in jeopardy."

Treasury Assistant Secretary Tim Massad pointed out that 259 of 303 banks that exited TARP were smaller institutions with less than $10 billion in assets.

"We believe our overall approach recognizes that each bank's situation is unique, and that such approach is better suited to fulfilling our statutory responsibilities than attempting to devise standard discounts or terms for all situations," Massad said in a letter to SIGTARP responding to an audit.

Massad also said that if a bank finds it is unable to pay the elevated dividend, it would not necessarily default or fail. Repayment is voluntary, he said. While the Treasury does have the contractual right to appoint board directors in cases where a bank is unable to meet the payment, it has only done so for six banks.

But SIGTARP raised the point that as long as these TARP dollars are outstanding, recovery at these smaller firms will remain sluggish and lending in their local economies — vital for job growth — would remain constricted.

"These are banks whose names are on the backs of Little League jerseys. If they suffer the communities they're in suffer," Romero said.

Write to Jon Prior.

Follow him on Twitter @JonAPrior.