The Federal Reserve settled with the top mortgage servicers for a possible fine of up to $767 million in possible penalties as part of the foreclosure consent orders signed last year.

If the servicers do not live up to the foreclosure settlement between state and federal regulators announced Thursday, then the firms will have to pay the difference between the potential fine and what they did pay in relief.

The Fed could charge $275 million to JPMorgan Chase (JPM), $207 million to Ally Financial (GJM), $175.5 million to Bank of America (BAC), $87 million to Wells Fargo (WFC) and $22 million to Ally Financial (GJM).

The Office of the Comptroller of the Currency had a similar arrangement with the servicers for up to $394 million in fines.