Not every mortgage servicer was required to sign consent orders with the Office of the Comptroller of the Currency, the Federal Reserve or the Office of Thrift Supervision. But for them, a blend of 50 state attorneys general and smaller regulators are quietly building their case for applying separate punitive damages. When foreclosure issues arose in the servicing industry last fall, the problems did not solely belong to the largest institutions. Smaller companies had to hold up foreclosures fix faulty affidavits and other breakdowns in the loss- mitigation process. Along with the 10 major settlements announced Wednesday, the OTS cracked down on four smaller thrifts that serviced loans as well. But some of the other financial firms that flesh out the top 25 spots in terms of total mortgage servicing fall between the regulatory cracks. Litton Loan Services, which is currently being shopped by Goldman Sachs (GS), halted foreclosures in October 2010 to refile affidavits. Saxon Mortgage, owned by investment bank Morgan Stanley (MS) fell under scrutiny as well. American Home Mortgage has been under investigation from the Texas AG Greg Abbott since 2010. And Ocwen Financial Corp. (OCN), which never instituted a foreclosure moratorium of its own, did fall under investigation. Each of those mortgage servicers is registered to do business in their respective states as non-bank lenders. Therefore these companies do not fall under the jurisdiction of either the OCC, Fed, or OTS. A spokesperson with the Office of Financial Regulation for the State of Florida did confirm the state regulator was still looking into Ocwen, which has offices in Orlando. Litton and Saxon may have to answer to their respective state attorneys general. Shortly after the foreclosure problems surfaced, the 50 state AGs announced a coalition with state regulators to look into the issue. Federal regulators such as the OCC, the OTS, the Fed, the Treasury Department, the Department of Housing and Urban Development and the Department of Justice hopped aboard. However, the OCC, OTS and the Fed split off during settlement negotiations to sign their own consent orders with the servicers listed on Wednesday. As part of the agreement, a variety of loss-mitigation protocols will be put in place including single-point of contact, a review of modifications and quality control checks with third-party vendors. While these regulators made room for a monetary sanction, none has been announced yet. The 50 state AGs and the smaller state regulators, though, are mired in their push for harsher penalties such as mandatory principal reduction, modifications and even fines as high as $25 billion. An agreement among their own ranks avoided consenus when four Republican AGs sent a letter to investigation leader Iowa AG Tom Miller saying the crack down he was pushing for was too stringent and would further harm the market. Those AGs included Abbott and Florida AG Pam Bondi, the same jurisdiction as the state regulator looking into Ocwen. Acting Comptroller of the Currency John Walsh and Iowa AG Miller said the settlement reached this week will not undermine the still ongoing negotiations. But homeowner advocacy groups such as the National People’s Action aren’t so certain and said they hoped the AGs would come down harder on these companies regardless of where they fall in the regulatory jurisdictions. “American families are depending on their Attorneys General now more than ever to reach a settlement with the banks that truly reforms the industry and brings homeowners and neighborhoods real relief,” the group said. Write to Jon Prior. Follow him on Twitter @JonAPrior.