The Treasury Department will withhold payments once again to Bank of America (BAC) and JPMorgan Chase (JPM) for their poor performance modifying mortgages in the second quarter. Both banks need substantial improvement to their operations within the Home Affordable Modification Program, according to the compliance review conducted by the Treasury. In the first quarter, the federal agency elected to withhold payments from these two banks and Wells Fargo (WFC). Since then, Wells made the mandated improvements. Though some moderate work is still needed from the bank, the Treasury will return previously withheld funds. If BofA and Chase continually fail to make the corrections, the Treasury could permanently reduce payments to the banking giants. The Treasury compliance team looked at how each participating servicer performed when contacting homeowners, evaluating them for the program and how they assisted with any questions or document submissions. Including these judgments, the Treasury also looks at HAMP data gathered by Fannie Mae to determine which servicers need improvement. Since HAMP launched in March 2009, servicers started more than 791,000 permanent modifications with roughly 28,000 reported in July, according to the latest Treasury data. Roughly 1.6 million trials have been extended, and more than 763,000 canceled due to a redefault or not enough information was submitted. “While tens of thousands of additional homeowners benefit from the administration’s programs each month, we need to keep the pressure on servicers to effectively assist those homeowners who are still struggling and eligible for assistance,” said Treasury Assistant Secretary for Financial Stability Tim Massad. “We continue to make significant improvements to our processes and controls. We expect future scorecards will reflect that,” a Chase spokesman said. A Bank of America spokesman said ratings did improve for the bank in nearly all ratings and metrics. One in four of all HAMP modifications belong to BofA, and the spokesman said the bank is committed to preventing foreclosures for unemployed, underemployed and other troubled borrowers. “We are working to achieve the ratings necessary to reinstate incentives, but we are not driven by that goal,” the spokesman said. The Treasury uses Troubled Asset Relief Program funds to pay servicers $1,000 for every permanent modification and another $1,000 every year the new loan is current. Including BofA, Chase, and Wells, 10 servicers still show a need for some improvement, unchanged from the previous review. The Treasury determined American Home Mortgage Servicing, Citigroup‘s (C) CitiMortgage, Ocwen Financial Corp. (OCN) and Select Portfolio Servicing each need to make moderate improvements. Ally Financial‘s (GJM) GMAC Mortgage, Goldman Sachs‘ (GS) now sold Litton Loan Servicing and OneWest Bank were revised to need only minor improvements. “These assessments provide an unprecedented level of information about servicer performance and are designed to help more eligible homeowners walk away from this process with better results,” Massad said. Write to Jon Prior. Follow him on Twitter @JonAPrior.