After 10 years of development, at a cost of $250 million to the state, SAP Public Services Inc. could not get a new computer system to spit out a proper payroll once during an eight-month trial run.

State Controller John Chiang terminated the contract in February and asked for the state’s money back. He was refused, so his office filed suit in Sacramento County Superior Court this week, asking for unspecified damages. Controller spokesman Jacob Roper told the Los Angeles Times the state wants $150 million.

The project, MyCalPays, is supposed to replace the poorly-connected computer systems that the controller’s office uses to pay approximately 240,000 civil service employees. When, or if, completed, MyCalPays would handle payroll for 160 agencies and integrate six different human resources systems. 

In his complaint, Chiang said “significant errors” during the first months of testing “resulted in under- or over-compensation of wages, failure to report contributions to retirement accounts, failure to issue child support payments to the correct custodial spouse, and failure to transmit data regarding medical and other insurance deductions leading to the denial of benefits.”

SAP spokesman Andy Kedzie told the Sacramento Bee, “SAP software is not the culprit here. Our software works exactly as it’s designed to work. We stand behind our software and our actions.”

An August report (pdf) prepared for the state Senate Committee on Budget and Fiscal Review spread the blame around. “It will be difficult to determine, if anyone ever does, exactly what doomed the project, the largest payroll modernization in the nation,” the report said.

It blamed the controller’s office for delivering “upbeat reports to legislators” about the project during some of its “darkest moments.” And the state was chastised for demanding more customization of off-the-shelf-software than is customarily sought. But it ripped SAP for a project that “suffered from lapses in due diligence, a failure to resolve core issues raised early and often, chronic turnover in leadership and what may have been unrealistic expectations.”

The debacle is the latest in a series of technology failures experienced by state government. In March 2012, the billion-dollar California Court Case Management System (CCMS) was terminated in midstream after the state had expended $500 million. CCMS was supposed to replace 70 different systems already in place—many of which cannot talk to each—and result in a single system for all 58 Superior Courts while enabling the public to e-file documents, access information and make payments via the internet.

But unlike those situations, the Senate report predicted, failure to implement this program is not an option. The Vietnam-era computer system is written in code “maintained and operated by a dwindling number of retirement-age specialists who still understand the outdated computer language.”

Vince Brown, the project’s former chief operating officer under former Controller Steve Wesley, told the committee, “I think everyone knows that at some point that payroll system is going to blow up for good.”

MyCalPays was conceived in the 1990s, launched as a feasibility study in 2003 and contracted out to SAP in 2005. It was to cost $132 million and be finished by June 2009. Off-the-shelf SAP software was to be installed by prime contractor Bearing Point Inc.

The project was going nowhere in 2009 when the state fired Bearing Point and suspended the project. Bearing Point then filed for bankruptcy. The project was reanimated in February 2010 and SAP took over duties as lead contract integrator of the system. The cost had now more than doubled to $305 million and the new completion date was set for September 2013.

In August 2011, the project was stumbling badly and the controller threatened to shut it down again. Most troubling was that the controller’s office and SAP each said it was the responsibility of the other party to resolve the project’s biggest problem: the inability to translate data from the old system to the new one, which is the project’s core responsibility.

The two sides squared off in October 2012, preparing for legal battle. When SAP requested mediation of the dispute, the controller fired them.

–Ken Broder

To Learn More:

California Controller Chiang Sues Firm over Failed Payroll System (by Jon Ortiz, Sacramento Bee)

California Sues SAP over Payroll System Failures (by Chris Megerian, Los Angeles Times)

State Fires Contractor and Wants $50 Million Back for Troubled Computer Project (by Ken Broder, AllGov California)

Crash Course: Failure to Heed Early Warnings, Troubles of the Past Contributed to Payroll System Collapse (by Jim Sweeney Dorothy Korber, California Senate Office of Oversight and Outcomes) (pdf)

State Controller’s Office v. SAP Public Services Inc. (Sacramento County Superior Court)