This series of five articles takes a closer look at software development, the reasons for failure and how to avoid them.
It takes years to build a reputation. It takes only one failure to ruin it—even if it was someone else’s fault. Take Edsel Ford. He was president of Ford for 24 years, directed the company to offer more cars than just the Model T, and established the Lincoln and Mercury car lines. Yet most people associate his name with the failed Ford Edsel introduced 16 years after his death.
For CIOs looking to establish their own legacy, or just keep their job, one of the biggest potential pitfalls is a failed software development project. And each project offers a multitude of ways to fail. Sometimes the software never fully makes it into use, such as the FBI’s virtual case file project (canceled in 2003 after spending more than $170M); K-Mart’s supply chain management system; or the IRS’s modernization program (ditched after spending $4B).
In other cases, it may have been better if the software was never deployed. Take the case of Oxford Health Plans’ new billing system that failed to send bills to subscribers, resulting in a quarterly loss and a $3.4B stock drop. Or when problems with Hershey’s ERP system led to a $151M loss. Even small errors, uncaught, can be costly. NASA’s $125M Mars Climate Observer failed because the manufacturer used English measurements for thrust, while NASA assumed metric units. It was only after the Observer crashed into Mars instead of landing this was discovered.
In October 2008, internet retailer Overstock.com had to restate five years of financials because its ERP system didn’t correctly account for customer refunds. “The short version is: when we upgraded our system, we didn’t hook up some of the accounting wiring. However, we thought we had manual fixes in place,” Overstock.com CEO Patrick Byrne told shareholders. “We’ve since found that these manual fixes missed a few of the unhooked wires.”
These types of anecdotes have been repeatedly supported by studies and surveys over the last two decades. Perhaps the most famous is the Standish Group’s CHAOS Study which found that 31% of software development projects were canceled before completion, and only 16% were completed on-time and on-budget. More recently, the Business Analysis Benchmark Report issued in 2008 by IAG Consulting of New Castle, Delaware found that when companies have poor requirements and business analysis capabilities, three quarters of their projects would fail. Yet, despite the dismal stories and statistics, there is reason for hope. Software development projects, with proper management, can achieve their goals.
From Art to Engineering
“A couple decades back it was an art. Now, we are converting it into an engineering discipline so there is a higher level of predictability. Bringing in delivery concepts like ISO or IEEE standards, and now the Software Engineering Institute’s Capability Maturity Model (CMM) helps establish credibility,” said Abid Ali Neemuchwala, VP and head of Global Process Excellence for Tata Consultancy Services (TCS), a Mumbai-based IT firm with 120,000 employees in 50 countries.
His firm measures software development success on three key parameters: on time, on budget and on target. But in addition to keeping its internal statistics on each area, Tata also surveys customers to determine their perceptions. The two don’t necessarily line up. For example, in the last quarter, out of roughly 15,000 deliverables, TCS’s internal numbers showed 96.7% of projects were delivered on time. But, in the customer perception, it was only 89%.
“We call it the quality of delivery vs. the quality of experience,” said Abid. “If I deliver nine out of 10 deliverables on time, but the customer considers the one that was late to be the most critical, he rates me as having a 50% ability to deliver on time.”
But it is the customer’s perception that matters most. So, while Tata initially focused on the hard delivery metrics, over the last several years it has been giving equal or greater attention to lowering the gap between quality of delivery and customer perception. “That is where you can convert the customer into someone who recommends you or does more business with you,” he said.
Implementing a Project Management
One of the obvious ways to improve software development success, as well as customer perception, is via project management. “The biggest failure is not having any sort of a project management discipline but just flying by the seat of your pants,” said Jeff Monteforte, president and co-founder of Exential, an IT Management Consulting firm in
He gives the example where the client had asked that the technical lead act as the project manager. “He was extremely detail oriented and didn’t see the larger organizational issues that were occurring,” said Monteforte. “As the deadline neared, we found ourselves with the project scope running amok, the budget being burned through and no visible way of delivering the expectations that had been set.”