The short answer is: It depends. The long answer is more involved. But, ultimately, there is no short, sweet or single answer to this question.
Over the past few years of reduced budgets and a shrinking economy, ROI (return on investment) has once again come back into favor as a measurement of how well an IT project is doing or will do. The trouble is many CIOs are still stuck wondering just how to measure it and, therefore, report it back to their bosses.
According to David Sanders, senior vice president of Bearing Point’s public sector integration service practice, ROI is an elusive target at best. It is rarely a single percentage indicating increased sales or reduced downtime or improved customer satisfaction; although it could be all of those things if that is what ‘IT Project X’ set out to accomplish.
“The problem with ROI is there are so many different versions of how you can do it,” he said. “One of the issues around ROI is it’s a not very well defined term. And the thing the CIO can do is to well define what ROI really means to that company. What performance measures it’s going to use and to make sure its not something done half way.”
But defining ROI in strictly IT terms can be an exercise in futility, said Bill Johnston, president of Alinean, an ROI tools and advisory firm.
Too many other immeasurables such as employee resistance to a new system upgrade can negate the benefits Project X was supposed reap. CIOs today are faced with so many factors when trying to understand ROI and where it comes from they need to step back from the technology and take a look at the big picture.
“It really is about creating business value,” Johnston said. “The question always becomes: ‘What is the point? Why are we doing this from a business standpoint?’ So, the first thing the CIO has to do is make sure that paradigm shift has taken place and it doesn’t become a discussion around bits and bytes; it becomes a business advantage, be it competitiveness or internal efficiency.”
What are the business goals this implementation project or integration package or CRM install suppose to accomplish? How sweeping will the changes be to business processes? Will Project X be accepted by our vendors? Our customers? Our employees?
These are the first questions to be asked, said John Jordan, a principal in the Office of the CTO, Americas Region, for Cap Gemini Ernst & Young. The technology itself is irrelevant without at least some understanding as to effect it will have on the organization as a whole. The days of installing technology for technology’s sake are over.
“How many technologists can really understand the metrics of the business process instead of saying ‘Hey, we’re going to have better server uptime?’ and better server uptime doesn’t mean anything,” he said. “One of the really key exercises is to have the ROI justification in the pay-off of business economics rather than technology economics, but to do that you have to understand the business economics and that’s hard to unravel.”
This can be done with existing systems too but it is harder. Benchmarking, the tool of choice in this exercise, only goes so far, said Sanders.
With technology changing so fast and the ability to integrate legacy systems into new applications, measures of past performance can be misleading and should only be used as part of the overall ROI equation. Basing future technology spends on how well similar, older technology performed probably won’t work very well. It’s the old apples to oranges analogy; they don’t compare but they do contrast.
Also, long term projects like ERP installs or platform change-overs, are not easily measured by short-term, financial analysis: the most common form of gauging ROI.
What does work, though, is carefully defining why you are doing what you are doing and what you expect to get out of it. Then measure those metrics the same way across the IT spectrum long term. Too many CIOs, just happy with getting a project completed, abandon these measurements soon thereafter and loose valuable ROI information, said Sanders.
“Make sure it’s done correctly and done right because there’s nothing worse than one project being handled one way and another project being done another way and not being really measured the same way,” he said. “So, make sure it’s a standard. If you treat it that way then, long term, people will react to it.”