It is a widely held belief that business analysts (BAs) must have a number of years experience in a domain they are to be discovering and documenting requirements in to be effective in that domain. It turns out that that assumption is usually false.
An IAG study, Business Analyst Benchmark – 2008, identifies the following types of business objectives from the least risky to the most risky with the average failure rate for 68% of companies without high BA skills:
- Regulatory compliance (27%)
- Information and access to information (41%)
- Communications with employees, suppliers or customers (46%)
- Launch new products (47%)
- Reduce direct costs (59%)
- Significantly re-engineering the business processes surrounding application (67%)
- Reduce transaction and indirect costs (68%)
Failure Rate Correlations
The rate of failure correlates to the type of the project without regard to whether the BA is a business domain SME or not. In other words, using a BA who is a SME in that domain on a regulatory compliance project would fail on average 27% of the time. However, using the same BA with the same SME knowledge in the same domain would fail an average 59% on a project to reduce direct costs. Conclusion: being a SME does not help the BA reduce risk for this type of project.
IAG defines two approaches for BAs to mitigate risk: they may rely on personal subject expertise; but must rely on professional requirements discovery techniques.
Personal subject matter expertise can be effective for the first two business objective types, regulatory compliance and data access. Professional requirements skills expertise should be used to mitigate risk for business the other five objectives, types 3 – 7. A BA SME can be advantageous for only the two least risky of the seven types of business objectives.
When a BA is too familiar with a domain the BA will share the same thought processes and concepts as the rest of the team for better or for worse. However, a BA with expert level skills who is new to the domain is much more likely to ask the “dumb” question that will uncover misaligned assumptions early on when they can be addressed at the lowest cost.
Additionally, expert level BAs are much more likely to enable development of an innovative project that delights stakeholders and helps the company succeed in the marketplace. Their projects result in higher customer satisfaction.
Financial Impact
IAG states that:
Companies using best requirements practices will estimate a project at $3 million and better than half the time will spend $3 million on that project. Including all failures, scope creep, and mistakes across the entire portfolio of projects, this group will spend, on average, $3.63 million per project.
Companies using poor requirements practices will estimate a project at $3 million and will be on budget less than 20% of the time. Fifty percent of time, the overrun on the project both in time and budget will be massive. Across the entire portfolio of successes and failures, this group with poor requirements practices will, on average, pay $5.87 million per project.
Companies without high level requirement skills spend $2.24 million more per project than companies with high level requirement skills. Unfortunately, many companies recruit BAs primarily because of their business domain knowledge and use the same BAs for all projects. Those companies pay a heavy price for that decision.
Case Study
At a major global company some time ago, executives determined that significant growth could be achieved by cross-selling services to existing customers. The service was one of the most profitable in the entire company. Over a two year period, a development team had tried to make this happen but it just wasn’t working because of the wrong understanding of what was needed to cross-sell properly.
While using a simulation approach (see previous CIOUpdate article, Using the Right Medium Gets Executive Buy In), the business subject matter expert (SME) for the project simulated a related part of the department’s process and was presenting it in a group meeting with both business and IT representatives. As the business SME was walking through the process under discussion, he stopped with an “Aha!” look on his face and said “I know why we have not been able to link [the service].” He then explained his insight to the group.
As a direct result of using interactive visual simulation, the root misconception that had plagued the team for two years—connecting to the wrong use-case—became clear. Additionally, a solution also became clear immediately. Everyone in attendance agreed they could now write accurate requirements and get this functionality into production quickly. They would then be able to cross sell this highly profitable service.
Consider: What was the cost of the failed attempts? How much revenue was lost due to missed cross selling for two years? How large was the opportunity cost due to the benefit stream from other ideas that could not be implemented as a result of not achieving this functionality the first time?
While we do not have enough information to calculate real numbers, we can conclude that these three items combined represented a significant negative impact to the bottom line for this company.