Yet despite the diversity, all data migration projects share a common trait: they are complex, time consuming and costly, and exceedingly prone to major overruns if not outright failure.
So how can you overcome the stumbling blocks? The starting point is project methodology. With the right project methodology in place, data migration can become much easier to deal with.
Don’t Think Linearly
It seems logical to look at data migration as a linear process with four distinct phases: analyze the source data; extract and transform it; validate and fix it as needed; and load it into the target. But the fact is, data migration is much more of an iterative process, and adopting a linear “waterfall” project methodology that is tuned to addressing each of these phases serially can be a big mistake.
Here is what typically happens in a migration project. Right off the bat, you can be hit by a lack of understanding of your data and source systems. You can analyze and analyze, and still be surprised as constraints invariably pop up that require further and more highly focused analysis.
Then as you move into the extract and transform phase, you can expect to be hit with shifting target system requirements, resulting in more analysis and project changes.
As you move into validation, further issues will surface that push you back into analysis mode and the creation of new extracts and transforms. Hence you are continually going back to the drawing board of analyzing your source systems.
A linear project methodology that treats each phase discretely can quickly break down and start burning time and money. The linear approach is simply not flexible and fluid enough to effectively handle the iterative nature of data migration.
Think Iteratively
A more appropriate methodology is one that anticipates the challenges intrinsic to data migration and that is adaptive to data migration’s iterative nature and the need for constant mid-project adjustments and refinements.
With an iterative methodology, you are able to continually repeat—or spiral through—analysis, extraction, validation, etc. without breaking time and money budgets because the four phases are interconnected and not discrete.
With an iterative methodology the stumbling blocks to successful data migration are minimized, if not completely eradicated. With the ability to cycle back and analyze thoroughly as you go comes an accurate and complete understanding of data and data sources.
The nimbleness this capability provides makes it easier to deal with data migration’s moving target syndrome, i.e., changes no longer send you all the way back to square one. The same holds true when dealing with complex target data-validations: they are much less likely to become show-stoppers.
Lastly, and not at all insignificantly, you gain a framework for promoting consistent best practices and building meaningful data migration expertise within your organization.
Just like reusable objects, these attributes can be leveraged repeatedly across projects to ensure their success. With data migration projects increasing in scope, complexity and number, this alone might be the biggest payoff.
Arvind Parthasarathi is director of solutions at Informatica, a data migration and master data management company. He has a wide experience with migration projects at some of the world’s leading ERP and supply chain implementations.