By definition, metadata is selected or summary information about data, i.e. name, length, valid values, or description of a data element. Metadata is stored in a data dictionary and repository and insulates the data warehouse from changes in the schema of operational systems.
The adoption of metadata by business has been slower than expected. Intuitively, one would think mining and categorizing data about data would be a no-brainier, especially for organizations that rely on enterprise applications to run a business.
It doesn’t help that the whole concept keeps getting re-categorized each time an organization defines a different type of metadata. The challenges in the proper definition have arisen because attempts to categorize and define metadata have so far been one-dimensional.
It is ironic that metadata, one of the functions of which is to define data, should go through challenges to its own definition.
Yet, even where metadata is being considered, it is still not well understood in terms of its scope, implementation and benefits. Listed below are some of the myths surrounding metadata that have adversely affected its importance within corporate culture.
Metadata Myths
Metadata is all about data warehousing. That most implementations of metadata target the data within a data warehouse is true but there is no reason that the implementation of metadata should be limited to a data warehouse.
Depending on the scope of implementation, metadata can be maintained on organizational processes, business indicators and metrics. Enterprise-level metadata is more useful than a solution that is data-warehouse centric.
Metadata should target data origination sources throughout the enterprise, such as transaction systems.
Metadata is someone else’s business. IT always operating under the pressures of time and money places little or no importance on developing metadata. It’s viewed as a maintenance tool, not as part and parcel of the development processes.
The reason? The fixed cost to create the metadata infrastructure within an organization. This is shortsighted.
The enterprise application portfolio of a company is akin to a network of local streets. But without metadata, there is no highway system to link neighborhoods and villages. Local streets are easy to justify and build, but their implementation (and therefore their benefits) are restricted to a limited population.
Companies that create metadata repositories in an unconnected, stovepipe fashion end up with redundant and passive metadata.
“I cannot afford metadata.” Yes, you can, but only if there is total corporate buy-in.
A policy decision about metadata should be made at the highest echelons of management. You do have to swallow the up-front fixed cost of creating pervasive metadata throughout an organization. But depending on the scope, a metadata solution will pay benefits many times over the cost of development.
Generally, ROI on metadata depends on the current state of metadata availability and its use within the organization.
It is directly proportional to the:
“The more the merrier” is the essential mantra behind a successful implementation of a metadata solution and keeping the ROI as high as possible.
For a very large organization ($10 billion in revenue or higher), a ROI of 500% is not uncommon. And sources of ROI can be both tactical and strategic.
Metadata has no flexibility. Most people believe that once metadata is created it changes little. This belief not only obscures the benefits of metadata but can also cause unintended harm to its user community.
While it is true that enterprise data changes more frequently than metadata, there are instances when metadata may evolve due to changes in the attributes of data although the data itself remains the same.
A passive metadata solution can do more harm than good when the users mistakenly base their decisions on outdated metadata. A successful metadata solution calls for real-time updating of key metrics.
Managers do not need metadata. Everyone can benefit from metadata whether it is a software developer, business development manager, data modeler or CEO.
Unlike most applications, an enterprise metadata solution has a more “global” reach and provides useful and hard to get information at a glance. This makes metadata useful to the strategic decision-making process. C-level management should have ready access to metadata.
Maintaining metadata is too political. Metadata provides deep insight into the target data that it represents. For this reason, access to metadata can be the cause of political quandary within a company.
A metadata solution, when implemented with proper security and context sensitivity can truly be a “win-win” proposition. Most concerns about data sharing can be dispelled by installing user sensitive authorization profiles.
This would enable the repository to capture and manipulate metadata on tax calculation, for example, but still safeguard this information from a field salesman, who wants to access metadata of a specific part number.
Piece-meal implementation of metadata is feasible. A long-lasting and successful metadata solution is mostly an all or nothing proposition. Different parts of the metadata solution come together to offer an all-encompassing knowledgebase, which instills confidence within the user community.
Most users of metadata are also the providers of metadata. A half-baked solution can quickly turn some users away. When they stop using metadata, they stop populating the metadata repository.
An incomplete metadata solution would drive away more users and would result in a vicious circle. Fewer metadata users result in obsolete metadata, which erodes its accuracy and completeness, which, in turn, drives more users away.
Attempts by firms to erect home-brewed metadata solutions have failed primarily because they underestimated the requirements of a successful metadata solution.
It is true that a central repository is the most obvious component of a metadata solution. But to keep the metadata active, complete and accurate, a successful implementation needs other components: triggers and alarms, data profilers, data crawlers, security administration, metadata administration, user interface, and the ability to interface with other tools and metadata repositories.
Utility of a metadata solution can rapidly degrade in the absence of one or more of these components.
With more than twenty years of experience in business and technology fields, Mark Robinson manages Greenbrier & Russel’s Business Intelligence practice. And Vivek Anand is a project manager within the Business Intelligence practice. He has more than 18 years of experience in applying information technology solutions to achieve business goals.