Notes on Life Cycles (Customer and Product)

March 23, 2011 by · Leave a Comment
Filed under: Uncategorized 

I have been sharing some thoughts on the concept of life cycles, both the life cycle of the product and the life cycle of the customer with postings at the Informatica Perspectives blog – check it out…

Managing History in Master Data Management

Yesterday was the first of a series of breakfast presentations I am making with Ataccama (a data quality and MDM tools company) on the value of master data management, data quality, and data governance. One of the attendees works for a company that has invested a significant amount of budget and effort in MDM, yet is finding some challenges today regarding the management of history, slowly-changing reference concepts, and associated semantics.
Read more

Assessing the Severity of False Positive and False Negative Identity Resolution

January 7, 2011 by · Leave a Comment
Filed under: Data Quality, Uncategorized 

I was recently humorously reminded that “almost doesn’t count except in horseshoes and hand grenades,” and that got me thinking about how the concept of “almost” maps to master data and more precisely, identity resolution. I have been interviewing business data consumers at a number of clients to solicit their perception of criticality of data flaws (as we do in all of our data quality assessments and impact analyses) and have found a significant disparity regarding the severity of false matches within different business contexts, even within the same organization.
As an example, let’s contrast marketing versus health care. Many marketing organizations rent or buy external data, which may or may not be combined with internal customer data. From a marketing perspective, the information may be used for sales campaigns for recruiting new customers or perhaps cross-selling additional products to existing customers. A false negative may lead to not correctly determining that an individual is already a customer, and sending out a mailing soliciting new business. This could be a bother for the customer, and I have met individuals who would cancel their relationship with an organization over this type of situation, but that is relatively sever and unusual. Most people would probably laugh about it, rip up the mailing, and never think about it again.
On the other hand, consider health care and clinical management. A false positive that links two distinct patient records into a single record could be catastrophic. For example overwriting an individual’s record and eliminating a notation regarding critical allergies to particular medications could, at the worst, lead to loss of life.
There are a lot of other scenarios in which identity resolution issues pose different levels of problems within the same organization, ranging from a nuisance with minimal extra work to serious consequences and a lot of investigation, research, analysis, reconciliations, and other kinds of rework. Yet when the severity of the impact is small, the business representatives often are willing to tolerate the existence of the problem and attack it reactively instead of eliminating any root causes. But another interesting aspect is that within each area of the business, there is limited visibility to the levels of severity of the identity resolution issues outside of their own context. So even in an organization in which each group’s exposure to these issues is relatively small, the cumulative impact may add up to be of interest at an organizational level. This can only be shown when you have well-defined business-oriented data quality metrics.

Thoughts on the Virtuous Cycle of Data Quality

November 16, 2010 by · Leave a Comment
Filed under: Data Quality, Uncategorized 

Data quality management incorporates a “virtuous cycle” in which continuous analysis, observation, and improvement lead to overall improvement in the quality of organizational information across the board, as is shown in the figure below. The objective of this cycle is to transition from being an organization in which the data stewards react to acute data failures into an organization that proactively controls and limits the introduction of data flaws into the environment.

The virtuous cycle incorporates five fundamental data quality management practices:

  1. Data quality assessment, as a way for the practitioner to understand the scope of how poor data quality affects the ways the ways that the business processes are intended to run, and to develop a business case for data quality management;
  2. Data quality measurement, in which the data quality analysts synthesize the results the assessment and concentrate on the data elements that are deemed critical based on the selected business users’ needs. This leads to the definition of performance metrics that feed management reporting via data quality scorecards;
  3. Integrating Data Quality into the Application infrastructure, by way of integrating data requirements analysis across the organization and by engineering data quality into the system development life cycle;
  4. Operational data quality improvement, where data stewardship procedures are used to manage identified data quality rules, conformance to acceptability thresholds , supported by
  5. Data quality incident management, which allows the data quality analysts to review the degree to which the data does or does not meet the levels of acceptability, report, log, and track issues, and document the processes for remediation and improvement.

Welcome, Readers

April 26, 2010 by · Leave a Comment
Filed under: Uncategorized 

Welcome to the website for my new book, “The Practitioner’s Guide to Data Quality Improvement.” While I continue to develop this site, my intent is for it to be a repository for my continuing thoughts about practical aspects of data quality management. I will be adding content over the next few weeks and start to flesh out the site strucure. Meanwhile, let me know if you have read the book and if you have specific thoughts!

« Previous Page