Reports of Data Quality’s Death are Greatly Exaggerated

This recent article about analytics suggests that IT executives worry too much about data quality. However, the provocative headline (well, at least provocative to me) is a conclusion drawn out of survey results, and that interpretation may reflect some flawed thinking.

My reading is this: In reporting on a speech given by IBM VP of information management strategy Andy Warzecha, the article’s author enumerates the “biggest barriers to broader use of analytics” resulting from an IBM survey of 3000 executives as (I added the numbering, by the way):

1. “Lack of understanding about how to use analytics to improve the business (38%)”
2. “Lack of bandwidth because of competing priorities (34%)”
3. “Lack of skills in lines of business (28%)”

The conclusion in a summary of the original paper (which appeared in the MIT Sloan Management Review) that “… getting the data right is not a top challenge organizations face when adopting analytics” appears to based on the fact that “concerns about the data (21%)” (number 8) and “the ability to get data (24%)” (number 4) are lower on the list.

However, let’s think about this a little bit more. First of all, the highest scoring obstacle (not knowing how to use analytics) is always going to be an issue in an organization that is not properly prepared to exploit new technology. Suggesting that “analytics” is a solution to some set of problems without understanding how that solution addresses those problems is always going to be a problem. In fact, replace the term “analytics” with any other technology du jour (CRM! ERP! MDM!) and my opinion is that the same percentage of executives would still lack understanding.

Let’s look at number 2: having to deal with competing priorities. I see this a lot in some of our clients, in which the desire to benefit from a technology exceeds the bandwidth of available resources to do the work needed to implement the technology (such as understanding how to use that technology, as an example). Of course, higher performing organizations manage their activities more efficiently, which allows them to allocate resources to applying analytics. Number 3 is interesting as well: “lack of skills in the line of business” – again, my opinion is that if this is an issue, the company probably has bigger problems than applying analytics.

By the way, it is worth looking at some of the other obstacles:

4. “Ability to get the data”
5. “Existing culture does not encourage sharing information”
6. “Ownership of the data is unclear or governance is ineffective”
7. “Lack of executive sponsorship”
8. “Concerns with the data”

In other words, out of the top 8 barriers, half reflect some issue with data (access, quality, sharing, governance).

By the way, I am not claiming that data quality should bubble up to the top of the list, especially in organizations that have (a) lots of data, (b) are prepared to change the way they operate based on the results of analysis, and (c) are willing to tolerate some amount of noise in their data. Look at companies like Amazon, Netflix, eBay, or Orbitz, which collects reams and reams of transactions and are able to consumer, analyze, and apply the results. If there are some errors in the data, it is probably OK, since a small amount of bad data has little overall effect on the aggregate results. In addition, in some scenarios, there is an ability to tolerate some incorrect conclusions because real-time performance monitoring is in place that allows the company to rapidly change direction if a decision turns out to be bad.

However, not every company is an Amazon or a Netflix, and the fact that 4 out of the top 8 obstacles to adopting analytics are data issues still says to me that data quality is not yet dead.

Comments

2 Comments on Reports of Data Quality’s Death are Greatly Exaggerated

  1. Steve Sarsfield on Tue, 15th Mar 2011 10:47 AM
  2. Thank you, David, for clarifying this. I can’t decide whether the author of the Information Week article was going for shock value or is simply uninformed. Either way, it’s an irresponsible headline and probably not worthy of our coverage.
    By the way, my next blog article will be entitled “Kids Worry Too Much About Vitamins” and then I’m planning on penning “Drivers Worry Too Much About Fuel”. Stay tuned.

  3. admin on Tue, 15th Mar 2011 4:01 PM
  4. Thanks Steve for the comment, I love it when people not only read the posts but also take the time to share their thoughts.

    Actually, the author of the IW article was noting what was in the original report, I think, which has a section that said

    “Despite popular opinion, getting the data right is not a top challenge organizations face when adopting analytics. Only about one out of five of respondents in our study cited concern with data quality or ineffective data governance as a primary obstacle…”

    But I got my stuff from looking at their graphic in the paper and drawing my own conclusions about what is meant by the obstacle categories. If I saw all of the surveys individually, I might have drawn a different conclusion because I think that 4 of those choices express “concern with data quality or ineffective data governance.”

    As far as whether it is worthy of coverage, I disagree – there is no such thing as “negative publicity”!

Tell me what you're thinking...
and oh, if you want a pic to show with your comment, go get a gravatar!