This is a repost of an article I wrote back in 1999, but I thought it might be interesting to recycle it, since it still seems relevant. I did edit it a little – I wrote it after the birth of my second child, who is now 11, so it did not make sense to refer to him as a baby ;-). Here it is:
A null value is a missing value. Yet a value that is “not there” may provide more information than one might think, because there may be different reasons that the value might be missing. A null value might actually represent an unavailable value, that the attribute is not applicable for this entity, that there is no value in the attribute’s domain that correctly classifies this entity, etc. Or the value may actually be missing!
Filed under: Business Intelligence, Data Analysis, Data Governance, Data Profiling, Data Quality, Master Data, Metadata
We are pleased to announce that our teaming partner CORMAC has been awarded the Data Management (DM) five year IDIQ contract by Centers for Medicare and Medicaid Services (CMS), an agency under Health and Human Services (HHS). As a teaming partner, Knowledge Integrity will work with CORMAC in support of data administration task orders. The total contract value is $350 million.
The purpose of this contract is to provide the following services for CMS:
1. Data Administration
2. Database Administration
3. Middleware and Message Queuing Administration
4. Business Intelligence (BI) and Extract-Transformation-Load (ETL) Tool Administration and Production Support
5. Data and Information Products Production Support
6. BI Tool, Data and Information Products, and BI-Related ETL Build and Deploy Services
7. Training/User Support
8. EDG Legacy Application and Database Maintenance Services
9. IDR ETL Development, View Development and Database Administration
I have nothing against data validation as a general practice. In fact, I might claim to be one of the more forceful proponents of validation as a practical methodology, having written a book that has guided the development of automated data validation tools. Yet validation only provides one level of trust when it comes to evaluating the quality of information. Read more
Filed under: Data Analysis, Data Governance, Data Profiling, Identity Resolution
Yesterday, Henrik Liliendahl Sørensen posted an interesting entry about data profiling, data values, and corresponding quality and completeness of the hierarchies associated with the data domain values used within a data set for any particular data element’s populated values. I’d like to jam along with that concept with respect to a conversation I had the other day that was essentially about capturing and tracking spend data, although the context was capturing and reporting the aggregate physician payments made by a pharmaceutical (or other covered manufacturer) to specific practitioners.
A long time ago I wrote an article about null values (you can still find it online at this location), and how there are often values inserted into a data element that are essentially absent values. You know what I am talking about, values like “Not available” or “N/A” that pepper the data set.