Using Data Integration Testing for Reconciling Production Data Assets
Filed under: Business Intelligence, Data Analysis, Data Governance, Data Integration, Data Quality
In my last post, we started to discuss the need for fundamental processes and tools for institutionalizing data testing. While the software development practice has embraced testing as a critical gating factor for the release of newly developed capabilities, this testing often centers on functionality, sometimes to the exclusion of a broad-based survey of the underlying data asset to ensure that values did not (or would not) incorrectly change as a result.
In fact, the need for testing existing production data assets goes beyond the scope of newly developed software. Modifications are constantly applied within an organization – acquired applications are upgraded, internal operating environments are enhanced and updated, additional functionality is turned on and deployed, hardware systems are swapped out and in, and internal processes may change. Yet there are limitations in effectively verifying that interoperable components that create, touch, or modify data are not impacted. The challenge of maintaining consistency across the application infrastructure can be daunting, let alone assuring consistency in the information results. Read more
Response to “Eight Problems with Big Data”
After reading Jay Stanley’s ACLU article on “Eight Problems with Big Data,” it is worth reflecting on what could be construed as a fear-mongering indictment of the use of big data analytics and the implication that big data analytics and its implementation of data mining algorithms are tantamount to all-out invasion of privacy. What is interesting, though, is the presumption that privacy advocates have been “grappling” with data mining since “not long after 9/11,” yet data mining was already quite a mature discipline by that point in time, as was the general use of customer data for marketing, sales, and other business purposes. Raising an alarm about “big data” and “data mining” today is akin to shutting the barn door decades after the horses have bolted. Read more
Outline for a Business Justification for Data Quality Improvement
Filed under: Data Analysis, Data Governance, Data Profiling, Data Quality
I am currently working on my presentation slides for the upcoming DataFlux IDEAS conference, and the topic I am discussing is building a business justification for data quality improvement. I intend to walk through the assessment process to understand specific business impacts of data flaws, then look at evaluating alternatives for improvement, and then developing a cost/benefit analysis.
Upcoming Webinar on Geographic Data and Spatial Analysis
Filed under: Business Intelligence, Data Analysis, Data Quality
On August 25, I will be presenting a Webinar in whichI will discuss concepts of geography, spatial analysis, location intelligence, and other aspects of the integration of location information as part of a business intelligence and analytics program. In this webinar I will discuss the basics of geographic data and spatial analysis, and how to extract knowledge based on geographic information and exploit it to drive efficiencies and profitability.
Live Webinar: Using Geographic and Spatial Data to Improve Business Analytics
When: August 25, 2011, Noon ET, 9:00 a.m. PT
Register now.
You will learn:
- About geographic data and spatial analysis
- Processes for integrating geographic data
- Best practices for data integration and data quality
This Webinar has limited capacity and will fill up quickly, so register early to ensure your spot.
Using Geographic and Spatial Data to Improve Business Analytics
Is Data Profiling a Commodity?
Filed under: Data Analysis, Data Governance, Data Profiling, Data Quality
Here are some quick thoughts about the basic functionality of data profiling that make me wonder about the degree to which it has become a commodity capability. If so, then I have a few observations at the end to make folks think about what they are using profiling for.
Now Featured at Alltop!
