Accuracy – How Come No One Talks About It?
Posted by 
Share this Post
Without accurate master data the business processes that depend on those data can be inefficient, ineffective, and costly or increase business risk.
It is very common for MDM vendors to tout the performance and scalability of the solutions they offer, often publishing technical write-ups from lab results or publicly citing specific technical data on data volumes and throughput performance. At Initiate we’re guilty. And so are many other vendors.
I find it interesting that despite the fact that an organization’s most critical business processes depend on high quality master data, vendors don’t really talk about the “accuracy” of their MDM solutions in enabling that master data to be accurate and complete.
Without accurate master data the business processes that depend on those data can be inefficient, ineffective, and costly or increase business risk. The accuracy requirements of MDM customers should always be thought of in terms of the business requirements.
For example, what is the consequence of a manufacturer mistakenly identifying the wrong part? I suspect it can’t be measured in dollars and cents and is more rightly measured in terms of safety.
What is the consequence of not accurately identifying a patient? We know errors in the medical community cost thousands of lives each year.
What is the extra cost of additional mailers for a marketing organization? Sometimes, this can cost companies hundreds of thousands of dollars, if not millions.
But, the reason you don’t hear vendors talk about accuracy is it can be difficult to actually have a very simple conversation; because accuracy at a technical level is not simply represented as a single number. If only it were that simple.
Accuracy comes down to discussing two different populations of data – one population that consists of a pair of records that match, and the other a pair of records for a population that doesn’t match.
The accuracy discussion for an organization implementing MDM boils down to a basic decision weighing the cost/benefit of minimizing false positives (matching two records that should not match) and false negatives (not matching two records that should match). Optimizing one impacts the other.
As our customers tune their implementations, their primary goal is typically to reach a false positive, and hence a false negative rate, their business can live with. My next blog in fact will explore the cost / benefit of decisions made with respect to these types of accuracy metrics with a specific example.
With this in mind, we recently released as part of the Initiate Master Data Service V9.0, a tool that makes it easy to visualize changes in accuracy produced by making changes to the customer’s MDM implementation.
This tool accelerates project level decisions around additional implementation efforts to improve data matching and provides a visual representation to non-technical users of accuracy metrics and simplifies conversations around the costs / benefits of making changes.
We know it’s critical for executive stakeholders to understand the business implications of MDM. We believe by enabling IT and business to have a straightforward conversation around accuracy, we have contributed to making those conversations easier.
5 Responses »
Trackbacks
Leave a Response
Entries(RSS)
Well stated, Rick. This got me thinking of how to map accuracy to benefit...
Perhaps there is a curve that could be identified?
The accuracy of Entity Resolution within MDM is directly proportional to risk incurred. The higher the risk, the more accurate you need to be. For instance, with patient data, if you are not accurate someone may be injured. That is some serious risk. However if you send a handful of mailers out to inaccurate addresses it is not as big a deal, the risk is much less.
Looking forward to your next post...
-Jim
Wait a minute... nobody talks about accuracy? Maybe you're talking to the wrong people, Rick.
I hear a lot of people talk about (the lack of) accuracy, completeness, correctness, validity, timeliness, and every other aspect of data quality. Note that I don't say "master data quality". It's wrong to assume that only master data needs to be accurate. It sure is a major element, but master data isn't the only place where we'd need quality data. In addition, I see a lot of organizations simply assuming that master data is of good quality, after all, it's the master, right? Bad master data that gets replicated and then used in operational application almost works as a poison pill, negatively affecting business processes.
If vendors don't talk about accuracy (I assume you mean MDM vendors, Rick) it's probably because they don't provide data quality technologies. There is only a handful of vendors that would cover both areas, MDM and DQ, but they sure talk a lot about how those technologies complement each other.
No argument on the business value of data quality (I prefer that term over accuracy, which is too restricted). There are countless case studies of organizations that reduced risk, fraud, or cost, increased margin, revenue, or other benefits, by addressing data quality.
Cheers,
Andy
Good post, Rick.
You ask some interesting questions. While I can't speak for each vendor, perhaps salespeople know that they only have a finite amount of prospective clients' time. Focusing on bells and whistles might, in their view, give them a better shot at landing a deal.
I'd loudly echo Andy's comments about all data needing to be accurate.
I am in the middle of a project now converting systems. One of our biggest challenges stems from legacy data. The new system is set up to do exactly what the client wants. However, the client's current data is preventing the new system from doing what it should do because of (you guessed it) the data.
Cheers,
Phil
Andy...and everyone...thanks for your comments. You're right, I was speaking about vendors mostly. I realize customers are talking about completeness, accuracy, validity, etc, and it's part of the same conversation we also have with every customer. I was speaking more about how at a technical level, various MDM vendors, including us, have published performance / scalability benchmarks, but I am not aware of MDM vendors publishing benchmarks related to the accuracy their offerings deliver. Interestingly, we have discovered duplication rates of 15-30% in files “cleaned” by other organizations.