Mastering Data Management

More in Government

• More Mobile, More Competitive Government
• We Are Our Own Columbos
• The End of the Social Security Number?
• Five Rules of Successful Identity and Social Network Analysis

More from Aeastman

• Not Your Grandmother’s MDM: Introducing MDM Application Toolkit
• What to Look for in a Master Student Index
• From K to 12 and Beyond: Tracking Student Progress
• Measuring Education Success with Longitudinal Data Systems

Share this Post

Recent Comments

• Lorraine Fernandes on Response: A Universal Identifier Isn’t a Silver Bullet
• Henrik Liliendahl Sørensen on Response: A Universal Identifier Isn’t a Silver Bullet
• Response: A Universal Identifier Isn’t a Silver Bullet | Mastering Data Management on 3 Best Practices in Identity Management

A reference architecture can help connect the dots - and systems - across agencies

John Foley of Information Week recently wrote about the technical task of connecting the dots. The main theme of his article was that getting the right information to the right people at the right time is a huge technical challenge.

Mr. Foley lists some of the technologies that address the problem of connecting the dots:

"But how do tidbits of data in a dozen different places get transformed into actionable insight? Some of the technologies and practices that come into play include enterprise content management, master data management (i.e. data governance), data cleansing, complex event processing, text mining, identity resolution, data integration middleware, data mining, BI tools, relational databases, and data warehouses. U.S. intelligence agencies have already implemented many of these capabilities, so the question is whether they've done so effectively and what more can be done."

Mr. Foley’s list of technologies is comprehensive and bears a strong resemblance to thoughts we’ve shared on the subject. Back in November, Forrester’s Gene Leganza joined us as we broadcast a webinar on our reference architecture for identifying persons of interest. You can find the webinar recording here.

We recognize that intelligence, national security, and law enforcement organizations have bits and pieces of the technologies Mr. Foley listed. What’s lacking is a clear vision of how to effectively combine those pieces to solve intelligence and security challenges.

We have learned that without entity resolution, those pieces fail to deliver their full potential (for more information on entity resolution see Entity Resolution 101).

Here are some of the technologies that intelligence, national security, and law enforcement organizations are using to connect the dots:

Visual Analytics
These products visualize large data sets as relationship networks, map overlays, or diagrams and charts. They help analysts see patterns and gain insights that they couldn’t get from looking at raw data. Yet without entity resolution, visual analytics merely visualizes dirty or duplicated data. Finding the needle in the haystack is even harder when you have duplicated haystacks.

Enterprise Search
These products allow you to search across disparate structured and unstructured data sets, giving you a single place from which you can look for information about a subject. But have you ever had to wade through hundreds of search results to find the data you were looking for?

Entity resolution determines which of those hundreds or thousands of records refer to the same individuals and aggregates all of the details into consolidated records. What’s more, entity resolution can find connections that simple searches can’t.

Entity resolution might connect a record without a name to another record without an address (e.g., because they share other common identifying information). A simple search on just the name would miss one of those records, unless entity resolution had already correlated them.

Event Processing
Rules engines, reasoning engines, predictive analysis, or complex event processing are all terms you might use to describe solutions that analyze transactions to detect or predict fraud, threats, or other noteworthy events. Entity resolution ensures that those transactions and events are associated with the right individuals, so that the resulting analysis doesn’t miss an important connection in the data.

Initiate’s reference architecture for connecting the dots understands the key role that entity resolution plays in enabling these technologies. Understanding which tools are appropriate for which problems, and how to put the pieces together, is a critical step on the path toward overcoming the technical challenge of connecting the dots.