Connecting the Dots Across the Intelligence Community
Posted by 
Share this Post
The intelligence community can leverage entity resolution to connect the dots.
We had previously defined "entity resolution" and discussed Gartner’s definition. Today, let’s examine a more specific use case: How can the intelligence community leverage entity resolution to connect the dots between agencies?
The Intelligence Community (IC) is comprised of many whose activities aim to protect national security. The involved agencies share a common mission to perform counter-terrorism and counter-intelligence activities that connect the dots between terrorists/foreign intelligence operatives, persons of interest, events or locations that might be of interest.
An intelligence analyst must identify, assess and monitor potential threats that are actively trying to mask their nature, status and intent. There are two ways that IC analysts gather and use data that could help with their investigation.
The first is to analyze records in databases that already exist.
The second involves analyzing raw reports of real-time data coming in from the field: snippets of information, pieces of recorded phone conversations and other data that does not yet reside in a database.
Recently collected field data is often put into a temporary data store so that analysts can quickly perform link analysis to determine whether people or events are related. This process attempts to resolve multiple conflicting reports and identify a single entity, which can be an extremely difficult and complex process.
Even when searching existing databases for potential threats, most intelligence analysts are accessing hundreds of disparate structured and unstructured data sets from public, internal, cross-organizational and clandestine sources. Each data set has its own schema, varies in terms of completeness and quality, and is typically voluminous and highly dynamic.
Clandestine data from covert sources are especially problematic as they often contain dirty and incomplete data. In addition, analysts are often required to conduct cross-language script matching.
Despite the challenges, there is high value in introducing technology that can accurately assist with identification and resolution of entities (individual and complex) and associated relationships (both declared and inferred) across multiple data sets.
When providing entity resolution technology to the IC, some additional concerns must be addressed. The community is one of low trust and analysts are likely to be especially suspect of technology that performs associative analysis – after all, that’s their job.
The parameters of any entity resolution technology used by IC analysts need to be able to be configured and adjusted to meet individual requirements, carry out additional levels of analysis and conduct hypothesis testing. Trust is only achieved when there is confidence that the results have mitigated false positive and false negative scenarios.
Despite standing mandates to share information more fully across the IC, there is reluctance on the part of data owners to release control of their data. It is highly desirable to provide a way to share information in a secure and need-to-know-only manner that upholds pedigree and lineage and gets information to the right analyst at the right time.
Finally, the IC operates on numerous classification networks, all of which contain valuable data that must be exchanged and integrated across domains. The IC can benefit greatly from technology that recognizes and manages data in a particular way based on how that data is tagged and classified as it comes from various networks ranging from UNCLASSIFIED to the highest level of classification.
In my next post, I’ll discuss an integrated law enforcement case study.
Leave a Response
Entries(RSS)