Taking Decisions about Information Value

This report comprises the findings of CREST funded research into project into making decisions about information value. It addresses an important challenge for intelligence analysts. Intelligence analysts are typically required to process large volumes of data in a timely manner in order to extract useful information and detect potential security threats. This relies on consistent judgements by the analyst in order to efficiently process the data and effectively identify useful information. Research and historical evidence have shown that analysts' judgements are often inconsistent due to the mass of data, the variation in
types and nature of intelligence information and the time pressures the analyst is operating within. Consequently, intelligence analysts will often take decisions that deviate significantly from those of their peers, from their own prior decisions, and from training rules that they themselves claim to follow. Such inconsistency is mainly due to two types of errors; noise and bias,
which complicate the intelligence analysis process and can result in key pieces of data being misclassified or overlooked with potential security threat implications. The proposed project aims to develop, train and evaluate an innovative analytic approach to address these errors and enable analysts to achieve better judgements about the value of elicited information from intelligence reports. The innovation is in the embedding a machine learning method called the Dominance-based Rough Set Approach (DRSA) algorithm within a tool that enables an intelligence analyst's interests and
behaviour to be captured. This is designed to evaluate the consistency of analysts' judgements at individual and group levels, as well as identifying key factors or biases which influence an analyst's decision making.