The proliferation of data accessibility has exacerbated the risk of shallowness in information analysis, making it increasingly difficult to tell when analysis is sufficient for making decisions or changing plans, even as it becomes increasingly easy to find seemingly relevant data. In addressing the risk of shallow analysis, the concept of rigor emerges as an approach for coping with this fundamental uncertainty, motivating the need to better define and understand analytical rigor.

Research Team

Emily S. Patterson

David D. Woods

Daniel J. Zelik



Research Support

Erik Aagard

Stoney Trent

Martin Voshell

Understanding Rigor in Information Analysis

Across information analysis domains, it is often difficult to recognize when analysis is inadequate for a given context. A better understanding of rigor is an analytic broadening check to be leveraged against this uncertainty. The purpose of this research is to refine the understanding of rigor, exploring the concept within the domain of intelligence analysis. Nine professional intelligence analysts participated in a study of how analytic rigor is judged. The results suggest a revised definition of rigor, reframing it as an emergent multi-attribute measure of sufficiency rather than as a measure of process deviation. Based on this insight, a model for assessing rigor was developed, identifying eight attributes of rigorous analysis. Finally, an alternative model of briefing interactions is proposed that integrates this framing of rigor into an applied context. This research, although specific in focus to intel analysis, shows the potential to generalize across forms of information analysis.


The topic landscape below provides a brief history and overview of our research into understanding rigor in information analysis. This landscape summarizes the developing understanding of rigor relative to the following five topic areas: (1) Exploring rigor to determine how it can be defined in the context of information analysis. (2) Studying professional analysts to identify the cues that they use to infer and evaluate rigor. (3) Assessing rigor by identifying metrics that differentiate between high and low rigor processes. (4) Applying the concept of rigor through innovations that address the challenges of analysis. (5) Looking Forward to discover how understanding rigor shapes the future of information analysis.

Understanding Rigor in Information Analysis Topic Landscape



The following paper, presented at the Eighth International Naturalistic Decision Making Conference, provides a more formal overview of our current research.

Modeling Rigor in Information Analysis: A Metric for Rigor Poster (PDF)



David Woods’ discussion of our analytical rigor research at a Google TechTalk provides a dynamic presentation of the material. Google TechTalks are designed to disseminate a wide spectrum of views on topics including Current Affairs, Science, Medicine, Engineering, Business, Humanities, Law, Entertainment, and the Arts. This talk was originally recorded on on April 10, 2007.

Reducing the Risk of Shallow Information Analysis Google TechTalk





Studying the Professional Analyst

Our exploration of analytical rigor is based, in part, on a research study involving professional intelligence analysts. This study examined how professional analysts judge the rigor behind an analysis. The study investigates the challenges that inhibit the understanding of rigor in intelligence analysis and explores cues used by analysts to identify analytic rigor—or lack of rigor. Nine professional intelligence analysts participated in a modified elicitation by critiquing method study, embedded in a scenario walkthrough. Findings from the study indicate that, while professional intelligence analysts can make perceptive assessments about the quality of an analysis process based on product quality, these perceptions are apt to change with insight into the analytic process.


The following paper, presented at the Human Factors and Ergonomics Society 51st Annual Meeting, provides an overview of the research study and findings.

The Impact of Process Insight on Assessment of Analytic Rigor Poster (PDF)



A complete write up of the study background, methodology, analysis, and results can be found in my Master’s Thesis (CSEL Report 2007-TR-02).

Zelik, D., Patterson, E. S., & Woods, D. D. (2007, June). Understanding rigor in information analysis. Paper presented at the 8th International Conference on Naturalistic Decision Making, Pacific Grove, CA. (PDF) (VIDEO)



This recent poster provides a brief overview of our research on analytical rigor, contrasting a conventional perspective on rigor with a practice-driven perspective.

< BackHome.html

Analytical Rigor Poster (PDF)



This poster provides an overview of the rigor model, identifying the aspects of the attributes that contribute to low, moderate, and high rigor analysis processes. It also overviews the rigor metric as applied to the LNG Scenario study.