Published on GEOG 885: Advanced Analytic Methods in Geospatial Intelligence (https://www.e-education.psu.edu/geog885)

Home > GEOG 885 > Lesson 03: Structured Analytic Techniques

Lesson 03: Structured Analytic Techniques

Introduction

Introduction

Geospatial analysis can be very difficult to do well. Much of the difficulty is cognitive and not related to an individual's ability to use the technical tools, i.e., GIS. It takes far greater mental agility than gathering evidence supporting a single hypothesis that was pre-judged as the most likely answer. To develop and retain multiple spatial schemes in working memory and note how each item of information fits into each hypothesis is beyond the mental capabilities of most analysts. (Note: Working memory tasks include the active monitoring or manipulation of information or behaviors.)

Moreover, truly good geospatial analysis requires monitoring your progress, making changes and adapting the ways you are thinking. It's about self-reflection, self-responsibility, and initiative to achieve the analytic results within the time allotted. This mental agility can be accomplished with the help of a few simple thinking tools discussed here.

Lesson Objectives

At the end of this lesson, you will be able to:

  • Comprehend the role and purpose of Structured Analytic Techniques in geospatial analysis
  • Apply Structured Analytic Techniques in geospatial analysis

The Course Roadmap is intended to help you understand where we are in the overall learning process and to place our dual case study and project focus into context.

 Course Roadmap - more information in text description in link below
Figure 1: Course Roadmap
Click for text description of Figure 1

The image above shows a basic outline of assignments for all lessons in the course. Students are currently on Lesson 3. 

  • Lesson 1: Introduction and Review
    Case Study & Project
  • Lesson 2: Structured Geospatial Analytic Method (SGAM)
    Project
  • Lesson 3: Structured Analytical Techniques
    Project
  • Lesson 4: SGAM - Question and Grounding
    Case Study & Project
  • Lesson 5: SGAM - Hypothesis and Evidence
    Case Study & Project
  • Lesson 6: Diagnostic Techniques
    Project
  • Lesson 7: SGAM - Fusion and Conclusions
    Case Study
  • Lesson 8: Challenge Exercise
    Project
  • Lesson 9: Work Period
    Project
  • Lesson 10: Presentation
    Project

Lesson 3 is one week in length. (See the Calendar in Canvas for specific due dates.) To finish this lesson, you must complete the activities listed below. You may find it useful to print this page out first so that you can follow along with the directions.

Steps to Completing Lesson 03
Step Activity Access/Directions
1 Read the Lesson Overview and Checklist. You are in the Lesson 03 Checklist now. Click on the Previous Page to read the lesson Overview.
2 Read the Lesson Content. You are in the Lesson 03 online content now. Click on the Next Page to read the lesson content.
3 Read A Tradecraft Primer: Structured Analytic Techniques for Improving Intelligence Analysis [1]

Read or Listen to Richards Heuer's "The Evolution of Structured Analytic Techniques" [2] (Presentation to the National Academy of Science, National Research Council Committee on Behavioral and Social Science Research to Improve Intelligence Analysis for National Security, Washington, DC, December 8, 2009.)
There are three different styles of reading that are referred to in the lessons:
  • Scan: Do not deal with all of the content, but search through the material for a specific purpose or a specific word (or its synonym). Scanning is used for such purposes as finding the answer to a particular question.
  • Skim: To skim, read a page by reading the headings and first sentences of each paragraph or section.
  • Read: The purpose of this style is to understand the concepts and arguments that the text contains, and it should be preceded by the Skim reading style.
4 Read Lesson Summary. You are in the Lesson 3 online content now.

Questions?

If you have any questions now or at any point during this week, please feel free to post them to the GEOG 885 - General Discussion Forum. (That forum can be accessed at any time in Canvas by clicking on the Modules tab. The General Discussion forum is listed under the Orientation Section.)

Challenging Mindsets

Challenging Mindsets

Heuer makes three important points relative to intelligence in his work, the Psychology of Intelligence Analysis [3].

  • Human minds are ill-equipped ("poorly wired") to cope effectively with both inherent and induced uncertainty.
  • Increased knowledge of one's own inherent biases tends to be of little assistance to the analyst.
  • Tools and techniques that apply higher levels of critical thinking can substantially improve analysis of complex problems.

He provides the following series of images to illustrate how poorly we are cognitively equipped to accurately interpret the world.

Question #1: What did you see in the figure? The answer is at the bottom of this page.

Three triangles with text, see text description in link below
Figure 1
Source: Heuer, Psychology of Intelligence Analysis [3]
Click for text description of Figure 1

Figure 1 has three triangles, each one contains a common saying, except not quite right. From left to right the sayings are: Paris in the the Spring, Once in a a Lifetime, Bird in the the Hand

Question #2: Look at the drawing of the man in the upper right. Are the drawings all of men? The answer is at the bottom of this page.

 Two rows of four similar figures, see text description in link below
Figure 2
Source: Heuer, Psychology of Intelligence Analysis [3]
Click for text description of Figure 2

Figure has 2 rows of drawings. Four images are in each row, and all appear to be the same. The images can either look like a drawing of a man’s face or of a woman sitting. When you continue to look at each image and let your mind adjust to different possibilities, the drawings of a man also look like a sitting woman and the drawings of the sitting woman can also look like a man’s face. At the bottom of the figure, it says, “Impressions resist change.”

Question #3: What do you see - an old woman or a young woman? The answer is at the bottom of this page.

 Image of either a young woman or old woman, see text description in link below
Figure 3
Source: Heuer, Psychology of Intelligence Analysis [3]
Click for text description of Figure 3

Figure 3 appears to be a sketch of either an old woman or a young woman, depending on what you allow your mind to focus on. At the bottom of the image is written, “ It is difficult to look at the same information from different perspectives.”

Now look to see if you can reorganize the drawing to form a different image of a young woman; if your original perception was of an old woman, or of the old woman if you first perceived the young one.

According to Heuer, and as the above figures illustrate, mental models, or mindsets, or cognitive patterns are essentially the analogous image by which people perceive information. Even though every analyst sees the same piece of information, it is interpreted differently due to a variety of factors. In essence, one's perceptions are morphed by a variety of factors that are completely out of the control of the analyst. Heuer sees these cognitive patterns as potentially good and bad for the analyst. On the positive side, they tend to simplify information for the sake of comprehension, but they also bias interpretation. The key risks of mindsets are that:

  • analysts perceive what they expect to perceive;
  • once formed, they are resistant to change;
  • new information is assimilated, sometimes erroneously, into existing mental models; and
  • conflicting information is often dismissed or ignored.

Therefore, since all people observe the same information with inherent and different biases, Heuer believes an effective analysis method needs a few safeguards. The analysis method should:

  • encourage products that clearly show their assumptions and chains of inferences; and
  • emphasize procedures that expose alternative points of view.

What is required of analysts is a process for challenging, refining, and challenging again their own working mental models. This is a key component of his Structured Analytic Techniques (SATs), which include Analysis of Competing Hypotheses.

These problems notwithstanding, cognitive patterns are critical to allowing individuals to process what otherwise would be an incomprehensible volume of information. Yet, they can cause analysts to overlook, reject, or forget important incoming or missing information that is not in accord with their assumptions and expectations. Seasoned analysts may be more susceptible to these mindset problems as a result of their expertise and past success in using time-tested mental models.

Answers

Answer #1: The article is written twice in each of the three phrases. This is commonly overlooked because perception is influenced by our expectations about how these familiar phrases are normally written.

Answer #2: The above figure illustrates that mindsets tend to be quick to form but resistant to change by showing part of a longer series of progressively modified drawings that change almost imperceptibly from a man into a woman. The right-hand drawing in the top row, when viewed alone, has equal chances of being perceived as a man or a woman.

Answer #3: The old woman's nose, mouth, and eye are, respectively, the young woman's chin, necklace, and ear. The old woman is seen in profile looking left. The young woman is also looking left, but we see her mainly from behind, so most facial features are not visible. Her eyelash, nose, and the curve of her cheek may be seen just above the old woman's nose.

The Structured Analytic Techniques "Toolbox"

The Structured Analytic Techniques "Toolbox"

Structured techniques are simply a "box of tools" to help the analyst mitigate the adverse impact on analysis of one's cognitive limitations and pitfalls. Taken alone, they do not constitute an analytic method for solving geospatial analytic problems. The most distinctive characteristic is that structured techniques help to decompose one's geospatial thinking in a manner that enables it to be reviewed, documented, and critiqued.

Structured thinking in general, and structured geospatial thinking specifically, is at variance with the way in which the human mind is in the habit of working. Most people solve geospatial problems intuitively by trial and error. Structured analysis is a relatively new approach to intelligence analysis, with the driving forces behind the use of these techniques being:

  • an increased understanding of cognitive limitations and pitfalls that make intelligence analysis difficult,
  • prominent intelligence failures that have prompted reexamination of how intelligence analysis is generated,
  • DNI policy support and technical support for interagency collaboration, and
  • a desire by policy makers who receive analysis that it be more transparent as to how conclusions were reached.

In general, the Intelligence Community began focusing on structured techniques because analytic failures led to the recognition that it had to do a better job overcoming cognitive limitations, analytic pitfalls, and addressing the problems associated with mindsets. Structured analytic techniques help the mind think more rigorously about an analytic problem. In the geospatial realm, they ensure that our key geospatial assumptions, biases, and cognitive patterns are not just assumed correct but are well considered. The use of these techniques later helps to review the geospatial analysis and identify the cause of any error.

Moreover, a structured technique provides a variety of tools to help reach a conclusion. Even if both intuitive and scientific approaches provide the same degree of accuracy, structured techniques have value in that they can be easily used to balance the art and science of their analysis. It is clear is that structured methodologies are severely neglected by the geospatial community. Even in the rare cases where specific techniques are used, no one technique is appropriate to every step of the problem-solving process.

There are two ways to view the nature of these techniques. Heuer categorized structured techniques by how they help analysts overcome human cognitive limitations or pitfalls to analysis. Heuer's grouping is as follows:

  • Decomposition and Visualization: The number of things most people can keep in working memory at one time is seven, plus or minus two. Complexity increases geometrically as the number of variables increases. In other words, it is very difficult to do error-free analysis only in our heads. The two basic tools for coping with complexity in analysis are to: (1) break things down into their component parts, so that we can deal with each part separately, and (2) put all the parts down on paper or a computer screen in some organized manner such as a list, matrix, map, or tree so that we and others can see how they interrelate as we work with them. Many common techniques serve this purpose.
  • Indicators, Signposts, Scenarios: The human mind tends to see what it expects to see and to overlook the unexpected. Change often happens so gradually that we don’t see it, or we rationalize it as not being of fundamental importance until it is too obvious to ignore. Identification of indicators, signposts, and scenarios creates an awareness that prepares the mind to recognize change.
  • Challenging Mindsets: A simple definition of a mindset is “a set of expectations through which a human being sees the world.” Our mindset, or mental model of how things normally work in another country, enables us to make assumptions that fill in the gaps when needed evidence is missing or ambiguous. When this set of expectations turns out to be wrong, it often leads to intelligence failure. Techniques for challenging mindsets include reframing the question in a way that helps break mental blocks, structured confrontation such as devil’s advocacy or red teaming, and structured self-critique such as what we call a key assumptions check. In one sense, all structured techniques that are implemented in a small team or group process also serve to question your mindset. Team discussions help us identify and evaluate new evidence or arguments and expose us to diverse perspectives on the existing evidence or arguments.
  • Hypothesis Generation and Testing: “Satisficing” is the tendency to accept the first answer that comes to mind that is “good enough.” This is commonly followed by confirmation bias, which refers to looking at the evidence only from the perspective of whether or not it supports a preconceived answer. These are among the most common causes of intelligence failure. Good analysis requires identifying, considering, and weighing the evidence both for and against all the reasonably possible hypotheses, explanations, or outcomes. Analysis of Competing Hypotheses is one technique for doing this.
  • Group Process Techniques: Just as analytic techniques provide structure to our individual thought processes, they also provide structure to the interaction of analysts within a team or group. Most structured techniques are best used as a collaborative group process, because a group is more effective than an individual in generating new ideas and at least as effective in synthesizing divergent ideas. The structured process helps identify differences in perspective between team or group members, and this is good. The more divergent views are available, the stronger the eventual synthesis of these views. The specific techniques listed under this category, such as brainstorming and Delphi, are designed as group processes and can only be implemented in a group.

Others have grouped techniques by their purpose:

  • Diagnostic techniques are primarily aimed at making analytic arguments, assumptions, or intelligence gaps more transparent;
  • Contrarian techniques explicitly challenge current thinking; and
  • Imaginative thinking techniques aim at developing new insights, different perspectives and/or develop alternative outcomes. In fact, many of the techniques will do some combination of these functions.

These different groupings of the techniques notwithstanding, the analysts should select the technique that best accomplishes the specific task they set out for themselves. The techniques are not a guarantee of analytic precision or accuracy of judgments; they do improve the usefulness, sophistication, and credibility of intelligence assessments.

SATs and Geospatial Operations

SATs and Geospatial Operations

It is often difficult for an analyst to determine the next step in an analytic process, or to visualize how various techniques and tools fit together. Using the below as a list of the common GIS operations the analyst might use:

  • Data discovery and entry
  • Data conversion
  • Data validation
  • Spatial data management
  • Attribute data management
  • Data visualization
  • Data processing/analysis
  • Output of maps and reports

The Structured Geospatial Analytic Method (SGAM) provides the means to relate the analytical step to the Structured Analytic Technique (SAT) and then to the appropriate geospatial operation. The following table summarizes this mapping:

Overview of The Structured Geospatial Analytic Method
Structured Geospatial Analytic Method Step Structured Analytic Technique GIS Operation
Step 1: Question
  • Brainstorming
Step 2: Grounding
  • Brainstorming
  • Key Assumption Check
  • Quality of Information Check
  • Red Team
  • Data discovery and entry
  • Data conversion
  • Data validation
  • Spatial data management
  • Attribute data management
  • Data visualization
Step 3: Hypothesis Development
  • Brainstorming
  • Analysis of Competing Hypotheses (ACH)
  • Data visualization
Step 4: Evidence Development
  • Analysis of Competing Hypotheses (ACH)
  • Data visualization
  • Data processing/analysis
Step 5: Fusion
  • Analysis of Competing Hypotheses (ACH
  • Output of maps and reports
Step 6: Conclusions
  • Analysis of Competing Hypotheses (ACH)
  • Devil’s Advocacy
  • Output of maps and reports

Summary

Summary

The Intelligence Community began to use structured techniques because of analytic failures related to cognitive limitations and pitfalls or biases. The use of SATs does not guarantee getting intelligence analysis right, because there are so many uncertainties. SATs help to reduce the frequency and severity of error. These include SATs that partially overcome cognitive limitations, address analytic pitfalls, and confront the problems associated with mindsets. SATs help the mind think more rigorously about an analytic problem. Specifically, they ensure that assumptions, preconceptions, and mindsets are not taken for granted but are explicitly examined and tested. The use of SATs also helps to review the analysis and identify the cause of any error.

Looking ahead

The traditional approach to solving intelligence problems in general, and geospatial analysis in particular, is intuitive. It has been found that intelligence analysts who apply a process to an intelligence problem outperform those who rely on the intuitive approach. The next lesson discusses a method including high-level cognitive actions that structure the geospatial analysis process.


Source URL:https://www.e-education.psu.edu/geog885/l4.html

Links
[1] https://www.e-education.psu.edu/geog885/sites/www.e-education.psu.edu.geog885/files/geog885q//file/Lesson_03/Tradecraft_Primer-apr09.pdf [2] https://www.e-education.psu.edu/geog885/sites/www.e-education.psu.edu.geog885/files/file/Evolution_SAT_Heuer.pdf [3] https://www.e-education.psu.edu/geog885/sites/www.e-education.psu.edu.geog885/files/geog885q//file/Lesson_01/PsychofIntelNew.pdf