Published on GEOG 885: Advanced Analytic Methods in Geospatial Intelligence (https://www.e-education.psu.edu/geog885)

Home > GEOG 885 > Lesson 05: SGAM: Hypotheses and Evidence

Lesson 05: SGAM: Hypotheses and Evidence

Introduction

Introduction

The lesson addresses the two highlighted elements of the Structured Geospatial Analytic Method (SGAM):

Full SGAM description at https://www.e-education.psu.edu/geog885/l3_p3.html. Step 3: Hypothesis Development; Step 4: Evidence Development are highlighted
Figure 1: Elements of the Structured Geospatial Analytic Method
Source:  T. Bacastow, D. Bellafiore, D. Bridges, S. Harter, The Pennsylvania State University, 2010.

This lesson begins with developing hypotheses and evidence, which is the beginning of the sensemaking and Analysis of Competing Hypotheses (ACH) process. Developing good hypotheses requires both divergent thinking, to ensure that all hypotheses are considered, and convergent thinking to ensure that redundant and irrational hypotheses are eliminated. Evidence should include all factors, both spatial and non-spatial, that might have an impact on judgments about the hypotheses.

ACH is a general purpose intelligence analysis methodology developed by Richards Heuer while he was an analyst at the Central Intelligence Agency (CIA). ACH draws on the scientific method, cognitive psychology, and decision analysis. ACH became widely available when the CIA published Heuer’s The Psychology of Intelligence Analysis. The ACH methodology can help the geospatial analyst overcome cognitive biases common to analysis in national security, law enforcement, and competitive intelligence. ACH forces analysts to disprove hypotheses rather than jump to conclusions and permit biases and mindsets to determine the outcome. ACH is a very logical step-by-step process that has been incorporated into our Structured Geospatial Analytical Method. A synopsis of the method as it can be applied to geospatial analysis in discussed in Lessons 4,5, and 7. A complete discussion of ACH is found in Chapter 8 of Heuer’s book. [1]

Lesson Objectives

At the end of lessons 5, 6 & 7 you will be able to:

  • Understand Analysis of Competing Hypotheses (ACH).
  • Apply ACH in the context of the Structured Geospatial Analytic Method.

The Course Roadmap is intended to help you understand where we are in the overall learning process and to place our dual case study and project focus into context.

Course Roadmap: see text description in link below
Figure 2: Course Roadmap
Click for text description of Figure 2

The image above shows a basic outline of assignments for all lessons in the course. Students are currently on Lesson 5. 

  • Lesson 1: Introduction and Review
    Case Study & Project
  • Lesson 2: Structured Geospatial Analytic Method (SGAM)
    Project
  • Lesson 3: Structured Analytical Techniques
    Project
  • Lesson 4: SGAM - Question and Grounding
    Case Study & Project
  • Lesson 5: SGAM - Hypothesis and Evidence
    Case Study & Project
  • Lesson 6: Diagnostic Techniques
    Project
  • Lesson 7: SGAM - Fusion and Conclusions
    Case Study
  • Lesson 8: Challenge Exercise
    Project
  • Lesson 9: Work Period
    Project
  • Lesson 10: Presentation
    Project

Lesson 5 is one week in length. (See the Calendar in Canvas for specific due dates.) To finish this lesson, you must complete the activities listed below. You may find it useful to print this page out first so that you can follow along with the directions.

Steps to Completing Lesson 05
Step Activity Access/Directions
1 Read the lesson Overview and Checklist. You are in the Lesson 05 online content now. Click on the "Next Page" link to continue.
2
  • Read Lesson 05 online content.
  • Scan The Psychology of Intelligence Analysis [2]
  • Scan A Tradecraft Primer: Structured Analytic Techniques for Improving Intelligence Analysis [3]
There are three different styles of reading that are referred to in the lessons:
  • Scan: Do not deal with all of the content, but search through the material for a specific purpose or a specific word (or its synonym). Scanning is used for such purposes as finding the answer to a particular question.
  • Skim: To skim, read a page by reading the headings and first sentences of each paragraph or section.
  • Read: The purpose of this style is to understand the concepts and arguments that the text contains, and it should be preceded by the Skim reading style.
3 Participate in the Graded Discussion. Expand upon (add to or change) the example, DC Sniper: Analysis of Competing Hypothesis I [4] (Lesson 5, Part 5).

To participate in the discussion, please go to the Lesson 5 Discussion Forum in Canvas. (That forum can be accessed at any time by going to the Canvas link on the menu bar and then selecting Lesson 5 Discussion Forum from the appropriate weekly module.)
4 Complete the Imaginative Thinking Techniques Exercise. Complete the Imaginative Thinking Techniques Exercise [5] (Lesson 5, Part 6).  Post your results to your team's Canvas project dropbox.

NOTE: All of the team's work will be contained in a single document file uploaded to the team's dropbox in Canvas.
5 Read lesson Summary. You are in the Lesson 5 online content now.

Questions?

If you have any questions now or at any point during this week, please feel free to post them to the GEOG 885 - General Discussion Forum. (That forum can be accessed at any time in Canvas by clicking on the Modules tab. The General Discussion forum is listed under the Orientation Section.)

SGAM Step 3: Hypothesis

SGAM Step 3: Hypothesis

The third step in the process is to identify all hypotheses that merit detailed examination, keeping in mind that there is a distinction between the hypothesis generation and hypothesis evaluation. If the analyst does not begin with the correct hypothesis, it is unlikely to get the correct answer. Psychological research into how people go about generating hypotheses shows that people are actually rather poor at thinking of all the possibilities. Therefore, at the hypothesis generation stage, it is wise to bring together a group of analysts with different backgrounds and perspectives for a brainstorming session.

Brainstorming in a group stimulates the imagination and usually brings out possibilities that individual members of the group had not thought of. Experience shows that initial discussion in the group elicits every possibility, no matter how remote, before judging likelihood or feasibility. Only when all the possibilities are on the table is the focus on judging them and selecting the hypotheses to be examined in greater detail in subsequent analysis.

When screening out the seemingly improbable hypotheses, it is necessary to distinguish hypotheses that appear to be disproved from those that are simply unproven. For an unproven hypothesis, there is no evidence that it is correct. For a disproved hypothesis, there is positive evidence that it is wrong. Early rejection of unproven, but not disproved, hypotheses biases the analysis, because one does not then look for the evidence that might support them. Unproven hypotheses should be kept alive until they can be disproved.

One example of a hypothesis that often falls into this unproven but not disproved category is the hypothesis that an opponent is trying to deceive us. You may reject the possibility of denial and deception because you see no evidence of it, but rejection is not justified under these circumstances. If deception is planned well and properly implemented, one should not expect to find evidence of it readily at hand. The possibility should not be rejected until it is disproved, or, at least, until after a systematic search for evidence has been made and none has been found.

There is no "correct" number of hypotheses to be considered. The number depends upon the nature of the analytical problem and how advanced you are in the analysis of it. As a general rule, the greater your level of uncertainty, or the greater the impact of your conclusion, the more alternatives you may wish to consider. More than seven hypotheses may be unmanageable; if there are this many alternatives, it may be advisable to group several of them together for your initial cut at the analysis.

SGAM Step 4: Evidence

SGAM Step 4: Evidence

As stated earlier, much of geospatial intelligence work never departs the foraging loop (steps 1-4) and simply consists of extracting information and repackaging it without much actual analysis.

In assembling the evidence, list all the factors that may have an impact upon judgments about the hypotheses. Do not limit the factors to concrete evidence in the current intelligence reporting. Also, include all assumptions or logical deductions. These assumptions may generate strong preconceptions as to which hypothesis is most likely. Such assumptions often drive the final judgment, so it is important to include them in the list of "evidence."

First, list the general evidence that applies to all the hypotheses. Then consider each hypothesis individually, listing factors that tend to support or contradict each one. You will commonly find that each hypothesis leads you to ask different questions and, therefore, to seek out somewhat different evidence.

For each hypothesis, ask yourself this question: If this hypothesis is true, what should I expect to be seeing or not seeing? What are all the things that must have happened, or may still be happening, and that one should expect to see evidence of? If you are not seeing this evidence, why not? Is it because it has not happened, it is not normally observable, it is being concealed from you, or because you or the intelligence collectors have not looked for it?

Note the absence of evidence as well as its presence. For example, when weighing the possibility of military attack by an adversary, the steps the adversary has not taken to ready his forces for attack may be more significant than the observable steps that have been taken. In the case of a business, some competitors like to leak information to the press to condition the market, while others suddenly announce new offerings that take the industry by surprise. This recalls the Sherlock Holmes story, in which the vital clue was that the dog did not bark in the night. One's attention tends to focus on what is reported, rather than what is not reported. It requires a conscious effort to think about what is missing but should be present if a given hypothesis were true.

DC Sniper: Analysis of Competing Hypothesis I

DC Sniper Case: Example Hypothesis and Evidence Development

Submissions Instructions: Expand upon (add to or change) the example hypotheses and evidence. Post your analysis to the Lesson 5 Discussion Forum.

Purpose: To complete the ACH steps of hypothesis generation and evidence development.

General. Analysis of Competing Hypotheses, ACH, is a simple model for how to think about and fuse geospatial information into analytic problems. As we have included ACH in our geospatial analytic method, it takes the geospatial analyst through a process to make well-reasoned, analytical judgments using both non-geospatial and geospatial information. It is particularly useful for predictions of what is likely to happen in the future. It helps an analyst to minimize some of the cognitive limitations we discussed earlier. The key elements of ACH we are going to address in this exercise are:

  1. Identify the possible hypotheses to be considered. Use a group of analysts with different perspectives to brainstorm the possibilities.
  2. Make a list of evidence and arguments for and against each hypothesis.

According to Heuer, a hypothesis is a testable proposition about what is true, or about what has happened, is happening, or will happen. It should usually be worded as a positive statement that can be disproved. The most effective hypotheses meet two tests. (1) The hypotheses are mutually exclusive. That means if one hypothesis is true, then all other hypotheses must be false - in other words, no overlap between hypotheses. (2) The hypotheses cover all reasonable possibilities, including those that seem unlikely but not impossible. As evidence is collected and added to the matrix, you may find that hypotheses need to be reworded, added, deleted, or combined.

Again, according to Heuer, when identifying relevant evidence, consider each hypothesis, one at a time, and ask yourself: What evidence would I expect to see if this hypothesis is true? Also, what evidence would I expect not to see if this hypothesis is true? You will generally find that each hypothesis leads you to seek out somewhat different evidence. Evaluation of all reasonable hypotheses may require that you explore hypotheses you have not seriously considered before. To appreciate the geospatial aspects of the problem, it will be necessary to map the data!

What to do: Expand upon (add, delete, or modify information) the following hypothesis and evidence lists developed from the DC Sniper Case study:

  1. Initial hypotheses:
    • Disgruntled Michael's employee
    • Foreign terrorist
    • Serial killer
    • Domestic terrorist
  2. Initial evidence:
    • Conforms to the geospatial model of a serial criminal.
    • The majority of the shootings were at or near shopping centers. That is, the events are not near governmental buildings, indicating that government is not a target.
    • Shootings were all on major highways or interstates for a quick exit from the scene. This indicates a desire not to be caught.
    • There was only one shooting per location and often occurred at gas stations. This seems to indicate our Sniper is not in the role of a suicide terrorist.
    • Sighting of a blue car with two black men.
    • Sighting of a white van with two individuals at one killing.
    • Military caliber weapon (5.56mm). This caliber is common to the US military and is also a sporting cartridge.
    • Noise heard, but shooter never seen. This possibly indicates some sniper training.

Team Project: Analytic Question and Imaginative Thinking Techniques Exercise

Team Project: Analytic Question and Imaginative Thinking Techniques Exercise

To help develop the project, your team will use one of the Imaginative Thinking Techniques found in A Tradecraft Primer: Structured Analytic Techniques for Improving Intelligence Analysis [3].

We often overlook the obvious, formulate faulty conclusions, and fail to consider alternative possibilities, not because we lack the ability to think rationally, but because we fail to take charge of how we approach problems. We must not only think, we must think about how we think. This interactive online course defines what critical thinking is and how to do it.

Your team will use one of the following techniques to examine your analytic question:

  • Brainstorming; an unconstrained group process designed to generate new ideas and concepts.
  • Outside-in Thinking; used to identify the full range of basic forces, factors, and trends that would indirectly shape an issue.
  • Red Team Analysis; models the behavior of an individual or group by trying to replicate how an adversary would think about an issue.
  • Alternative Futures Analysis; systematically explores multiple ways a situation can develop when there is high complexity and uncertainty.

Complete the Imaginative Thinking Techniques Exercise. Post a summary of how you used one of the techniques and the results to your team's Canvas folder.

Summary

Summary

At the end of lesson 5 you should be able to:

  • Understand and apply the Systems Thinking Intelligence Framework.
  • Understand and apply Diagnostic Techniques.

Looking ahead

In Lesson 6, we will study the Imagination Techniques and Indicators and understand how they are applied.


Source URL:https://www.e-education.psu.edu/geog885/l6.html

Links
[1] https://www.e-education.psu.edu/geog885/sites/www.e-education.psu.edu.geog885/files/geog885q//file/Lesson_01/PsychofIntelNew.pdf [2] https://www.e-education.psu.edu/geog885/sites/www.e-education.psu.edu.geog885/files/geog885q//file/Lesson_05/PsychofIntelNew.pdf [3] https://www.e-education.psu.edu/geog885/sites/www.e-education.psu.edu.geog885/files/geog885q//file/Lesson_05/Tradecraft_Primer-apr09.pdf [4] https://www.e-education.psu.edu/geog885/node/2102 [5] https://www.e-education.psu.edu/geog885/node/2105