Published on GEOG 885: Advanced Analytic Methods in Geospatial Intelligence (https://www.e-education.psu.edu/geog885)

Home > GEOG 885 > Lesson 07: SGAM: Fusion and Conclusions

Lesson 07: SGAM: Fusion and Conclusions

Introduction

Introduction

The lesson addresses the two highlighted elements of the Structured Geospatial Analytic Method (SGAM):

Full SGAM description at https://www.e-education.psu.edu/geog885/l3_p3.html. Step 5: Fusion; Step 6: Develop Conclusions are highlighted
Figure 1
Source:  T. Bacastow, D. Bellafiore, D. Bridges, S. Harter, The Pennsylvania State University, 2010.

Figure 1: Structured Geospatial Analytic Method Steps

Fusion is a term commonly used to describe how an organization arrives at a decision based upon data - facts! Traditionally, the field has been thought to be the domain of mathematicians and statisticians who could use facts to justify a decision. The goal of the conclusion is to summarize and stress the main ideas. Report both the conclusions and the other hypotheses that were tested and rejected. Last, because events are dynamic and subject to a variety of influences, analytic conclusions are always tentative. Therefore, specify in advance that certain occurrences, if observed, could cause significant changes in the probability of the accepted or alternative hypotheses. Collection of additional information can also suggest possibilities that may occur in the future.

Lesson Objectives

At the end of lesson 7 you will be able to:

  • Understand and apply the concepts and techniques to fuse evidence and hypotheses in the ACH process.
  • Understand and apply the aspects of developing and rendering conclusions and milestones.

The Course Roadmap is intended to help you understand where we are in the overall learning process and to place our dual case study and project focus into context.

 Course Roadmap: see text description in link below for more information
Figure 2: Course Roadmap
Click for text description of figure 2

The image above shows a basic outline of assignments for all lessons in the course. Students are currently on Lesson 7. 

  • Lesson 1: Introduction and Review
    Case Study & Project
  • Lesson 2: Structured Geospatial Analytic Method (SGAM)
    Project
  • Lesson 3: Structured Analytical Techniques
    Project
  • Lesson 4: SGAM - Question and Grounding
    Case Study & Project
  • Lesson 5: SGAM - Hypothesis and Evidence
    Case Study & Project
  • Lesson 6: Diagnostic Techniques
    Project
  • Lesson 7: SGAM - Fusion and Conclusions
    Case Study
  • Lesson 8: Challenge Exercise
    Project
  • Lesson 9: Work Period
    Project
  • Lesson 10: Presentation
    Project

Lesson 7 is one week in length. (See the Calendar in Canvas for specific due dates.) To finish this lesson, you must complete the activities listed below. You may find it useful to print this page out first so that you can follow along with the directions.

Steps to Completing Lesson 07
Step Activity Access/Directions
1 Read the lesson Overview and Checklist. You are in the Lesson 07 online content now. Click on the "Next Page" link to continue.
2
  • Read Lesson 07 online content.
  • Scan The Psychology of Intelligence Analysis [1]
  • Scan A Tradecraft Primer: Structured Analytic Techniques for Improving Intelligence Analysis [2]
There are three different styles of reading that are referred to in the lessons:
  • Scan: Do not deal with all of the content, but search through the material for a specific purpose or a specific word (or its synonym). Scanning is used for such purposes as finding the answer to a particular question.
  • Skim: To skim, read a page by reading the headings and first sentences of each paragraph or section.
  • Read: The purpose of this style is to understand the concepts and arguments that the text contains, and it should be preceded by the Skim reading style.
3 Participate in the Graded Discussion Expand upon (add information to) the example, DC Sniper: Analysis of Competing Hypothesis I [3] (Lesson 7, Part 5).

To participate in the discussion, please go to the Lesson 7 Discussion Forum in Canvas. (That forum can be accessed at any time by going to the Canvas link on the menu bar and then selecting Lesson 7 Discussion Forum from the appropriate weekly module.)
4 Project Deliverable:  Tentative Conclusions Provide a summary of your use of the Analysis of Competing Hypothesis (ACH) technique and the tentative conclusions.  Post your results to your team's Canvas project dropbox.

NOTE: All of the team's work will be contained in a single document file uploaded to the team's dropbox in Canvas.
5 Read lesson Summary. You are in the Lesson 7 online content now.

Questions?

If you have any questions now or at any point during this week, please feel free to post them to the GEOG 885 - General Discussion Forum. (That forum can be accessed at any time in Canvas by clicking on the Modules tab. The General Discussion forum is listed under the Orientation Section.)

SGAM Step 5: Fusion

SGAM Step 5: Fusion

Information Fusion

Why is this called the fusion step? The integration of data, recorded from multiple modes, together with knowledge, is known as fusion (Esteban, J, 2005 [4]). In general, fusion is the process one goes through in an analytic method so that the influence of unreliable sources can be lowered compared to the reliable sources (Kludas, 2007 [5]). Therefore, the multi-modal (graphical and text) nature of geospatial intelligence data analysis to reduce the influence of unreliable sources is essentially a fusion process. Step 5 of the SGAM is where it happens.

Evidence and Hypothesis Matrix

Fusion Step A: Prepare a matrix with hypotheses across the top and evidence and arguments down the side. Please note that your evidence and arguments may or may not be geospatial in nature. The matrix gives an overview of all the significant components of your analytical problem.

 Matrix with the hypotheses written across the top row and the evidence/argument data are written down the first column. The inner cells are used to indicate which hypotheses are supported by each peice of evidence.
Figure 3: Evidence and Hypothesis Matrix
Source:  T. Bacastow, D. Bellafiore, D. Bridges, S. Harter, The Pennsylvania State University, 2010.

Fusion Step B: Working down the column to each piece of evidence and then across the rows of the matrix, examine one item of evidence at a time to see how consistent that item of evidence is with each of the hypotheses. Later you will work across the columns of the matrix, examining one hypothesis at a time, to see how consistent that hypothesis is with all the evidence.

 Evidence and Hypotheses matrix with an arrow pointing down next to the column with each piece of evidence numbered, showing how to work through the matrix as described in the text.
Figure 4: Working down the columns of the matrix
Source:  T. Bacastow, D. Bellafiore, D. Bridges, S. Harter, The Pennsylvania State University, 2010.

To fill in the matrix, take the first item of evidence and ask whether it is consistent with, inconsistent with, or irrelevant to each hypothesis. Then make a notation accordingly in the appropriate cell under each hypothesis in the matrix. The form of these notations in the matrix is a matter of personal preference. It may be pluses, minuses, and question marks. It may be C, I, and N/A standing for consistent, inconsistent, or not applicable. Or it may be some textual notation. Simply put, what you use is a shorthand representation of the complex reasoning that went on as you thought about how the evidence relates to each hypothesis. In some cases, it may be useful to refine this procedure by using a numerical probability, rather than a general notation such as plus or minus, to describe how the evidence relates to each hypothesis.

Analyze the "diagnosticity" of each piece of evidence and each argument as you complete the matrix. This step is the most important; it is the step that deviates most from the intuitive approach to geospatial analysis. Diagnosticity is how helpful the evidence or argument is in judging the relative likelihood of alternative hypotheses. Diagnosticity of evidence is an important concept that is unfamiliar to many geospatial analysts. Diagnosticity may be illustrated by a medical analogy. A high-temperature reading may have great value in telling a doctor that a patient is sick, but relatively little value in determining which illness a person is suffering from. Because a high temperature is consistent with so many possible hypotheses about a patient's illness, this evidence has limited diagnostic value in determining which illness (hypothesis) is the more likely one.

The matrix format helps you weigh the diagnosticity of each item of evidence. If an item of evidence seems consistent with all the hypotheses, it may have no diagnostic value. A common experience is to discover that most of the evidence supporting what you believe is the most likely hypothesis really is not very helpful, because that same evidence is also consistent with other hypotheses. When you do identify items that are highly diagnostic, these should drive your judgment.

Matrix Refinement

Fusion Step C: Refine the matrix by reconsidering the hypotheses, and delete evidence and arguments that have no diagnostic value. The wording of the hypotheses is critical to the conclusions one can draw from the analysis. By this point, you will have seen how the evidence breaks out under each hypothesis, and it is necessary to reconsider and reword the hypotheses. New hypotheses may need to be added, or finer distinctions may need to be made in order to consider all the significant alternatives. If there is little or no evidence that helps distinguish between two hypotheses, consider combining the hypotheses.

Also, reconsider the geospatial evidence. Question if your thinking about which hypotheses are most likely and least likely is influenced by factors that are not included in the evidence. If so, use the data and tools in hand to create this evidence and include it in your analysis. Delete from the matrix items of evidence or arguments that now seem unimportant or have no diagnostic value. Save these items in a separate list as a record of information that was considered.

SGAM Step 6: Conclusions

SGAM Step 6: Conclusions

Draw Your Tentative Conclusions

Conclusions Step A: You will now draw tentative conclusions about the relative likelihood of each hypothesis by trying to disprove the hypotheses rather than prove them. The matrix format gives an overview of all the evidence for and against all the hypotheses, so that you can examine all the hypotheses together and have them compete against each other. Previously you analyzed the "diagnosticity" of the evidence and arguments by working down and across the matrix, focusing on a single item of evidence and examining how it relates to each hypothesis. Now, work across and down the matrix, looking at each hypothesis and the evidence as a whole.

 Evidence and Hypotheses matrix with an arrow pointing to the right above the row with each hypothesis, showing how to work through the matrix as described in the text.
Figure 5: Working across the matrix
Source:  T. Bacastow, D. Bellafiore, D. Bridges, S. Harter, The Pennsylvania State University, 2010.

Analysts have a natural tendency to concentrate on confirming hypotheses they already believe to be true, and giving more weight to information that supports a favored hypothesis than to information that weakens it. Moreover, no matter how much information is consistent with a given hypothesis, you cannot prove that hypothesis is true, because the same information may also be consistent with one or more other hypotheses. On the other hand, a single item of evidence that is inconsistent with a hypothesis may be sufficient grounds for rejecting that hypothesis.

This step requires doing the opposite of what comes intuitively when evaluating the relative likelihood of alternative hypotheses, by looking for evidence or arguments that enable you to possibly reject the hypothesis or determine that it is unlikely. This follows a fundamental concept of the scientific method of rejecting or eliminating hypotheses, while tentatively accepting only those hypotheses that cannot be refuted. The scientific method obviously cannot be applied without the analyst's intuitive judgment, but the principle of seeking to disprove hypotheses, rather than confirm them, helps to overcome the natural tendency to favor one hypothesis.

In examining the matrix, look at the minuses, or whatever other notation you used to indicate geospatial evidence that may be inconsistent with a hypothesis. The hypothesis with the fewest minuses is probably the most likely one. The hypothesis with the most minuses is probably the least likely one. The fact that a hypothesis is inconsistent with the evidence is certainly a sound basis for rejecting it. The pluses, indicating evidence that is consistent with a hypothesis, are far less significant. It does not follow that the hypothesis with the most pluses is the most likely one, because a long list of evidence that is consistent with almost any reasonable hypothesis can be easily made. What is difficult to find, and is most significant when found, is hard evidence that is clearly inconsistent with a reasonable hypothesis.

This initial ranking by number of minuses is only a rough ranking, however, as some evidence obviously is more important than other evidence, and degrees of inconsistency cannot be captured by a single notation such as a plus or minus. By reconsidering the exact nature of the relationship between the evidence and the hypotheses, you will be able to judge how much weight to give it. Analysts who follow this procedure often realize that their judgments are actually based on very few factors rather than on the large mass of information they thought was influencing their views.

Applying Judgment

Conclusions Step B: The matrix does not dictate your conclusion. Rather, it should accurately reflect a judgment of what is important and how the evidence relates to the probability of each hypothesis. You, not the matrix, makes the decision. The matrix only organizes your analysis, to ensure consideration of all the possible interrelationships between evidence and hypotheses, and to identify those items that heavily influence your reasoning.

The matrix may show a hypothesis is probable, and you may disagree. If so, it is because you omitted from the matrix evidence or arguments that influenced on your judgment. Go back and add the evidence or argument, so that the analysis reflects your best judgment. Following this procedure will cause you to consider things you might otherwise have overlooked or revise your earlier estimate of the relative probabilities of the hypotheses. When you are done, the matrix serves as an audit trail of your thinking and analysis.

Importantly, this process forces you to spend more analytical time than you otherwise would on what you had thought were the less likely hypotheses. The seemingly less likely hypotheses usually involve more work. What you started out thinking was the most likely hypothesis tends to be based on a continuation of your recalled geospatial experiences and patterns. A principal advantage of the analysis of competing hypotheses to geospatial intelligence is that it forces you to consider the alternatives.

Sensitivity Analysis

Conclusions Step C: You should analyze how sensitive your conclusion is to a few critical items of evidence. This is done by considering the consequences for your geospatial analysis if that evidence were wrong, misleading, or subject to a different interpretation. In Step 5, you identified the evidence and arguments that were most diagnostic, and later you used these findings to make tentative judgments about the hypotheses. Now, go back and question the key items of evidence that drive the outcome of the analysis:

  • Are there questionable assumptions that underlie your understanding and interpretation?
  • Are there alternative explanations or interpretations?
  • Could the evidence be incomplete and, therefore, misleading?

If there is any concern at all about denial of information and/or deception, this is an appropriate place to consider that possibility. Look at the sources of key evidence:

  • Are any of the sources known to be controlled by a distrusted actor?
  • Could the sources have been duped and provided information that was planted?
  • Could the information have been manipulated?

Put yourself in the shoes of a deception planner to evaluate motive, opportunity, means, costs, and benefits of deception as they might appear to the opponent.

Report Your Conclusions

Conclusions Step D: In this step, you may decide that additional research is needed to check key judgments. It may be appropriate to go back to check original source materials rather than relying on someone else's interpretation. In writing your report, it is desirable to identify critical assumptions that went into your interpretation and to note that your conclusion is dependent upon the validity of these assumptions. If your report is to be used as the basis for decision-making, it is appropriate to provide the relative likelihood of alternative possibilities. Analytical judgments are never certain. Decision-makers need to make decisions on the basis of a full set of alternative possibilities, not just the single most likely alternative. You should consider a fallback plan in case one of the less likely alternatives turns out to be true.

Identify Milestones to Monitor

Conclusions Step E: Analytical conclusions are always tentative, therefore it is important to identify milestones that may indicate events are taking a different course than predicted. It is always helpful to specify things one should look for that suggest a significant change in the probabilities. This is useful for intelligence consumers who are following the situation on a continuing basis. Specifying in advance what would cause you to change your mind will also make it more difficult for you to rationalize such developments as not really requiring any modification of your judgment.

DC Sniper Case Study: Analysis of Competing Hypothesis II

DC Sniper Case Study: Analysis of Competing Hypothesis II

Submissions Instructions: Answer the questions at the end of this page. Post your analysis to the Lesson 7 Discussion Forum.

Purpose: To complete the ACH steps of Fusion and Conclusion.

General: The following information is provided:

A matrix with hypotheses across the top and evidence and arguments down the side are developed. Note that your evidence and arguments may or may not be geospatial in nature.

Hypotheses Matrix
Evidence/Argument H1 H2 H3 H4
Michael's
employee
Foreign
terrorist
Serial
killer
Domestic
terrorist
1 Conforms to the geospatial model of a serial criminal
2 All DC-area killings occurred within 30 days
3 The majority of the shootings were at or near shopping centers
4 Shootings were all on major highways or interstates
5 There was only one shooting per location and often occurred at gas stations
6 Sighting of a blue car with two black men
7 Sighting of a white van with two individuals at one killing
8 Military caliber weapon (5.56mm)
9 Noise heard but shooter never seen

Work down the evidence column and across the rows of the matrix, examining one item of evidence at a time to see how consistent that item of evidence is with each of the hypotheses. Make a notation C, I, and N/A standing for consistent, inconsistent, or not applicable.

Hypotheses Matrix with Notations
Evidence/Argument H1 H2 H3 H4
Michael's
employee
Foreign
terrorist
Serial
killer
Domestic
terrorist
1 Conforms to the geospatial model of a serial criminal I C I C
2 All DC-area killings occurred within 30 days C C CC C
3 The majority of the shootings were at or near shopping centers I CC C CC
4 Shootings were all on major highways or interstates I CC CC CC
5 There was only one shooting per location and often occurred at gas stations I II CC CC
6 Sighting of a blue car with two black men I I I I
7 Sighting of a white van with two individuals at one killing I C C C
8 Military caliber weapon (5.56mm) C C C C
9 Noise heard but shooter never seen I I CC CC

Analyze the "diagnosticity" of each piece of evidence. We discover that the evidence of the rifle's caliber is not very helpful because it is consistent with all hypotheses. Refine the matrix by reconsidering the hypotheses and delete evidence and arguments that have no diagnostic value. In the case, I deleted the "Military caliber weapon (5.56mm)" evidence since it offered no diagnosticity.

Hypotheses Matrix with Notations
Evidence/Argument H1 H2 H3 H4
Michael's
employee
Foreign
terrorist
Serial
killer
Domestic
terrorist
1 Conforms to the geospatial model of a serial criminal I C I C
2 All DC-area killings occurred within 30 days I CC C CC
3 The majority of the shootings were at or near shopping centers I CC CC CC
4 Shootings were all on major highways or interstates I II CC CC
5 There was only one shooting per location and often occurred at gas stations I II C I
6 Sighting of a blue car with two black men I C C C
7 Sighting of a white van with two individuals at one killing I C C CC
8 Noise heard but shooter never seen I I CC C

Examine each hypothesis one at a time by looking down the column to consider each hypothesis as a whole. Draw tentative conclusions by trying to disprove the hypotheses. Look at the “I”s to indicate evidence that may be inconsistent with a hypothesis. Generally, the hypothesis with the fewest “I”s is probably the most likely one. The hypothesis with the most “I”s is probably the least likely one.

Hypotheses Matrix with Summary
Evidence/Argument H1 H2 H3 H4
Michael's
employee
Foreign
terrorist
Serial
killer
Domestic
terrorist
Summary I=-8
C=0
I=-5
C=7
I=-1
C=10
I=-1
C=11

What to do: Complete the questions concerning the Fusion and Conclusions in the DC Sniper Case study:

  1. What conclusion would you draw from this analysis?
  2. What is offered is an example. How would you improve this ACH analysis?
  3. What events (milestones) might cause you to change your conclusions?
  4. Now that you have completed this effort using the SGAM, how would you compare this analysis to the Wikipedia summary [6] of the DC Sniper case?

Team Project: ACH and Tentative Conclusions

Team Project: ACH and Tentative Conclusions

Provide a summary of your use of the Analysis of Competing Hypothesis (ACH) technique and the tentative conclusions. Post your results to your team's Canvas folder.

Summary

Summary

Most geospatial analysts go about their business by picking out what they suspect intuitively is the most likely answer, then looking at the available information from the point of view of whether or not it supports this answer. If the evidence seems to support the favorite hypothesis, the analyst looks no further. If it does not, the analyst either rejects the evidence as misleading or develops another hypothesis and goes through the same procedure again. Decision analysts call this a satisficing strategy. Satisficing is picking the first solution that seems satisfactory, rather than going through all the possibilities to identify the very best solution. There may be several seemingly satisfactory solutions, but there is only one best solution. SGAM uses Analysis of Competing Hypotheses (ACH) which requires an analyst to explicitly identify all the reasonable alternatives and have them compete against each other for the analyst's favor, rather than evaluating their plausibility one at a time.

Looking ahead

In Lesson 8, we will continue work on the Capstone project.


Source URL:https://www.e-education.psu.edu/geog885/l8.html

Links
[1] https://www.e-education.psu.edu/geog885/sites/www.e-education.psu.edu.geog885/files/geog885q//file/Lesson_07/PsychofIntelNew.pdf [2] https://www.e-education.psu.edu/geog885/sites/www.e-education.psu.edu.geog885/files/geog885q//file/Lesson_07/Tradecraft_Primer-apr09.pdf [3] https://www.e-education.psu.edu/geog885/l8_p5.html [4] https://uhra.herts.ac.uk/bitstream/handle/2299/2246/902341.pdf;sequence=1 [5] http://vision.unige.ch/publications/postscript/2007/Kludas.amr07.
pdf [6] https://en.wikipedia.org/wiki/D.C._sniper_attacks