Confirmation Bias in the Analysis of Remote Sensing Data
Paul E. Lehner, Leonard Adelman,
Robert J. DiStasio Jr., Marie C. Erie, Janet S. Mittel, Sherry L. Olson
Abstract
In practical application, analysis of remote sensing data requires a mix of technical
analysis and best expert judgment. Unfortunately, a substantial experimental literature on
judgment indicates that expert judgment is systematically flawed. In particular, experts
are prone to a confirmation bias – where focus on a proposed hypothesis leads the expert
to seek and overweigh confirming versus disconfirming evidence. In remote sensing, this
predicts a tendency toward false positives in interpretation - concluding the evidence
supports a hypothesis when it doesn’t. In this paper, we empirically examine
confirmation bias in technical data analysis, along with an approach to mitigating this
bias that systematically promotes consideration of alternative causes in the analysis.
Results suggest that analysts do exhibit confirmation bias in their technical analysis of
remote sensing data; and furthermore that structured consideration of alternative causes
mitigates this bias.
Introduction
Consider the following hypothetical scenario. As a result of recent flooding, there are
hundreds of possible locations where caustic chemicals may be leaking into the
waterways. Scarce containment resources must be quickly dispatched to the few
locations where spills have actually occurred. Response time is critical. At the site of a
particular chemical plant there is concern that ethylene glycol, a dangerous substance, is
leaching into the waterways upstream from a residential area. A flyover was conducted
in response to the flooding, where a small plume of discolored water was visible from the
air. Spectral data from this flight was processed to examine the area of the observed
plume. Analysis of the spectral sensor data is a highly technical data analysis task
requiring a specialist in spectral remote sensing,