Computational mechanisms for gaze direction in interactive visual
environments
Robert J. Peters∗
Department of Computer Science
University of Southern California
Laurent Itti†
Departments of Computer Science, Neuroscience and Psychology
University of Southern California
Figure 1: Four game frames with the eye position of one human (small cyan square, arrows) gazing back and forth between
Mario and his enemy in a video game console system. Computational image-processing heuristics drawn from biological vision
could help such systems predict and adapt to the behavior of their human operators.
Abstract
Next-generation immersive virtual environments and video
games will require virtual agents with human-like visual at-
tention and gaze behaviors. A critical step is to devise ef-
ficient visual processing heuristics to select locations that
would attract human gaze in complex dynamic environ-
ments. One promising approach to designing such heuris-
tics draws on ideas from computational neuroscience. We
compared several such heuristics with eye movement record-
ings from five observers playing video games, and found that
heuristics which detect outliers from the global distribution
of visual features were better predictors of human gaze than
were purely local heuristics. Heuristics sensitive to dynamic
events performed best overall. Further, heuristic prediction
power differed more between games than between differ-
ent human observers. Our findings suggest simple neurally-
inspired algorithmic methods to predict where humans look
while playing video games.
CR Categories:
H.1.2
[Models and Principles]:
User/Machine Systems—Software Psychology I.2.10 [Ar-
tificial Intelligence]: Vision and Scene Understanding—
Perceptual Reasoning I.3.3 [Computer Graphics]:
Pic-
ture/Image Generation—Viewing Algorithms
∗e-mail:
rjpeters@usc.edu;
website:
http://ilab.usc.edu/rjpeters/
†e-mail: itti@usc.edu; website: http://ilab.usc.edu/
Keywords: visual attention, active vision, eye movements,
computational modeling, video games, immersi