Four Top Reasons Mutual Information Does Not Quantify Neural Information Processing
Don H. Johnson
Rice University
Houston, Texas
Oral or Poster Presentation
1
Introduction
The mutual information between the stimulus□ and the response, whether the response be that of a single
neuron or of a population, has the form
□□□
□□
□ □□□
□ □
□ □ □
(1)
where □ □ is the probability distribution of the response for a given stimulus condition and □□
is the probability distribution of the stimulus. It is important to note that this probability distribution is
defined over the entire stimulus space. For example, if black-and-white images serve as the stimulus,
□□ is defined over all positive-valued signals having a compact two-dimensional domain. To measure
mutual information, the experimenter defines a stimulus set □□ □□ and, from the measured response,
estimates □ □, the probability distribution of the response under each stimulus condition. The mutual
information is estimated as [1]
□□□
□
□ □ □□
□ □
□ □ □□
(2)
where □□ is the probability of the th stimulus occurring.
In information theory, the mutual information is seldom used save for finding the capacity , defined to
be the maximum value of the mutual information over all stimulus probability distributions.
□□□
What other uses mutual information might have can be questioned. Because of the way mutual information
is calculated and used, several important issues not involving empirical concerns arise.
Mutual information depends on the stimulus probabilities. As can be seen from its definition (1) or
its estimate (2), mutual information depends on the stimulus probabilities. Mutual information measures
how different, in a statistical sense, the stimulus and response are. It equals zero when the response is
statistically independent of the stimulus and equals either □□ (2) or (1) when the response directly
reflects the stimulus.1 Mutual inf