Joint Committee on Quantitative Assessment of Research
A report from the International Mathematical Union (IMU) in
cooperation with the International Council of Industrial and
Applied Mathematics (ICIAM) and the Institute of Mathematical
Robert Adler, John Ewing (Chair), Peter Taylor
This is a report about the use and misuse of citation data in the assessment of scientific research. The
idea that research assessment must be done using "simple and objective" methods is increasingly
prevalent today. The "simple and objective" methods are broadly interpreted as bibliometrics, that is,
citation data and the statistics derived from them. There is a belief that citation statistics are inherently
more accurate because they substitute simple numbers for complex judgments, and hence overcome
the possible subjectivity of peer review. But this belief is unfounded.
• Relying on statistics is not more accurate when the statistics are improperly used. Indeed,
statistics can mislead when they are misapplied or misunderstood. Much of modern
bibliometrics seems to rely on experience and intuition about the interpretation and validity of
• While numbers appear to be "objective", their objectivity can be illusory. The meaning of a
citation can be even more subjective than peer review. Because this subjectivity is less obvious
for citations, those who use citation data are less likely to understand their limitations.
• The sole reliance on citation data provides at best an incomplete and often shallow
understanding of research—an understanding that is valid only when reinforced by other
judgments. Numbers are not inherently superior to sound judgments.
Using citation data to assess research ultimately means using citation‐based statistics to rank things—
journals, papers, people, programs,