Annotated Memory References:
A Mechanism for Informed Cache Management
Alvin R. Lebeck, David A. Raymond, Chia-Lin Yang, Mithuna S. Thottethodi
Department of Computer Science
Durham, NC 27708 USA
Processor cycle time continues to decrease faster than main memory access times, placing higher demands on
cache memory hierarchy performance. To meet these demands, conventional cache management techniques that
rely solely on naive hardware must be augmented with more sophisticated techniques. This paper investigates
Informed Caching Environments (ICE) where software can assist in cache management. By exposing some cache
management mechanisms and providing an efficient interface for their use, software can complement existing hard-
ware techniques by providing hints on how to manage the cache.
In this paper, we focus primarily on a mechanism for software to convey information to the memory hierarchy.
We introduce a single instruction—called TAG—that can annotate subsequent memory references with a number of
bits, thus avoiding major modifications to the instruction set. Simulation results show that annotating all memory
reference instructions in the SPEC95 benchmarks increases execution time between 0% and 2% for both statically
and dynamically scheduleded processors. We show that exposing cache management mechanisms to software can
increase the performance of three media benchmarks (epic, pegwit, ijpeg) by managing replacement and block size
more effectively. We use source code inspection to annotate a subset of the load instructions such that we can
mark cache blocks for retain (epic) in a 4-way set associative cache, utilize a smaller cache block size (pegwit,
ijpeg), or default to normal cache operation. For the benchmarks considered, the results indicate that our methods
provide between 13% and 20% speedups on a 4-issue dynamically scheduled processor.
One of the key challenges facing computer architects is the increasing discrepanc