Cognitive maps were proposed as an alternative to stimulus-response explanations of animal
behavior. Although the concept of cognitive maps advanced treatments of complex animal
behavior, it has remained resistant to theoretical definition. A simplified perspective on
cognitive maps that focused on spatial behavior and the construction of spatial maps has
provided an important approach to understanding the role of the hippocampus in spatial
behavior and spatially modulated neural activity, particularly within the hippocampus.
However, this perspective leaves open many questions on how spatial maps and neural
activities within the hippocampus are used and how they contribute to selection of adaptive
A reinforcement learning approach to animal behavior was used to develop a theory of
cognitive map function. Reinforcement learning provides a theoretical framework within
which the components of cognitive map function can be readily defined and explored. This
approach addresses long-standing criticisms of cognitive map theory by explicit mapping
of stimuli to action via specific, albeit behaviorally unobservable, computations. Flexible
behavior associated with cognitive maps implies the use of transition models in reinforcement
learning algorithms. In contrast to model-free algorithms that depend on current
experience only, model-based reinforcement algorithms represent sensory or state information
beyond the modeled animal’s current sensory experience. As a result, model-based
reinforcement learning provides a principled approach to analysis of neural representations
and the dynamic processes that support cognition.
Neurophysiological recordings in the hippocampus showed that apparent noise present in
spatially modulated place cell activity could be explained as coherent spatial representations
that deviated from the animal’s position on the maze. These non-local representations were
associated with fast spatial representation dynamics and were typically found when the
animal was at feeder locations or choice points. Non-local representations at choice points
shifted forward of the animal to potential future spatial positions and were associated with theta and gamma local field potential activity. Forward-shifted spatial representations were
associated with vicarious-trial-and-error behaviors and were task and experience dependent.
In sum, these results suggest how cognitive maps in the hippocampus can contribute to
selection of adaptive actions through the construction of past events and potential future experiences.