Model-Free and Model-Based Influence on Choice in Rodents and Interactions between Hippocampus and Dorsomedial Prefrontal Cortex during Deliberation

Abstract

Decision making is driven by multiple, somewhat independent systems within the brain. One of these systems makes slow, deliberative decisions, and is thought to be driven by a model-based neural algorithm, in that it learns an internal model of the world which it uses to make decisions. Another system makes fast, habitual choices, and is hypothesized to depend on a model-free neural algorithm, in that it does not learn a model of the world, but simply stores state-action-reward associations. While the habitual system is relatively well-studied, the neural underpinnings of the deliberative system are less clear. Specifically, it is not known how areas comprising the deliberative system, such as prefrontal cortex and the hippocampus, share information on fast timescales. Also, representations of contingency information in prefrontal areas have previously been impossible to disambiguate from the encoding of other time-varying information. In this thesis, we adapted for rats a task which enabled the dissociation of model-based from model-free influence on choice, and we found evidence for both model-based and model-free control. We also developed a simpler task which caused rats to repeatedly transition between deliberative and habitual modes. On this second task, we found that both dmPFC and CA1 encoded information about task contingencies, while simultaneously representing unrelated time-varying information. Lastly, we examined interactions between dmPFC and CA1 on fast timescales, and found that both areas represented prospective information simultaneously, but that the content of this prospective information was not always identical between the two areas. Activity in dmPFC predicted whether HPC would represent prospective information on broad timescales, and prospective representation in HPC changed reward encoding in dmPFC on faster, sub-second timescales. Our work begins to bridge the neural underpinnings of decision making in rodents and the algorithms by which they select actions, confirms that the deliberative system represents contingency information, and uncovers asymmetries in the transfer of information between dmPFC and HPC.