Behavioral Social Neuroscience Seminar

Broadly construed, learning is an agent's adaptive response to the environment that potentially enables it to improve its chances of gaining rewarding outcomes. I show that human behavior typically thought of as irrational can be reinterpreted as optimal exploratory behavior. The problem is that learning often requires foregoing rewarding choices to make choices that are directed towards dissipating uncertainty about the structure of the environment. This raises the question of how much to explore in an unknown environment. For a rational agent, how much to explore is controlled by the agent s beliefs about the value of novel states, which we refer to as the agent s aspiration. To examine the effect of aspiration on exploratory behavior in humans, we conducted a series of experiments that explore whether prior beliefs and inferences about the reward potential of the environment controls investment in exploratory behavior. We show that prior beliefs about the generative model for the environment can account for individual differences in exploration, that statistical information about reward magnitudes modifies exploration, and that knowledge of other's reward experience exceeds their creates a complete reinitialization of exploration. Theoretically, we have developed a hierarchical model-based Bayesian reinforcement-learning framework that can mimic all of these behaviors by manipulating the agent s initial prior belief about the size of the state space and/or the magnitude of potential rewards.