T&C Chen Center for Social and Decision Neuroscience Seminar

Abstract: A standard assumption in decision science is that low-effort, model-free learning is reflexive and continuously employed, while more complex model-based strategies are only used when the rewards they generate are worth the additional effort (i.e., opportunistically). We present evidence from behavioural, pupil, and neuroimaging data refuting this assumption in several ways. First, we show that the class of multi-stage decision tasks commonly used to disentangle model-free and model-based influences on behaviour are only able to make this distinction if participants fully understand the task and limit their strategies to the specific algorithms being compared. These conditions do not appear to have been met in previously published studies. Moreover, improving participants' understanding of the task removes any evidence of model-free behaviour. Second, we find that task instructions that lead to more correct model-based behaviour reduce rather than increase mental effort. This is inconsistent with cost-benefit arbitration about whether to add opportunistic model-based strategies to the fixed cost of reflexive model-free learning. Third, we demonstrate that previous reports of combined model-free and model-based reward prediction errors in the ventral striatum are based on a flawed statistical analysis and are most likely spurious. Analyses with an appropriate model yield no evidence of a model-free prediction error signal in the ventral striatum. Together, our physiological and behavioural data suggest that model-free learning may not be automatic after all, contrary to long-standing assumptions. Thus, our results call for re-evaluation of this core assumption in influential theories of learning and decision-making.