Reaching to multiple potential targets: An optimal control perspective

Abstract

Living in a dynamic environment, we must be able to make flexible plans that can handle ambiguity and changes in goals while acting. Recent studies suggest that brain builds multiple competing plans related to potential goals and use perceptual information to drive this competition, until a single policy is selected. We propose an extended optimal control framework to model human behavior in tasks with multiple goals and show that goal competition is a natural by-product of handling goal uncertainty. We show how an agent's optimal policy in the presence of goal ambiguity, can be expressed as a weighted mixture of multiple control policies, each of which produces a sequence of actions associated with a specific target. At any instant, weighting factor is an inference of which goal's policy is best to follow, starting from the current state. Simulations of our multiple-goal optimal control model replicated reaching strategies observed in several human studies. Finally, we made novel predictions about the effects of the spatial probability distributions of the candidate targets and their expected pay-off values on the optimal policy.