Motivation, learning, planning and strategy change in brain and behaviour

Deciding between apples and oranges has been an age-old question not just for hungry shoppers but within the field of decision-making research. However, very rarely have researchers considered the possibility to reject either and move on to the next shelf or even leave the shop altogether. I have previously argued that such a sequential decision making framework is not just essential for understanding foraging (1) in the wild, but also ecological, real life, behaviour (2)(3). In general, while some simplifications of decision making in real-life environments are necessary for laboratory settings, past studies have often removed essential complexity. A common reductionist approach making focuses exclusively on a framing in neuro-economic terms of one-shot decisions. However, this approach often neglects fundamental temporal aspects of decision-making in the real world: behaviour is often a sequence of choices, requiring planning and complex learning. Only by examining cognition, emotions and behaviour embedded in these extended sequences can we gain a full understanding of brain function. For example, we have recently proposed a cognitive model for sequential search decisions and its underlying neural dynamics (4). We then linked changes in decision inertia during sequential decision making to Apathy. Compulsivity was primarily linked to metacognitive changes in peoples believes about chasing and pre-emptive avoidance (in press, PLOS Biology). Beyond the labs interest in sequential decision and prospection, there are several other areas of research listed below: 1) Inspired by the conception of risk taking in ecology and biology, we have developed a cognitive model of dynamic risk cognition. Importantly, while economists and psychologist see risk taking primarily as a trait (e.g. one can be a generally risk averse or seeking person), biologist think of it more as a result intrinsic and extrinsic risk pressure. For example, if an animal is hungry it will engage in risk it would otherwise avoid. We have previously build a simple cognitive model of value modulation by risk pressure and linked it to brain processes using fMRI (5). In the future we want to extend this approach to include dynamic pressures to play it safe as well as looking at the effects of varying assets and wealth on risk taking. In addition, we want to understand how people differ in dynamic risk taking by taking a large online sample and understand the evolving neural dynamics of modulation using EEG/MEG. 2) My lab investigates decision strategy flexibility more broadly. In other words, rather than just looking at how different decision features such as reward probabilities and magnitudes are flexibly weighted based on changing environmental pressure, we want to understand how people flexibly organize their overall behaviour in cognitive modes i.e. decision strategies (inferred through Hidden Markov models) and link those to their underlying brain states using fMRI and EEG/MEG. 3) We have also recently developed a new model predicting intrinsic motivation and motivational collapse across tasks in macaque monkeys and linked it to differences in neural activity using fMRI and changes with neurostimulation. We now want to use the same model to look at the effects on neural populations as measured using electrode recordings. We also want to develop a human version measuring intrinsic motivation collapse. 4) Complex sequential behaviours have to be supported by learning. We have worked on multiple representations of changing reward environments in the anterior cingulate cortex (6, 7) and how the changeability of the reward environment can affect how rare reward experiences are processed in orbitofrontal cortex (8). In the future we want to understand the brain mechanisms of complex attribution, credit assignment and reward learning. 5) Bringing our work in planning and sequential pursuit together with the reward learning, we have been working on models of reward trend estimation and prediction. We have already built computational and cognitive models capable of prediction in a neurophysiologically plausible way and collected MEG data. In the future we want to understand when reward trend estimation is triggered and traded-off with non-directional learning mechanisms, followed by finding the neural mechanism using fMRI. 6) Lastly, we also work on social computational cognitive neuroscience (9). In particular, we want to understand what computations can support complex social cognitions in the prefrontal cortex. To achieve this, we want to combine complex tasks that allow participants to pursue different kinds of social and non-social behaviours with hidden markov models to extract distinct behavioural modes and corresponding brain states using fMRI and EEG/MEG. If you are interesting in working on or develop and lead a project in any of those subject areas, but would like to hear more details on the potential projects outlined or want to conceive a related project together based on your own ideas, please do not hesitate to get in touch with me nils.kolling@inserm.fr . Regarding skill learning, my work within cognitive computational neuroscience includes the following approaches: -Computational cognitive modelling of complex task behaviour (including hidden markov modes) and task development. - Electrophysiological recordings using MEG and EEG in humans and electrode recordings in macaques -Localization and description of neural mechanisms using fMRI in humans and macaques -Pharmacological manipulations and magnetic resonance spectroscopy in humans -Effects of lesions and neurostimulation -Large scale online experiments using Java-script, hierarchical models and discovery-confirmation sample approaches Students should have some knowledge of programming in R or one of Matlab or Python before the internship starts. I can help students with useful resources, tutorials and support between now and project start. Close daily supervision will be provided for conception, task design, analysis and interpretation, depending on the project. In addition, the student will be integrated in the lab and learn what the larger teams is up to. There is certainly the potential for a motivated student finish a project from the work in the internship. Furthermore, it is possible to build on the project for future Phd applications. All supervision and writing will be in English. References/ recent publications 1. N. Kolling, T. E. J. Behrens, R. B. Mars, M. F. S. Rushworth, Neural mechanisms of foraging. Science. 336, 95–98 (2012). 2. N. Kolling, T. Akam, (Reinforcement?) Learning to forage optimally. Curr. Opin. Neurobiol. 46, 162–169 (2017). 3. N. Kolling, J. X. O’Reilly, State-change decisions and dorsomedial prefrontal cortex: the importance of time. Curr Opin Behav Sci. 22, 152–160 (2018). 4. N. Kolling, J. Scholl, A. Chekroud, H. A. Trier, M. F. S. Rushworth, Prospection, Perseverance, and Insight in Sequential Behavior. Neuron. 99, 1069-1082.e7 (2018). 5. N. Kolling, M. Wittmann, M. F. S. Rushworth, Multiple Neural Mechanisms of Decision Making and Their Competition under Changing Risk Pressure. Neuron. 81, 1190–1202 (2014). 6. D. Meder, N. Kolling, L. Verhagen, M. K. Wittmann, J. Scholl, K. H. Madsen, O. J. Hulme, T. E. J. Behrens, M. F. S. Rushworth, Simultaneous representation of a spectrum of dynamically changing value estimates during decision making. Nat Commun. 8, 1–11 (2017). 7. M. K. Wittmann, N. Kolling, R. Akaishi, B. K. H. Chau, J. W. Brown, N. Nelissen, M. F. S. Rushworth, Predictive decision making driven by multiple time-linked reward representations in the anterior cingulate cortex. Nature Communications. 7, 12327 (2016). 8. J. Grohn, U. Schüffelgen, F.-X. Neubert, A. Bongioanni, L. Verhagen, J. Sallet, N. Kolling, M. F. S. Rushworth, Multiple systems in macaques for tracking prediction errors and other types of surprise. PLoS Biol. 18, e3000899 (2020). 9. N. Kolling, M. Braunsdorf, S. Vijayakumar, H. Bekkering, I. Toni, R. B. Mars, Constructing Others’ Beliefs from One’s Own Using Medial Frontal Cortex. J. Neurosci. 41, 9571–9580 (2021).