Computational psychiatry

Making decisions in environments with few choice options is easy. We select the action that results in the most valued outcome. Making decisions in more complex environments, where the same action can produce different outcomes in different conditions, is much harder. In such circumstances, we propose that accurate action selection relies on top-down control from the prelimbic and orbitofrontal cortices over striatal activity through distinct thalamostriatal circuits. We suggest that the prelimbic cortex exerts direct influence over medium spiny neurons in the dorsomedial striatum to represent the state space relevant to the current environment...

Reduced social motivation is a hallmark of individuals with autism spectrum disorders (ASDs). Although the exact neural mechanisms are unclear, oxytocin has been shown to enhance motivation and attention to social stimuli, suggesting a potential to augment social reinforcement learning as the central mechanism of behavioral interventions in ASD. We tested how reinforcement learning in social contexts and associated reward prediction error (RPE) signals in the nucleus accumbens (NAcc) were modulated by intranasal oxytocin...

In many studies of attention-deficit hyperactivity disorder (ADHD), stimulus encoding and processing (perceptual function) and response selection (executive function) have been intertwined. To dissociate deficits in these functions, we introduced a task that parametrically varied low-level stimulus features (orientation and color) for fine-grained analysis of perceptual function. It also required participants to switch their attention between feature dimensions on a trial-by-trial basis, thus taxing executive processes...

BACKGROUND: In substance-dependent individuals, drug deprivation and drug use trigger divergent behavioral responses to environmental cues. These divergent responses are consonant with data showing that short- and long-term adaptations in dopamine signaling are similarly sensitive to state of drug use. The literature suggests a drug state-dependent role of learning in maintaining substance use; evidence linking dopamine to both reinforcement learning and addiction provides a framework to test this possibility...

Learning-based therapies, such as cognitive-behavioral therapy, are used worldwide, and their efficacy is endorsed by health and research funding agencies. However, the mechanisms behind both their strengths and their weaknesses are inadequately understood. Here we describe how advances in computational modeling may help formalize and test hypotheses regarding how patients make inferences, which are core postulates of these therapies. Specifically, we highlight the relevance of computations with regard to the development, maintenance, and therapeutic change in psychiatric disorders...

Most psychiatric disorders emerge during childhood and adolescence. This is also a period that coincides with the brain undergoing substantial growth and reorganisation. However, it remains unclear how a heightened vulnerability to psychiatric disorder relates to this brain maturation. Here, we propose 'developmental computational psychiatry' as a framework for linking brain maturation to cognitive development. We argue that through modelling some of the brain's fundamental cognitive computations, and relating them to brain development, we can bridge the gap between brain and cognitive development...

Depression is a debilitating condition with a high prevalence. Depressed patients have been shown to be diminished in their ability to integrate their reinforcement history to adjust future behaviour during instrumental reward learning tasks. Here, we tested whether such impairments could also be observed in a Pavlovian conditioning task. We recruited and analysed 32 subjects, 15 with depression and 17 healthy controls, to study behavioural group differences in learning and decision-making. Participants had to estimate the probability of some fractal stimuli to be associated with a binary reward, based on a few passive observations...

Cognitive models aim to explain complex human behavior in terms of hypothesized mechanisms of the mind. These mechanisms can be formalized in terms of mathematical structures containing parameters that are theoretically meaningful. For example, in the case of perceptual decision making, model parameters might correspond to theoretical constructs like response bias, evidence quality, response caution, and the like. Formal cognitive models go beyond verbal models in that cognitive mechanisms are instantiated in terms of mathematics and they go beyond statistical models in that cognitive model parameters are psychologically interpretable...

Important real-world decisions are often arduous as they frequently involve sequences of choices, with initial selections affecting future options. Evaluating every possible combination of choices is computationally intractable, particularly for longer multistep decisions. Therefore, humans frequently use heuristics to reduce the complexity of decisions. We recently used a goal-directed planning task to demonstrate the profound behavioral influence and ubiquity of one such shortcut, namely aversive pruning, a reflexive Pavlovian process that involves neglecting parts of the decision space residing beyond salient negative outcomes...

A cognitive map has long been the dominant metaphor for hippocampal function, embracing the idea that place cells encode a geometric representation of space. However, evidence for predictive coding, reward sensitivity and policy dependence in place cells suggests that the representation is not purely spatial. We approach this puzzle from a reinforcement learning perspective: what kind of spatial representation is most useful for maximizing future reward? We show that the answer takes the form of a predictive representation...

Motivation deficits, such as apathy, are pervasive in both neurological and psychiatric diseases. Even when they are not the core symptom, they reduce quality of life, compromise functional outcome and increase the burden for caregivers. They are currently assessed with clinical scales that do not give any mechanistic insight susceptible to guide therapeutic intervention. Here, we present another approach that consists of phenotyping the behaviour of patients in motivation tests, using computational models...

Humans and animals are capable of evaluating actions by considering their long-run future rewards through a process described using model-based reinforcement learning (RL) algorithms. The mechanisms by which neural circuits perform the computations prescribed by model-based RL remain largely unknown; however, multiple lines of evidence suggest that neural circuits supporting model-based behavior are structurally homologous to and overlapping with those thought to carry out model-free temporal difference (TD) learning...

Decision-making in the real world presents the challenge of requiring flexible yet prompt behavior, a balance that has been characterized in terms of a trade-off between a slower, prospective goal-directed model-based (MB) strategy and a fast, retrospective habitual model-free (MF) strategy. Theory predicts that flexibility to changes in both reward values and transition contingencies can determine the relative influence of the two systems in reinforcement learning, but few studies have manipulated the latter...

Prediction-error signals consistent with formal models of "reinforcement learning" (RL) have repeatedly been found within dopaminergic nuclei of the midbrain and dopaminoceptive areas of the striatum. However, the precise form of the RL algorithms implemented in the human brain is not yet well determined. Here, we created a novel paradigm optimized to dissociate the subtypes of reward-prediction errors that function as the key computational signatures of two distinct classes of RL models-namely, "actor/critic" models and action-value-learning models (e...

BACKGROUND: Addiction is supposedly characterized by a shift from goal-directed to habitual decision making, thus facilitating automatic drug intake. The two-step task allows distinguishing between these mechanisms by computationally modeling goal-directed and habitual behavior as model-based and model-free control. In addicted patients, decision making may also strongly depend upon drug-associated expectations. Therefore, we investigated model-based versus model-free decision making and its neural correlates as well as alcohol expectancies in alcohol-dependent patients and healthy controls and assessed treatment outcome in patients...

Aims: To advance translational studies of the role of reward prediction error (PE) in alcohol use disorder, the present study sought to develop and conduct an initial test of an alcohol-specific PE task paradigm using functional magnetic resonance imaging in humans. Methods: Alcohol dependent or social drinkers received small tastes of their preferred alcohol beverage or control beverage, with preceding visual cues indicating whether alcohol (or water) would be delivered...

The problem of whether and how information is integrated across hierarchical brain networks embodies a fundamental tension in contemporary cognitive neuroscience, and by extension, cognitive neuropsychiatry. Indeed, the penetrability of perceptual processes in a 'top-down' manner by higher-level cognition-a natural extension of hierarchical models of perception-may contradict a strictly modular view of mental organization. Furthermore, some in the cognitive science community have challenged cognitive penetration as an unlikely, if not impossible, process...

BACKGROUND: Stress is widely known to alter behavioral responses to rewards and punishments. It is believed that stress may precipitate these changes through modulation of corticostriatal circuitry involved in reinforcement learning and motivation, although the intervening mechanisms remain unclear. One candidate is inflammation, which can rapidly increase following stress and can disrupt dopamine-dependent reward pathways. METHODS: Here, in a sample of 88 healthy female participants, we first assessed the effect of an acute laboratory stress paradigm on levels of plasma interleukin-6 (IL-6), a cytokine known to be both responsive to stress and elevated in depression...

Influential models of schizophrenia suggest that patients experience incoming stimuli as excessively novel and motivating, with important consequences for hallucinatory experience and delusional belief. However, whether schizophrenia patients exhibit excessive novelty value and whether this interferes with adaptive behaviour has not yet been formally tested. Here, we employed a three-armed bandit task to investigate this hypothesis. Schizophrenia patients and healthy controls were first familiarised with a group of images and then asked to repeatedly choose between familiar and unfamiliar images associated with different monetary reward probabilities...

According to current concepts, major depressive disorder is strongly related to dysfunctional neural processing of motivational information, entailing impairments in reinforcement learning. While computational modelling can reveal the precise nature of neural learning signals, it has not been used to study learning-related neural dysfunctions in unmedicated patients with major depressive disorder so far. We thus aimed at comparing the neural coding of reward and punishment prediction errors, representing indicators of neural learning-related processes, between unmedicated patients with major depressive disorder and healthy participants...