Bolles, R.C., Holtz, R., Dunn, T., & Hill, W. (1980). Comparisons of Stimulus Learning and Response Learning in a Punishment Situation. Learning and Motivation, 11, 78-96.
Early on in the study of learning, learning was believed to consist of the attachment of a response to a stimuli (an S-R association). Later, in contrast with the long-held conventional view that all learning was of the S-R form, alternative forms were proposed by Pavlov and others. Examples include stimulus learning (S-S*) and response learning (R-S*). What type of learning underlied punishment, for example? This stimulated debate. In common punishment paradigms, it became obvious that is was unclear whether the animal was learning that shock is correlated there contextually with the bar (stimulus) or whether shock is correlated with its behavior of pressing the bar (response). The purpose of this paper was to attempt to disentangle these different forms of learning experimentally. Four novel experimental paradigms were explored.
Experiment 1 contained one single bar in the chamber which could either be pressed or pulled. Animals had to alternate their behavioral response (press to pull, and back again) in order to be rewarded. Punishment was delivered on every tenth press for half the animals or every tenth pull for the other half. Results showed a rapid initial suppression (fear to environmental stimuli), but later a return to baseline for the unpunishment response and continued suppression for the punished response. This seems to be evidence for both types of learning taking place within one paradigm. Experiment 2 simply adjusted the response contingencies (up and down) to see if results would be sensitive to this type of experimental manipulation. Instead of an FR-10 punishment schedule, rats were shocked on FR-4 or FR-25. FR-4 showed dramatic differences in responding from the outset. FR-25 differences only emerged in the second day of punishment.
Experiment 3 used two bars, each of which could be either pressed or pulled. Thus, four punishment conditions were possible, punishing a left-press, a left-lift, a right-press, or a right-lift. Results showed that when a rat was punished for a left-lift, for example, it quickly stopped lifting AND pressing the left bar. However, it continued to lift AND press the right bar. Thus, learning, in this case, seems to be mostly about stimuli. Experiment 4, like Experiment 2, changed the contingencies. Punishment was shifted from an FR-1 schedule to an FR-10 schedule. Punishment conditions were: press or lift left bar, press or lift right bar, lifting left or right, and pressing left or right. Across the conditions, the general trend that emerged was a rapid emergence of stimulus learning and then a slower but undeniable development of response learning.
Showing posts with label Learning. Show all posts
Showing posts with label Learning. Show all posts
Wednesday, April 29, 2009
Thursday, October 9, 2008
Ethanol-conditioned flavor preferences compared with sugar- and fat-conditioned preferences in rats
Ackroff, K., Rozental, D., Scalfani, A. (2004). Ethanol-conditioned flavor preferences compared with sugar- and fat-conditioned preferences in rats. Physiology & Behavior, 81, 699-713.
Past studies in rats have suggested that the postingestive effects of various nutrients can condition strong flavor preferences. Research has been conducted on ethanol using oral conditioning (where the ethanol is mixed in with the flavoring, thus adding its own flavor to the mixture) and intragastic conditioning (where the ethanol is injected into the rat’s stomach after it ingests the flavoring, thus eliminating the ethanol’s flavor as a conditioning factor). However, the findings of these studies have been somewhat contradictory. Mehiel and Bolles (1988) found that rats equally preferred a flavor paired with ethanol and a flavor paired with sucrose, while Sherman et al. (1983) found that rats preferred a flavor paired with glucose to a flavor paired with ethanol. These studies different in terms of design, route of administration, and sugar used, but it is unclear which of these factors are responsible for the confliction. The present study examined this question in a series of four related experiments.
In the first experiment, sucrose and ethanol were compared to each other as well as to a control of water. These nutrients were each paired with a different flavor and administered through intragastric infusion. Unlimited access to alternating flavor/nutrient combinations was provided during a training period and then the amount of each combination consumed was measuring during a test in which the rats could choose between different combinations. The results were that (1) the rats preferred both sucrose- and ethanol-paired flavors to the water-paired flavor, (2) the rats strongly preferred the sucrose-paired flavor to the ethanol-paired flavor, and (3) the rats ingested more sucrose mixture than ethanol mixture during the training period.
The second experiment attempts to control for finding (3) in experiment one by adding the additional constraint that sucrose mixture consumption was limited to the previous day’s ethanol mixture consumption. Findings (1) and (2) did not change as a result.
In the third experiment, the oral conditioning method was used in place of intragastric infusion for nutrient administration. Once again, the findings did not change.
Finally, in the fourth experiment, ethanol was compared to fructose and corn oil using the intragastic infusions. Similar to the sucrose findings, the rats preferred these new sugars to ethanol, while still preferring ethanol to water.
This study supports the findings of Sherman et al. (1983) and eliminates training intakes, route of administration, and specific sugar/fat as explanations of Mehiel and Bolles’ (1988) contradictory findings. Something other than energy concentration affects the efficacy of ethanol as a reinforcer, making it less powerful than other sugars/fats.
Past studies in rats have suggested that the postingestive effects of various nutrients can condition strong flavor preferences. Research has been conducted on ethanol using oral conditioning (where the ethanol is mixed in with the flavoring, thus adding its own flavor to the mixture) and intragastic conditioning (where the ethanol is injected into the rat’s stomach after it ingests the flavoring, thus eliminating the ethanol’s flavor as a conditioning factor). However, the findings of these studies have been somewhat contradictory. Mehiel and Bolles (1988) found that rats equally preferred a flavor paired with ethanol and a flavor paired with sucrose, while Sherman et al. (1983) found that rats preferred a flavor paired with glucose to a flavor paired with ethanol. These studies different in terms of design, route of administration, and sugar used, but it is unclear which of these factors are responsible for the confliction. The present study examined this question in a series of four related experiments.
In the first experiment, sucrose and ethanol were compared to each other as well as to a control of water. These nutrients were each paired with a different flavor and administered through intragastric infusion. Unlimited access to alternating flavor/nutrient combinations was provided during a training period and then the amount of each combination consumed was measuring during a test in which the rats could choose between different combinations. The results were that (1) the rats preferred both sucrose- and ethanol-paired flavors to the water-paired flavor, (2) the rats strongly preferred the sucrose-paired flavor to the ethanol-paired flavor, and (3) the rats ingested more sucrose mixture than ethanol mixture during the training period.
The second experiment attempts to control for finding (3) in experiment one by adding the additional constraint that sucrose mixture consumption was limited to the previous day’s ethanol mixture consumption. Findings (1) and (2) did not change as a result.
In the third experiment, the oral conditioning method was used in place of intragastric infusion for nutrient administration. Once again, the findings did not change.
Finally, in the fourth experiment, ethanol was compared to fructose and corn oil using the intragastic infusions. Similar to the sucrose findings, the rats preferred these new sugars to ethanol, while still preferring ethanol to water.
This study supports the findings of Sherman et al. (1983) and eliminates training intakes, route of administration, and specific sugar/fat as explanations of Mehiel and Bolles’ (1988) contradictory findings. Something other than energy concentration affects the efficacy of ethanol as a reinforcer, making it less powerful than other sugars/fats.
Monday, October 6, 2008
Evidence for episodic memory in a Pavlovian conditioning procedure in rats
O'Brien, Jamus; Sutherland, Robert J. Evidence for episodic memory in a Pavlovian conditioning procedure in rats. Hippocampus. Vol 17(12) 2007, 1149-1152.
Several requirements have been proposed for establishing episodic memory in nonlinguistic species. Clayton et al. (2003) suggest that episodic memory competence requires integrated representations of “what", "where", and "when” content that can be flexibly updated as more information is gathered. Tests of episodic memory should be novel and unexpected, exceed the capacity of short-term memory (Dere et al., 2006), be unsolvable by familiarity judgments (Gallistel, 1990), and involve memories formed in unique one-trial learning episodes (Morris, 2001). In this article, O’Brien and Sutherland design and run an experiment that attempts to meet these requirements while testing for the flexible and integrated representations required for episodic memory.
In this experiment, twenty-two male Long-Evans rats were run through three experimental phases. In Phase 1, rats were given time to explore two different “contexts” (boxes A and B). These boxes were Plexiglas modular test chambers with steel grid floors. Box A had black colored walls and was scented with Quatzyl-D-Plus, while box B was white and scented with Clinicide. Each rat visited Box A on three successive mornings and Box B on three successive evenings. In Phase 2, rats were exposed to a single immediate shock in a “chimerical” box that was half black, half white, and unscented. For half the rats, this occurred in the morning and for the other half, it occurred in the evening; this is the study’s independent variable. In Phase 3, during mid-day, rats were placed in one of the contexts from Phase 1 (either Box A or Box B). Fear responses (this study’s dependent variable) were then measured by timing conditioned “freezing” behavior.
A repeated measure ANOVA was conducted on the percent of time spent freezing during Phase 3, and it was predicted that rats placed in the context congruent to the time of day of their shock would exhibit more freezing than rats placed in the incongruent context. (For example, if a rat was in Box A in the morning and Box B in the evening during Phase 1 and received its shock in the morning during Phase 2, then the congruent context would be Box A and the incongruent context would be Box B.) The results were a significant relationship in the direction predicted (F(1,20) = 45.0, P < 0.001).
These findings support the idea that rats acquire memories laden with temporal context (or “when” content), such as the time of day, which is an important requirement of establishing episodic memory competence. Further studies could continue this line of research by establishing evidence for rats generating “what” and “where” content, and by exploring the nature of rats’ temporal cues – possibly endogenous circadian oscillators (Gallistel, 1990) or the age of granule cells in the dentate gyrus of the hippocampus (Aimone et al., 2006).
Several requirements have been proposed for establishing episodic memory in nonlinguistic species. Clayton et al. (2003) suggest that episodic memory competence requires integrated representations of “what", "where", and "when” content that can be flexibly updated as more information is gathered. Tests of episodic memory should be novel and unexpected, exceed the capacity of short-term memory (Dere et al., 2006), be unsolvable by familiarity judgments (Gallistel, 1990), and involve memories formed in unique one-trial learning episodes (Morris, 2001). In this article, O’Brien and Sutherland design and run an experiment that attempts to meet these requirements while testing for the flexible and integrated representations required for episodic memory.
In this experiment, twenty-two male Long-Evans rats were run through three experimental phases. In Phase 1, rats were given time to explore two different “contexts” (boxes A and B). These boxes were Plexiglas modular test chambers with steel grid floors. Box A had black colored walls and was scented with Quatzyl-D-Plus, while box B was white and scented with Clinicide. Each rat visited Box A on three successive mornings and Box B on three successive evenings. In Phase 2, rats were exposed to a single immediate shock in a “chimerical” box that was half black, half white, and unscented. For half the rats, this occurred in the morning and for the other half, it occurred in the evening; this is the study’s independent variable. In Phase 3, during mid-day, rats were placed in one of the contexts from Phase 1 (either Box A or Box B). Fear responses (this study’s dependent variable) were then measured by timing conditioned “freezing” behavior.
A repeated measure ANOVA was conducted on the percent of time spent freezing during Phase 3, and it was predicted that rats placed in the context congruent to the time of day of their shock would exhibit more freezing than rats placed in the incongruent context. (For example, if a rat was in Box A in the morning and Box B in the evening during Phase 1 and received its shock in the morning during Phase 2, then the congruent context would be Box A and the incongruent context would be Box B.) The results were a significant relationship in the direction predicted (F(1,20) = 45.0, P < 0.001).
These findings support the idea that rats acquire memories laden with temporal context (or “when” content), such as the time of day, which is an important requirement of establishing episodic memory competence. Further studies could continue this line of research by establishing evidence for rats generating “what” and “where” content, and by exploring the nature of rats’ temporal cues – possibly endogenous circadian oscillators (Gallistel, 1990) or the age of granule cells in the dentate gyrus of the hippocampus (Aimone et al., 2006).
Tuesday, September 9, 2008
Switching on and off fear by distinct neuronal circuits
Herry, C. et al. (2008). Switching on and off fear by distinct neuronal circuits. Nature, 454, 600-605.
Whereas firing of amygdala neurons is necessary for retrieval of conditioned fear memories, extinction of these fear memories is thought to be controlled by constraining this neural activity by local inhibitory circuitry (under the influence of mPFC). However, fear extinction is known to be a fragile behavioral state, readily influenced by context, i.e. changing context can result in spontaneous recovery. This raises the question of whether there are specialized circuits driving behavioral transitions in opposite directions, namely fear-on and fear-off. This paper showed that neurons in the BA could be divided into distinct functional classes: those exhibiting selective increases in CS+ evoked spike firing during and after fear conditioning (fear neurons) and those exhibiting selective increases in CS+ evoked spike firing during extinction (extinction neurons). Further, close analysis revealed that these two groups were not only functionally different but also differentially connected, with (1) fear neurons selectively receiving input from the hippocampus, and (2) extinction neurons being reciprocally connected to the mPFC while fear neurons only projected unidirectionally to the mPFC. This would indicate that co-localized within the same nucleus, two discrete neuronal circuits exist, intermingled in a salt-and-pepper-like manner. Their close anatomical proximity may serve to facilitate local interactions, although these mechanisms remain unexplored. Taken together with evidence showing emotional perseveration (persistent lack of state change) concomitant with inactivation of the BA, results suggest that the BA is unlikely to be associated with the storage, retrieval, or expression of conditioned fear and extinction memories, but is more likely to mediate context-dependent behavioral transitions between low and high fear states.
Whereas firing of amygdala neurons is necessary for retrieval of conditioned fear memories, extinction of these fear memories is thought to be controlled by constraining this neural activity by local inhibitory circuitry (under the influence of mPFC). However, fear extinction is known to be a fragile behavioral state, readily influenced by context, i.e. changing context can result in spontaneous recovery. This raises the question of whether there are specialized circuits driving behavioral transitions in opposite directions, namely fear-on and fear-off. This paper showed that neurons in the BA could be divided into distinct functional classes: those exhibiting selective increases in CS+ evoked spike firing during and after fear conditioning (fear neurons) and those exhibiting selective increases in CS+ evoked spike firing during extinction (extinction neurons). Further, close analysis revealed that these two groups were not only functionally different but also differentially connected, with (1) fear neurons selectively receiving input from the hippocampus, and (2) extinction neurons being reciprocally connected to the mPFC while fear neurons only projected unidirectionally to the mPFC. This would indicate that co-localized within the same nucleus, two discrete neuronal circuits exist, intermingled in a salt-and-pepper-like manner. Their close anatomical proximity may serve to facilitate local interactions, although these mechanisms remain unexplored. Taken together with evidence showing emotional perseveration (persistent lack of state change) concomitant with inactivation of the BA, results suggest that the BA is unlikely to be associated with the storage, retrieval, or expression of conditioned fear and extinction memories, but is more likely to mediate context-dependent behavioral transitions between low and high fear states.
Monday, September 8, 2008
mPFC neurons signal memory for fear extinction
Milad, M.R. & Quirk, G.J. (7 November 2002). Neurons in the medial prefrontal cortex signal memory for fear extinction. Nature, 420, 70-74.
Extinction is a process thought to form a new memory that inhibits the once-learned conditioned response. This paper suggests that consolidation of extinction learning potentiates activity in the infralimbic cortex (IL) of the mPFC which inhibits fear during subsequent encounters with fear stimuli. Electrophysiological recording showed that IL activity remained unresponsive during the conditioning phase and also during extinction training on Day 1. However, by Day 2, activity in the IL in response to tone was present from the start of the extinction phase. Further, stimulation of the IL paired with tone presentation resulted in less freezing behavior and also accelerated extinction learning. Therefore, enhanced extinction learning could be mediated directly by the stimulation or indirectly by the behavioral feedback of decrease freezing. Since the BLA sends excitatory projections to IL, it is possible that these inputs serve to potentiate IL neurons during the consolidation of extinction. The IL is then likely to inhibit expression of fear behavior via its projections to intercalated (ITC) cells in the CE, dampening the output of the amygdala. Pairing reminder stimuli with activation of the ventral mPFC through transcranial magnetic stimulation (TMS) might help strengthen extinction of fear in clinical settings.
Extinction is a process thought to form a new memory that inhibits the once-learned conditioned response. This paper suggests that consolidation of extinction learning potentiates activity in the infralimbic cortex (IL) of the mPFC which inhibits fear during subsequent encounters with fear stimuli. Electrophysiological recording showed that IL activity remained unresponsive during the conditioning phase and also during extinction training on Day 1. However, by Day 2, activity in the IL in response to tone was present from the start of the extinction phase. Further, stimulation of the IL paired with tone presentation resulted in less freezing behavior and also accelerated extinction learning. Therefore, enhanced extinction learning could be mediated directly by the stimulation or indirectly by the behavioral feedback of decrease freezing. Since the BLA sends excitatory projections to IL, it is possible that these inputs serve to potentiate IL neurons during the consolidation of extinction. The IL is then likely to inhibit expression of fear behavior via its projections to intercalated (ITC) cells in the CE, dampening the output of the amygdala. Pairing reminder stimuli with activation of the ventral mPFC through transcranial magnetic stimulation (TMS) might help strengthen extinction of fear in clinical settings.
Neural mechanisms of extinction
Quirk, G.J. & Mueller, D. (2007). Neural mechanisms of extinction learning and retrieval. Neuropsychopharmacology Reviews, 1-17.
The simplest form of emotional regulation is extinction, is which conditioned responding to a stimulus decreases when the reinforcer is omitted. Exinction, like any learning process, occurs in 3 phases: acquisition, consolidation, and retrieval. Cannabinoid and opioid receptors appear to be implicated in the acquisition of extinction since anandamide and opioid antagonists impair within-session extinction of fear. Consolidation appears to depend on protein synthesis within the BLA, frequency bursting of the infralimbic region (IL) of the vmPFC shortly after extinction, and general involvement of the hippocampus, especially in tasks such as inhibitory avoidance and contextual fear. Retrieval of extinction memories involves the expression of inhibitory circuitry and is highly context-specific. Inhibition circuitry within the amygdala includes local inhibitory neurons within the BLA and CE, as well as islands of GABAergic neurons between these two sites known as the intercalated (ITC) cells. ITC cells could serve as a site of extinction memory since they inhibit CE output neurons and BLA neurons, acting as an off-switch for the amygdala. ITC cells receive strong projection from the IL mPFC, and IL activity is correlated with the extent of extinction retrieval. In fact, electrical stimulation of IL reduces conditioned fear and strengthens extinction memory. The prelimbic (PL) mPFC, on the other hand, excites fear expression and can augment fear expression via projections to the basal nucleus of the amygdala. Thus, the PFC can fully control overall fear expression. Individuals with PTSD show reduced vmPFC and hippocampal volume and activity, as well as increased amygdala activity. Stress may also impair extinction, since chronic stress is shown to decrease dendritic branching and spine count in hippocampus and mPFC, but increase it in BLA, which could be expected to increase conditioning and impair extinction. Pharmacological adjuncts to current extinction-based exposure therapies may accelerate and strengthen extinction. Among them D-cycloserine, yohimbine, sulpiride, and methylene blue show promise. Administration of glucocorticoids such as cortisol before exposure therapy may also help.
The simplest form of emotional regulation is extinction, is which conditioned responding to a stimulus decreases when the reinforcer is omitted. Exinction, like any learning process, occurs in 3 phases: acquisition, consolidation, and retrieval. Cannabinoid and opioid receptors appear to be implicated in the acquisition of extinction since anandamide and opioid antagonists impair within-session extinction of fear. Consolidation appears to depend on protein synthesis within the BLA, frequency bursting of the infralimbic region (IL) of the vmPFC shortly after extinction, and general involvement of the hippocampus, especially in tasks such as inhibitory avoidance and contextual fear. Retrieval of extinction memories involves the expression of inhibitory circuitry and is highly context-specific. Inhibition circuitry within the amygdala includes local inhibitory neurons within the BLA and CE, as well as islands of GABAergic neurons between these two sites known as the intercalated (ITC) cells. ITC cells could serve as a site of extinction memory since they inhibit CE output neurons and BLA neurons, acting as an off-switch for the amygdala. ITC cells receive strong projection from the IL mPFC, and IL activity is correlated with the extent of extinction retrieval. In fact, electrical stimulation of IL reduces conditioned fear and strengthens extinction memory. The prelimbic (PL) mPFC, on the other hand, excites fear expression and can augment fear expression via projections to the basal nucleus of the amygdala. Thus, the PFC can fully control overall fear expression. Individuals with PTSD show reduced vmPFC and hippocampal volume and activity, as well as increased amygdala activity. Stress may also impair extinction, since chronic stress is shown to decrease dendritic branching and spine count in hippocampus and mPFC, but increase it in BLA, which could be expected to increase conditioning and impair extinction. Pharmacological adjuncts to current extinction-based exposure therapies may accelerate and strengthen extinction. Among them D-cycloserine, yohimbine, sulpiride, and methylene blue show promise. Administration of glucocorticoids such as cortisol before exposure therapy may also help.
Sunday, September 7, 2008
Dopamine gates LTP in lateral amygdala
Bissiere, S., Humeau, Y., & Luthi, A. (June 2003). Dopamine gates LTP induction in lateral amygdala by suppressing feedforward inhibition. Nature Neuroscience, 6, 6, 587-591.
It has been known that both long-term potentiation (LTP) and concomitant activation of dopaminergic nerves to the amygdala underlie the acquisition of fear conditioning. In fact, dopamine is known to be released in the amygdala during stress and intra-amygdala injection of dopamine receptor antagonists prevents fear conditioning. This study investigated the mechanisms supporting this and showed how dopamine could modulate fear conditioning by modulating inhibitory synaptic transmission within the amygdala. Specifically, D2 dopamine receptors could enable the induction of LTP by suppressing feedforward inhibition from local inhibitory interneurons.
It has been known that both long-term potentiation (LTP) and concomitant activation of dopaminergic nerves to the amygdala underlie the acquisition of fear conditioning. In fact, dopamine is known to be released in the amygdala during stress and intra-amygdala injection of dopamine receptor antagonists prevents fear conditioning. This study investigated the mechanisms supporting this and showed how dopamine could modulate fear conditioning by modulating inhibitory synaptic transmission within the amygdala. Specifically, D2 dopamine receptors could enable the induction of LTP by suppressing feedforward inhibition from local inhibitory interneurons.
Friday, September 5, 2008
Neuronal Signalling of Fear Memory
Maren, S. & Quirk, G.J. (November 2004). Neuronal Signalling of Fear Memory. Nature Reviews Neuroscience, 5, 844-850.
Plasticity within the CNS is necessary for the representation of new information, and can range from synthesis and insertion of synaptic proteins to whole-brain synchronization of neuronal activity. Pavlovian fear conditioning is an especially interesting phenomenon since such fear memories are acquired rapidly and are long-lasting. Research first noticed conditioning-induced changes in the midbrain, thalamus, and cortex; however, it was unclear whether or not these were primary sites of plasticity or were simply downstream from other plastic sites. Eventually the lateral nucleus of the amygdala (LA), receiving direct projections from the auditory thalamus, was posited to be vital for auditory fear conditioning. The dorsal subdivision of the nucleus (LAd) seems to be the first site in the auditory pathway to show associative plasticity that is not fed forward passively from upstream sites, is not dependent on downstream sites, and is crucial for conditioned behavior. And LA neurons appear to drive plasticity at both thalamic and cortical levels.
Fear memories are useful to anticipate and respond to dangers within the environment. However, when signals for aversive events no longer predict those events, fear to those signals subsides. This is an inhibitory learning process known as extinction. It appears that although fear subsides after extinction, the fear memory is not erased. Extinction seems to be highly context dependent and sometimes short-lived. Fear responses can be spontaneously recovered over time. It seems biology has deemed it better to fear than not to fear. It is more likely that additional memories which interfere with pre-existing excitatory responses are learned in the extinction process. Again the amygdala seems to be essentially involved here. Further, the mPFC, which has an inhibitory influence on both the LA and the CE (the main output regions of the amygdala) through a rich network of inhibitory interneurons embedded in the amygdala, appears to be a major participant, and is perhaps modulated by context via hippocampus.
Plasticity within the CNS is necessary for the representation of new information, and can range from synthesis and insertion of synaptic proteins to whole-brain synchronization of neuronal activity. Pavlovian fear conditioning is an especially interesting phenomenon since such fear memories are acquired rapidly and are long-lasting. Research first noticed conditioning-induced changes in the midbrain, thalamus, and cortex; however, it was unclear whether or not these were primary sites of plasticity or were simply downstream from other plastic sites. Eventually the lateral nucleus of the amygdala (LA), receiving direct projections from the auditory thalamus, was posited to be vital for auditory fear conditioning. The dorsal subdivision of the nucleus (LAd) seems to be the first site in the auditory pathway to show associative plasticity that is not fed forward passively from upstream sites, is not dependent on downstream sites, and is crucial for conditioned behavior. And LA neurons appear to drive plasticity at both thalamic and cortical levels.
Fear memories are useful to anticipate and respond to dangers within the environment. However, when signals for aversive events no longer predict those events, fear to those signals subsides. This is an inhibitory learning process known as extinction. It appears that although fear subsides after extinction, the fear memory is not erased. Extinction seems to be highly context dependent and sometimes short-lived. Fear responses can be spontaneously recovered over time. It seems biology has deemed it better to fear than not to fear. It is more likely that additional memories which interfere with pre-existing excitatory responses are learned in the extinction process. Again the amygdala seems to be essentially involved here. Further, the mPFC, which has an inhibitory influence on both the LA and the CE (the main output regions of the amygdala) through a rich network of inhibitory interneurons embedded in the amygdala, appears to be a major participant, and is perhaps modulated by context via hippocampus.
Monday, March 31, 2008
Aversive learning enhances perceptual and cortical discrimination
Li, W, Howard, J.D., Parrish, T.B., & Gottfried, J.A. (March 28, 2008). Aversive learning enhances perceptual and cortical discrimination of indiscriminable odor cues. Science, Vol. 319, 1842-1844.
With this study, the authors explored the impact of aversive conditioning on olfactory discrimination. While most conditioning studies examine the acquisition of new behavioral responses (CR) to formerly benign stimuli presentations, this examined how associative learning can actually alter the perceptual processing of the conditioned stimulus (CS) itself. Following a conditioning regimen, behavioral accuracy for distinguishing by smell between a previously indistinguishable pair of molecules (CS+) rose by more than a factor of 2, exceeding both chance and preconditioning performance. Interestingly, following conditioning, no improvement in distinguishing between the unconditioned control pair (CS-) was witnessed, indicating that these effects are specific to the CS+. After conditioning, reorganization of neural coding was also observed in the posterior piriform cortex, where neural representations of odor identity are maintained. This may shed new light on anxiety disorders which are characterized by exaggerated sensory sensitivity and hypervigilance, potentially self-reinforcing patterns.
With this study, the authors explored the impact of aversive conditioning on olfactory discrimination. While most conditioning studies examine the acquisition of new behavioral responses (CR) to formerly benign stimuli presentations, this examined how associative learning can actually alter the perceptual processing of the conditioned stimulus (CS) itself. Following a conditioning regimen, behavioral accuracy for distinguishing by smell between a previously indistinguishable pair of molecules (CS+) rose by more than a factor of 2, exceeding both chance and preconditioning performance. Interestingly, following conditioning, no improvement in distinguishing between the unconditioned control pair (CS-) was witnessed, indicating that these effects are specific to the CS+. After conditioning, reorganization of neural coding was also observed in the posterior piriform cortex, where neural representations of odor identity are maintained. This may shed new light on anxiety disorders which are characterized by exaggerated sensory sensitivity and hypervigilance, potentially self-reinforcing patterns.
Thursday, March 6, 2008
Parallel incentive processing: an integrated view of amygdala function
Balleine, B.W. & Killcross, S. (May 2006). Parallel incentive processing: an integrated view of amygdala function. Trends in Neuroscience, Vol. 29, 272-279.
Although the amygdala has been long studied for its involvement in emotional learning and memory, the exact nature of its involvement is still disputed. Historically, a serial model has predominated, with the lateral nucleus detecting threatening stimuli and the central nucleus initiating expression of defensive behaviors and other bodily responses associated with fear reactivity. However, in this paper Balleine and Killcross opine perhaps it's time to consider alternative models, and propose a model which, based on appetitive conditioning studies, has the basolateral and central nuclei operating independently and in parallel to mediate incentive processes in both appetitive and aversive situations. They suggest the basolateral nucleus encodes emotional events with reference to their particular sensory features, while the central nucleus provides affective significance to processing, motivating or inciting responses and actions.
Although the amygdala has been long studied for its involvement in emotional learning and memory, the exact nature of its involvement is still disputed. Historically, a serial model has predominated, with the lateral nucleus detecting threatening stimuli and the central nucleus initiating expression of defensive behaviors and other bodily responses associated with fear reactivity. However, in this paper Balleine and Killcross opine perhaps it's time to consider alternative models, and propose a model which, based on appetitive conditioning studies, has the basolateral and central nuclei operating independently and in parallel to mediate incentive processes in both appetitive and aversive situations. They suggest the basolateral nucleus encodes emotional events with reference to their particular sensory features, while the central nucleus provides affective significance to processing, motivating or inciting responses and actions.
Tuesday, March 4, 2008
Classical fear conditioning in functional neuroimaging
Buchel, C. & Dolan, R.J. (2000). Classical fear conditioning in functional neuroimaging. Current Opinion in Neurobiology, 10, 219-223.
A brief overview of the brief history of the examination of classical conditioning with functional neuroimaging. Early PET studies showed a striking absence of expected amygdala activation, but later demonstrated the expected amygdala involvement. More recently, 'backward masking' designs indicate a hemispheric difference when the CS+ was presented out of awareness, with greater activation observed in the right amygdala. fMRI studies showed amygdala participation during initial acquisition and early phases of extinction, and also demonstrated the characteristic decreases in amygdala activation over time. Interestingly, blocked fMRI designs revealed that social phobics do not show the 'physiological' decrease of amygdala activation over time. Finally, the paper points out the controversy about the role of the amygdala. One camp regards the amygdala as a rapid subcortical information processing unit that is continuously involved in the processing of CSs in aversive classical conditioning, producing deliberately high "false alarm" rates and being kept under the supervision of cortical controllers. The other camp sees the amygdala as enabling or permitting associative plasticity that encodes acquired sensory contingencies which are later expressed at a cortical level; once the association has been learned, the systems mediating the modulation of plasticity disengage and hence we see the decline in amygdala activation. More on this debate here.
A brief overview of the brief history of the examination of classical conditioning with functional neuroimaging. Early PET studies showed a striking absence of expected amygdala activation, but later demonstrated the expected amygdala involvement. More recently, 'backward masking' designs indicate a hemispheric difference when the CS+ was presented out of awareness, with greater activation observed in the right amygdala. fMRI studies showed amygdala participation during initial acquisition and early phases of extinction, and also demonstrated the characteristic decreases in amygdala activation over time. Interestingly, blocked fMRI designs revealed that social phobics do not show the 'physiological' decrease of amygdala activation over time. Finally, the paper points out the controversy about the role of the amygdala. One camp regards the amygdala as a rapid subcortical information processing unit that is continuously involved in the processing of CSs in aversive classical conditioning, producing deliberately high "false alarm" rates and being kept under the supervision of cortical controllers. The other camp sees the amygdala as enabling or permitting associative plasticity that encodes acquired sensory contingencies which are later expressed at a cortical level; once the association has been learned, the systems mediating the modulation of plasticity disengage and hence we see the decline in amygdala activation. More on this debate here.
Neural mechanisms of extinction learning and retrieval
Quirk, G.J. & Mueller, D. (2008). Neural mechanisms of extinction learning and retrieval. Neuropsychopharmacology, 33, 56-72.
Early in the study of classical conditioning, Pavlov observed spontaneous recovery of responding to an extinguished conditioned stimulus. This witnessed 'uncovering phenomena', brought on by a change in context or stimulus presentation, led to the belief that extinction is not erasure of a previous fear memory but rather the learning of an additional inhibitory memory. This paper reviews what has been learned about extinction learning ever since. Like other types of learning, extinction occurs in three phases: acquisition, consolidation, and retrieval. Acquisition seems to depend on the basolateral amygdala (BLA) and the ventrolateral periaqueductal gray (vlPAG) structures. Consolidation seems to be most dependent on the BLA, where the learning of new memories (requiring protein synthesis) appears to take place. It also seems to rely on the involvement of the prefrontal cortex and the hippocampus. Retrieval requires expression of an inhibitory memory, and as such, during retrieval we see activation of inhibitory networks in the amygdala, cortical control of amygdala inhibition by the IL mPFC, and contextual regulation provided by the hippocampus and mPFC. Thus, like classical conditioning, extinction seems to be distributed across a network of structures, rather than centered in any one particular area.
The paper also raises many fascinating special issues. One challenges the Pavlovian idea that extinction is purely an additional inhibitory memory: recent evidence seems to indicate that extinction leads to reversal of conditioning-induced phosphorylation of CREB, indicating some erasure of the original BLA fear memory!
It also discusses anxiety disorders and PTSD which may be caused by a failure to retrieve an extinction memory generated in extinction-based treatment. Subjects with PTSD show reduced volume and activity in the vmPFC and hippocampus areas, along with increased activity in the amygdala, suggesting inhibitory control and contextual modulation of extinction may be compromised. It also mentions that chronic stress can impede extinction-based therapies, decreasing dendritic branching and spine count in the vmPFC and hippocampus, and increasing dendritic branching and spine count in the BLA, thereby enhancing conditioning effects and impairing extinction. Recently, deficits in fear extinction observed in these human disorders have been combated with pharmacological agents, facilitating extinction of the fear memory with the help of D-cycloserine and impairing fear memory reconsolidation with the B-adrenergic receptor blocker propranolol.
Early in the study of classical conditioning, Pavlov observed spontaneous recovery of responding to an extinguished conditioned stimulus. This witnessed 'uncovering phenomena', brought on by a change in context or stimulus presentation, led to the belief that extinction is not erasure of a previous fear memory but rather the learning of an additional inhibitory memory. This paper reviews what has been learned about extinction learning ever since. Like other types of learning, extinction occurs in three phases: acquisition, consolidation, and retrieval. Acquisition seems to depend on the basolateral amygdala (BLA) and the ventrolateral periaqueductal gray (vlPAG) structures. Consolidation seems to be most dependent on the BLA, where the learning of new memories (requiring protein synthesis) appears to take place. It also seems to rely on the involvement of the prefrontal cortex and the hippocampus. Retrieval requires expression of an inhibitory memory, and as such, during retrieval we see activation of inhibitory networks in the amygdala, cortical control of amygdala inhibition by the IL mPFC, and contextual regulation provided by the hippocampus and mPFC. Thus, like classical conditioning, extinction seems to be distributed across a network of structures, rather than centered in any one particular area.
The paper also raises many fascinating special issues. One challenges the Pavlovian idea that extinction is purely an additional inhibitory memory: recent evidence seems to indicate that extinction leads to reversal of conditioning-induced phosphorylation of CREB, indicating some erasure of the original BLA fear memory!
It also discusses anxiety disorders and PTSD which may be caused by a failure to retrieve an extinction memory generated in extinction-based treatment. Subjects with PTSD show reduced volume and activity in the vmPFC and hippocampus areas, along with increased activity in the amygdala, suggesting inhibitory control and contextual modulation of extinction may be compromised. It also mentions that chronic stress can impede extinction-based therapies, decreasing dendritic branching and spine count in the vmPFC and hippocampus, and increasing dendritic branching and spine count in the BLA, thereby enhancing conditioning effects and impairing extinction. Recently, deficits in fear extinction observed in these human disorders have been combated with pharmacological agents, facilitating extinction of the fear memory with the help of D-cycloserine and impairing fear memory reconsolidation with the B-adrenergic receptor blocker propranolol.
Monday, March 3, 2008
Human amygdala activation during conditioned fear acquisition and extinction
LaBar, K.S., Gatenby, J.C., Gore, J.C., LeDoux, J.E., & Phelps, E.A. (May 1998). Human amygdala activation during conditioned fear acquisition and extinction: a mixed-trial fMRI study. Neuron, Vol. 20, 937-945.
The amygdala is believed to be a key component in a network mediating survival functions by coordinating behavioral plans of action based on the integration of exteroceptive and interoceptive information. The amygdala, in particular, has been thought to be the structure responsible for detecting and reacting to potentially threatening environmental stimuli through classical conditioning learning.
However, attempts to investigate amygdala function in humans has produced inconsistent results, with failures to notice increased amygdala blood flow in PET being most surprising. Difficulties may stem from the small size and troublesome subcortical placement of the structure, and that amygdalar responses are relatively transient to discrete cues, have low spontaneous neuronal firing rates, and exhibit marked habituation (gradual signal intensity reduction). This study attempted to overcome these difficulties by using a mixed-trial fMRI design. This time, results successfully showed amygdala/periamygdaloid cortex involvement during both conditioned fear acquisition and extinction, biased towards the right hemisphere in both cases.
Previous lesion studies have shown the integrity of the amygdala is required for expression of learned conditioned fear associations. However, the temporal pattern of amygdala activity (greatest during early acquisition and early extinction, and degrading over time) suggests that this activity may only partially underlie expression. The paper offers a hypothesis that the observed activation may be related to encoding the emotional meaning of the conditioned stimulus. This is consistent with with the amygdala activity witnessed during the initial stages of learning (when stimulus is novel) and during early extinction (when the meaning of the stimulus has changed).
The amygdala is believed to be a key component in a network mediating survival functions by coordinating behavioral plans of action based on the integration of exteroceptive and interoceptive information. The amygdala, in particular, has been thought to be the structure responsible for detecting and reacting to potentially threatening environmental stimuli through classical conditioning learning.
However, attempts to investigate amygdala function in humans has produced inconsistent results, with failures to notice increased amygdala blood flow in PET being most surprising. Difficulties may stem from the small size and troublesome subcortical placement of the structure, and that amygdalar responses are relatively transient to discrete cues, have low spontaneous neuronal firing rates, and exhibit marked habituation (gradual signal intensity reduction). This study attempted to overcome these difficulties by using a mixed-trial fMRI design. This time, results successfully showed amygdala/periamygdaloid cortex involvement during both conditioned fear acquisition and extinction, biased towards the right hemisphere in both cases.
Previous lesion studies have shown the integrity of the amygdala is required for expression of learned conditioned fear associations. However, the temporal pattern of amygdala activity (greatest during early acquisition and early extinction, and degrading over time) suggests that this activity may only partially underlie expression. The paper offers a hypothesis that the observed activation may be related to encoding the emotional meaning of the conditioned stimulus. This is consistent with with the amygdala activity witnessed during the initial stages of learning (when stimulus is novel) and during early extinction (when the meaning of the stimulus has changed).
Friday, February 29, 2008
Brain systems mediating aversive conditioning
Buchel, C., Morris, J., Dolan, R.J., & Friston, K.J. (May 1998). Brain systems mediating aversive conditioning: an event-related fMRI study. Neuron, Vol. 20, 947-957.
Classical conditioning refers to a type of associative learning whereby a previously neutral stimulus (CS) comes to elicit a behavioral response by being paired with an aversive unconditioned stimulus (US). This study implemented a human classical conditioning paradigm in which images of faces (CS) were paired with an aversive tone (US). To assess which areas of the brain were related to conditioning, event-related fMRI responses were compared between presentation of conditioned stimuli (CS+) and unconditioned stimuli (CS-) after skin conductance indicated the conditioning regimen was completed successfully. To be more accurate, CS- was compared to the occasional trials of CS+ which were not followed by a tone.
Unequivocal differential responses were found in two cortical areas: the anterior cingulate cortex (ACC) and the anterior insula. These structures receive input from the amygdala, which also shows interesting activation patterns during this paradigm. The lateral amygdala in particular shows time-dependent neural responses, with higher than baseline responses at first but habituating over time. A possible explanation is a negative feedback loop, an analgesia kicked off by the amygdala and mediated by endogenous opioids which leads to reduced conditioning over time. Finally, differential activation was also witnessed in the red nucleus together with premotor structures, characteristic of response expression.
Happy 50th post!!! And happy bissextile day!!!
Classical conditioning refers to a type of associative learning whereby a previously neutral stimulus (CS) comes to elicit a behavioral response by being paired with an aversive unconditioned stimulus (US). This study implemented a human classical conditioning paradigm in which images of faces (CS) were paired with an aversive tone (US). To assess which areas of the brain were related to conditioning, event-related fMRI responses were compared between presentation of conditioned stimuli (CS+) and unconditioned stimuli (CS-) after skin conductance indicated the conditioning regimen was completed successfully. To be more accurate, CS- was compared to the occasional trials of CS+ which were not followed by a tone.
Unequivocal differential responses were found in two cortical areas: the anterior cingulate cortex (ACC) and the anterior insula. These structures receive input from the amygdala, which also shows interesting activation patterns during this paradigm. The lateral amygdala in particular shows time-dependent neural responses, with higher than baseline responses at first but habituating over time. A possible explanation is a negative feedback loop, an analgesia kicked off by the amygdala and mediated by endogenous opioids which leads to reduced conditioning over time. Finally, differential activation was also witnessed in the red nucleus together with premotor structures, characteristic of response expression.
Happy 50th post!!! And happy bissextile day!!!
Wednesday, February 27, 2008
Conflict Monitoring and the ACC
Botvinick, M.M., Cohen, J.D., & Carter, C.S. (December 2004). Conflict monitoring and anterior cingulate cortex: An update. Trends in Cognitive Sciences, Vol. 8, No. 12, 539-546.
Activity in the dorsal anterior cingulate cortex (ACC) shows up in a variety of tasks. For example, a transient potential (known as the error-related negativity, or ERN) is elicited from the posterior ACC in response to error commission. And a similar evoked potential, the feedback-related negativity (FRN), occurs in response to error feedback and may derive from the same portion of the cingulate that generates the ERN. These activity patterns during commission of errors led researchers to suggest an 'error detection' function for the ACC.
However, tasks which require overriding of habitual responses and tasks which require selecting among a set of equally permissible responses also yield ACC activation. This led researchers to put forth a 'conflict detection' theory of ACC function, with the structure being especially responsible for selecting between competing motor responses.
However, recent studies have proposed other unifying theories to explain the role of the ACC beyond just error detection and conflict monitoring. Some suggest the ACC serves to evaluate action outcomes, performing cost-benefit analyses on possible outcomes and using reward-related information to guide action selection. This 'action-outcome evaluation' view is particularly consonant with other research connecting the mesencephalic dopamine system with the ACC in reinforcement learning.
Activity in the dorsal anterior cingulate cortex (ACC) shows up in a variety of tasks. For example, a transient potential (known as the error-related negativity, or ERN) is elicited from the posterior ACC in response to error commission. And a similar evoked potential, the feedback-related negativity (FRN), occurs in response to error feedback and may derive from the same portion of the cingulate that generates the ERN. These activity patterns during commission of errors led researchers to suggest an 'error detection' function for the ACC.
However, tasks which require overriding of habitual responses and tasks which require selecting among a set of equally permissible responses also yield ACC activation. This led researchers to put forth a 'conflict detection' theory of ACC function, with the structure being especially responsible for selecting between competing motor responses.
However, recent studies have proposed other unifying theories to explain the role of the ACC beyond just error detection and conflict monitoring. Some suggest the ACC serves to evaluate action outcomes, performing cost-benefit analyses on possible outcomes and using reward-related information to guide action selection. This 'action-outcome evaluation' view is particularly consonant with other research connecting the mesencephalic dopamine system with the ACC in reinforcement learning.
Tuesday, January 15, 2008
The Amygdala
LeDoux, J. (October 2007). The amygdala, Current Biology, Vol. 17, R868-R874.
This is a fantastic overview of a very important brain structure by one of the world's leading experts on the subject. The article covers the amygdala's anatomic organization (its different nuclei), its connectivity (both inputs and outputs), its cellular mechanisms (neurotransmission and neuromodulation), its role in emotional processing, and its implication in a variety of human disorders. The article even covers neuronal processes related to classical conditioning (i.e. fear memory consolidation and reconsolidation), such as changes in synaptic strength (upregulation of post-synaptic receptors by LTP), structural changes in synaptic connectivity (post-synaptic cytoskeletal alterations, presumably dendritic and microtubule changes), and pre-synaptic feedback mechanisms (release of nitric oxide as a messenger).
This is a fantastic overview of a very important brain structure by one of the world's leading experts on the subject. The article covers the amygdala's anatomic organization (its different nuclei), its connectivity (both inputs and outputs), its cellular mechanisms (neurotransmission and neuromodulation), its role in emotional processing, and its implication in a variety of human disorders. The article even covers neuronal processes related to classical conditioning (i.e. fear memory consolidation and reconsolidation), such as changes in synaptic strength (upregulation of post-synaptic receptors by LTP), structural changes in synaptic connectivity (post-synaptic cytoskeletal alterations, presumably dendritic and microtubule changes), and pre-synaptic feedback mechanisms (release of nitric oxide as a messenger).
Saturday, December 1, 2007
The Neural Basis of Human Error Processing
Holroyd, C.B. and Coles, M.G.H. (2002). The Neural Basis of Human Error Processing: Reinforcement Learning, Dopamine, and the Error-Related Negativity, Psychological Review, Vol. 109, No. 4, 679-709.
It is clear that our superior ability to learn from consequences makes us quite exceptional animals. But what are the neural substrates for reinforcement learning? This article takes a stab at that question.
Researchers have inferred the existence of a generic, high-level error-processing system in the brain for some time. When human participants commit errors in a wide variety of psychological tasks, a negative deflection is witnessed in EEG data, deemed the error-related negativity (ERN), which appears to be generated from anterior cingulate cortex (ACC). On the other hand, researchers have argued that the mesencephalic dopamine system conveys reinforcement learning signals to the basal ganglia and frontal cortex, where they are used to facilitate development of adaptive behavioral programs. This article proposes a hypothesis which unifies the two – specifically, when human subjects commit errors the dopamine system conveys a negative reinforcement learning signal to the frontal cortex where it generates an ERN by disinhibiting the dendrites of motor neurons in the ACC (hence the negative potential seen).
To dig a bit deeper... first the mesencephalic dopamine system. This is a small collection of nuclei including the substantia nigra pars compacta and the ventral tegmental area (VTA) that project diffusely to the basal ganglia and frontal cortex. The consequence of stimulation from this area appears to reinforce learning, solidifying behavior. After learning how to complete a task properly, presentation of a reward elicits a phasic response in dopamine neurons. When a reward is better than predicted, a positive dopamine signal is elicited. And, as expected, when (i) an expected reward is not delivered, (ii) a reward is worse than predicted, or (iii) punishment is administered instead, mesencephalic dopamine neurons decrease their firing rate, falling below baseline. Interestingly, over time and practice on a task, the presentation of the reward no longer elicits the phasic dopaminergic response; instead, the conditioned stimulus predicting delivery of the reward elicits the phasic activity. As such, the phasic dopaminergic activity is said to propagate “back in time” from the reward to the conditioned stimulus with learning. Thus, the mesencephalic dopamine system can be understood to produce predictive and critical error signals which can be used by other parts of the brain for reinforcement learning.
Now the ERN. The ERN is a negative wave pattern witnessed during commission of an error, essentially the brain’s “Oh shit!” signal. The amplitude of the ERN increases with incentive (e.g. financial reward) and with the degree of error (i.e. very wrong errors as opposed to only slightly incorrect). The ERN can be elicited (i) by presentation of negative feedback to the participant, or (ii) by detection of error commission itself.
The theory that this paper offers is that the ACC, which generates the ERN and receives input from numerous semi-independent command structures, is responsible for conflict monitoring in the brain – detecting competing choices from these multiple motor controllers and resolving the response conflict. Its job is therefore to identify which of its input are best suited for carrying out the task – serving as a motor control filter – and finally transforming multiple intentions into a unitary action. They contend that the ACC is “trained” into choosing the correct controller by the mesencephalic dopamine system’s reinforcement learning signals, and that the ERN essentially reflects transmission of this reinforcement learning signal to the ACC. The mesencephalic dopamine system, then, plays the role of an adaptive critic, assigning a “goodness” or “badness” to witnessed response outcomes, and communicates its opinion to the ACC to bias future decision-making.
The article used behavioral data (a probabilistic learning task) to support this hypothesis, as well as computer simulation modeling which predicted the observed behavioral results.
It is clear that our superior ability to learn from consequences makes us quite exceptional animals. But what are the neural substrates for reinforcement learning? This article takes a stab at that question.
Researchers have inferred the existence of a generic, high-level error-processing system in the brain for some time. When human participants commit errors in a wide variety of psychological tasks, a negative deflection is witnessed in EEG data, deemed the error-related negativity (ERN), which appears to be generated from anterior cingulate cortex (ACC). On the other hand, researchers have argued that the mesencephalic dopamine system conveys reinforcement learning signals to the basal ganglia and frontal cortex, where they are used to facilitate development of adaptive behavioral programs. This article proposes a hypothesis which unifies the two – specifically, when human subjects commit errors the dopamine system conveys a negative reinforcement learning signal to the frontal cortex where it generates an ERN by disinhibiting the dendrites of motor neurons in the ACC (hence the negative potential seen).
To dig a bit deeper... first the mesencephalic dopamine system. This is a small collection of nuclei including the substantia nigra pars compacta and the ventral tegmental area (VTA) that project diffusely to the basal ganglia and frontal cortex. The consequence of stimulation from this area appears to reinforce learning, solidifying behavior. After learning how to complete a task properly, presentation of a reward elicits a phasic response in dopamine neurons. When a reward is better than predicted, a positive dopamine signal is elicited. And, as expected, when (i) an expected reward is not delivered, (ii) a reward is worse than predicted, or (iii) punishment is administered instead, mesencephalic dopamine neurons decrease their firing rate, falling below baseline. Interestingly, over time and practice on a task, the presentation of the reward no longer elicits the phasic dopaminergic response; instead, the conditioned stimulus predicting delivery of the reward elicits the phasic activity. As such, the phasic dopaminergic activity is said to propagate “back in time” from the reward to the conditioned stimulus with learning. Thus, the mesencephalic dopamine system can be understood to produce predictive and critical error signals which can be used by other parts of the brain for reinforcement learning.
Now the ERN. The ERN is a negative wave pattern witnessed during commission of an error, essentially the brain’s “Oh shit!” signal. The amplitude of the ERN increases with incentive (e.g. financial reward) and with the degree of error (i.e. very wrong errors as opposed to only slightly incorrect). The ERN can be elicited (i) by presentation of negative feedback to the participant, or (ii) by detection of error commission itself.
The theory that this paper offers is that the ACC, which generates the ERN and receives input from numerous semi-independent command structures, is responsible for conflict monitoring in the brain – detecting competing choices from these multiple motor controllers and resolving the response conflict. Its job is therefore to identify which of its input are best suited for carrying out the task – serving as a motor control filter – and finally transforming multiple intentions into a unitary action. They contend that the ACC is “trained” into choosing the correct controller by the mesencephalic dopamine system’s reinforcement learning signals, and that the ERN essentially reflects transmission of this reinforcement learning signal to the ACC. The mesencephalic dopamine system, then, plays the role of an adaptive critic, assigning a “goodness” or “badness” to witnessed response outcomes, and communicates its opinion to the ACC to bias future decision-making.
The article used behavioral data (a probabilistic learning task) to support this hypothesis, as well as computer simulation modeling which predicted the observed behavioral results.
Subscribe to:
Posts (Atom)