Advanced

Affective–associative two-process theory : a neurocomputational account of partial reinforcement extinction effects

Lowe, Robert; Almér, Alexander; Billing, Erik; Sandamirskaya, Yulia and Balkenius, Christian LU (2017) In Biological Cybernetics 111(5-6). p.365-388
Abstract

The partial reinforcement extinction effect (PREE) is an experimentally established phenomenon: behavioural response to a given stimulus is more persistent when previously inconsistently rewarded than when consistently rewarded. This phenomenon is, however, controversial in animal/human learning theory. Contradictory findings exist regarding when the PREE occurs. One body of research has found a within-subjects PREE, while another has found a within-subjects reversed PREE (RPREE). These opposing findings constitute what is considered the most important problem of PREE for theoreticians to explain. Here, we provide a neurocomputational account of the PREE, which helps to reconcile these seemingly contradictory findings of within-subjects... (More)

The partial reinforcement extinction effect (PREE) is an experimentally established phenomenon: behavioural response to a given stimulus is more persistent when previously inconsistently rewarded than when consistently rewarded. This phenomenon is, however, controversial in animal/human learning theory. Contradictory findings exist regarding when the PREE occurs. One body of research has found a within-subjects PREE, while another has found a within-subjects reversed PREE (RPREE). These opposing findings constitute what is considered the most important problem of PREE for theoreticians to explain. Here, we provide a neurocomputational account of the PREE, which helps to reconcile these seemingly contradictory findings of within-subjects experimental conditions. The performance of our model demonstrates how omission expectancy, learned according to low probability reward, comes to control response choice following discontinuation of reward presentation (extinction). We find that a PREE will occur when multiple responses become controlled by omission expectation in extinction, but not when only one omission-mediated response is available. Our model exploits the affective states of reward acquisition and reward omission expectancy in order to differentially classify stimuli and differentially mediate response choice. We demonstrate that stimulus–response (retrospective) and stimulus–expectation–response (prospective) routes are required to provide a necessary and sufficient explanation of the PREE versus RPREE data and that Omission representation is key for explaining the nonlinear nature of extinction data.

(Less)
Please use this url to cite or link to this publication:
author
organization
publishing date
type
Contribution to journal
publication status
published
subject
keywords
Affect, Associative two-process theory, Decision making, Partial reinforcement, Reinforcement learning
in
Biological Cybernetics
volume
111
issue
5-6
pages
365 - 388
publisher
Springer
external identifiers
  • scopus:85029510456
  • wos:000415625500004
ISSN
0340-1200
DOI
10.1007/s00422-017-0730-1
language
English
LU publication?
yes
id
7bbdc536-5629-4458-b7e2-64d01db138c4
date added to LUP
2017-09-29 13:15:18
date last changed
2018-02-19 01:56:36
@article{7bbdc536-5629-4458-b7e2-64d01db138c4,
  abstract     = {<p>The partial reinforcement extinction effect (PREE) is an experimentally established phenomenon: behavioural response to a given stimulus is more persistent when previously inconsistently rewarded than when consistently rewarded. This phenomenon is, however, controversial in animal/human learning theory. Contradictory findings exist regarding when the PREE occurs. One body of research has found a within-subjects PREE, while another has found a within-subjects reversed PREE (RPREE). These opposing findings constitute what is considered the most important problem of PREE for theoreticians to explain. Here, we provide a neurocomputational account of the PREE, which helps to reconcile these seemingly contradictory findings of within-subjects experimental conditions. The performance of our model demonstrates how omission expectancy, learned according to low probability reward, comes to control response choice following discontinuation of reward presentation (extinction). We find that a PREE will occur when multiple responses become controlled by omission expectation in extinction, but not when only one omission-mediated response is available. Our model exploits the affective states of reward acquisition and reward omission expectancy in order to differentially classify stimuli and differentially mediate response choice. We demonstrate that stimulus–response (retrospective) and stimulus–expectation–response (prospective) routes are required to provide a necessary and sufficient explanation of the PREE versus RPREE data and that Omission representation is key for explaining the nonlinear nature of extinction data.</p>},
  author       = {Lowe, Robert and Almér, Alexander and Billing, Erik and Sandamirskaya, Yulia and Balkenius, Christian},
  issn         = {0340-1200},
  keyword      = {Affect,Associative two-process theory,Decision making,Partial reinforcement,Reinforcement learning},
  language     = {eng},
  number       = {5-6},
  pages        = {365--388},
  publisher    = {Springer},
  series       = {Biological Cybernetics},
  title        = {Affective–associative two-process theory : a neurocomputational account of partial reinforcement extinction effects},
  url          = {http://dx.doi.org/10.1007/s00422-017-0730-1},
  volume       = {111},
  year         = {2017},
}