Emotion-Based Reinforcement Learning

Abstract

Studies have shown that counterfactual reasoning can shape human decisions. However, there is a gap in the literature between counterfactual choices in description-based and experience-based paradigms. While studies using description-based paradigms suggest participants maximize expected subjective emotion, studies using experience-based paradigms assume that participants learn the values of options and select what maximizes expected utility. In this study, we used computational modeling to test 1) whether participants make emotion-based decisions in experience-based paradigms, and 2) whether the impact of regret depends on its degree of unexpectedness as suggested by the current regret theory. The results suggest that 1) participants make emotion-based choices even in experience-based paradigms, and 2) the impact of regret is greater when it is expected than when it is unexpected. These results challenge the current theory of regret and suggest that reinforcement learning models may need to use counterfactual value functions when full information is provided.