To simulate or not? Comment on steingroever, Wetzels, and Wagenmakers (2014)

Abstract

Steingroever, Wetzels, and Wagenmakers (2014) conducted a detailed investigation of 3 popular reinforcement-learning models for the Iowa gambling task using 2 model comparison techniques: a post hoc fit criterion and a simulation method. However, these 2 methods yield inconsistent results regarding which model should be preferred as a description of underlying psychological processes. Here, we describe the benefits of each method in an attempt to develop a more balanced view of how to utilize these model comparison techniques, and we outline the risks of focusing on a single method to make inferences about the overall utility of a model. Also, we make several suggestions about how applied research should evaluate candidate cognitive models, and we offer guidelines for future research aimed at identifying "good" models for decomposing and explaining participants' performance.