Publications
Detailed Information
Reinforcing an image caption generator using off-line human feedback
Cited 0 time in
Web of Science
Cited 18 time in Scopus
- Authors
- Issue Date
- 2020-04
- Publisher
- AAAI press
- Citation
- AAAI 2020 - 34th AAAI Conference on Artificial Intelligence, Vol.34 No.3, pp.2693-2700
- Abstract
- Human ratings are currently the most accurate way to assess the quality of an image captioning model, yet most often the only used outcome of an expensive human rating evaluation is a few overall statistics over the evaluation dataset. In this paper, we show that the signal from instance-level human caption ratings can be leveraged to improve captioning models, even when the amount of caption ratings is several orders of magnitude less than the caption training data. We employ a policy gradient method to maximize the human ratings as rewards in an off-policy reinforcement learning setting, where policy gradients are estimated by samples from a distribution that focuses on the captions in a caption ratings dataset. Our empirical evidence indicates that the proposed method learns to generalize the human raters judgments to a previously unseen set of images, as judged by a different set of human judges, and additionally on a different, multidimensional side-by-side human evaluation procedure.
- Files in This Item:
- There are no files associated with this item.
Item View & Download Count
Items in S-Space are protected by copyright, with all rights reserved, unless otherwise indicated.