Publications
Detailed Information
Video-Text Compliance: Activity Verification Based on Natural Language Instructions
Cited 0 time in
Web of Science
Cited 0 time in Scopus
- Authors
- Issue Date
- 2019
- Publisher
- IEEE COMPUTER SOC
- Citation
- 2019 IEEE/CVF INTERNATIONAL CONFERENCE ON COMPUTER VISION WORKSHOPS (ICCVW), pp.1503-1512
- Abstract
- We define a new multi-modal compliance problem, which is to determine if the human activity in a given video is in compliance with an associated text instruction. Learning at the junction of vision and text for the compliance problem requires addressing the challenges caused by irregularities in videos and ambiguities in natural language. Successful solutions to the compliance problem could enable automatic compliance checking and efficient feedback in many real-world settings. To this end, we introduce the Video Text Compliance (VTC) dataset, which contains videos of atomic activities, along with text instructions and compliance labels. The VTC dataset is constructed by an auto augmentation technique, preserves privacy, and contains over 1.2 million frames. Finally we present ComplianceNet, a novel end-to-end trainable network to solve the video-text compliance task. Trained on the VTC dataset, ComplianceNet improves the baseline accuracy by 27.5% on average. We plan to release the VTC dataset to the community for future research.
- ISSN
- 2473-9936
- Files in This Item:
- There are no files associated with this item.
Related Researcher
- College of Engineering
- Department of Electrical and Computer Engineering
Item View & Download Count
Items in S-Space are protected by copyright, with all rights reserved, unless otherwise indicated.