Graph neural network-based propagation effects modeling for detecting visual relationships among construction resources

Abstract

© 2022Detecting visual relationships among construction resources plays a pivotal role in understanding complex construction scenes and performing vision-based site monitoring and digitalization. Despite extensive efforts, the propagation effects of different resource-to-resource interactions were overlooked and thus, it is still challenging to precisely detect entangled and intertwined visual relationships from actual construction images. To address the challenge, this study proposes a semantic graph neural network approach that structuralizes construction resources and their entangled interactions in the form of a graph, and simulates the propagation effects using a neural message passing mechanism. The experimental results showed that the proposed approach achieved 77.1% F-score—11.5% higher than the performance of the baseline model. This suggests the positive impacts of the propagation effects and the applicability of the proposed approach. These findings can help understand what are actually happening at construction sites automatically and provide valuable insights for future vision-based monitoring studies.