Publications

Detailed Information

Self-evolving vision transformer for chest X-ray diagnosis through knowledge distillation

Cited 18 time in Web of Science Cited 22 time in Scopus
Authors

Park, Sangjoon; Kim, Gwanghyun; Oh, Yujin; Seo, Joon Beom; Lee, Sang Min; Kim, Jin Hwan; Moon, Sungjun; Lim, Jae-Kwang; Park, Chang Min; Ye, Jong Chul

Issue Date
2022-07
Publisher
Nature Publishing Group
Citation
Nature Communications, Vol.13 No.1, p. 3848
Abstract
Although deep learning-based computer-aided diagnosis systems have recently achieved expert-level performance, developing a robust model requires large, high-quality data with annotations that are expensive to obtain. This situation poses a conundrum that annually-collected chest x-rays cannot be utilized due to the absence of labels, especially in deprived areas. In this study, we present a framework named distillation for self-supervision and self-train learning (DISTL) inspired by the learning process of the radiologists, which can improve the performance of vision transformer simultaneously with self-supervision and self-training through knowledge distillation. In external validation from three hospitals for diagnosis of tuberculosis, pneumothorax, and COVID-19, DISTL offers gradually improved performance as the amount of unlabeled data increase, even better than the fully supervised model with the same amount of labeled data. We additionally show that the model obtained with DISTL is robust to various real-world nuisances, offering better applicability in clinical setting. Although deep learning-based computer-aided diagnosis systems have recently achieved expert level performance, developing a robust model requires large, high-quality data with annotations. Here, the authors present a framework which can improve the performance of vision transformer simultaneously with self-supervision and self-training.
ISSN
2041-1723
URI
https://hdl.handle.net/10371/184722
DOI
https://doi.org/10.1038/s41467-022-31514-x
Files in This Item:
There are no files associated with this item.
Appears in Collections:

Related Researcher

  • College of Medicine
  • Department of Medicine
Research Area Radiology

Altmetrics

Item View & Download Count

  • mendeley

Items in S-Space are protected by copyright, with all rights reserved, unless otherwise indicated.

Share