Publications
Detailed Information
Self-guided contrastive learning for BERT sentence representations
DC Field | Value | Language |
---|---|---|
dc.contributor.author | Kim, Taeuk | - |
dc.contributor.author | Yoo, Kang Min | - |
dc.contributor.author | Lee, Sang-Goo | - |
dc.date.accessioned | 2022-06-24T00:25:48Z | - |
dc.date.available | 2022-06-24T00:25:48Z | - |
dc.date.created | 2022-05-04 | - |
dc.date.issued | 2021-08 | - |
dc.identifier.citation | ACL-IJCNLP 2021 - 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing, Proceedings of the Conference, pp.2528-2540 | - |
dc.identifier.uri | https://hdl.handle.net/10371/183728 | - |
dc.description.abstract | © 2021 Association for Computational LinguisticsAlthough BERT and its variants have reshaped the NLP landscape, it still remains unclear how best to derive sentence embeddings from such pre-trained Transformers. In this work, we propose a contrastive learning method that utilizes self-guidance for improving the quality of BERT sentence representations. Our method fine-tunes BERT in a self-supervised fashion, does not rely on data augmentation, and enables the usual [CLS] token embeddings to function as sentence vectors. Moreover, we redesign the contrastive learning objective (NT-Xent) and apply it to sentence representation learning. We demonstrate with extensive experiments that our approach is more effective than competitive baselines on diverse sentence-related tasks. We also show it is efficient at inference and robust to domain shifts. | - |
dc.language | 영어 | - |
dc.publisher | Association for Computational Linguistics (ACL) | - |
dc.title | Self-guided contrastive learning for BERT sentence representations | - |
dc.type | Article | - |
dc.citation.journaltitle | ACL-IJCNLP 2021 - 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing, Proceedings of the Conference | - |
dc.identifier.wosid | 000698663100197 | - |
dc.identifier.scopusid | 2-s2.0-85115408815 | - |
dc.citation.endpage | 2540 | - |
dc.citation.startpage | 2528 | - |
dc.description.isOpenAccess | N | - |
dc.contributor.affiliatedAuthor | Lee, Sang-Goo | - |
dc.type.docType | Conference Paper | - |
dc.description.journalClass | 1 | - |
- Appears in Collections:
- Files in This Item:
- There are no files associated with this item.
Item View & Download Count
Items in S-Space are protected by copyright, with all rights reserved, unless otherwise indicated.