Publications

Detailed Information

SCOPA: Soft Code-Switching and Pairwise Alignment for Zero-Shot Cross-lingual Transfer

Cited 0 time in Web of Science Cited 5 time in Scopus
Authors

Lee, Dohyeon; Lee, Jaeseong; Lee, Gyewon; Chun, Byung Gon; Hwang, Seung-Won

Issue Date
2021-10
Publisher
Association for Computing Machinery
Citation
International Conference on Information and Knowledge Management, Proceedings, pp.3176-3180
Abstract
© 2021 ACM.The recent advent of cross-lingual embeddings, such as multilingual BERT (mBERT), provides a strong baseline for zero-shot cross-lingual transfer. There also exists increasing research attention to reduce the alignment discrepancy of cross-lingual embeddings between source and target languages, via generating code-switched sentences by substituting randomly selected words in the source languages with their counterparts of the target languages. Although these approaches improve the performance, naively code-switched sentences can have inherent limitations. In this paper, we propose SCOPA, a novel technique to improve the performance of zero-shot cross-lingual transfer. Instead of using the embeddings of code-switched sentences directly, SCOPA mixes them softly with the embeddings of original sentences. In addition, SCOPA utilizes an additional pairwise alignment objective, which aligns the vector differences of word pairs instead of word-level embeddings, in order to transfer contextualized information between different languages while preserving language-specific information. Experiments on the PAWS-X and MLDoc dataset show the effectiveness of SCOPA.
URI
https://hdl.handle.net/10371/184198
DOI
https://doi.org/10.1145/3459637.3482176
Files in This Item:
There are no files associated with this item.
Appears in Collections:

Altmetrics

Item View & Download Count

  • mendeley

Items in S-Space are protected by copyright, with all rights reserved, unless otherwise indicated.

Share