Publications
Detailed Information
Bitwidth-Adaptive Quantization-Aware Neural Network Training: A Meta-Learning Approach
DC Field | Value | Language |
---|---|---|
dc.contributor.author | Youn, Jiseok | - |
dc.contributor.author | Song, Jaehun | - |
dc.contributor.author | Kim, Hyung-Sin | - |
dc.contributor.author | Bahk, Saewoong | - |
dc.date.accessioned | 2024-05-14T08:06:20Z | - |
dc.date.available | 2024-05-14T08:06:20Z | - |
dc.date.created | 2023-03-22 | - |
dc.date.issued | 2022 | - |
dc.identifier.citation | Lecture Notes in Computer Science, Vol.13672, pp.208-224 | - |
dc.identifier.issn | 0302-9743 | - |
dc.identifier.uri | https://hdl.handle.net/10371/202136 | - |
dc.description.abstract | Deep neural network quantization with adaptive bitwidths has gained increasing attention due to the ease of model deployment on various platforms with different resource budgets. In this paper, we propose a meta-learning approach to achieve this goal. Specifically, we propose MEBQAT, a simple yet effective way of bitwidth-adaptive quantization-aware training (QAT) where meta-learning is effectively combined with QAT by redefining meta-learning tasks to incorporate bitwidths. After being deployed on a platform, MEBQAT allows the (meta-)trained model to be quantized to any candidate bitwidth with minimal inference accuracy drop. Moreover, in a few-shot learning scenario, MEBQAT can also adapt a model to any bitwidth as well as any unseen target classes by adding conventional optimization or metric-based meta-learning. We design variants of MEBQAT to support both (1) a bitwidth-adaptive quantization scenario and (2) a new few-shot learning scenario where both quantization bitwidths and target classes are jointly adapted. Our experiments show that merging bitwidths into meta-learning tasks results in remarkable performance improvement: 98.7% less storage cost compared to bitwidth-dedicated QAT and 94.7% less back propagation compared to bitwidth-adaptive QAT in bitwidth-only adaptation scenarios, while improving classification accuracy by up to 63.6% compared to vanilla meta-learning in bitwidth-class joint adaptation scenarios. | - |
dc.language | 영어 | - |
dc.publisher | Springer Verlag | - |
dc.title | Bitwidth-Adaptive Quantization-Aware Neural Network Training: A Meta-Learning Approach | - |
dc.type | Article | - |
dc.identifier.doi | 10.1007/978-3-031-19775-8_13 | - |
dc.citation.journaltitle | Lecture Notes in Computer Science | - |
dc.identifier.wosid | 000897093900013 | - |
dc.identifier.scopusid | 2-s2.0-85142745643 | - |
dc.citation.endpage | 224 | - |
dc.citation.startpage | 208 | - |
dc.citation.volume | 13672 | - |
dc.description.isOpenAccess | Y | - |
dc.contributor.affiliatedAuthor | Kim, Hyung-Sin | - |
dc.contributor.affiliatedAuthor | Bahk, Saewoong | - |
dc.type.docType | Proceedings Paper | - |
dc.description.journalClass | 1 | - |
- Appears in Collections:
- Files in This Item:
- There are no files associated with this item.
Item View & Download Count
Items in S-Space are protected by copyright, with all rights reserved, unless otherwise indicated.