Detailed Information

MERCI: Efficient embedding reduction on commodity hardware via sub-query memoization

Cited 0 time in Web of Science Cited 10 time in Scopus

Lee, Yejin; Seo, Seong Hoon; Choi, Hyunji; Sul, Hyoung Uk; Kim, Soosung; Lee, Jae Wook; Ham, Tae Jun

Issue Date
Association for Computing Machinery
International Conference on Architectural Support for Programming Languages and Operating Systems - ASPLOS, pp.302-313
© 2021 ACM.Deep neural networks (DNNs) with embedding layers are widely adopted to capture complex relationships among entities within a dataset. Embedding layers aggregate multiple embeddings- A dense vector used to represent the complicated nature of a data feature-into a single embedding; such operation is called embedding reduction. Embedding reduction spends a significant portion of its runtime on reading embeddings from memory and thus is known to be heavily memory-bandwidth-bound. Recent works attempt to accelerate this critical operation, but they often require either hardware modifications or emerging memory technologies, which makes it hardly deployable on commodity hardware. Thus, we propose MERCI, Memoization for Embedding Reduction with ClusterIng, a novel memoization framework for efficient embedding reduction. MERCI provides a mechanism for memoizing partial aggregation of correlated embeddings and retrieving the memoized partial result at a low cost. MERCI substantially reduces the number of memory accesses by 44% (29%), leading to 102% (74%) throughput improvement on real machines and 40.2% (28.6%) energy savings at the expense of 8×(1×) additional memory usage.
Files in This Item:
There are no files associated with this item.
Appears in Collections:


Item View & Download Count

  • mendeley

Items in S-Space are protected by copyright, with all rights reserved, unless otherwise indicated.