Optimizing Memory Management Systems for High Performance and Scalability

Abstract

One common characteristic of modern workloads which appeared with recent computing paradigms including cloud, big data and machine learning is memory intensiveness. Such workloads usually have huge working sets that cannot be fully accommodated in DRAM in many case. Those also tend to show only low locality so that the small CPU cache cannot hide DRAM or lower level memory access overhead.

Meanwhile, computing hardware has also evolved to keep pace with this change. (1) Computing systems are increasing the size of their main memory so that those could accommodate more of the huge working sets. As a result, data center servers utilizing few hundreds of gigabytes of DRAM have been common and even terabytes of DRAM equipped systems exist. (2) Massive parallelism is becomming common and essential. CPU vendors have started to increase the number of CPU cores instead of the CPU frequency due to the heat dissipation and power consumption problem since the early 2000s. Prevalent datacenter systems provide few hundreds of CPU cores; Few thousands of CPU cores are not rare. Such many-core systems are normally constructed in non-uniform memory access (NUMA) architecture. Therefore, efficient, effective and NUMA-awared use of this parallelism is especially important for the memory intensive workloads.

Compared to these rapid changes of workload characteristics and hardware, memory management system software has not sufficiently optimized. Consequently, the memory management system software has been a bottleneck. In other words, the memory intensive modern workloads cannot fully utilize the evolved modern hardware unless the underlying memory management system is completely optimized. This paper provides an overview of a few limitations in existing memory management systems and introduces two optimization approaches for high performance and scalability of the memory management systems. The first approach improves the performance of the memory systems by guaranteeing huge page utilization under memory fragmentation situation. For the guarantee, we introduce a contiguous memory allocator that guarantees success and low latency of its allocations. The second approach intends to optimize the NUMA-aware system scalability. For that, we optimize virtual memory address space management system by substituting virtual memory area (VMA) managing red-black tree protection from global reader-writer locking to an RCU extension. Because no RCU extension including state-of-the-arts are NUMA oblivious, we also designed new RCU extension that provides NUMA-aware scalable update-side synchronization.