High Performance Hybrid Memory Systems with 3D-stacked DRAM
Doctoral thesis, 2020

The bandwidth of traditional DRAM is pin limited and so does not scale well with the increasing demand of data intensive workloads. 3D-stacked DRAM can alleviate this problem providing substantially higher bandwidth to a processor chip. However, the capacity of 3D-stacked DRAM is not enough to replace the bulk of the memory and therefore it is used together with off-chip DRAM in a hybrid memory system, either as a DRAM cache or as part of a flat address space with support for data migration. The performance of both above alternative designs is limited by their particular overheads. This thesis proposes new designs that improve the performance of hybrid memory systems. It does so first by alleviating the overheads of current approaches and second, by proposing a new design that combines the best attributes of DRAM caching and data migration while addressing their respective weaknesses. The first part of this thesis focuses on improving the performance of DRAM caches. Besides the unavoidable DRAM access to fetch the requested data, tag access is in the critical path adding significant latency and energy costs. Existing approaches are not able to remove these overheads and in some cases limit DRAM cache design options. To alleviate the tag access overheads of DRAM caches this thesis proposes Decoupled Fused Cache (DFC), a DRAM cache design that fuses DRAM cache tags with the tags of the on-chip Last Level Cache (LLC) to access the DRAM cache data directly on LLC misses. Compared to current state-of-the-art DRAM caches, DFC improves system performance by 11% on average. Finally, DFC reduces DRAM cache traffic by 25% and DRAM cache energy consumption by 24.5%. The second part of this thesis focuses on improving the performance of data migration. Data migration has significant performance potential, but also entails overheads which may diminish its benefits or even degrade performance. These overheads are mainly due to the high cost of swapping data between memories which also makes selecting which data to migrate critical to performance. To address these challenges of data migration this thesis proposes LLC guided Data Migration (LGM). LGM uses the LLC to predict future reuse and select memory segments for migration. Furthermore, LGM reduces the data migration traffic overheads by not migrating the cache lines of memory segments which are present in the LLC. LGM outperforms current state-of-the art data migration, improving system performance by 12.1% and reducing memory system dynamic energy by 13.2%. DRAM caches and data migration offer different tradeoffs for the utilization of 3D-stacked DRAM but also share some similar challenges. The third part of this thesis aims to provide an alternative approach to the utilization of 3D-stacked DRAM combining the strengths of both DRAM caches and data migration while eliminating their weaknesses. To that end, this thesis proposes Hybrid2, a hybrid memory system design which uses only a small fraction of the 3D-stacked DRAM as a cache and thus does not deny valuable capacity from the memory system. It further leverages the DRAM cache as a staging area to select the data most suitable for migration. Finally, Hybrid2 alleviates the metadata overheads of both DRAM caches and migration using a common mechanism. Depending on the system configuration, Hybrid2 on average outperforms state-of-the-art migration schemes by 6.4% to 9.1%, compared to DRAM caches Hybrid2 gives away on average only 0.3%, to 5.3% of performance offering up to 24.6% more main memory capacity.

DRAM caches

Data migration

3D-stacked DRAM

Hybrid memory systems

EDIT 8103, EDIT building, Chalmers Campus Johanneberg, Gothenburg
Opponent: Babak Falsafi, Professor at EPFL, Switzerland

Author

Evangelos Vasilakis

Chalmers, Computer Science and Engineering (Chalmers), Computer Engineering (Chalmers)

Computers have revolutionized the way we interact with information through ever more efficient designs that are faster, take less space, and consume less energy. The current computing paradigm is based on two distinct parts that play an equally important role; the processor and the memory system. Both have benefited from the advances in transistor manufacturing, albeit in different ways. Processors have become faster while memory has become bigger, this creates a problem as the increasing disparity in speed between them means that, in many cases, the processor is constrained by the memory system.

In recent years, 3D-stacking has enabled placing the memory much closer to the processor and also to increase the available bandwidth, which is essential to satisfy the ever increasing processing power. However, 3D-stacking comes with other limitations which mandate that 3D-stacked memory must be used in tandem with conventional memory, creating hybrid memory systems. Hybrid memory system architectures try to combine memory technologies with complementary characteristics to provide a more efficient memory system. This thesis proposes novel hybrid memory system architectures that improve the performance and efficiency of existing approaches.

Meeting Challenges in Computer Architecture (MECCA)

European Commission (EC) (EC/FP7/340328), 2014-02-01 -- 2019-01-31.

Secure Hardware-Software Architectures for Robust Computing Systems (SHARCS)

European Commission (EC) (EC/H2020/644571), 2015-01-01 -- 2018-12-31.

Energy-efficient Heterogeneous COmputing at exaSCALE (ECOSCALE)

European Commission (EC) (EC/H2020/671632), 2015-10-01 -- 2018-12-31.

Subject Categories

Computer Engineering

Communication Systems

Computer Systems

Areas of Advance

Information and Communication Technology

ISBN

978-91-7905-311-6

Doktorsavhandlingar vid Chalmers tekniska högskola. Ny serie: 4778

Publisher

Chalmers

EDIT 8103, EDIT building, Chalmers Campus Johanneberg, Gothenburg

Online

Opponent: Babak Falsafi, Professor at EPFL, Switzerland

More information

Latest update

6/5/2020 4