Adapting Memory Hierarchies for Emerging Datacenter Interconnects
Journal article, 2015

Efficient resource utilization requires that emerging datacenter interconnects support both high performance communication and efficient remote resource sharing. These goals require that the network be more tightly coupled with the CPU chips. Designing a new interconnection technology thus requires considering not only the interconnection itself, but also the design of the processors that will rely on it. In this paper, we study memory hierarchy implications for the design of high-speed datacenter interconnects particularly as they affect remote memory access and we use PCIe as the vehicle for our investigations. To that end, we build three complementary platforms: a PCIe-interconnected prototype server with which we measure and analyze current bottlenecks; a software simulator that lets us model microarchitectural and cache hierarchy changes; and an FPGA prototype system with a streamlined switchless customized protocol Thunder with which we study hardware optimizations outside the processor. We highlight several architectural modifications to better support remote memory access and communication, and quantify their impact and limitations.

memory hierarchy

time shared memory

datacenter network

high-speed interconnect


Tao Jiang

Chinese Academy of Sciences

Rui Hou

Chinese Academy of Sciences

Jianbo Dong

Chinese Academy of Sciences

L. Chai

Chinese Academy of Sciences

Sally A McKee

Chalmers, Computer Science and Engineering (Chalmers), Computer Engineering (Chalmers)

B. Tian

National High Performance Integrated Circuit Design Center (Shanghai)

L. Zhang

Chinese Academy of Sciences

N. H. Sun

Chinese Academy of Sciences

Journal of Computer Science and Technology

1000-9000 (ISSN) 1860-4749 (eISSN)

Vol. 30 1 97-109

Subject Categories

Computer and Information Science



More information

Latest update