Understanding the behavior of in-memory computing workloads
Paper i proceeding, 2014

© 2014 IEEE. The increasing demands of big data applications have led researchers and practitioners to turn to in-memory computing to speed processing. For instance, the Apache Spark framework stores intermediate results in memory to deliver good performance on iterative machine learning and interactive data analysis tasks. To the best of our knowledge, though, little work has been done to understand Spark's architectural and microarchitectural behaviors. Furthermore, although conventional commodity processors have been well optimized for traditional desktops and HPC, their effectiveness for Spark workloads remains to be studied. To shed some light on the effectiveness of conventional generalpurpose processors on Spark workloads, we study their behavior in comparison to those of Hadoop, CloudSuite, SPEC CPU2006, TPC-C, and DesktopCloud. We evaluate the benchmarks on a 17-node Xeon cluster. Our performance results reveal that Spark workloads have significantly different characteristics from Hadoop and traditional HPC benchmarks. At the system level, Spark workloads have good memory bandwidth utilization (up to 50%), stable memory accesses, and high disk IO request frequency (200 per second). At the microarchitectural level, the cache and TLB are effective for Spark workloads, but the L2 cache miss rate is high. We hope this work yields insights for chip and datacenter system designers.


Tao Jiang

Chinese Academy of Sciences

Q. Zhang

Chinese Academy of Sciences

Rui Hou

Chinese Academy of Sciences

L. Chai

Chinese Academy of Sciences

Sally A McKee

Chalmers, Data- och informationsteknik, Datorteknik

Z. Jia

Chinese Academy of Sciences

N. Sun

Chinese Academy of Sciences

IISWC 2014 - IEEE International Symposium on Workload Characterization



Data- och informationsvetenskap