Delegation sketch: A parallel design with support for fast and accurate concurrent operations
Paper in proceedings, 2020

Sketches are data structures designed to answer approximate queries by trading memory overhead with accuracy guarantees. More specifically, sketches efficiently summarize large, high-rate streams of data and quickly answer queries on these summaries. In order to support such high throughput rates in modern architectures, parallelization and support for fast queries play a central role, especially when monitoring unpredictable data that can change rapidly as, e.g., in network monitoring for large-scale denial-of-service attacks. However, most existing parallel sketch designs have focused either on high insertion rate or on high query rate, and fail to support cases when these operations are concurrent. In this work we examine the trade-off between query and insertion efficiency and we propose Delegation Sketch, a parallelization design for sketch-based data structures to efficiently support concurrent insertions and queries. Delegation Sketch introduces a domain splitting scheme that uses multiple, parallel sketches to ensure all occurrences of a key fall into the same sketch. We complement the design by proposing synchronization mechanisms that facilitate delegation of insertion and queries among threads, enabling it to process streams at higher rates, even in the presence of concurrent queries. We thoroughly evaluate Delegation Sketch across multiple dimensions (accuracy, scalability, query rate and input skew) on two massively parallel platforms (including a NUMA architecture) using both synthetic and real data. We show that Delegation Sketch achieves from 2.5X to 4X higher throughput, depending on the rate of concurrent queries, than the best performing alternative, while at the same time maintaining better accuracy at the same memory cost.

Author

Charalampos Stylianopoulos

Chalmers, Computer Science and Engineering (Chalmers), Networks and Systems (Chalmers)

Ivan Walulya

Chalmers, Computer Science and Engineering (Chalmers), Networks and Systems (Chalmers)

Magnus Almgren

Chalmers, Computer Science and Engineering (Chalmers), Networks and Systems (Chalmers)

Olaf Landsiedel

Chalmers, Computer Science and Engineering (Chalmers), Networks and Systems (Chalmers)

Marina Papatriantafilou

Chalmers, Computer Science and Engineering (Chalmers), Networks and Systems (Chalmers)

Proceedings of the 15th European Conference on Computer Systems, EuroSys 2020

15th European Conference on Computer Systems, EuroSys 2020
Heraklion, Greece,

Integrated cyber-physical solutions for intelligent distribution grid with high penetration of renewables (UNITED-GRID)

European Commission (Horizon 2020), 2017-11-01 -- 2020-04-30.

Subject Categories

Computer Engineering

Computer Science

Computer Systems

DOI

10.1145/3342195.3387542

ISBN

9781450368827

More information

Latest update

9/1/2020 1