Lachesis: A Middleware for Customizing OS Scheduling of Stream Processing Queries
Paper in proceeding, 2021
Motivated by the above, we explore the feasibility and benefits of custom scheduling without alterations to SPEs but, instead, by orchestrating the OS scheduler (e.g., using nice and cgroup) to enforce the scheduling goals. We propose Lachesis, a standalone scheduling middleware, decoupled from any specific SPE, that can schedule multiple streaming applications, run in one or many nodes, and possibly multiple SPEs. Our evaluation with real-world and synthetic workloads, several SPEs and hardware setups, shows its benefits over default OS scheduling and other state-of-the-art schedulers: up to 75% higher throughput, and 1130x lower average latency once such SPEs reach their peak processing capacity.
operating systems
stream processing
scheduling
Author
Dimitrios Palyvos-Giannas
Network and Systems
Gabriele Mencagli
University of Pisa
Marina Papatriantafilou
Network and Systems
Vincenzo Massimiliano Gulisano
Network and Systems
Middleware 2021 - Proceedings of the 22nd International Middleware Conference
365-378
978-1-4503-8534-3 (ISBN)
Virtual Event, Canada,
AUTOSPADA (Automotive Stream Processing and Distributed Analytics) OODIDA Phase 2
VINNOVA (2019-05884), 2020-03-12 -- 2022-12-31.
Future factories in the Cloud (FiC)
Swedish Foundation for Strategic Research (SSF) (GMT14-0032), 2016-01-01 -- 2020-12-31.
INDEED
Chalmers, 2016-01-01 -- 2020-12-31.
STAMINA - GE
Göteborg Energi, Foundation for Research and Developmen, 2017-01-01 -- 2021-12-31.
HARE: Self-deploying and Adaptive Data Streaming Analytics in Fog Architectures
Swedish Research Council (VR) (2016-03800), 2017-01-01 -- 2020-12-31.
Subject Categories
Computer Engineering
Computer Science
Computer Systems
Areas of Advance
Energy
DOI
10.1145/3464298.3493407
ISBN
9781450385343