Haren: A Framework for Ad-Hoc Thread Scheduling Policies for Data Streaming Applications
Paper i proceeding, 2019

In modern Stream Processing Engines (SPEs), numerous diverse applications, which can differ in aspects such as cost, criticality or latency sensitivity, can co-exist in the same computing node. When these differences need to be considered to control the performance of each application, custom scheduling of operators to threads is of key importance (e.g., when a smart vehicle needs to ensure that safety-critical applications always have access to computational power, while other applications are given lower, variable priorities).

Many solutions have been proposed regarding schedulers that allocate threads to operators to optimize specific metrics (e.g., latency) but there is still lack of a tool that allows arbitrarily complex scheduling strategies to be seamlessly plugged on top of an SPE. We propose Haren to fill this gap. More specifically, we (1) formalize the thread scheduling problem in stream processing in a general way, allowing to define ad-hoc scheduling policies, (2) identify the bottlenecks and the opportunities of scheduling in stream processing, (3) distill a compact interface to connect Haren with SPEs, enabling rapid testing of various scheduling policies, (4) illustrate the usability of the framework by integrating it into an actual SPE and (5) provide a thorough evaluation. As we show, Haren makes it is possible to adapt the use of computational resources over time to meet the goals of a variety of scheduling policies.

Scheduling

Stream processing

Middleware

Författare

Dimitrios Palyvos-Giannas

Chalmers, Data- och informationsteknik, Nätverk och system

Vincenzo Massimiliano Gulisano

Chalmers, Data- och informationsteknik, Nätverk och system

Marina Papatriantafilou

Chalmers, Data- och informationsteknik, Nätverk och system

13th ACM International Conference on Distributed and Event-based Systems
Darmstadt, Germany,

Molnbaserade produkter och produktion (FiC)

Stiftelsen för Strategisk forskning (SSF), 2016-01-01 -- 2020-12-31.

INDEED

Chalmers, 2016-01-01 -- 2020-12-31.

HAREN: Självdistribuerad och anpassningsbar dataströmningsanalys i dimman

Vetenskapsrådet (VR), 2017-01-01 -- 2020-12-31.

STAMINA - GE

Göteborg Energi, Forskningsstiftelsen, 2017-01-01 -- 2021-12-31.

Ämneskategorier

Datorteknik

Datavetenskap (datalogi)

Datorsystem

Styrkeområden

Informations- och kommunikationsteknik

DOI

10.1145/3328905.3329505

Mer information

Senast uppdaterat

2019-09-04