Deterministic Real-Time Analytics of Geospatial Data Streams through ScaleGate Objects

Paper in proceeding, 2015

In this work we present the design, implementation and evaluation of our approach to solve the DEBS 2015 Grand Challenge. Our work studies how ScaleGate, a concurrent implementation of a recently proposed abstract data type, that articulates data access in parallel data streaming, can be leveraged to partition the Grand Challenge analysis among an arbitrary number of processing units. ScaleGate aims not only at supporting high throughput and low latency parallel streaming analysis, but also at guaranteeing deterministic processing, which is one of the biggest challenges in parallelizing computation while maintaining consistency. Our main contribution is a new perspective for addressing the high throughput, low latency and determinism challenges of parallel data streaming by letting such challenges permeate the entire analysis framework, down to its underlying shared data objects. As a result, we propose shared data objects that balance independent actions among processing threads in order to guarantee high throughput, while providing the necessary synchronization for deterministic processing.