eChIDNA: Continuous Data Validation in Advanced Metering Infrastructures
Paper i proceeding, 2018

New laws and regulations increase the demands for a more data-intense metering infrastructure towards more adaptive electricity networks (aka smart grids). The automatic measuring, often involving wireless communication, introduces errors both in software and during data transmission. These demands, as well as the large data volumes that need to be validated, present new challenges to utilities. First, measurement errors cannot be allowed to propagate to the data stored by utilities. Second, manual fixing of errors after storing is not a feasible option with increasing data volumes and decreasing lead times for new services and analysis. Third, validation is not only to be applied to current readings but also to past readings when new types of errors are discovered. This paper addresses these issues by proposing a hybrid system, eChIDNA, utilizing both the store-then-process and the data streaming processing paradigms, enabling for high throughput, low latency distributed and parallel analysis. Validation rules are built upon this paradigm and then implemented on the state of the art Apache Storm Stream Processing Engine to assess performance. Furthermore, patterns of common errors are matched, triggering alerts as a first step towards automatic correction of errors. The system is evaluated with production data from hundreds of thousands of smart meters. The results show a performance in the thousands messages per second realm, showing that stream processing can be used to validate large volumes of meter data online with low processing latency, identifying common errors as they appear. The results from the pattern matching are cross-validated with system experts and show that pattern matching is a viable way to minimize time required from human operators.

Författare

Johan Swetzén

Chalmers, Data- och informationsteknik

Joris Van Rooij

Chalmers, Data- och informationsteknik, Nätverk och system

Vincenzo Massimiliano Gulisano

Chalmers, Data- och informationsteknik, Nätverk och system

Magnus Almgren

Chalmers, Data- och informationsteknik, Nätverk och system

Marina Papatriantafilou

Chalmers, Data- och informationsteknik, Nätverk och system

2018 IEEE International Energy Conference (ENERGYCON)

2018 IEEE International Energy Conference (ENERGYCON)
Limassol, Cyprus,

STAMINA -WASP

Wallenberg AI, Autonomous Systems and Software Program, 2016-04-04 -- 2020-04-06.

Integrated cyber-physical solutions for intelligent distribution grid with high penetration of renewables (UNITED-GRID)

Europeiska kommissionen (Horisont 2020), 2017-11-01 -- 2020-04-30.

STAMINA - GE

Göteborg Energi, Forskningsstiftelsen, 2017-01-01 -- 2021-12-31.

Ämneskategorier

Datorteknik

Datavetenskap (datalogi)

Datorsystem

Styrkeområden

Energi

DOI

10.1109/ENERGYCON.2018.8398800

Mer information

Senast uppdaterat

2019-07-02