eChIDNA: Continuous Data Validation in Advanced Metering Infrastructures
Paper in proceedings, 2018

New laws and regulations increase the demands for a more data-intense metering infrastructure towards more adaptive electricity networks (aka smart grids). The automatic measuring, often involving wireless communication, introduces errors both in software and during data transmission. These demands, as well as the large data volumes that need to be validated, present new challenges to utilities. First, measurement errors cannot be allowed to propagate to the data stored by utilities. Second, manual fixing of errors after storing is not a feasible option with increasing data volumes and decreasing lead times for new services and analysis. Third, validation is not only to be applied to current readings but also to past readings when new types of errors are discovered. This paper addresses these issues by proposing a hybrid system, eChIDNA, utilizing both the store-then-process and the data streaming processing paradigms, enabling for high throughput, low latency distributed and parallel analysis. Validation rules are built upon this paradigm and then implemented on the state of the art Apache Storm Stream Processing Engine to assess performance. Furthermore, patterns of common errors are matched, triggering alerts as a first step towards automatic correction of errors. The system is evaluated with production data from hundreds of thousands of smart meters. The results show a performance in the thousands messages per second realm, showing that stream processing can be used to validate large volumes of meter data online with low processing latency, identifying common errors as they appear. The results from the pattern matching are cross-validated with system experts and show that pattern matching is a viable way to minimize time required from human operators.

Author

Johan Swetzén

Chalmers, Computer Science and Engineering (Chalmers)

Joris Van Rooij

Chalmers, Computer Science and Engineering (Chalmers), Networks and Systems (Chalmers)

Vincenzo Massimiliano Gulisano

Chalmers, Computer Science and Engineering (Chalmers), Networks and Systems (Chalmers)

Magnus Almgren

Chalmers, Computer Science and Engineering (Chalmers), Networks and Systems (Chalmers)

Marina Papatriantafilou

Chalmers, Computer Science and Engineering (Chalmers), Networks and Systems (Chalmers)

2018 IEEE International Energy Conference (ENERGYCON)

2018 IEEE International Energy Conference (ENERGYCON)
Limassol, Cyprus,

STAMINA - WASP

Wallenberg AI, Autonomous Systems and Software Program, 2016-04-04 -- 2020-04-06.

Integrated cyber-physical solutions for intelligent distribution grid with high penetration of renewables (UNITED-GRID)

European Commission (Horizon 2020), 2017-11-01 -- 2020-04-30.

STAMINA - GE

Göteborg Energi, Foundation for Research and Developmen, 2017-01-01 -- 2021-12-31.

Subject Categories

Computer Engineering

Computer Science

Computer Systems

Areas of Advance

Energy

DOI

10.1109/ENERGYCON.2018.8398800

More information

Latest update

7/2/2019 2