EPypes: a framework for building event-driven data processing pipelines
Journal article, 2019

Many data processing systems are naturally modeled as pipelines, where data flows though a network of computational procedures. This representation is particularly suitable for computer vision algorithms, which in most cases possess complex logic and a big number of parameters to tune. In addition, online vision systems, such as those in the industrial automation context, have to communicate with other distributed nodes. When developing a vision system, one normally proceeds from ad hoc experimentation and prototyping to highly structured system integration. The early stages of this continuum are characterized with the challenges of developing a feasible algorithm, while the latter deal with composing the vision function with other components in a networked environment. In between, one strives to manage the complexity of the developed system, as well as to preserve existing knowledge. To tackle these challenges, this paper presents EPypes, an architecture and Python-based software framework for developing vision algorithms in a form of computational graphs and their integration with distributed systems based on publish-subscribe communication. EPypes facilitates flexibility of algorithm prototyping, as well as provides a structured approach to managing algorithm logic and exposing the developed pipelines as a part of online systems.

Robotics

Distributed systems

Event-driven systems

Computational graph

Python

Algorithm development

Pipeline

Concurrency

Computer vision

Publish-subscribe

Author

Oleksandr Semeniuta

Norwegian University of Science and Technology (NTNU)

Petter Falkman

Chalmers, Electrical Engineering, Systems and control, Automation

PEERJ COMPUTER SCIENCE

2376-5992 (ISSN)

e176

Subject Categories

Embedded Systems

Computer Science

Computer Systems

DOI

10.7717/peerj-cs.176

More information

Latest update

11/5/2019