Function-as-a-Service Performance Evaluation: A Multivocal Literature Review
Artikel i vetenskaplig tidskrift, 2020

Function-as-a-Service (FaaS) is one form of the serverless cloud computing paradigm and is defined through FaaS platforms (e.g., AWS Lambda) executing event-triggered code snippets (i.e., functions). Many studies that empirically evaluate the performance of such FaaS platforms have started to appear but we are currently lacking a comprehensive understanding of the overall domain. To address this gap, we conducted a multivocal literature review (MLR) covering 112 studies from academic (51) and grey (61) literature. We find that existing work mainly studies the AWS Lambda platform and focuses on micro-benchmarks using simple functions to measure CPU speed and FaaS platform overhead (i.e., container cold starts). Further, we discover a mismatch between academic and industrial sources on tested platform configurations, find that function triggers remain insufficiently studied, and identify HTTP API gateways and cloud storages as the most used external service integrations. Following existing guidelines on experimentation in cloud systems, we discover many flaws threatening the reproducibility of experiments presented in the surveyed studies. We conclude with a discussion of gaps in literature and highlight methodological suggestions that may serve to improve future FaaS performance evaluation studies.

Multivocal Literature Review

Cloud Computing

Benchmarking

Serverless

Performance

Function-as-a-Service

Författare

Joel Scheuner

Chalmers, Data- och informationsteknik, Software Engineering

Philipp Leitner

Chalmers, Data- och informationsteknik, Software Engineering

Journal of Systems and Software

0164-1212 (ISSN)

Vol. 170 110708

Styrkeområden

Informations- och kommunikationsteknik

Ämneskategorier

Programvaruteknik

Datavetenskap (datalogi)

DOI

10.1016/j.jss.2020.110708

Relaterade dataset

Replication Package for "Function-as-a-Service Performance Evaluation: A Multivocal Literature Review" [dataset]

DOI: 10.5281/zenodo.3906613

Mer information

Senast uppdaterat

2020-09-22