Towards Measuring and Understanding Performance in Infrastructure- and Function-as-a-Service Clouds
Licentiate thesis, 2020

Context. Cloud computing has become the de facto standard for deploying modern software systems, which makes its performance crucial to the efficient functioning of many applications. However, the unabated growth of established cloud services, such as Infrastructure-as-a-Service (IaaS), and the emergence of new services, such as Function-as-a-Service (FaaS), has led to an unprecedented diversity of cloud services with different performance characteristics.

Objective. The goal of this licentiate thesis is to measure and understand performance in IaaS and FaaS clouds. My PhD thesis will extend and leverage this understanding to propose solutions for building performance-optimized FaaS cloud applications.

Method. To achieve this goal, quantitative and qualitative research methods are used, including experimental research, artifact analysis, and literature review.

Findings. The thesis proposes a cloud benchmarking methodology to estimate application performance in IaaS clouds, characterizes typical FaaS applications, identifies gaps in literature on FaaS performance evaluations, and examines the reproducibility of reported FaaS performance experiments. The evaluation of the benchmarking methodology yielded promising results for benchmark-based application performance estimation under selected conditions. Characterizing 89 FaaS applications revealed that they are most commonly used for short-running tasks with low data volume and bursty workloads. The review of 112 FaaS performance studies from academic and industrial sources found a strong focus on a single cloud platform using artificial micro-benchmarks and discovered that the majority of studies do not follow reproducibility principles on cloud experimentation.

Future Work. Future work will propose a suite of application performance benchmarks for FaaS, which is instrumental for evaluating candidate solutions towards building performance-optimized FaaS applications.



Cloud Computing




Opponent: Alessandro Papadopoulos, Mälardalen University, Sweden


Joel Scheuner

Chalmers, Computer Science and Engineering (Chalmers), Software Engineering (Chalmers), Software Engineering for Cyber Physical Systems

A Cloud Benchmark Suite Combining Micro and Applications Benchmarks

ACM/SPEC International Conference on Performance Engineering Companion,; (2018)p. 161-166

Paper in proceedings

Estimating Cloud Application Performance Based on Micro-Benchmark Profiling

2018 IEEE 11th International Conference on Cloud Computing (CLOUD),; (2018)p. 90-97

Paper in proceedings

Serverless Applications: Why, When, and How?

IEEE Software,; Vol. 38(2021)p. 32-39

Journal article

Function-as-a-Service Performance Evaluation: A Multivocal Literature Review

Journal of Systems and Software,; Vol. 170(2020)

Journal article

Transpiling Applications into Optimized Serverless Orchestrations

2019 IEEE 4th International Workshops on Foundations and Applications of Self* Systems (FAS*W),; Vol. June 2019(2019)p. 72-73

Paper in proceedings

Areas of Advance

Information and Communication Technology

Subject Categories

Software Engineering

Computer Science


Chalmers University of Technology


Opponent: Alessandro Papadopoulos, Mälardalen University, Sweden

More information

Latest update