Demonstrating the Benefits of Service-Aware Pod Autoscaling with Shared Resources

Paper i proceeding, 2023

Service providers can leverage shared resources to reduce the overall amount of required resources while keeping acceptable Quality of Service (QoS) levels. Kubernetes (K8s) provides a Horizontal Pod Autoscaling (HPA) mechanism that allows to automatically adjust the number of Pods to closely follow the user demand variations over time. To properly leverage shared resources with HPA, service providers need to limit the use of dedicated resources and overprovisioning. However, in the case of traffic spikes, there may not be enough resources to satisfy the demand. The HPA, which relies on resource usage to drive the scaling, is unaware of how many requests could not be served with the required QoS. This might result in an underestimation of the number of required Pods to be added, leading to additional QoS degradation. This demonstration showcases the effectiveness of a new Pod autoscaling mechanism (i.e., Service Aware Pod Autoscaling (SAPA)) that relies on user request measurements from the service load balancer to better estimate the number of required Pods. SAPA allows selecting the amount of Pod resources (dedicated and shared) in a simple way. We demonstrate the benefits of SAPA by comparing it to a K8s cluster based on the traditional HPA in terms of resource usage and service latency.