Scaling Kubernetes on demand

Keda and Keda HTTP Add-On

Posted by eumel8 on November 05, 2023 · 14 mins read

Sustainable computing

Climate change is on everyone’s lips. Meanwhile, the world’s data centers are happily humming and humming along. We will address the topic of sustainable computing in the next article. First of all, this is about scaling on demand, so our workload in the Kubernetes cluster is scaled as needed using the Horizontal Pod Autoscaler from 1 to infinity. But what if that 0 to infinity were possible?


Keda - Kubernetes Event Driven Autoscaling. A pithy term and just as brilliant. I installed my workload in the cluster and scaled the deployment to 0. Everything is “ready to go”. The go then comes from an event, old school would now be a cron job that scales the workload up at 8 a.m. and down again at 6 p.m. Anyway. But what would be the scenario of a service that is rarely used, such as a website that someone only visits once in a while, like this one? Okay, let’s ignore all the spam and bots, we can worry about those later. Maybe one or two visitors come by here during the day. And for them we open our store by starting a pod after the first request in the browser, which houses an Nginx web server, which then delivers this content here and shows it to the visitor in the browser with a short delay. When the page is loaded, Keda waits a while and scales the deployment back to 0. So we save computer resources and electricity and thus protect the environment.


Our Kubernetes cluster is a K3S that manages itself with Rancher. K3S comes with Traefik as an ingress controller as standard. We have to make a few adjustments there:

kubectl -n kube-system edit traefik

We add the following lines:

   valuesContent: |-
         allowemptyservices: true
         allowExternalNameServices: true
         allowemptyservices: true
         allowExternalNameServices: true

We’ll see why in a moment.

Install Keda and Keda HTTP Addon

One thing in advance: Keda is very clean and structured. In the Documentation you can quickly find the options for installing Keda. The documentation is also structured according to standards for technical writing: it goes from easy steps to complicated ones, superficial and general descriptions lead to a lot of depth, such as the detailed description of all Helm Chart parameters. Something like this is rare and shows a lot of love for the project. Thanks to the high security standards, hardly any adjustments are necessary. We can start with the Helm installation:

helm repo add keda
helm repo update
helm upgrade -i keda keda/keda --namespace keda --create-namespace --set rbac.aggregateToDefaultRoles=true
helm upgrade -i http-add-on keda/keda-add-ons-http --namespace keda --set rbac.aggregateToDefaultRoles=true

Here we have decided on a cluster-wide installation. The CRDs and cluster-wide RBAC are installed, with the rights being aggregated to the respective default users such as admin. Project owners can then later manage Keda for their app in their Rancher project. Another option would be the Keda operator or namespaced installation. However, we don’t want to burden the user with managing the Kedas components. He wants it to be as simple as possible, hence this approach here.

Demo app & demo user

The demo user owns a project in Rancher and is the project owner there. In the demoapp project we create a demoapp namespace.

The app we use is a demo app, let’s use the Flask app here. We install this into the demoapp namespace.

kubectl -n demoapp apply -f
kubectl -n demoapp apply -f

Now all we need is one ingress and our app would be accessible to the world. But let’s look at the Keda architectural picture again:

Between Ingress and Service there is the Intereceptor through which we have to pass the traffic. Our smuggler is called relink and is deployed in the demoapp namespace:

cat <<EOF | kubectl -n demoapp apply -f -
apiVersion: v1
kind: Service
   name: relink
   type: ExternalName
   externalName: keda-add-ons-http-interceptor-proxy.keda.svc.cluster.local

The interceptor service runs in a different namespace. This service redirects traffic there. The Ingress now has relink as backend:

cat <<EOF | kubectl -n demoapp apply -f -
kind: Ingress
   name: demoapp
   ingressClassName: traefik
   - host:
       - backend:
             name: relink
               number: 8080
         path: /
         pathType: Prefix

In order for ExternalName to be accepted as an ingress endpoint by Traefik, the changes to Traefik’s Helmchartconfig were necessary at the beginning.


After our last step, our workload ends up somewhere in nirvana. It is not transparent to the project owner where he is sending the traffic. Of course, consultation with the cluster owner is necessary here.

Now to tackle scaling, we need an HTTPScaledObject:

cat <<EOF | kubectl -n demoapp apply -f -
kind: HTTPScaledObject
     name: demoapp
     scaledown period: 10
         deployment: demoapp
         service: demo app
         port: 80
         min: 0
         max: 10
     targetPendingRequests: 1

So the Keda HTTP Interceptor should forward our traffic to the demoapp service in our demoapp namespace. The minimum replica is 0, so no pods of the app are running:

$ kubectl -n demoapp get deployments.apps demoapp
demoapp   0/0     0            0           23h

We can also query the status of the Keda and it looks something like this:

kubectl -n demoapp describe demoapp
     Message: Identified HTTPScaledObject creation signal
     Reason: PendingCreation
     Status: Unknown
     Timestamp: 2023-11-04T16:42:14Z
     Type: Pending
     Message: App ScaledObject created
     Reason: AppScaledObjectCreated
     Status: True
     Timestamp: 2023-11-04T16:42:14Z
     Type: Created
     Message: Finished object creation
     Reason: HTTPScaledObjectIsReady
     Status: True
     Timestamp: 2023-11-04T16:42:14Z
     Type: Ready
Events: <none>

If we now open our demo app, it will take a moment and then the app will be available:

$ curl
You requested: a

We set the inactive timeout to 10 seconds. So the app is asleep again before we can look in the event log to see what happened:

$ kubectl -n demoapp get events -w=1
LAST SEEN   TYPE     REASON                       OBJECT                          MESSAGE
77s         Normal   KEDAScaleTargetActivated     scaledobject/demoapp            Scaled apps/v1.Deployment demoapp/demoapp from 0 to 1
77s         Normal   ScalingReplicaSet            deployment/demoapp              Scaled up replica set demoapp-6b6f4bc684 to 1 from 0
77s         Normal   SuccessfulCreate             replicaset/demoapp-6b6f4bc684   Created pod: demoapp-6b6f4bc684-s4gng
77s         Normal   Scheduled                    pod/demoapp-6b6f4bc684-s4gng    Successfully assigned demoapp/demoapp-6b6f4bc684-s4gng to k3s-test-server-2
77s         Normal   Pulled                       pod/demoapp-6b6f4bc684-s4gng    Container image "" already present on machine
77s         Normal   Created                      pod/demoapp-6b6f4bc684-s4gng    Created container demoapp
77s         Normal   Started                      pod/demoapp-6b6f4bc684-s4gng    Started container demoapp
66s         Normal   KEDAScaleTargetDeactivated   scaledobject/demoapp            Deactivated apps/v1.Deployment demoapp/demoapp from 1 to 0
66s         Normal   ScalingReplicaSet            deployment/demoapp              Scaled down replica set demoapp-6b6f4bc684 to 0 from 1
66s         Normal   SuccessfulDelete             replicaset/demoapp-6b6f4bc684   Deleted pod: demoapp-6b6f4bc684-s4gng
66s         Normal   Killing                      pod/demoapp-6b6f4bc684-s4gng    Stopping container demoapp

That’s it! The concept is adapted from Idle Instances from Openshift.

However, there are many other options for scaling. Also worth mentioning:


kind: ScaledObject
   name: demo-keda-scaledobject
     apiVersion: apps/v1
     kind: Deployment
     name: demoapp
   pollingInterval: 10 # Optional. Default: 30 seconds
   cooldownPeriod: 300 # Optional. Default: 300 seconds
   minReplicaCount: 0 # Optional. Default: 0
   maxReplicaCount: 6 # Optional. Default: 100
   fallback: # Optional. Section to specify fallback options
     failureThreshold: 3 # Mandatory if fallback section is included
     replicas: 1
   advanced: # Optional. Section to specify advanced options
     horizontalPodAutoscalerConfig: # Optional. Section to specify HPA related options
       behavior: # Optional. Use to modify HPA's scaling behavior
           stabilizationWindowSeconds: 150
             - type: Percent
               value: 100
               periodSeconds: 15
     - type: prometheus
         serverAddress: http://prometheus-operated.demoapp:9090/
         metricName: flask_http_request_duration_seconds # Note: name to identify the metric, generated value would be `prometheus-http_requests_total`
         query: sum(rate(flask_http_request_duration_seconds_count{path="a"}[1m])) # Note: query must return a vector/scalar single element response
         threshold: '1'
         # Optional fields:
         ignoreNullValues: "true" # Default is `true`, which means ignoring the empty value list from Prometheus. Set to `false` the scaler will return error when Prometheus target is lost

In this example, a monitoring instance with Prometheus is still running. We install another ServiceMonitor:

kubectl -n demoapp apply -f

and could then control our scaling with PromQL queries. Of course it doesn’t work with the Flask metrics, because our Flask app would have to run at least once. But it may be suitable for larger applications where, for example, a smaller part runs permanently and the large Java container is only started for certain requests. This is just an idea.


Our on-demand scaling now works fine. If we let it loose on the Internet, it would hardly be able to calm down because there are tons of bots out there trying to spy on our app.

Here are a few ideas to curb traffic:

Traefik offers the Middleware resource to whitelist IPs:

kind: middleware
   name: test-ipwhitelist

Blacklists work the other way around:

kind: middleware
   name: test-ipwhitelist

A deeper possibility is an application firewall in front of the ingress controller. Or you can integrate dynamic spam lists (DNBL) in the Nginx of the ingress controller.


Scaling on demand may only be a small contribution to protecting the environment and your wallet if you have to pay for CPU and memory. But it is a start and thanks to the excellent Keda project it is also possible in open source.