Technical Articles
Scale your Applications using Custom Metrics Feature of Application Autoscaler
- Is your application unable to cope with large amounts of requests during peak hours?
- Have you seen a decrease in application performance in high-traffic conditions?
- Are you looking to reduce your runtime costs when there is less traffic?
- Do you want to scale your application based on your custom metrics (other than CPU, memory, throughput, reponsetime) ?
Application Autoscaler automatically adds or removes application instances/nodes based on the workload. This can be done by defining an autoscaling policy, containing scaling rules.
In this guide, we will demonstrate how one can use the custom metrics feature (released in app-autoscaler version 3.0.0) using mTLS authentication,
Application Autoscaler Metrics
Let’s quickly discuss the available metrics that Application Autoscaler supports (as of release 9.3.0)
| Metric | Type | Technical Metric Name |
| Memory Used | standard | memoryused |
| Memory Utilization | standard | memoryutil |
| CPU Utilization | standard | cpu |
| Throughput | standard | throughput |
| Response Time | standard | responsetime |
| Custom Metrics | user-defined | arbitrary |
| Refer to the Application Autoscaler User Guide to learn more about the metrics | ||
Besides standard metrics, which are mostly available in the container (where the app is running), Application Autoscaler also provides applications to scale out or scale back in based on user-defined metrics types aka. custom metrics. This makes it possible to enhance application performance.
We will use a sample Golang application, deployed on Cloud Foundry which uses the Application Autoscaler custom metrics feature using mTLS authentication. The application can be downloaded from the GitHub repo.
For instance, the application should scale out (add one more instance) if the server is too busy (HTTP 503 Status code) and then scale back in (remove the added instance) if the service is responding with HTTP 200 (normal state).
Custom Metrics Using MTLS
Mutual Transport Layer Security (mTLS) is a certificate-based authentication mechanism, in which both client and server communicate by exchanging x.509 digital certificates. This enables trust and secure communication between them.
Example Scenario
In order to send custom metrics by a cf application to the Application Autoscaler service using mTLS, the following are the high-level steps that need to be performed in the cf application
- Read the app identity certificate and key pair (stored in the app container)
- Create a CA certificate pool and supply certificate to it
- Create an HTTP client and provide the CA certificate pool and certificate
- Send HTTP POST request to autoscaler
The above steps are already implemented in our sample Golang application. Now we have to define our scaling policy which Application Autoscaler uses to perform scaling operations.
Scaling Policy
Our sample Golang application should scale out (add one more instance) if the server is too busy. (HTTP 503 Status code) and then scale in (remove the added instance) if the service responds with HTTP 200.
Prerequisites
Following are the environment and application-specific requirements for this guide.
- BTP account is required. If you don’t have one, try the free trial account
- Cloud Foundry Runtime entitlement is required
- Application Autoscaler entitlement is pre-configured
- CF org and space should be available in the Cloud Foundry environment
- CF CLI is installed locally to interact with the Cloud Foundry environment
- Golang application having RESTful API. If you don’t have one, try the sample application from GitHub repo
- Following Autoscaling policy is required
{
"instance_min_count": 1,
"instance_max_count": 2,
"scaling_rules": [
{
"metric_type": "tooManyRequestCustomMetrics",
"breach_duration_secs": 60,
"threshold": 200,
"operator": ">",
"cool_down_secs": 60,
"adjustment": "+1"
},
{
"metric_type": "tooManyRequestCustomMetrics",
"breach_duration_secs": 60,
"threshold": 200,
"operator": "<",
"cool_down_secs": 60,
"adjustment": "-1"
}
]
}custom-metrics-policy.json
Let’s understand the Application Autoscaler scaling policy
- Scale-out
- if the metric tooManyRequestCustomMetrics is greater than 200 and it remains greater than 200 for 60 seconds, Autoscaler should perform the scale out (add one more instance/node) of the app.
- Scale-in
- Similarly, scale in (remove one instance) of an app if the metric tooManyRequestCustomMetrics is less than 200 for 60 seconds. The app instance count remains 1 due to “instance_min_count”: 1.
Walkthrough
Target CF Org and Space
Login to cf and target the cf organization and space, where you want to deploy the application
cf target -o ak_autoscaler_dev -s ak-test-space
Push Golang Application
You need to clone the repository first (if you haven’t yet)
# Clone the Golang Application git repository
git clone https://github.com/SAP-samples/cf-autoscaler-custom-metrics-mtls.git
# Navigate to the directory
cd golang-autoscaler-custom-metrics
# Build application
make build
Let’s push an app to the Cloud Foundry. The following app manifest is used available here
---
applications:
- name: golang-autoscaler-custom-metrics
instances: 1
memory: 128M
disk_quota: 128M
stack: cflinuxfs4
buildpacks:
- binary_buildpack
command: ./golang-autoscaler-custom-metrics
cf push --no-start -f deploy/app-manifest.yml -p deploy/build
Output: cf push an app
Create Autoscaler Service Instance
Now, we want to use the Autoscaler service. So, we create a CF service instance of type autoscaler
cf create-service autoscaler standard ak-test-autoscaler
| Commands | Description |
| autoscaler | Name of the service (available as cf marketplace) |
| standard | Plan type of Autoscaler service |
| ak-test-autoscaler | Name of the service instance |
output: create a service instance
Bind Service Instance with the Golang App
Bind the created ak-test-autoscaler service instance with the Golang app and assign the policy (defined above) as a file
cf bind-service golang-autoscaler-custom-metrics \
ak-test-autoscaler \
-c deploy/custom-metrics-policy.json
The console output will suggest to restage golang-autoscaler-custom-metrics. This is because when a service is bound to an existing app, the VCAP_SERVICES environment variable is populated with the information e.g. credentials information of that service.
output: cf bind service
As soon as we bind the service to the Golang app, we can check the bound Autoscaler service.
TIP – Additionally, Autoscaler plugin can be used to attach the policy. If you have not provided the policy with the cf bind-service command, the following can also be used to attach the policy to the app.
cf attach-autoscaling-policy \
golang-autoscaler-custom-metrics \
deploy/custom-metrics-policy.json
Let’s have a look at VCAP_SERVICES environment variable
$ cf env golang-autoscaler-custom-metrics
output: cf env app
In the output, we can see the custom metric mtls_url under the credentials section. Our Golang app uses mtls_url to submit tooManyRequestCustomMetrics custom metric to Autoscaler service.
By using mtls_url, the Golang app uses the certificate-based authentication (mTLS), provided by Autoscaler Custom Metrics feature.
Start Golang App
Start Golang app to consider binding changes
cf start golang-autoscaler-custom-metrics
output: cf start app
Now, our Golang app is running with one instance. We can perform scaling through the Autoscaler service which is waiting to receive custom metrics from the Golang app.
Verify App Scale-Out
We now have to send custom metrics to Autoscaler using the HTTP GET endpoint (provided by the app).
– Grep the route of the app
cf app golang-autoscaler-custom-metrics | grep routes
– Using Postman/Curl, perform the following
$ curl https://golang-autoscaler-custom-metrics.cfapps.sap.hana.ondemand.com/busy/201
{"message":"I am busy with value 201.000000"}%
By running cf app golang-autoscaler-custom-metrics again, we now have two instances of our Golang App (after 60-80 seconds)
output: scale out (2 Instances)
Verify App Scale-In
Now, we want our application to scale in if the requests are below 200 (threshold). Similarly, we also have an HTTP GET endpoint to decrease the instance via curl/postman (whichever you prefer)
$ curl https://golang-autoscaler-custom-metrics.cfapps.sap.hana.ondemand.com/not-busy/190
{"message":"I am not busy with value 190.000000"}%
We now have one instance of our Golang App as required
output: scale in (1 Instance)
Monitor Scaling Events
Sometimes it is important to know how and when scaling events happen. This can be find out using the following three ways
App Autoscaler CLI Plugin
Install the plugin — cf install-plugin -r CF-Community app-autoscaler-plugin
Read more about the CF App Autoscaler CLI Plugin
$ cf autoscaling-history golang-autoscaler-custom-metrics
Retrieving scaling event history for app golang-autoscaler-custom-metrics...
Scaling Type Status Instance Changes Time Action Error
dynamic succeeded 2->1 2022-12-30T16:38:08+01:00 -1 instance(s) because tooManyRequestCustomMetrics <= 200test-unit for 60 seconds
dynamic succeeded 1->2 2022-12-30T16:18:08+01:00 +1 instance(s) because tooManyRequestCustomMetrics >= 200test-unit for 60 seconds
CF App Events
Cloud Foundry provides viewing events of an app. App events can be created, upload, scaled, and so on. For more information on auditing capabilities in Cloud Foundry, have a look at Audit Event on Cloud Foundry documentation.
However, we are only interested to view app scaling events at the moment.
cf events golang-autoscaler-custom-metrics
Getting events for app golang-autoscaler-custom-metrics in org ak_autoscaler_dev / space ak-test-space as ak-user@abc.com...
time event actor description
2022-12-30T16:38:08.00+0100 audit.app.process.scale instances: 1
2022-12-30T16:18:08.00+0100 audit.app.process.scale instances: 2Autoscaler Application Dashboard
Autoscaler Application Dashboard is a CF app available to SAP customers that comes out of the box when you use the Application Autoscaler service.
Conclusion
- We created a sample Golang app that sends metrics to the Autoscaler service.
- We learned how to use mTLS-based authentication to send custom metrics.
- The application was scaled out and scaled in based on the metrics sent by the Golang app.
- Additionally, we also learned how to monitor scaling events using different methods and tools.