In this post I describe how to scale a microservice by deploying it in multiple apps on SAP BTP, Cloud Foundry runtime. I describe my experience with achieving this using multiple modules in a multitarget application (mtar). SAP BTP, Cloud Foundry runtime allows scaling of apps by adding more instances and more memory. It was possible always to run more apps too. I present a case here for why this might be a simpler solution to some problems.
Introduction
Scaling solutions to meet the demand is an important consideration in microservices design. But when the boundaries of a microservice were decided, this may not have been a consideration. This decision might have been dictated more by team velocity and paved paths from tools. Resource and scaling requirements of workloads for a microservice may not have been even apparent at the time the decision was made. It might not have been a deliberate choice or the choice may not have been apparent. Whatever the reason, is restructuring the microservice the only choice now?
Context
If you look at how to scaling a microservice that is deployed as an app in SAP BTP, Cloud Foundry runtime the obvious choices are memory which you can increase up to 8 GB (maximum instance memory per application) and the number of instances. Allocated CPU scales linearly with the allocated memory up to 2 CPUs for 8 GB memory. Depending on if the process is memory bound or CPU bound, you can decide on the memory required to handle the expected load and split this across a number of instances (minimum is two to have high availability). These choices apply for an application in SAP BTP Cloud Foundry, runtime. But you have workloads in a microservice which is deployed as an application that have different scaling needs for these dimensions. Some need a lot of memory but wouldn't benefit from increased number of instances. Others need very little memory per request but receive a high number of requests; better to have more instances. Some may need a lot of CPU but may be needed only at specific periods.
Solution
It was always possible to deploy the same build results for a microservice as several applications.
a micro service deployed as multiple applications
Here is an example of deploying the same same build results as multiple apps:
cf push cf-bl-app-1 -path gen/srv --no-route
# We will explcitly bind to nicely named route
cf push cf-bl-app-2 -path gen/srv --no-route
# We will explcitly bind to nicely named routeNote that deployment is usually described in an App manifest or a Multitarget Application Development Descriptor where scaling and routes are also specified. I am splitting these into multiple commands for illustration. There are commands for copying package of an app to another. I let these pass too for simplicity in illustration.
Then each application can be scaled independently.
cf scale cf-bl-app-1 -i 10 -k 1G -m 1G
# -i Number of instances
# -k Disk limit (e.g. 256M, 1024M, 1G)
# -m Memory limit (e.g. 256M, 1024M, 1G)Each application can be a mapped to a different route and so workloads can be executed on the dedicated application when the clients choose the right host.
cf create-route cfapps.us10.hana.ondemand.com \
--hostname risk-risk-management-mkjy
cf create-route cfapps.us10.hana.ondemand.com \
--hostname mitigation-risk-management-mkjy
cf map-route cf-bl-app-1 cfapps.us10.hana.ondemand.com \
--hostname risk-risk-management-mkjy
cf map-route cf-bl-app-2 cfapps.us10.hana.ondemand.com \
--hostname miti-risk-management-mkjy
For example Application Router configuration could be (for illustrating) as follows:
"routes": [
{
"source": "^/service/risk(.*)$",
"destination": "risk-service"
} ,
{
"source": "^/service/mitigation(.*)$",
"destination": "mitigation-service"
}
]with destinations configured as follows:
destinations:
[
{"name":"risk-service",
"url":"https://risk-risk-management-mkjy.cfapps.us10.hana.ondemand.com"},
{"name":"mitigation-service",
"url":"https://miti-risk-management-mkjy.cfapps.us10.hana.ondemand.com"}
]When different workloads are attached to different paths segments, routes can be created with the same hostname but different paths. The different applications can be mapped to these different routes. For example the commands shown below could be used for this.
cf create-route cfapps.us10.hana.ondemand.com \
--hostname risk-management-mkjy \
--path service/risk
cf create-route cfapps.us10.hana.ondemand.com \
--hostname risk-management-mkjy \
--path service/mitigation
cf map-route cf-bl-app-1 cfapps.us10.hana.ondemand.com \
--hostname risk-management-mkjy \
--path service/risk
cf map-route cf-bl-app-2 cfapps.us10.hana.ondemand.com \
--hostname risk-management-mkjy \
--path service/mitigationThen the clients would not need to know the different hosts to be used for different services. Instead they can use the microservice deployed as multiple applications without the knowledge how it is deployed using the same hostname. This is illustrated with configuration for Application Router as a client below.
"routes": [
{
"source": "^/service/risk(.*)$",
"destination": "business-logic-app"
} ,
{
"source": "^/service/mitigation(.*)$",
"destination": "business-logic-app"
}
]destinations:
[
{"name":"business-logic-app",
"url":"https://risk-management-mkjy.cfapps.us10.hana.ondemand.com"}
]Multitarget Applications
Most applications are deployed as multitarget applications (MTA) in SAP BTP, Cloud Foundry environment. Once deployed, modules in an MTA manifest as applications. We need to have multiple modules in an MTA to have multiple applications in SAP BTP Cloud Foundry, runtime like described earlier. But the modules can share the same content or build result. This is illustrated in the following snippet from an MTA descriptor.
modules:
- name: risk-management-risk
type: nodejs
path: gen/srv
parameters:
buildpack: nodejs_buildpack
command: cds serve RiskService
instances: 2
memory: 112M
disk-quota: 400M
routes:
- route: ${myhostname}/service/risk
requires:
- name: risk-management-db
- name: risk-management-uaa
- name: risk-management-mitigation
type: nodejs
path: gen/srv
parameters:
buildpack: nodejs_buildpack
command: cds serve MitigationService
instances: 4
memory: 96M
disk-quota: 400M
routes:
- route: ${myhostname}/service/mitigation
requires:
- name: risk-management-db
- name: risk-management-uaaBoth the modules refer to the same source (referred to by path) for the deployed content. Each module may have its own command so as to activate only the functionality it serves. This is not mandatory as the requests for other paths are not routed to the app at all. But the modules refers to the same shared resources. Environment variables could be used to have different database pool sizes or control workload specifications for used services if the microservice makes use of these features.
The build process can be optimized by adding build dependencies so that the source code is not built multiple times redundantly.
- name: risk-management-mitigation
....
build-parameters:
requires:
- name: risk-management-risk
builder: custom
commands: []
....Shared Module Binary
Even though we now have multiple modules using the same content, the content is packaged twice (or as many times as there are modules) in the generated archive. It is possible to avoid duplicate content in an MTA archive by sharing the content among modules. Shared Module Binaries is supported in SAP BTP for deploying Multitarget Applications. The support described uses mtad.yaml and MANIFEST.MF files But in a typical workflow, these files are generated by Cloud MTA Build Tool from mta.yaml file. We can achieve the desired result with the following changes to path and build-parameters. Path now points to the previous module and the build-parameters indicate that there is no source to be bundled for this module.
- name: risk-management-mitigation
path: risk-management-risk
...
build-parameters:
no-source: true
requires:
- name: risk-management-risk
...With these changes, even though the generated mtad.yaml file is as specified for shared module binary, the MANIFEST.MF is not. The generated MANIFEST.MF contains an entry like the following for the module that includes the content:
Name: risk-management-risk/data.zip
MTA-Module: risk-management-risk
Content-Type: application/zipIt contains no entries for the modules with no content (no-source: true). We need to correct the above entry to be the following:
Name: risk-management-risk/data.zip
MTA-Module: risk-management-risk, risk-management-mitigation
Content-Type: application/zip
So this workflow, currently, requires post build patching to correct the generated file MANIFEST.MF and enable this feature. I used the below script for the same. It relies on a copy of MANIFEST.MF I changed and saved and its original (MANIFEST_org.MF) before the change in a directory named META-INF.bak.
mtar='risk-management.mtar'
mbt build -t . --mtar $mtar
jar -xvf $mtar META-INF/MANIFEST.MF
diff3 -am META-INF/MANIFEST.MF META-INF.bak/MANIFEST_org.MF \
META-INF.bak/MANIFEST.MF > META-INF/MANIFEST.MF
jar -uMvf $mtar META-INF/MANIFEST.MFThis worked so far. The file MANIFEST.MF does not change unless new modules are added (or deleted or renamed). So one could just have a good copy and update the archive with it.
Conclusion
In this article I showed a method of scaling microservices deployed as applications in SAP BTP Cloud Foundry, runtime that was always possible but not frequently used. I present the case that this is the easiest way to partition workloads in a microservice.
Have you used this in your microservices? How did that work for you. Please let me know in the comments.
- SAP Managed Tags:
