SAP Event Mesh supports mTLS [2]: How to adjust my application

This tutorial is about SAP Event Mesh (aka SAP Enterprise Messaging) running on SAP Business Technology Platform (SAP BTP, aka SAP Cloud Platform) in Cloud Foundry environment.
We learn how to use mTLS instead of basic authentication, when our application calls the Event Mesh REST API.

Quicklinks:
Quick Steps
Sample Code

Content

0.   Prerequisites
00. Preparation
1.   Standard App
2.   Adjust to mTLS
2.1. Rebind Service
2.2. Redeploy with adjusted manifest.yml
2.3. Update Service Instance
3.   More Info
3.1. Validity/Expiration
3.2. JWT
A    Appendix: Project Files
1. Standard App
2.1. Rebind
2.2. Adjusted Manifest
2.3. Update Service
3. Native Code

Introduction

In the previous blog post we’ve learned what’s all about it:
SAP Event Mesh offer a REST API and it provides the necessary credentials for using it.
These credentials are used to follow the OAuth flow: to fetch a JWT token which is required for using the REST API.
To fetch a JWT token, we have to authenticate, by sending user/password, or more precisely, clientid and clientsecret (basic authentication).
After switching to mTLS, we authenticate with a client certificate which Event Mesh provides to us instead of the clientsecret.

In the previous blog post, we’ve learned how to handle this authentication mechanism in a manual request, using Postman as REST client tool.
Today we’re creating a sample application which calls the REST API and is supposed to follow the mTLS approach.

We’re describing a few scenarios which all start from a standard deployment and move to mTLS. They differ in the way how the adjustment is done.

0. Prerequisites

The previous blog post with all its explanations is a recommended reading.
There’s a silly questions’n’answers blog post, recommended only if you aren’t familiar at all with mTLS and credential rotation.
Today we’re creating a very simple sample application based on Node.js, it is very simple, but basic node.js skills are required.
Our cloud deployments and configurations are done with the Cloud Foundry command line client.

00. Preparation

We create a project folder c:\mtls containing following files:

c:\mtls
config-messaging1.json
manifest.yml
package.json
server.js

See Appendix 1 for file content.

1. Standard App

Let’s quickly create a simple app that uses normal OAuth credentials to call REST API of Event Mesh.

1.1. Create Service Instance

We create standard service instance with usual params:

config-messaging1.json

{
  "emname": "mtlsxsappname",
  "namespace": "my/mtls/app",
  . . .
  "options": {
      . . .
  },
  "rules": {
      . . .

And the command:
cf cs enterprise-messaging default mtlsMsg -c config-messaging1.json

1.2.Create application

Our app just performs the easiest use case, it simply uses the REST API to view the existing queues.
Our app is a server app based on Node.js and “express” and we deploy it to Cloud Foundry environment of SAP BTP.

To use the Event Mesh REST API, we need to access the binding information for 2 aspects:
1. We need the base URL of REST API.
2. For authentication, we need to fetch a JWT token, thus we need the credentials.

Before we actually start with the code and the deployment, I give you a preview of how the app environment will look like.

1. The REST URL for managing queues
We get the base URI from path
VCAP_SERVICES.enterprise-messaging.management[0].uri

The uri property gives us the base URL on this landscape and in order to use the REST API, we need to check the documentation for the uris of the different endpoints.
In our example, we have to concatenate the base URI with the segments for reading queues:
/hub/rest/api/v1/management/messaging/queues

2. The credentials of Event Mesh

Event Mesh REST API is protected with OAuth 2.0 so we need to fetch a JWT token, before calling the API.
To fetch a JWT token, we need credentials and the token endpoint, which we find in the binding at following path:
VCAP_SERVICES.enterprise-messaging.uaa

Coming to our application code, we now know which parts of the environment we need to access.
To access the binding itself we use the convenience library @sap/xsenv :

const BINDING = xsenv.getServices({ myMessaging: { tag: 'enterprise-messaging' } })

Above snippet reads the VCAP_SERVICES section and searches for an entry which contains the given value for the tag-property.
The result is assigned as value to the variable which we define with name myMessaging.

To fetch the JWT token, we use the client lib @sap/xssec which comes very convenient, as we will see.

xssec.requests.requestClientCredentialsToken(null, uaa, null, null, (error, token)=>{

Note:
If you prefer using native https module, see Appendix 3.

Once we have the token, we use it to call the REST endpoint for reading queues:

const options = {
   headers: { Authorization: 'Bearer ' + jwtToken }

In brief:
-> Access the binding and feed the xsuaa client lib with it.
-> Access the binding and use the url to read the queues

Below is a collection of snippets:

const BINDING = xsenv.getServices({ myMessaging: { tag: 'enterprise-messaging' } })

app.get('/queues', async (req, res) => {
    const response = await getQueues(BINDING.myMessaging)            


async function getQueues( msgCredentials){   
    // fetch JWT token
    const msgUaa = msgCredentials.uaa 
    const jwtToken = await _fetchJwtToken(msgUaa)

    // fetch queues
    const restUri = msgCredentials.management[0].uri
    const queuesUrl = `${restUri}/hub/rest/api/v1/management/messaging/queues`
    const options = {
        headers: { Authorization: 'Bearer ' + jwtToken }
    }
    const response = await fetch(queuesUrl, options)


async function _fetchJwtToken (uaa){
        xssec.requests.requestClientCredentialsToken(null, uaa, null, null, (error, token)=>{

See Appendix 1 for full content.

I’d like to point your attention to these lines

const uaa = msgCredentials.uaa 
   xssec.requests.requestClientCredentialsToken(null, uaa, null, null, (error, token)=>{

We’re using the client lib for fetching the required JWT token.
We can see that we can directly pass the credentials from the binding to the client lib.
Without even looking at it.
That’s very convenient.
The lib will even take care of appending the /oauth/token segments to the url.
Nice.

1.3. Deploy

After deploy, we can optionally create a queue in the messaging dashboard, just to see any entry in the array of queues.
Anyways, we invoke our endpoint and see the available queues in the browser.
In my example:
https://mtlsapp.cfapps.eu10.hana.ondemand.com/queues

2. Adjust to mTLS

In the screenshot above, we’ve seen the uaa section in our app binding.
It contains the credentials required to fetch the JWT token.
As we’ve seen, the credentials consist of properties clientid and clientsecret, which are used like user/password in basic authentication request.
Now we want to switch to “mutual TLS”, to use client certificate and private key, instead of basic authentication.

How to start?
The required change is in the configuration of the Event Mesh service instance.
In the first chapter, we created the Event Mesh service instance with standard params.
Which means that Event Mesh is configured with standard credential type:
In the environment we see the property “credential-type” being specified as “binding-secret”.
Which is the default.

However, here comes an interesting information:
If nothing is specified by us, then the new instance comes with two implicit credential types:
“binding-secret” and “x509”.

The first one is used as per default.
This means:
In our first chapter we created an instance without specifying any security settings.
As such, implicitly the following settings have been used:

{
  "emname": "mtlsxsappname",
  . . .
  "xs-security": {
    "oauth2-configuration": { 
      "credential-types": [
        "binding-secret", 
        "x509"
      ]
    }
  }
}

With other words:
The “mTLS” is already supported.
We only need to use it.

How to use the “x509” credential type?

As mentioned, the first credential type in the list is used, which means that this kind of credentials will be written into the environment
Now we want the certificate information to be written into the environment, instead of the client secret.

We have 3 possibilities:

1. Select the desired credential type via binding parameters:
We can rebind the service to our app and add the parameters.
2. Specify the desired credential type in the manifest:
During deployment, the parameters are considered for the binding.
3. Update the service instance itself
We can add “xs-security” settings to our config file and update the service instance.

OK, so let’s try it.

2.1. Rebind Service

As mentioned, we want to describe 3 scenarios of switching from normal credential type to mTLS.
We start with the situation as of chapter 1:
We’ve created service without any xs-security related setting in the Event Mesh config.
We’ve deployed the app and we can verify that the credential type is “binding-secret”:
The command cf env mtlsapp shows it:

1. Unbind

Now we want to get rid of the clientid/clientsecret, so we run the following command, to unbind the Event Mesh service instance from our app:
cf unbind-service mtlsapp mtlsMsg

2. Re-bind

Now we want to rebind the service instance and we want to control the binding with a configuration file. This configuration will instruct the service broker to switch to mTLS.
For defining the binding parameters, we create a file called e.g. config-binding.json with content:

{
  "xsuaa": {
    "credential-type": "x509", 
      "x509": {
        "key-length": 4096, 
        "validity": 1, 
        "validity-type": "DAYS"
    }   
  }
}

Explanation of the settings can be found in previous blog post.

Now we rebind the service instance to our app, but we provide the binding params as follows:
cf bind-service mtlsapp mtlsMsg -c config-binding.json

Afterwards we check the app environment again, and this time we see:

Yes, we can see the new credentials-type and also the certificate and key are there.
Although we already can see the new environment has been provided, we need to restage our app, before we can test it:
cf restage mtlsapp

Afterwards, we can invoke our app:
https://mtlsapp.cfapps.eu10.hana.ondemand.com/queues

The endpoint still works fine.
Hey….
Isn’t that remarkable?
It still works fine….
We’ve changed the environment to mTLS and haven’t changed our app and it still works fine!
That might be surprising:
We don’t need to do ANY change in our code.
We read the credentials from binding, and we don’t need to care whether there is a certificate or a clientsecret.
We just pass the credentials to the helper library which takes care of fetching the JWT token.

const uaa = msgCredentials.uaa
xssec.requests.requestClientCredentialsToken(null, uaa, null, null, (error, token)=>{

Note:
We don’t need to adapt the code, because we’re using the xssec node module which supports both types of credentials.
Note:
You just might need to update to current version of xssec

Note:
If you still prefer using native https module, see Appendix 3.

Now we want to have a look into the log: cf logs mtlsapp
There we can see that the cnf claim has been filled:

This property is only available when a client certificate has been used to fetch the JWT token.

2.2. Redeploy with adjusted manifest.yml

Wiring back…
We start again from scratch. Again our starting situation is the one of an innocent user who has created Event Mesh service instance without specifying any xs-security param.
We’ve deployed our app and checked in the env that our credential type is binding-secret.
OK.

Now we want to switch to mTLS, but we don’t want to use the “unbind-service” and “bind-service” commands.
(too lazy)
Instead, we want to define the necessary binding parameters directly in the color manifest.yml file.
This is nice, because the correct binding will be always there when anybody deploys our app.
However, for our demo, it requires that we delete the app, such that the binding can be done from scratch.
Defining binding params in the manifest looks like this:

    services:
      - name: mtlsMsg
        parameters:
          xsuaa:
            credential-type: x509
            x509:
              validity: 1
              validity-type: DAYS

See appendix 2.2 for complete manifest file.

After modifying our manifest, we delete our app cf d -r -f mtlsapp
and deploy again cf push
And then we check the env to see the desired new credential type is “x509”.

2.3. Update Service Instance

This chapter is relevant for older instances which were created before mTLS support was introduced by Event Mesh
In this case we need to use a different approach:
We need to update the service instance, such that it supports the new x509 credential type.
Because it is not implicitly defined.
So we update the service instance and afterwards rebind the app.

A good side effect:
No app downtime.

To reproduce the legacy scenario, we need to create an Event Mesh instance with legacy credential type.
So let’s create a new instance for now, and use a separate config file:

config-messaging-legacy.json

    "xs-security": {
        "oauth2-configuration": { 
            "credential-types": [
                "instance-secret"
            ]
        }
    }

The complete file content can be found in Appendix 2.3.

And the command:
cf cs enterprise-messaging default mtlsMsg -c config-messaging-legacy.json

Afterwards we deploy our app, which never requires any change.

After deploy, we check the env or the log and make sure that the current credential type is “instance-secret”.

OK.
So now we’ve reproduced the legacy situation (instance-secret is legacy, as it doesn’t support credentials rotation)

Now we prepare our new config file for updating the Event Mesh instance:

config-messaging-legacy-update.json

    "xs-security": {
        "oauth2-configuration": { 
            "credential-types": [
                "x509",
                "binding-secret",
                "instance-secret"
            ]

The complete file content can be found in Appendix 2.3.

Note:
We’ve specified all 3 credential types.
Thus we keep our changes backwards compatible.
However, after some time of making sure that it is safe, we can remove the other credential types and keep only x509

The update command:
cf update-service mtlsMsg -c config-messaging-legacyupdate.json

To make the changes effective, we need to rebind the updated service instance.
Thus
unbind: cf us mtlsApp mtlsMsg
and
rebind: cf bs mtlsApp mtlsMsg
Finally we need to restage the app (if we want to check the log).

Note:
Binding parameters are not required, because we’ve put the x509 credential type in the first place.
So this one will be written into the env.

Afterwards we can check the env and we’ll get the desired credential-type as x509 and we get the certificate and key.

3. Optional: More Info

Let’s briefly have a look at some interesting stuff.

3.1. Validity/Expiration

Once the certificate has expired, or better, before it expires, we have to  rebind the service to get a new certificate.

The validity can be checked with the following command:
openssl x509 -in cert.txt -noout -enddate

The result is the end date of the validity period of the certificate cert.txt
The command for printing the whole decoded certificate:
openssl x509 -noout -text -in cert.txt

If you see this error X509_V_ERR_CERT_HAS_EXPIRED then you’re too late and you don’t need to check anymore. Just rebind and good.

The openssl tool is availble in the Git Bash, or can be downloaded.

To control the validity we can modify the binding parameters:

         "validity": 3, 
         "validity-type": "YEARS"

3.1. JWT Token

In case of mTLS, the issed JWT contains a property called cnf.
With better words: The JWT contains a cnf claim.
We’ve seen it already in the log (in the first chapter it was empty) and we’ve wondered what it means.

cnf stands for “confirmation” and is used as “proof of possession”.
Means:
When we requested the JWT token, we’ve sent a client certificate for authentication.
And now this token contains an information about that certificate.
With other words: the JWT is bound to a certificate.
This needs to be confirmed.
To do so, there are several methods.
One method is called x5t#S256 (as printed in the screenshot)
Sounds crazy…
Let’s try to break it down.

To bind a JWT to a certificate, it is obviously not desired to copy the whole certificate into the cnf claim.
Instead, the hash of the certificate is used.
Such hash is also called thumbprint or fingerprint or digest.
This thumbprint of the x509 certificate (-> x5t) is created with “SHA-256” algorithm (-> S256)
So now the name of the confirmation method sounds less crazy.

The value of this x5t… property is the thumbprint of the client certificate.
The receiver of the token has to validate the token and it will use the thumbprint found in the cnf claim.

Summary

We already knew that Event Mesh can be configured to support mTLS for delivering JWT tokens.
Today we’ve learned how to deal with it in the context of a deployed application.

– In new scenarios, for switching to mTLS, we only need to rebind Event Mesh service and provide binding params.
– In new scenarios, for switching to mTLS, we only need to provide binding params in manifest.yml
– In legacy scenarios, we need to update the service instance and provide credential type as x509 in config file.

Application code doesn’t need to be changed, if xssec lib is used.
The client lib takes care of retrieving the certificate and private key and configure the token fetch request properly

Quick Steps

Switch to mTLS via binding params

{
  "xsuaa": {
    "credential-type": "x509", 
      "x509": {
        "key-length": 4096, 
        "validity": 7, 
        "validity-type": "DAYS"
    }   
  }
}

cf us mtlsapp mtlsMsg
cf bs mtlsapp mtlsMsg -c config-binding.json
cf restage mtlsapp

To enable mTLS in manifest.yml

    services:
      - name: mtlsMsg
        parameters:
          xsuaa:
            credential-type: x509
            x509:
              validity: 7
              validity-type: DAYS

Links

SAP Event Mesh landing page.
Event Mesh docu about JSON params.
Event Mesh documentation about REST API.
Specification RFC about PEM and representation format in single lines.
Openssl documentation.
Postman documentation for working with certificates.
Security Glossary.

Appendix 1: Project Files for Standard App

config-messaging1.json

{
  "emname": "mtlsxsappname",
  "namespace": "my/mtls/app",
  "version": "1.1.0",
  "options": {
      "management": true,
      "messagingrest": true,
      "messaging": true
  },
  "rules": {
      "queueRules": {
          "publishFilter": [
              "${namespace}/*"
          ],
          "subscribeFilter": [
              "${namespace}/*"
          ]
      },
      "topicRules": {
          "publishFilter": [
              "${namespace}/*"
          ],
          "subscribeFilter": [
              "${namespace}/*"
          ]
      }
  }
}

manifest.yml

---
applications:
  - name: mtlsapp
    memory: 128M
    routes:
    - route: mtlsapp.cfapps.eu10.hana.ondemand.com
    services:
    - mtlsMsg

package.json

{
  "dependencies": {
    "express": "^4.17.1",
    "passport": "^0.4.0",
    "@sap/xsenv": "latest",
    "@sap/xssec": "latest",
    "node-fetch": "2.6.2"
  }
}

server.js

const fetch = require('node-fetch')
const xssec = require('@sap/xssec')
const xsenv = require('@sap/xsenv')
const express = require('express')
const app = express()

const BINDING = xsenv.getServices({ myMessaging: { tag: 'enterprise-messaging' } })

/* App Server */
app.listen(process.env.PORT, function () { console.log('===> Server started') })

/* App Endpoints */
app.get('/queues', async (req, res) => {
    const response = await getQueues(BINDING.myMessaging)            
    res.send(`Found following queues: '${response}'`)
})

/*  Helper */

async function getQueues( msgCredentials){   
    // fetch JWT token
    const uaa = msgCredentials.uaa 
    const jwtToken = await _fetchJwtToken(uaa)

    // fetch queues
    const restUri = msgCredentials.management[0].uri
    const queuesUrl = `${restUri}/hub/rest/api/v1/management/messaging/queues`
    const options = {
        headers: { Authorization: 'Bearer ' + jwtToken }
    }
    const response = await fetch(queuesUrl, options)
    return await response.text() 
}

async function _fetchJwtToken (uaa){
    return new Promise ((resolve, reject) => {
        console.log(`===>[mtlsApp] credential type: ${uaa['credential-type']}`)
        xssec.requests.requestClientCredentialsToken(null, uaa, null, null, (error, token)=>{            
            const tokenInfo = new xssec.TokenInfo(token)
            console.log(`===>[mtlsApp] cnf: ${JSON.stringify(tokenInfo.getPayload().cnf)}`)
            resolve(token)
        })
    })  
}

Appendix 2.1: Rebind

config-binding.json

{
  "xsuaa": {
    "credential-type": "x509", 
      "x509": {
        "key-length": 4096, 
        "validity": 1, 
        "validity-type": "DAYS"
    }   
  }
}

Appendix 2.2: Adjusted Manifest

manifest.yml

---
applications:
  - name: mtlsapp
    memory: 128M
    routes:
    - route: mtlsapp.cfapps.eu10.hana.ondemand.com
    services:
      - name: mtlsMsg
        parameters:
          xsuaa:
            credential-type: x509
            x509:
              validity: 1
              validity-type: DAYS

Appendix 2.3: Update Service

config-messaging-legacy.json

{
  "emname": "mtlsxsappname",
  "namespace": "my/mtls/app",
  "version": "1.1.0",
  "options": {
      "management": true,
      "messagingrest": true,
      "messaging": true
  },
  "rules": {
      "queueRules": {
          "publishFilter": [
              "${namespace}/*"
          ],
          "subscribeFilter": [
              "${namespace}/*"
          ]
      },
      "topicRules": {
          "publishFilter": [
              "${namespace}/*"
          ],
          "subscribeFilter": [
              "${namespace}/*"
          ]
      }
  },
    "xs-security": {
        "oauth2-configuration": { 
            "credential-types": [
                "instance-secret"
            ]
        }
    }
}

config-messaging-legacy-update.json

{
  "emname": "mtlsxsappname",
  "namespace": "my/mtls/app",
  "version": "1.1.0",
  "options": {
      "management": true,
      "messagingrest": true,
      "messaging": true
  },
  "rules": {
      "queueRules": {
          "publishFilter": [
              "${namespace}/*"
          ],
          "subscribeFilter": [
              "${namespace}/*"
          ]
      },
      "topicRules": {
          "publishFilter": [
              "${namespace}/*"
          ],
          "subscribeFilter": [
              "${namespace}/*"
          ]
      }
  },
    "xs-security": {
        "oauth2-configuration": { 
            "credential-types": [
                "x509",
                "binding-secret",
                "instance-secret"
            ]
        }
    }
}

Appendix 3.: Native Code

server.js

async function _fetchJwtToken (uaa){
    return new Promise ((resolve, reject) => {
        const https = require('https')  

        const options = {
            cert:  '-----BEGIN CERTIFICATE-----\nMIIFuzCCA6OgAwIBAgIRAP+ZJ3u/ktIKjlf1Xtaqb/swDQYJKoZIhvcNAQELBQAweTELMAkGA1UEBhMCREUxDTALBgNVBAcMBEVVM. . . xnLq83g==\n-----END CERTIFICATE-----\n-----BEGIN CERTIFICATE-----\nMIIGYDCCBEigAwIBAgITcAAAAAR7yX5CNr+FlgAAAAAABDANBgkqhkiG9w0BAQsFADBNMQswCQYDVQQGEwJERTERMA8GA1UEBwwIV. . . b4MDZWYksZY/0+6nRvEvg==\n-----END CERTIFICATE-----\n-----BEGIN CERTIFICATE-----\nMIIFZjCCA06gAwIBAgIQGHcPvmUGa79M6pM42bGFYjANBgkqhkiG9w0BAQsFADBNMQswCQYDVQQGEwJERTERMA8GA1UEBwwIV2Fsb. . . va5Kbng/u20u5ylIQK8fcIaUp7MVBIVZ\n-----END CERTIFICATE-----',
            key: '-----BEGIN RSA PRIVATE KEY-----\nMIIEpAIBAAKCAQEA23DStxj7PemUT8WpiRjgrXtQFvY/zavi3KrtnTx++hLZRwAlb1VqeoC8kakdAT58zEhAaU4Us35//MQYxyaFB52L/RlgRK+zCS8z2dlV/9g6DOGy5xFF0M++3w5c2AA/pzEw==\n-----END RSA PRIVATE KEY-----',
            host: uaa.certurl.replace('https://', ''),
            path: '/oauth/token',
            method: 'POST',
            headers: {
                'Content-Type': 'application/x-www-form-urlencoded'
            }
        }

        const data = `client_id=${uaa.clientid}&grant_type=client_credentials&response_type=token`
        const req = https.request(options, (res) => {
            let response = ''
            res.on('data', (chunk) => {
                response += chunk
            })
            res.on('end', ()=>{
                return resolve(JSON.parse(response).access_token)
            })
         })
               
         req.write(data)
         req.end()           
    })
}