SAP Event Mesh: Sample: Webhook with Security [3]: alternative protection

This tutorial is about SAP Event Mesh (aka SAP Enterprise Messaging) running on SAP Business Technology Platform (SAP BTP, aka SAP Cloud Platform).
We send events to Event Mesh which delivers them via Webhook Subscription. The receiving endpoint is protected with OAuth and scope.

Quicklinks:
Quick Steps
Sample Code
Previous Blog Post

Content

0. Prerequisites
00. Preparation
1. Event Mesh
2. Send Events
3. Receive Events
3.1. Create XSUAA instance
3.2. Create Application
3.3. Webhook Subscription
3.4. Logs
A Sample Code

Overview

This blog post describes an alternative to the previous blog, where the scenario and configurations are explained in detail. So in case of doubts, please refer to that posting.
Today we’re showing an alternative way of securing the webhook endpoint.
In brief:
Instead of configuring the webhook subscription with our own XSUAA credentials, we use the Event Mesh credentials.

If you understand what I mean, you can stop reading and watch cat videos.
Everybody else hopes (like me) to find clarification in below explanations.

Let’s have a look at the 2 scenarios and compare.

In previous tutorial, we did what seemed the most natural way to us:
We protected our endpoint with our XSUAA binding and provided the credentials to Event Mesh.
Also, we defined a scope and assigned the scope to our own OAuth client (xsuaa instance).

The diagram above shows how Event Mesh fetches a JWT token from our own XSUAA.
It also shows that our scope is assigned to our own OAuth client.

Now today, we still protect our endpoint with our own XSUAA binding.
There’s no change in the code of our receiver application.
The difference is that we configure the webhook subscription with the credentials of Event Mesh.
See below:

Hope that the diagram clarifies a bit the scenario:
Event Mesh fetches a JWT token from the Event Mesh XSUAA instance and sends it to our endpoint, where it is validated with our XSUAA.

How does that work?
Event Mesh credentials are actually used to protect the Event Mesh REST API.
We need the Event Mesh credentials when we call the Event Mesh REST API.

How can these credentials be used to call OUR endpoint?
The solution is the GRANT statement on our side and the AUTHORITIES on Event Mesh side.

Above diagram shows that our application has its binding to an instance of XSUAA and this instance is used to protect our app.
Event Mesh has its own XSUAA instance which is used to protect the Event Mesh API.
And Event Mesh uses its own instance to fetch a token which is sent to our endpoint.

There are 2 hurdles:

1. OAuth client
Every XSUAA instance actually represents an OAuth client which has a client id and a client secret.
When the Event Mesh client calls the XSUAA authorization server asking for a JWT token, then this token will contain the client id.
When our endpoint receives that token, it will validate the token. This validation is done by the JWTStrategy of xssec library.
This JWTStrategy object is configured with our instance of XSUAA.

As such, the validation will check if OUR client id is contained in the JWT token.
Foreign clients will be rejected.
Means that the Event Mesh client would be rejected, normally.
So how can Event Mesh call our endpoint with foreign client?

2. The scope
In addition, we require that the incoming token contains a scope.
We define this scope in the configuration of our XSUAA instance.
Such a scope lives in the context of the subdomain in our cloud subaccount.
It can be assigned to human users, if it is wrapped in a role.
It can be assigned to applications (for client_credentials scenario, where an app calls another app) by using the authorities statement In the previous blog we learned this mechanism.
But how can we assign the scope to a foreign application?

There’s one solution for both hurdles:
In our own XSUAA configuration, we define the scope and we also declare a grant.
Which means, that we decide to grant that scope to a specific foreign OAuth client.
With other words, we allow that the foreign client may obtain that scope.
With other colors: we allow that the orange client becomes green.

But that’s only half the way:
That foreign OAuth client also needs to accept that scope.

If both conditions are met, then the foreign OAuth client (in the same subdomain/subaccount) will get that scope, when it requests a JWT token.

Hey, this is the solution for hurdle nr. 2
What about hurdle nr. 1 ?

True.
In fact, the foreign client is still a foreign client and not accepted by the JWTStrategy validation.
Answer:
There’s another mechanism.
Whenever a scope is granted to a foreign OAuth client, then XSUAA will:
–  write the scope into the scope claim of the JWT token
–  take care of the audience.

What does that mean?
The “audience” is the intended receiver of the JWT token.
I mean, if I carry (bear) a key, then the key knows, which doors it can open.
So in our case, the audience is
– always the own client, the Event-Mesh-XSUAA, the Event Mesh REST API
– but also our protected endpoint, means our own app-XSUAA

How can our own app-XSUAA be written into the audience?
It is the value of xsappname property, which  is written into the aud claim.
Remember that whenever we define a scope, we concatenate the name of the scope with the xsappname value:

"name": "$XSAPPNAME.scopeforhookorcrook"

XSUAA will split the received scope name and it will write the xsappname into the aud field of the JWT token.

Repeat:
Event Mesh will receive a JWT token and send it to our endpoint.
That JWT token will have OUR correct scope.
But it will have a foreign wrong clientId.
However, it will have OUR correct xsappname in the audience field.
This means that OUR xsappname is an audience for the token.
With other words, the token is in fact intended for our xsappname.
As such, when OUR endpoint validates the incoming token, using the JWTStrategy configured with OUR instance of XSUAA, then the validation will succeed.
Because it will find its own xsappname in the foreign token.
That’s good.
And our manual scope check will find OUR scope in the foreign JWT token as well.

All good.

HAPPY END!

To learn about the mentioned mechanism, I recommend this blog post.
It describes how 2 instances of XSUAA communicate with each other, I mean, how a scope is granted to a foreign OAuth client.

However…..
???
….there’s one more little trick.

The reader of the tutorial will realize that we talk about “applications” in the perspective of OAuth clients.
When we create an instance of XSUAA, we define an “xsappname”. This is the name of the “application”.
The statement grant-as-authority-to-apps is meant to grant the scope to an app which has an “xsappname”
On the other side, the receiving instance of XSUAA has to declare the “authorities” with the “ACCEPT” statement.
With other words: granting and accepting works only between instances of XSUAA.
But Event Mesh is not an “instance of XSUAA”.
OMG !
So the trick is to declare the “authorities” and ACCEPT statement in the configuration of Event Mesh client and then Event Mesh will take care of delegating it to the Event-Mesh-XSUAA.

The syntax of the authorities property in the Event-Mesh-config is the same as in the XSUAA-config:

"authorities": ["$ACCEPT_GRANTED_AUTHORITIES"]

That’s all what I wanted to explain.

Short Summary:
We declare a grant in our app.
We declare an accept in the Event Mesh
Then the scenario works

All that is confusing, when talking in theory.
So let’s get the hands dirty and get the described scenario running.

0. Prerequisites

The previous blog and all of its text is a prerequisite, it contains more details and explanations.

This tutorial is based on productive account in SAP BTP.
If you’re using Trial account, you need to deploy a sender app (see sample code)
Furthermore, you need to be aware of the differences between Trial and productive account, respectively dev and default service plans.
Differences are listed here.

00. Preparation

We need a project containing an app which is used to expose a webhook endpoint for receiving the messages.
To send messages, we use the test tool of the Event Mesh dashboard.
Alternatively, the sender application can be deployed and used to send messages, as described in previous blog post.
Project structure with root and app folder:

C:\hookorcrook
receiver

1. Event Mesh

We need an instance of SAP Event Mesh service and we need access to the Event Mesh dashboard.

1.1. Create Service Instance

To create instance of Event Mesh service, “default” plan, we need configuration parameters which we store in a config file in the project folder:
C:\hookorcrook\config-messaging.json
The content of this file can be found in the Appendix 1 section.
The creation command:
cf cs enterprise-messaging default hookorcrookMsg -c config-messaging.json

The relevant snippet:

{
  "emname": "hookorcrookmessagingclient",
  . . .
  "authorities": ["$ACCEPT_GRANTED_AUTHORITIES"]
}

Explanation
The authorities property is in fact the central learning of this blog post.
It is meant to declare which scopes are accepted.
A different xsuaa instance grants a scope to this instance.
And now, this instance declares that it accepts that scope.
Why we need it?
As we know, Event Mesh sends a JWT token to the protected webhook endpoint, when forwarding the messages.
And now we want that this token contains the relevant scope.
The relevant scope is the one that is defined by the receiver app.
It is defined and required.
And in order to make the scenario work, the receiver app grants it to the Event Mesh instance.

Note:
This explanation is not completely precise.
The truth is:
An instance of XSUAA grants the scope to a different instance of XSUAA.
Not to any other service, like Event Mesh.
Event Mesh handles it, as is bound to its own instance of XSUAA and it delegates the authorities to the XSUAA.
The documentation of the authorities property in XSUAA can be found here.

Service Key
There’s one additional step required:
Later, when creating the webhook subscription, we will need to access the credentials of Event Mesh service instance.
Since we don’t bind our application to Event Mesh, we must create a service key:
cf csk hookorcrookMsg sk
Above command creates a service key with name sk for the Event Mesh instance
Then we view the content of the service key and take a note of the uaa properties which we need for the webhook subscription
cf service-key hookorcrookMsg sk

Summary
If we want to use the Event Mesh uaa credentials in the webhook subscription, then we need the authorities statement in the Event Mesh config.

1.2. Dashboard

A description about how to get access to the dashboard (subscribe and assign the required roles) can be found in previous blog post.

Queue
We create a queue with name “hookorcrookQueue”

2. Send Events

The previous description about sending events is still valid.

2.1. Sender App

A sender app can be used if desired, sample code can be found in the previous blog post.
Note:
Make sure to adapt the messaging configuration in the config-messaging.json file

2.2. Use Dashboad

To make life easier, we use the test tool in the dashboard for sending events.
Note:
In our scenario we’re using plain text as content type.
Our receiver app assumes the event payload to be plain text.
That’s why we leave the default setting in the test tool.

3. Receive Events

The application code for the receiver webhook endpoint is basically the same as in previous blogs.
The difference is in the configuration of the service instances and of the webhook subscription.

3.1. Create instance of XSUAA service

Again, we want to protect our app with OAuth and therefore we need to create an instance of XSUAA service.
We use a config file xs-security.json which we create in the app folder C:\hookorcrook\receiver.
The content can be copied from Appendix 2 section.

Creation command to be executed inside the receiver folder:
cf cs xsuaa application hookorcrookXsuaa -c xs-security.json

The configuration is the same like in previous blog, the only difference is in the security configuration.
Here we need to grant the scope to the foreign XSUAA instance.

. . .
"scopes": [{
    "name": "$XSAPPNAME.scopeforhookorcrook",
    "grant-as-authority-to-apps": ["$XSSERVICENAME(hookorcrookMsg)"]
. . .

The property declares that we want to grant the currently defined scope to another “app”.
In this context, “app” refers to an OAuth “application” for which an OAuth client is registered. This client is represented by an instance of XSUAA and it is referenced by its “xsappname”.
As such, the value of the “grant” property is an “xsappname”.
In our example, we use a shortcut, which makes use of the standard variable $XSSERVICENAME and the value is the name of the instance of Event Mesh, which we created in previous section of this blog post.
In case the name doesn’t match correctly, you get a helpful error message.

The documentation can be found here (section “Granting scopes”).

3.2. Create Application

The application code is all the same like in previous blog post.

Only one small addition:
We want to print some of the information that is sent along with the JWT bearer token.
The information in the log should clarify the scenario.
We want to see:
The clientid which has requested the token.
The audience for which the token is intended.
The scopes contained in the token.

console.log(`===> [/webhook/hookorcrook] JWT client_id: ${req.tokenInfo.getPayload().client_id}`)
console.log(`===> [/webhook/hookorcrook] JWT aud: ${req.tokenInfo.getPayload().aud}`)
console.log(`===> [/webhook/hookorcrook] JWT scopes: ${req.tokenInfo.getPayload().scope}`)

The full sample code can be found in Appendix 2.

3.3. Create Webhook Subscription

After deployment of our receiver app, we can create the webhook subscription.
Note that today the configuration differs from previous tutorials:
To retrieve the oauth credentials, we don’t access the environment of our app.
With other words, cf env is not the statement of the day.
As mentioned earlier, today we don’t want to provide the credentials of our own XSUAA into the dialog.
We want to use the credentials of the Event Mesh.
As such, the featured command is
cf service-key hookorcrookMsg sk

*applause*

Once we have the service key of Event Mesh, we search fo the uaa section (not messaging, etc) and find the clientid, clientsecret and url properties.
Don’t forget to complete the url by appending /oauth/token
Example:
https:// abc123trial.authentication.sap.hana.ondemand.com/oauth/token

And the URL of the webhook,
Example
https://hookorcrookreceiver.cfapps.eu10.hana.ondemand.com/webhook/hookorcrook
Default Content-Type:
text/plain

Below screenshot shows the configuration with the long cryptic cliend id and secret of Event Mesh:

After creation of the webhook subscription we start observing the log:
cf logs hookorcrookreceiver

And now: fire !

3.4. Understanding the log

After firing an event, we get the following output from receiver:

Unreadable.
So I’ve cleared it a bit for you:

Above screenshot shows the client id of the OAuth client, which has obtained a JWT token.
This is the client id which we have entered in the webhook subscription dialog.
So it works as expected.
Below screenshot shows the content of the aud claim:

We can see that the same client is also an audience. This is obvious.
The client is of course always allowed to access a resource for which it has credentials.
The interesting point is the second entry in the audience:
It is the xsappname of our own app-XSUAA.
We can see that it looks completely different.
And yes, it is also expected:
That’s how the grant mechanism works, as explained above.
Nevertheless, we can be happy that it works as expected.
Happy End.

Just one last screenshot:
We don’t want to close the tutorial without checking that the granted scope is really contained in the JWT token as desired:

We can see the xsappname of our receiver app concatenated with the scope name.
Just what we did in the xs-security.json config.
And just what we also check in the code of the webhook, to make sure that the webhook can only be accessed by the Event Mesh, to which we granted the scope.

Happy End

Summary

Again, todays setup was to create a webhook endpoint that is protected with OAuth and scope.
Today we learned how to configure the scenario such that the webhook subscription in the dashboard can be configured with the credentials of Event Mesh, instead of the own app-credentials.
Nevertheless, the own app-credentials are still used for protection.
The trick was to define a grant in the own app, and to declare an authorities in the Event Mesh.

Links

XSUAA documentation about the xs-security.json parameters.
Event Mesh documentation about Entitlements.

Quick Steps

In own app xsuaa, define scope and grant:

"name": "$XSAPPNAME.scopeforhookorcrook",
"grant-as-authority-to-apps": ["$XSSERVICENAME(myMsgInstance)"]            

In service descriptor of Event Mesh, accept it (top-level:

"authorities": ["$ACCEPT_GRANTED_AUTHORITIES"]

No change in code required.
Webhook is configured with oauth and credentials of Event Mesh.
Service Key for Event Mesh might be required.

Appendix 1: Service Descriptor

config-messaging.json

{
  "emname": "hookorcrookmessagingclient",
  "namespace": "hook/or/crook",
  "version": "1.1.0",
  "options": {
      "management": true,
      "messagingrest": true,
      "messaging": true
  },
  "rules": {
      "queueRules": {
          "publishFilter": [
              "${namespace}/*"
          ],
          "subscribeFilter": [
              "${namespace}/*"
          ]
      },
      "topicRules": {
          "publishFilter": [
              "${namespace}/*"
          ],
          "subscribeFilter": [
              "${namespace}/*"
          ]
      }
  },
  "authorities": ["$ACCEPT_GRANTED_AUTHORITIES"]
}

Appendix 2: Receiver App

Note: everything can be copy&pasted, only URLs need to be adapted

xs-security.json

{
    "xsappname": "hookorcrookxsappname",
    "tenant-mode": "dedicated",
    "scopes": [
        {
            "name": "$XSAPPNAME.scopeforhookorcrook",
            "description": "Scope required to access webhook endpoint",
            "grant-as-authority-to-apps": ["$XSSERVICENAME(hookorcrookMsg)"]            
        }
    ]
}

manifest.yml

---
applications:
  - name: hookorcrookreceiver
    memory: 64M
    routes:
    - route: hookorcrookreceiver.cfapps.sap.hana.ondemand.com
    services:
      - hookorcrookXsuaa

package.json

{
  "dependencies": {
    "@sap/xsenv": "latest",
    "@sap/xssec": "latest",
    "express": "^4.17.1",
    "passport": "^0.4.0"
  }
}

server.js

const xsenv = require('@sap/xsenv')
const serviceBindings = xsenv.getServices({ 
   myXsuaa: {tag: 'xsuaa'}
})
const XSUAA_CREDENTIALS = serviceBindings.myXsuaa

const express = require('express')
const app = express();
const xssec = require('@sap/xssec')
const passport = require('passport')
const JWTStrategy = xssec.JWTStrategy
passport.use('XSUAA_BINDING', new JWTStrategy(XSUAA_CREDENTIALS))
app.use(passport.initialize())
app.use(express.text())


app.listen(process.env.PORT,  () => { console.log('===> Server started') })

app.post('/webhook/hookorcrook', passport.authenticate('XSUAA_BINDING', {session: false}), (req, res) => {
    console.log(`===> [/webhook/hookorcrook] JWT client_id: ${req.tokenInfo.getPayload().client_id}`)
    console.log(`===> [/webhook/hookorcrook] JWT aud: ${req.tokenInfo.getPayload().aud}`)
    console.log(`===> [/webhook/hookorcrook] JWT scopes: ${req.tokenInfo.getPayload().scope}`)

    const auth = req.authInfo  
    if (! auth.checkScope(XSUAA_CREDENTIALS.xsappname + '.scopeforhookorcrook')) {
        console.log(`===> [/webhook/hookorcrook] ERROR scope for webhook access ('${XSUAA_CREDENTIALS.xsappname}.scopeforhookorcrook') is missing`)
        res.status(403).end('Forbidden. Authorization for webhook access is missing.')
    }else{
        console.log(`===> [/webhook/hookorcrook] Received message with payload: ${req.body}`)
        res.status(201).send()
    }      
})