This blog is part of a
little series about
SAP Cloud Platform Job Scheduler
Have you configured a job to call your oauth-protected endpoint?
And does it fail?
And do you get an error like 401 or 403?
And you hate it?
If yes, you should go should give this blog a chance and hopefully it will solve your problem…!
I’ve listed possible reasons for failure and possible ways how to find the error and solve the problem.
I’m not explaining much, you can find detailed descriptions in the blogs of
this series
And please:
If you have another tip how you solved a problem, please add it in the comment section
1. Jobscheduler instance creation
When creating the instance of jobscheduler service, you have to pass the following parameter:
{
"enable-xsuaa-support": true
}
See
here for details
2. Jobscheduler instance creation
When creating the instance of jobscheduler service, you have to choose service plan
standard
Otherwise, OAuth flow is not fully supported for your endpoint, and authorization will always fail
3. XSUAA instance creation
When creating the instance of “Authorization and Trust Management” (XSUAA), you have to choose service plan as
application or
broker
See
here for details
4. XSUAA scope contains 'grant...'
Background:
You create an application and you define “scopes” to control the access to it.
Means, a user who calls your app needs to have that “scope”
Thus Jobscheduler must have it as well
But Jobscheduler is not a user, so the scope must be assigned via “GRANT” in the xs-security.json file
"scopes": [{
"name": "$XSAPPNAME.scopeformyapp",
"grant-as-authority-to-apps": ["$XSSERVICENAME(my_jobscheduler_instance_name)"]
}]
This is essential!
When Jobscheduler calls your app, it sends a JWT token, and this token will contain the required scope - but only if it has been granted
See
here for details
5. xs.security.json
In case of changes to
xs-security.json:
The instance of XSUAA needs to be updated, re-deploy of app is not sufficient.
If you've modified the
xs-security.json file, then you need to exeucte the following command to make the changes available in the instance of xsuaa:
cf update-service yourXsuaa -c xs-security.json
Note that updating a service instance can only be done on command line, it is not supported in the cockpit.
Note:
After changing the xsuaa instance, you need to bind it again to your app.
See section 8
6. SAP_JWT_TRUST_ACL
Background:
Jobscheduler cannot read the credentials of your app (I mean the OAuth credentials which are provided to your app, when you bind it to xsuaa
So it cannot send the expected
clientid
Instead, jobscheduler sends its own
clientid
But, that clientid is wrong, so it is rejected, unless your app declares an exlicit TRUST to it
To specify that trust, your app has to provide the following environment variable, which specifies the trusted clientid.
You can specify that clientid (you can find it in the credentials section of the jobscheduler, bound to your app), or just an asterisk (obviously, not recommended in productive scenarios)
The mentioned environment variable can be set in the
manifest.yml file:
env:
SAP_JWT_TRUST_ACL: >
[
{"clientid":"*", "identityzone":"*"}
]
Environment variables can also be set via command line or in the cloud cockpit
Note: make sure not to forget all the brackets.
Note: there must be the square brackets as well
See
here for details
7. Check the TRUST
After deployment of your app, it might make sense to check if the TRUST property has properly reached the cloud
It can be checked in the cloud cockpit, in the details screen of your app, in the section user-provided variables
Or you can use command line
cf env yourAppName
Result in user-provided section:
SAP_JWT_TRUST_ACL: [
{"clientid":"*", "identityzone":"*"}
]
Note: make sure that the quotation marks are there
8. Laundry program for binding
In
tutorial no 3, we‘ve learned that the order of creation and binding of service instances is crucial, because otherwise the jobscheduler doesn’t get the required scope assignment. And as a consequence, calling your endpoint will fail
Remember:
Create Jobs first, but
Bind xsusa first again
Instead of going through the process of creation and binding again, you can just execute I(what I call) the laundry program.
I mean, just unbind the servce instances, then bind again in the required order
These are the commands to be executed on command line
Make sure to adapt
your app name and
your service instance name
First unbind both instances:
cf unbind-service yourApp yourXsuaa
cf unbind-service yourApp yourJobscheduler
cf restage yourApp
Then bind xsuaa:
cf bind-service yourApp yourXsuaa
cf restage yourApp
Finally bind jobscheduler:
cf bind-service yourApp yourJobscheduler
cf restage yourApp
(I confess that the amount of commands it a bit paranoid...)
9. Analyze the error
In the Jobscheduler run log, check the error message:
The error status is 401 or 403?
In case of 403, the authentication is fine, but authorization is missing. This would be an indicator that the jobscheduler doesn't have the required scope
10. Call your endpoint manually
Make sure that your endpoint behaves as expected.
I mean, it is possible that the problem is not with Jobscheduler, but with the endpoint
To verify it, invoke the endpoint URL with a REST client (e.g. Postman)
You need to follow the OAuth flow, it is described
here or
more detailed here
11. View the content of the token
When calling the endpoint manually, like described in previous section, you fetch the JWT token manually.
Once you have it, you can decode it.
Then check out the scopes section: how does the required scope look like?
Remember it and compare with the token and the scope which is sent by Jobscheduler (see next section)
Find the required
description here
12. View the content of the token sent by Jobscheduler
This is the essential check:
Does the JWT token which is sent by Jobscheduler contain the required scope?
This might be not easy to identify.
To verify it, hook into your app, before your endpoint fails.
You need to access the “request” object and get the “authorization” header.
The value of this header contains the token, you can decode it
Check if the required scope is there
See here how it is done in Node.js
13. Other attempt to view the content of the token
If previous way is not possible:
Add another endpoint to your app.
No frameworks.
No protection, no security checks.
The endpoint should just
read the jwt token
In addition, in the code, you can also access the TRUST environment variable, and check if it contains valid JSON.
I mean, e.g. if the quotation marks were lost, then parsing the content would fail, because not valid JSON
14. One more attempt to view the content of the token
If previous way is not possible:
Increase the log level of the involved components, like
xssec, or app router
xssec:
To increase the log level, add this variable to your environment:
DEBUG=xssec:*
App router:
In case you have an application router involved, you can also configure it to write millions of log entries.
E.g. run this command
cf set-env yourAppName XS_APP_LOG_LEVEL DEBUG
In addition, more environment variables to add:
REQUEST_TRACE = TRUE
DEBUG=*
15. Really the last attempt to view the content of the token
If there’s no other option, you can write a separate app, which does nothing than read the JWT token.
Similar like above attempt, but this time a separate app
Here's why it is the last attempt: a separate app is not the same app, so it might not be really helpful.
Nevertheless, you bind it to the same instance of xsuaa and jobscheduler (remember the order) like your productive app
1. Hopefully you're little familiar with Node.js, so you can create a meaningful app with below files
The code of the app can be found in the appendix
2. Deploy
3. Create a Job
4. Then check the log (jobscheduler run log and also Cloud Platform log).
There you can find the information about the JWT token which was sent by Jobscheduler.
16. One more hint
Check out the Guided Answers
here.
17. Issues with Dashboard
In case you face problems with display of Job Scheduler Dashboard:
Are you using old Microsoft browser?
Background:
Newer UI5 versions (from 1.88) don’t support the legacy browsers IE11 and Edge Legacy (Edge HTML) anymore (
more info)
To solve the problem, you should use modern browser
If you find a bug or have added enhancements: please let us know
Appendix: Troublemaker app files
Please don’t expect too much from this app:
It only prints the content of the JWT token which it receives
And it writes the value of the TRUST param to the log
That’s already all.
But it might be helpful....
Has already be useful for Werner (thanks, Werner...)
Once you see the scope contained in the token, you might be surprised, maybe the scope-check was not properly implemented, who knows…
The tiny app is written in node.js, so you should get familiar with node if you aren’t yet
😉
You have to create a folder in your file system
Then create the 3 files inside
Adapt the service names in the manifest file
Enhance the implementation as desired
Make sure to write a comment to share your enhancements with the community
To deploy, navigate into the project directory and run
cf push.
Finally, here are the required files:
manifest.yml
To deploy the app to the SAP Cloud Platform, we need a manifest file
Make sure to adapt the names of the service instances.
Ideally, just use the manifest of your productive app
---
applications:
- name: troublemaker
host: troublemaker
memory: 64M
buildpacks:
- nodejs_buildpack
services:
- your_xsuaa_instance
- your_jobscheduler_instance
env:
SAP_JWT_TRUST_ACL: >
[
{"clientid":"*", "identityzone":"*"}
]
package.json
{
"main": "server.js",
"dependencies": {
"express": "^4.16.3"
}
}
server.js
const express = require('express');
const app = express();
/* The endpoint */
app.get('/job', function(req, res){
const authHeader = req.headers.authorization;
const acl = process.env.SAP_JWT_TRUST_ACL
const jwtCheckResult = _checkAuthorization(authHeader)
const jobscheduler_client = jwtCheckResult.clientid
const jobscheduler_identityzone = jwtCheckResult.identityzone
const trustCheckResult = _checkTrust(acl, jobscheduler_client, jobscheduler_identityzone)
if(jwtCheckResult.isValid && trustCheckResult.isValid){
res.status(200).send(`Successfully checked JWT token and TRUST. \n Token contains scopes: ${jwtCheckResult.scopes} \n for jobscheduler-client: ${jobscheduler_client}` )
}else{
res.status(403).send(`Validation failed. Invalid JWT and/or TRUST. See log for details`)
}
});
/* The server */
app.listen(process.env.PORT, function(){})
/* The helpers */
const _checkAuthorization = function(authHeader){
console.log(`===> [Check JWT] Checking the authorization...`)
if (! authHeader){
console.log('===> [Check JWT]: Endpoint was called with no authorization header at all')
return {isValid: false, message: 'Request does not contain authorization header'}
}
if(! authHeader.includes('earer')){
console.log(`===> [Check JWT]: Endpoint was called with authorization header, but it does not contain bearer token: ${authHeader}`)
return {isValid: false, message: 'Authorization header of request does not contain bearer token'}
}
var theJwtToken = authHeader.substring(7);
if(! theJwtToken){
console.log(`===> [Check JWT]: Endpoint was called with bearer-token header, but the token seems to be invalid: ${authHeader}`)
return {isValid: false, message: 'Authorization header contains invalid JWT token'}
}
console.log(`===> [Check JWT] the received JWT token: \n${theJwtToken}`)
let jwtBase64Encoded = theJwtToken.split('.')[1];
if(! jwtBase64Encoded){
console.log(`===> [Check JWT]: Endpoint was called with bearer-token header, but the token seems to have invalid structure: Must have 3 segments separated with dot.`)
return {isValid: false, message: 'Authorization header contains invalid JWT token'}
}
let jwtDecodedAsString = Buffer.from(jwtBase64Encoded, 'base64').toString('ascii');
console.log(`===> [Check JWT] the decoded JWT token: ${jwtDecodedAsString}`)
let jwtDecodedJson = undefined
console.log(`===> [Check JWT] parsing the JWT token...`)
try {
jwtDecodedJson = JSON.parse(jwtDecodedAsString);
} catch (error) {
console.log(`Endpoint was called with bearer-token header, but parsing the token caused error: ${error}`)
return {isValid: false, message: 'Error while parsing JWT token. See log for details.'}
}
// JWT token is valid, however, checking the scopes has to be done manually
console.log(`===> [Check JWT]: JWT: scopes: ${jwtDecodedJson.scope}`);
console.log(`===> [Check JWT]: JWT: client_id: ${jwtDecodedJson.client_id}`);
console.log(`===> [Check JWT]: JWT: identity_zone: ${jwtDecodedJson.zid}`);
console.log(`===> [Check JWT]: JWT: user: ${jwtDecodedJson.user_name}`);
return {
isValid: true,
message: `Successfully parsed JWT token. See log for more details. \n Scopes: ${jwtDecodedJson.scope} \n client: ${jwtDecodedJson.client_id} \n user: ${jwtDecodedJson.user_name}`,
clientid: jwtDecodedJson.client_id,
identityzone: jwtDecodedJson.zid,
scopes: jwtDecodedJson.scope
}
}
const _checkTrust = function (aclstring, clientSentByJobScheduler, identitySentByJobscheduler) {
console.log('===> [Check TRUST] Checking the The variable SAP_JWT_TRUST_ACL...')
if(! aclstring){
console.log('===> [Check TRUST] The variable SAP_JWT_TRUST_ACL was not found in the app environment')
return {isValid: false, message: 'Missing environment variable SAP_JWT_TRUST_ACL'}
}
let aclJson = undefined
try {
aclJson = JSON.parse(aclstring)
} catch (error) {
console.log(`===> [Check TRUST]: Error while parsing env var: SAP_JWT_TRUST_ACL. Value is not valid JSON: ${error}`);
return {isValid: false, message: 'Error while parsing SAP_JWT_TRUST_ACL. Value is not valid JSON. See log for details'}
}
console.log(`===> [Check TRUST]: value of SAP_JWT_TRUST_ACL is valid JSON`);
// check correct format
const aclIsArray = Array.isArray(aclJson)
console.log(`===> [Check TRUST]: value of SAP_JWT_TRUST_ACL is valid Array: ${aclIsArray}`);
if (! aclIsArray){
console.log(`===> [Check TRUST]: Incorrect format of SAP_JWT_TRUST_ACL: Value is NOT array`);
return {isValid: false, message: 'Incorrect format of SAP_JWT_TRUST_ACL: Value is NOT array. See log for details'}
}
// check if jobscheduler (foreign) clientid is trusted
var foundMatch = false;
for (var aclEntry in aclJson) {
let clientidInAcl = aclJson[aclEntry].clientid
let identityzoneInAcl = aclJson[aclEntry].identityzone
console.log(`===> [Check TRUST]: value of SAP_JWT_TRUST_ACL[i].clientid: ${clientidInAcl} and identityzone: ${identityzoneInAcl}`);
if (((clientidInAcl === clientSentByJobScheduler) || (clientidInAcl === '*'))
&& ((identityzoneInAcl === identitySentByJobscheduler) || (identityzoneInAcl === '*'))) {
foundMatch = true;
break;
}
}
if (foundMatch) {
console.log(`===> [Check TRUST]: the jobscheduler clientid and identityzone are contained in TRUST. All fine ;-)`);
return {isValid: true, message: 'The jobscheduler clientid and identityzone are contained in TRUST'}
} else {
console.log(`===> [Check TRUST]: Error: the jobscheduler clientid is NOT contained in TRUST. Result: 403 - Forbidden`);
return {isValid: false, message: 'The jobscheduler clientid and identityzone NOT contained in TRUST_ACL'}
}
}
