This is a searchable description of the content of a live stream recording, specifically “Ep.57 - Setting up our first Workflow definition on CF” in the “Hands-on SAP dev with qmacro” series. There are links directly to specific highlights in the video recording. For links to annotations of other episodes, please see the “Catch the replays” section of the series blog post.
This episode was streamed live on Fri 13 Mar and is approximately 65 minutes in length. The stream recording is
available on YouTube.
Brief synopsis: Now we have our Workflow tools set up on Cloud Foundry, it’s time to put them to use, with a simple workflow definition that we’ll create and deploy. What’s more, we dive into a little OAuth 2.0 adventure of discovery!
00:02:55 After a brief moment of fail on my part where I’d forgotten to reconfigure the mic settings since switching back to the MBP for this broadcast, my audio improves so that you can hear what I’m saying.
00:04:00 Showing off my Asus Chromebox 3 (i5, 8GB RAM) which, unusually, is not in use so I can hold it up the camera.
00:04:40 Switching to show
thomas.jung’s
update on LinkedIn about the SAP Community Coding Challenge Finalist Announcement (with a brief return to the mic fail as it’s configured on a per-scene basis). The announcement is on 17 Mar - and you can head on over to Tom’s scheduled live stream session
here to set a reminder. Find out more about the challenge series in Tom’s post “
SAP Community Coding Challenge Series”.
00:07:15 Mentioning a new live stream initiative, which is related to our Hands-on SAP dev series. It’s called
Brambleweeny Cluster Experiments and it’s for exploring and learning about clouds, clusters and more, using the physical hardware medium of Raspberry Pi computers. I’ve not set a schedule up yet, and may just do this on a fairly ad-hoc basis, but in any case, watch out on my YouTube
home page for upcoming live stream episodes and set reminders as appropriate. These topics are indeed relevant for us as developers and technicians in the SAP tech universe, and will become more so over time too.
00:09:20 Looking at where we’d left off at the end of the last episode, with a fully deployed Workflow tools environment in my Cloud Foundry (CF) dev space, including the running
workflowtilesApprouter
app and the
workflowtilesFLP
app which is in the stopped status as the task (of deploying content to the portal service) had completed successfully.
00:12:00 Jumping to the tiles in the Fiori Launchpad site, which are available (and to which we’re directed) at the
/cp.portal
path, as defined in the
xs-app.json
. Following this, I share some thoughts on how it’s important to properly understand OAuth in this context, i.e. in the context of exercising APIs on SAP Cloud Platform Cloud Foundry, to which end I intend to dig in a little bit to better grok how things fit together.
00:16:50 To create a new workflow definition, and deploy it, we create a new Multi-Target Application template based project. This appears initially very empty, which makes sense as we haven’t defined any modules (or dependencies) yet. The entirety of the project looks like this:
Episode57/
|
+-- mta.yaml
Even the
mta.yaml
contents are very minimal:
ID: Episode57
_schema-version: '2.1'
version: 0.0.1
00:17:50 Using the context menu, we add a new Workflow Module which then appears in our project, so that we have something recognisable (from working in the SAP Web IDE with the Workflow service on Neo in the past), within our MTA project. We end out with the simplest workflow definition possible (the flow starts and then immediately ends) but it’s enough for now. More importantly, this is what we end up with in our project structure:
Episode57/
|
+-- SimpleWorkflow/
| |
| +-- forms/
| +-- sample-data/
| +-- scripts/
| +-- webcontent/
| +-- workflows/
| |
| +-- simpleworkflow.workflow
|
+-- mta.yaml
(At this point
rsletta arrives on the scene, prompting me to remind folks of the SAP Online Track update he posted recently:
SAP Online Track - Weekly Update #2.)
We’ve seen this type of module before, as a ‘content deployment’ type - the
workflowtilesFLP
module is of type
com.sap.portal.content
, and this time we have a slightly more generic
com.sap.application.content
content deployment type that describes our
SimpleWorkflow
module.
00:20:50 While we’re checking this type, we modify the name references for the workflow service instance in our
mta.yaml
which have been injected when we created the new
SimpleWorkflow
module, also changing the type of that resource to “existing” (
org.cloudfoundry.existing-service
).
At this point, I also notice that the parameters in this resource definition, are probably not required if we’re specifying an existing service … but then forget to come back and test that theory. While writing up these annotations, I did test it, removing the
parameters
node of the
workflow
resource definition:
resources:
- name: workflow
parameters: <--- removed
service-plan: standard <--- removed
service: workflow <--- removed
type: org.cloudfoundry.existing-service
I can happily confirm that after removing these three lines the deployment still succeeded. Makes sense, right?
00:22:50 Having checked through everything we build and deploy the MTA (in the same way we would build and deploy any MTA) which causes the
simpleworkflow
workflow definition to be deployed to our instance of the workflow service.
00:24:55 Checking the deployment log, we notice this line:
Creating service key “SimpleWorkflow-workflow-credentials” for service “workflow”…
and we’ll see this new service key shortly.
00:25:25 We see, in the Workflow Definitions app, that we now have this new
simpleworkflow
definition alongside a definition I’d deployed earlier this week (
orderprocess
) from another MTA project “OrderFlow”. We test the definition out by creating an instance of it, and all looks well (remember, we have to widen the status filter to be able to see instances in COMPLETED status).
00:27:10 At this point, our thoughts turn towards using the SAP API Business Hub, specifically to use the Workflow API to list those definitions. We’ve used the API Hub many times before, for example in the context of the Workflow API for Neo, and the context of the Business Rules API for Cloud Foundry. Here, there are a couple of differences worth mentioning:
- The Workflow API for Neo uses Basic Authentication and (as a consequence) also requires the use of CSRF tokens
- The Business Rules API for Cloud Foundry divides the “API space” differently, in that it has separate API groups for designtime and runtime artifacts (think of this distinction as the difference between managing the artifacts in the repository - designtime - and actually invoking those APIs - runtime)
Here, the Workflow API for CF, like all the APIs I’ve seen for CF, doesn’t use Basic Authentication, but uses OAuth 2.0. This is an open standard that goes way beyond what Basic Authentication has to offer, and supports not only authentication but also authorisation, and does that across multiple parties - users, agents (clients), authentication servers and resource servers. It’s an accomplished but complex standard … and because it’s at the heart of many things we’ll be doing on a day to day basis in real life with Cloud Foundry, it’s important to understand properly.
Considering how the Business Rules APIs are organised, it’s worth noticing that the Workflow APIs are organised slightly differently. There’s no distinction between designtime and runtime (if you look at the
specifications on the API Hub you’ll see that a single API endpoint covers definitions and instances for tasks, workflows and more) but there is a distinction between this main endpoint, known as the “Workflow REST API”, and the “Workflow OData API” which is (surprise surprise) an OData service for task collection management, used in the My Inbox app, for example.
Actually, you can see these endpoints spelled out in the service key data, the relevant section of which looks like this:
{
"endpoints": {
"workflow_odata_url": "https://api.workflow-sap.cfapps.eu10.hana.ondemand.com/workflow-service/odata",
"workflow_rest_url": "https://api.workflow-sap.cfapps.eu10.hana.ondemand.com/workflow-service/rest"
},
...
}
00:29:30 Anyway, getting back to what we want to focus on here, which is an initial exploration of OAuth, specifically, as we’ll see shortly, the “Client Credentials Grant” flow. We start with the very convenient configurable “Environments” feature of the API Hub. We’ll be eventually able to summise, by poking this black box feature with a stick, that it performs the “Client Credentials Grant” flow on our behalf, which is nice.
00:30:50 In order to complete an environment configuration (modulo the slight hiccup with that feature during the live stream), we need to take information from our service key data from the workflow service instance that we have.
First, we need the starting endpoint URL, which is our EU10 specific one, i.e.
https://api.workflow-sap.cfapps.eu10.hana.ondemand.com/workflow-service/rest
. This is, as you’ve probably spotted, exactly the value of our
workflow_rest_url
endpoint that we looked at earlier, within the
endpoints
section.
Next, we need OAuth credentials, in the form of a client ID and secret. And, you guessed it, they’re also in the service key data, in the
uaa
section.
The authentication endpoint is next, in the form of the “Token URL” (i.e. the URL that we’re going to send our request for a token to). This is (surprise surprise) also in the service key data, at the
uaa.url
node, and in our case it’s
https://i347491trial.authentication.eu10.hana.ondemand.com
which is based on the combination of the generic
https://authentication.eu10.hana.ondemand.com
URL with a hostname prefix reflecting my subaccount name
i347491trial
.
00:36:20 Now that we have our “EU10” environment configured, we go to make an API call to list the workflow definitions, expecting to see the two we know about, i.e.
simpleworkflow
and
orderprocess
. We use the “Try out” facility for this API endpoint:
GET /v1/workflow-definitions
BUT … this is what we get:
{
"error": {
"message": "User does not have sufficient privileges."
}
}
which accompanies an HTTP 403 response. Ouch! Why is this? Well, read on to find out. It will all make sense, when we dig into this a bit more.
00:37:20 While trying to figure out why we get a 403 response (“Forbidden”, of course, learn your
HTTP response codes if you don’t know them already!), we go over to the subaccount level security settings and check that the role collection “workflow”, containing a myriad roles relating to workflow (WorkflowAdmin, WorkflowContextViewer, WorkflowDeveloper, et al.) giving me pretty much all access, is assigned to my user. Which it is.
00:38:45 Taking a step back to look at the documentation at the top of this API endpoint, we see this:
Roles permitted to execute this operation: - Global roles: WorkflowViewer, WorkflowAdmin, WorkflowDeveloper - Scope: WORKFLOW_DEFINITION_GET
The thing is, I have those roles listed, but
I am not involved in the authentication flow here.
00:40:40 Checking the SAP Help Portal, we see there are two sections relating to accessing workflow APIs using OAuth:
While the “Authorization Code Grant” flow would involve me as a participant in the flow, what’s being used in the API Hub environment and what it does for us, is the “Client Credentials Grant” flow. This doesn’t involve any request to an end user (me) to supply credentials (my email address and password), so how could any roles assigned to me be relevant here? They’re not.
The “Client Credentials Grant” flow is something we’ve actually seen - and used - before. It involves requesting an access token, using the client ID and secret, and then using that token as a bearer token in an authentication header when making the actual API calls. To refresh your memory, you might want to check out the annotation blog post for, or recording of,
Ep.51 - More fun with Business Rules API and OAuth.
00:43:00 In this flow, it’s not any global roles that are relevant - it’s the
scopes that are relevant, the scopes that are, in the form of so-called “authorities”, assigned to the service instance. And the scope that’s required in this particular case is, as we now know, “WORKFLOW_DEFINITION_GET”.
00:44:20 So to solve the authentication issue we have, we need to add that scope to the service instance. We do that using the command line CF CLI
cf
.
First, let’s see what service keys there are:
-> cf service-keys workflow
Getting keys for service instance workflow as dj.adams@sap.com...
name
OrderProcess-workflow-credentials
SimpleWorkflow--workflow-credentials
Now, let’s have a look at the second one specifically:
-> cf service-key workflow SimpleWorkflow-workflow-credentials
Getting key SimpleWorkflow-workflow-credentials for service instance workflow as dj.adams@sap.com...
{
"content_endpoint": "https://api.workflow-sap.cfapps.eu10.hana.ondemand.com/workflow-deploy/rest/internal/v1",
"endpoints": {
"workflow_odata_url": "https://api.workflow-sap.cfapps.eu10.hana.ondemand.com/workflow-service/odata",
"workflow_rest_url": "https://api.workflow-sap.cfapps.eu10.hana.ondemand.com/workflow-service/rest"
},
"html5-apps-repo": {
"app_host_id": "1365363a-6e04-4f43-876a-67b81f32306e,1a5b93af-f1af-4acf-aee0-8c6cc8d3f315,8964e911-e35d-4cfd-972e-08e681a2df0f,9ea7410f-80ea-4b19-bbf0-4fca238ef098"
},
"saasregistryappname": "workflow",
"sap.cloud.service": "com.sap.bpm.workflow",
"uaa": {
"apiurl": "https://api.authentication.eu10.hana.ondemand.com",
"clientid": "sb-clone-9316c623-329a-4b3f-aacd-13cf03...",
"clientsecret": "8f72614b-db26-42dd-9293-6139b1ffcecc$P-zEsLD...",
"identityzone": "i347491trial",
"identityzoneid": "119daf21-ae67-4ced-9bd3-5cddb5...",
"sburl": "https://internal-xsuaa.authentication.eu10.hana.ondemand.com",
"tenantid": "119daf21-ae67-4ced-9bd3-5cddb5...",
"tenantmode": "dedicated",
"uaadomain": "authentication.eu10.hana.ondemand.com",
"url": "https://i347491trial.authentication.eu10.hana.ondemand.com",
"verificationkey": "-----BEGIN PUBLIC KEY-----MIICIjANBgkqhkiG9w0BAQEFAAOCAg8AMIICCgKCAgEAwThn6OO9kj0bchkOGkqYBnV1dQ3zU/xtj7Kj7nDd8nyRMcEWCtVzrzjzh- [ ] sRhlrzlRIEY82wRAZNGKMnw7cvCwNixcfcDJnjzgr2pJ+5/yDZUc0IXXyIWPZD+XdL+0EogC3d4+fqyvg/BF/F0t2hKHWr/UTXE6zrGhBKaL0d8rKfYd6olGWigFd+3+24CKI14zWVxUBtC+P9Fhngc9DRzkXqhxOK/EKn0HzSgotf5duq6Tmk9DCNM4sLW4+ERc6xzrgbeEexakabvax/Az9WZ4qhwgw+fwIhKIC7WLwCEJaRsW4m7NKkv+eJR2LKYesuQ9SVAJ3EXV86RwdnH4uAv7lQHsKURPVAQBlranSqyQu0EXs2N9OlWTxe+FyNkIvyZvoLrZl/CdlYc8AKxRm5rn2/88nkrYQ0XZSrnICM5FRWgVF2hn5KfZGwtBN85/D4Yck6B3ocMfyX7e4URUm9lRPQFUJGTXaZnEIge0R159HUwhTN1HvyXrs6uT1ZZmW+c3p47dw1+LmUf/hIf8zd+uvHQjIeHEJqxjqfyA8yqAFKRHKVFrwnwdMHIsRap2EKBhHMfeVf0P2th5C9MggYoGCvdIaIUgMBX3TtCdvGrcWML7hnyS2zkrlA8SoKJnRcRF2KxWKs355FhpHpzqyZflO5l98+O8wO...==-----END PUBLIC KEY-----",
"xsappname": "clone-9316c623-329a-4b3f-aacd-13cf030e0216!b35963|workflow!b10150"
}
}
00:45:35 This is great, and we’ll use this shortly, but for now, we want to add that scope to the instance.
We can do this with the CF cli again, like this:
-> cf update-service workflow -c '{"authorities": ["WORKFLOW_DEFINITION_GET"]}'
Updating service instance workflow as dj.adams@sap.com...
OK
(Note that this list is absolute, not relative, so whatever is specified in the
[...]
list
replaces what’s already there.)
00:47:40 Now when we try the API call in the API Hub again, it works! We have the appropriate authorisations to list the workflow definitions, and we see the result:
[
{
"id": "simpleworkflow",
"version": "2",
"name": "simpleworkflow",
"createdBy": "sb-clone-9316c623-329a-4b3f-aacd-13cf030e0216!b35963|workflow!b10150",
"createdAt": "2020-03-13T10:49:17.018Z",
"jobs": []
},
{
"id": "orderprocess",
"version": "1",
"name": "orderprocess",
"createdBy": "sb-clone-9316c623-329a-4b3f-aacd-13cf030e0216!b35963|workflow!b10150",
"createdAt": "2020-03-10T11:01:02.084Z",
"jobs": []
}
]
This also suggests that each time we make an API call in the API Hub, it probably goes through the entire Client Credentials Grant flow (otherwise, if it had reused the token, it would still have failed. In other words, the token returned reflects what’s allowed).
00:49:00 31a8856c1f6f4bcfa7f3d890a0b88fd2 mentions something in the chat that is a bit of a hint which reminds me of what we’ve seen before but sort of ignored until now. That is, when creating a workflow service instance in the Cockpit with the “New Instance” button, we go through the wizard and there’s a “Specify Parameters (Optional)” step … that has a default value that looks like this:
{
"authorities": []
}
In other words, when we created the workflow instance a while back, we created it and
explicitly (albeit unknowingly, at the time) said that it should have (from an Client Credentials Grant OAuth flow perspective) no authorities, no scopes. So it’s no wonder that when we first tried to make the API call to
/v1/workflow-definitions
in the API Hub, we were greeted by a 403 response!
00:51:00 We double check our understanding by removing the “WORKFLOW_DEFINITION_GET” scope again from the instance, like this:
-> cf update-service workflow -c '{"authorities":[]}'
Updating service instance workflow as dj.adams@sap.com...
and noting that on retrying this API call, we get the 403 error again. Nice!
00:53:20 In the final stage of this episode, there’s just enough time to dig in a little bit to how this Client Credentials Grant flow works … and in fact, regulars of this live stream series already know how it works, because we’ve used it before, albeit in a different context (the Business Rules CF API).
00:54:00 A short digression (digress? moi?) on what I think is a nicer way to enter JSON to a command. Instead of typing this:
-> cf update-service workflow -c '{"authorities": ["WORKFLOW_DEFINITION_GET"]}'
which involves lots of commas, double quotes and so on, we can type this:
-> cf update-service workflow -c $(yq r -j -)
authorities:
- WORKFLOW_DEFINITION_GET
which uses the
yq
tool to convert the YAML to JSON. Now I like that!
00:55:30 To work through this, we start at the command line, bearing in mind the prerequisites described in the documentation for
Access Workflow APIs using OAuth 2.0 Authentication (Client Credentials Grant).
We can use the command line to our advantage (remember,
#TheFutureIsTerminal!), as follows.
First, we grab the service keys data for the workflow instance, specifically for the
SimpleWorkflow-workflow-credentials
service key:
cf service-key workflow SimpleWorkflow-workflow-credentials > keys.json
At this point we note that the output from various
cf
commands is really rather messy, in that we get headings as well as data, which we have to tidy up each time. Ugh. Anyway.
We can now grab service key values using
jq
, which is nice; and we take a closer look at all the keys including the
endpoints
and
uaa
nodes that we saw earlier.
00:58:20 Given that one of the philosophies of the Unix command line is “small pieces loosely joined”, we create a simple throwaway shell function (called ‘skv’, for ‘service key value’) that gives us some convenience for dipping into this rich seam of values:
-> skv () { jq -r "$1" keys.json; }
Now we can use this very comfortably, like this:
-> skv .uaa.clientid
sb-clone-9316c623-329a-4b3f-aacd-13cf03...
00:59:30 At this stage, we turn to the next part of the documentation which instructs us to request an access token from the OAuth 2.0 endpoint, which we do, also updating the
.netrc
values for our authentication endpoint as we want to tell
curl
to use
.netrc
.
This is what we end up doing to make the access token request and store the result in a file:
-> curl -n -v -X POST "$(skv .uaa.url)/oauth/token?grant_type=client_credentials" > token.json
01:02:50 Now we have the access token to use in an authorisation header for the actual API call, as we’ve done before, and we get the result we’re expecting!
-> curl -H "Authorization: Bearer $(jq -r .access_token token.json)" \
> "$(skv .endpoints.workflow_rest_url)/v1/workflow-definitions" | jq .
[
{
"id": "simpleworkflow",
"version": "2",
"name": "simpleworkflow",
"createdBy": "sb-clone-9316c623-329a-4b3f-aacd-13cf030e0216!b35963|workflow!b10150",
"createdAt": "2020-03-13T10:49:17.018Z",
"jobs": []
},
{
"id": "orderprocess",
"version": "1",
"name": "orderprocess",
"createdBy": "sb-clone-9316c623-329a-4b3f-aacd-13cf030e0216!b35963|workflow!b10150",
"createdAt": "2020-03-10T11:01:02.084Z",
"jobs": []
}
]
And that’s it for this episode. Hopefully that makes some sense, there’s a lot more to discover, especially the other flow which is the Authorisation Code Grant type, which we’ll try next time.
Just before we finish,
iinside points out that there’s also
cf curl
which is definitely worth looking into for more predictable and parseable output from
cf
. Thanks Max!