Query on CMIS Repository #2

[lalitmohan]

Welcome back! In my first blog post(click here), we learned about CMIS standards to comprehend the possibilities of the SAP Document Management Service with a step-by-step process for creating an instance and establishing a session to query the Repository by browsing it in the CMIS workbench.

This Workbench, a CMIS Client, is distributed as a standalone application to explore a working CMIS implementation. Nevertheless, we have popular libraries that we may use to create clients to work on Repositories.

Please look over the standard client libraries.

This blog post aims to create a custom CMIS client to connect with the SAP Document Management Service instance and establish a session to execute a query on the Repository. We will also try to understand the different OAuth 2.0 flows (or grants) to retrieve an Access Token for accessing the instance.

Prerequisite

1. It would be ideal if you had already followed and completed the steps mentioned in my previous article(click here) before moving on to this one. 


2. You need to have a server-side javascript runtime environment called NodeJS. Download and install at least version 17 if you don't already have it.

There are numerous command-line snippets in this step-by-step blog post that must be placed into a command-line window. Any of the samples provided for macOS/Linux or without a platform description can be run in either the bash or zsh shells, which are these platforms' default shells.
Windows users are advised to use the Git BASH, which is a component of the Git for Windows installation and comprises the fundamental UNIX command-line tools, as an alternative.

Let's proceed step by step

Before we start, if you haven't already, create a new directory called "sdmapps." Open a terminal on your computer and type in the following:

mkdir sdmapps sdmapps/node

OR Clone the repository and fetch node branch as shown below:

git clone -b node https://github.com/alphageek7443/sdmapps

Please note that if you clone the repository, you can skip to Step 6. You only need to consider copying your service key file "default-key.json" to the src/keys folder.

To start, change to the node directory inside the sdmapps:

cd sdmapps/node

The OAuth 2.0 authorization framework allows a third-party application to request limited access to an HTTP service on behalf of a resource owner by orchestrating an approval interaction between the resource owner and the HTTP service or by allowing the third-party application to request access on its behalf.

An application must perform the steps to get resource access authorization. The authorization framework provides several grant types to address different scenarios:

First Scenario (Client credential flow)

The Client credential flow is one of the most straightforward flows we generally use to get the API response using the service key. With machine-to-machine (M2M) applications, such as CLIs, daemons, or services running on your back-end, the system authenticates and authorizes the app rather than a user.

Step 1: Create an Application

Create a new dedicated directory for your Node.js application called "clientflow".

mkdir clientflow

To start the application setup, change to the "clientflow" directory and execute “npm init” in the command line. This will walk you through creating a package.json file.

cd clientflow

npm init

Once the above command is executed, Terminal will ask you to provide some information to initialize your package.json file inside your project folder. An important field is a name; you can provide "clientflow" as a name.

package name: (clientflow)

version: (1.0.0)

description:

entry point: (index.js)

test command:

git repository:

keywords:

author:

license: (ISC)

We have initialized the project now; let's install the necessary npm packages using the following command:

npm install cmis express node-fetch

Every time you hit the space bar after the first folder name in a mkdir command, Terminal understands it as a new mkdir command. Create required folders by using the following command:

mkdir web src src/keys src/libs

Step 2: Get the Service key

Ensure you have already logged in to your BTP subaccount and created a Document Management Service, Integration option service instance. Also, check you have a service key to access it. If not, please follow my blog post(click here).

Download the Service key and redirect the output to a "default-key.json" file by using the following command:

cf service-key sdmsrv default-key | sed 1d > src/keys/default-key.json

open VS Code from the terminal by typing:

code .

Update script tag in package.json and add "type": "module"

"type": "module",

  "scripts": {

    "start": "node index.js"

  },

Now your application looks similar to the below screenshot:

Step 3: Get the Access Token

Create a script file auth.js under a directory called libs inside the src folder and paste the following javascript code to authenticate the application by passing Client ID and Client Secret. Authorization Server validates the request and responds with an access token. 

import fetch from 'node-fetch';



class OAuth{

    constructor(clientId, clientSecret,authUrl){

        this.clientId = clientId;

        this.clientSecret = clientSecret;

        this.authUrl = authUrl;

    }



    async getToken(){

        return await this.fetchToken().then(res => res);

    }



     async fetchToken(){

        return await fetch(this.authUrl+

            '/oauth/token?grant_type=client_credentials',{

                headers: { 'Authorization': 'Basic ' + Buffer.from(

                    `${this.clientId}:${this.clientSecret}`,

                     'binary').toString('base64') }})

            .then(res=> res.json())

            .then(res => res.access_token)

            .catch(error => console.error(error))

    }

}



export default OAuth

 

Step 4: Query on CMIS Repository

Create a script file index.js under the application root directory and paste the following JavaScript code to create an instance of an Express application. Set up a route with path "/documents" which includes a callback to connect with the CMIS session to get the document list and send it as a response. 

import express from 'express'

import cmis from 'cmis';



import keys from './src/keys/default-key.json' assert { type: 'json' };

const {uaa:{clientid},uaa:{clientsecret},uaa:{url},uri,...other} = keys 



import OAuth from './src/libs/auth.js';



const app = express();

const oauth = new OAuth(clientid,clientsecret,url);

const port = process.env.PORT || 3004;



app.use(express.static("web"));



app.get('/token', async(req,res) => {

    var token = await oauth.getToken()

    res.send(token)

})



app.get('/documents', async(req,res) =>{

    var session = new cmis.CmisSession(uri+'browser');

    var token = await oauth.getToken()

    session.setToken(token).loadRepositories()

    .then(() => session.query("select * from cmis:document"))

    .then(data => res.send(data))

})



app.listen(port, () => { console.log(`Explore http://localhost:${port}`) });

Step 5: Add HTML Page

Create a static HTML page under the web directory and paste the below HTML content with the relative link to access the recently created request methods.

<!DOCTYPE html>

<html lang="en">

<head>

    <meta charset="UTF-8">

    <meta http-equiv="X-UA-Compatible" content="IE=edge">

    <meta name="viewport" content="width=device-width, initial-scale=1.0">

    <title>Documents</title>

</head>

<body>

    <h1>Document Management Service, Integration options</h1>

    <h2>Custom CMIS Client</h2>



    <a href="https://blogs.sap.com/token">Get Token</a><br/>

    <a href="https://blogs.sap.com/documents">Get Documents</a>

</body>

</html>

 

Step 6: Run your application 

After we have added all these files, we are ready to run our first application using the following command:

npm run start

If you are using the VScode editor, your application looks similar to the screenshot below.



To access the URL in the browser, click on the link that appears in the integrated Terminal of VSCode.

Now Click on the "Get Documents" to get the list of documents in the CMIS Repository.

Second Scenario (Authorization code flow)

The Authorization Code flow is the most commonly used flow, designed especially for server-side applications that can maintain the confidentiality of their Client Secrets. It's one of the redirection-based flows.

Standard web applications are server-side applications where the source code is not publicly exposed; they can use the Authorization Code Flow, which exchanges an Authorization Code for a token. Your app must be server-side because, during this exchange, you also pass your application's client Secret, which must always be kept secure, and you will have to store it in your client.

Step 7: Copy content and Create a New Application

Goto the Terminal again, If still, you are in the "clientflow" directory, change your directory one step back to "node" using the following command:

cd ..

Please note that if you clone the repository, you can skip to Step 10. You only need to consider copying your service key file "default-key.json" to the src/keys folder.

Create a Copy of the "clientflow" directory with the name "authorizeflow" having the same content and change the directory to "authorizeflow":

cp -R clientflow authorizeflow

cd authorizeflow

Install the dependency needed for this section, then launch VS Code using the following command:

npm install universal-cookie 

code .

Open package.json and change the name property "name" with "authorizeflow". Your application looks like the below screenshot.

Step 8: Get the Access Token

Update the script file auth.js under a directory called libs inside the src folder and paste the following boilerplate(standard) code and save.

import Cookies from "universal-cookie";

import fetch from "node-fetch";

const cookieName = "SDMCookie";

const CALLBACK_URI = "/myCallbackURI";



class OAuth{

    constructor(clientId,clientSecret,authUrl){

        this.clientId = clientId;

        this.clientSecret = clientSecret;

        this.authUrl = authUrl;

    }



    getToken(req){

        const cookies = new Cookies(req.headers.cookie);

        return cookies.get(cookieName);

    }



    async fetchToken(code,redirectUri) {

        const params = new URLSearchParams();

        params.set('client_id', this.clientId);

        params.set('client_secret', this.clientSecret);

        params.set('code', code);

        params.set('redirect_uri', redirectUri);

        params.set('grant_type', 'authorization_code');



        const response = await fetch(`${this.authUrl}/oauth/token`,{

            method: 'post',

            headers: {

                'Content-Type': 'application/x-www-form-urlencoded',

                'Accept': 'application/json'

            },

            body: params

        });



        const json = await response.json();

        return json.access_token;

    }



    getMiddleware() {

        return async (req, res, next) => {

            const redirectUri = `${req.protocol}://${

                req.get("host")}${CALLBACK_URI}`;

            if (req.url.startsWith(CALLBACK_URI)) {

                const code = req.query.code;

                if (code) {

                    const token = await this.fetchToken(code, redirectUri);

                    res.cookie(cookieName, token, { 

                        maxAge: 1000 * 60 * 120, httpOnly: true,  path: "/", });

                }

                res.redirect("/");

            } else if (!this.getToken(req)) {

                res.redirect(`${this.authUrl}/oauth/authorize?client_id=${

                    encodeURIComponent(this.clientId)}&redirect_uri=${

                        encodeURIComponent(redirectUri)}&response_type=code`);

            } else {

                next();

            }

        };

    }

}



export default OAuth

As you can see, this boilerplate authentication code is used for logging the user using single sign-on and for the user’s convenience; it also saves the obtained access token as a cookie so that it can be used until it expires.

Step 9: Query on CMIS Repository

Update the script file index.js under the application root directory and paste the following JavaScript code to create an instance of an Express application. Set up a route with path “/documents” and include a callback to connect with the CMIS session and send the document list as a response.

import express from 'express';

import cmis from "cmis";



import keys from './src/keys/default-key.json' assert { type: 'json' };

const {uaa:{clientid},uaa:{clientsecret},uaa:{url},uri,...other} = keys 





import OAuth from "./src/libs/auth.js";



const app = express()

const oauth = new OAuth(clientid,clientsecret,url);

const port = process.env.PORT || 3004;



app.use(oauth.getMiddleware())

app.use(express.static("web"));



app.get('/token', async(req,res) => {

    var token = await oauth.getToken(req)

    res.send(token)

});



app.get('/documents', async(req, res) => {

    const accessToken = oauth.getToken(req);

    var session = new cmis.CmisSession(uri+'browser');

    session.setToken(accessToken).loadRepositories()

        .then(() => session.query("select * from cmis:document"))

        .then(data => res.send(data))

        .catch(error => {

            res.send(`${error.response.status} - ${error.message}`)

        })

});



app.listen(port, () => { console.log(`Explore http://localhost:${port}`) });

Step 10: Run your application 

After we have added all these files, we are ready to run our second application using the following command:

npm run start

If you are using the VScode editor, your application looks similar to the screenshot below.



To access the URL in the browser, click on the link that appears in the integrated Terminal of VSCode.
Now click on the "Get Documents" to get the list of documents in the CMIS Repository, but this time you are getting a 403 - Forbidden error.

 

Step 11: Troubleshooting

In the OAuth 2.0 protocol, The client can access protected resources by giving the access token to the resource server. The resource server must check the access token to ensure it is legitimate, that its scope includes the requested resource, and that it has not expired.

Applications request permission to use resources identified by scopes. Scopes are an important concept in OAuth 2.0. They specify exactly the reason for which access to resources may be granted. Acceptable scope values and the resources they relate to depend on the Resource Server.

Let's examine the first scenario (the client credential flow), which was executed successfully and check app scopes by decoding its access token.
Use the following commands to change the directory to "clientflow" and to start the application again.

cd ../clientflow

npm run start

Verify that the application has been granted the permissions required to access API. You need to check the scope claim in the decoded JWT's payload. It should match the necessary permissions for the endpoint being accessed.
Let's decode the access token using any JWT decoder of your choice.

Decoded content reveals that the access token contains the four different scopes for accessing SAP Document Management Service Resources.

scope: <Application identifier>.sdmbusinessadmin

corresponding Role: SDM_BusinessAdmin

Resources: Retentions and Holds on CMIS objects.

 

scope: <Application identifier>.sdmuser

corresponding Role: SDM_User

Resources: Document Management or CMIS APIs.

 

scope: <Application identifier>.sdmmigrationadmin

corresponding Role: SDM_MigrationAdmin

Resources: Migration APIs.

 

scope: <Application identifier>.sdmadmin

corresponding Role: SDM_Admin

Resources: Admin or Repository management APIs.

Please take note of the Application identifier, which we will use to create the new role collection.

Let’s examine the second scenario (Authorization code flow), which is not executed as aspected.
Similarly, Change the directory to “authorizeflow", then start the application again, grab the access token and decode it.

Here, decoded content reveals that the access token contains the scope but has no scope which is required for accessing Document Management Service Resources.

Let’s add them in the next step.

Step 12: Assign a Role Collection to a User

Open SAP BTP Cockpit, Go to the Subaccount, Choose Security -> Role Collections, then Choose the (+) icon to create a new role collection; enter the name SDM Users, Choose to Create.

The new role collection appears in the list but doesn't contain any roles. To add:

1. Choose the "SDM Users" role collection.


2. Choose Edit.


3. Open the value help for Role Name.

Select noted Application Identifier from the dropdown.

1. Choose to Add,


2. then Choose Save.

Your user must be assigned to role collection with the necessary scopes to access the application:

1. Enter the E-Mail Address of your user.


2. Choose Save.

Your browser saves token information from the earlier application version in its cache and cookies, clears them and follow the step to get the new token. After decoding, the newly generated token, you can check if it contains the scope, which includes roles required for accessing the Document Management service instance.

Now, you get the same result as you got in Client flow.

What’s next?

This concludes the blog entry; I hope now you can understand how to build a custom client to access or query on CMIS-compliant Repositories. Here I have also explained how the OAuth 2.0 flows and grants are means of obtaining access tokens that permit an application to access resources and to perform specific actions by the scope granted during the authorization. Follow up on the topics we covered in there in these blog posts:

#3: Use CAP Java SDK to build a basic CAP application to integrate with the SAP Document Management ...

Keep your anticipation high for the upcoming episode, which you will see in great detail.

Stay curious!

Reference & Further Reading

CmisJS
SAP Document Management Service
SAP Document Management Service Authorization
The OAuth 2.0 Authorization Framework