Query on CMIS Repository #1

Blog post series #1 – CMIS standards and a step-by-step procedure for setting up an SAP Document Management Service instance. #2 – Created a unique CMIS client to establish a connection with the SAP Document Management Service instance and launch a session to query the Repository.(click here) #3 – Use CAP Java SDK to build a basic CAP application to integrate with the SAP Document Management Service instance(click here)

You probably weren’t very concerned about your software’s relational database when you built an application that uses one for data persistence. The company using your programme may have a standard database, and you may have preferences, but until you were doing something distinctive, it didn’t matter. 

The standardization of SQL is the only thing that makes developers’ database agnosticism possible. Applications were previously created for a specific relational back end, and switching databases required transferring the code, which was, at best, an expensive endeavor and, at worst, might be completely unworkable.

To modern developers who are used to tools like ODBC and JDBC that can abstract away the specifics of a particular database implementation, the idea of developing apps that only operate with a specific database appears strange. 

The same issue those people facing pre-SQL standardization encountered also existed for developers creating applications that needed to employ Enterprise Content Management (ECM) systems for data persistence: There was an API for each ECM system.

That’s where CMIS comes in.

What is CMIS Open Standard?

Content Management Interoperability Service (CMIS) is a vendor-neutral, language-independent specification for working with ECM systems. Because of this, many companies already offer (or will soon provide) CMIS interfaces for their systems, giving them a significant edge over competing specifications. Furthermore, CMIS client libraries are currently available in various programming languages. Numerous of these are accessible as Open Source implementations under the Apache Chemistry umbrella project.

 

Document Management Service, Integration Option

SAP Document Management Service is exposed using the OASIS standard protocol Content Management Interoperability Service (CMIS). On top of this protocol, it provides a document service client API for simpler consumption. The Apache Chemistry Project offers this API, which is the Open Source library OpenCMIS. Chemistry components are used in-app and server implementations of the Document Management Service, Integration Option.

Customers can connect to their chosen repository using the information from the CMIS repository.

CMIS defines the following primary base types: Document objects, Folder objects, Relationship objects, Policy objects, and Item objects, But Currently, Document Management Service, Integration Option supports only document and folder objects.

The purpose of this blog article is to examine the capabilities of the Document Management Service, Integration Option. To do this, we must first become familiar with CMIS and Apache Chemistry, including how to retrieve repository data and conduct CMIS repository queries.

Prerequisite

You need to have:

1. A valid SAP Business Technology Platform with Document Management Service, Integration option setup. Please refer to this link for the detailed steps.
2. Add the entitlements for Document Management service, repository option(the storage offering of SAP Document Management Service) to the same subaccount where Document Management Service, Integration Option instance is created.
3. The Cloud Foundry command-line tools installed; if not, please follow Install the Cloud Foundry Command Line Interface (CLI)
4. Choose VS Code as an editor. If you don’t have it, go to the Visual Studio Code homepage.
5. Git is the version control system that you need to download files. If you don’t have it, go to Git downloads, pick the installer appropriate for your operating system and install it.
6. Install a Java VM. At least, Java 8 is required. For example, download and install SapMachine 11.

There are numerous command-line snippets in this step-by-step blog post that must be placed into a command-line window. Any of the samples provided for macOS/Linux or without a platform description can be run in either the bash or zsh shells, which are these platforms’ default shells.
Windows users are advised to use the Git BASH, which is a component of the Git for Windows installation and comprises the fundamental UNIX command-line tools, as an alternative.

Let’s proceed step by step

Before we will start first create a new dedicated directory for your development called “sdmapps”. open a terminal on your computer and type in the following:

mkdir sdmapps

OR Clone this repo as shown below:

git clone https://github.com/alphageek7443/sdmapps

To start, change to the “sdmapps” directory.

cd sdmapps

Step 1: Check Service & Available Plan

Next, log in to your BTP Cloud Foundry Account, In the Terminal, type in the following:

cf login -a <URL>

Replace the section <URL> with the API Endpoint which is displayed on the left-hand side of your subaccount Overview page. When prompted, enter your username and password. Once you are logged in, you should see your API endpoint, user, org, and space and then use the following command to check the service and available plan.

cf m -s sdm

Step 2: Create Service Instance 

Run the following command to create the SAP Document Management service instance:

cf cs sdm free sdmsrv

The SAP Document Management Service, we wish to connect with is sdm. free, the service plan we will use, and sdmsrv as the name of the Service Instance.

We can use the following commands to check the status while the creation of the service instance is in progress.

cf service sdmsrv

Step 3: Create Service Key

To create the credentials necessary to interface directly with a service instance we have created, we can utilise service keys. Local clients, applications running in other spaces, or other entities outside of your deployment can use service instances using this service key.

we can use the below command to generate a key for the sdmsrv instance.

cf csk sdmsrv default-key

Once the service key has been created, you will see JSON output containing API endpoint URL and User Account and Authentication (UAA) details By executing the following Command.

cf service-key sdmsrv default-key

In the service key details, we take note of those four property values (I’ve marked them in the screenshot above) clientid, clientsecret, url and Uri.

Step 4: Generate API Authorization Bearer Token

In order to consume any of the SAP Document Management Service instances, we will need an Authorization Bearer Token. We can use any Rest Client to generate a bearer token.

Use the rest-client of your choice. I will use vs code’s rest client plugin which is incredibly simple to use. open VS Code from the terminal by typing:

code .

The REST Client plugin requires just a plain text file with the extension .http or .rest. Create a file with the extension http and Copy and paste the following basic syntax.

### url value inside JSON node from service key. 
@url = <url> 

### clientid value inside JSON node from service key.
@clientid = <clientid>

### clientsecret value inside JSON node from service key.
@clientsecret = <clientsecret> 

### get Access Token
# @name getAccessToken
GET  {{url}}/oauth/token?grant_type=client_credentials HTTP/1.1
Authorization: Basic {{clientid}}:{{clientsecret}} 

Copy url value inside uaa JSON node from service key and append query /oauth/token?grant_type=client_credentials.

In HTTP HEADER authorization use Basic Auth type and copy clientid and clientsecret value inside the uaa JSON node from service key as username and password.

Once you prepared a request, click the Send Request link above the request (this will appear if the file’s language mode is HTTP, by default .http files are like this)

After a successful call, an access token is returned in property access_token. With the access token, it is possible to access the repository. Please take note of it.

Step 5: Create a Repository 

Now Create a request in the same file to connect the instance of Document Management Service, Integration Option to the Internal Repository, which is the Document Management service, repository option, a storage offering of Document Management Service.

### sdm ecmservice uri value inside JSON from service key 
@uri = <uri>

## access token value inside JSON from getAccessToken result 
@token = {{ getAccessToken.response.body.access_token }}

### create Repository
# @name createRepository
POST {{uri}}/rest/v2/repositories HTTP/1.1
Authorization: Bearer {{token}}
Content-Type: application/json

{
  "repository": {
        "displayName": "ZDOC-INT",
        "description": "ZDOC-INT Internal repository",
        "repositoryType": "internal",
        "isVersionEnabled":"true",
        "isVirusScanEnabled":"true",
        "skipVirusScanForLargeFile": "false",
        "hashAlgorithms":"SHA-256"
  }
}

Copy uri value inside uaa JSON node from service key and append /rest/v2/repositories.

In HTTP HEADER authorization use Bearer and copy token value inside JSON from getAccessToken Result.

Again, Once you prepared a request, click the Send Request link above the request in the same file.

After a successful call, a Repository ID is returned in property cmisRepositoryId, which is required to upload a document in the next call. Please take note of it.

Step 6: Add Document 

We have created the repository. Now we can create a document object of the specified type(given by the cmis:objectTypeId property) in the (optionally) specified location.

### Repository ID value inside JSON from createRepository result 
@repositoryId = <repositoryId>

### sdm ecmservice uri value inside JSON from service key
@uri = https://api-sdm-di.cfapps.eu10.hana.ondemand.com

### Upload Document 
# uploadDocument
POST {{uri}}/browser/{{repositoryId}}/root HTTP/1.1
Accept: application/json
DataServiceVersion: 2.0
Authorization: Bearer {{token}}
Content-Length: 726
Content-Type: multipart/form-data; boundary=----WebKitFormBoundary7MA4YWxkTrZu0gW

------WebKitFormBoundary7MA4YWxkTrZu0gW
Content-Disposition: form-data; name="cmisaction"

createDocument
------WebKitFormBoundary7MA4YWxkTrZu0gW
Content-Disposition: form-data; name="propertyId[0]"

cmis:name
------WebKitFormBoundary7MA4YWxkTrZu0gW
Content-Disposition: form-data; name="propertyValue[0]"

invoice-3.pdf
------WebKitFormBoundary7MA4YWxkTrZu0gW
Content-Disposition: form-data; name="propertyId[1]"

cmis:objectTypeId
------WebKitFormBoundary7MA4YWxkTrZu0gW
Content-Disposition: form-data; name="propertyValue[1]"

cmis:document
------WebKitFormBoundary7MA4YWxkTrZu0gW
Content-Disposition: form-data; name="media"; filename="sample-invoice-3.pdf"
Content-Type: application/pdf

< .\docs\sample-invoice-3.pdf
------WebKitFormBoundary7MA4YWxkTrZu0gW--

Again, Once you have prepared a request, click the Send Request link above the request in the same file.

You can add multiple documents to your repository if you update your payload first, like in the screenshot described above.

Step 7: Access The Repository in the CMIS workbench

Download CMIS Workbench using the following command

curl -O https://archive.apache.org/dist/chemistry/opencmis/1.1.0/chemistry-opencmis-workbench-1.1.0-full.zip

Use the following command to unzip the downloaded zip file:

unzip -q chemistry-opencmis-workbench-1.1.0-full.zip -d CMIS-Workbench 

Now change your directory and run the following command

 cd CMIS-Workbench
./workbench.sh

The workbench should start soon. You must enter your connection details as shown in the screenshot. Add the access token to the Username and the API endpoint with the suffix “/browser.” Keep the Password blank. Don’t forget to verify “OAuth 2.0” (Bearer Token). The VS code Rest Client’s access token can be copied. You should be able to connect to your BTP DMS instance after clicking Load Repositories.

Click Login and select the repository you created in the previous step. You will then see all the content under the storage. As a result, you may check to see if your files have successfully been uploaded to your SAP BTP DMS repository, which adheres to the CMIS standard.

 

Step 8: Query on CMIS Repository

Go to the CMIS Workbench now and select Query.

The default SQL that is in the Query pane can be executed. You may see query results with all the document properties by clicking the Query button:

To view the object property values for the query results, take a moment to scroll right in the query output. You might need to make the columns wider to see the column names and values. The columns can be moved to the right or left to alter their order.

What’s next?

This concludes this blog entry. I hope that by being familiar with CMIS and Apache Chemistry, you will be able to evaluate the capabilities of the Document Management Service and Integration Option. Follow up on the topics we covered in there in these blog posts:

#2: create a custom CMIS client to connect with the SAP Document Management Service instance
#3: Use CAP Java SDK to build a basic CAP application to integrate with the SAP Document Management Service instance

Keep your anticipation high for the upcoming episode, which you will see in great detail.

Stay curious!

Reference & Further Reading

Feature Scope Description for SAP Cloud Platform Document Management
User SAP Document Service to store document files
CMIS (Content Management Interoperability Services)