Integrating AWS (DynamoDB) with SAP CPI

former_member86721 · ‎02-19-2021

As the world is moving towards the Cloud from On-Premise, the need of integrating with Cloud service provider raises. AWS is one of the major cloud service provider and hence, the integration of SAP (or) other third party systems with the services in AWS is needed. AWS has n number of dedicated services and each one serves it own purposes. As an integration tool, SAP CPI should be able to integrate with AWS services too.

One of the easiest and existing possibility to integrate with AWS through SAP CPI is via Advantco Adapter. SAP has collaborated with Advantco and they have published an Amazon WS Adapter which can be installed in SAP CPI to enable the integration with AWS. The Adapter installation needs to be done via Eclipse Oxygen. Installation guide and User guide has been published by SAP in the discover section of SAP CPI. We can able to see the list of documents attached in the document section.

Advantco adapter is available in both sender and Receiver. The sender adapter enables us to integrate with AWS services AWS S3 and AWS SQS whereas the receiver adapter enables us to integrate with services S3, SNS, SQS and SWF.

As the Advantco adapter enables us to integrate with only few major services in AWS, we have to think out of the box to integrate with other services.

In this document, we are going to see on how to integrate with one of the other AWS services (Dynamo DB) which is not available with Advantco Adapter.

Amazon DynamoDB is a fully managed NoSQL database service that provides fast and predictable performance with seamless scalability.

Requirement:

Integrating SAP CPI with AWS services.

Solution:

In the first few sections of our blog, we will see how to set up a DynamoDB in AWS account which we will be integrating from SAP CPI.

Followed by that, we will look into see how to access the Dynamo DB from Postman.

Finally, we will see the steps to be followed in the SAP CPI to achieve our need.

AWS Trial Account Setup:

Login to https://aws.amazon.com/account/sign-up and create a free account with your Gmail.

Once the account is created, navigate to the same page and Sign In to the console with your user name and password by selecting the root user.

Creating DynamoDB:

Navigate to DynamoDB from AWS console Home page --> Create table --> Enter Table name and Primary key.

Now, the table will be created with the primary key field alone. You can insert more columns through the Create Item option. Anyways, DynamoDB is capable of creating the columns at runtime if it has received any create request with column name not described in the schema.

Creating IAM User with roles to access DynamoDB:

Navigate to console home screen and search for service “IAM Console”

Navigate to Users --> Add user --> Input your user name --> Select Access type as Programmatic access à Click Next.

Select “Attach existing policies directly” --> Search DynamoDB --> Select “AmazonDynamoDBFullAccess”(Allows to perform CRUD operations on DynamoDB) -->Click Next --> Skip next screen and then, Click Next.

Click Next and you could see a Success message for creation of user along with the Access key and Secret access key.

This Access key and Secret access key is needed for authentication while communicating to AWS either via Postman (or) CPI.

Save these creds for later reference (or) you can download the .csv file as well.

Connecting DynamoDB from Postman:

In this topic, we are going to see how we can connect DynamoDB and perform CRUD operation from Postman.

Create post request from Postman and enter the request URL as “https://dynamodb.{{regionname}}.amazonaws.com/”.

The above URL is the standard URL to access the DynamoDB where the region name is the region in which our DynamoDB resource has been created. In our case, its “ap-south-1”

Choose the Authentication method as “AWS Signature”. Input the “AccessKey” and “SecretKey” which we have downloaded in the previous step.

Select Advanced option. Input your region name in the field “AWS Region” and Input “dynamodb” in the field “Service Name”.

Add a custom header which has been highlighted in the screenshot where the header “Content-Type” specifies the request type as “json” and the header “X-Amz-Target” specifies which one of the CRUD operation need to be performed.

We have to post the data in the json format in the specific structure. The structure to perform the Create/Put operation has been shown in the screenshot.

Now, we are all set to post this message to AWS and let we see whether this row has got created in the DynamoDB.

Click Send and check whether you have got 200 status. If yes, logon to AWS DynamoDB and you can observe that the entry has been added.

In the similar way, you can perform all the CRUD operations. The only action needed is, the input body and the header “X-Amz-Target” needs to be altered accordingly.

The header value for the “X-Amz-Target” for the different CRUD operation are as follows.
- Create - DynamoDB_20120810.PutItem
- Get - DynamoDB_20120810.GetItem
- Update - DynamoDB_20120810.UpdateItem
- Delete - DynamoDB_20120810.DeleteItem

The body structure for different CRUD operation has been shown below.

Connecting DynamoDB from SAP CPI

Get: Update: Delete:

Connecting DynamoDB from SAP CPI:

We are going to follow the same procedure in CPI like we did in Postman. But, the Postman has the inbuild capacity to calculate the “Host” and “AWS Signature” based on the inputs that we have given.

These two things need to be carried out exclusively in CPI to integrate the DynamoDB. I have achieved this via Groovy.

The sender http is to trigger a integration flow. In the content modifier, hardcode the body we used in the postman. This can be populated dynamically either through Mapping (or) with content modifier itself in our real time case.

Add an header as shown in the below screenshot

We have added the necessary headers and body with the help of Content modifier.

The left out part is the Authentication. AWS supports only the AWS signature authentication which is not available by default in any of the adapter in CPI. So, we are going to calculate this AWS signature through Groovy script and place it as authorization header.

Add a Groovy script and place the below code from snippet.

import com.sap.gateway.ip.core.customdev.util.Message;

import java.util.HashMap;

import java.text.DateFormat;

import java.text.SimpleDateFormat;

import java.security.MessageDigest;

import javax.crypto.Mac;

import javax.crypto.spec.SecretKeySpec;

import com.sap.it.api.ITApiFactory;

import com.sap.it.api.securestore.SecureStoreService;

import com.sap.it.api.securestore.UserCredential;



def Message processData(Message message) {

    //************* REQUEST VALUES *************

    String method = 'POST';

     def headers = message.getHeaders();

     String host = headers.get("Host");

     String endpoint = 'https://'+ host+'/'

    // POST requests use a content type header. For DynamoDB, the content is JSON.

    String amz_target = headers.get("X-Amz-Target");

    String content_type = headers.get("Content-Type");

    

    // Request parameters for Create/Update new item--passed in a JSON block.

    String body = message.getBody(java.lang.String) as String;

    String request_parameters = body;

    

   // Read AWS access key from security artifacts. 

   def secureStoreService = ITApiFactory.getApi(SecureStoreService.class, null);

   //AWS_ACCESS_KEY

   def aws_access_key = secureStoreService.getUserCredential("AWS_ACCESS_KEY");

   if (aws_access_key == null){

      throw new IllegalStateException("No credential found for alias 'AWS_ACCESS_KEY' ");

   }

   //AWS_SECRET_KEY

  def aws_secret_key = secureStoreService.getUserCredential("AWS_SECRET_KEY");

   if (aws_secret_key == null){

      throw new IllegalStateException("No credential found for alias 'AWS_SECRET_KEY' ");

   }

   //AWS_REGION

   def aws_region = secureStoreService.getUserCredential("AWS_REGION");

   if (aws_region == null){

      throw new IllegalStateException("No credential found for alias 'AWS_REGION' ");

   }

   

   //AWS_SERVICE_NAME

   def aws_service = secureStoreService.getUserCredential("AWS_SERVICE_NAME");

   if (aws_service == null){

      throw new IllegalStateException("No credential found for alias 'AWS_SERVICE_NAME' ");

   }

   

    String access_key = new String(aws_access_key.getPassword());

    String secret_key = new String(aws_secret_key.getPassword());

    String service =   new String(aws_service.getPassword());

    String region =   new String(aws_region.getPassword());

   

    // Create a date for headers and the credential string

    def date = new Date();

    DateFormat dateFormat = new SimpleDateFormat("yyyyMMdd'T'HHmmss'Z'");

    dateFormat.setTimeZone(TimeZone.getTimeZone("UTC"));//server timezone

    String amz_date = dateFormat.format(date);

    dateFormat = new SimpleDateFormat("yyyyMMdd");

    dateFormat.setTimeZone(TimeZone.getTimeZone("UTC"));//server timezone

    String date_stamp = dateFormat.format(date);

    

    // ************* TASK 1: CREATE A CANONICAL REQUEST *************

    // http://docs.aws.amazon.com/general/latest/gr/sigv4-create-canonical-request.html

    

    // Step 1 is to define the verb (GET, POST, etc.)--already done.

    

    // Step 2: Create canonical URI--the part of the URI from domain to query 

    // string (use '/' if no path)

    String canonical_uri = '/';

    

    // Step 3: Create the canonical query string. In this example, request

    // parameters are passed in the body of the request and the query string is blank.

    String canonical_querystring = '';

    

    // Step 4: Create the canonical headers. Header names must be trimmed

    // and lowercase, and sorted in code point order from low to high. Note that there is a trailing \n.

    String canonical_headers = 'content-type:' + content_type + '\n' + 'host:' + host + '\n' + 'x-amz-date:' + amz_date + '\n' + 'x-amz-target:' + amz_target + '\n';

    

    // Step 5: Create the list of signed headers. This lists the headers

    // in the canonical_headers list, delimited with ";" and in alpha order.

    // Note: The request can include any headers; canonical_headers and

    // signed_headers include those that you want to be included in the

    // hash of the request. "Host" and "x-amz-date" are always required.

    // For DynamoDB, content-type and x-amz-target are also required.

    String signed_headers = 'content-type;host;x-amz-date;x-amz-target';

    

    // Step 6: Create payload hash. In this example, the payload (body of the request) contains the request parameters.

    String payload_hash = generateHex(request_parameters);

    

    // Step 7: Combine elements to create canonical request

    String canonical_request = method + '\n' + canonical_uri + '\n' + canonical_querystring + '\n' + canonical_headers + '\n' + signed_headers + '\n' + payload_hash;

    

    // ************* TASK 2: CREATE THE STRING TO SIGN*************

    // Match the algorithm to the hashing algorithm you use, either SHA-1 or SHA-256 (recommended)

    String algorithm = 'AWS4-HMAC-SHA256';

    String credential_scope = date_stamp + '/' + region + '/' + service + '/' + 'aws4_request';

    String string_to_sign = algorithm + '\n' +  amz_date + '\n' +  credential_scope + '\n' +  generateHex(canonical_request);

    

    // ************* TASK 3: CALCULATE THE SIGNATURE *************

    // Create the signing key using the function defined above.

    byte[] signing_key = getSignatureKey(secret_key, date_stamp, region, service);

    

    // Sign the string_to_sign using the signing_key

    byte[] signature = HmacSHA256(string_to_sign,signing_key);

    

     /* Step 3.2.1 Encode signature (byte[]) to Hex */

    String strHexSignature = bytesToHex(signature);

    

    // ************* TASK 4: ADD SIGNING INFORMATION TO THE REQUEST *************

    // Put the signature information in a header named Authorization.

    String authorization_header = algorithm + ' ' + 'Credential=' + access_key + '/' + credential_scope + ', ' +  'SignedHeaders=' + signed_headers + ', ' + 'Signature=' + strHexSignature;

    

    // For DynamoDB, the request can include any headers, but MUST include "host", "x-amz-date",

    // "x-amz-target", "content-type", and "Authorization". Except for the authorization

    // header, the headers must be included in the canonical_headers and signed_headers values, as

    // noted earlier. Order here is not significant.



    // set X-Amz-Date Header and Authorization Header

    message.setHeader("X-Amz-Date",amz_date);

    message.setHeader("Authorization", authorization_header);

      

    return message;

}



String bytesToHex(byte[] bytes) {

    char[] hexArray = "0123456789ABCDEF".toCharArray();            

    char[] hexChars = new char[bytes.length * 2];

    for (int j = 0; j < bytes.length; j++) {

        int v = bytes[j] & 0xFF;

        hexChars[j * 2] = hexArray[v >>> 4];

        hexChars[j * 2 + 1] = hexArray[v & 0x0F];

    }

    return new String(hexChars).toLowerCase();

} 



String generateHex(String data) {

    MessageDigest messageDigest;



    messageDigest = MessageDigest.getInstance("SHA-256");

    messageDigest.update(data.getBytes("UTF-8"));

    byte[] digest = messageDigest.digest();

    return String.format("%064x", new java.math.BigInteger(1, digest));

}



byte[] HmacSHA256(String data, byte[] key) throws Exception {

    String algorithm="HmacSHA256";

    Mac mac = Mac.getInstance(algorithm);

    mac.init(new SecretKeySpec(key, algorithm));

    return mac.doFinal(data.getBytes("UTF8"));

}



byte[] getSignatureKey(String key, String dateStamp, String regionName, String serviceName) throws Exception {

    byte[] kSecret = ("AWS4" + key).getBytes("UTF8");

    byte[] kDate = HmacSHA256(dateStamp, kSecret);

    byte[] kRegion = HmacSHA256(regionName, kDate);

    byte[] kService = HmacSHA256(serviceName, kRegion);

    byte[] kSigning = HmacSHA256("aws4_request", kService);

    return kSigning;

}

You can observe that, there were four inputs which were required predominantly in the groovy script to proceed with the further steps.

These four parameters is actually the same value what we have given in the postman. Create a secure parameter for each one respectively with the same naming conventions what you have declared in the groovy script.

Navigate to Monitoring Overview à Manage Security à Security Material à “Create” à “Secure Parameter”.

Now, we are almost set. The related headers are created and AWS signature has been calculated and set as header in the groovy script.

Now, make a call to AWS DynamoDB via http adapter with the help of request reply. Then, deploy the IFlow and trigger from Postman to check whether the things are working fine.

*** While Testing the flow for first time, you may get into error as AWS DynamoDB host was not trusted by default in SAP CPI. In this case, download the certificate and add it in SAP CPI trust store.

Conclusion:

We can make other CRUD operation possible via the same process but the header and body needs to be changed accordingly in the similar way we did it in the postman.

We doesn’t need any special adapters like Advantco to handle the integration with AWS via CPI. It’s made possible with simple HTTP adapter with the combination of groovy script.

DynamoDB can be used for lookups if we have AWS as well in our working environment. This also make more sense in Standard edition of CPI where in the Storage are limited and Open connectors are not available.

Also, we can do similar kind of integration with other services in AWS as well which i will try to cover in the further blogs.

References:

https://docs.aws.amazon.com/amazondynamodb/latest/APIReference/API_Operations_Amazon_DynamoDB.html

https://docs.aws.amazon.com/general/latest/gr/sigv4-create-canonical-request.html

Integrating AWS (DynamoDB) with SAP CPI

SAP PI for Beginners

ABAP 7.40 Quick Reference

Fiori: technical installation and configuration of one app from A - Z