Connect to AzureBlob from PI/PO: Using REST API

Introduction:

In my previous blog post I described how to connect to AzureBlob from CPI using Azure Storage jar. In this article I am going to discuss how to use REST API call from PI/PO to upload a file into AzureBlob container.

Prerequisite:

To start with this development you need the below information from AzureBlob administrator.

1. AzureBlob account name
2. Container name
3. Shared AccessKey
4. Version

Go though this Microsoft document to understand how the REST API for PUT Blob works.

Development of Java Mapping:

We need to create a java mapping to construct the header parameters required for REST API call. The mandatory header parameters are:

Authorization : To construct the Authorization parameters we need to follow the below steps:

1. Construct the String to Sign.
2. Decode the Base64 storage key.
3. Use the HMAC-SHA256 algorithm and the decoded storage key from previous step to compute a hash of the string to sign.
4. Base64 encode the hash and include this in the Authorization header.

Date or x-ms-date : This specifies Coordinated Universal Time (UTC) for the request. The format should be “E, dd MMM yyyy HH:mm:ss”

x-ms-version : This specifies the version of the operation to use for this request.

Content-Length: This specifies the length of the request.

x-ms-blob-type: This specifies the type of blob to create: block blob, page blob, or append blob. In our example we need to use block blob, so I will assign it as BlockBlob.

Content-Type : This specifies content type of the blob. Though it is optional, need to create this header unless you want to use the default type “application/octet-stream”.

Here is the code for Java mapping I used:

package pi.mapping.azureblob;

import java.io.ByteArrayOutputStream;
import java.io.InputStream;
import java.io.OutputStream;
import java.security.InvalidKeyException;
import java.text.SimpleDateFormat;
import java.util.Base64;
import java.util.Date;

import javax.crypto.Mac;
import javax.crypto.spec.SecretKeySpec;
import com.sap.aii.mapping.api.AbstractTransformation;
import com.sap.aii.mapping.api.DynamicConfiguration;
import com.sap.aii.mapping.api.DynamicConfigurationKey;
import com.sap.aii.mapping.api.StreamTransformationException;
import com.sap.aii.mapping.api.TransformationInput;
import com.sap.aii.mapping.api.TransformationOutput;

public class AzureBlobConnect extends AbstractTransformation {
	public String length = "";
	public AzureBlobConnect() {		
	}

	public void transform(TransformationInput input, TransformationOutput output) throws StreamTransformationException {
		try {		
			InputStream in = input.getInputPayload().getInputStream();
			OutputStream out = output.getOutputPayload().getOutputStream();
			// Get parameters defined in the ICo
			String blobtype = input.getInputParameters().getString("Blobtype");
			String version = input.getInputParameters().getString("Version");
			String storageKey = input.getInputParameters().getString("AccessKey");
			String account = input.getInputParameters().getString("AzureAccountName");
			String container = input.getInputParameters().getString("AzureContainer");
			String filename = input.getInputParameters().getString("FileName");
			String contentType = input.getInputParameters().getString("ContentType");
			String addTimeStamp = input.getInputParameters().getString("AddTimeStamp");
			String timeStampPattern = input.getInputParameters().getString("TimeStampPattern");
			
			// Construct Current time in specific format required to be used for REST API call as header
			String pattern = "E, dd MMM yyyy HH:mm:ss";
			SimpleDateFormat simpleDateFormat = new SimpleDateFormat(pattern);
			String timenow = simpleDateFormat.format(new Date()) + " GMT";
			
			// Construct file name
			if (addTimeStamp.equalsIgnoreCase("y")) {
				SimpleDateFormat simpleDateFormatTimeStamp = new SimpleDateFormat(timeStampPattern);
				if (filename.contains(".")) {
					int indexofdot = 0;
					indexofdot = filename.indexOf('.');
					filename = filename.substring(0, indexofdot) + "_" + simpleDateFormatTimeStamp.format(new Date())
							+ "." + filename.substring(indexofdot + 1, filename.length());
				} else
					filename = filename + "_" + simpleDateFormatTimeStamp.format(new Date());

			}
			String len = execute(in, out);
			
			// Construct StringToSign to be used to create Hash
			String StringToSign = "PUT\n\n\n" + len + "\n\n" + contentType + "\n\n\n\n\n\n\nx-ms-blob-type:" + blobtype
					+ "\nx-ms-date:" + timenow + "\nx-ms-version:" + version + "\n/" + account + "/" + container + "/"
					+ filename;
			byte[] secretKey = Base64.getMimeDecoder().decode(storageKey.getBytes());
			byte[] digest = null;
			
			//Construct digest from StringToSign using HmacSHA256 algorithm
			try {
				Mac mac = Mac.getInstance("HmacSHA256");
				SecretKeySpec secretKeySpec = new SecretKeySpec(secretKey, "HmacSHA256");
				mac.init(secretKeySpec);
				digest = mac.doFinal(StringToSign.getBytes());
			} catch (InvalidKeyException e) {
				throw new RuntimeException("Invalid key exception while converting to HMac SHA256");
			}
			//Encode digest to Base64
			String auth = Base64.getMimeEncoder().encodeToString(digest);
			//Create Dynamic configuration headers
			DynamicConfiguration conf = input.getDynamicConfiguration();

			DynamicConfigurationKey key1 = DynamicConfigurationKey.create("http://sap.com/xi/XI/System/REST",
					"XHeaderName1");
			DynamicConfigurationKey key2 = DynamicConfigurationKey.create("http://sap.com/xi/XI/System/REST",
					"XHeaderName2");
			DynamicConfigurationKey key3 = DynamicConfigurationKey.create("http://sap.com/xi/XI/System/REST",
					"XHeaderName3");
			DynamicConfigurationKey key4 = DynamicConfigurationKey.create("http://sap.com/xi/XI/System/REST",
					"XHeaderName4");
			DynamicConfigurationKey key5 = DynamicConfigurationKey.create("http://sap.com/xi/XI/System/REST",
					"XHeaderName5");
			DynamicConfigurationKey key6 = DynamicConfigurationKey.create("http://sap.com/xi/XI/System/REST",
					"XHeaderName6");
			DynamicConfigurationKey key7 = DynamicConfigurationKey.create("http://sap.com/xi/XI/System/REST",
					"XHeaderName7");
			//Put values in header parameters
			conf.put(key1, blobtype);
			conf.put(key2, version);
			conf.put(key3, timenow);
			conf.put(key4, "SharedKey " + account + ":" + auth);
			conf.put(key5, len);
			conf.put(key6, contentType);
			conf.put(key7, filename);
		} catch (Exception e) {
			throw new StreamTransformationException(e.getMessage());
		}
	}

	public String execute(InputStream in, OutputStream out) throws Exception {
		byte[] content = new byte[16384];

		ByteArrayOutputStream byt = new ByteArrayOutputStream();
		int i = 0;
		while ((i = in.read(content, 0, content.length)) != -1) {
			byt.write(content, 0, i);
		}
		byt.flush();
		content = byt.toByteArray();
		// String str = new String(content);
		length = Integer.toString(byt.size());
		out.write(content);
		return length;
	}

}

Please note: I am passing the file name in the Integrated Configuration and present timestamp is concatenated with this file name to construct the final name of the file.

You need to add this java mapping in operation mapping after you generate the required target structure (using an message mapping/XSLT mapping or another java mapping) and bind the required parameters. These parameters needs to be passed from ICo as described in the next section.

Configuration In Integration Directory :

Integrated Configuration Set Up:

I have assigned values to the below parameters in ICo as these are used in Java mapping:

1. AccessKey
2. AddTimeStamp
3. AzureAccountName
4. AzureContainer
5. Blobtype
6. ContentType
7. FileName
8. TimeStampPattern
9. Version

Receiver REST Channel Configuration:

I have replaced the dynamic configuration headers in REST channel to create respective http headers and passed them with API call.

Other configurations are set up as per the integration requirement and not described in this blog post.

Once the configuration is completed, run the end to end process to check whether the file is updated in AzureBlob.

Issues Faced:

1. I could send only XML or JSON files with this set up in PO as REST channel supports only XML or JSON. From CPI using same concept in groovy script I could send flat files as well. If anyone could send flat file from REST channel please mention in comment.
2. For xml, extra attributes standalone=”no” is added in channel level, so the content length determined in java mapping varies with the actual content length sent out from REST channel, hence the API call fails. You might need to modify your code to avoid this error.

Conclusion:

This blog post describes how to use REST API to put files in Azure Blob container from SAP PI/PO. We can use the same concept to build groovy script to use PUT Blob REST API from CPI tenant.

Cheers,

Suman