Technical Articles
Connect to AzureBlob from CPI: Using Azure Storage jar
Introduction:
I came across a requirement to upload file to AzureBlob container from CPI. There are multiple ways to achieve this : using REST API or using standard jar provided by MS.
In this blog post I will discuss about uploading files to AzureBlob container using standard jar available in maven repository.
Steps to Configure:
I followed the below blog post and done some modification:
https://blogs.sap.com/2019/08/30/how-to-connect-azure-using-scpi-upload-file-to-azure-blob/
Step 1: I downloaded Azure Storage Client SDK from https://mvnrepository.com/artifact/com.microsoft.azure/azure-storage/
Step 2: I uploaded this .jar file as resource to my iFlow:
Step 3: I added properties using groovy or content modifier. I used content modifier and externalized so that I can configure as per my need.
Step 4: I created a Groovy script to directly upload the file to the container
/*
Script to upload files to Azure Blob folder using parameters provided
*/
import com.sap.gateway.ip.core.customdev.util.Message
import java.util.HashMap
import com.microsoft.azure.storage.*
import com.microsoft.azure.storage.blob.*
import java.text.SimpleDateFormat
def Message processData(Message message) {
def body = message.getBody(java.lang.String) as String
map = message.getProperties()
String accountName=map.get("AzureAccountName")
String accountKey = map.get("AzureAccountKey")
String containerRef =map.get("ContainerRef")
String fileNameScheme = map.get("Filename")
String timestamp = map.get("AddTimestamp")
String storageConnectionString = "DefaultEndpointsProtocol=https;" + "AccountName=" + accountName+ ";" + "AccountKey=" + accountKey
CloudStorageAccount account = CloudStorageAccount.parse(storageConnectionString)
CloudBlobClient serviceClient = account.createCloudBlobClient()
CloudBlobContainer container = serviceClient.getContainerReference(containerRef)
if(timestamp.equals("Y")){
String pattern = "yyyyMMddHHmmss";
SimpleDateFormat simpleDateFormat = new SimpleDateFormat(pattern);
String date = simpleDateFormat.format(new Date());
fileNameScheme = fileNameScheme + "_" + date;
}
String fileName = fileNameScheme+ ".csv"
String fileContent = body
byte[] fileBytes = fileContent.getBytes()
CloudBlockBlob blob = container.getBlockBlobReference(fileName)
blob.uploadFromByteArray(fileBytes, 0, fileBytes.length)
blob.getProperties().setContentType("text/plain;charset=utf-8")
blob.uploadProperties()
message.setBody("File is created with name : "+fileName)
return message;
}
When the CPI flow runs successfully, file is uploaded if the properties mentioned are correct. We don’t need any particular adapter for this iFlow as this script takes care of uploading the files.
Conclusion:
Using this concept you can upload a file to AzureBlob from CPI without any adapter, using jar provided in Maven repository.
In the next blog post I will describe how we can use REST API to achieve the same.
Check this blogpost to get and process a file from azure blob container using CPI.
Cheers,
Suman
Can we pull all azure blob contents in a container using single rest URI call in SAP PO or CPI. Let's say I have four blobs with names abc,abcd,abcdef,def. I need to pull blob contents of abc,abcd,abcdef based on some file name pattern in single REST URI call. Is that possible
I have not tried that. But you can explore different download methods as mentioned in the document and try.
Hi Suman,
Can you share the iflow screenshot here... wanted to know how u used it
Hi Suman,
Thanks for sharing the blog. Could you pls describe the steps to place the file from SCPI to azure blob storage using REST API.
Hi Siva,
You can follow the below blog post which describes how to achieve this using SAP PO and do the necessary changes to adapt to CPI.
https://blogs.sap.com/2021/03/22/connect-to-azureblob-from-pi-po-using-rest-api
Regards,
Suman
Hello Suman!
Do you happen to know the solution to this issue?: org.apache.camel.CamelExecutionException: Exception occurred during execution on the exchange: Exchange[ID-4371506a-0610-4ce3-7cb8-0095-1628398407818-70-1], cause: java.lang.NoClassDefFoundError: javax/crypto/Mac
Hello Ivan,
hope you are doing fine.
We receive this error as well on CPI CF - did you have a chance to resolve that ?
Many thanks in advance,
Julian
Hello Suman,
Thanks for this blog it is very useful.
We have one scenario to pull file from Azure blob using CPI and put it into SAP.
Could you please suggest how to proceed for this scenario ,is it possible using GETBLOB REST API or any other approach would be more convenient.
Thanks and Regards
Saurabh
Hi Saurabh,
You can use the below code similar to the upload:
try{
CloudStorageAccount account = CloudStorageAccount.parse(storageConnectionString)
CloudBlobClient serviceClient = account.createCloudBlobClient()
CloudBlobContainer container = serviceClient.getContainerReference(containerRef)
CloudBlockBlob blob = container.getBlockBlobReference(fileName)
message.setBody(blob.downloadText())
}
catch(Exception e){
throw new Exception("file can't be downloaded")
}
I have roughly written the code, you can change according to your requirement and let me know.
Regards,
Suman Saha
Hi Suman,
Thanks for your swift response, Could you please confirm over below points :
i. Does storage Jar file import will help in this scenario as well.
ii.Since the source is Azure Blob container, we do not need sender since groovy will pull the file.
iii. For connectivity ,parameter used in groovy is sufficient or do we need anything else.
Thanks and Regards
Saurabh
Hi,
The filename needs to be provided in the content modifier as a parameter. Everything else will be the same.
Yes you need to start with a trigger and schedule that.
Regards,
Suman
Hi,
You can follow the blogpost:
Get File from AzureBlob using CPI: Using Azure Storage jar | SAP Blogs
Regards,
Suman
Thanks a lot Suman, Really appreciate it.
Let me try and I will get back to you if I stuck anywhere.
Thanks & Regards
Saurabh
You can directly achieve this using HTTP connector in CPI . You can reach out to me if you need any help.
Thanks for your valuable input.
Thanks and Regards
Saurabh
Hey could we connect regarding this ?
Have anyone faced the below issue.
com.sap.it.rt.adapter.http.api.exception.HttpResponseException: An internal server error occured: javax/crypto/Mac.
Kindly help.
I try to upload PDFs, but some pictures are not shown. Any idea ?
Thanks a lot, Suman. I'm able to upload document now.
I faced problem due to key as InvalidKeyException: Storage Key is not a valid base64 encoded string then realized my key is incorrect from other blogs.
You need to grab the key from the azure portal. Go to the dashboard. Select your storage account. You can then select "Access keys". Just pick the top key.