Technical Articles
Get File from AzureBlob using CPI: Using Azure Storage jar
Introduction:
I have described how to upload a file in azureblob storage container in my previous blog post
In this blog post I will discuss about downloading file from AzureBlob container using standard jar available in maven repository.
Steps to Configure:
Step 1: I downloaded Azure Storage Client SDK from https://mvnrepository.com/artifact/com.microsoft.azure/azure-storage/
Step 2: I uploaded this .jar file as resource to my iFlow:
Step 3: I added properties using content modifier and externalized so that I can configure as per my need. Please note I have saved the Azure account key as secure parameter in security material.
Step 4: I created a Groovy script to directly download the file from the container and set as body.
/*
Script to download file to Azure Blob folder using parameters provided
*/
import com.sap.gateway.ip.core.customdev.util.Message
import java.util.HashMap
import com.microsoft.azure.storage.*
import com.microsoft.azure.storage.blob.*
import com.sap.it.api.ITApiFactory;
import com.sap.it.api.securestore.SecureStoreService;
import com.sap.it.api.securestore.UserCredential;
import com.sap.it.api.securestore.exception.SecureStoreException;
def Message processData(Message message) {
def body = message.getBody(java.lang.String) as String
map = message.getProperties()
String accountKey = null;
String accountName=map.get("AzureAccountName")
String AzureAccountKey = map.get("AzureAccountKey")
String containerRef =map.get("ContainerRef")
String fileName = map.get("Filename")
def securestoreApi = ITApiFactory.getService(com.sap.it.api.securestore.SecureStoreService.class, null)
try{
accountKey = String.valueOf(securestoreApi.getUserCredential(AzureAccountKey).getPassword())
} catch(Exception e){
throw new SecureStoreException("Secure Parameter not available")
}
String storageConnectionString = "DefaultEndpointsProtocol=https;" + "AccountName=" + accountName+ ";" + "AccountKey=" + accountKey
try{
CloudStorageAccount account = CloudStorageAccount.parse(storageConnectionString)
CloudBlobClient serviceClient = account.createCloudBlobClient()
CloudBlobContainer container = serviceClient.getContainerReference(containerRef)
CloudBlockBlob blob = container.getBlockBlobReference(fileName)
message.setBody(blob.downloadText())
}
catch(Exception e){
throw new Exception("file can't be downloaded")
}
return message;
}
I have configured trigger to schedule the CPI flow to run at particular time. When the CPI flow runs successfully, file is downloaded and added to the message body if the properties mentioned are correct. We don’t need any particular adapter for this iFlow as this script takes care of downloading file.
Conclusion:
Using this concept you can download a file from AzureBlob using CPI without any adapter, using jar provided in Maven repository.
Cheers,
Suman Saha
Thank you for the blog Suman.
We have a use case where we need to upload/download files to/from Azure Data Lake Gen1. Would you have any recommendations on the best way to deal with this (I know there are Azure DL Gen1 REST APIs but thought there could be other ways).
Many thanks.
Regards,
Alexey
Hi Alexey,
The best way should be using REST API. You can find detailed documentation on official site to connect using REST API.
Regards,
Suman
Thank you, Suman.
Hi Suman,
Thanks for the blog,
If you could clarify on the below points it would be great help :
If the scenario is - Container Name -X - Folder A- Folder-File and A is dynamic could be changed based on the date ,then the file should be picked from Container Name X- Folder B - Folder-file. In this case this solution will work ?
Hi Suman,
I am doing IDOC to Azure Blob integration,IDOC to post in blob floder as xml file.
So I am followed your blog ContainerRef i used flodername (if1-data).
done changes in script instaded of .csv i used .xml now i am getting below error.
is there any way to find what is the error getting lin Blob side also.
Thanks,
Kumar