Skip to Content

Objective: To calculate MD5 checksum value of a flat file and send this value in a checksum file to FTP/SFTP server.


Approaches Tried:

We use following approaches in general to achieve certain functionality which is not readily achieved in SAP PI.

  • Modules from SAP.
  • Java mapping through customized .jar files.
  • OS command lines.
  • User Defined Functions (UDFs).

Details on various approaches:

  • Modules from SAP:

We do not have any inbuilt module in SAP PI which can generate checksum of a file especially a non-xml file. Writing a customized module requires larger efforts and expertise to develop and deploy.

  • Java mapping through customized .jar files:
    • This is most popular approach of attaining some functionality which is not readily available with in SAP PI.
    • Java mapping uses .jar files which are used in Integration Engine with in SAP PI.
    • It operates on XML form of data and suitable for the nominal file size which PI can process without altering its performance.
    • With respect to our requirement it has two shortcomings:
      1. It receives data from source side in the form of xml into an input stream. It performs operation on this xml and xml tags. But we need to calculate checksum on a flat file not on its xml version.
      2. It uses ESR objects to identify source data. As it is a quarterly activity and files will come in chunks of 5 MB – 10 MB, the final file will have a size of >= 40 MB. It is huge file and will get stuck in PI and will not get processed.
      3. We need to calculate checksum on whole file; not on its individual chunks.
    • Owing to above shortcomings we cannot employ this approach to generate MD5 checksum.

  • OS command lines:
    • We do not have any inbuilt command line to generate checksum for SAP PI on windows server.
    • OS command can be used with MD5 checksum utilities available on internet but that may expose system to malware.
    • Using a java program through OS command line efforts at par with developing a UDF or java mapping which are easier to implement.
    • Most importantly it is difficult to track any failure in OS command line as the communication channel never displays any error if OS command line does not run properly.

  • User Defined Functions:
    • UDFs are easier to develop, compile and deploy.
    • Though UDFs too use ESR objects but it may operate independent of the data being processed.
    • UDFs have advantage that PI server (AL11 Path) is directly accessible to it.
    • UDFs can be written to operate on a flat file (in AL11 path) of any size without converting it to XML.


  • Why do we choose UDF over other approaches?
    • UDFs have advantage that it can operate on flat files as a whole though it operates in integration Engine.
    • The flat files (for which we need to calculate checksum) on PI server (AL11 path) are always at its disposal.
    • The interface using this UDF can be run whole day, thus constantly searching for the files on the path for which checksum is needed to be calculated.
    • It performs MD5 Checksum logic on flat file not on its xml version.
    • It does not use ESR objects of the file for which checksum is needed to be calculated thus not converting the file to xml  adding to that whatever be the size of the file it can calculate checksum value without putting any load on PI once the file is available on AL11 path.
    • We can use same UDF to calculate checksum of any number of interfaces with any file size.

  • Conclusion:
    • UDFs can operate on flat file without changing it XML
    • UDFs can operate on flat file of any size thus be ridding PI of any stress due to file size.
  • Process of implementation:
    • We are implementing the UDF in message mapping itself as shown below:


UDF.jpg


    • The Complete Code is:


——————————————————————————————————————————————————————

try

{

  String Path = “<File path on AL11>”+”Filename”;

  String datafile = Path;

 

         MessageDigest md = MessageDigest.getInstance(“MD5”);

         FileInputStream fis = new FileInputStream(datafile);

         byte[] dataBytes = new byte[1024];

 

         int nread = 0;

  

         while ((nread = fis.read(dataBytes)) != -1) {

           md.update(dataBytes, 0, nread);

         };

 

         byte[] mdbytes = md.digest();

 

         //convert the byte to hex format

         StringBuffer sb = new StringBuffer(“”);

         for (int j = 0; j < mdbytes.length; j++) {

                    sb.append(Integer.toString((mdbytes[j] & 0xff) + 0x100, 16).substring(1));

         }

         return sb.toString();

}

catch (Exception e)

{

return “Error”;

}

———————————————————————————————————————————————————

You will be needed to place the file in any AL11 path which is accessible to SAP PI service User.

AL11.jpg


    • This UDF can be used in message mapping:


Mapping.jpg



    • Which in turn generated MD5 checksum of the file abc.csv (test file) which is place on some AL11 path as shown below, accessible to UDF and will pass the MD5 checksum value to target field which can be redirected to generate a checksum file on FTP/SFTP server.

Mapping Test.jpg


    • The final checksum file will look like as shown below. (The text with in file can be customized in UDF)


Checksum.jpg


* Please note that the above process is very Basic approach to calculate MD5 checksum. It can be implemented directly on FTP server instead of first moving file to AL11 path. For which you will need to use the library org.apache.commons to use FTP upload download. You will need to get this library placed on PI server to be able to import it while developing UDF.

To report this post you need to login first.

Be the first to leave a comment

You must be Logged on to comment or reply to a post.

Leave a Reply