Skip to Content

  Abstract

+
Customers often ask for massive file transfers for the interfaces affecting their systems.
This blog describes the realization of a custom adapter module in SAP PI (Process Integration) 7.1 to manage the proper acquisition
of chunk files pertaining to a single large file.
+

 

 

  Referenced sites

 
How to Create Modules for the JEE Adapter Engine

 

 

  Scope

  In order to send large files, the sender system divides them into smaller chunk files (e.g.: 5 MB) to send in parallel.

  Once this operation is over, an additional end-of-job file (.eoj) containing the list of chunk files is produced and sent.

  Assuming that the sender system does not guarantee the sending order, the eoj file must enable the check on the proper reception of all

  chunk files and determine their acquisition in the settled sequence, in order to reconstruct the original file.   

 

  The process therefore provides for the system that receives the chunk files to wait for the arrival of the end-of-job file.

  Once the eoj file is obtained, the receiving system checks that all files listed have been properly received. If so, these files are

  renamed to be acquired in order. Otherwise the receiving system reports the missed reception and checks again after a preset time. 

 

 

 

  Realization

 

  Following the directions given in the document , a custom module (called Chunk Message CheckCMC) has been created for the file adapter

  in SAP PI 7.1 system. This module is tailored for the FTP protocol, and it directly gets from the adapter all information necessary to access.

  The only information that must be provided in input is the new extension of the chunk file to be renamed (otherwise .ch will be used by default).   

 

  In order to implement the interface that runs the CMC custom module, you just need to use objects from the Integration Builder

  (design objects are not necessary). For example, you can define an integrated configuration object with a File sender and a File receiver channel,

  using dummy interfaces (eg: Dummy_Out and Dummy_In) and the same business system as sender and receiver.

 

  The file sender channel must be configured in order to acquire the end-of-job files and must provide for an adapter module CMC at the beginning

  of the processing sequence, before the File adapter. You also have to insert the parameter ChunkExt to define the extension of the renamed files. 

 

 

 

 

 

 

 

  Java code

  Below is the java code that implements the adapter module Chunk Message Check. i; i++)

     {

      newFileName = oldFileName =  “”;

      oldFileName = ((String)chunkfiles.get(i));

      newFileName = oldFileName+this.ChunkExt;

      ftpClient.rename(oldFileName, newFileName);     

     }     

     audit.addAuditLogEntry(amk, AuditLogStatus.SUCCESS,

     “CMC: all chunks RENAMED!”);

    }

    else

    {

     //audit.addAuditLogEntry(amk, AuditLogStatus.ERROR,

     //”CMC: some file MISSING “);

     throw new Exception(“Some chunk file missing: “counter” of “+chunkfiles.size());

    }

   } else {

    audit.addAuditLogEntry(amk, AuditLogStatus.ERROR,

    “CMC: login fail”);

   }

  } catch (IOException ioe) {

   audit.addAuditLogEntry(amk, AuditLogStatus.ERROR,

     “CMC: “+ioe.getMessage());

   throw new MessagingException(ioe);

  } catch (Exception e){

   throw new MessagingException(e);

  } 

  finally {

   try {   

    if(login)

    {

     // When logout success the logout method returns true. 

     boolean logout = ftpClient.logout();

     if (logout) {

      audit.addAuditLogEntry(amk, AuditLogStatus.SUCCESS,

      “CMC: logout from FTP server “);

     }

    }

    // Closes the connection to the FTP server 

    ftpClient.disconnect();

   } catch (IOException e) {

    throw new MessagingException(e);

   }

  }

}

}

 

 

  Example run

 

 

  A file test.eoj has been added to perform a test of the CMC module; within this file there are two file names which are the chunks

  deriving from the splitting of a larger file.

 

 

   

  The communication channel FILESenderTest reads the test.eoj file and looks for the chunks described in it.

Since there are no chunk files in the directory, the communication channel ends with processing error, showing the number of chunks detected
and the number of those expected (0/2 in this case). The channel will attempt a new check after 60 seconds, as defined in the configuration.

 

 

 

  Below is the log concerning the steps performed by the communication channel FILESenderTest in the attempt to identify the chunks expected

  in the file test.eoj.

 

 

 

 

  By entering the expected chunk files (chunk1.txt and chunk2.txt), the next processing loop of the FILESenderTest channel allows

  the identification of each chunk listed in the test.eoj file, and executes the renaming in order to get the chunks in the correct order by a

  further standard communication channel.

 

 

 

 

 

  In this case the FILESenderTest communication channel correctly performs the process (as it is shown below), identifying and renaming

  all chunk files and moving the file test.eoj from the reading directory.

 

 

 

 

   

 

The directory now contains the chunk files properly renamed and ready to be acquired in the correct order by another
communication channel for the acquisition in Exactly Once In Order mode.

 

To report this post you need to login first.

2 Comments

You must be Logged on to comment or reply to a post.

  1. Anurag Garg
    HI Allesandro,

    Its a very useful blog for a common issue.

    Can you please let me know if FTP Parameters can be fetched in the way you have used:
    this.FtpUser = channel.getValueAsString(“ftp.user”);
       this.FtpPassword = channel.getValueAsString(“ftp.password”);
       this.FtpHost = channel.getValueAsString(“ftp.host”);
       this.FtpDirectory = channel.getValueAsString(“ftp.sourceDir”);

    Are these parameters fetched from Channel FTP Parameters like Server , User Name, Password, Source Directory?
    Suggest a SAP Document on this info.

    REgards,
    Anurag

    (0) 
    1. Alessandro Menale Post author
      Hi Anurag, thank you!

      I guess there is no documentation about the parameters that you can use to read channel information.
      In order to get this data I have used an helper method of the Channel class.

      I’m going to write another short blog in order to describe this in detail.

      Regards,
      Alessandro

      (0) 

Leave a Reply