Adapter Module to stop processing of duplicate file (FTP Location)

[Former Member]

 We had a requirement in one of our project, to stop the processing of duplicate files at the adapter level i.e. the message should not enter  the integration engine, if the file has been already processed earlier. There was a chance that client might place a file at FTP location more than ones.

Here we have used two text file at server to maintain the name of processed file and the password of the FTP. Example: name.txt and FtpPassword.txt.

The developed module code has the following functionality.

• Whenever a file will be picked from a FTP location, the module code will check the existence of that file name in the text file placed at the server. Example name.txt.

•  If the name of the file already exist, the code will make the file empty by making URL connection with the FTP server (so that adapter will delete that file when pinged to the server next time. For that we had set the property “do not process empty file” in the ftp channel). An error will be thrown.

• Else the name will be written in the text file (server) i.e. name.txt.

•  The module code will make a URL connection with the FTP server by picking the password from a text file stored at the server.Example FtpPassword.txt.

     To implement this functionality, we had written a module code. This module code will keep the name of files as record which will be checked before processing of the next file. If the file has been already processed, it will throw an error at adapter level which in turn can trigger alerts. 

For step by Step procedure of  the creation of adapter module, please refer the following pdf:

http://www.sdn.sap.com/irj/sdn/go/portal/prtroot/docs/library/uuid/3bdc14e1-0901-0010-b5a9-a01e29d75...

The code for the adapter module:

<span>package com.sap;import javax.ejb.SessionBean;import javax.ejb.SessionContext;import javax.ejb.CreateException;import com.sap.aii.af.service.auditlog.*;/*** @ejbHome <{com.sap.aii.af.mp.module.ModuleHome}>* @ejbLocal <{com.sap.aii.af.mp.module.ModuleLocal}>* @ejbLocalHome <{com.sap.aii.af.mp.module.ModuleLocalHome}>* @ejbRemote <{com.sap.aii.af.mp.module.ModuleRemote}>* @stateless* @transactionType Container*/import com.sap.aii.af.mp.module.*;import com.sap.aii.af.ra.ms.api.Message;import java.util.Hashtable;import java.io.*;import java.io.BufferedReader;import java.io.FileReader;import java.io.IOException;import java.net.*;import java.util.List;import java.util.ArrayList;public class CustomFileBean implements SessionBean, Module { public void ejbRemove() {}    public void ejbActivate() {}    public void ejbPassivate() {}    public void setSessionContext(SessionContext context) {  myContext = context;                                                       } private SessionContext myContext; public void ejbCreate() throws CreateException {}  public ModuleData process(ModuleContext moduleContext, ModuleData  inputModuleData) throws ModuleException{       Object obj = null;   // Handler to get Principle data     Message msg = null;  // Handler to get Message object     String fileName = null;  // File Name which is being processed           URL FileLocation;     obj = inputModuleData.getPrincipalData();     msg = (Message) obj;     Hashtable mp = (Hashtable)     inputModuleData.getSupplementalData("module.parameters");// Get the Supplemental data which is available as hash table     if (mp != null) fileName = (String)mp.get("FileName"); // Get the file name which is being processed           AuditMessageKey amk = new AuditMessageKey(msg.getMessageId(),AuditDirection.INBOUND);// to put the message in the audit log              String Location = moduleContext.getContextData("Location");   // moduleContext is used to retrieve data supplied by user           String FtpUser = moduleContext.getContextData("FtpUser");     // these data will be extracted from the module configuration of the channel           String PassLocation = moduleContext.getContextData("PassLocation");// which is passed as parameter value      //this code will read names of files which has been already processed and convert them into array so that it can be easily compared    //with the next name of file       List wordList = new ArrayList();       BufferedReader br = null;     try {     br = new BufferedReader( new FileReader(Location) );                    String word;//       loop and read a line from the file as long as we dontget null     while( ( word = br.readLine() ) != null )//       add the read word to the wordList       wordList.add( word );      }      catch(IOException ex){       ex.printStackTrace();                           }            finally {         try {//          attempt to close the file         br.close();          } catch( IOException ex ) {          ex.printStackTrace();               }           }        try{//    initialize a new string array equal to the size of the wordList      String[] words = new String[ wordList.size() ];//    call the wordList's toArray method to and transfer itemsfrom//    wordList to our string array words      wordList.toArray( words );      Writer output = null;          File file = new File(Location);      output = new BufferedWriter(new FileWriter(file));      int flag = 0;      for( int i = 0; i < words.length; i++ )      {      String text = words[i]+"<br /> ";      output.write(text);      if(words[i].equals(fileName))      flag=1;              }      // if file has not been processed earlier    if (flag==0)    {     //write the name file to the list    output.write(fileName);    output.close();       }          // if the file has been processed earlier       else     {     output.close();     try{      BufferedReader Read = new BufferedReader(new FileReader(PassLocation)); //to read the password of the FTP server      Audit.addAuditLogEntry(amk, AuditLogStatus.ERROR,"File has been already processed");            FileLocation = new URL("</span><a class="jive-link-external-small" href="ftp://">ftp://</a><span>"+FtpUser+":"+Read.readLine()+"@"+msg.getMessageProperty("</span><a class="jive-link-external-small" href="http://sap.com/xi/XI/System/File">http://sap.com/xi/XI/System/File</a><span>","SourceFTPHost")+ msg.getMessageProperty("</span><a class="jive-link-external-small" href="http://sap.com/xi/XI/System/File">http://sap.com/xi/XI/System/File</a><span>","Directory")+"/"+fileName+ "");      //msg.getMessageProperty("</span><a class="jive-link-external-small" href="http://sap.com/xi/XI/System/File">http://sap.com/xi/XI/System/File</a><span>","SourceFTPHost") this is used to retrieve the name of host. For this adapter specific message attributes must be enabled in the channel.       DeleteFile(FileLocation); //to delete the file             }     catch (Exception e)         {          throw new ModuleException(fileName+"should be deleted");         }       throw new ModuleException(fileName+"is already processed");     }          }    catch( Exception e ) {        ModuleException me = new ModuleException(e);                        throw me;         }   return inputModuleData;        } public void DeleteFile(URL FileLocation) throws Exception{    URLConnection uc = FileLocation.openConnection() ;  // to establish connection with FTP server using URL  uc.setDoOutput(true);  OutputStream out = uc.getOutputStream();  out.flush();//to make the file empty  out.close();//close the file so that connection can be terminated    } }</span>

And to maintain jndi name edit ejb-j2ee-engine.xml:

"

This jndi Name will be used at the time of calling this module in communication channel.

Scenario to test the Custom Module:

Configuration of sender channel: To pick any file available at FTP location

To delete the file :

The adapter specific message attribute must be checked (so that message can hold all the checked value which will be used by module code).

The Module Tab of the Sender communication channel.

        FtpUser: user name of the client FTP        

        Location: The complete paths of the text file which will keep the record of the processed file

        PassLocation:  The complete paths of the text file which will maintain the password of the FTP client

Testing

Case1: 

When a new file will be placed at FTP location.Example: sanjaisw.xml

we can see in the text file name.txt : sanjaisw.xml has not been processed earlier:

In SXMB_MONI

The text file on the server: the file name sanjaisw.xml has been added at the last place as shown below.

Case2:    When a duplicate file will be placed at FTP location:

Example: sandeep.xml (we can see below, sandeep.xml has been already processed)

Place the file at FTP Location

After some time file will be deleted as shown below:

But this file will not be send to the integration engine as shown below. Here it is showing only one file has been processed and that is the previous one.

We can see error: Go to Runtime Workbench à Message Monitoring à select Adapter engine XI1 and click on display as shown below:

 

After that select your message.

And we can see there is no change in the text file that is stored at server: