Skip to Content

Recently we have faced couple of issues where PI File sender channel failed to pick the file from source file server because some times it is going to hung state and some times it is getting locked for some of the business critical interfaces.

We tried to avoid the above situations by trying different options like refreshing the channel by using availability time planning and setting up time out parameter in sender File channel .Though these workarounds minimized the above situations ,client is not fully satisfied .He is looking for some thing more …that is the situation where in order to convince the client I’m asked to write a custom module/Unix script to check the age of the files in input folders of all sender systems which uses file server .

Best options to fulfill the above requirement is to develop custom module or Unix script.I opted developing custom module to suffice the requirement  .

Custom module development:


Prerequisites:

For connecting to different source file servers and to get the file names and age of the files , i have used the Apache open source API .It is free to use 🙂

You can download the jar file from the below link.

Apache Commons Net –  Download Commons Net

For module Creation below mentioned PI specific jar files are required.

Modules.JPG

You can refer the below link for steps to develop custom module.

http://www.sdn.sap.com/irj/scn/go/portal/prtroot/docs/library/uuid/c0b39e65-981e-2b10-1c9c-fc3f8e6747fa?overridelayout=t…


Module Code :


/**

*

*/

package com.Venkat.sappi.AgingAlerts;

//Java imports

import org.apache.commons.net.ftp.FTPClientConfig;

import org.apache.commons.net.ftp.FTPFile;

import org.apache.commons.net.ftp.FTPClient;

import org.apache.commons.net.ftp.FTPFileFilter;

import java.util.Date;

import javax.ejb.SessionBean;

import javax.ejb.SessionContext;

//Module imports

import com.sap.aii.af.lib.mp.module.Module;

import com.sap.aii.af.lib.mp.module.ModuleContext;

import com.sap.aii.af.lib.mp.module.ModuleData;

import com.sap.aii.af.lib.mp.module.ModuleException;

import com.sap.aii.af.service.auditlog.Audit;

import com.sap.engine.interfaces.messaging.api.Message;

import com.sap.engine.interfaces.messaging.api.MessageDirection;

import com.sap.engine.interfaces.messaging.api.MessageKey;

import com.sap.engine.interfaces.messaging.api.MessagePropertyKey;

import com.sap.engine.interfaces.messaging.api.PublicAPIAccessFactory;

import com.sap.engine.interfaces.messaging.api.auditlog.AuditAccess;

import com.sap.engine.interfaces.messaging.api.auditlog.AuditLogStatus;

import com.sap.engine.interfaces.messaging.api.exception.InvalidParamException;

import com.sap.tc.logging.Location;

/**

* @author : Venkat Nimmagadda

* @version : 0.1

* @Purpose : To find the age of the files and notify us.

* @Created on :01-Jun-2013

*

*/

public class AgingAlertsBean implements SessionBean, Module {

    public static final String VERSION_ID = “$Id://tc/aii/30_REL/src/_adapters/_sample/java/user/FileStatusCheckBean.java#1 $”;

    static final long serialVersionUID = 7435850550539048633L;

    private Message msg = null;

    private MessageKey key = null;

    private String count;

    private String inpTag;

    private String endTag;

    private String timeInterval;

    private String Out;

    private String Check;

    private int countVal;

    private String[] server;

    private int[] subCount;

    private String[] SubCount;

    private String[] userName;

    private String[] password;

    private String[] DateFormat;

    private String[] OSType;

    private String[][] fileName;

    private String[][] fileAge;

    private int[][] fileAgeVal;

    private String[][] dirPath;

    private StringBuffer Out_Final = new StringBuffer(“”);

    private java.util.Date md ;

    private long diff;

    private long diffMinutes;

    private String Output;

    private SessionContext myContext;

    public void ejbRemove() {

    }

    public void ejbActivate() {

    }

    public void ejbPassivate() {

    }

    public void setSessionContext(SessionContext context) {

        myContext = context;

    }

    public ModuleData process(ModuleContext moduleContext,

            ModuleData inputModuleData) throws ModuleException {

        String SIGNATURE = “process(ModuleContext moduleContext, ModuleData inputModuleData)”;

        Location location = null;

        AuditAccess audit = null;

        // Create the location always new to avoid serialisation/transient of

        // location

        try {

            location = Location.getLocation(this.getClass().getName());

        } catch (Exception t) {

            t.printStackTrace();

            ModuleException me = new ModuleException(

                    “Unable to create trace location”, t);

            throw me;

        }

        try {

            msg = (Message) inputModuleData.getPrincipalData();

            if (msg.getMessageDirection().equals(MessageDirection.OUTBOUND))

                key = new MessageKey(msg.getMessageId(),

                        MessageDirection.OUTBOUND);

            else

                key = new MessageKey(msg.getMessageId(),

                        MessageDirection.INBOUND);

            audit = PublicAPIAccessFactory.getPublicAPIAccess()

            .getAuditAccess();

            audit.addAuditLogEntry(key, AuditLogStatus.SUCCESS,

            “AgingAlertsBean: Module called”);

        } catch (Exception e) {

            audit.addAuditLogEntry(key, AuditLogStatus.ERROR,

            “$$$ Error while retrieving message ID/Audit trace”);

        }

        if (moduleContext != null) {

            count = moduleContext.getContextData(“Count”);

            audit.addAuditLogEntry(key, AuditLogStatus.SUCCESS,”Input value given for \”Count\” is :” + count );

            if (count == null) {

                throw new ModuleException(

                “Specified ‘Count’ in Module Context is null. Please check the module configuration”);

            }

            inpTag = moduleContext.getContextData(“InpTag”);

            audit.addAuditLogEntry(key, AuditLogStatus.SUCCESS,”Input value given for \”InpTag\” is :” + inpTag );

            if (inpTag == null) {

                throw new ModuleException(

                “Specified ‘InpTag’ in Module Context is null. Please check the module configuration”);

            }

            endTag = moduleContext.getContextData(“EndTag”);

            audit.addAuditLogEntry(key, AuditLogStatus.SUCCESS,”Input value given for \”EndTag\” is :” + endTag );

            if (endTag == null) {

                throw new ModuleException(

                “Specified ‘EndTag’ in Module Context is null. Please check the module configuration”);

            }

       

            countVal = Integer.parseInt(count);// converting string value to

            // int

       

            server = new String[countVal];

            userName = new String[countVal];

            password = new String[countVal];

            SubCount = new String[countVal];

            DateFormat = new String[countVal];

            OSType = new String[countVal];

            try{

                for (int i = 0; i < countVal; i++) {

                    SubCount[i] = moduleContext.getContextData(“SubCount_”+i);

                    audit.addAuditLogEntry(key, AuditLogStatus.SUCCESS,”Input value given for \”SubCount_”+i+” is :” +SubCount[i] );

                    server[i] = moduleContext.getContextData(“Server_”+ i);

                    audit.addAuditLogEntry(key, AuditLogStatus.SUCCESS,”Input value given for \”Server_”+i+” is :” +server[i] );

                    userName[i] = moduleContext.getContextData(“UserName_”+i);

                    audit.addAuditLogEntry(key, AuditLogStatus.SUCCESS,”Input value given for \”UserName_”+i+” is :” +userName[i] );

                    password[i] = moduleContext.getContextData(“pwd_Password_”+i);

                    audit.addAuditLogEntry(key, AuditLogStatus.SUCCESS,”Input value given for \”pwd_Password_”+i+” is :” +password[i] );

                    DateFormat[i] = moduleContext.getContextData(“dateFormat_”+i);

                    audit.addAuditLogEntry(key, AuditLogStatus.SUCCESS,”Input value given for \”dateFormat_”+i+” is :” +DateFormat[i] );

                    OSType[i] = moduleContext.getContextData(“osType_”+i);

                    audit.addAuditLogEntry(key, AuditLogStatus.SUCCESS,”Input value given for \”osType_”+i+” is :” +OSType[i] );

                }//closing brace for if

            }//closing brace for try

            catch(Exception e){audit.addAuditLogEntry(key, AuditLogStatus.ERROR,”Exception occured while reading input values for server ,username ,pwd and SubCount ” +

                    “Error message :”+e.getMessage()+”Stack trace”+e.getStackTrace());

            throw new ModuleException(“Exception occured while reading input values for server ,username ,pwd and SubCount “);

            }

            subCount = new int[countVal];

            for (int t = 0; t < countVal; t++) {

                subCount[t] = Integer.parseInt(SubCount[t]);

            }//closing brace for for

            dirPath = new String[countVal][25];

            fileName = new String[countVal][25];

            fileAge = new String[countVal][25];

            fileAgeVal = new int[countVal][25];

            try{

                for (int i = 0; i < countVal; i++) {

                    for (int j = 0; j < subCount[i]; j++) {

                        dirPath[i][j] = moduleContext.getContextData(“DirPath_”+i+j);

                        audit.addAuditLogEntry(key, AuditLogStatus.SUCCESS,”Input value given for \”DirPath_”+i+j+” is :” +dirPath[i][j] );

                        fileName[i][j] = moduleContext.getContextData(“FileName_”+i+j);

                        audit.addAuditLogEntry(key, AuditLogStatus.SUCCESS,”Input value given for \”FileName_”+i+j+” is :” +fileName[i][j] );

                        fileAge[i][j] = moduleContext.getContextData(“FileAge_”+i+j);

                        audit.addAuditLogEntry(key, AuditLogStatus.SUCCESS,”Input value given for \”FileAge_”+i+j+” is :” +fileAge[i][j] );

                        fileAgeVal[i][j] = Integer.parseInt(fileAge[i][j]);

                    }//closing brace for forloop

                }//closing brace for forloop

            }////closing brace for try

            catch(Exception e){audit.addAuditLogEntry(key, AuditLogStatus.ERROR,”Exception occured while reading input values for Dirpath ,FileName” +

                    “Error message :”+e.getMessage()+”Stack trace”+e.getStackTrace());

            throw new ModuleException(“Exception occured while reading input values for Dirpath ,FileName”);

            }

        }// closing brace for if

        for (int k = 0; k < countVal; k++) {

            // Start creating a FTPClient instance:

            FTPClient client = new FTPClient();

            // Connect now to a remote FTP service:

            if(OSType[k].equalsIgnoreCase(“UNIX”))

            {FTPClientConfig conf = new FTPClientConfig(FTPClientConfig.SYST_UNIX);

            conf.setDefaultDateFormatStr(DateFormat[k]);

            client.configure(conf);

            }

            if(OSType[k].equalsIgnoreCase(“NT”))

            {FTPClientConfig conf = new FTPClientConfig(FTPClientConfig.SYST_NT);

            conf.setDefaultDateFormatStr(DateFormat[k]);

            client.configure(conf);

            }

            if(OSType[k].equalsIgnoreCase(“AS400”))

            {FTPClientConfig conf = new FTPClientConfig(FTPClientConfig.SYST_AS400);

            conf.setDefaultDateFormatStr(DateFormat[k]);

            client.configure(conf);

            }

            if(OSType[k].equalsIgnoreCase(“VMS”))

            {FTPClientConfig conf = new FTPClientConfig(FTPClientConfig.SYST_VMS);

            conf.setDefaultDateFormatStr(DateFormat[k]);

            client.configure(conf);

            }

            if(OSType[k].equalsIgnoreCase(“OS2”))

            {FTPClientConfig conf = new FTPClientConfig(FTPClientConfig.SYST_OS2);

            conf.setDefaultDateFormatStr(DateFormat[k]);

            client.configure(conf);

            }

            if(OSType[k].equalsIgnoreCase(“L8”))

            {FTPClientConfig conf = new FTPClientConfig(FTPClientConfig.SYST_L8);

            conf.setDefaultDateFormatStr(DateFormat[k]);

            client.configure(conf);

            }

            if(OSType[k].equalsIgnoreCase(“NETWARE”))

            {FTPClientConfig conf = new FTPClientConfig(FTPClientConfig.SYST_NETWARE);

            conf.setDefaultDateFormatStr(DateFormat[k]);

            client.configure(conf);

            }

            if(OSType[k].equalsIgnoreCase(“MACOS_PETER”))

            {FTPClientConfig conf = new FTPClientConfig(FTPClientConfig.SYST_MACOS_PETER);

            conf.setDefaultDateFormatStr(DateFormat[k]);

            client.configure(conf);

            }

            if(OSType[k].equalsIgnoreCase(“MVS”))

            {FTPClientConfig conf = new FTPClientConfig(FTPClientConfig.SYST_MVS);

            conf.setDefaultDateFormatStr(DateFormat[k]);

            client.configure(conf);

            }

            if(OSType[k].equalsIgnoreCase(“OS400”))

            {FTPClientConfig conf = new FTPClientConfig(FTPClientConfig.SYST_OS400);

            conf.setDefaultDateFormatStr(DateFormat[k]);

            client.configure(conf);

            }

            try{

                client.connect(server[k]);// connecting to ftp server

                client.login(userName[k], password[k]);// Giving credentials

                audit.addAuditLogEntry(key, AuditLogStatus.SUCCESS,”Successfully connected to server  “+ server[k] +”. “);

                audit.addAuditLogEntry(key, AuditLogStatus.SUCCESS,”Successfully connected to server  “+ client.getReplyString());

            }

            catch(Exception e){

                e.printStackTrace();

                audit.addAuditLogEntry(key, AuditLogStatus.ERROR,”Exception occured while connecting to ” + server[k]+” :”+”Error message :”+e.getMessage()+”Stack trace”+e.getStackTrace());

                throw new ModuleException(“Exception occured while connecting to ” + server[k]);

            }

            try{

                for (int l = 0; l < subCount[k]; l++) {

                    final String pattern = fileName[k][l].replace(“.”, “\\.”).replace(“*”, “.*”);

                    audit.addAuditLogEntry(key, AuditLogStatus.SUCCESS,

                            “Pattern : ” + pattern);

                    FTPFileFilter filter = new FTPFileFilter() {

                        public boolean accept(FTPFile ftpFile) {

                            return (ftpFile.isFile() && ftpFile.getName().matches(pattern));

                        }

                    };

                    FTPFile[] fileNames ;

                    try{

                        fileNames = client.listFiles(dirPath[k][l], filter);

                        audit.addAuditLogEntry(key, AuditLogStatus.SUCCESS,”Successfully executed listNames command in the “+ server[k]+ “in directory “+dirPath[k][l]);

                    }

                    catch(Exception e){                   

                        audit.addAuditLogEntry(key, AuditLogStatus.ERROR,”Exception occured while executing listNames command in the “+ server[k]+ “in directory “+dirPath[k][l]+”. Error message :”+e.getMessage()+”Stack trace”+e.getStackTrace());

                        throw new ModuleException(“Exception occured while executing listNames command on server : ” + server[k]);

                    }

                    for (int m = 0; m < fileNames.length; m++) {

                       audit.addAuditLogEntry(key, AuditLogStatus.SUCCESS,”Files in the server  “+server[k]+” that matched pattern are :” + fileName[k][l]);

                        try{

                            md = fileNames[m].getTimestamp().getTime();

                            audit.addAuditLogEntry(key, AuditLogStatus.SUCCESS,”Last modified time stamp for file “+fileNames[m].getName() +” is :” + fileNames[m].getTimestamp().getTime());

                        }

                        catch(java.lang.IllegalStateException e)

                        {

                            e.printStackTrace();

                            audit.addAuditLogEntry(key, AuditLogStatus.ERROR,”client is not connected or not authenticated while executing modifiedDate command”);

                            throw new ModuleException(“client is not connected or not authenticated while executing modifiedDate command ” );

                        }

                        catch(Exception e)

                        {

                            e.printStackTrace();

                            audit.addAuditLogEntry(key, AuditLogStatus.ERROR,”Operation failed while executing modifiedDate command. “+”Error message :”+e.getMessage());

                            throw new ModuleException(“Operation failed while executing modifiedDate command ” );

                        }

                        Date today = new Date();

                        try{

                            diff = today.getTime() – md.getTime();

                            diffMinutes = ((diff / (60 * 1000)) – 60);

                            audit.addAuditLogEntry(key, AuditLogStatus.SUCCESS,”Time difference with respect to current time for file ” +fileNames[m].getName()+ ” is :              “+diffMinutes);

                        }

                        catch(Exception e)

                        {

                            e.printStackTrace();

                            throw new ModuleException(“Exception occured while calculating time difference with respect to current time for file ” +fileNames[m].getName());

                        }

                        try{

                            if (diffMinutes > fileAgeVal[k][l]) {

                                audit.addAuditLogEntry(key, AuditLogStatus.SUCCESS,fileNames[m].getName()+ ” file has exceeded threshold time”);

                                Out_Final

                                .append(

                                        fileNames[m].getName()

                                        + ” in ” + server[k]

                                                          + ” in dir “

                                                          + dirPath[k][l]).append(System.getProperty(“line.separator”));

                            }// closing brace for if

                        }catch(Exception e)

                        {

                            e.printStackTrace();

                            throw new ModuleException(“Exception occured while checking age of the file ” +fileNames[m].getName() );

                        }

                        //}// closing brace for if                   

                    }// closing brace for for loop

                    //java.util.Arrays.fill(fileNames,””);

                }// closing brace for for loop

                try { if (client.isConnected()) {           

                    client.logout();

                    client.disconnect();}

            

                }// end brace for try   

                catch (Exception e) {

                    e.printStackTrace();

                    throw new ModuleException(“Exception occured while disconnecting from the server : ” + server[k]);                                           

                }

            }catch (Exception e) {

                e.printStackTrace();

                throw new ModuleException(“Exception occured while working on ” + server[k]);                                           

            }

        }// for loop closing brace

        if (Out_Final.toString() == null

                || Out_Final.toString().trim().length() == 0) {

            Check = “NotValid”;

            audit.addAuditLogEntry(key, AuditLogStatus.SUCCESS,

            “None of the files crossed the threshold time value”);

        } else {

            Check = “Valid”;

            //audit.addAuditLogEntry(key, AuditLogStatus.SUCCESS,

            //”Files which crossed the threshold time value are :\n”+Out_Final.toString());

        }

        Out = inpTag

        + “<Status>” + Check + “</Status>” + “<Content>”

        + Out_Final.toString()

        + “</Content>” + endTag;

        Out_Final.setLength(0);

        try {

            msg.getMainPayload().setContent(Out.getBytes(“UTF-8”));

        } catch (InvalidParamException e) {

            audit.addAuditLogEntry(key, AuditLogStatus.ERROR, e.getMessage());

            throw new ModuleException(e);

        } catch (Exception e) {

            audit.addAuditLogEntry(key, AuditLogStatus.ERROR, e.getMessage());

            audit.addAuditLogEntry(key, AuditLogStatus.SUCCESS,

            “Data passed across to message stream”);

            inputModuleData.setPrincipalData(msg);

            audit.addAuditLogEntry(key, AuditLogStatus.SUCCESS,

            ” Data assigning to message stream Completed”);

            throw new ModuleException(e);

        }

        return inputModuleData;

    }

}//closing brace for class

In Part2 of this blog i will explain how to use this custom module by developing a file to mail interface in PI .

To report this post you need to login first.

Be the first to leave a comment

You must be Logged on to comment or reply to a post.

Leave a Reply