Additional Blogs by Members
cancel
Showing results for 
Search instead for 
Did you mean: 
Former Member
0 Kudos

This blog is a continuation of Generic Approach for Validating Incoming Flat File in SAP XI - Part 1. In the previous blog I did mention the factors against which I am trying to validate the incoming file. This blog specifies the java-mapping program for fulfilling this requirement. The Mapping Program is divided into three-class files for modularity.



The various class files are:



    • BaseValidation.java


    • InterfaceSpecificValidate.java




I have uploaded two jar files for this purpose. The first one would contain Hierarchy.class and BaseValidate.class. This jar file is uploaded to a generic software component version. There is a Usage Dependency between Generic Software Component Version and Stream Specific Software Components. The second Jar would contain InterfaceSpecificValidate.class. This jar should be imported for individual interfaces.




Source Code for Hierarchy.class:







BaseValidate.java contains the logic for Validating incoming file and creating ERROR_REASON node in addition to STATUS and FILE TYPE nodes. The purpose of having this modularity is to avoid duplication of efforts in case of change in requirements during Testing. This would increase maintainability of the code.



Source Code For BaseValidate.java


import java.io.BufferedReader;
import java.io.BufferedWriter;
import java.io.File;
import java.io.FileReader;
import java.io.FileWriter;
import java.io.InputStream;
import java.io.OutputStream;


import javax.xml.parsers.DocumentBuilder;
import javax.xml.parsers.DocumentBuilderFactory;
import javax.xml.transform.Transformer;
import javax.xml.transform.TransformerFactory;
import javax.xml.transform.dom.DOMSource;
import javax.xml.transform.stream.StreamResult;
import java.util.Map;
import java.util.Vector;
import org.w3c.dom.Document;
import org.w3c.dom.Element;
import org.w3c.dom.Node;
import org.w3c.dom.NodeList;
import org.w3c.dom.Text;


import org.w3c.dom.Document;
import org.w3c.dom.Element;
import org.w3c.dom.Text;


public class BaseValidate
{


     private int errorStatus = 0;
     private Document docOut;
     private Element root_element = null;
     private Element errMsg = null;
     private String count = "";
     private Text errMsgValue = null;
     private Document document;
     private DOMSource domS =null;
     private Element file_type = null;
     private Vector b;
     private Element ROW = null;
     private Element recordset1 = null;
     private Element data = null;
     private int found_header = 0;
     private int found_trailer = 0;
     private Vector Parent = new Vector(100);
     private Vector subParents = new Vector(100);
     private boolean KeyValueFound = false;
     private int index = 0;
     private int indexChild = 0;
     private int counter = 0;


     public void mainFunction(InputStream in, OutputStream out, Vector v2, String ele)
     {
          String keyId = null;
          String record = null;
          String a = null;
          String keyId_actual = null;


          try
          {


               DocumentBuilderFactory factory = DocumentBuilderFactory.newInstance();
               DocumentBuilder builder = factory.newDocumentBuilder();
               //create a DOM structure for input
               document = builder.parse(in);
               //create a DOM structure for output
               docOut = builder.newDocument();
               TransformerFactory tf = TransformerFactory.newInstance();
               Transformer transform = tf.newTransformer();
               // look at the tag 'ROW'
               NodeList list = document.getElementsByTagName("ROW");
               // need to be changed for every interface - create the root node
               root_element = docOut.createElement(ele);
               Text rowValue = docOut.createTextNode(ele);
               root_element.appendChild(rowValue);
               docOut.appendChild(root_element);
               recordset1 = docOut.createElement("recordset");
               Text recordsetValue1 = docOut.createTextNode("recordset");
               recordset1.appendChild(recordsetValue1);
               root_element.appendChild(recordset1);
               data = docOut.createElement("DATA");
               Text dataValue = docOut.createTextNode("DATA");
               data.appendChild(dataValue);
               root_element.appendChild(data);
               //end of modification


               System.out.println(v2.size());
               Hierarchy h = new Hierarchy(v2);
               System.out.println(h.v.size());
               for(int m =0; m<h.v.size(); m++)
               {
                    //to get all the parent nodes
                    Hierarchy hierarchy1 = (Hierarchy) h.v.get(m);


                    String[] recordParent = hierarchy1.childParent[0].split(",");
                    if(recordParent[2].equals("null"))
                         Parent.add(recordParent[0]);
                    else
                    {
                         if (!(subParents.contains(recordParent[2])))
                              subParents.add(recordParent[2]);
                    }
               }
               System.out.println("Sub parent = " + subParents);
               b = new Vector();
               for(int i=0; i<list.getLength(); i++)
               {
                    //To get all the incoming nodes in a Vector for Hierarchical validations
                    Node node = list.item(i);
                    node = node.getFirstChild();
                    record = node.getNodeValue();
                    String[] recordValue = record.split(",");
                    String transId = recordValue[0];
                    transId = transId.substring(1, (transId.length()-1));
                    b.add(i,transId);
                    ROW = docOut.createElement("ROW");
                    rowValue = docOut.createTextNode(record);
                    ROW.appendChild(rowValue);
                    root_element.appendChild(ROW);
               }


               // create the FILE_TYPE Node alone
               file_type = docOut.createElement("FILE_TYPE");
               root_element.appendChild(file_type);


               for (int i=0; i<list.getLength(); i++)
               {
                    //if found then look for the value of row
                    Node node = list.item(i);
                    node = node.getFirstChild();
                    String recordset = node.getNodeValue();
                    if (recordset.endsWith(","))
                         recordset += " ";
                  String[] recordsetValue = recordset.split(",");
                    keyId_actual = recordsetValue[0];
                    keyId = keyId_actual.substring(1,(keyId_actual.length()-1));
                    KeyValueFound = false;
                    counter++;
                    count = "" + counter ;
                    System.out.println("Key Id = " + keyId);
                    checkParentExists(keyId, recordsetValue, h,i);
                    if (KeyValueFound == false)
                    {
                         String msg = keyId_actual + ": Segment Unknown at Line ";
                         msg = msg + count;
                         createErrorNode(msg);
                    }
               }


               if (found_header == 0)
               {
                    createErrorNode("A00: Header Missing");
               }


               if(found_trailer == 0)
               {
                    createErrorNode("Z99: Trailer Missing");
               }
               String fileStatus = "" + errorStatus;
               errMsg = docOut.createElement("STATUS");
               errMsgValue = docOut.createTextNode(fileStatus);
               errMsg.appendChild(errMsgValue);
               root_element.appendChild(errMsg);


               domS = new DOMSource(docOut);
               transform.transform((domS),new StreamResult(out));


          }catch(Exception e){e.printStackTrace();}


     }


     private void checkParentExists(String keyId, String[] recordsetValue, Hierarchy h, int keyIndex)
     {
          Hierarchy  hierarchy2, hierarchy1;
          boolean parentFound;
          boolean childFound = true;
          String childSearch = new String();
          for(int i=0; i<h.v.size(); i++)
          {
               hierarchy1 = (Hierarchy) h.v.get(i);
               if(hierarchy1.transType.equals(keyId) )
               {
                    String []record = hierarchy1.childParent[0].split(",");
                    KeyValueFound = true;
                    System.out.println("Key Id in check parent = " + KeyValueFound );
                    if(record[1].equals("header"))
                         found_header = 1;


                    if(record[1].equals("trailer"))
                         found_trailer = 1;


                    // check for hierarchy
                    if(subParents.contains(keyId))
                    {
                         for(int l = i1; l<h.v.size(); l+)
                         {
                              //to check if a mandatory child is missing
                              hierarchy2 = (Hierarchy) h.v.get(l);
                              for(int s = 0; ; s++)
                              {
                                   if(hierarchy2.childParent[s] == null)
                                        break;
                                   else
                                   {
                                        String[] recordChild = hierarchy2.childParent[s].split(",");
                                        if((recordChild[1].equals("header")) || (recordChild[1].equals("trailer")))
                                             break;
                                        if (recordChild[2].equals(keyId) && recordChild[3].equals("true"))
                                        {
                                             childFound = false;
                                             childSearch = recordChild[0];
                                             if(b.contains(recordChild[0]))
                                             {
                                                  for(int k =keyIndex1; k<b.size(); k+)
                                                  {
                                                       String c = (String) b.get(k);
                                                       if(recordChild[0].equals(c))
                                                       {
                                                            childFound = true;
                                                            break;
                                                       }
                                                       else if (Parent.contains(recordChild[0]))
                                                       {
                                                            childFound = false;
                                                            break;
                                                       }
                                                  }
                                             }
                                        }
                                   }
                              }
                              if(childFound == false)
                              {
                                   createErrorNode(childSearch + " - Mandatory child " + keyId  + " Missing after Line: "+ count);
                                   System.out.println(childSearch);
                                   childFound = true;
                              }
                         }
                    }


                    if(!(record[2].equals("null")))     //if the node is a child node
                    {
                         for(int s = 0; ; s++)
                         {
                              parentFound = false;
                              if(hierarchy1.childParent[s] == null)
                                   break;
                              else
                              {
                                   String[] nChild = hierarchy1.childParent[s].split(",");
                                   if(s == 0)
                                        indexChild = b.indexOf(nChild[0],(indexChild+1));
                                   for (int k= indexChild-1; k>0; k--)
                                   {
                                        String PrevElement = (String) b.get(k);
                                        if(PrevElement.equals(nChild[2]))
                                        {
                                             parentFound = true;
                                             break;
                                        }
                                   }
                                   if(parentFound == true)
                                        break;
                              }
                         }
                         if(parentFound == false)
                         {
                              String msg = keyId + "- Incorrect parent at Line: ";
                              msg = msg + count;
                              createErrorNode(msg);
                         }


                    }
                    checkFieldsRecordset(recordsetValue, hierarchy1.Fields, hierarchy1.transType,record[1], hierarchy1);
                    break;
               }
          }     // end of for loop


     }


     private void checkFieldsRecordset(String[] recordsetValue, String[] fields, String recordId,String recordType, Hierarchy hierarchy)
     {
          String fileType = null;
          String orgId = null;
          int flag_v;
          int flag_i;
          int column;


          int recordLen = recordsetValue.length;
          try
          {
               for(int i = 0; ; i++)
               {
                    if((fields.length -1) != recordLen)
                    {
                         createErrorNode("Number of fields incorrect at Line: "+count);
                         break;
                    }
                    else
                    {
                         if(     fields[i] != null)
                         {
                              String[] fieldAttribute = fields[i].split(",");
                              if(recordType.equals("header"))
                              {
                                   if (fieldAttribute[0].equals("FILE_TYPE"))
                                   {
                                        Text fileTypeValue = docOut.createTextNode(recordsetValue[i]);
                                        file_type.appendChild(fileTypeValue);
                                        fileType = new String(recordsetValue[i]);
                                        fileType = fileType.substring(1,(fileType.length()-1));
                                   }
                              }


                              if(recordType.equals("trailer") )
                              {
                                   if(convertToIntegerString(recordsetValue[i],"NUMC") && recordId.equals("Z99"))
                                        if ( (fieldAttribute[0].equals("RECORD_COUNT")) && ((counter - 2) != Integer.parseInt(recordsetValue[i])))
                                              createErrorNode("The number of detail records contained within the file does not match to RECORD_COUNT in " + recordId);
                              }


                              if( fieldAttribute[2].equals("NUMC") )
                               {
                                   checkNumericMandatory(fieldAttribute[0],fieldAttribute[1],recordsetValue[i],fieldAttribute[3],recordId,fieldAttribute[2]);
                               }
                               else if ( fieldAttribute[2].equals("DOUB"))
                               {
                                   checkNumericMandatory(fieldAttribute[0],fieldAttribute[1],recordsetValue[i],fieldAttribute[3],recordId,fieldAttribute[2],fieldAttribute[4]);
                               }


                              if(fieldAttribute[2].equals("TEXT"))
                                   checkTextMandatory(fieldAttribute[0],fieldAttribute[1],recordsetValue[i],fieldAttribute[3],recordId);


                              if(fieldAttribute[2].equals("DATE"))
                                   checkDateMandatory(fieldAttribute[0],fieldAttribute[1],recordsetValue[i],fieldAttribute[3],recordId);


                              if(fieldAttribute[2].equals("TIME"))
                                   checkTimeMandatory(fieldAttribute[0],fieldAttribute[1],recordsetValue[i],fieldAttribute[3],recordId);


                         }
                         else
                              break;
                    }


               }
          }
          catch(Exception ex)
          {
                    ex.printStackTrace();
          }
     }


     private void checkTimeMandatory(String fieldName, String mandatory, String recordsetValue, String maxLen, String recordId)
     {
          if (recordsetValue.equals("") || recordsetValue.equals(" "))
          {
               if (mandatory.equals("true"))
               {
                    String msg = fieldName.concat("- Mandatory Segment Missing under ");
                    msg = msg.concat(recordId);
                    msg = msg.concat("at Line: ");
                    msg = msg.concat(count);
                    createErrorNode(msg);
               }
          }
          else
          {


               if ( recordsetValue.length() == 6 )
               {
                    if ( validateTimeString(recordsetValue) == false )
                    {
                         String msg = fieldName + "- Incorrect Time at Line: ";
                         msg = msg + count;
                         createErrorNode(msg);
                    }
               }
               else
               {
                    String msg = fieldName + "- Incorrect Time at Line: ";
                    msg = msg + count;
                    createErrorNode(msg);
               }
          }
     }


     private void checkDateMandatory(String fieldName, String mandatory, String recordsetValue, String maxLen, String recordId)
     {
          if (recordsetValue.equals("") || recordsetValue.equals(" "))
          {
               if (mandatory.equals("true"))
               {
                    String msg = fieldName + "- Mandatory Segment at Line: ";
                    msg = msg + count;
                    createErrorNode(msg);
               }
          }
          else
          {


               if ( (recordsetValue.length()) == 8 )
               {
                    if ( validateDateString(recordsetValue) == false )
                    {
                         String msg = fieldName + ": Incorrect Date at Line: ";
                         msg = msg + count;
                         createErrorNode(msg);
                    }
               }
               else
               {
                    String msg = fieldName + ": Incorrect Date under ";
                    msg = msg + recordId;
                    createErrorNode(msg);
               }
          }
     }


     private void checkNumericMandatory(String fieldName, String mandatory, String recordsetValue, String maxLen, String recordId, String dataType)
     {
          if (recordsetValue.equals("") || recordsetValue.equals(" "))
          {
               if (mandatory.equals("true"))
               {
                    String msg = fieldName + "- Mandatory Segment Missing at Line: ";
                    msg = msg + count;
                    createErrorNode(msg);
               }
          }
          else
          {
               if (recordsetValue.endsWith("-"))
                    recordsetValue = recordsetValue.substring(0,(recordsetValue.length()-1));
               if ( convertToIntegerString(recordsetValue,dataType) )
               {


                    int len = Integer.parseInt(maxLen);


                    if (dataType.equals("DOUB") )//decimal also contain within string
                         len = len + 1;
                    if ( recordsetValue.length() > len )
                    {
                         String msg = fieldName + ": exceeds the maximum length at Line: ";
                         msg = msg + count;
                         createErrorNode(msg);
                    }


               }
               else
               {
                    String msg = fieldName + " - Incorrect number at Line: ";
                    msg = msg + count;
                    createErrorNode(msg);
               }
          }
     }


     private void checkNumericMandatory(String fieldName, String mandatory, String recordsetValue, String maxLen, String recordId, String dataType, String decimal)
     {
          String str;
          checkNumericMandatory(fieldName, mandatory, recordsetValue, maxLen, recordId, dataType);
          if( decimal!=null && decimal.length() >0 )
          {
               int len = Integer.parseInt(decimal);
               int index = recordsetValue.indexOf(".");
               if(index>0)
               {
                    index++;
                    if (recordsetValue.endsWith("-"))
                         str = recordsetValue.substring(index, (recordsetValue.length()-1));
                    else
                         str = recordsetValue.substring(index);
                    if(len < str.length())
{
String msg = fieldName + " - Incorrect decimal value at Line: ";
msg = msg + count;
createErrorNode(msg);
}
}
}

}


private void checkTextMandatory(String fieldName, String mandatory, String recordsetValue, String maxLen, String recordId)
{
if ((recordsetValue.length() - 2) == 0)
{
if (mandatory.equals("true"))
{
String msg = fieldName + "- Mandatory Segment Missing at Line: ";
msg = msg + count;
createErrorNode(msg);
}
}
else
{
int len = Integer.parseInt(maxLen);

if ((recordsetValue.length() - 2) > len)
{
String msg = fieldName + "- exceeds the maximum length at Line: ";
msg = msg + count;
createErrorNode(msg);
}

if(recordsetValue.startsWith("""))
{ }
else
{
String msg = fieldName + "- is not a Text field at Line: ";
msg = msg + count;
createErrorNode(msg);
}
}
}

private void createErrorNode(String msg)
{
errMsg = docOut.createElement("ERR_RECORD");
errMsgValue = docOut.createTextNode(msg);
errMsg.appendChild(errMsgValue);
root_element.appendChild(errMsg);
errorStatus = 1;
}

/* private void checkGenNumber(String fileName ,String genNum_fileType, String orgId)
{
orgId = orgId.substring(1, orgId.length() - 1);
String filename = fileName + orgId;
filename = filename + ".txt";
genError = false;
try
{
File f = new File(filename);
if(f.exists())
{
BufferedReader genIn = new BufferedReader(new FileReader(filename));
String str = "";
while ((str = genIn.readLine()) != null)
{
if(str.equals(genNum_fileType))
{
createErrorNode("Duplicate File");
genError = true;
break;
}
}
genIn.close();
}
if(genError == false)
{
BufferedWriter genOut = new BufferedWriter(new FileWriter(filename, true));
genOut.write(genNum_fileType);
genOut.newLine();
genOut.close();
}

}
catch(Exception e)
{
e.printStackTrace();
}

}
*/
private boolean convertToIntegerString(String s, String dataType)
{

try
{

if(dataType.equals("NUMC"))
{
long l = Long.parseLong(s);
}

if(dataType.equals("DOUB"))
{
new Double(s);
}

}
catch(NumberFormatException nex){return false;}
catch(Exception e) {e.printStackTrace(); }
return true;
}

private boolean validateDateString(String str)
{

try
{
//date format recirved str = 20050411
//check year
String strYear = str.substring(0,4);
int year = Integer.parseInt(strYear);

if (year < 1900 )
return false;

//check month
String strMonth = str.substring(4,6);
int month = Integer.parseInt(strMonth);

if(month<1 || month>12)
return false;

//check date
String date = str.substring(6);
int dateValue = Integer.parseInt(date);

if (dateValue < 1)
return false;
else
{
if (month%2==0)
{
if ( month > 8 && dateValue >31)
return false;

if (year%4 == 0 && month==2)
{
if(dateValue>29)
return false;
}
else
{
if (year%4 != 0 && month==2)
{
if(dateValue>28)
{
return false;
}
}
}

if ( dateValue > 30)
return false;
}
else
{
if(month >8 && dateValue > 30)
return false;
else if(dateValue >31)
return false;
}
}
}catch(Exception e) {e.printStackTrace();}
return true;
}

private boolean validateTimeString(String s)
{

try
{
//time format recirved s = HHMMSS
//check hour
String hour = s.substring(0,2);
int hourValue = Integer.parseInt(hour);

if (hourValue < 0 || hourValue > 23 )
return false;

//check minute
String minute = s.substring(2,4);
int minuteValue = Integer.parseInt(minute);

if(minuteValue<0 || minuteValue>59)
return false;

//check second
String sec = s.substring(4);
int secValue = Integer.parseInt(sec);

if (secValue < 0 || secValue>59)
return false;

}catch(Exception e) {e.printStackTrace();}
return true;
}

}






In InterfaceSpecificValidate.java program we need to specify the structure of the incoming file.



Source Code for InterfaceSpecificValidate.java







ASSUMPTIONS


There are some things that need to be taken care in the Interface Specific Validation program:



1. The size of the FIELDS array must be actual size of the array + 1.




2. The following variables of HierarchyBuild() method in the InterfaceSpecificValidate java class depict the following:

==> TransType: Specify Record Id

==> ChildParent: Must contain Hierarchical Information

==> Fields: Should contain FieldName, Mandatory/optional, TEXT/NUMC/DATE/TIME, Maximum Allowed Length




3.The array of class Variable childParent must necessarily be specified in the format specified below:


childParent[0] = new String("A, B,C, D ");

The values that should be specified for the following fields are:

A -> Record Id

B -> header/trailer/space -> If Record Id is not a Header or Trailer record, then specify SPACE

C -> null/Parent Id -> If Record Id deos not have Parent then specify ‘null’ else specify Parent node

D -> true/false – If Record Id is a Mandatory Child of Parent mentioned in .i>"C", specify "true" else "false"


Note: There should not be any additional spaces which specifying Childparent.




4. Fields array in the above program must be specified in the following format:

Fields[0] = new String("E,F,G,H");



E -> Field Name

F -> true/false (to state mandatory or optional field)

G -> TEXT/NUMC/DOUB/DATE/TIME (Field Indicator)

H -> The maximum length of the field should be specified.

3 Comments