Data Mining using APriori Algorithm in XI – Part III

[Former Member]

The Support Calculation process is the second phase of the APriori algorithm, which is done after the Candidate Generation process ( jeyakumar.muthu2/blog/2005/12/19/data-mining-using-apriori-algorithm-in-xi-150-part-ii ).

This process counts the Support for every candidate that is generated in the Candidate Generation process.

So, this process needs two input files. One is the output file of Candidate Generation process and other is the initial input file. The first file has the following XSD structure,

The second input file has the following XSD structure,

The support is calculated by the number of times the candidate set has occurred in the itemset. So the output file has the candidates and their corresponding support counts.

The output file has the following XSD structure,

In the message mapping, the source message has two message types.

Next we will look at Message Mapping. In this, both the input files are given as input to the User-Defined function, which will generate the candidate sets and their support counts separately.

In this mapping, we have to change the context of ITEMSET to FC_Input_MT. This is shown in the following picture.

Next is the User-Defined function, which will generate the support counts for every candidate.

The User-Defined Function

Code Sample

ArrayList[] listItem = new ArrayList[arrayItemset.length];ArrayList[] listCandidate = new ArrayList[arrayCandidate.length];ArrayList listResult = new ArrayList();ArrayList listTemp = new ArrayList();ArrayList listCount = new ArrayList();ArrayList listPruned = new ArrayList();int intResult = 1;boolean flag = false; for(int intCounter1 =0; intCounter1 < arrayItemset.length; intCounter1++){ StringTokenizer st = new StringTokenizer(arrayItemset[intCounter1],","); listItem[intCounter1] = new ArrayList(20);  while (st.hasMoreTokens())   listItem[intCounter1].add(st.nextToken()); listItem[intCounter1].trimToSize();}  for(int intCounter1 =0; intCounter1 < arrayCandidate.length; intCounter1++){ StringTokenizer st = new StringTokenizer(arrayCandidate[intCounter1],","); listCandidate[intCounter1] = new ArrayList(20);  while (st.hasMoreTokens())   listCandidate[intCounter1].add(st.nextToken()); listCandidate[intCounter1].trimToSize();}    for(int intCounter1 = 0; intCounter1 < listCandidate.length; intCounter1++) for(int intCounter2 = 0; intCounter2 < listItem.length;intCounter2++)  if((listItem[intCounter2].size() >= listCandidate[intCounter1].size()) && listItem[intCounter2].containsAll(listCandidate[intCounter1]))   listTemp.add(listCandidate[intCounter1]);  for(int intCounter1 = 0; intCounter1 < listTemp.size(); intCounter1++){ for(int intCounter2 = 0; intCounter2 < listResult.size(); intCounter2++){  if(listTemp.get(intCounter1).equals(listResult.get(intCounter2))){   listCount.set(intCounter2,(new Integer(((Integer) listCount.get(intCounter2)).intValue()+ 1)));   flag = true;  } } if(! flag){  listResult.add(intResult,listTemp.get(intCounter1));  listCount.add(intResult, new Integer(1));  intResult++; } else  flag = false;}for(int intCounter1 = 0; intCounter1 < listCount.size(); intCounter1++) result.addValue( ( (Integer) listCount.get(intCounter1) ).toString()  ) ;

The Integration Process

In this integration process, the Fork node has two branches. In every branch, there is one Receive node. The Receive nodes in two branches are responsible for getting the input files. The Fork node gets terminated when the two input files are fetched. Then the mapping is done by the Transformation and the output is sent through the Send node.

Finally, do all the necessary configuration settings for this Support Calculation process.

The Input File

The Output File