Group, Sort and handle Duplicate XML records – using UDF
Below User Define Functions (UDF) logic can be used to group, sort and handle duplicate XML records.
Note: –
- Number of queue entries in each input field, to UDF should be equal (except key fields). Use ‘mapWithDefault’ for all optional input fields.
- Input fields context should be removed. Use ‘removeContexts’ OR right click on field and then change ‘Context’.
- Create UDF with Execution Type as ‘All Values of Queue’. Input all related fields to one ‘multiple input and multiple output UDF’. When match is found, write all related fields to output.
- Please tryout below examples in sandbox. And then solve your actual requirement.
Grouping XML records
Here is input XML and expected output XML: –
Mapping: –
UDF: –
UDF will search ‘Name_People’ in ‘Name_Record’, if it finds a match, corresponding ‘Where’ and ‘Phone’ will be written to target fields with Context Change. Context Change will be added to output ‘Record_out’ for each ‘Name_People’.
Sorting XML records
Here is input XML and expected output XML: –
Mapping: –
Mapping is same as above, add standard function ‘sort’ to key field. UDF logic is same as above. Output XML will be sorted with Name_People.
Handling Duplicate XML records
Here is input XML and expected output XML: – Duplicate records are ignored.
Mapping: –
UDF: –
HashSet will return true, if unique value is added. It will return false, if duplicate is added.
UDF code for reference
// Grouping XML records.
public void udf_getCorrespondingRecord(String[] Name_People, String[] Name_Record, String[] Where, String[] Phone, ResultList Record_out, ResultList Where_out, ResultList Phone_out, Container container) throws StreamTransformationException{
for(int i = 0; i < Name_People.length; i ++) {
for(int j = 0; j < Name_Record.length; j ++) {
if(Name_People[ i ].equals(Name_Record[ j ])) {
Record_out.addValue(" ");
Where_out.addValue(Where[ j ]); Phone_out.addValue(Phone[ j ]);
Where_out.addValue(ResultList.CC); Phone_out.addValue(ResultList.CC);
}
}
Record_out.addValue(ResultList.CC);
}
}
//If match has to be found using a combination of two fields. Please use logic something like this in above UDF:
//if((Name_People[ i ] + Where[ i ]).equals(Name_People[ j ] + Where[ j ]))
//------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
//------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
// Handle duplicate XML records.
public void udf_getUnique(String[] Id, String[] Name, String[] field1, ResultList Record_out, ResultList Id_out, ResultList Name_out, ResultList field1_out, Container container) throws StreamTransformationException{
Set<String> unique = new HashSet<String>();
for(int i = 0; i < Id.length; i ++) {
if(unique.add( Id[ i ] )) { // True if unique.
Record_out.addValue(" ");
Id_out.addValue(Id[ i ]); Name_out.addValue(Name[ i ]); field1_out.addValue(field1[ i ]);
Id_out.addValue(ResultList.CC); Name_out.addValue(ResultList.CC); field1_out.addValue(ResultList.CC);
}
}
}
Good one Raghu Vamseedhar Reddy KadipiReddy
Well, you don't believe the same concept was in my mind from last few weeks for one of complex interface requirement which was recently go-live but at the moment with a java mapping and I am planning to replace it with a Graphical mapping for a better visibility in general and for enhancements by support team later
May be you can consider adding below points,
Thanks for sharing a nice pattern. And keep blogging your ideas ....
Best Regards,
Praveen Gujjeti
Thank you Praveen, for feedback.
I myself written Java mapping for grouping XML records, previously. I agree with you, now we can implement more transformations using graphical mapping.
Cheers!! and Thanks for a version (V2) change 🙂
good one 🙂 Keep sharing 🙂
Thanks for Sharing this it'll help us like Newbie's in Java
Thanks a lot bro 🙂
Thank you for this, I'll try it out some time. 🙂
Nice blog, Keep sharing.
Good One..Keep blogging !!
Hi Raghu ,Thanks for sharing this it's really helped me a lot.
can you please tell this, for duplicate XML UDF. If I want add or concatenate the Field one . this is the requirement I got .
Please share your thoughts
Thanks
Babu