Update 19 May 2015: Added new converter Base64EncodeConverter

Update 30 Apr 2015: Added new converter Base64DecodeConverter

Introduction

Over the past months, I have introduced various custom adapter modules that are configurable and reusable; aiming to handle various format conversions to/from XML that are not available in standard adapter modules.

I have recently refactored the different modules and consolidated them into a single module, FormatConversionBean. The benefits of this refactoring are:-

  • Single point of entry for the different types of conversion
  • Refactoring of factory class utilizing ConverterFactory allows dynamic instantiation of converters
  • Easily extensible to new converter formats by extending class AbstractModuleConverter
  • Conforms to the Open Close Principle whereby new converters can be added without modification to existing code

Converter Class List

Below is the comprehensive list of the converter classes that are part of FormatConversionBean now. This list will be updated as and when new converter classes are introduced in the future.

Converter Class Source Format Target Format Reference
Excel2XMLTransformer Excel XML ExcelTransformBean Part 1: Convert various Excel formats to simple XML easily
XML2ExcelTransformer XML Excel ExcelTransformBean Part 2: Convert simple XML to various Excel formats easily
DeepPlain2XMLConverter Deep Plain Text XML DeepFCCBean – The better FCC at meeting your deep (structure) needs! (Part 2 – Flat File to Deep XML)
XML2DeepPlainConverter XML Deep Plain Text DeepFCCBean – The better FCC at meeting your deep (structure) needs! (Part 1 – Deep XML to Flat File)
JSON2XMLConverter JSON XML JSONTransformBean Part 1: Converting JSON content to XML
XML2JSONConverter XML JSON JSONTransformBean Part 2: Converting XML to JSON content
Base64DecodeConverter Base64 (Plain/XML) Binary Base64DecodeConverter – Base64 decoding made easy!
Base64EncodeConverter Binary Base64 (Plain/XML) Base64EncodeConverter – Base64 encoding made easy!

Usage of Module in Communication Channel

Module Processing Sequence

Number Module Name Type Module Key

<Depending on position of module in chain>

1) Asynchronous scenario,

Normally before the last module in channels

2) Synchronous scenario,

Before last module to convert request payload

After last module to convert response payload

Custom_AF_Modules/FormatConversionBean Local Enterprise Bean <Any Arbitrary Value>

Module Configuration

For configuration of parameters, refer to parameter list of each converter.

Source Code and Deployment Archive

The Java source codes for the module are located in the following GitHub repositories.

GitHub repository for equalize-xpi-modules

The EAR file for deployment (compiled on NWDS 7.31 SP13 Patch 0) is also available from the latest repository release below. The EAR file can be deployed directly on PI/PO systems with versions that are same/higher than 7.31 SP13 or 7.4 SP8.

Latest release for repository equalize-xpi-modules

To set up the EJB/EAR projects on NWDS for further customization, refer to the following.

NWDS version Reference

7.3x or higher

Setup com.equalize.xpi.af.modules Adapter Module project in NWDS easily using EGit

7.1x

Recompile com.equalize.xpi.af.modules as EJB 2.1 modules in NWDS 7.1x

Further Reference

If you plan to download the source codes into your own NWDS installation to make changes and/or develop new modules, the following blog would be useful for performing testing in NWDS prior to deployment into the PI system.

Standalone testing of Adapter Module in NWDS

Guide on EAR deployment via NWDS

Deploying .EAR files Using NWDS

To report this post you need to login first.

30 Comments

You must be Logged on to comment or reply to a post.

    1. Eng Swee Yeoh Post author

      Hi Avinash

      Thanks for your comment. You are the first to inform me that the modules’ EAR file has been successfully deployed and I really appreciate this feedback. It is good to know that it is working on other systems apart from mine.

      Rgds

      Eng Swee

      (0) 
  1. Ryan Crosby

    Nice work Eng Swee!

    I don’t have any specific needs for this now but I will have to remember it for the future.

    Regards,

    Ryan Crosby

    (0) 
  2. Shaibayan Chakrabarti

    Hi Eng Swee Yeoh and Avinash Ayanala ,

    I am working currently on a requirement where the input is proxy and output is a file in excel format. I am using SAP PI 7.31 .

    I have some queries related to this.

    Is the below the correct EAR that we need to deploy or it contains only the JSON2XML converter. My exact need is to convert XML to EXCEL but I want to try the FormatConversionBean as a whole since it contains the entire module and I would not have to deploy any other separately. So do we need to deploy only the below EAR file?

    com.equalize.xpi.af.modules.app.ear 

    I will provide feedback and updates once I am done.

    Please do let me know.

    Thanks,

    Shaibayan

    (0) 
  3. Juan Vasquez

    Hi Eng, can you help me please with this issue

    I am using the Adapter Module success, in the receiver channel

    but i need to send some header values:

    Cache-Control: no-cache

    Content-Type: application/json

    Application-Secret: 6474b25daaaaaaaaaacc7c9c892195352354facd

    where can i send it?

    what is the better channel type soap http (axis)

    the content is in the right way, you can see in the immage

    thanks a lot2015-04-28 13_43_34-Message Editor - SAP NetWeaver Administrator.png

    (0) 
    1. Eng Swee Yeoh Post author

      Hi Juan

      Thanks for your interest in FormatConversionBean. Glad to know that the JSON transformation is working well for you.

      As to your query regarding custom HTTP headers, since it’s unrelated to this module, I’d suggest that you open a new thread in the forum for this issue instead.

      As always, please provide as much details/screenshots as possible in the thread, i.e. PI/PO version & SP, scenario overview & details, adapter types, error messages, etc.

      Rgds

      Eng Swee

      (0) 
  4. Yee Loon Khoo

    Hi Eng Swee,

    Feedback on my trial.

    Deployed EAR in PO server 7.31, so far tested excel2xml working well, able to convert accordingly to xml. Nice! Many end users still love excel for manual upload, this will help a lot.

    Your blog, example and github code is very useful and inspiring. Thanks again.

    Regards,

    Yee Loon

    (0) 
    1. Eng Swee Yeoh Post author

      Dear Yee Loon

      Thanks for your feedback. I am very pleased to hear that is is working well. Please feel free to let me know if you do encounter any issues if you do get around to testing the modules more.

      Best regards,

      Eng Swee

      (0) 
  5. Varun Mukund

    HI Eng Swee Yeoh,

    Firstly, let me congratulate you on the blog. It is awesome and very helpful.

    I deployed the module and tested the Deep flat to XML conversion. I wanted to report a small bug. On the variable for fixedFieldLengths since you are doing a number format check after removing commas, the module will throw error any time you exceed 2,147,483,647 which is the maximum value int type supports. May be you can use a string.matches(“\\d+?”);


    Once again, amazing work πŸ™‚


    Regards

    Varun

    (0) 
    1. Eng Swee Yeoh Post author

      Hi Varun

      Thanks for your kind comments and also the feedback regarding the bug.

      Would you be able to provide some more details so that I can try to reproduce it? Can you provide screenshots of the module configuration as well as the error you are getting?

      Rgds

      Eng Swee

      (0) 
      1. Varun Mukund

        Hi Eng Swee Yeoh,

        To reproduce the error, you just need to do an FCC for a fixed field length file and you need to ensure that the recordset.fixedFieldLengths parameter has a value which exceeds 2147483647 after you remove the commas. I think if you test with a file having 5 columns with each column having a fixed length of 25, you should get the error.

        The exact location of error is at the class com.equalize.xpi.af.modules.util.RecordTypeParameters on line 54. I removed the try/catch block and used the below snippet. It worked like a charm πŸ™‚

        if(!lengthsWithoutComma.matches(“\\d+?”))

        {

        throw new ModuleException(“Maintain only integers separated by commas for ‘” + fieldFixedLengthsName);

        }

        (0) 
        1. Eng Swee Yeoh Post author

          Hi Varun

          Ah, thanks for elaborating further. I’ve fixed the bug accordingly and published a new release for the EAR file as well as the source code.

          Thanks again for reporting the bug. It is much appreciated! πŸ™‚

          Rgds

          Eng Swee

          (0) 
  6. Anil Kumar Vaitla

    Hi Eng Swee Yeoh,

    Tried DeepPlain2XML and XML2DeepPlain in the File and JMS adapters and they worked great.

    Thanks for the detailed blog it was very helpful.

    (0) 
  7. Vadim Klimov

    Hi Eng Swee,

    Thank you for developing and sharing this module – very helpful, and your efforts are very much appreciated. Recently we used it for XML to JSON conversion in JMS based integration scenario – worked perfectly well.

    One question to you. Manifest contained in a project, specifies implementation vendor ID = “sap.com”. As a result, corresponding component gets deployed with vendor = “sap.com”, which is normally reserved / used by SAP standard components. Would you suggest downloading source code provided by you, adopting manifest and building deployable, or may you adjust vendor identifier to something different from “sap.com” and rebuild EAR so that it already contains custom vendor identifier?

    Regards,

    Vadim

    (0) 
    1. Eng Swee Yeoh Post author

      Hi Vadim

      Thank you for your feedback. I’m really glad to hear that the conversion worked well.

      Regarding the Manifest file, to be honest, this is something that I’m not that familiar with. I checked both the EAR and EJB projects – there is no manifest file maintained in the EAR project and the one in the EJB project does not specify the vendor ID (very little info in that file).

      It looks like the MANIFEST.MF file within the EAR deployable file is automatically generated when I perform Export > SAP EAR file for the EAR project. I guess I can of course manually edit the entry in the generated EAR file, but do you know if there is some other automatic way to achieve this – maybe manually maintaining a manifest file in the EAR project?

      Have you adjusted the vendor identifier before for some custom adapter module that you have developed? Let me know if you do, I’m all up for modifying it accordingly for the EAR files I provide.

      Rgds

      Eng Swee

      (0) 
      1. Ryan Crosby

        Hi Eng Swee,

        I was able to achieve this in the past by adding the following tag into the application-j2ee-engine.xml for the EAR file:

        <provider-name>idexx.com</provider-name>

        Regards,

        Ryan Crosby

        (0) 
      2. Vadim Klimov

        Hi Eng Swee,

        I assume, if manifest files (MANIFEST.MF and SAP_MANIFEST.MF) are not present in project structure, then NWDS auto-generates them when building EJB and EAR projects, and uses standard attribute values (this is where vendor = “sap.com” comes from). Since in built EAR file, after I decompile it, I can see manifest files:

        com.equalize.xpi.af.modules.app decompiled.png

        I resolved this issue by explicit creation of manifest files within a project. I didn’t create them in NWDS (but it is worth trying this – probably, it will be much simpler), but used build tools and specified what kind of manifest file I would like to assemble when building EJB project. To be more precise, I used Gradle (http://gradle.org/) in order to automate bundling the deployable. In Gradle script, I made use of Java plugin and its property manifest, where I provided attributes and their custom values which I want to put into manifest file. Usage of Gradle purely for this task may be a bit overkilling – but I used Gradle also for dependency management (especially when adapter module / custom job was using external libraries and I wanted to insert some of them, but not all, into the same deployable) and entire compilation and assembly of JARs and EARs, so manifest creation was just one of tasks of executed Gradle script.

        Regards,

        Vadim

        (0) 
      3. Eng Swee Yeoh Post author

        Hi Vadim, Ryan

        Thank you so much for both your feedback. I’ve definitely learnt something new πŸ™‚

        I’ve managed to adjust the value generated in the MANIFEST.MF file by adding the provider name into the application-j2ee-engine.xml file as suggested by Ryan. That worked really well and once it’s deployed into the SAP system, the vendor value is reflected correctly in the different views in NWA.

        I’ve updated the source code and also released a new EAR file in the GitHub repository for this change.

        Thanks also for the info on Gradle. Will definitely have to look into it at some point! πŸ˜‰

        Rgds

        Eng Swee

        (0) 
  8. bang sungtae

    Thank you

    The information provided by you became great help to me.

    But i was occur problem in my source of NWDS

    APIAccessFactory.java and MessageLogger.java Is not find in MessageLoggerHelper.java

    Where is find JAVA Source? and .jar download url?

    Regards,

    Bang

    (0) 
    1. Eng Swee Yeoh Post author

      Hi Bang

      Glad to know this was helpful for you.

      The pre-packaged Adapter library that comes with NWDS 7.31 are reduced versions and do not contain all classes for some packages. As such, you will need to get the full JAR file for com.sap.aii.af.ms.ifc_api.jar from your PI server.


      Refer screenshot below that shows the package and classes that are in that JAR file. This will enable MessageLoggerHelper.java to compile successfully.

      /wp-content/uploads/2016/01/lib_867776.png

      Refer to the following Wiki on where to get that file in your PI system.

      Where to get the libraries for XI development – Process Integration – SCN Wiki

      After you have retrieved that file, create a User Library in NWDS (give it any name) that includes that JAR file. Then add the newly created User Library to the EJB project’s build path.

      Rgds

      Eng Swee

      (0) 
      1. bang sungtae

        Dear Eng

        Thank you, kind explanation for me

        Is not found MessageLogger.class in com.sap.aii.af.ms.ifc.api.jar in PI 7.3

        So, i’m going to after annotation in MessageLogger.class

        Regards,

        Bang

        (0) 
  9. Hetal Shah

    Hi Eng,

     

    Very nice work with providing us this wholesome solution.

    I am planning to use this solution for my below requirement. I have JSON file which i will need to read and update that data into Database tables.

     

    So can you please let me know once i read this file using File adapter and custom JSONtoXML converter module, how can i map it to the database tables?

     

    Also i will let you know once i deploy your module in our server, we are on SAP PO 7.4 SP9 how does it work out.

     

    Thanks,

    Hetal

    (0) 
  10. Hetal Shah

    Any idea how can i deploy the EAR file on SAP PO 7.4? I do not need to make any changes in the code you provided or any enhancements, hence if you can let me know how to deploy it directly on SAP PO.

     

    Do i still need to have NWDS to deploy the file on the server?

     

    Thanks,

    Hetal

    (0) 
  11. vk k

    Β 

    Hi Eng Swee,

    I have been fan of this bean and I have been using this quiet aΒ number of times.

    Recently, I ran into trouble when I am using xml2json converter in this bean, when I am passing 800 + rows of XML, total of 1700kb , it takes around 9 minutes to perform the conversion.

    Do you think it is normal ?

    Β 

    Regards,

    Vk

    Β 

    (0) 

Leave a Reply