Demystifying Custom Adapter Development: Part 1a – Cloning the Sample JCA Adapter
Update 9 May 2016: Due to limitation of number of uploaded images allowed in a post, Part 1 of this series has been split further into two parts (1a – this post, and 1b – Demystifying Custom Adapter Development: Part 1b – Cloning the Sample JCA Adapter)
Introduction
One of the final frontiers of Java development in PI is the development of a custom adapter. Unlike other Java developments in PI (mappings, adapter modules) which have extensive coverage in terms of documentation and guides, custom adapter development materials are few and far between. Although the following SAP Library link provides the steps for Developing a Java Adapter for SAP PI, these steps have not been updated since XI 3.0 days and in practice, the steps are somewhat vague with certain information missing that is required for proper development and deployment of the adapter.
With all these “hurdles” and no proper guide, it is no wonder that the following blog mentions that there is a high learning curve involved and the cost may come up to $500K.
Do you need a custom adapter in PI?
Due to these prohibitive factors, many alternative workarounds have been used to meet the various integration needs. Some of these involves developing IO connection in Java mappings or adapter modules even though these violates EJB’s specification.
Over the next few weeks, I’ll be sharing a series of posts covering the various aspects of custom adapter development. The aim is to shorten the learning curve by illustrating both the basics as well as the key points of such developments.
Without further ado, we will begin with the necessary setup and configuration to reproduce a clone copy of the sample JCA adapter provided by SAP.
Step 1 – Getting All the Necessary Files
As the first step, we need to get all the necessary files required to set up the project correctly. These are:-
i) The source code for the sample JCA adapter
ii) The configuration and deployment descriptor files
iii) The reference JAR library files for adapter development
i) Sample adapter source code
The source code is no longer provided in the installation SCA files. However these are available as an attachment in an older version of the following SAP Note:-
1004000 – SAP NetWeaver PI Adapter/Module Development: API Changes
Switch to version 20 of the note and download the attached ZIP file.
Extract the ZIP file into a suitable temporary location in the local computer.
ii) Configuration and deployment files
These files are not included in the above ZIP file (and even the external link provided in the SAP Note is not working). However, these are included in the Adapter Framework SCA file that can be downloaded from SMP.
Go to the Download Software section of SMP, and select Support Packages & Patches.
Navigate to the section for XI Adapter Framework as shown below and download the XIAF SCA file. Note that although the screenshot shows the SCA file that is relevant to my PI system, it is actually not really important which version of PI or SP level that is downloaded, as we just need a copy of the files.
Open the SCA file in a ZIP utility like 7-Zip, and navigate to the following RAR entry within the SCA file to access the sample adapter’s file.
Open the RAR file and extract the following two folders to a temporary location in the local computer.
iii) PI adapter development reference libraries
Although NWDS 7.31 (and above) contains built-in libraries that are sufficient for mapping or adapter module development, there are a few more library files required for adapter development.
Retrieve the following four files from the PI system (either by accessing them directly at OS level or using an approach like Browsing File System of a Java-only System)
/usr/sap/<SID>/J<nr>/j2ee/cluster/bin/ext/security.class/lib/sap.com~tc~sec~ssf.jar
/usr/sap/<SID>/J<nr>/j2ee/cluster/bin/ext/tc~bl~txmanager~plb/lib/private/sap.com~tc~bl~txmanagerimpl~plb~impl.jar
/usr/sap/<SID>/J<nr>/j2ee/cluster/bin/core_lib/sap.com~tc~bl~guidgenerator~impl.jar
/usr/sap/<SID>/J<nr>/j2ee/cluster/bin/interfaces/appcontext_api/lib/private/sap.com~tc~je~appcontext_api~API.jar
Step 2 – Setting Up the Project in NWDS
With all the necessary files in place, we can proceed to set up the following projects in NWDS:-
i) Java Project for the adapter source codes
ii) Connector Project for the RAR file generation
i) Adapter Java Project
Create a new Java project in NWDS. The project name is arbitrary but some of the subsequent files will refer to it, so it is recommended to follow some form of naming convention.
Next, create a package within the project. The package name needs to be exactly com.sap.aii.af.sample.adapter.ra (don’t worry, this will be refactored later) in order to match the package declaration in the sample source codes.
With the package in place, copy the sample source codes that were extracted above (in Step 1 – i) and paste it into the package.
Initially, there will be various errors for the source codes. These need to be rectified by configuring the build path. Add the following two built-in libraries to the project.
Next, add the four PI adapter libraries as External JARs as shown below. Once this is done, the project will compile and build successfully.
ii) Connector Project
Select File > New > Connector Project to create the RAR project and accept the default values.
The project will be created with default files in the META-INF folder.
These need to be replaced by the following seven files that were extracted from the XIAF SCA file above (in Step 1 – ii).
Copy and paste the files into the META-INF folder (overwriting any existing files). The final view of the RAR project is as follows.
Step 3 – Modifying the Source Code and Configuration Files
As the existing sample JCA adapter is normally already deployed in the PI system, we need to make some modifications to the codes and configuration files in order to be able to deploy a clone copy of the adapter.
i) Rename the package in the Java project
Right click on the package com.sap.aii.af.sample.adapter.ra and select Refactor > Rename, providing an appropriate name according to a suitable naming convention. This automatically updates all the underlying source codes with the new package name.
ii) Update adapter type and namespace in SPIManagedConnectionFactory.java
Open the above class, and navigate to the run() method. Modify the values for adapterType and adapterNamespace.
Note that this is a very crucial step, as it affects the actual registration of the adapter to the framework during deployment. The combination of these two values must be a unique pair.
iii) Update the deployment descriptor ra.xml
Open the ra.xml file in the META-INF folder of the RAR project. Update the following three values with the same package name that was used in item i) above.
In the same file, update the following values for the adapter type and namespace with the same value used in item ii) above.
iv) Update log-configuration.xml
Update the following values in the log configuration files. These values affect the logging that is displayed in NWA’s Log Viewer.
v) Update the deployment file SAP_MANIFEST.MF
This file affects the deployment of the adapter. The combination of keyvendor and keyname values must be a unique pair so that it does not clash with other applications that are already deployed in the system.
The continuation steps for Part 1 are listed in the following post:-
Demystifying Custom Adapter Development: Part 1b – Cloning the Sample JCA Adapter
Other Parts of this Series
Demystifying Custom Adapter Development: Part 1b – Cloning the Sample JCA Adapter
Demystifying Custom Adapter Development: Part 2 – Examining the Adapter’s Key Classes and Methods
Demystifying Custom Adapter Development: Part 3 – Examining the Deployment and Metadata Files
Demystifying Custom Adapter Development: Part 4 – Modifying the Adapter’s Functionality
Hi Eng,
Excellent blog, thank for sharing this.
Regards,
Praveen.
Hi Praveen
Thanks for your comment. This took a while coming, but hopefully it will be beneficial to the community 😉
Regards
Eng Swee
Hi Eng,
Some of the screen shots are not appearing in the blog, can you fix them?
Regards,
Praveen.
Hi Eng,
Superb one. Thanks a lot for writing another very useful and informative blog. This will definately help the community.
Thanks,
Apu
Hi Eng,
Thanks for sharing this wonderful blog.
Regards
Bhargava Krishna
Hi Eng Swee,
Really outstanding and long awaited blog! Such blog is a "must have" reading for those interested in advanced topics in PI/PO development space. Level of details and hands-on examples that you embedded in the blog, definitely provide good level of clarity in regards to different bits and aspects that form the work flow of the adapter development. Lack of high quality materials in the area of custom adapter development makes it even much more valuable and shall lift the veil of secrecy of how the custom adapter can be created.
Thank you for your efforts you invested into preparation of this blog and bringing it to SCN community. Hats off for your ongoing hard work and looking forward to reading your next blogs in this series!
Best regards,
Vadim
Really nice write up of how to make an adapter. I think you covered all the relavant points for it.
Excellent blog Eng Swee, this should take a place in ever PI developer's blog series.
Excellent.! Much needed.
Thanks and Regards,
Sunil
Excellent , thanks for sharing it ...
Thanks everyone for all the positive feedback 🙂
Praveen,
I had some issues with the content editor while preparing this post. I thought these were already fixed when I published the final version. I have since checked this post on a number of computers and do not see any issues with any of the images. Can you try again (maybe refresh cache) or on another computer? If it still does not work, can you please list down which image (step and sub-step numbers) that are not displaying, and I'll upload them again? Thanks.
Vadim,
Thanks for such encouraging words. In the past, custom adapters have been nearly exclusively restricted to the realm of SAP partners. I hope that this will help bring such development to the "masses". On one hand, there continue to be new challenges in integration that are not currently supported by SAP's offerings. On the other hand, there is a wealth of expertise within this community that have the capabilities to meet these challenges. My hope is that this provides a viable alternative to developers to meet these new challenges. Along those lines, I'd also wish that this will open up opportunities for open-source solutions developed by the community that can complement SAP's own functionality.
Regards
Eng Swee
Excellent blog ...!! as always it is explained nicely..!! Keep up d good work..
Regards
S Tomar
Hi Eng,
Thank you for this excellent blog !
Some of the images are basically giving an authorization error and appearing only as a link - would be really helpful if you could please update these.
1) Step 4 - section ii) : metadata2.png : http://scn.sap.com/servlet/JiveServlet/showImage/102-72414-18-940128/metadata2.png
2) Step 5 - section ii) : exp_rar1.png: http://scn.sap.com/servlet/JiveServlet/showImage/940151/exp_rar1.png
3) Step 6: channel.png : http://scn.sap.com/servlet/JiveServlet/showImage/102-72414-18-940074/channel.png
monitor.png: http://scn.sap.com/servlet/JiveServlet/showImage/102-72414-18-940075/monitor.png
Regards
Subhro
Hi Subhro
Thanks for your feedback and also listing the images that were problematic.
I just realised from the following thread that there were some limitations on the uploaded pictures.
Limit on no of pictures uploaded in a blog? | SCN
This might have been the cause of the issue I faced whilst preparing the post. Due to the limitation mentioned, I have now split this post into part 1a and 1b. Do check if you still have any issues accessing the images on both parts. If you do, I would appreciate it if you let me know.
Thanks again,
Eng Swee
Hello Eng Swee,
I thought I'd get around to trying this out and when I did, I hit one snag.
SPIManagedConnectionFactory.java uses import com.sap.guid.GUID;
I was getting a class Not found as this class was not in my build path and hence ended up using the Jar as mentioned in this blog : Finding com.sap.guid.IGUID (or any other class) on the PI Server
I am on NWDS 731 SP 06 corresponding to my PI Server and hence I think this could be a requirement for lower versions of NWDS. I will keep you updated on how my journey pans out as I continue from here on!
Regards
Bhavesh
Hi Bhavesh
The class is included in JAR file sap.com~tc~bl~guidgenerator~impl.jar that is listed in Step 1 (iii).
Let me know how your journey goes 🙂 Always open to feedback if there are any parts that can be improved.
Regards
Eng Swee
You are indeed right. I definitely need to get my eyes tested 🙂
Will definitely share my feedback but honestly so far this has been extremely detailed! Much appreciated!
You really should - I'm concerned cause this is not the first time you've said this 😛 😆
I'm trying to figure out, what is the entry point into the adapter? Where does the data get into the
data get into the adapter? In the SPIManagedConnectionFactory.java file there the methods
start() and startMCF(). start() is executed when the JCA adapter is started by the J2EE component.
If this is the entry point; how can data be sent to a specific channel from the
SPIManagedConnectionFactory instance? Reading further through the code there
are lines like:
that seem to use a ModuleProcessorFactory to get an instance of ModuleProcessor
and then send a ModuleData object to a specific channel known by its
identifier. Does the ModuleProcessor write the ModuleData object to the
channel here?
I need to write an adapter that is able to periodically update an outbound
channel and am not sure if this is a good starting point.
Hi Sbt M
Try to read through the other parts of this blog series, I've covered the receiver adapter in Part 4 and sender adapter (in polling mode) in Part 5.
Not too sure what you are trying to achieve, but if you still can't get by, I'd suggest you open up a thread in the Q&A section.
Regards
Eng Swee
Hi Eng Swee,
I’ve read all posts and opened a thread here.
I’m working on an adapter that must connect to an EIS, retrieve some records and write them to a
channel.
From my understanding, the work flow of the adapter shown in this tutorial is the following:
(1) SPIManagedConnectionFactory emulates a client in a separate thread reading contents
from local files
(2) It then creates a ModuleData object which it sends to a ModuleProcessor instance
(3) The ModuleProcessor then sends this data to a CCInteraction object that
manages the communication between the adapter and an EIS
Something like (taken from the code):
Is this logic correct?
Regards,
Sebastian
Hi Eng Swee,
Just to let you know SAP Note 1004000 is updated and a link to sample adapter source code is included, so there is no need for switching to version 20 in order to download the source code.
Regards,
Kristina
Hi Kristina
Thanks for your comment. However, I can't seem to find it in the latest version 23 - any thoughts?
Hi Eng Swee!
Isn't it what you're looking for?
Regards, Evgeniy.
Hi Evgeniy
You are absolutely right. There was an MDocs link there before but it never worked in the past, so it didn't cross my mind that Kristina was referring to that. By "a link to the source code", I immediately thought it was a link in the attachment section of the Note like how it was in version 20.
Good catch, and thanks for pointing me in the right direction.
Eng Swee
Hi Eng Swee!
Thanks for your wonderful blogs.
Regards, Evgeniy.
Hi Eng,
Thank you for such a detailed and wonderful series of blogs...
I was trying to open the sample code file from the link in V23 of the note and I am getting an error that access for share has expired. Also we are not able to change the version to V20 in the updated launchpad.
Any Ideas to get the sample code file?
Thank you,
Ajay
Hi Ajay
I provided a link to my GitHub repository in part 5. It contains the code post the changes implemented throughout the series.
Regards
Eng Swee
Thanks Eng.. Got It!!!! 🙂
Hi Eng,
Amazing blog … I have followed the steps and I have created my own receiver adapter, which working fine for Sync & Async messages.
Anyway, I have noticed, that my adapter is not working in Async – Sync bridges …
The replaceInterface (ResponseOnewayBean) parameter does not substitute the sender interface as it should, and the response ICO is not found.
I have noticed in several posts … that this module and parameter (ResponseOnewayBean – replaceInterface) does not work with several adapters (RFC …).
So … I have the following doubt:
Thanks in advance.
Mikel
Hi Mikel
Unfortunately, I don't have the answer for that. I haven't used a custom adapter in a bridge before. Maybe you can look into the source codes of other adapters (that support bridges) as well as the bridge related modules to see if you can find something there. Good luck!
I´ve been looking into it ... and I found why the replace interface is not working:
For the replaceInterface to work properly on the ResponseOnewayBean, the message arriving to the module must have the following class:
The problem, is that the code that the sample adapter has to create new messages, which is based in the class Message does not create this kind of message (APLICATION_RESPONSE), but another kind of message (APLICATION_MESSAGE).
This causes that later on ... the interface is not replaced in ResponseOnewayBean.
Hi Eng,
I am a middleware developer from non SAP background. This series was extremely helpful and I was able to build custom SAP PO Adapter to connect with Apache Pulsar using Azure OAuth 2.0 authentication. Thanks for the documentation.
Thanks and Regards,
Sudarsan V