Versioning in SAP PI - Part 2, the detail

[robert_warde4]

I have been tasked with understanding how SAP PI can support different interface versions. Specifically how we will support a SAP system that is being rolled out across the globe.

I recently wrote a short blog entry Versioning in SAP PI about versioning and had a limited response so I decided the investigate the matter in a little more detail. The following is based upon my own analysis, interaction with other developers, SAP Developer Network, SAP Service Support and external developer resources.

Change Types

Firstly let’s understand the various change types:

Changes that are Backwards Compatible

• Addition of new operations
• Addition of new data types or modified data types

Changes that are not backwards compatible.

• Removing an operation
• Renaming an operation
• Changing parameters
• Changing Structures

There are three known strategies for versioning:

Strict - Any changes result in a new version of the interface. It does not support backwards or forwards compatibility.

Flexible - Any incompatible changes result in a new version of the interface. The interface is designed to support backwards compatibility but not forwards compatibility. This means that any backwards-compatible change is considered safe in that it ends up extending or augmenting a service without affecting the interface.

Loose - Any incompatible change results in a new version of the interface. The interface is designed to support backwards compatibility and forwards compatibility. Instead of accommodating known data requirements, special features are used to make parts of the service intrinsically extensible so that they remain able to support a broad range of future, unknown data exchange requirements. This will result in vague interfaces that place the burden of validation on the underlying service logic. Not a recommended approach.

How do we indicate a new version? Most developers and developer resources suggest using the XML namespace to clearly delineate the version. A namespace value is sent with every SOAP message, HTTP Post or XML document. This allows the system to correctly determine what to do with an incoming message.

Options

With regards to PI the following options were considered

a) Create a new SWCV.

According to SAP documentation ‘a software component version is a shipment unit for design objects in the ES Repository’. You use a SWCV to group together objects that are shipped or installed together.’ SAP uses this approach for the majority of their standard content. You can’t copy objects between releases of a SWCV but use the Transfer Design Objects Function (ESR Menu under tools). This tool will transfer objects only if the source objects are more recent than the target objects or the target objects don’t exist. You can preview the changes and see what will be updated (source version is more recent), what can’t be transferred (the target version is more recent) or what does not need to be transferred (the versions are the same).

b) Create a new namespace with a version number.

Create a new namespace using the same name as the original namespace but adding a version number to the end. This will differentiate the versions although any service consumers will have to reference a different endpoint.

c) Ensure that the interface is backwards compatible.

Ensure all changes to the interfaces are backwards compatible. For example any new fields are optional and coding or mapping changes support both releases.

d)  Add a version number to the object names

Add a version number to the object names within the same SWVC and Namespace.

Analysis

a) Create a new SWCV.

Although this is appropriate for deploying different versions of a particular software package it doesn’t allow both versions to be used. Integration Directory objects refer to the interface by Communications Component/Interface Name and Namespace and not the SWCV.  The system will not allow you to add two interfaces from separate SWCVs to the same Communications Component Interface Name and Namespace. Therefore this approach will not work.

b) Create a new namespace with a version number.

This approach allows for changes to be implemented without impact to existing interfaces. The current interface would be copied to the new namespace, modified and in the case of an inbound interface the source system would be given the new target. In the case of outbound interfaces the target system would find that the payload would contain a new namespace. 

c) Ensure that the interface is backwards compatible.

Backwards compatibility isn’t easy to achieve. The functional requirements evolve over time, planned changes are deferred or cancelled and what seems like a simple change that may not have any impact may change significantly. This may result in having to unpick changes to implement subsequent changes. It also means that interfaces are not being developed correctly. This presents a high risk to existing interfaces and is not a recommended approach.

d)  Add a version number to the object names

Although this would work it would beconfusing to maintain and is not a clean and elegant solution. I believe this presents a high risk to existing interfaces. It would be very easy to apply a change to the wrong version. This is not a recommended approach.

Proposal

Based upon my analysis I would like to propose option B, using a new namespace for all future development and supporting flexible versioning. At a high level it would work as follows:

1. Requirements reviewed, assessed
2. Compatibility
   1. Changes not backwards compatible?
      1. Prior version frozen, new release created
      2. Development to use current release
   2. Changes backwards compatable
      1. Modify existing version
3. Development
4. Deployment
5. New Version
   1. Existing consumers migrate when appropriate.
6. Updated Version
   1. Made available to all consumers on this release.

….repeat

It is very important the process is documented and agreed with the relevant teams. We need to ensure that all parties are aware of the process and buy into it. Supporting documentation, including specifications must be updated so it becomes easy to identify the release they relate to.It is more important than ever that supporting documentation is maintained and completed in a timely manner. Developers working in different geographies need to rely on the documentation they have access to.

A very specific namespace naming convention should be used to make it easy to identify the release that the corresponding objects belong to.

Namespaces should be used to differentiate development work conducted between different projects or areas.  The naming conventions that we will therefore adhere to are as follows:

urn:xxxxx.com:PI:<Software Component>:<Region if not in SWC>:<Area>:<Sub-Area>:<Version>

             E.g. urn:xxxxx.com:PI:A_B2B_RSP:B2B:CRMPunchout:YYYY

Benefits

No impact on existing interfaces

Ability to support aggressive project timescales

Customers can migrate when they want

Concerns

a) Freezing code, It is not possible to check out OR lock code via a transport. PI has the ability to restrict access to SWCV’s, Namespaces and individual objects via permissions and this is currently used to ensure objects cannot be created or edited in the Q, R and P systems. However, this may become too onerous to maintain but it would work. We have processes in place to ensure that all development requests are managed centrally, are documented and reviewed prior undertaking any work.

b) Migration to new releases. As the other countries play catch-up we will need to support migration to the later releases and then, when appropriate, decommission the earlier releases. How is the catch-up managed? It is annual, bi annual or quarterly?

c) Bug fixing. We must ensure that any bug fixes requested are documented and reviewed to see if they are appropriate. We must ensure that these are bug fixes and not an attempt to slip changes in unannounced.

d) Change Requests. We will need to plan any changes and determine how they will be incorporated into existing developments.

e) Communication. The changes, specifically the new endpoints should be clearly communicated to the project responsible.

Testing

To validate the proposal I performed a test in development. I took a simple HTTP to RFC synchronous interface as an example. Given a material code and language the interface will return the material code and material description from R/3. I tested this before attempting to create a new version.

1 Within the Enterprise Services Repository.

1. Created a new namespace
2. Copied ALL objects to the new namespace
3. Checked the copied objects to ensure that they were copied correctly.
4. Modify the message mapping to return a different message in the response
5. Activated the objects

2 Within the Integration Directory (configuration)

1. Create a new scenario. Same name, adding release as per the namespace.
2. Update communications component to use new interface
3. Create new configuration to use new objects
4. Activate objects

The request will now have a new namespace but everything else is the same. Each interface was tested and both worked as expected.

The test highlighted the following points

1. Certain objects, such as data types retain the original namespace once copied. A test should be performed with all object types so that post copy actions are documented.
2. Documentation should indicate what objects in the Integration Directory MAY NOT need to be duplicated.  This may include Communications Components, Parties, and Communications Channels etc.  There is no reason to duplicate configuration objects that will not be impacted by the change. Although it makes sense to duplicate all the ESR objects certain ID objects will remain the same.

References: 

http://www.ibm.com/developerworks/webservices/library/ws-version/

http://soa.dzone.com/news/web-service-contract

http://www.oracle.com/technetwork/articles/web-services-versioning-094384.html