Skip to Content
Technical Articles
Author's profile photo Kamlesh Zanje

Script collection reusable artifact in SAP Cloud Integration

Introduction

With the 5.22.x/6.14.x release, SAP Cloud Integration provides a new artifact type known as Script Collection to reuse the scripts in multiple integration flows. In this blog, I will give a detailed overview on this feature.

This feature is described in the SAP Help Portal (see Script Collection Artifact).

What is script collection artifact?

Script Collection is an artifact that is a bundle of scripts. It is a reusable artifact for the scripts that can be developed and deployed independently. Create a script collection within an integration package so that you create scripts in it and then can be referenced from multiple integration flows within that package. The supported resources that can be maintained in the script collection artifact are:

  1. Groovy scripts
  2. JavaScripts
  3. Jar files.

You can modify/enhance scripts without a need to redeploy integration flows that reference these scripts through script collection.

Why script collection artifact?

At present, scripts, jars resources are maintained in an integration flow project. This leads to duplication if the same resource is used by more than one integration flows in your tenant. This causes high memory consumption and maintenance of the scripts is very difficult.

To avoid duplicates, endorse reusability, ease maintenance, and reduce memory footprints, script collection artifact has been introduced.

How to use script collection artifact?

In this section, I will explain through a guided example, how to use this feature and different steps involved in using a script collection.

A typical workflow to use a script collection is as follows

Example

Let us understand the significance of the script collection step by step through the example.

In the script collection artifact, we will create a log payload groovy resource that will be referenced in the integration flow. The groovy resource will read the payload from the incoming message and attach the same to the message log.

Now we will start following the workflow.

Create Script Collection

Open an integration package and choose to edit. In the Artifact tab choose Add “Script Collection”.Image 1: Artifact Type for the script collection.

In the Add Script Collection, you will notice two options

  1. Create script collection from scratch.
  2. Upload any existing script collection artifact project.

Create a script collection artifact from scratch by providing required details and choose Ok.Image 2: Create a script collection artifact

Script collection artifact is created and maintained in the artifact list view.Image 3: Script collection artifact is created

Choose and open the script collection artifact. The artifact is in the read mode. On the left side you will notice a resource view and on the right side, an editor pane for the resource.

To perform the following task in the resource view of script collection artifact, you must edit the artifact.

  1. Create a script resource
  2. Upload a script resource.

Image 4: Script collection artifact viewer in the read mode.

On edit of script collection, options to create and upload the resources will be available. In the resource view, choose Upload.Image5: Script collection artifact viewer in the edit mode

Image6: Upload the resource in Script Collection

In the Source list, you will notice two options.

File System – To upload a script resources from your local file system

Integration Flow – To upload a script resources from integration flows in your tenant.Image7:  Option to upload the script resource from File system.

Image8:  Option to upload/add the script resources from Integration Flow

We will upload the Groovy script resource (Log Payload) from the File system.

For this, create a Log Payload file (groovy script resource) with the following snippet.

——————-

import com.sap.gateway.ip.core.customdev.util.Message;

import java.util.HashMap;

 

def Message processData(Message message) {

              def body = message.getBody(java.lang.String) as String;

              def messageLog = messageLogFactory.getMessageLog(message);

              messageLog.addAttachmentAsString(“Encoded payload”, body, “text/plain”);

              return message;

——————-

Select the File system source and browse the Log Payload file that that you envisioned to upload.Image9: Upload the Groovy script resource from the File system.

On click of the Add button, the resource will be uploaded and shall be maintained in the resource view.Image 10: Groovy script resource is maintained in the resource view

You can click the file from the resource view and open it in the corresponding editor on the right-hand side for viewing/modification/enhancements.Image 11: Resource is opened in its corresponding editor for any modification.

You can save the script collection artifact by clicking Save or Save as version.Image12: Script collection artifact is saved.

Consume Script Collection

Consumption of a script collection in an integration flow is a two-step process.

  1. In the first step, you create a reference to the script collection in your integration flow.
  2. In the second step, you can assign a script resource from the referenced script collection to the script flow step of the integration flow.

Create two integration flows to consume the exercise.

  1. Integration flow consumer 1
  2. Integration flow consumer 2

Image1: Two different integration flow artifacts are created.

This is timer-based integration flow with content modifier and groovy script.

Ensure to provide the message payload in the content modifier. You can define your own payload or use the following sample payload.

———————–

<root>

<orders>

<orderno>OD100

</orderno>

<orderno>OD200

</orderno>

<orders>

</root>

————————

Let us understand and follow the consumption steps in an integration flow.

Step 1:  How to create a reference to the script collection in your integration artifact

Open the integration flow in edit mode and navigate to the resource view of the integration flow. Switch to the Tab “References”.Image2:  References tab in the resource view of integration flow

Click “Script collection” under the menu “Add References”. A dialog will appear to add script collection as a reference to the integration flow.

Image3: Script collection in the Add Reference menu of the resource view

In the References dialog box, script collections that are part of the integration package are listed.

Note – The script collection artifact must be in the same integration package where the integration flow is. You can’t consume a script collection artifact from another Integration package.Image4: Script collection artifacts listed in the References dialog

Select one or more script collection artifacts to which you want to create a referenceImage5: Script collection artifacts is selected as a reference to Integration flow.

Referenced script collection artifact is maintained in the resource view and the script resources of the referenced script collection artifact is available for the assignment in the flow step.Image6: Referenced script collection is maintained in Resource view.

Step 2: How to assign a script resource from the referenced script collection artifact to the script flow step of the integration flow.

Choose the groovy script step from the integration flow modelling area and click the “Assign” speed button.

Select Resource dialog will have two tabs

  • Local Resources
    • List of local script files that are available in your Integration Flow.
  • Referenced Resources
    • List of the referenced script files that are available in your integration flow.

Image8: Referenced script files available in the integration flow for the assignment.

Select the Script collection artifact from the dropdown. Choose the resource that you would like to assign to the flow step.

Image9: Referenced groovy script resource is selected for the assignment to the flow step.

Referenced groovy script resource is assigned to the flow step and in the properties view script file will noticed.Image10: Referenced groovy script resource is assigned to the flow step.

After you consumed the groovy script resource from the referenced script collection artifact, save the integration flow.Image11:  Integration flow consumer 1 is saved

Similarly, you can model another integration flow, add script collection artifact as a reference to the integration flow, assign the groovy script resource file to the flow step and save the integration flow as well.Image12: Integration flow consumer 2 is saved

Deploy Script Collection

Select the script collection artifact and from the action menu, choose Deploy.Image1: Choose action Deploy

On Deployment of script collection artifact, you will experience success message.Image2: Deployment of script collection is successful

Deploy Integration Flow

Deploy both the integration flows one after another.Image1: Integration flow consumer 1 is deployed successfully

Image2: Integration flow consumer 2 is deployed successfully

Monitor Script Collection and Integration Flow

In the Manage integration content section of the operations view, you will notice the runtime status of the deployed script collection artifact. The semantic status should be Started. Similarly, the runtime status of the deployed integration flows should be started.Image1: Semantic state of the script collection and integration flow artifact is started.

In the Monitoring processing view, the integration flow status should be completed, and in the attachments section, you will notice a payload (Encoded payload) that was attached to the message logs through a groovy script.

Image2:  Encoded payload in the Attachments section.

On clicking Encoded payload link, you will notice the payload that was attached to message logs.Image3: Payload in the message log.

Hope you have learned the benefits of the script collection artifact through this simple exercise.

Now lets us understand some functionality of script collection artifact.

You can create Groovy and JavaScript resource from scratch. A predefined template for each resource is createdImage1: Create a new groovy and javascript

You can open the resource in the editor by clicking the file name and it shall open in the corresponding editor. You can enhance the file as per your scenario.Image2:Resource is opened in the editor for modification

You can upload multiple resources in the script collection artifact. Choose upload and browse multiple resources.     Image3: Upload multiple resources

You can upload multiple resources from the integration flow in the script collection artifact. Choose Integration Flow as a source, select an integration package and integration flow of your choice and choose the resources that you wish to add in the script collection artifact.Image4: Add the resources from the Integration flow

You can manage the resources from the resource view. Lifecycle operations available are download and delete.Image5: Lifecyle operation to download and delete the resource.

Benefits

  • Reusability– Without a script collection, you must manually maintain the same copy of the resource in multiple integration flows.  Using script collection, you maintain the scripts in one container and refer them in multiple integration flows through reference model.
  • Reduce Maintenance Efforts– since script resource is referenced from the script collection in multiple integration flows, you edit the script only once if any changes are anticipated.
  • Reduce File Size and Memory Usage– By avoiding copy of the same script resources in the multiple integration flow, we are reducing the memory footprints.
  • Avoid Duplicates– Maintaining duplicate copy of the script resources is eradicated through script collection artifact.

Best practice and recommendation

  1. Avoid producing incompatible changes in the existing function of the script resource that is being referenced in the script step of the integration flow. Incompatible change can be one of the following:
    • Adding a new parameter(s) to an existing function
    • Renaming the type of parameter of an existing function
    • Renaming the function name.
  2. Instead of enhancing existing function, our recommendation would be to create a new function.

Next Steps

  • Usability improvements in the usage of the script collection.
  • Support more lifecycle operations such as rename, copy etc. for the resources in the script collection.
  • Consumption of the script collection in any package of tenant workspace.
  • Dependency management for the script collection.

I hope, you can benefit from the feature presented in this blog post.

In case of questions or feedback, please feel free to comment on this blog.

Assigned tags

      18 Comments
      You must be Logged on to comment or reply to a post.
      Author's profile photo sri v
      sri v

      Hi Kamlesh,

      Nice blog!

      Seeing this new feature, I tried to create a script collection artifact and added few groovy script files from my local system. Upon deploy, I am seeing the below error. Could you please check and let me know what would be went wrong.

      ERROR - FAILED to start bundle from path: /app/data/deploy/bundle-pickup/ScriptCollection.jar org.osgi.framework.BundleException: Unable to resolve ScriptCollection [866](R 866.0): missing requirement [ScriptCollection [866](R 866.0)] osgi.wiring.package;

      I believe if this artifact is accessible across package would be more reusable and your comments please.

      Regards, Sri

      Author's profile photo Kamlesh Zanje
      Kamlesh Zanje
      Blog Post Author

      Hello Sri,

      I'll involve developer to trouble shoot the issue. Thanks for reporting.

      Coming back to artifact accessibility across package, we have plans to support this capability as well. Its in our radar Sri.

      Regards,

      Kamlesh.

      Author's profile photo sri v
      sri v

      Thank you for quick reply Kamlesh and waiting for the fix.

      Regards,

      Sri

      Author's profile photo Kamlesh Zanje
      Kamlesh Zanje
      Blog Post Author

      Hello Sri,

      We have tried deploying the script collection artifact with resources in the factory tenant and it worked.

      Probably we need some more data points from you to troubleshoot the issue that you are facing.

      Either you can drop a mail to me - kamlesh.zanje@sap.com  or report a ticket on LOD-HCI-PI-WT-IFL.

      Regards,

      Kamlesh.

      Author's profile photo Kamlesh Zanje
      Kamlesh Zanje
      Blog Post Author

      Hello Sri and Daniel,

      My team had analysed the issue. Basically, the worker set was not updated to the latest version hence the deployed script collection failed to start. We have informed cloud ops team to update the worker set to the latest version. Issue will be resolved shortly.

      Regards,

      kamlesh.

      Author's profile photo Allen Chew
      Allen Chew

      Great blog and great feature for scripts re-usability although is within the same package for a start. We would be more happy if the scripts could be reuse across packages - also glad that it would be part of the pipeline at later stage.

      Author's profile photo Kamlesh Zanje
      Kamlesh Zanje
      Blog Post Author

      Hello Allen,

      Thanks for reply.

      Reusability across package and Remote OData APIs for script collection is in roadmap.

      We will make the availability of the same transparent in product documentation and blog.

      Regards,

      Kamlesh.

      Author's profile photo Srini Reddy
      Srini Reddy

      Hello Kamlesh Zanje

      It is a nice feature to build reusable scripts.

      I don't see an OData API to maintain script collection in api.sap.com. When can we expect an OData API to automate the deployment of script collections?

      https://api.sap.com/api/IntegrationContent/overview

       

      Thanks,

      Srini

      Author's profile photo Kamlesh Zanje
      Kamlesh Zanje
      Blog Post Author

      Hello Srini,

      Thanks for appreciating reusable script feature. OData remote APIs is not available for script collection.

      We have few increments in our roadmap

      1. Script collection reusability across the packages.
      2. OData remote APIs for performing script collection artifact CRUD and its other operations.

       

      Regards,

      Kamlesh.

      Author's profile photo Eng Swee Yeoh
      Eng Swee Yeoh

      Hi Kamlesh,

       

      I have a custom Java archive (JAR file) that I want to reuse across different integration flows. Instead of adding them as a resource in each of the flows (which causes redundancy), I thought using script collection would be appropriate, since it supports JAR files.

       

      After doing so, in an integration flow I have a Groovy script which uses one of the classes in the JAR file (the script collection has been referenced in the flow). When I execute a message for the integration flow, it fails with the error that it cannot resolve the class.

       

      Looks like scripts from the integration flow cannot resolve classes in JAR files of the script collection.

       

      However, if I move the script from the integration flow to the script collection, and reference the script from the flow, then it works. But this approach is not useful, because that would mean that every script that needs to use classes in the JAR file would need to be put in the script collection.

       

      Is this a bug?

       

      Regards,

      Eng Swee

      Author's profile photo Kamlesh Zanje
      Kamlesh Zanje
      Blog Post Author

      Hello Eng Swee,

      Custom Java archive(jar file) of the script collection artifact is consumable by the groovy scripts maintained in the script collection. You cannot reference Jar file of the script collection  in the local groovy of the iflow. This is the reason groovy scripts from the integration flow could not resolve classes in JAR files of the script collection.

      As we explained in this blog, the idea is to bring reusability of the scripts via script collection and try to avoid the usage of local groovy scripts. In order to do so you need to move your local groovy scripts from the iflow to script collection. It would be one time activity but would bring the benefits as mentioned in the blog.

      This is not a bug.

       

      Do you see moving the scripts in the script collection is a tedious task. I would like to understand from your perspective the challenge in performing this activity.

       

      Regards,

      Kamlesh.

       

      Author's profile photo Eng Swee Yeoh
      Eng Swee Yeoh

      Hi Kamlesh,

       

      Thank you for your response.

       

      The issue is not so much that it is tedious to move scripts into the script collection, but of storing the right "object" in the right place. My view is that the script collection contains objects (JAR, Groovy, JavaScript files) that are reused in more than 1 integration flow.

       

      In the real-life use case that we have, there is a custom JAR file that contains common logic that are used across many integration flows. Imagine the following simplified example:

      IFlow A contains Groovy Script A, which refers to classes in the custom JAR file

      IFlow B contains Groovy Script B, which refers to classes in the custom JAR file

       

      Currently, the custom JAR file has to be uploaded to both IFlow A & B, but with the script collection functionality, it would be nice to move it to a "central" script collection. Now, Groovy Script A is only relevant to IFlow A, and Groovy Script B is only relevant to IFlow B - so there is no reuse for both this scripts, and therefore I don't see it pertinent that they are "relocated" into the central script collection.

      I don't want the script collection to end up containing many instances of single use scripts - this also could potentially cause an issue of determining where each script is referenced.

      Another point is also an impact on runtime and deployment. If Groovy Script B is located in the central script collection, changes to it would involve redeploying the script collection. This could impact messages being processed in-transit on IFlow A which actually has no dependency at all on Groovy Script B.

       

      I hope this helps provide a clearer picture of the use case.

       

      Regards,

      Eng Swee

       

      P.S: I really think this script collection is a great addition to Cloud Integration, and only hope that my feedback would make it even better 😉

       

      Author's profile photo Kamlesh Zanje
      Kamlesh Zanje
      Blog Post Author

      Hello Eng Swee

      Thanks for sharing your view. Will evaluate the possibility.

      Regards,

      Kamlesh.

      Author's profile photo Helmut Skolaut
      Helmut Skolaut

      Hi Kamlesh,

      first of all - thanks for this comprehencive blog - i am got stuck already with the very first step - i cannot create an Artifact "Script Collection":

       

      Is there any mystical i need to activate? I am seeing the option when I am adding a Script-Groovy Step to my IFlow.

      Good regards
      Helmut

      PS: i'm on 6.16.20 (trial)

      Author's profile photo Kamlesh Zanje
      Kamlesh Zanje
      Blog Post Author

      Hello Helmut,

      Thanks for reporting this issue. Can you please provide answer for the following questions. This will help to trouble shoot the issue.

      1. In which trial landscape your tenant exist,
        eu10, us10, ap21.
      2. Share external ID of the tenant
      3. Share tenant url

      Regards,

      Kamlesh.

      Author's profile photo Helmut Skolaut
      Helmut Skolaut

      Hello Kamlesh,

      how can i check this? When go via the "About Menu" i am seeing this version number:

      Thanks for helping me with this issue.

      BR Helmut

       

      Author's profile photo Kamlesh Zanje
      Kamlesh Zanje
      Blog Post Author

      Hello Helmut,

      Please mail me your tenant url.

      Email - kamlesh.zanje@sap.com

      Regards,

      Kamlesh.

      Author's profile photo Helmut Skolaut
      Helmut Skolaut

      Just to update the thread - as also confirmed by SAP - it was a temporary problem in some of the trail instances that has been fixed now.