Disclaimer
Material in this blog post is provided for information and technical features demonstration purposes only. The described technique is only applicable for non-productive usage and under no circumstances shall it be considered for utilization in production-oriented scenarios in CPI tenants, as it might introduce excessive overcomplication to troubleshooting process and security risk associated with potential lack of retrospective traceability and auditability of executed integration flow logic. An area where usage of the technique might be considered, is various proof of concepts that involve development of CPI iFlows.
Intro
When it comes to production-oriented development in CPI, the organization in general and the developer in particular shall be inclined to set up the appropriate development lifecycle infrastructure that aims enhancement of developer efficiency and development quality, as well as automation of recurrent steps that form parts of development, testing and deployment processes. There are nice blog posts out there written by engswee.yeoh and addressing developer productivity and aspects of unit testing of Groovy scripts. An intention is to develop representative and meaningful tests alongside the actual script, so that tests will cover critical (if not all) areas of the script functionality. Consequently, such unit tests can be re-executed locally on the developer machine and a script under test can be validated before being introduced to the iFlow in CPI.
In contrast to production-oriented development, it is commonly not feasible or not reasonable to prepare test specifications for the developed proof of concept iFlow and scripts within it. Development happens at a very high pace and consists of many amendments and changes that are not necessarily unit tested before deployment to the CPI tenant. It is a common case when the iFlow undergoes many iterative changes, each requiring the iFlow to be saved, deployed and started before it can be tested end to end – these steps are repeated and take time. One of areas that often requires several iterations of adjustments, is script steps within the iFlow. In between Groovy and JavaScript scripting languages that are supported by CPI, Groovy is way more popular choice among CPI developers – hence, in this blog post, we will focus on Groovy scripts, and terms "Groovy script" and "script" will be used interchangeably.
In this blog post, I would like to illustrate a technique that, when applied under certain circumstances, can be utilized to reduce a number of redeployments of the iFlow when working with scripts. The technique is not supposed to be used in production-oriented development, but might be useful when working on proof of concepts.
Overview
Fundamental idea of the technique is to replace static script with a generic wrapper that can execute a script logic that is dynamically submitted to it during execution of the iFlow. Since there are no changes to the iFlow at design time (assuming we only change script logic and no other steps of the iFlow), there is no need to save and re-deploy such an iFlow every time we need to introduce a change to a script.
Underlying concept is based on scripting nature of Groovy, where a currently executed script can instruct Groovy scripting engine to load an arbitrary submitted script code from a given location, parse and run it. Location of such dynamically called script code can vary – it can be a local script file, or externally located script, or a code snippet submitted to the script by other means. We can utilize this concept and pass code snippet together with (as a part of) the call to the iFlow – as a result, a message sent to the iFlow will not only contain data that needs to processed by the iFlow, but it will also include information on how data shall be processed.
This concept has much wider application and is effectively used in local environments when testing Groovy scripts. Now it is time to illustrate how it can be brought from the ground to the cloud and how it can be used in context of CPI development. Here I’m going to cover in details the particular case and implementation example that can be further adapted as per specific needs.
Implementation
In the demo, code snippet is going to be submitted in a request message body:
Obviously, this is not the only way to deliver code snippet to the wrapper script. For HTTP requests, code snippet can be passed as a part of the message body (in case the message body shall also include application data), or in a custom header (preliminarily code snippet shall be encoded – for example, using Base64 – as well as overall header value length shall be considered), or retrieved from some other location accessible from the Groovy script step within the iFlow.
In sake of simplicity, the iFlow consists of only one script step, which is a Groovy step implementing a generic wrapper and expecting dynamically submitted script code that is to be passed to it. No other steps are added to the iFlow deliberately, so that we can see pure effect of the script on the entire outcome of the iFlow execution at a later stage.
The entire logic of the wrapper script is based on usage of
GroovyShell that allows invocation of a Groovy script from another Groovy script. There are few other alternatives that can be used to implement dynamic loading of Groovy scripts and that are not covered in this blog post – those are based on usage of GroovyScriptEngine and GroovyClassLoader. A reason why they are not used here is because GroovyScriptEngine has advanced support for dependent scripts, and GroovyClassLoader has advanced support for complex hierarchy of classloaders and additional capabilities to support loading of Groovy classes – when dealing with many Groovy script steps in CPI, both features are not somewhat commonly utilized, but it is rational to be aware of these alternatives in case GroovyShell will not be fit for purpose in certain circumstances.Below is code snippet of the wrapper script:
import com.sap.gateway.ip.core.customdev.util.Message
import org.codehaus.groovy.control.CompilerConfiguration
import org.codehaus.groovy.control.customizers.ImportCustomizer
Message processData(Message message) {
def body = message.getBody(java.io.Reader)
def script = new StringBuilder()
def config = new CompilerConfiguration()
def customizer = new ImportCustomizer()
def binding = new Binding()
binding.message = message
body.eachLine { line ->
line = line.trim()
if (line.startsWith('import ')) {
def importClassName = line.replaceAll(';', '').minus('import ')
customizer.addImport(importClassName)
} else if (!line.empty) {
script << "${line}\n"
}
}
config.addCompilationCustomizers(customizer)
def shell = new GroovyShell(binding, config)
shell.evaluate(script.toString())
message = binding.message
return message
}Let’s have a walkthrough its most critical and essential parts. Considering the following part of the blog post will contain references to specific code lines, let me provide a screenshot of the code snippet above from the code editor:
Code line 7. Given we expect called script’s code snippet to arrive in the message body, we firstly need to retrieve the body from message instance.
Code lines 9-12. We will need several objects that contain code of the called script to be executed, configuration for Groovy Shell and binding between the wrapper script and the called script in particular:
- Builder (script) is used to collect code of the called script. In the demo, code snippet is going to be small and will not occupy much space, so we are going to use
StringBuilder, but if there is possibility of submitted large blocks of code in code snippet, we shall take care of optimization of memory consumed by the wrapper script and avoid potential suboptimal memory consumption – in such cases, we might want to switch to usage ofReaderandWriterobjects to manipulate submitted code snippet. - Compiler configuration (config) is used to tweak and customize behaviour of Groovy compiler – for example, by setting specific supported compiler properties and flags, specifying used imports. In this demo, we will use compiler configuration to add imports that are required by the called script and that are passed as a part of the message body alongside code snippet. The wrapper script contains a minimal number of imports required for its own execution, but any specific imports that are required for correct execution of the called script, are not known to the wrapper script – hence, we can dynamically specify them when sending the request.
- Import customizer (customizer) is used to hold information about imports that are required by the called script.
- Binding (binding) is used to bind variables between the wrapper script and the called script and allow exchange of data between them. For example, in the demo, we will use binding to pass Message object from the wrapper script to the caller script (to allow manipulation with this object within the caller script), and then get it back from the called script and return as an outcome of the wrapper script execution.
Code line 14. Here we set up binding between variable message of the wrapper script with the similarly named variable that will be accessible within the called script. Variables don’t necessarily need to be named similarly and can carry different names, but usage of same names here reduces confusion.
Code lines 16-24. We go through code snippet retrieved from the message body line by line and split them into two categories:
- Lines that contain import statements are retrieved from code snippet and added to import customizer to consolidate script imports.
- Lines that contain code of the called script are retrieved from code snippet and passed to builder to consolidate script code).
Import customizer provides various methods to add imports – for example, it is possible to add static imports, wildcard imports. In the demo, we will not need static imports, and wildcard imports are generally considered as an anti-pattern as in many cases, they can be replaced with specific class imports to improve readability and avoid potential import ambiguities and conflicts, so we will not use them here. Given that Groovy doesn’t enforce usage of semicolons at the end of statements, and different developers have different styles – some prefer usage of semicolons to emphasize end of statement, whereas others prefer more compact code and avoid usage of redundant characters – we’d rather let the wrapper script support both styles when retrieving imports.
As it can be evidenced, we don’t run any validations against obtained code snippet – for example, we don’t validate imported classes against rules for fully qualified names of classes (which can be done using regular expression) and we don’t run syntax checks for parsed code of the called script before executing it. These checks are omitted in the demo and we assume correctness of input submitted in the message body – if this assumption turns to be wrong, the wrapper script is going to fail, resulting in subsequent failure of message processing by the iFlow.
The iFlow also doesn’t check if the message body exists at all – it is assumed that code snippet is provided in the body of the submitted request, otherwise the iFlow shall not be called. Should this be a production-oriented scenario, such condition should have been considered and handled appropriately – for example, by checking message body size before making use of it within the script, or by checking a value of the header Content-Length of the request message against equality to zero (which would be a reflection of a message with the empty body).
Code line 26. After we have collected all imports into import customizer, we can add it to compiler configuration.
Code lines 28-29. Groovy Shell is created and configured by submitting binding and compiler configuration to it, followed by execution of the called script with the help of the shell. In simple cases, it is possible to use Groovy Shell without binding and/or compiler configurations, but in this demo, we need them both for a good reason.
Note that here we use method
GroovyShell.evaluate() – that is a convenient way of running a script once and combining steps of parsing script and executing it. An alternative to this would be usage of combination of methods GroovyShell.parse() (to parse the called script) and GroovyShell.run() (to run the called script) – this is useful and more optimal when the script has to be parsed once and executed several times. As this is not the case here in the demo, we gravitate towards a simpler form.Code lines 31-33. Message variable is read from the one that has been bound to the called script, back in the wrapper script and finally returned from the wrapper script to the executed route.
Alternatively, we could reduce number of code lines in the wrapper script by omitting explicit assignment of the bound variable back to the message variable and returning content of the bound variable in the wrapper script straightaway – here, they are separated and split into two steps in sake of better readability and illustration.
Test
Let’s now put the iFlow under test. Given demo implementation of the wrapper script expects the script code to arrive in the message body, we are going to compose a corresponding request and send it to the endpoint of the iFlow using Postman. For illustrative purposes, code snippet implements logic to compose a JSON object with elements that contain both static content (fixed value) and dynamic content (message ID and current timestamp) and pass it to a body of the response message, and given object content is marshalled to JSON formatted output, we also want to set a specific content type header that indicates the response message body is a JSON document – Content-Type = application/json. Full code snippet used in the request mesage is provided below and highlighted with red colour on the following screenshot.
import groovy.json.JsonOutput;
def body = JsonOutput.toJson(
messageId: "${message.headers['SAP_MessageProcessingLogID']}",
timestamp: "${new Date().format('yyyy-MM-dd HH:mm:ss.SSS')}",
text: "For demonstration purposes only"
);
message.setBody(body);
message.setHeader('Content-Type', 'application/json');
Let’s pay attention to a response that has been received (highlighted with blue colour on the screenshot above) – this is exactly what we would expect a response message to look like, if the code that has been submitted in the request, would have been placed in the Groovy step script in the iFlow! We can evidence that the wrapper script successfully identified import requirement for
JsonOutput and executed the script that was passed to it in the request.CPI Message Monitor provides additional details and evidences that the message triggered execution of the wrapper script and demonstrates message body before and after a script step execution – all together confirming once again that execution of dynamically submitted code snippet was carried and fulfilled as expected:
Alternative implementation by Eng Swee Yeoh
After publication of this blog post, engswee.yeoh made a proposal how the wrapper script can be implemented in an alternative way, making source code of the script significantly more compact and straightforward. The idea behind the version provided by Eng Swee, is to pass the entire Groovy script to the iFlow and consequently to the wrapper script, and not to consider input as a combination of imports section and code snippet that have to be handled in two different ways, as well as making usage of bindings redundant. Let me express gratitude to Eng Swee for exploring, testing and suggesting a more compact alternative, and provide the version that he shared, in here.
Below is code snippet of the wrapper script that has to be embedded into a Groovy script step of the iFlow:
import com.sap.gateway.ip.core.customdev.util.Message
Message processData(Message message) {
def reader = message.getBody(Reader)
Script script = new GroovyShell().parse(reader)
message = script.processData(message)
return message
}An amended sample request that was used in the demo, has to be slightly adjusted and shall result into the following message body to conform to the modified wrapper script:
import com.sap.gateway.ip.core.customdev.util.Message
import groovy.json.JsonOutput
Message processData(Message message) {
def body = JsonOutput.toJson(
messageId: "${message.headers['SAP_MessageProcessingLogID']}",
timestamp: "${new Date().format('yyyy-MM-dd HH:mm:ss.SSS')}",
text: "For demonstration purposes only"
)
message.setBody(body)
message.setHeader('Content-Type', 'application/json')
return message
}Outro
This was indeed a simple example, but I hope it illustrated the described concept – without changing the wrapper script and keeping it generic, we could flexibly compose the entire actually required custom script logic outside of CPI (here, in Postman) and let CPI execute it at runtime without necessity of making a single change in the iFlow implementation after it has been deployed. We can now send various requests (with different scripts in the message body) to this iFlow, and each time the script in the request will get changed, the same iFlow will behave differently.
This kind of flexibility has certain payoff – transparency and traceability. I would like to conclude this blog post by placing emphasis on what has been mentioned earlier: this concept is useful when rapidly developing and tweaking proof of concepts, but it is an inappropriate approach for production-oriented scenarios. In cases when script code is not settled as a part of the iFlow definition during design time, but is submitted and executed at runtime, it is much more complex to ensure consistency of executed flow logic. Not only it exposes a corresponding integration scenario to possible inconsistency and lack of predictability of flow logic, but raises a security concern – if flow logic is impacted from outside of the deployed iFlow, we set strong dependency on security of externally and dynamically submitted script code and enter discussion of analysis and assurance of the code (being it static or dynamic) that is executed at runtime, to be not malicious and to be free of security vulnerabilities.
- SAP Managed Tags:
20 Comments
