In this article I want to show you that (and how) it is possible to write mappings and scripts in SAP Cloud Platform Integration (CPI) in C#.NET (or any other .NET core compatible language).
Before we start, a few words about the background and my motivation. If you are only interested in the solution, you can skip the following paragraph. At the end of the article I will reflect on the sense of the undertaking and outline my vision and wishes.
Two things drove me to this proof of concept: First, the fact that I have been programming in C# for over 10 years now and prefer to use it wherever possible. (Which unfortunately doesn't happen too often in the SAP world - but what is not today can be tomorrow. ?) And secondly, my curiosity to frequently explore the limits of a system anew.
After the decision was made that I would like to try running .NET Core applications on the CPI, I created a SAP Cloud Platform Trial Account, created a CPI instance and explored the general conditions.
By using a small IFlow and a Groovy script I checked the operating system the CPI runs on, the write permissions on the hard disk and the file execution rights. Therefore I wrote the following script:
I did not include a separate check/property in the response object to determine whether processes may be executed, since a process has to be executed to read the operating system version (=call the execute() function). As soon as the script runs without errors, the execution rights shuld also be granted. The result of the script looked like this:
The conditions seemed optimal. Necessary write and execute permissions were available and for Ubuntu Linux there are even precompiled SDKs for Microsoft .NET Core, which only need to be unpacked but not executed for installation. So nothing stood in the way of the implementation.
The goal of the proof-of-concept was to write a Groovy script as simple as possible that does the following things:
As a "showcase" I thought about building an IFlow, which can be reached via HTTP/GET (so it can be called easily in a web browser) and which generates a QRCode from the string given in the url parameter "payload". The QRCode generation should be done in the .NET Core environment.
First I designed an IFlow, which meets the above requirements.
The IFlow gets along with a HTTP sender adapter and a Groovy script. In this Groovy-Script I implemented all the above mentioned requirements (build environment, set up code, compilation, ...). Let's have a look at this script step by step now.
Since the script is a bit longer, let's take a look at it step by step. At the end of this blog post you will find the complete script again.
We start with the imports necessary for the function as well as auxiliary variables that we need in the further course. In the first lines of the processData function we read the url parameters that are given when the IFlow is called and take the value from the url parameter payload and store it in the variable qrCodeContent. We want to encode it's value later as QR Code.
Then we create the variable out, which we will use to intercept and document command line outputs of any process calls we do inside the Groovy Script. (The contents of out will by default not be returned by the IFlow, but might be useful for debugging purposes).
We get the path of the default temporary directory of the CPI node, append a folder name ("dotnet_core_sdk") for our .NET environment and save it into the variable tmpDir. In the following line we create this directory by use of the createDirectories function.
Finally, we create an auxiliary variable named "dotnet" which contains the full path to the .NET Core binary. With this auxiliary variable we can later address the .NET environment. So let's head to the next code block...
In the next step we "install" the .NET Core Framework on the CPI instance. For this we first check, via File.exists() function, whether the dotnet file already exists, because this would mean that the installation has already taken place. We do this, because we don't want to have to perform the installation again, every time we call the interface.
If the installation does not exist yet, we download the appropriate archive (for the previously determined operating system - see section Motivation). In this case the SDK for Ubuntu/Linux x64. An overview of all precompiled and read-to-go SDKs can be found here: https://dotnet.microsoft.com/download/dotnet-core/3.1
(Note: I will not go into detail regarding the used helper functions downloadFile and execCmd here. However, they are listed in the complete script at the end of the article. Feel free to explore them there.)
Next we unpack the downloaded file to the hard disk of the CPI using the "tar zxf" command. By running the command the installation of the SDK is already finished. (Wasn't that easy?)
Then we set up the C#.NET project. Again, this normally should not be necessary for every interface run, so we use the File.exists() function to check if the project files already exist. If the project does not exist, we create a folder for the C# solution using the createDirectories function and initialize a new project folder using the dotnet new command line command. Afterwards we install, by use of the dotnet add ... package command, a nuget package for the generation of QR Codes.
After the project is set up, we create a multiline string variable (by using triple ticks ''') where we store our C# code and then write it into the "Programs.cs" file of our new project.
Some words to the C# code itself:
By completing the former step, the SDK is installed, the C# project is set up, implemented and ready to be compiled now. In the next and last step we only have to compile and execute the C# program and catch its return value to use the QR Code data in the CPI IFlow.
Using the command line command dotnet run ... we execute our C# project. When we call it, we pass the contents of url parameter of the IFlow, which we have previously stored in the variable qrCodeContent, and pass them as a command line parameter to the C# application. The result of the execution of the C# application (=a string with Base64with encoded PNG image data) is stored in the variable qrCodeBase64.
We set the "Content-Type" header of the CPI message to make the interface caller aware, that the data returned is of type image/png. Last but not least we write the Base64 bytes into the message body by use of Groovy's decodeBase64 function. Now our interface is ready for use.
(Note: For debugging purposes you could also write the string variable "out" into the message body instead of the Base64 data. This should contain a detailed log about the individual processing steps of the interface. It's not necessary, but If you may want to under stand the interface better, feel free to comment it in.)
To test the interface, it now only needs to be called in any browser (or testing tool like Postman). The value you enter in the parameter "payload" is then encoded as QR code and displayed in the browser.
(Just scan it with your smartphone to proof it's functionality...)
The first run of the interface, which downloads and installs the SDK, sets up the C# project, compiles and runs it takes ~ 30-35 seconds. All follow-up runs (which only run the C# app) take round about 4-6 seconds. This is not really fast compared to similar CPI interfaces that run on Groovy scripts only. But as said - this is a proof of concept. I think if .NET would be integrated officially in a professional manner, there would be no noticable performance lags.
Here once again the complete script including all help functions:
But what have I achieved now? I have shown that it is basically technically possible to execute C# or .NET core code on the CPI platform (at least the Cloud Foundry-based - on NEO process exec rights were missing in my tenant). However, the path shown is by no means practicable, and it goes without saying that the practices shown above are not likely to be used productively.
I think that despite the lack of practicability, it is obvious that .NET support in the CPI would be technically (and certainly cleanly) implementable. Personally, I would welcome it, if in the future, besides JavaScript and Groovy(-Script), C#.NET would also be available as language of choice in the CPI. I see several advantages here:
I spun the whole idea a bit further and made some little mockups that show how I would imagine an integration of .NET in the CPI.
If I could make a wish, there would be (as shown in the screenshot above) additionally "C#" as script language to choose from.
In the script settings you could then (similar to the resource tab) add arbitrary nuget packages that were preloaded when deploying an IFlow.
In the script editor there would be syntax highlighting and a code template similar to the Groovy template. The SAP message class could be included by using directive (in the background via DLL). Nuget packages, which you have set in the script block settings, would also only have to be included per using directive...
And now it's your turn! I would love to hear your thoughts and ideas on the topic! Would you welcome the integration of .NET in the CPI, do you get along with JavaScript and Groovy Script or would you like to have support for a completely different language? In the spirit of "Bring your own language" (BYOL) - which language/framework/ecosystem would you bring along if you had the free choice? I am looking forward to your comments!
Before we start, a few words about the background and my motivation. If you are only interested in the solution, you can skip the following paragraph. At the end of the article I will reflect on the sense of the undertaking and outline my vision and wishes.
Motivation and preparation
Two things drove me to this proof of concept: First, the fact that I have been programming in C# for over 10 years now and prefer to use it wherever possible. (Which unfortunately doesn't happen too often in the SAP world - but what is not today can be tomorrow. ?) And secondly, my curiosity to frequently explore the limits of a system anew.
After the decision was made that I would like to try running .NET Core applications on the CPI, I created a SAP Cloud Platform Trial Account, created a CPI instance and explored the general conditions.
By using a small IFlow and a Groovy script I checked the operating system the CPI runs on, the write permissions on the hard disk and the file execution rights. Therefore I wrote the following script:
import com.sap.gateway.ip.core.customdev.util.Message
import groovy.json.JsonOutput
Message processData(Message message) {
def response = [
OS: ("lsb_release -a".execute().text =~ /Description:\t*(.*)/).findAll()[0][1],
CanWriteToTemp: new File(System.getProperty("java.io.tmpdir")).canWrite(),
]
message.setHeader("Content-Type", "application/json")
message.setBody(JsonOutput.toJson(response))
return message
}I did not include a separate check/property in the response object to determine whether processes may be executed, since a process has to be executed to read the operating system version (=call the execute() function). As soon as the script runs without errors, the execution rights shuld also be granted. The result of the script looked like this:
{
"OS": "Ubuntu 18.04.4 LTS",
"CanWriteToTemp": true
}The conditions seemed optimal. Necessary write and execute permissions were available and for Ubuntu Linux there are even precompiled SDKs for Microsoft .NET Core, which only need to be unpacked but not executed for installation. So nothing stood in the way of the implementation.
The Implementation - How to run .NET (Core) on CPI
The goal of the proof-of-concept was to write a Groovy script as simple as possible that does the following things:
- "Install" the .NET Core SDK on the CPI node
- Generate a C# console project
- Transfer C# code from CPI-Groovy script into the .NET project
- (Re)load nuget packages
- Compile and execute C# project
As a "showcase" I thought about building an IFlow, which can be reached via HTTP/GET (so it can be called easily in a web browser) and which generates a QRCode from the string given in the url parameter "payload". The QRCode generation should be done in the .NET Core environment.
First I designed an IFlow, which meets the above requirements.
The IFlow gets along with a HTTP sender adapter and a Groovy script. In this Groovy-Script I implemented all the above mentioned requirements (build environment, set up code, compilation, ...). Let's have a look at this script step by step now.
The almighty Groovy Script
Since the script is a bit longer, let's take a look at it step by step. At the end of this blog post you will find the complete script again.
We start with the imports necessary for the function as well as auxiliary variables that we need in the further course. In the first lines of the processData function we read the url parameters that are given when the IFlow is called and take the value from the url parameter payload and store it in the variable qrCodeContent. We want to encode it's value later as QR Code.
import com.sap.gateway.ip.core.customdev.util.Message
import java.nio.file.*
import java.net.URLDecoder
import java.nio.charset.StandardCharsets
Message processData(Message message) {
//Get QR Code input from CPI url parameter named "payload"
def qParams = message.getHeaders().get("CamelHttpQuery")?.split('&')
def qParamsMap = qParams.collectEntries { param ->
param.split('=').collect {
URLDecoder.decode(it, StandardCharsets.UTF_8.name())
}
}
def qrCodeContent = qParamsMap["payload"]
//Setup environment variables
def out = ""
def tmpDir = System.getProperty("java.io.tmpdir") + "/dotnet_core_sdk"
Files.createDirectories(Paths.get(tmpDir));
def dotnet = "${tmpDir}/dotnet"Then we create the variable out, which we will use to intercept and document command line outputs of any process calls we do inside the Groovy Script. (The contents of out will by default not be returned by the IFlow, but might be useful for debugging purposes).
We get the path of the default temporary directory of the CPI node, append a folder name ("dotnet_core_sdk") for our .NET environment and save it into the variable tmpDir. In the following line we create this directory by use of the createDirectories function.
Finally, we create an auxiliary variable named "dotnet" which contains the full path to the .NET Core binary. With this auxiliary variable we can later address the .NET environment. So let's head to the next code block...
In the next step we "install" the .NET Core Framework on the CPI instance. For this we first check, via File.exists() function, whether the dotnet file already exists, because this would mean that the installation has already taken place. We do this, because we don't want to have to perform the installation again, every time we call the interface.
If the installation does not exist yet, we download the appropriate archive (for the previously determined operating system - see section Motivation). In this case the SDK for Ubuntu/Linux x64. An overview of all precompiled and read-to-go SDKs can be found here: https://dotnet.microsoft.com/download/dotnet-core/3.1
(Note: I will not go into detail regarding the used helper functions downloadFile and execCmd here. However, they are listed in the complete script at the end of the article. Feel free to explore them there.)
//Install dotnet SDK if necessary
if (!new File(dotnet).exists()) {
out += "No dotnet found. Starting download.\r\n"
downloadFile("https://download.visualstudio.microsoft.com/download/pr/fdd9ecec-56b4-40f4-b762-d7efe24fc3cd/ffef51844c92afa6714528e10609a30f/dotnet-sdk-3.1.403-linux-x64.tar.gz", "${tmpDir}/dotnet-sdk-3.1.403-linux-x64.tar.gz")
out += "Download complete. Start install.\r\n"
out += "tar zxf ${tmpDir}/dotnet-sdk-3.1.403-linux-x64.tar.gz -C ${tmpDir}".execute().text
out += execCmd("${dotnet} --version")
}Next we unpack the downloaded file to the hard disk of the CPI using the "tar zxf" command. By running the command the installation of the SDK is already finished. (Wasn't that easy?)
Then we set up the C#.NET project. Again, this normally should not be necessary for every interface run, so we use the File.exists() function to check if the project files already exist. If the project does not exist, we create a folder for the C# solution using the createDirectories function and initialize a new project folder using the dotnet new command line command. Afterwards we install, by use of the dotnet add ... package command, a nuget package for the generation of QR Codes.
//Setup script project enviroment
if (!new File("${tmpDir}/mapping").exists()) {
//Create project directory
Files.createDirectories(Paths.get("${tmpDir}/mapping"));
//Initiate a new C#.NET project
out += execCmd("${dotnet} new console --force --output ${tmpDir}/mapping")
//Add Nuget package to project solution
out += execCmd("${dotnet} add ${tmpDir}/mapping package QRCoder")
//Create C#-script as multiline Groovy string
def cSharpCode = '''using System;
using QRCoder;
namespace mapping {
class Program {
static void Main(string[] args) {
var qrCodePayload = args[0];
var qrGenerator = new QRCodeGenerator();
QRCodeData qrCodeData = qrGenerator.CreateQrCode(qrCodePayload, QRCodeGenerator.ECCLevel.Q);
var qrCode = new PngByteQRCode(qrCodeData);
var qrCodeImageAsBytes = qrCode.GetGraphic(10);
Console.WriteLine(Convert.ToBase64String(qrCodeImageAsBytes));
}
}
}'''
//Write C# code to C# project solution
def codeFile = new File("${tmpDir}/mapping/Program.cs")
codeFile.write(cSharpCode)
}After the project is set up, we create a multiline string variable (by using triple ticks ''') where we store our C# code and then write it into the "Programs.cs" file of our new project.
Some words to the C# code itself:
The C# code itself takes the first command line parameter passed to the application. Then it encodes it as a QR code, renders it as PNG graphic and outputs the graphic's data as a Base64 encoded string by writing it out to the command line. (Funfact: The used QR library, called QRCoder, is by the way one of my hobby projects.)
By completing the former step, the SDK is installed, the C# project is set up, implemented and ready to be compiled now. In the next and last step we only have to compile and execute the C# program and catch its return value to use the QR Code data in the CPI IFlow.
//Run C# script
def qrCodeBase64 = execCmd("${dotnet} run --project ${tmpDir}/mapping \"${qrCodeContent}\"").trim()
out += "Data received from .NET environment: ${qrCodeBase64}"
//Comment this line in and comment out the two-after, to show debug information
//message.setBody(out)
//Set image header and QR Code bytes to message
message.setHeader("Content-Type", "image/png")
message.setBody(qrCodeBase64.decodeBase64())
return message
}Using the command line command dotnet run ... we execute our C# project. When we call it, we pass the contents of url parameter of the IFlow, which we have previously stored in the variable qrCodeContent, and pass them as a command line parameter to the C# application. The result of the execution of the C# application (=a string with Base64with encoded PNG image data) is stored in the variable qrCodeBase64.
We set the "Content-Type" header of the CPI message to make the interface caller aware, that the data returned is of type image/png. Last but not least we write the Base64 bytes into the message body by use of Groovy's decodeBase64 function. Now our interface is ready for use.
(Note: For debugging purposes you could also write the string variable "out" into the message body instead of the Base64 data. This should contain a detailed log about the individual processing steps of the interface. It's not necessary, but If you may want to under stand the interface better, feel free to comment it in.)
Demonstration - Let's create a QR code
To test the interface, it now only needs to be called in any browser (or testing tool like Postman). The value you enter in the parameter "payload" is then encoded as QR code and displayed in the browser.
(Just scan it with your smartphone to proof it's functionality...)
Performance of the interface
The first run of the interface, which downloads and installs the SDK, sets up the C# project, compiles and runs it takes ~ 30-35 seconds. All follow-up runs (which only run the C# app) take round about 4-6 seconds. This is not really fast compared to similar CPI interfaces that run on Groovy scripts only. But as said - this is a proof of concept. I think if .NET would be integrated officially in a professional manner, there would be no noticable performance lags.
The complete Groovy script
Here once again the complete script including all help functions:
import com.sap.gateway.ip.core.customdev.util.Message
import java.nio.file.*
import java.net.URLDecoder
import java.nio.charset.StandardCharsets
Message processData(Message message) {
//Get QR Code input from CPI url parameter
def qParams = message.getHeaders().get("CamelHttpQuery") ? .split('&')
def qParamsMap = qParams.collectEntries {
param - >
param.split('=').collect {
URLDecoder.decode(it, StandardCharsets.UTF_8.name())
}
}
def qrCodeContent = qParamsMap["payload"]
//Setup environment variables
def out = ""
def tmpDir = System.getProperty("java.io.tmpdir") + "/dotnet_core_sdk"
Files.createDirectories(Paths.get(tmpDir));
def dotnet = "${tmpDir}/dotnet"
//Install dotnet SDK if necessary
if (!new File(dotnet).exists()) {
out += "No dotnet found. Starting download.\r\n"
downloadFile("https://download.visualstudio.microsoft.com/download/pr/fdd9ecec-56b4-40f4-b762-d7efe24fc3cd/ffef51844c92afa6714528e10609a30f/dotnet-sdk-3.1.403-linux-x64.tar.gz", "${tmpDir}/dotnet-sdk-3.1.403-linux-x64.tar.gz")
out += "Download complete. Start install.\r\n"
out += "tar zxf ${tmpDir}/dotnet-sdk-3.1.403-linux-x64.tar.gz -C ${tmpDir}".execute().text
out += execCmd("${dotnet} --version")
}
//Setup script project enviroment
if (!new File("${tmpDir}/mapping").exists()) {
Files.createDirectories(Paths.get("${tmpDir}/mapping"));
out += execCmd("${dotnet} new console --force --output ${tmpDir}/mapping")
out += execCmd("${dotnet} add ${tmpDir}/mapping package QRCoder")
def cSharpCode = ''
'using System;
using QRCoder;
namespace mapping {
class Program {
static void Main(string[] args) {
var qrCodePayload = args[0];
var qrGenerator = new QRCodeGenerator();
QRCodeData qrCodeData = qrGenerator.CreateQrCode(qrCodePayload, QRCodeGenerator.ECCLevel.Q);
var qrCode = new PngByteQRCode(qrCodeData);
var qrCodeImageAsBytes = qrCode.GetGraphic(10);
Console.WriteLine(Convert.ToBase64String(qrCodeImageAsBytes));
}
}
}
''
'
def codeFile = new File("${tmpDir}/mapping/Program.cs")
codeFile.write(cSharpCode)
}
//Run given script
def qrCodeBase64 = execCmd("${dotnet} run --project ${tmpDir}/mapping \"${qrCodeContent}\"").trim()
out += qrCodeBase64
out += "List directory (${tmpDir}/mapping):\r\n" + "ls -lla ${tmpDir}/mapping".execute().text + "\r\n"
//message.setBody(out)
message.setHeader("Content-Type", "image/png")
message.setBody(qrCodeBase64.decodeBase64())
return message
}
def execCmd(def command){
def sout = new StringBuilder(), serr = new StringBuilder()
def proc = command.execute()
proc.consumeProcessOutput(sout, serr)
proc.waitForOrKill(30000)
return (sout.length() > 0 ? "${sout}\r\n" : "") + (serr.length() > 0 ? "Error: ${serr}\r\n" : "")
}
def downloadFile(def url, def filename) {
new URL(url).openConnection().with { conn ->
new File(filename).withOutputStream { out ->
conn.inputStream.with { inp ->
out << inp
inp.close()
}
}
}
}Vision, Dreams and Mockups
But what have I achieved now? I have shown that it is basically technically possible to execute C# or .NET core code on the CPI platform (at least the Cloud Foundry-based - on NEO process exec rights were missing in my tenant). However, the path shown is by no means practicable, and it goes without saying that the practices shown above are not likely to be used productively.
So was the work for nothing?
I think that despite the lack of practicability, it is obvious that .NET support in the CPI would be technically (and certainly cleanly) implementable. Personally, I would welcome it, if in the future, besides JavaScript and Groovy(-Script), C#.NET would also be available as language of choice in the CPI. I see several advantages here:
- Lower inhibition threshold for .NET developers to deal with CPI
- Achieve greater user acceptance and user base (since companies can draw on the know-how of their .NET developers additionally)
- Access to nuget, a huge ecosystem of helpful libraries that can be easily integrated and used and thus would push the boundries of integration in a positive manner
- (Perhaps a) strengthening of the partnership with Microsoft? (I guess this would be entirely in line with the joint efforts on many fronts such as SAP on Azure, etc.)
I spun the whole idea a bit further and made some little mockups that show how I would imagine an integration of .NET in the CPI.
If I could make a wish, there would be (as shown in the screenshot above) additionally "C#" as script language to choose from.
In the script settings you could then (similar to the resource tab) add arbitrary nuget packages that were preloaded when deploying an IFlow.
In the script editor there would be syntax highlighting and a code template similar to the Groovy template. The SAP message class could be included by using directive (in the background via DLL). Nuget packages, which you have set in the script block settings, would also only have to be included per using directive...
Feedback - It's your turn
And now it's your turn! I would love to hear your thoughts and ideas on the topic! Would you welcome the integration of .NET in the CPI, do you get along with JavaScript and Groovy Script or would you like to have support for a completely different language? In the spirit of "Bring your own language" (BYOL) - which language/framework/ecosystem would you bring along if you had the free choice? I am looking forward to your comments!
- SAP Managed Tags:
2 Comments
