Bring your own language into SAP Data Intelligence – julia

This blog explains how to run code in SAP Data Intelligence with the programing language of your choice.

There are many programming languages that are directly supported ( Python, Go, JavaScript) via the use of the respective out of the box Operators. The current blog explains how to address the case of a language which is not directly supported. The only requirement for achieving this in ensuring you have a docker image available to run your language.

As an example, I am demonstrating how to run julia inside SAP Data Intelligence.

Create your Docker file

Log on to SAP Data Intelligence and access the Modeler app.

Click the Repository tab, right click the dockerfiles folder and choose the Create Docker File option.

Name the Dockerfile and press ok.

You now need to write the script that defines the Dockerfile.

For this, you can take a look at the Dockerfile shown in org.opensuse.

In the end, your file should look something like this (notice the FROM julia.latest row):

FROM julia:latest
RUN groupadd -g 1972 vflow && useradd -g 1972 -u 1972 -m vflow
USER 1972:1972
WORKDIR /home/vflow
ENV HOME=/home/vflow


You then need to add a tag to the Dockerfile so it can be used with an Operator.

Finally, save your Dockerfile and press build button. After the build is done, the respective status should show as completed.

Create your Operator

Now you need to create an Operator that is able to receive and run julia code.

For this, copy the Command Executor Operator as follows.

Go into the Repository tab and access the com.sap.system.commandExecutor folder. Right click the operator.json file and select the Export File option.

Now create a new folder under the Operators folder where you can store your custom Operators.

Inside this new folder, create a folder for your julia Operator and import the commandExecutor operator.json file you had exported.

Double click the json file to edit your new operator.

Go to the Show JSON option and change the description.

Switch to the Show Form option, choose the Tags tab and add julia as a tag. This will ensure that the Operator runs with the Dockerfile you created.

Go to the Configuration tab and set the cmdLine argument to julia:

Optional – add your own icon to the Operator

You can upload a .svg file to use as your Operator’s icon.

Save the operator.

Test you Operator in a Graph

I will show how to create a simple graph that uses the following Operators:

• One instance of the Terminal Operator that will be used to type julia code
• One instance of our julia Operator
• One instance of the Terminal Operator that will show the output of julia’s execution

Create the Graph

Go to the Graphs tab and create a new Graph as shown:

Save the Graph.

Run the Graph

Once the Graph is saved, you can run it.

After the Graph is running, click the Show Status Details link, right-click on the first instance of the Terminal operator and select Open UI.

You are now ready to try out some julia code!

Type the following example instructions in your terminal window:

Now go back to the run details and open the UI for the second Terminal.

You will see the output your Operator produced after running your instructions:

Summary

It is possible to execute code in SAP Data Intelligence for a programming language that is not directly supported by the tool. For that you need to:

• Deploy a Dockerfile that allows you to run that language
• Create a customer Operator (e.g., via an extension of the Command Executor Operator ) that runs with that Dockerfile