Technical Articles
Bring your own language into SAP Data Intelligence – julia
This blog explains how to run code in SAP Data Intelligence with the programing language of your choice.
There are many programming languages that are directly supported ( Python, Go, JavaScript) via the use of the respective out of the box Operators. The current blog explains how to address the case of a language which is not directly supported. The only requirement for achieving this in ensuring you have a docker image available to run your language.
As an example, I am demonstrating how to run julia inside SAP Data Intelligence.
Create your Docker file
Log on to SAP Data Intelligence and access the Modeler app.
Click the Repository tab, right click the dockerfiles folder and choose the Create Docker File option.
Name the Dockerfile and press ok.
You now need to write the script that defines the Dockerfile.
For this, you can take a look at the Dockerfile shown in org.opensuse.
In the end, your file should look something like this (notice the FROM julia.latest row):
FROM julia:latest
RUN groupadd -g 1972 vflow && useradd -g 1972 -u 1972 -m vflow
USER 1972:1972
WORKDIR /home/vflow
ENV HOME=/home/vflow
You then need to add a tag to the Dockerfile so it can be used with an Operator.
Finally, save your Dockerfile and press build button. After the build is done, the respective status should show as completed.
Create your Operator
Now you need to create an Operator that is able to receive and run julia code.
For this, copy the Command Executor Operator as follows.
Go into the Repository tab and access the com.sap.system.commandExecutor folder. Right click the operator.json file and select the Export File option.
Now create a new folder under the Operators folder where you can store your custom Operators.
Inside this new folder, create a folder for your julia Operator and import the commandExecutor operator.json file you had exported.
Double click the json file to edit your new operator.
Go to the Show JSON option and change the description.
Switch to the Show Form option, choose the Tags tab and add julia as a tag. This will ensure that the Operator runs with the Dockerfile you created.
Go to the Configuration tab and set the cmdLine argument to julia:
Optional – add your own icon to the Operator
You can upload a .svg file to use as your Operator’s icon.
Save the operator.
Test you Operator in a Graph
I will show how to create a simple graph that uses the following Operators:
- One instance of the Terminal Operator that will be used to type julia code
- One instance of our julia Operator
- One instance of the Terminal Operator that will show the output of julia’s execution
Create the Graph
Go to the Graphs tab and create a new Graph as shown:
Save the Graph.
Run the Graph
Once the Graph is saved, you can run it.
After the Graph is running, click the Show Status Details link, right-click on the first instance of the Terminal operator and select Open UI.
You are now ready to try out some julia code!
Type the following example instructions in your terminal window:
Now go back to the run details and open the UI for the second Terminal.
You will see the output your Operator produced after running your instructions:
Summary
It is possible to execute code in SAP Data Intelligence for a programming language that is not directly supported by the tool. For that you need to:
- Deploy a Dockerfile that allows you to run that language
- Create a customer Operator (e.g., via an extension of the Command Executor Operator ) that runs with that Dockerfile
Hi Guilherme,
Thanks for this informative article. Is there are reason why you declare three base images in the dockerfile? Shouldn't be "FROM julia:latest" enough?
Thanks
Yann
Hi Yann,
Good point. There is no reason. I have updated the article.
Thanks!
Guilherme
Hi Guilherme,
Thanks for the informative post! Is there also a way to add the script tab in the operator editor(akin to python3 operator) so that the ability to directly code inside the operator becomes available?
Thanks- Ujwal
Hi Ujwal,
Thanks for your question! What you request would indeed be very helpful, but I am still trying to find out how to achieve it. I will keep you posted.
Guilherme
Hello Guilherme and Ujwal,
I use the Constant Generator to pass code into my custom Haskell operator:
Best regards