Personal Insights
SAP Data Intelligence: Custom python operator for beginners
If you are a beginner in SAP Data Intelligence (DI) and python, this blog post is for you. In this post I am going to show how you can create a python custom operator in DI based on my personal experience. Spoiler – it’s fast and easy 😊 For this purpose, we are going to use three simple examples:
-
Hello World
-
Sum of two numbers using input ports
-
Sum of two input values using configuration window
Let’s start by logging into SAP Data Intelligence and opening DI Modeler. Go to the “Operators” tab and click on “+” button to create an operator.
Then enter a name for your operator and choose “Python3 Operator” for the base operator field.
Add an output port: name “output”, data type – string.
Example 1: Hello World
We will start with the simplest example – print “Hello World”. Go to the “Script” tab, delete “file://script.py” string. Add the code below, upload icon for the operator and click on “Save”.
string = "Hello World"
api.send("output", string)
Now we are going to use this operator in the pipeline. Go to the “Graphs” tab on the left side and click on “+” button. Search for your operator in the search box (in my example it is “Python3 example”) and drag and drop it. Add “Terminal” operator and connect them. Each port has a port type. Moreover, the input and output ports must be compatible. In our example the python operator has the output port with the data type string, and “Terminal” has an input port – string, too. Save the pipeline and run it.
As soon as the pipeline has a status “running” right click on the terminal and click on “Open UI”. A new tab should be opened in the web browser:
Example 2: Sum of two numbers using input ports
We are going to reuse the existing operator from the first example. Right click on the python operator and click on “Open operator editor”. Add two input ports “input1” and “input2” with the type “int64”. Through these input ports the python operator will receive two numbers. After this add the code below in the “Script” tab and save the operator:
def on_input(input1, input2):
sums = input1 + input2
api.send("output", str(sums))
api.set_port_callback(["input1","input2"], on_input)
Explanation: Operators in DI react to events via their input ports. Here, we defined the function on_input that runs only when it calls (event – data is received by input ports). In this example the operator waits on input1 and input2. The variable sums has a data type integer, and the output – string. That’s why we are using the function str() to convert integer into string.
In order to use the modified operator in the modeler, you should delete it from the pipeline and drag &drop it one more time. In the pipeline below there are two more python operators that send numbers 2 and 3.
Example 3: Sum of two input values using configuration.
In the “Operator Editor” open the “Configuration” tab. To add parameters, click on the “+” button. We are going to create two input fields “input1” and “input2” for two numbers:
After that go to “Script” to change it. You will get an access to the parameters (input1 and input2) through api.config. Save the operator.
sums = api.config.input1 + api.config.input2
api.send("output", str(sums))
In the configuration window for the python operator enter numbers, save and run the pipeline.
So, now it’s your turn to create your own first python operator in DI 😊 You can get access to the trial version here. Moreover, you will find SAP Data Intelligence tutorials for beginners here.
The provided examples should be noticed like the small illustration of python operator in SAP Data Intelligence. In the next blog post I will demonstrate you the combination of ABAP CDS Reader and python operator.
I hope you find this post helpful. For any questions or feedback just leave a comment below this post. Thanks for reading. Stay tuned 😊
Great blog to get me started, thank you!
I'm happy to hear that 🙂
Very nice blog!!! Thanks!!
Hi Joel, I hope it helps you to start!
Great Blog Yulia .Specially the third part was very helpful.
Thanks.
Hi Shakti, glad to hear that 🙂
Hi Yuliya, great post, thanks a lot and keep on posting 🙂 Thanks
Hi Yuliya, but when you try to add a python code with import hana library:
import pandas as pd
import hana_ml.dataframe as dataframe
from hdbcli import dbapi
the execution code fail:
Group messages: Group: default; Messages: Graph failure: operator.TEST_PARAMB:testparamb1: Error while executing Python Operator's user provided script: No module named 'hana_ml' [line 2]
you know how do you add hana module from python3 operator
Hi Aristobulo,
this error occurs due to the fact, that the base docker image for python does not contain the hana_ml library. You need to create a dockerfile that adds the hana_ml library.
Hope the following hyperlinks might help you:
https://help.sap.com/viewer/1c1341f6911f4da5a35b191b40b426c8/Cloud/en-US/03d1ef5a40d8426d9e92ec0d3e469df5.html
https://blogs.sap.com/2019/12/13/some-notes-on-docker-file-creation-on-sap-data-intelligence/
Best wishes,
Tim
Hi Tim, yes I install the library and works, soo Thanks
Hey Yuliya 😀
Really great blog post and definitely recommended for getting started with Custom Python Operators in DI!
Hi Yuliya,
First very thanks for this blog and this definitely will help people like me who are very novice into SAP Data Science and ML field.
As suggested by you I thought of exploring the examples mentioned here using SAP DI Trial account,
But I am not able to figure out how to land into DI Modeler in Trial account. Below is the page where am landing into, can you please assist here.
I got the solution 🙂
https://developers.sap.com/tutorials/dataintelligence-trial-v2-setup.html
Thanks,
Sijin
Hi,
thanks for that Blog Post.
Is something similar possible with R?
Just found this https://blogs.sap.com/2019/12/05/sap-data-intelligence-create-your-first-ml-scenario-with-r/. But thats with a openAPI Server and I want to provide the Parameters similar to the api.config in Python directly over the api Object.
Thanks and best
Niclas
Hi Yuliya,
Can we do the same thing with one input?
input --> Python operator ----->> terminal
Input will have details of number 1 and number 2.
How can we pass multiple values with single port,
Can you give some examples or a code snippet for sending and receiving?
Or how can we pass values in Arrays. what will be the type of ports.