Skip to Content
Author's profile photo Thorsten Schneider

FAQs for SAP Data Hub, developer edition

This page is used to collect frequently asked questions related to SAP Data Hub, developer edition. We will update it regularly.

Where do I find the SAP Data Hub, developer edition?

You find an overview in this blog post. And you can get started by looking at this tutorial.

Can the developer edition be used with Docker Toolbox?

Yes, you can run the SAP Data Hub, developer edition with Docker Toolbox (although we recommend to use Docker for Windows respectively Docker for Mac, since Docker Toolbox is considered a legacy solution).

  • Ensure that the Linux VM used to run Docker is sufficiently sized (at least 2 cores, recommended 4 cores, 8 GB of RAM). You can adjust the sizing via Virtualbox.
  • Publishing the services running inside the Docker container to 127.0.0.1 (localhost) does not work. Instead you can publish and access them to the docker-machine ip.

ERROR: could not handle api call, failure reason : execution of scheduler plan failed

If this error occurs while storing data in SAP Vora (during turorial Store sensor data in SAP Vora), check that you have correctly maintained the connection between the Kafka Consumer operator and the ToString Converter operator.

You need to connect the message port of the Kafka Consumer operator to the ininterface port of the ToString Converter operator.

Where do I find logs inside the Docker container?

You find the logs related to SAP Data Hub (and SAP Vora) inside the Docker container in the directory /var/log/vora.

How can I access SAP Data Hub, developer edition from “other” hosts?

You have set up SAP Data Hub, developer edition on a host different from your local computer or inside a VM (via Docker Toolbox). You cannot access SAP Data Hub.

You are running the Docker container for SAP Data Hub, developer edition with docker run -ti –publish 127.0.0.1:8090:8090 –publish 127.0.0.1:8998:8998 –publish 127.0.0.1:9225:9225 –publish 127.0.0.1:50070:50070 –name datahub –hostname datahub –network dev-net datahub run –agree-to-sap-license –hdfs –livy as per this tutorial.

When using this tutorial to set up SAP Data Hub, developer edition you bind the container ports to the loopback IP address (aka “localhost”) to restrict access. Hence you can only access it on the host it is set up on.

  • For accessing SAP Data Hub, developer edition in conjunction with Docker Toolbox, look at “Can the developer edition be used with Docker Toolbox?”.
  • For accessing SAP Data Hub, developer edition on “another” host (different from your local computer), you can do without binding the container ports to an IP address. For details refer to the README file which you have downloaded with the developer edition. Keep in mind: SAP Data Hub, developer edition is not intended to provide a multi-user environment.

ERROR: No matching manifest for unknown…

You try to use SAP Data Hub, developer edition in conjunction with Docker for Windows and get an error “No matching manifest for unknown…”. Please ensure that you use Linux (and not Windows) containers.

Assigned Tags

      33 Comments
      You must be Logged on to comment or reply to a post.
      Author's profile photo Chris Bethune
      Chris Bethune

      Hello, I was able to successfully setup SAP Data Hub as covered in the tutorial at https://www.sap.com/developer/tutorials/datahub-docker-setup.html and the status loop seems to be running as expected.

      However, I am not able to access any of the URLs from step 5 Smoke Test

      http://localhost:8090 (SAP Data Hub - Data Pipelines)
      http://localhost:9099 (Apache Zeppelin)
      http://localhost:9225 (SAP Vora Tools)
      http://localhost:50070 (Apache Hadoop User Interface)

      I suspect there are some changes I need to make in the Virtual Box network settings.

      I did notice that an IP of 172.18.0.2 is created in the Docker containter while observing the status loop so I tried with this IP instead of localhost but still get a "Site Cannot Be Reached - ERR_CONNECTION_REFUSED" error when trying to navigate to the URLs.

      I have also tried changing the network adapters one at a time to use Bridged Mode.  The default settings created in Virtual Box are:

      Adapter 1:   NAT

      Adapter 2:   Host Only Adapter (Virtual Box Host Only Ethernet Adapter #3)

      Any advice on what I need to modify so that I am able to access the URL's and start working with Data Hub?

      I am using Docker Toolbox since I am running Windows 7.

      Author's profile photo Ayman Salem
      Ayman Salem
      in Host Only Adapter (Virtual Box Host Only ...) check which address you use, and also check that Host Only Adapter is active

      And when you run the docker you must use --publish ex_port:port,  (ex: --publish 50001:8090: Here the port 8090 of the container will be mapped to port 50001).

      Then, to access the "Data Hub"  use  http://192.168.99.100:50001

       

      I hope that will help you.

       

      Author's profile photo Serge Muts
      Serge Muts

      Hi Chris,

      let me know if Ayman's solution works.

      Here is what worked for me based on changes to the documented step 4:

      1. find the docker-machine ip address:

      $ docker-machine ip

      192.168.99.100

      2. Publish using the docker-machine ip address:

      $ docker run -ti --publish 192.168.99.100:8090:8090 --publish 192.168.99.100:9099:9099 --publish 192.168.99.100:9225:9225 --publish 192.168.99.100:50070:50070 --name datahub --hostname datahub --network dev-net datahub run --agree-to-sap-license --hdfs –zeppelin

       

      Now these work:

      http://192.168.99.100:8090/

      http://192.168.99.100:9099/#/

      http://192.168.99.100:9225/web/

      http://192.168.99.100:50070/

      Regards,

      Serge

      Author's profile photo Chris Bethune
      Chris Bethune

       

      I ended up getting it running using your process here and the issued IP address.  I may either set it to permanent or reconfigure for DHCP.   First I need to try and acquire more RAM for my laptop so I can run it without risking a crash.  I wanted to prove out I could get it running before investing in hardware upgrades.   🙂

      Author's profile photo Ingo Peter
      Ingo Peter

      Hello,

      This works for me as well. But how could I use the hostname instead of the IP-address of the docker machine? Using the hostname throws the error

      docker: invalid publish opts format (should be name=value but got '<hostname>:8090:8090')

      Thx, Ingo

      Author's profile photo Ayman Salem
      Ayman Salem

      I just want to add that the "IP address 192.168.99.100" was assigned by the DHCP server it may vary in another installation of VirtualBox.

      • make sure that the DHCP server is active for the "VirtualBox Host-Only Adapter" that you are using for the "default" VM.

      So, you can also run the datahub without IP in the command.

      here's what I'm using:

      $ docker run -ti --publish 50000:9225 --publish 50001:8090 --publish 50002:9099 --publish 50070:50070 --publish 8500:8500 --publish 14000:14000 --name devedition --hostname devedition --net dev-net datahub run --agree-to-sap-license --hdfs --zeppelin

       

      Regards,

      Ayman

      Author's profile photo Chris Bethune
      Chris Bethune

       

      Thanks all for the suggestions.  Note my reply above, I was able to get it running finally.

      Author's profile photo Abdelkader Abdelaziz
      Abdelkader Abdelaziz

      Hi Thorsten,

      Thanks for the tutorial, I was able to run it until the last

      step :https://www.sap.com/developer/tutorials/datahub-pipelines-storeinvora.html

      where i get the following message :

      "Error during init of 'com.sap.vora.hdfsLoader': failed to ping SQL connection with DSN v2://vora:2202/?binary=true: timeout. Please ensure host is up and reachable"

      It seems that port 2202 is not working. any hint how to debug this issue?

      Thanks

      Abdelkader

       

       

       

      Author's profile photo Ayman Salem
      Ayman Salem

      Hallo Abdelkader,

       

      you can check the status of the Vora services under http://<host/ip address>:8500

      if all services are running, then check the firewall on your system (disable it or allow port access)

       

      Ayman

       

       

      Author's profile photo Deepak Varandani
      Deepak Varandani

      Hi Thorsten,

      I have installed SAP Data Hub, developer Edition on my Windows 7 (8GB RAM) laptop using docker toolbox. I have successfully build the docker image of Dev edition, but not able to run docker container based on above datahub image. Please find below error screenshot for your reference.

       

       
      
      
      Regards,
      Deepak Varandani
      Author's profile photo Thorsten Schneider
      Thorsten Schneider
      Blog Post Author

      Hi Deepak,

      hard to say what goes wrong there. How much CPUs/RAM have you given to the Linux VM (see also question "Can the developer edition be used with Docker Toolbox?")?

      Are you using the newest version of SAP Data Hub (i.e. when did you download it)? And does the problem stay the same even when you try a second time / after restarting your computer?

      Cheers

      Thorsten

      Author's profile photo Former Member
      Former Member

      Hi Thorsten,

       

      I am trying to install DH Developer Edition on my desktop using Docker but I am not able to build the image. See the error on the screenshot.

      I hope you can help me, Thanks a lot.

       

      David

       

      Author's profile photo Thorsten Schneider
      Thorsten Schneider
      Blog Post Author

      Hi David,

      two questions

      1) which operating system / Docker version are you using?

      2) do you build the image from inside a corporate (and have you set the necessary proxy in the Dockerfile)?

      Cheers

      Thorsten

      Author's profile photo Former Member
      Former Member

      Thorsten I am using Windows 10 Pro with Docker version 18.03.

      About the second question I am using my personal computer, I am not inside a corporate and I have set the necessary proxy in the Dockerfile using:

      ENV http_proxy=http://myproxy:8080

      ENV https_proxy=http://myproxy:8080

       

      Thanks

       

       

       

       

      Author's profile photo Thorsten Schneider
      Thorsten Schneider
      Blog Post Author

      Hi David,

      but you go via a proxy? Have you tried without setting the proxy?

      If that does not help, can you somehow (mail?) send me the complete log of the build process? Thanks.

      Best regards

      Thorsten

      Author's profile photo Former Member
      Former Member

      Thorsten I think you can't post your mail here but could you send me a mail to df.boada914@gmail.com so I can send you the complete log of the build process

       

      Thank you very much.

      Author's profile photo Gerd Jaehrling
      Gerd Jaehrling

      Hi,

      I downloaded the latest version 1.2 but I'm getting an error using the following command:

      docker build --tag datahub_1.2 --build-arg VORA_USERNAME=vora --build-arg VORA_PASSWORD=******** .

       

      2018-05-04T22:08:49+0000 -------- executing LIVY_download --------
        % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                       Dload  Upload   Total   Spent    Left  Speed
        0     0    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--     0curl: (7) Failed to connect to mirror.netcologne.de port 80: Connection refused
      2018-05-04T22:08:49+0000 ERROR: Couldn't download Apache Livy
      The command '/bin/sh -c /dev-edition-helper.sh LIVY_download' returned a non-zero code: 1
      

      Kindly advise

      cheers

      G.

      Author's profile photo Gerd Jaehrling
      Gerd Jaehrling

      ok, I solved it on my own.

      I changed the file dev-edition-helper.sh in the directory DatahubDevEdition/files and replaced the existing line for the LIVY_URL with some other mirror from https://www.apache.org/dyn/closer.lua/incubator/livy/0.5.0-incubating/livy-0.5.0-incubating-bin.zip

      For example:

      #   LIVY_URL="http://mirror.netcologne.de/apache.org/incubator/livy/${LIVY_VERSION}/livy-${LIVY_VERSION}-bin.zip"
         LIVY_URL="http://ftp-stud.hs-esslingen.de/pub/Mirrors/ftp.apache.org/dist/incubator/livy/${LIVY_VERSION}/livy-${LIVY_VERSION}-bin.zip"
      

      After that the build runs.

      Cheers

      Gerd

      Author's profile photo Thorsten Schneider
      Thorsten Schneider
      Blog Post Author

      Hi Gerd,

      the mirror had a problem on the last weekend. Cheers

      Thorsten

      Author's profile photo Sharad Singh
      Sharad Singh

      Hi Thorsten,

      Thanks for creating such a wonderful blog.

      I am stuck at the last step of the blog "Store sensor data in SAP Vora". After executing pipeline, I am getting error " Error during init of 'com.sap.vora.hdfsLoader': failed to ping SQL connection with DSN v2://vora:2202/?binary=true: timeout. Please ensure host is up and reachable".

      Kindly advise.

      Thanks.

       

      Author's profile photo Sharad Singh
      Sharad Singh

      I solved the issue. Increased no of cpu for VM.

      Author's profile photo Tatiana Signe Nguelok
      Tatiana Signe Nguelok

      Thorsten Schneider    

      I am trying to create a Docker File in the SAP Data Hub Pipeline Modeler. I have created the folder and gave a name for my Docker File.  The Docker File Build is started, but i am recieve an error that the docker image cannot build then “Cannot connect to the Docker daemon at unix:///var/run/docker.sock”.

      Do you have a suggestion to slove this issue?

      Thanks,

      Tatiana

      Author's profile photo Thorsten Schneider
      Thorsten Schneider
      Blog Post Author

      Hi Tatiana,

      at the moment you are not able to do this with the developer edition. Normally (i.e. not in the developer edition) we will spin up containers when running pipelines. These will run on Kubernetes and they will be the runtime environment for the operators.

      With the (current) developer edition we only deliver a single docker image / container. You are not able to define your own Dockerfiles for new operators. Simply spoken the container which runs the developer edition itself is also the runtime environment for all operators.

      If you like to build own operators AND create dedicated Docker files for these, please use our trial edition.

      We know that this behavior is not ideal, but it will stay like this at least for 2.3 / 2.4.

      Sorry!

      Cheers
      Thorsten

      Author's profile photo Tatiana Signe Nguelok
      Tatiana Signe Nguelok

      Thanks for the Information!

      Author's profile photo Carlos Mendez
      Carlos Mendez

      Hi.

      I'm getting this same error.

      Is Data Hub staying like this even with Developer Edition 2.3?

      I'm trying to build my own operator and it is not possible.

       

      Thanks.

      Author's profile photo Thorsten Schneider
      Thorsten Schneider
      Blog Post Author

      Hi,

      yes. This will stay like this at least for 2.3 and 2.4. You can alternatively look at the SAP Data Hub, trial edition which does not have this restriction.

      Best regards
      Thorsten

      Author's profile photo Hobart Liu
      Hobart Liu

      Hi,

      If I want to use pandas/numpy/scikit-learn python libraries, I need to first create a docker file in the datahub. If the datahub developer edition does not support creating new docker file, is there any workaroundthat allows me to use these python libraries?

       

      Thanks!

      Hobart

      Author's profile photo Thorsten Schneider
      Thorsten Schneider
      Blog Post Author

      Hi Hobart,

      you can try to install the libraries directly into the Docker OS, i.e. you exec bash inside the container and do the installation.

      Cheers
      Thorsten

      Author's profile photo Julian Neugebauer
      Julian Neugebauer

      Hey Thorsten,

      after installing everything and getting it to run on Docker for Linux, I integrated a Hadoop Cluster to Read and Write the Data to.

      Now I would love to try the Leonardo MLF Client, but when it comes to configuring it with the rigth API’s from the SAP Business Hub I really am not able to make it work.

      Any hints or documentation you could refere me to?

       

      Many thanks in advance!

      Julian

      Author's profile photo Thorsten Schneider
      Thorsten Schneider
      Blog Post Author

      Hey Julian,

      question: I think we have an example graph for MLF (I am not sure, if it is delivered as part of the developer edition... need to check, but will not find the time today).

      Will try to look into or ask somebody to look into.

      BR

      Thorsten

      Author's profile photo Julian Neugebauer
      Julian Neugebauer

      Hey Thorsten,

      Thank you for looking into it!

      Looking forward to hearing from you

      Julian

       

      Author's profile photo Terry Penna
      Terry Penna

      Running Set up for SAP Data Hub, developer edition 2.4 tutorial and I am getting an error when I run the following docker command per the tutorial:

      PS C:\SAP\dh_dev2_4\DatahubDevEdition> docker build --tag datahub .
      Sending build context to Docker daemon 1.028GB
      Step 1/14 : ARG BASEIMAGE="sapdatahub/dev-edition-base:15.0-01"
      Step 2/14 : FROM ${BASEIMAGE}
      Get https://registry-1.docker.io/v2/sapdatahub/dev-edition-base/manifests/15.0-01: unauthorized: incorrect username or password

      I am logged onto the website https://developers.sap.com/tutorials/datahub-docker-v2-setup.html and in those steps it does not give me a user name and password and the docker command does not ask for one?

       

       

      Author's profile photo Thorsten Schneider
      Thorsten Schneider
      Blog Post Author

      Hi Terry,

      no imediate idea.... I just tested on my laptop and all works as expected.

      You should not need a username / password.

      If the problem is still there, can you give some more details about the environment you are using? Windows 10... with Docker for Windows?

      What happens when you run "docker pull sapdatahub/dev-edition-base:15.0-01"?

      Cheers
      Thorsten