Monitor your SAP Data Hub Distributed Runtime with Grafana, Kibana and Prometheus

With my Complex Data Pipelines scheduled successfully on the SAP Data Hub, I want to monitor my SAP Data Hub Distributed Runtime more closely running on my SUSE CaaS Platform.

Luckily, the SAP Data Hub Distributed Runtime already comes with 3 monitoring respective analytics tools, namely Grafana, Kibana and Prometheus. The only slight challenge is to determine their URLs if you missed the respective messages from the installation log:

Grafana

export GRAFANA_NODE_IP=$(kubectl get pods --namespace vora -o wide | awk '!/datasource/ && /crazy-crocodile-grafana/ {print $7}')
export GRAFANA_NODE_PORT=$(kubectl get --namespace vora -o jsonpath="{.spec.ports[0].nodePort}" services crazy-crocodile-grafana)
export GRAFANA_FQDN=$(nslookup $GRAFANA_NODE_IP | awk '/name =/ {print $4}' | sed 's/.$//g')
printf '\nThe Grafana URL and port is: http://%s:%d\n\n' $GRAFANA_FQDN $GRAFANA_NODE_PORT

Grafana shows me the status of my SAP Data Hub Distributed Runtime SUSE CaaSP worker node. The default user/password is admin/admin.

Kibana

export KIBANA_NODE_IP=$(kubectl get pods --namespace vora -o wide | awk '/kibana/ {print $7}')
export KIBANA_NODE_PORT=$(kubectl get --namespace vora -o jsonpath="{.spec.ports[0].nodePort}" services kibana-logging)
export KIBANA_FQDN=$(nslookup $KIBANA_NODE_IP | awk '/name =/ {print $4}' | sed 's/.$//g')
printf '\nThe Kibana URL and port is: http://%s:%d\n\n' $KIBANA_FQDN $KIBANA_NODE_PORT

Kibana shows me the activity of my individual SAP Data Hub Distributed Runtime pods with dlog being the busies and operator the laziest one.

Prometheus

export NODE_PORT=$(kubectl get --namespace vora -o jsonpath="{.spec.ports[0].nodePort}" services crazy-crocodile-prometheus-server)
export NODE_IP=$(kubectl get nodes --namespace vora -o jsonpath="{.items[0].status.addresses[0].address}")
echo http://$NODE_IP:$NODE_PORT

Prometheus finally is a very powerful tool for defining matrices and alerts concerning my SAP Data Hub Distributed Runtime but in this simple example just monitoring itself.