Leverage your SAP Data Hub Adapter with SAP Agile Data Preparation

[architectSAP]

With my SAP Data Hub connected to SAP Vora and Hadoop, I would like to leverage my respective SAP Data Hub Adapter with SAP Agile Data Preparation.

To start with, I upgrade the environment from my SAP Agile Data Preparation Tutorial to SAP Agile Data Preparation 1.0 SP23 as follows (again please check the respective PAM for the supported combinations):

• HANA Version 2.00.024.01


• HANA_IM_DP Version 2.2.2


• HCO_RULE_FW Version 1.11.5


• HANA_IM_ESS Version 2.2.0


• HANA_IM_ADP Version 1.23.1

Next, I install the standalone JDBC adapter for my Hortonworks Data Platform installation as per the SAP HANA Smart Data Integration and SAP HANA Smart Data Quality Installation and Configuration Guide:

cp hive-jdbc-1.2.1000.2.6.3.0-235-standalone.jar /usr/sap/dataprovagent/lib/

And configure the HiveAdapter from SAP ADP based on the HiveServer2 JDBC URL that I get from Ambari:

jdbc:hive2://my.ambari.server:2181/;serviceDiscoveryMode=zooKeeper;zooKeeperNamespace=hiveserver2

Then I adopt my SAP Data Pipeline to also write my IoT sensor information into a HDFS file:



And configure Data Access Using an External Table:

CREATE EXTERNAL TABLE IF NOT EXISTS raspi ( c0 STRING, c1 TIMESTAMP, c2 DECIMAL ) ROW FORMAT DELIMITED FIELDS TERMINATED BY ',' STORED AS TEXTFILE location '/tmp/';

In addition, I must adjust some HDFS access privileges:

hadoop fs -chmod o+rx /tmp/ambari-qa

hadoop fs -chmod o+rx /tmp/ambari-qa/staging

hadoop fs -chmod o+rx /tmp/hive

hadoop fs -chmod o+rx /tmp/hive/ambari-qa

hadoop fs -chmod o+rx /tmp/hive/anonymous

hadoop fs -chmod o+rx /tmp/hive/vora

With this I can connect SAP ADP to this file:



And, for example, assess its quality as usual, but of course there is much more I could do with ADP (I shared SAP Agile Data Preparation in action on YouTube):



In comparison, profiling the same HDFS file from the SAP Data Hub Cockpit looks like this (the values are slightly different since taken a bit earlier and therefore with a little less data):



This way I very conveniently leverage my SAP Data Hub Adapter for SAP ADP access to my HDFS landscape.