Leverage your SAP Data Hub Adapter with SAP Agile Data Preparation
With my SAP Data Hub connected to SAP Vora and Hadoop, I would like to leverage my respective SAP Data Hub Adapter with SAP Agile Data Preparation.
To start with, I upgrade the environment from my SAP Agile Data Preparation Tutorial to SAP Agile Data Preparation 1.0 SP23 as follows (again please check the respective PAM for the supported combinations):
- HANA Version 2.00.024.01
- HANA_IM_DP Version 2.2.2
- HCO_RULE_FW Version 1.11.5
- HANA_IM_ESS Version 2.2.0
- HANA_IM_ADP Version 1.23.1
Next, I install the standalone JDBC adapter for my Hortonworks Data Platform installation as per the SAP HANA Smart Data Integration and SAP HANA Smart Data Quality Installation and Configuration Guide:
cp hive-jdbc-1.2.1000.2.6.3.0-235-standalone.jar /usr/sap/dataprovagent/lib/
And configure the HiveAdapter from SAP ADP based on the HiveServer2 JDBC URL that I get from Ambari:
jdbc:hive2://my.ambari.server:2181/;serviceDiscoveryMode=zooKeeper;zooKeeperNamespace=hiveserver2
Then I adopt my SAP Data Pipeline to also write my IoT sensor information into a HDFS file:
And configure Data Access Using an External Table:
CREATE EXTERNAL TABLE IF NOT EXISTS raspi ( c0 STRING, c1 TIMESTAMP, c2 DECIMAL ) ROW FORMAT DELIMITED FIELDS TERMINATED BY ',' STORED AS TEXTFILE location '/tmp/';
In addition, I must adjust some HDFS access privileges:
hadoop fs -chmod o+rx /tmp/ambari-qa
hadoop fs -chmod o+rx /tmp/ambari-qa/staging
hadoop fs -chmod o+rx /tmp/hive
hadoop fs -chmod o+rx /tmp/hive/ambari-qa
hadoop fs -chmod o+rx /tmp/hive/anonymous
hadoop fs -chmod o+rx /tmp/hive/vora
With this I can connect SAP ADP to this file:
And, for example, assess its quality as usual, but of course there is much more I could do with ADP (I shared SAP Agile Data Preparation in action on YouTube):
In comparison, profiling the same HDFS file from the SAP Data Hub Cockpit looks like this (the values are slightly different since taken a bit earlier and therefore with a little less data):
This way I very conveniently leverage my SAP Data Hub Adapter for SAP ADP access to my HDFS landscape.