Trust your data with lineage information from SAP Data Hub

[architectSAP]

In this blog I build on my scenario to Measure your Raspberry Pi sensor data on the go with the Smart Business Service.

To use Graph lineage with SAP Data Hub 2.5.1 on SUSE CaaSP or AWS, please apply the following patch: In the Modeler open the graph.json located in graphs/com/sap/dh/task/vflowlineageextractor. Click on the operator called [INTERNAL] Vflow Lineage Extractor. In its configuration, change the cmdLine to the following (i.e. remove the --debug flag) such that it reads: npm run -s vflow-operator -- VflowExtractorOperator --extractionId=${extractionIdentifier}. Now the graph will run fine for the user who made that change. For the change to be available to all users, activate the modified graph.json into the tenant space via the System Management's Files browser..

Again, I am a data scientist with the requirement to Visualize your Raspberry Pi sensor data with the Smart Business Service:



From my data stewardship team, I got an OData service based on a SAP HANA Calculation View:

<?xml version="1.0"?>

<edmx:Edmx

	xmlns:edmx="http://schemas.microsoft.com/ado/2007/06/edmx"

	xmlns:sap="http://www.sap.com/Protocols/SAPData" Version="1.0">

	<edmx:DataServices

		xmlns:m="http://schemas.microsoft.com/ado/2007/08/dataservices/metadata" m:DataServiceVersion="2.0">

		<Schema

			xmlns:d="http://schemas.microsoft.com/ado/2007/08/dataservices"

			xmlns:m="http://schemas.microsoft.com/ado/2007/08/dataservices/metadata"

			xmlns="http://schemas.microsoft.com/ado/2008/09/edm" Namespace="c4pa.c4pa">

			<EntityType Name="c4paType" sap:semantics="aggregate">

				<Key>

					<PropertyRef Name="ID"/>

				</Key>

				<Property Name="ID" Type="Edm.String" Nullable="false" MaxLength="2147483647" sap:filterable="false"/>

				<Property Name="Device" Type="Edm.String" MaxLength="255" sap:label="Device" sap:aggregation-role="dimension"/>

				<Property Name="Created" Type="Edm.DateTime" sap:label="Created" sap:aggregation-role="dimension"/>

				<Property Name="Timestamp" Type="Edm.DateTime" sap:label="Timestamp" sap:aggregation-role="dimension"/>

				<Property Name="Temperature" Type="Edm.Double" sap:filterable="false" sap:label="Temperature" sap:aggregation-role="measure"/>

			</EntityType>

			<EntityContainer Name="c4pa" m:IsDefaultEntityContainer="true">

				<EntitySet Name="c4pa" EntityType="c4pa.c4pa.c4paType"/>

			</EntityContainer>

		</Schema>

	</edmx:DataServices>

</edmx:Edmx>

And a respective Connection in SAP Data Hub so that I can check its Fact Sheet:



But more importantly, with Dataset Lineage:



I can also check where this data comes from, in this instance from an SAP Cloud Platform IoT table:



I can also see what Graph stores the data there:



Based on this I can e.g. determine what IoT scenario the sensor data comes from, which in consequence increases my trust in the data.