SAP Data Hub 2.6 introduces Data Governance Rules. In comparison to other rule frameworks, like the SAP HANA Rules Framework, they are not defined against specific data sets but metadata Parameters instead. Therefore, they can be applied to any data set that supports their Parameters.
To illustrate this, I would like to reuse an existing SAP Cloud Platform IoT scenario, so lets Restore your NEO Internet of Things Service scenarios with Apache Kafka and the SAP Data Hub but store the data not only in SAP HANA but also in a Google Cloud Storage and Microsoft Azure Data Lake (the code for this SAP Data Hub Graph can be found in the appendix):
With my IoT scenario in the bag, I define a Rule with Parameter Degrees and a Condition that the temperature information that I receive from my data center should remain below 32 degrees Celsius:
Then I create a matching Rulebook and import my Rule into it. This allows me to bind this Rule to my Google Cloud Storage CSV file, where I store my data centre temperature information. SAP Data Hub automatically picks up the field that matches my Parameter:
Next, I run my Rulebook with the result that my data center is to warm according to my Threshold that at least 80% of the time the temperature should be below 32 degrees Celsius. I also get a glimpse into when and how badly the Condition failed:
Accordingly, I put in better air conditioning and the situation improves by 19.52% and according to my Thresholds from red to orange:
This improvement is also confirmed by the SAP Data Hub Fact Sheet of this data set in that lower than 32 degrees Celsius values are now the most common:
The cool thing however is, that this rule is not defined against this specific data set but defined independently and then bound to it. Therefore, I can bind it to another data set that supports the same Parameter, in my case additional temperature information from a secondary sensor in my data center. This is stored in SAP HANA and SAP Data Hub again detects the field that matches my Parameter in the respective Calculation View:
Running the Rulebook again reveals that I am fine now according to my Thresholds in the part of my data center with the new air conditioning but still fail in the other part that I just added and therefore miss my overall target:
This is of course a very simple example, but I hope it shows how you can gain insights into your data with SAP Data Hub Rules.
To illustrate this, I would like to reuse an existing SAP Cloud Platform IoT scenario, so lets Restore your NEO Internet of Things Service scenarios with Apache Kafka and the SAP Data Hub but store the data not only in SAP HANA but also in a Google Cloud Storage and Microsoft Azure Data Lake (the code for this SAP Data Hub Graph can be found in the appendix):
With my IoT scenario in the bag, I define a Rule with Parameter Degrees and a Condition that the temperature information that I receive from my data center should remain below 32 degrees Celsius:
Then I create a matching Rulebook and import my Rule into it. This allows me to bind this Rule to my Google Cloud Storage CSV file, where I store my data centre temperature information. SAP Data Hub automatically picks up the field that matches my Parameter:
Next, I run my Rulebook with the result that my data center is to warm according to my Threshold that at least 80% of the time the temperature should be below 32 degrees Celsius. I also get a glimpse into when and how badly the Condition failed:
Accordingly, I put in better air conditioning and the situation improves by 19.52% and according to my Thresholds from red to orange:
This improvement is also confirmed by the SAP Data Hub Fact Sheet of this data set in that lower than 32 degrees Celsius values are now the most common:
The cool thing however is, that this rule is not defined against this specific data set but defined independently and then bound to it. Therefore, I can bind it to another data set that supports the same Parameter, in my case additional temperature information from a secondary sensor in my data center. This is stored in SAP HANA and SAP Data Hub again detects the field that matches my Parameter in the respective Calculation View:
Running the Rulebook again reveals that I am fine now according to my Thresholds in the part of my data center with the new air conditioning but still fail in the other part that I just added and therefore miss my overall target:
This is of course a very simple example, but I hope it shows how you can gain insights into your data with SAP Data Hub Rules.
Appendix
{
"properties": {
},
"description": "",
"processes": {
"kafkaconsumer1": {
"component": "com.sap.kafka.consumer2",
"metadata": {
"label": "Kafka Consumer",
"x": 17,
"y": 192,
"height": 80,
"width": 120,
"config": {
"kafkaVersion": "0.10.1",
"connectionType": "connection management",
"connection": {
"configurationType": "Configuration Manager",
"connectionID": "Kafka"
},
"topics": "bme680"
}
}
},
"14multiplexer1": {
"component": "com.sap.system.multiplexer.1-4",
"metadata": {
"label": "1:4 Multiplexer",
"x": 201.99999904633,
"y": 191,
"height": 82,
"width": 120,
"extensible": true,
"config": {
}
}
},
"terminal1": {
"component": "com.sap.util.terminal",
"metadata": {
"label": "Terminal",
"x": 517.99999713898,
"y": 372,
"height": 80,
"width": 120,
"ui": "dynpath",
"config": {
}
}
},
"toblobconverter1": {
"component": "com.sap.util.toBlobConverter",
"metadata": {
"label": "ToBlob Converter",
"x": 402.99999809265,
"y": 252,
"height": 50,
"width": 50,
"config": {
}
}
},
"toblobconverter2": {
"component": "com.sap.util.toBlobConverter",
"metadata": {
"label": "ToBlob Converter",
"x": 402.99999809265,
"y": 162,
"height": 50,
"width": 50,
"config": {
}
}
},
"toblobconverter3": {
"component": "com.sap.util.toBlobConverter",
"metadata": {
"label": "ToBlob Converter",
"x": 402.99999809265,
"y": 72,
"height": 50,
"width": 50,
"config": {
}
}
},
"tostringconverter1": {
"component": "com.sap.util.toStringConverter",
"metadata": {
"label": "ToString Converter",
"x": 402.99999809265,
"y": 342,
"height": 50,
"width": 50,
"config": {
}
}
},
"formatconverter1": {
"component": "com.sap.util.formatConverter",
"metadata": {
"label": "Format Converter",
"x": 517.99999713898,
"y": 12,
"height": 80,
"width": 120,
"config": {
"targetFormat": "CSV",
"fields": "millis,temperature"
}
}
},
"formatconverter2": {
"component": "com.sap.util.formatConverter",
"metadata": {
"label": "Format Converter",
"x": 517.99999713898,
"y": 132,
"height": 80,
"width": 120,
"config": {
"targetFormat": "CSV",
"fields": "millis,temperature,pressure,humidity"
}
}
},
"formatconverter3": {
"component": "com.sap.util.formatConverter",
"metadata": {
"label": "Format Converter",
"x": 517.99999713898,
"y": 252,
"height": 80,
"width": 120,
"config": {
"targetFormat": "CSV",
"fields": "millis,temperature,pressure,humidity"
}
}
},
"saphanaclient1": {
"component": "com.sap.hana.client2",
"metadata": {
"label": "SAP HANA Client",
"x": 702.9999961853,
"y": 72,
"height": 80,
"width": 120,
"config": {
"connection": {
"connectionProperties": {
"additionalHosts": [
],
"host": "host",
"password": "",
"port": 9000,
"useTLS": false,
"user": ""
},
"configurationType": "Configuration Manager",
"connectionID": "J0G"
},
"tableName": "\"C4PA\".\"T_IOT_985EAC8DCBD198C302F5\"",
"tableColumns": [
{
"name": "G_CREATED",
"type": "TIMESTAMP"
},
{
"name": "C_TEMPERATURE",
"type": "DOUBLE"
}
]
}
}
},
"writefile1": {
"component": "com.sap.storage.write",
"metadata": {
"label": "Write File",
"x": 702.9999961853,
"y": 192,
"height": 80,
"width": 120,
"config": {
"service": "ADL",
"adlConnection": {
"connectionProperties": {
"accountName": "",
"clientId": "",
"clientKey": "",
"rootPath": "",
"tenantId": ""
},
"configurationType": "Configuration Manager",
"connectionID": "architectIT"
},
"path": "bme680.csv"
}
}
},
"writefile2": {
"component": "com.sap.storage.write",
"metadata": {
"label": "Write File",
"x": 702.9999961853,
"y": 312,
"height": 80,
"width": 120,
"config": {
"service": "GCS",
"gcsBucket": "architectsap",
"path": "bme680.csv",
"gcsConnection": {
"connectionProperties": {
"keyFile": "",
"projectId": "project-id",
"rootPath": ""
},
"configurationType": "Configuration Manager",
"connectionID": "architectSAP"
},
"gcsContentType": "application\/vnd.ms-excel"
}
}
}
},
"groups": [
],
"connections": [
{
"metadata": {
"points": "325.9999990463257,205 353.9999985694885,205 353.9999985694885,97 397.99999809265137,97"
},
"src": {
"port": "out1",
"process": "14multiplexer1"
},
"tgt": {
"port": "ininterface",
"process": "toblobconverter3"
}
},
{
"metadata": {
"points": "325.9999990463257,223 369.9999985694885,223 369.9999985694885,187 397.99999809265137,187"
},
"src": {
"port": "out2",
"process": "14multiplexer1"
},
"tgt": {
"port": "ininterface",
"process": "toblobconverter2"
}
},
{
"metadata": {
"points": "141,223 168.99999952316284,223 168.99999952316284,232 196.99999904632568,232"
},
"src": {
"port": "message",
"process": "kafkaconsumer1"
},
"tgt": {
"port": "in1",
"process": "14multiplexer1"
}
},
{
"metadata": {
"points": "325.9999990463257,241 369.9999985694885,241 369.9999985694885,277 397.99999809265137,277"
},
"src": {
"port": "out3",
"process": "14multiplexer1"
},
"tgt": {
"port": "ininterface",
"process": "toblobconverter1"
}
},
{
"metadata": {
"points": "325.9999990463257,259 353.9999985694885,259 353.9999985694885,376 397.99999809265137,376"
},
"src": {
"port": "out4",
"process": "14multiplexer1"
},
"tgt": {
"port": "inmessage",
"process": "tostringconverter1"
}
},
{
"metadata": {
"points": "456.99999809265137,367 484.9999976158142,367 484.9999976158142,412 512.999997138977,412"
},
"src": {
"port": "outstring",
"process": "tostringconverter1"
},
"tgt": {
"port": "in1",
"process": "terminal1"
}
},
{
"metadata": {
"points": "456.99999809265137,277 484.9999976158142,277 484.9999976158142,292 512.999997138977,292"
},
"src": {
"port": "outbytearray",
"process": "toblobconverter1"
},
"tgt": {
"port": "input",
"process": "formatconverter3"
}
},
{
"metadata": {
"points": "456.99999809265137,187 484.9999976158142,187 484.9999976158142,172 512.999997138977,172"
},
"src": {
"port": "outbytearray",
"process": "toblobconverter2"
},
"tgt": {
"port": "input",
"process": "formatconverter2"
}
},
{
"metadata": {
"points": "456.99999809265137,97 484.9999976158142,97 484.9999976158142,52 512.999997138977,52"
},
"src": {
"port": "outbytearray",
"process": "toblobconverter3"
},
"tgt": {
"port": "input",
"process": "formatconverter1"
}
},
{
"metadata": {
"points": "641.999997138977,52 669.9999966621399,52 669.9999966621399,121 697.9999961853027,121"
},
"src": {
"port": "output",
"process": "formatconverter1"
},
"tgt": {
"port": "data",
"process": "saphanaclient1"
}
},
{
"metadata": {
"points": "641.999997138977,172 669.9999966621399,172 669.9999966621399,232 697.9999961853027,232"
},
"src": {
"port": "output",
"process": "formatconverter2"
},
"tgt": {
"port": "inFile",
"process": "writefile1"
}
},
{
"metadata": {
"points": "641.999997138977,292 669.9999966621399,292 669.9999966621399,352 697.9999961853027,352"
},
"src": {
"port": "output",
"process": "formatconverter3"
},
"tgt": {
"port": "inFile",
"process": "writefile2"
}
}
],
"inports": {
},
"outports": {
}
}- SAP Managed Tags:
