Technical Articles

Ferry Djaja

October 8, 2019 11 minute read

Send IoT Data to the IOTA Tangle with SAP HANA XSA and Analytics Cloud

In this blog, we’ll learn how to send the IoT data to the IOTA Tangle with SAP HANA XSA and display it real time in SAP Analytics Cloud.

In order to build this app, you need to have the following components:

IOTA Tangle (www.iota.org)
SAP HANA XS Advanced (XSA)
NodeJS
Socket.IO
SAP Analytics Cloud

IOTA

IOTA is an open source distributed ledger called Tangle that is developed for Internet of Things (IoTs) or Machine to Machine (M2M) economy. But its technology is also well suited for payments between humans as well. You can read more information on iota.org.

Scenarios

In this tutorial, we will create simple scenarios to send the IoT temperature data to the IOTA Tangle using Masked Authentication Messaging (MAM) which means that:

The message is encrypted (Masked)
The message is confirmed to be coming from the device (Authenticated).
A continuous message stream is created on the Tangle and will continue until the device stop publishing the data (Messaging).

IOTA MAM makes possible for the the devices to encrypt the data streams and securely store those in Tangle each on separate address and only authorized parties that will be able to read and parse the message streams. Only those with the right channel IDs (The channel ID is also called the root) can get access to the message.

There are different type of privacy modes when we publish the message with MAM to Tangle:

Public mode: Anyone has access to the message.
Private mode: Only the publisher has access to the message.
Restricted mode: Anyone with a side_key has access to the message. The message is encrypted by the side_key. To decrypt, use the same side_key.

In a nutshell, we will create the following parts:

Sensor Device: NodeJS app mam_publish.js to emulate the sensor temperature data reading and send the data in MAM to Tangle with restricted mode. We will use the side_key to encrypt and decrypt the data.
NodeJS & HTML App in HANA XSA:
NodeJS app to receive the stream of data from Tangle and send to HTML app via websocket, socket.io that we installed in HANA XSA.
HTML app to embed the SAP Analytics Cloud in an iFrame and receive the stream of data from NodeJS App and send to SAP Analytics Cloud.
SAP Analytics Cloud Analytics Application.
To display the temperature data in real time without need to refresh the browser.

Sensor Device

Install the required libraries:
```
npm install moment iota.lib.js
```

Create NodeJS app mam_publish.js to emulate the sensor reading and publish the data to Tangle.

/*
Author: Robert Lie (mobilefish.com)

The mam_publish.js file publishes random generated numbers on the tangle using MAM.
This file will work on a computer or Raspberry Pi.
The published data can be viewed using the mam_receive.js file or
https://www.mobilefish.com/services/cryptocurrency/mam.html (Select option: Data receiver)

Usage:
1)  You can change the default settings: MODE, SIDEKEY, SECURITYLEVEL or TIMEINTERVAL
    If you do, make the same changes in mam_receive.js file.
2)  Start the app: node mam_publish.js

More information:
https://www.mobilefish.com/developer/iota/iota_quickguide_raspi_mam.html
*/

const Mam = require('./lib/mam.client.js');
const IOTA = require('iota.lib.js');
const moment = require('moment');
const iota = new IOTA({ provider: 'https://node.deviceproof.org' });

const MODE = 'restricted'; // public, private or restricted
const SIDEKEY = 'mysecret'; // Enter only ASCII characters. Used only in restricted mode
const SECURITYLEVEL = 3; // 1, 2 or 3
const TIMEINTERVAL  = 30; // seconds

// Initialise MAM State
let mamState = Mam.init(iota, undefined, SECURITYLEVEL);

// Set channel mode
if (MODE == 'restricted') {
    const key = iota.utils.toTrytes(SIDEKEY);
    mamState = Mam.changeMode(mamState, MODE, key);
} else {
    mamState = Mam.changeMode(mamState, MODE);
}

// Publish data to the tangle
const publish = async function(packet) {
    // Create MAM Payload
    const trytes = iota.utils.toTrytes(JSON.stringify(packet));
    const message = Mam.create(mamState, trytes);

    // Save new mamState
    mamState = message.state;
    console.log('Root: ', message.root);
    console.log('Address: ', message.address);

    // Attach the payload.
    await Mam.attach(message.payload, message.address);

    return message.root;
}

const generateJSON = function() {
    // Generate some random numbers simulating sensor data
    const data = Math.floor((Math.random()*89)+10);
    const dateTime = moment().utc().format('DD/MM/YYYY hh:mm:ss');
    const json = {"data": data, "dateTime": dateTime};
    return json;
}

const executeDataPublishing = async function() {
    const json = generateJSON();
    console.log("json=",json);

    const root = await publish(json);
    console.log(`dateTime: ${json.dateTime}, data: ${json.data}, root: ${root}`);
}

// Start it immediately
executeDataPublishing();

setInterval(executeDataPublishing, TIMEINTERVAL*1000);

SAP HANA XSA

Setup Project

Logon to SAP HANA XSA Web IDE and create new project from template then click Next to continue.
Give a project name ziota and click Next to continue.
On this screen, just click Next to continue.
Set Service (srv) and Database (db) to Not included. We will add the NodeJS module later. Click Next to continue.
On the confirmation page, click Finish to complete the setup.
You will see the ziota project created in the workspace.

Add NodeJS Module

We will create the NodeJS app in HANA XSA.

Right click on the ziota project and select New > Node.js Module.
Give a module name srv and click Next to continue.
Don’t tick the Enable XSJS support. Click Next to continue.
Click Finish to complete.
Now you should see the srv folder.

Open server.js in srv folder and replace with the following codes:

/*eslint no-console: 0, no-unused-vars: 0, no-undef:0, no-process-exit:0*/
/*eslint-env node, es6 */
"use strict";
const port = process.env.PORT || 3000;
const server = require("http").createServer();

const cds = require("@sap/cds");
//Initialize Express App for XSA UAA and HDBEXT Middleware
const xsenv = require("@sap/xsenv");
const passport = require("passport");
const xssec = require("@sap/xssec");
const xsHDBConn = require("@sap/hdbext");
const express = require("express");
global.__base = __dirname + "/";

//Iota 
const Mam = require('./lib/mam.client.js');
const IOTA = require('iota.lib.js');
const iota = new IOTA({
	provider: 'https://node.deviceproof.org'
});

const MODE = 'restricted'; // public, private or restricted
const SIDEKEY = 'mysecret'; // Enter only ASCII characters. Used only in restricted mode

let root;
let key;
let Socket = null;

//logging
var logging = require("@sap/logging");
var appContext = logging.createAppContext();

//Initialize Express App for XS UAA and HDBEXT Middleware
var app = express();

//Compression
app.use(require("compression")({
	threshold: "1b"
}));

//Helmet for Security Policy Headers
const helmet = require("helmet");
app.use(helmet());
app.use(helmet.contentSecurityPolicy({
	directives: {
		defaultSrc: ["'self'"],
		styleSrc: ["'self'", "sapui5.hana.ondemand.com"],
		scriptSrc: ["'self'", "sapui5.hana.ondemand.com"]
	}
}));
// Sets "Referrer-Policy: no-referrer".
app.use(helmet.referrerPolicy({
	policy: "no-referrer"
}));

passport.use("JWT", new xssec.JWTStrategy(xsenv.getServices({
	uaa: {
		tag: "xsuaa"
	}
}).uaa));
app.use(logging.middleware({
	appContext: appContext,
	logNetwork: true
}));
app.use(passport.initialize());
app.use(
	passport.authenticate("JWT", {
		session: false
	})
);

// Initialise MAM State
let mamState = Mam.init(iota);

// Set channel mode
if (MODE == 'restricted') {
    key = iota.utils.toTrytes(SIDEKEY);
    mamState = Mam.changeMode(mamState, MODE, key);
} else {
    mamState = Mam.changeMode(mamState, MODE);
}

// Receive data from the tangle
const executeDataRetrieval = async function(rootVal, keyVal) {
    let resp = await Mam.fetch(rootVal, MODE, keyVal, function(data) {
        let json = JSON.parse(iota.utils.fromTrytes(data));
        
        Socket.emit('client_data', {'dateTime': json.dateTime, 'data': json.data})
        console.log(`dateTime: ${json.dateTime}, data: ${json.data}`);
    });

    executeDataRetrieval(resp.nextRoot, keyVal);
}

app.get("/node", (req, res) => {
	var root = req.query.root;
	console.log(root);
	executeDataRetrieval(root, key);

	res.send("Use the root:" + root);
});

//Start the Server 
server.on("request", app);

// use socket.io
var io = require('socket.io').listen(server);
// define interactions with client
io.sockets.on('connection', function (socket) {
	Socket = socket;
});

//Start the Server 
server.listen(port, function () {
	console.info(`HTTP Server: ${server.address().port}`);
});

We include the iota libraries and define the iota provider from Devnet https://node.deviceproof.org.
We will use the restricted mode with sidekey to publish the data to Tangle.
We subscribe to channel ID (root) to receive the stream of data from Tangle and send to HTML app via websocket.

Open package.json, we will install the required libraries for iota: iota.lib.js and socket.io: socket.io & socket.io-client.
Replace with the following codes:

{
    "name": "serve",
    "description": "Generated from ../package.json, do not change!",
    "version": "1.0.0",
    "dependencies": {
        "@sap/cds": "^3.10.0",
        "express": "^4.17.1",
        "@sap/xssec": "^2.1.17",
		"@sap/xsenv": "^2.0.0",
        "hdb": "^0.17.0",
        "@sap/hdbext": "^6.0.0",
		"@sap/hana-client": "^2.4.139",
		"@sap/textbundle": "latest",
		"@sap/logging": "^5.0.1",
		"@sap/audit-logging": "^3.0.0",
		"nodemailer": "^6.2.1",
		"passport": "~0.4.0",
		"async": "^3.0.1",
		"ws": "^7.0.0",
		"accept-language-parser": "latest",
		"node-xlsx": "^0.15.0",
		"node-zip": "~1.1.1",
		"xmldoc": "~1.1.2",
		"winston": "^3.2.1",
		"body-parser": "^1.19.0",
		"elementtree": "latest",
		"then-request": "latest",
		"compression": "~1.7",
		"helmet": "^3.18.0",
		"iota.lib.js": "^0.4.7",
		"socket.io": "^2.2.0",
		"socket.io-client": "^2.2.0"
    },
    "engines": {
        "node": "^8.9",
        "npm": "^6"
    },
    "devDependencies": {},
    "scripts": {
    	"postinstall": "cds build/all --project .. --clean",
		"start": "node server.js"
    },
    "i18n": {
		"folders": [
			"_i18n"
		]
	},
	"cds": {
		"data": {
			"driver": "hana"
		}
	}
}

Create a folder lib in srv and copy mam.client.js from https://github.com/ferrygun/ziota/blob/master/srv/lib/mam.client.js.

Add Web Module

We will create the HTML App in HANA XSA.

Right click on the ziota project and select New > Basic HTML5 Module.
Give a module name web. Click Next to continue.
Click Finish to complete.
Now you should see the web folder.

Open index.html and replace the content with the following codes:

<html>
  <title>IOTA Tangle with SAP HANA XSA and Cloud Analytics</title>
  <body>
  	<script src="https://ajax.googleapis.com/ajax/libs/jquery/3.4.1/jquery.min.js"></script>
    <script src="/socket.io/socket.io.js"></script>
    <script>
		function bindEvent(element, eventName, eventHandler) {
            if (element.addEventListener){
                element.addEventListener(eventName, eventHandler, false);
            } else if (element.attachEvent) {
                element.attachEvent('on' + eventName, eventHandler);
            }
        }
        
        var socket = io.connect();
        
        var iframeSource = 'REPLACE_WITH_SAP_ANALYTICS_CLOUD_APP_URL';
        // Create the iframe
        var iframe = document.createElement('iframe');
        iframe.setAttribute('src', iframeSource);
        iframe.setAttribute('id', 'the_iframe');
        iframe.setAttribute('frameborder', 0);
        iframe.setAttribute('style', 'overflow: hidden; height: 100%; width: 100%; position: absolute;');

        document.body.appendChild(iframe);
        
        // Send a message to the child iframe        
        var iframeEl = document.getElementById('the_iframe');
        // Send a message to the child iframe
        var sendMessage = function(msg) {
            // Make sure you are sending a string, and to stringify JSON
            iframeEl.contentWindow.postMessage(msg, '*');
        };
        
        
        // Listen to message from child window
        bindEvent(window, 'message', function (e) {
            console.log(e.data);
        });
        
        socket.on('client_data', function(data){
         	console.log(data.data);
         	var temperature = data.data.toString();
         	sendMessage(temperature);
         });
	</script>
  </body>
</html>

We need to get the SAP Analytics Cloud App URL later and update the variable iframeSource.

Open xs-app.json and replace with the following codes:

{
	"welcomeFile": "index.html",
	"authenticationMethod": "route",
	"routes": [{
		"source": "/node(.*)",
		"destination": "srv_api",
		"csrfProtection": true,
		"authenticationType": "none"
	},{
		"source": "/socket.io(.*)",
		"destination": "srv_api",
		"csrfProtection": true,
		"authenticationType": "none"
	}]
}

Update mta.yaml and xs-security.json

Open mta.yaml and replace with the following codes:

ID: ziota
_schema-version: "2.1"
version: 0.0.1
modules:
 - name: srv
   type: nodejs
   path: srv
   parameters:
     memory: 512M
     disk-quota: 256M
   provides:
     - name: srv_api
       properties:
         url: '${default-url}'
   requires:
     - name: zearnpfe-uaa

 - name: web
   type: html5
   path: web
   requires:
     - name: zearnpfe-uaa
     - name: srv_api
       group: destinations
       properties:
         name: srv_api
         url: '~{url}'
         forwardAuthToken: true

resources:
  - name: zearnpfe-uaa
    type: com.sap.xs.uaa-space
    parameters:
      config-path: ./xs-security.json

You need to adjust xsuaa parameter accordingly.

Create xs-security.json in the root folder and insert the following codes:

{
	"xsappname": "ziota",
	"scopes": [{
		"name": "$XSAPPNAME.Display",
		"description": "display"
	}, {
		"name": "$XSAPPNAME.Create",
		"description": "create"
	}, {
		"name": "$XSAPPNAME.Edit",
		"description": "edit"
	}, {
		"name": "$XSAPPNAME.Delete",
		"description": "delete"
	}, {
		"name": "$XSAPPNAME.DataGenerator",
		"description": "data generator"
	}, {
		"name": "xs_authorization.read",
		"description": "Read authorization information from UAA"
	}, {
		"name": "xs_authorization.write",
		"description": "Write authorization information to UAA"
	}, {
		"name": "$XSAPPNAME.ODATASERVICEUSER",
		"description": "Enter"
	}, {
		"name": "$XSAPPNAME.ODATASERVICEADMIN",
		"description": "Enter"
	}],
	"attributes": [{
		"name": "client",
		"description": "Session Client",
		"valueType": "int"
	}, {
		"name": "country",
		"description": "country",
		"valueType": "s"
	}],
	"role-templates": [{
		"name": "Viewer",
		"description": "View all records",
		"scope-references": [
			"$XSAPPNAME.Display"
		],
		"attribute-references": [
			"client", "country"
		]
	}, {
		"name": "Editor",
		"description": "Edit and Delete records",
		"scope-references": [
			"$XSAPPNAME.Create",
			"$XSAPPNAME.Edit",
			"$XSAPPNAME.Delete",
			"$XSAPPNAME.Display",
			"$XSAPPNAME.DataGenerator",
			"$XSAPPNAME.ODATASERVICEUSER",
			"$XSAPPNAME.ODATASERVICEADMIN"
		],
		"attribute-references": [
			"client"
		]
	}]
}

SAP Analytics Cloud

Logon to SAP Analytics Cloud and create Analytic Application.
In canvas, insert Numeric Point Indicator chart Chart_1.
Create script variable int1 to store the temperature data.
Create the calculated measure in the Chart_1.
On onPostMessageReceived, insert the following code to get the data stream from the child frame.
```
int1 = ConvertUtils.stringToInteger(message);
```
Click Save and click Run Analytics Application.
You should see the following screen. Copy the URL and update the variable iframeSource in index.html with this URL

Run the App

Run the Sensor Device app mam_publish.js and get the channel ID (root).
Run the NodeJS and HTML App in HANA XSA. Make sure they have both run succesfully.
At this moment, you will get the blank screen and if you open the developer console, you will get this error message.
To resolve this issue, go back to SAP Analytics Cloud and go to System > Administration.
Select App Integration, click Add a Trusted Origin then insert the HTML App URL and save it.
Now reopen the HTML App URL and you should see the screen without error.

Open a new web browser to subscribe to channel ID (root).

https://<HTML_APP_URL>:<HTML_APP_PORT>/node?root=CHANNEL_ID

Now you should see the stream of temperature data coming from sensor device to SAP Analytics Cloud in real time.

You can find the source codes on my Git and I hope you learn something new on Iota. Please let me know if you have any comments or questions.

References:

4 Comments

You must be Logged on to comment or reply to a post.

Hellvon Dirck

October 13, 2019 at 7:47 pm

Hi Please make a video recording of the above steps of accomplishments. please. Great stuff you have posted. we are already going hands on in it and getting in trouble lately .Voice over explanation would be awsome ..

Ferry Djaja

Blog Post Author

October 14, 2019 at 10:12 am

Hi Hellvon Dirck ,

I would love to, but I need sometime on this. In the mean while, can you let me know which part that is not working ?

Cheers,

Ferry

Raik Kulinna

October 17, 2019 at 12:53 pm

?thanks for that great results.

PS: If you have some more time for showcases then you might try as well the integration of IOTA nodes indirectly over SAP Data Hub or over SAP Data Intelligence.

Christian Bremser

October 6, 2021 at 8:01 am

Thank you for this impressive blog post, Ferry.

How did you build the connection between SAC and HANA in your example? In order to create a graphic in an analytical application, a model is necessary, isn't it? Is it a "SAP Data Warehouse Cloud Analytical Dataset" or what did you choose?