Technical Articles
Send IoT Data to the IOTA Tangle with SAP HANA XSA and Analytics Cloud
In this blog, we’ll learn how to send the IoT data to the IOTA Tangle with SAP HANA XSA and display it real time in SAP Analytics Cloud.
In order to build this app, you need to have the following components:
- IOTA Tangle (www.iota.org)
- SAP HANA XS Advanced (XSA)
- NodeJS
- Socket.IO
- SAP Analytics Cloud
IOTA
IOTA is an open source distributed ledger called Tangle that is developed for Internet of Things (IoTs) or Machine to Machine (M2M) economy. But its technology is also well suited for payments between humans as well. You can read more information on iota.org.
Scenarios
In this tutorial, we will create simple scenarios to send the IoT temperature data to the IOTA Tangle using Masked Authentication Messaging (MAM) which means that:
- The message is encrypted (Masked)
- The message is confirmed to be coming from the device (Authenticated).
- A continuous message stream is created on the Tangle and will continue until the device stop publishing the data (Messaging).
IOTA MAM makes possible for the the devices to encrypt the data streams and securely store those in Tangle each on separate address and only authorized parties that will be able to read and parse the message streams. Only those with the right channel IDs (The channel ID is also called the root) can get access to the message.
There are different type of privacy modes when we publish the message with MAM to Tangle:
- Public mode: Anyone has access to the message.
- Private mode: Only the publisher has access to the message.
- Restricted mode: Anyone with a side_key has access to the message. The message is encrypted by the side_key. To decrypt, use the same side_key.
In a nutshell, we will create the following parts:
- Sensor Device: NodeJS app mam_publish.js to emulate the sensor temperature data reading and send the data in MAM to Tangle with restricted mode. We will use the side_key to encrypt and decrypt the data.
- NodeJS & HTML App in HANA XSA:
NodeJS app to receive the stream of data from Tangle and send to HTML app via websocket, socket.io that we installed in HANA XSA.
HTML app to embed the SAP Analytics Cloud in an iFrame and receive the stream of data from NodeJS App and send to SAP Analytics Cloud. - SAP Analytics Cloud Analytics Application.
To display the temperature data in real time without need to refresh the browser.
Sensor Device
- Install the required libraries:
npm install moment iota.lib.js
- Create NodeJS app mam_publish.js to emulate the sensor reading and publish the data to Tangle.
/* Author: Robert Lie (mobilefish.com) The mam_publish.js file publishes random generated numbers on the tangle using MAM. This file will work on a computer or Raspberry Pi. The published data can be viewed using the mam_receive.js file or https://www.mobilefish.com/services/cryptocurrency/mam.html (Select option: Data receiver) Usage: 1) You can change the default settings: MODE, SIDEKEY, SECURITYLEVEL or TIMEINTERVAL If you do, make the same changes in mam_receive.js file. 2) Start the app: node mam_publish.js More information: https://www.mobilefish.com/developer/iota/iota_quickguide_raspi_mam.html */ const Mam = require('./lib/mam.client.js'); const IOTA = require('iota.lib.js'); const moment = require('moment'); const iota = new IOTA({ provider: 'https://node.deviceproof.org' }); const MODE = 'restricted'; // public, private or restricted const SIDEKEY = 'mysecret'; // Enter only ASCII characters. Used only in restricted mode const SECURITYLEVEL = 3; // 1, 2 or 3 const TIMEINTERVAL = 30; // seconds // Initialise MAM State let mamState = Mam.init(iota, undefined, SECURITYLEVEL); // Set channel mode if (MODE == 'restricted') { const key = iota.utils.toTrytes(SIDEKEY); mamState = Mam.changeMode(mamState, MODE, key); } else { mamState = Mam.changeMode(mamState, MODE); } // Publish data to the tangle const publish = async function(packet) { // Create MAM Payload const trytes = iota.utils.toTrytes(JSON.stringify(packet)); const message = Mam.create(mamState, trytes); // Save new mamState mamState = message.state; console.log('Root: ', message.root); console.log('Address: ', message.address); // Attach the payload. await Mam.attach(message.payload, message.address); return message.root; } const generateJSON = function() { // Generate some random numbers simulating sensor data const data = Math.floor((Math.random()*89)+10); const dateTime = moment().utc().format('DD/MM/YYYY hh:mm:ss'); const json = {"data": data, "dateTime": dateTime}; return json; } const executeDataPublishing = async function() { const json = generateJSON(); console.log("json=",json); const root = await publish(json); console.log(`dateTime: ${json.dateTime}, data: ${json.data}, root: ${root}`); } // Start it immediately executeDataPublishing(); setInterval(executeDataPublishing, TIMEINTERVAL*1000);
SAP HANA XSA
Setup Project
- Logon to SAP HANA XSA Web IDE and create new project from template then click Next to continue.
- Give a project name ziota and click Next to continue.
- On this screen, just click Next to continue.
- Set Service (srv) and Database (db) to Not included. We will add the NodeJS module later. Click Next to continue.
- On the confirmation page, click Finish to complete the setup.
- You will see the ziota project created in the workspace.
Add NodeJS Module
We will create the NodeJS app in HANA XSA.
- Right click on the ziota project and select New > Node.js Module.
- Give a module name srv and click Next to continue.
- Don’t tick the Enable XSJS support. Click Next to continue.
- Click Finish to complete.
- Now you should see the srv folder.
- Open server.js in srv folder and replace with the following codes:
/*eslint no-console: 0, no-unused-vars: 0, no-undef:0, no-process-exit:0*/ /*eslint-env node, es6 */ "use strict"; const port = process.env.PORT || 3000; const server = require("http").createServer(); const cds = require("@sap/cds"); //Initialize Express App for XSA UAA and HDBEXT Middleware const xsenv = require("@sap/xsenv"); const passport = require("passport"); const xssec = require("@sap/xssec"); const xsHDBConn = require("@sap/hdbext"); const express = require("express"); global.__base = __dirname + "/"; //Iota const Mam = require('./lib/mam.client.js'); const IOTA = require('iota.lib.js'); const iota = new IOTA({ provider: 'https://node.deviceproof.org' }); const MODE = 'restricted'; // public, private or restricted const SIDEKEY = 'mysecret'; // Enter only ASCII characters. Used only in restricted mode let root; let key; let Socket = null; //logging var logging = require("@sap/logging"); var appContext = logging.createAppContext(); //Initialize Express App for XS UAA and HDBEXT Middleware var app = express(); //Compression app.use(require("compression")({ threshold: "1b" })); //Helmet for Security Policy Headers const helmet = require("helmet"); app.use(helmet()); app.use(helmet.contentSecurityPolicy({ directives: { defaultSrc: ["'self'"], styleSrc: ["'self'", "sapui5.hana.ondemand.com"], scriptSrc: ["'self'", "sapui5.hana.ondemand.com"] } })); // Sets "Referrer-Policy: no-referrer". app.use(helmet.referrerPolicy({ policy: "no-referrer" })); passport.use("JWT", new xssec.JWTStrategy(xsenv.getServices({ uaa: { tag: "xsuaa" } }).uaa)); app.use(logging.middleware({ appContext: appContext, logNetwork: true })); app.use(passport.initialize()); app.use( passport.authenticate("JWT", { session: false }) ); // Initialise MAM State let mamState = Mam.init(iota); // Set channel mode if (MODE == 'restricted') { key = iota.utils.toTrytes(SIDEKEY); mamState = Mam.changeMode(mamState, MODE, key); } else { mamState = Mam.changeMode(mamState, MODE); } // Receive data from the tangle const executeDataRetrieval = async function(rootVal, keyVal) { let resp = await Mam.fetch(rootVal, MODE, keyVal, function(data) { let json = JSON.parse(iota.utils.fromTrytes(data)); Socket.emit('client_data', {'dateTime': json.dateTime, 'data': json.data}) console.log(`dateTime: ${json.dateTime}, data: ${json.data}`); }); executeDataRetrieval(resp.nextRoot, keyVal); } app.get("/node", (req, res) => { var root = req.query.root; console.log(root); executeDataRetrieval(root, key); res.send("Use the root:" + root); }); //Start the Server server.on("request", app); // use socket.io var io = require('socket.io').listen(server); // define interactions with client io.sockets.on('connection', function (socket) { Socket = socket; }); //Start the Server server.listen(port, function () { console.info(`HTTP Server: ${server.address().port}`); });
- We include the iota libraries and define the iota provider from Devnet https://node.deviceproof.org.
- We will use the restricted mode with sidekey to publish the data to Tangle.
- We subscribe to channel ID (root) to receive the stream of data from Tangle and send to HTML app via websocket.
- Open package.json, we will install the required libraries for iota: iota.lib.js and socket.io: socket.io & socket.io-client.
Replace with the following codes:{ "name": "serve", "description": "Generated from ../package.json, do not change!", "version": "1.0.0", "dependencies": { "@sap/cds": "^3.10.0", "express": "^4.17.1", "@sap/xssec": "^2.1.17", "@sap/xsenv": "^2.0.0", "hdb": "^0.17.0", "@sap/hdbext": "^6.0.0", "@sap/hana-client": "^2.4.139", "@sap/textbundle": "latest", "@sap/logging": "^5.0.1", "@sap/audit-logging": "^3.0.0", "nodemailer": "^6.2.1", "passport": "~0.4.0", "async": "^3.0.1", "ws": "^7.0.0", "accept-language-parser": "latest", "node-xlsx": "^0.15.0", "node-zip": "~1.1.1", "xmldoc": "~1.1.2", "winston": "^3.2.1", "body-parser": "^1.19.0", "elementtree": "latest", "then-request": "latest", "compression": "~1.7", "helmet": "^3.18.0", "iota.lib.js": "^0.4.7", "socket.io": "^2.2.0", "socket.io-client": "^2.2.0" }, "engines": { "node": "^8.9", "npm": "^6" }, "devDependencies": {}, "scripts": { "postinstall": "cds build/all --project .. --clean", "start": "node server.js" }, "i18n": { "folders": [ "_i18n" ] }, "cds": { "data": { "driver": "hana" } } }
- Create a folder lib in srv and copy mam.client.js from https://github.com/ferrygun/ziota/blob/master/srv/lib/mam.client.js.
Add Web Module
We will create the HTML App in HANA XSA.
- Right click on the ziota project and select New > Basic HTML5 Module.
- Give a module name web. Click Next to continue.
- Click Finish to complete.
- Now you should see the web folder.
- Open index.html and replace the content with the following codes:
<html> <title>IOTA Tangle with SAP HANA XSA and Cloud Analytics</title> <body> <script src="https://ajax.googleapis.com/ajax/libs/jquery/3.4.1/jquery.min.js"></script> <script src="/socket.io/socket.io.js"></script> <script> function bindEvent(element, eventName, eventHandler) { if (element.addEventListener){ element.addEventListener(eventName, eventHandler, false); } else if (element.attachEvent) { element.attachEvent('on' + eventName, eventHandler); } } var socket = io.connect(); var iframeSource = 'REPLACE_WITH_SAP_ANALYTICS_CLOUD_APP_URL'; // Create the iframe var iframe = document.createElement('iframe'); iframe.setAttribute('src', iframeSource); iframe.setAttribute('id', 'the_iframe'); iframe.setAttribute('frameborder', 0); iframe.setAttribute('style', 'overflow: hidden; height: 100%; width: 100%; position: absolute;'); document.body.appendChild(iframe); // Send a message to the child iframe var iframeEl = document.getElementById('the_iframe'); // Send a message to the child iframe var sendMessage = function(msg) { // Make sure you are sending a string, and to stringify JSON iframeEl.contentWindow.postMessage(msg, '*'); }; // Listen to message from child window bindEvent(window, 'message', function (e) { console.log(e.data); }); socket.on('client_data', function(data){ console.log(data.data); var temperature = data.data.toString(); sendMessage(temperature); }); </script> </body> </html>
- We need to get the SAP Analytics Cloud App URL later and update the variable iframeSource.
- Open xs-app.json and replace with the following codes:
{ "welcomeFile": "index.html", "authenticationMethod": "route", "routes": [{ "source": "/node(.*)", "destination": "srv_api", "csrfProtection": true, "authenticationType": "none" },{ "source": "/socket.io(.*)", "destination": "srv_api", "csrfProtection": true, "authenticationType": "none" }] }
Update mta.yaml and xs-security.json
- Open mta.yaml and replace with the following codes:
ID: ziota _schema-version: "2.1" version: 0.0.1 modules: - name: srv type: nodejs path: srv parameters: memory: 512M disk-quota: 256M provides: - name: srv_api properties: url: '${default-url}' requires: - name: zearnpfe-uaa - name: web type: html5 path: web requires: - name: zearnpfe-uaa - name: srv_api group: destinations properties: name: srv_api url: '~{url}' forwardAuthToken: true resources: - name: zearnpfe-uaa type: com.sap.xs.uaa-space parameters: config-path: ./xs-security.json
You need to adjust xsuaa parameter accordingly. - Create xs-security.json in the root folder and insert the following codes:
{ "xsappname": "ziota", "scopes": [{ "name": "$XSAPPNAME.Display", "description": "display" }, { "name": "$XSAPPNAME.Create", "description": "create" }, { "name": "$XSAPPNAME.Edit", "description": "edit" }, { "name": "$XSAPPNAME.Delete", "description": "delete" }, { "name": "$XSAPPNAME.DataGenerator", "description": "data generator" }, { "name": "xs_authorization.read", "description": "Read authorization information from UAA" }, { "name": "xs_authorization.write", "description": "Write authorization information to UAA" }, { "name": "$XSAPPNAME.ODATASERVICEUSER", "description": "Enter" }, { "name": "$XSAPPNAME.ODATASERVICEADMIN", "description": "Enter" }], "attributes": [{ "name": "client", "description": "Session Client", "valueType": "int" }, { "name": "country", "description": "country", "valueType": "s" }], "role-templates": [{ "name": "Viewer", "description": "View all records", "scope-references": [ "$XSAPPNAME.Display" ], "attribute-references": [ "client", "country" ] }, { "name": "Editor", "description": "Edit and Delete records", "scope-references": [ "$XSAPPNAME.Create", "$XSAPPNAME.Edit", "$XSAPPNAME.Delete", "$XSAPPNAME.Display", "$XSAPPNAME.DataGenerator", "$XSAPPNAME.ODATASERVICEUSER", "$XSAPPNAME.ODATASERVICEADMIN" ], "attribute-references": [ "client" ] }] }
SAP Analytics Cloud
- Logon to SAP Analytics Cloud and create Analytic Application.
- In canvas, insert Numeric Point Indicator chart Chart_1.
- Create script variable int1 to store the temperature data.
- Create the calculated measure in the Chart_1.
- On onPostMessageReceived, insert the following code to get the data stream from the child frame.
int1 = ConvertUtils.stringToInteger(message);
- Click Save and click Run Analytics Application.
- You should see the following screen. Copy the URL and update the variable iframeSource in index.html with this URL
Run the App
- Run the Sensor Device app mam_publish.js and get the channel ID (root).
- Run the NodeJS and HTML App in HANA XSA. Make sure they have both run succesfully.
- At this moment, you will get the blank screen and if you open the developer console, you will get this error message.
- To resolve this issue, go back to SAP Analytics Cloud and go to System > Administration.
- Select App Integration, click Add a Trusted Origin then insert the HTML App URL and save it.
- Now reopen the HTML App URL and you should see the screen without error.
- Open a new web browser to subscribe to channel ID (root).
https://<HTML_APP_URL>:<HTML_APP_PORT>/node?root=CHANNEL_ID
- Now you should see the stream of temperature data coming from sensor device to SAP Analytics Cloud in real time.
You can find the source codes on my Git and I hope you learn something new on Iota. Please let me know if you have any comments or questions.
Hi Please make a video recording of the above steps of accomplishments. please. Great stuff you have posted. we are already going hands on in it and getting in trouble lately .Voice over explanation would be awsome ..
Hi Hellvon Dirck ,
I would love to, but I need sometime on this. In the mean while, can you let me know which part that is not working ?
Cheers,
Ferry
?thanks for that great results.
PS: If you have some more time for showcases then you might try as well the integration of IOTA nodes indirectly over SAP Data Hub or over SAP Data Intelligence.
Thank you for this impressive blog post, Ferry.
How did you build the connection between SAC and HANA in your example? In order to create a graphic in an analytical application, a model is necessary, isn't it? Is it a "SAP Data Warehouse Cloud Analytical Dataset" or what did you choose?
Cheers,
Christian