NodeMCU and SAP HCP IoT

My previous blog focused on the MQTT protocol and delivered an IoT prototype that enabled NodeMCU to communicate wireless with a local MQTT message broker using the pub/sub pattern. This blog focuses on integrating with the SAP Hana Cloud Platform (“HCP”). HCP is an open platform-as-a-service which provides in-memory database and application services. As a 20 year SAP veteran I believe this platform enables  the traditional On-Premise ERP customer to embrace the benefits from the Lean IT thinking while maintaining the levels of governance and compliance of an enterprise grade ERP system.

Ok, ok, stepping down from my SAP soapbox, lets focus on the task at hand. My previous blogs introduced the world of IoT one prototype at the time:

1. An introduction to ESP and NodeMCU an IoT platform and flashing firmware with modules required for prototyping.
2. Blink! the IoT equivalent of Hello World! using the Lua programming language.
3. Connected to an IoT device, a DHT22 sensor capturing temperature and humidity information.
4. “Cut the Cord” and leverage the MQTT protocol to wireless publish device specific information to a central message broker.

How nice would it be to connect an ESP8266 micro controller to SAP HCP? Very! Building upon my previous prototypes this blog connects the DHT22 temperature and humidity sensor to the HCP in an autonomous mode (without a broker) using the HTTP protocol. For the Lua script to work, additional NodeMCU modules are utilized: rtcmem and http – ensure these are included in your NodeMCU firmware. Early adopters on the SAP HCP IoT bandwagon like my friend and former colleague Jan Penninkhof did not have this luxury!

SAP Hana Cloud Platform

The SAP HCP contains a set of services to establish integration of devices on the edge. The services provide interfaces to define messages types, register devices and standardized methods to store data in HCP. These services are enabled by two main components: Remote Device Management Services (“RDMS”) and IoT Message Management Services (“MMS”). The HCP IoT services cockpit provides access to these functions as visualized below*.

The SAP iot-starterkit github repository clearly describes the steps to configure the HCP services. They key steps to enable NodeMCU communication using the HTTP protocol are the following:

1. Create the IoT outbound message type with two entries: `temp` and `humi` both of type string. Note down the ‘Device Type ID’ for usage in the Lua script.
2. Configure ESP8266 as the device type and assign the newly created message type.
3. Create NodeMCU device and assign to ESP8366 device type. Upon completion take note of the ‘OAuth Access Token’ and the corresponding ‘Device ID’. Together with the device type ID this information is needed for the Lua script.

Upon completion of this device management configuration the NodeMCU/ESP8266 is ready to communicate with the SAP HCP platform.

Code

The advantage of using the NodeMCU platform is that several pre-build modules and libraries can be utilized to construct an IOT solution which minimizes energy usage while communicating with the HCP wireless.

config.lua
The config.lua file contains the variables and constants required for the successful execution of this prototype.

--[[
config.lua | Tiest van Gool
Global variables definitions for usage across various lua scripts
--]]

--wifi module
wifi_ssid = "%YOURNETWORKID%"  
wifi_password = "%YOURPASSWORD%"

--dht module
dht_pin = 2  -- Pin connected to DHT22 sensor
dht_temp_calc = 0  -- Calculated temperature
dht_humi_calc = 0  -- Calculated humidity
http_temp = 0      -- Temperature for publication
http_humi = 0      -- Humidity for publication

--sensor reading interval
dsleep_time = 60000 --sleep time in us

-- Status Message
print("Global variables loaded")

http.lua
The Lua script below – an extension of the mqtt script – leverages the rtcmem and http modules to provide additional functions. Lets have a closer look at the individual functions in the code:

• func_wifi_conn() – Establishes the connection to the wireless network using the `wifi` module. The network name and password are defined in config.lua script.
• func_read_dht() – Utilizes NodeMCU’s `dht` module to capture the temperature and humidity information from the dht22 sensor. The variables `http_temp` and `http_humi` store the measured values.
• func_http_post() – This function is core to connecting an IOT device wireless to a server and leverages the `http` module utilizing the POST request method. The http message consists of a number of components:
  • url – Contains the https url to the HCP iotservices, replace %SDNUSERID% with your account information and %DEVICEID% with the device id generated during the device configuration.
  • headers – OAuthorization and the JSON content type of the message are defined. The bearer should reference the %OAUTHACCESSTOKEN% which was received during the configuration of the device in HCP.
  • body – The actual message content to be sent to the HCP. The message stores the measured values `http_temp` and `http_humi` in the corresponding fields `temp` and `humi` of the message type configured in the HCP. Replace the %DEVICETYPEID% with the value as generated in the HCP.
  • callback – Evaluates the result of the http post request. In both cases an information message is thrown and the deep sleep mode is entered.
• func_rtcmem_read() – Reads the previously captured temperature and humidity values from user memory. User memory is not impacted by deep sleep thus making the measurements available upon wake up.
• func_rtcmem_write() – The NodeMCU’s `trcmem` module is invoked to store the sensor data captured in `http_temp` and `http_humi` in user memory.

The remainder two functions are triggered by event loops and implement the logic that establishes connectivity with the SAP HCP, publishes the captured sensor data and initiates the device’s deep sleep cycle.

• func_read_data() – Event loop 1 measures current temperature and humidity and retrieves the previous values stored in the user memory. Temperature values are compared and if there is no difference the device enters deep sleep mode to conserve energy. If a difference is observed connectivity with the wireless network is established and an event timer is started which initiates the posting of the http message to the HCP.
• func_post_loop() – The second event loop is responsible for posting the http message once wifi connectivity has been established. If no wifi connectivity is established the event timer restarts the posting process. This loop is required to account for the delay in connecting to the wifi network.

The http.lua script below contains the code of all functions discussed and is ready for loading onto NodeMCU.

--[[
http.lua | Tiest van Gool
Script establishes dht module and temperature and humidity is retrieved.
When a discrepancy between previous and new temperature reading is identified, wifi connectivity is established, information sent to SAP HCP and deep sleep initiated.
If no discrepancy is found, deep sleep is immediately initiated in order to preserve energy.
--]]

-- Load global user-defined variables
dofile("config.lua")

-- Connect to the wifi network using wifi module
function func_wifi_conn()
  wifi.setmode(wifi.STATION)
  wifi.sta.config(wifi_ssid, wifi_password)
  wifi.sta.connect()
end

-- Read out DHT22 sensor using dht module
function func_read_dht()
  status, temp, humi, temp_dec, humi_dec = dht.read(2)
  if( status == dht.OK ) then
-- Integer firmware using this example
    print("DHT Temperature: "..math.floor(temp).."."..temp_dec.." C")
    http_temp = math.floor(temp).."."..temp_dec
    print("DHT Humidity: "..math.floor(humi).."."..humi_dec.." %")
    http_humi = math.floor(humi).."."..humi_dec
-- Float firmware using this example
--    print("DHT Temperature: "..temp.." C")
--    print("DHT Humidity: "..humi.." %")
  elseif( dht_status == dht.ERROR_CHECKSUM ) then          
    print( "DHT Checksum error" )
  elseif( dht_status == dht.ERROR_TIMEOUT ) then
    print( "DHT Time out" )
  end
end

-- Publish temperature readings and activate deep sleep
function func_http_post()
  http.post('https://iotmms%SAPUSERID%.hanatrial.ondemand.com/com.sap.iotservices.mms/v1/api/http/data/%DEVICEID%',
  'Authorization: Bearer %OAUTHACCESSTOKEN%\r\nContent-Type: application/json\r\n',
  '{"mode":"sync","messageType":"%DEVICETYPEID%","messages":[{"temp":'..http_temp..',"humi":'..http_humi..'}]}',
  function(code, data)
    if (code < 0) then
      print(code, "HTTP request failed")
      print("Going into deep sleep mode for "..(dsleep_time/1000).." seconds.")
      node.dsleep(dsleep_time*1000)
    else
      print(code, data)
      print("Going into deep sleep mode for "..(dsleep_time/1000).." seconds.")
      node.dsleep(dsleep_time*1000)
    end
  end)
end

-- Read previous temperature and humidity measurements
function func_rtcmem_read()
  rtcmem_temp = rtcmem.read32(0)
  rtcmem_humi = rtcmem.read32(1)
  print("Previous measurements retrieved from memory: ", rtcmem_temp, rtcmem_humi)
--  end
end

-- Write new temperature and humidity measurements to persistent memory
function func_rtcmem_write()
  rtcmem.write32(0, (http_temp*1000))
  rtcmem.write32(1, (http_humi*1000))
  print("Temperature and humidity written to memory")
end

-- Read and compare sensor data and determine further processing
function func_read_data()
  func_read_dht()       --Retrieve sensor data
  print("Current temp ", http_temp)
  func_rtcmem_read()    --Retrieve temp and humi from memory
  print("Previous temp ", (rtcmem_temp/1000))
  if (http_temp*1000) == rtcmem_temp then --No change, go into deep sleep
    print("No change in temperature")
    print("Going into deep sleep mode for "..(dsleep_time/1000).." seconds.")
    node.dsleep(dsleep_time*1000)
  else                                    --Change, capture new sensor data
    func_wifi_conn()    --Connect to WIFI network
    tmr.alarm(2,500,tmr.ALARM_AUTO,function() func_post_loop() end)
  end
end

-- Post and store newly capture sensor data
function func_post_loop()
  if wifi.sta.status() == 5 then --STA_GOTIP
    print("Connected to "..wifi.sta.getip())
    tmr.stop(2)         --Exit loop
    func_rtcmem_write() --Write temp and humi to memory
    func_http_post()    --Post HTTP message to HCP and go to sleep
  else
    print("Still connecting...")
  end
end

tmr.alarm(1,500,tmr.ALARM_SINGLE,function() func_read_data() end)

The init.lua script should be modified to execute the newly created script by modifying the following commfand: FileToExecute=”http.lua”. Upload the scripts NodeMCU device.

SAP HCP IoT Starterkit

The SAP IoT Starterkit provides a number of scenarios to jumpstart to visualize the data captured by the NodeMCU board. In this prototype I’ve leverage the scenarion which creates a Java and UI5 web application using the HCP Persistence Service.

In order to create the web application created clone the source  repository to your local machine. I suggest using Eclipse and its import function to clone the master of the IoT Starterkit GIT repository.

Once successfully cloned slight changes must be made to the example web application to integrate with the message type we created earlier. The application can be found under: src > apps > java > consumption > com.sap.iot.starterkit.ui.

Locate the com.sap.iot.starterkit.ui/src/main/webapp/js/view/outbound.view.js file and replace the function `createDataSet` with the code below. This code snippet below replaces the source of the graph dimension to field `G_CREATED` which contains the date+timestamp of a sensor reading. The data visualized in the line graph will be `temp` which is stored in the IoT table field `C_TEMP`.

createDataSet: function() {
		var oController = this.getController();
		
        return new sap.viz.ui5.data.FlattenedDataset( {
			dimensions: [ {
				name: "timestamp",
				value: {
					path: "data>G_CREATED",
					formatter: function( oValue ) {
						return oController.formatDate( oValue );
					}
				}
			} ],
			measures: [ {
				name: "temp",
				value: "{data>C_TEMP}"
			} ],
			data: {
				path: "data>/"
			}
		} );
	},

A second adjustment must be made in the function `createMeasureFeed` to visualize the correct information on the graph axis:

createMeasureFeed: function() {
		return new sap.viz.ui5.controls.common.feeds.FeedItem( {
			"uid": "primaryValues",
			"type": "Measure",
			"values": [ "temp" ]
		} );
	},

Once these code changes have been made the actual web application must be build so it can be deployed on the HCP as Java application. In Eclipse right click the com.sap.iot.starterkit.ui node, select Run-as and select new configuration under Maven build. In order to generate a .war file set the goal `clean install`. Once successfully completed a deploy ready .war file is created in the target directory.

The .war file can be deployed on the HCP using the HCP cockpit > Java Application > Deploy Application. Once complete do not yet start, we need to establish data bindings between the sensor data sets and the IoT service.

The cloned iot-starterkit application in Eclipse comes in handy here as well. In the HCP cockpit > Connectivity > Destinations > Import Destination. Navigate to the com.sap.iot.starterkit.ui/destinations folder and you will find two files: iotmms and iotrdms. Import both files and adjust the url, username and password with your account information.

Final step is the actual data binding , select the uploaded Java application and Configuration > Data Source Bindings > New Binding. Create the new binding for the application to leverage the same database schema as the IOTMMS application, something like: /%SAPUSERID%trial.iotmms.web/bindings. Start the application service and select its url.

If all went well you should be able to see the readings of your DHT22 sensor in the SAP Hana Cloud Platform. If your IoT device has run for a while you should see something like the graph below. Well done!

* Source: Github repository for SAP IoT Starterkit