E2E: SAP Cloud Platform Internet of Things to SAP Analytics Cloud

Using SAP Cloud Platform IoT service to ingest edge device data is very handy. I recently wrote a small blog how to use Python for sending in the data by MQTT.

This blog is about consuming the received data from IoT service by first storing the data with SAP HANA Cloud, then providing it to SAP Analytics Cloud to derive insights or predictions from it.

What we will look at in this blog is:

1. Configuring SAP IoT Cockpit to send the edge device data by HTTP
2. Creating a CAP-defined data persistence for SAP HANA Cloud
3. Publishing OData services to SAP Analytics Cloud
4. Setting up the SAP Analytics Cloud Connection

Note: Authorization checks are not part of the blog for the sake of brevity. You must however consider these in your application. You can read my blog on how to implement them with CAP.

Overview of setup

High-level architecture and scope of this post

Configuring SAP IoT Cockpit to send the edge device data by HTTP

To start off, we need to be clear about the capabilities defined for the sensor type(s) that we want to integrate. As a recap, let’s check out the SAP Cloud Platform Internet of Things data model:

SAP Cloud Platform Internet of Things: Data model

We have set up our device in the IoT cockpit. As a next step it we need to know how the platform communicates the device data. For that, we can make use of a test REST server (I use beeceptor) which receives data from the SAP IoT and shows it to us. Of course, you would only do that with a web-based service if there are no data security restrictions!

Let’s set up an HTTP configuration pointing to our test server as well as a data selector configuration to direct the incoming device data to our HTTP configuration.

HTTP configuration to test REST server using JSON data notation

Data selector setup to use the HTTP configuration for all incoming IoT data

We are good to send test data from the device! Now let’s see what happened on the receiver side, our test REST server.

SAP IoT: Ingested data received by mock server

There we have our data! Let’s head for modeling!

Creating a CAP-defined data persistence for SAP HANA Cloud

We need a development environment for Cloud Foundry, that can be SAP Business Application Studio or Visual Studio Code with the according plugins. I’m developing it today in VS Code.

After initializing our workspace (cds init <project> etc.), we create two files, db/ambient-model.cds and src/ambient-service.cds. The first representing the data model on HANA, the second exposing the model as an OData service so that we can store data from SAP IoT.

namespace ambience;

// Data as provided by SAP IoT service
entity IoTData {
    key tenantId            : String(40);
    key capabilityId        : String(40);
    key sensorId            : String(40);
    key timestamp           : Double; // Unix timestamps
    gatewayTimestamp        : Double;
    measures: Composition of many {
        CO2         : Integer;
        temperature : Double;
        humidity    : Double;
    }
};

The data model for HANA based on our testing with the REST server

using ambience as a from '../db/ambient-model';

service AmbientDataService {
    entity AmbientData @insertonly as projection on a.IoTData;
}

The service to allow inserting data from SAP IoT

Build the CDS to generate the artifacts ($env:CDS_ENV = “production” ; cds build on Windows. Other OSs please read here). Deploy the DB (cf push -f gen\db), then the service (cf push -f gen\srv –random-route). You will get a URL after the last command ran through. Paste it in the browser and check it:

CDS services provided in Cloud Foundry environment

We need to switch the URL in SAP IoT Service HTTP Configuration to the URL that the service provides. in our case the URL would be:

https://<provided by your deployment>.hana.ondemand.com/ambient-data/AmbientData

Let’s play in some fresh data after the change and check it in the database explorer. We should see it inserted into HANA.

SAP IoT data in HDB table

Let’s now move to the next step.

Publishing OData services to SAP Analytics Cloud

That’s a tiny step, we just flatten the data in a simple read-only view. Let’s add this view to the db/ambient-model and the service to the src/ambient-service.cds.

define view iotDataSac as select from IoTData{
    key IoTData.sensorId,
    key ADD_SECONDS(TO_TIMESTAMP ('1970-01-01 00:00:00'), IoTData.timestamp / 1000) as timestamp: Timestamp,
    measures.CO2, 
    measures.temperature, 
    measures.humidity
};

SAC view on the data with timestamp conversion from Unix data type (ambient-model.cds)

using ambience as a from '../db/ambient-model';

service AmbientDataService {
    entity AmbientData @insertonly as projection on a.IoTData;
    entity AmbientDataSac @readonly as projection on a.iotDataSac; 
}

SAC service added to ambient-service.cds

Let’s build and deploy in the same way explained in the previous step.

We now have a view that can be processed by SAP Analytics Cloud.

Setting up the SAP Analytics Cloud Connection

Final step of this blog, we head over to the SAP Analytics Cloud and define our data connection. We provide an OData service from SAP HANA Cloud so let’s set it up as such.

SAC Connection details – Note: Specify the route to the service

We can now create a model based on the data. Let’s first add a few more datapoints ?

Selection of model based on previous created connection

Selection of OData entity: SAC view

Model creation: Query creation wizard

Finally, we are good to create the model – setting the measures and data formats right and are ready to consume it in e.g. a time series prediction.

SAP Analytics Cloud: Model creation final step

Closing

Let me know your thoughts or experiences setting up similar scenarios. Hope it was helpful getting started with IoT data in SAP Analytics Cloud!