Capture event streams from IoT devices and perform predictive analytics using HCP – Part 1
I had earlier posted a blog which shows how to use Remote Data Sync service on HCP to have bi-directional synchronization of data between many remote databases at the edge and SAP HANA cloud database at the center. This is perfect for occasionally connected devices. In this blog series, I wanted to take a different approach and use some of the services in HCP to capture event streams in real-time, process the data and apply predictive analytics to predict future outcomes.
There are three services I will use to achieve this – Smart Data Streaming, IoT Services & Predictive Services.
In the initial part of the blog series, I will focus on Smart Data Streaming and IoT concepts and will later cover Predictive services.
The objective of this blog series is to show how one could use Smart Data Streaming (SDS) to capture and filter sensor messages from an IoT device and later apply predictive analytics to gain some insights.
Smart Data Streaming
Smart Data Streaming (SDS) service processes high-velocity, high-volume event streams in real-time. It allows you to capture data arriving continuously from devices and applications, watch for trends & pattern and can generate notification and alerts accordingly.
Here are some popular use cases
- IoT – capture event streams from smart devices and sensors to predict failures
- Gain more insights from Social Media by analysing the external users in real time
- Users action in a website are analysed in real time to accordingly show best offers
- IT system events are continuously monitored to detect threats by looking for specific patterns
- Access to a HCP Account which has a HANA Platform Edition (SDS is not yet available on trial, however Predictive services are available as of today)
- Smart Data Streaming service provisioned for the HCP account
- HANA Studio on your laptop
Create users in SDS:
Once your HANA DB and SDS is provisioned, you will need to setup users. Navigate to the SAP HANA Cockpit and create two users.
SDSADMIN – Use this database user to perform policy administration functions, such as granting and revoking privileges.
HANAUSER1 – Use this database user to connect to any web service or use streaming in HANA Studio
Installing Smart Data Streaming components:
You will need to download and install some components on your laptop to connect to the SDS service on HCP. You will need to install the below two items.
- Streaming client package – contains streaming command line tools and set of adapters to connect to various data sources
- HANA Studio Plugin – required to create streaming projects
More information on how to install the client package and HANA Studio plugin can be found in SAP Help.
Setting up permissions:
In your environment variables, set Streaming home to the directory where Streaming client package is installed
Assign the below roles to the SDSUSER.
You need to grant permissions to HANAUSER1 user before it can connect to any web services or use streaming in SAP HANA studio.
Start streamingclusteradmin in interactive mode for your SAP HANA instance and issue the below commands
grant perm all on all to user HANAUSER1
You can now login to SAP HANA Cockpit as SDSADMIN user and verify the permissions. Once you are in the HANA Cockpit, click on the Streaming permissions tile
Notice that I have permission “all” against privileges/resource types for the HANAUSER1.
In the next part, I will create a Streaming Project using HANA Studio.