Connecting IoT Services to Streaming Analytics in SAP Cloud Platform
The most popular use cases for SAP Cloud Platform Streaming Analytics (which is the cloud version of SAP HANA smart data streaming) are around the Internet of Things. Therefore, many use cases will want to use Streaming Analytics in conjunction with the SAP Cloud Platform Internet of Things service.
These two services are complimentary:
- The IoT service provides a gateway for receiving data from smart devices, and can either store the incoming device data in the HANA database and/or can direct the incoming data to a Streaming Analytics project for processing/analysis
- Streaming Analytics provides the ability to process/analyze the data, with the ability to do things such as:
- watch for patterns or trends – generate alerts or initiate a response when specific situations are detected
- filter, aggregate or sample the data – reduce the amount of data being captured in the HANA database
- enrich or transform the data, combine raw events into higher level events that get recorded in the database
- calculate KPIs or summary data that is recorded in the database or displayed on a dashboard
So here’s how to configure the IoT Service to send device data to a streaming analytics project:
Configure the IoT Message Management service to send specific data streams to the Streaming Analytics service for processing
Here’s the section of the IoT services documentation on how to configure the SDS Processing Service in IoT Services. This configures MMS routing, mapping specific message types from specific device types to SDS input streams.
You can also configure this connection from the IoT Service Cockpit:
- From HCP Cockpit, go to the IoT Service
- From the IoT Services Cockpit, select the MMS Cockpit
3. Configure processing service mappings
4. Add a mapping
5. Add a processing service
6. Select “sds”
7. Fill in the form to configure the SDS processing service as follows:
|SDS Streaming Web Service host||
where: hdb1 is the HANA database name, xyz123 is the hcp account name and us1 is the hcp landscape. See this blog for more info on figuring out your SWS host name
|SDS Input Stream||
the name of the CCL input stream that will receive the data
|SDS Streaming Web Service Port||443 (don’t change this)|
the name of the SDS project to publish to
|SDS Streaming Web Service Protocol||wss (don’t change this)|
|SDS Streaming Web Service Password||These are the credentials for a HANA user that has been granted SDS permission to publish to projects in the workspace where this SDS project is running|
|SDS Streaming Web Service User name|
name of the SDS workspace where your SDS project is running
Configuring the SDS Project to receive data
See the IoT Services documentation on the SDS processing service.
Here’s an example of a CCL input stream set up to receive data from MMS:
CREATE INPUT STREAM T_IOT_ M0T0Y0P0E1 SCHEMA ( g_device string, g_created msdate, c_sensor string, c_value string, c_timestamp msdate);
Note that the schema of the CCL input stream must reflect the MMS message type you have mapped to SDS. The schema must have one column (field) corresponding to each field in the MMS message type, and the datatypes must match (or be compatible). These columns must be named beginning with the prefix c_ . There must be two additional for the device ID (g_device) and the timestamp (g_created) of the message insert; these are not part of the MMS message type but are generated automatically.
If you have set everything up but don’t see data arriving in the SDS project, here are suggested steps to troubleshoot the problem.
- Check that the SDS Streaming Web Service is running (via the HANA Cockpit)
- Using HANA Studio, connect to your SDS server and use the run test tools to confirm you can publish to the SDS project:
- Open a stream viewer on the input stream of your project
- Use the manual input tool to send an input event to the project
- Confirm that it appears in the stream viewer
- If the project isn’t running, start it. If you can’t start it, check the project log to see why (see step 4 below)
- Check the MMS log to see if it reports any errors connect to SDS or publishing to SDS
- Go to the Java Applications page in HCP Cockpit
- Click on the iotmms application
- View the Default Trace log file (scroll down to “Most Recent Logging”) and see if any errors are indicated
- View the SDS log files
- In HANA Studio, go to the SAP HANA Administration Console perspective
- Right click on your HANA system in the Systems view and Open the Administration view
- In the Administration View, go to the Diagnosis Files tab, find both the streaming project log file and the streaming server log file. Both will start with “streamingserver…”. The server file will end in .log and the project log file will have the workspace and project name in the file name and will end in .trc. Examine both of these for indicators of the problem
I have done the steps as your instructions, but an error occurred during connect to Streaming Analytics from SAP HANA Studio, The streaming project runs in the HCP HANA streaming server, but still I can't connect to the server that I added. and the HANA streaming server was down due to attempt many times.
Can you please help on it or give a further advise ?
What specific error(s) are you receiving?
Thank you for sharing the details how Streaming Analytics and IoT services can be complementary.
Just want to know if we can also connect Cloud Foundry IoT Service to SDS Processing service?
Thank you in advance,
No, not yet.
Hi, How can we make this work for IoT Service 4.0 that does not have Message Management Service Cockpit?