Message Processing in HCP IoT Service
Few months back, Message Processing functionality was introduced in HANA Cloud Platform IoT services. This provides more flexibility to process incoming messages. You no longer need to persist the messages in HCP, you could either forward the incoming message to a web service, message broker or Smart Data Streaming depending on your requirements.
In this blog, I am going to focus on the SQL message processing option. This is the default processing service which stores all the incoming messages in a database table. If none of these options are maintained, we have seen how MMS automatically creates a message table with a name which matches the Message type ID. It follows the naming scheme T_IOT_[Message Type ID]. You also would recollect that there would be two columns named G_DEVICE and G_CREATED which represent the device ID and timestamp when the message was stored in the table. All the remaining message fields would be mapped to a column prefixed with C_.
With SQL Message processing service, you longer need to stick to a I_IOT_MessageID table with such column definitions. You can create your own table, in your own schema and have columns with the name you provide.
Navigate to the HCP IoT services cockpit to create Message Type/Device Type and Devices
In the below screen, I have created a message type with two fields
I have assigned the message type to a device type.
Finally I have created a Device for the above device type.
In the payload, I provide the new table name along with the mappings of the message fields with table columns. Notice that I have set “add_default_columns” to true. This will automatically add G_DEVICE and G_CREATED columns in the table.
When I click on SEND, it would provide a below response stating that the service mapping has been registered.
Navigate to the MSS cockpit and click on the “Configure Processing Service Mappings”
You will notice there is a Processing service entry of type SQL. If you want to add more service mappings, you could do it straight from this UI too.
In the detail screen, you will be able to verify the table name along with the message field mappings to table columns
Now, we can test the message by using the test client provided in MMS cockpit.
I am going to use HTTP APIs to test the IoT messages from thecockpit
You will notice a success message as shown below
You can use the Service mappings to create a flexible chain of message processing by forwarding the IoT messages to other web services and event stream processors. I have also posted on how to configure Service mappings for Smart Data Streaming in Capture event streams from IoT devices and perform predictive analytics using HCP – Part 4
Hi,
I am trying the above.
I get a success for create service mapping. [{"msg":"Processing service mapping with device type [d0t0y0p0e1_11] and message type [m0t0y0p0e1_11] has been registered"}
]
In Processing Message Mapping screen, i can see an entry created and also its details.
But i do not see the device/message type in device management/Service cockpit screen .
Without Device ID for this new config, i cannot push data from HTTP API client.
Can you please let me know if i have missed any steps? and how to fix it.
Thanks,
When registering a mapping, use IDs and not Device Type / Message Type names.
Hi,
I used numbers for device type and message types, still get the same message.
{"msg":"Processing service mapping with device type [10011001] and message type [01100110] has been registered"}
Thanks,
Please, use not just arbitrary numbers but the IDs of the MT/DT. You get an ID after you register MT/DT in IoT Cockpit.
Sorry, My bad.
I have corrected, have attached screenshot with details.
Can you please advice. Am I missing anything?
Thanks,
Seems to be correct. Now when you will send messages (with that message type id) to MMS on behalf of your concrete device (which is assigned to that device type), they will be processed by the processing service(s) you have mapped to that MT/DT pair.
Thanks.
issue is Device is still not registered. So I cannot push data.
To send message, I need Device ID to be appended at the end of 'Data Endpoint'.
Thanks for your patience.
Sorry, but I do not really understand your point. If the last screenshot lists your devices, then select one. On the right hand side (right UI panel) you will see its ID (alphanumeric ID separated with dashes)
The *berry* devices were manually created for another set of Mt/DT and are not for the MT/DT which I created just now. so, I think I cant use those. 😕
Then create a new device and assign it to the desired MT. I also encourage you to re-check the HCP IoT Services documentation SAP HANA Cloud Platform Internet of Things Services
P.S. I am afraid our discussion goes beyond "clarification comment for the current blog post". Unless you are asking for clarification/correction of some part of the Document, please create a new Discussion marked as a Question. The Comments section of a Blog (or Document) is not the right vehicle for asking questions as the results are not easily searchable. Once your issue is solved, a Discussion with the solution (and marked with Correct Answer) makes the results visible to others experiencing a similar problem. If a blog or document is related, put in a link. Please, get familiar with the The SCN Rules of Engagement
Hi Anton,
I got it working.
I manually created the device also, and I am able to send messages and also table is created.
I thought, .../v1/api/http/processing would register the device and then I have to use the registered device to push Data.
Thanks a lot for your patience and help.
Regards,
Great blog,
I am trying to add processing service for the mapping, however  i could not list any processing services
Â
assume i have created a device type for eg) esp12 and msg type like temperature msg with relevant id’s populated as shown in the image below.
Â
What am i missing?.
Â
Â
--Update
It started working after a day, seems like HCP takes some time to refresh the services..
Â
Â
Â
Hi Anand, it could have been an intermittent issue. Good to know that it works now.
Hi Murali,
Thank you for the excellent blog on Processing services.
Just want to know if processing Services available in Cloud Foundry IoT Services as well?
Thank you and Best Regards,
Venu
Thanks Murali for the information.
Just want to understand if we have something similar available with IoT V4 services in Cloud Foundry? If yes, can you please share some information on that one.
Thanks,
Amit.