In previous blogs we explained some of the SAP IoT services like Data Ingestion, Onboarding & Device Management and other low-level services that helped us to define our Device and Thing Model together with the master data and measures associated. In this blog I’ll like to focus on some other services required to take advantage of the measures retrieved.
On the IoT word we talk about big amounts of data being generated by devices equipped with sensors. First topic that needs to be understood is where the data is stored and how it is managed to provide performances to applications developed on top of that data, let’s talk about Time series in the first section of this blog.
Once the data stored and managed we need to analyze and react on the data to designate business actions to be executed. We will cover that topic in the following sections talking about Rules, Actions and Query Services.
A first service proposed by SAP IoT to help us store and manage the big amounts of data is the Time Series Management.
There are 3 different storage types that differ in access speed and costs:
It is the highest performance storage alternative. SAP IoT stores in Hot Storage aggregates of time series data as well as event data.
Those aggregates are used to deliver data very quickly, sub second, to applications. Applications do not require raw data for most scenarios. E.g. a time series chart can typically display not more than 1900 data points on an HD display. If a sensor collects per second data, all charts showing a time window larger than 30 min must use aggregated data. Aggregates are stored for 1 year and then moved to warm storage.
Apart from providing aggregates, on hot storage also simple analytical queries can be directly executed on this storage.
All aggregates in hot storage are automatically calculated at data ingestion time for the time windows: 2 min, 1 hour, 1 day, 1 week.
Several aggregates are available by default: First , Last, Timestamp of first, Timestamp of last, Min, Max, Timestamp of min, Timestamp of max, Average, …
Used to store and read raw time series data and deliver it quickly to applications.
The raw data stored in warm storage has a standard retention period of 60 days but can be set to a custom retention period.
Able to store large amounts of raw time series data cost efficiently for an almost unlimited amount of time. It is optimized for cost efficient long-term storage. Accordingly, reading from cold storage is not as performant as from warm storage.
Check the blog FAQs for Big Data & Analytics on Timeseries within SAP IoT Application Enablement (Leonardo Foundati... for more details on SAP IoT data storage.
The Rule-based IoT Data Processing service allows you to create some conditions on your measured values and fire an Event in the case the conditions are met. You can create rules on the streamed data or for some specific periods of time. You don’t need then to read in your application all the measures nor to run tasks to check measures in a certain period of time in the past, you better define rules to get measures automatically filtered for you.
The next screen shows the IoT Rules section proposing tiles for defining Rules and Rule Contexts.
You start with a virtual thing that you want to monitor, pick the properties to be observed, and associate them both by creating a Rule Context. Then you define the threshold values to be checked for each property and what shall happen if that particular value is observed in the stream of incoming real time data or over a certain period of time with a Rule.
In the particular case shown here we are creating a streaming rule on the Thing Type “SMBRobot”, we check if the luminosity property is lower than 50 lux and if the condition is met then we fire an IoT Event with the name “LowLumEvent”. Once this event fired we can define an action that will react on that event, check next section for more details on Actions.
Once an event is fired by a Rule we can define the Actions that will be executed. In the SAP IoT we have 2 tiles associated to Actions: Actions and Notifications & Email Templates. We will concentrate in the Actions, the Notifications & Email Setup will help you prepare Actions templates.
Once you select the Actions tile you can define your action based on your Rules.
In order to call your backend you need to define a Destination in your BTP account specifying the URL to be called when the action is executed, in our case we call the Messages oData service:
Query Models are structured like OLAP views and provide extensible metadata, such as labels, units of measures, and hierarchical relations. Both the content and the metadata are provided through standard interfaces so that you can analyze and visualize data with the analytics tools of your choice, for example, SAP Analytics Cloud.
To create a Query Model you can use the Query Modeler tile of the SAP IoT services:
Query Models are defined by:
Here you can see the definition of a Query Model based on the Luminosity Average measure aggregation having the Hour and ItemCode of the Thing as Dimensions:
Check the Creating a Query Model help documentation for more details on how to create a Query Model.
If a user is assigned to this particular role collection, then the user is able to consume the query model as for instance in SAP Analytics Cloud.
Tenant Administration is of a group of apps covering organizational tasks to be done during onboarding of companies and persons to the SAP IoT platform. A tenant is a technical entity offered by the platform and is used by administrators (or automatically in the background by the Companies app) to establish a 1:1 relationship between a tenant and a company. For SAP IoT being a multi tenancy platform where many companies are hosted on the same hardware simultaneously, it is crucial to strictly separate the data that belongs to different companies. This is accomplished with the tenant concept. The tenant serves as a container for company-specific data and isolates this data from the data of other companies.
The Tenant Administration apps comprise the following:
For more details on Tenant Administration please check Tenant Administration help documentation.
In addition to these organizational aspects, there is a second area of system administration: The definition and modeling of things, that is, the digital twins of real-world objects that a company wants to manage. For this type of administrative tasks, SAP offers a second group of apps, namely the Thing Modeler apps that we already covered in the blog SAP IoT – Digital Twin.
In the blog SAP IoT – How to define your Device Model and start persisting measures I already shared some documentation and samples on how to define your Device Model and how to retrieve measures via Internet of Things APIs, please check that blog to refresh your knowledge and get all documentation links.
Then the API Reference for SAP IoT services compiles a set of REST-based as well as OData-based services to store and retrieve data efficiently like for example:
Several videos are available in the SAP Internet of Things Youtube academy:
The blog Access the SAP IoT Application Enablement APIs using Postman shows how to obtain the token required to do further API calls.
Also important to note, is the availability of the SAP IoT SDK for Node.js in GitHub. The SAP IoT SDK for Node.js implements the consumer side usage of the most frequent used APIs of SAP IoT. All API calls will be automatically enriched with authorization information to ensure a simple consumption of the SAP IoT services. The GitHub repository includes the SDK documentation, how to install it and several showcases on how to implement apps on top of the SAP IoT SDK for Node.js.
Check the full set SAP IoT blog series:
Along the blog I provide many pointers to already available documents and tutorials covering the different topics I talk about; my goal is to compile all the links together on these blogs and not to repeat the materials already available. Please play the game and click on the different links not to miss any detail!
Happy IoT!
On the IoT word we talk about big amounts of data being generated by devices equipped with sensors. First topic that needs to be understood is where the data is stored and how it is managed to provide performances to applications developed on top of that data, let’s talk about Time series in the first section of this blog.
Once the data stored and managed we need to analyze and react on the data to designate business actions to be executed. We will cover that topic in the following sections talking about Rules, Actions and Query Services.
Time Series
A first service proposed by SAP IoT to help us store and manage the big amounts of data is the Time Series Management.
There are 3 different storage types that differ in access speed and costs:
Hot Storage
It is the highest performance storage alternative. SAP IoT stores in Hot Storage aggregates of time series data as well as event data.
Those aggregates are used to deliver data very quickly, sub second, to applications. Applications do not require raw data for most scenarios. E.g. a time series chart can typically display not more than 1900 data points on an HD display. If a sensor collects per second data, all charts showing a time window larger than 30 min must use aggregated data. Aggregates are stored for 1 year and then moved to warm storage.
Apart from providing aggregates, on hot storage also simple analytical queries can be directly executed on this storage.
All aggregates in hot storage are automatically calculated at data ingestion time for the time windows: 2 min, 1 hour, 1 day, 1 week.
Several aggregates are available by default: First , Last, Timestamp of first, Timestamp of last, Min, Max, Timestamp of min, Timestamp of max, Average, …
Warm Storage
Used to store and read raw time series data and deliver it quickly to applications.
The raw data stored in warm storage has a standard retention period of 60 days but can be set to a custom retention period.
Cold Storage
Able to store large amounts of raw time series data cost efficiently for an almost unlimited amount of time. It is optimized for cost efficient long-term storage. Accordingly, reading from cold storage is not as performant as from warm storage.
Check the blog FAQs for Big Data & Analytics on Timeseries within SAP IoT Application Enablement (Leonardo Foundati... for more details on SAP IoT data storage.
Rules
The Rule-based IoT Data Processing service allows you to create some conditions on your measured values and fire an Event in the case the conditions are met. You can create rules on the streamed data or for some specific periods of time. You don’t need then to read in your application all the measures nor to run tasks to check measures in a certain period of time in the past, you better define rules to get measures automatically filtered for you.
The next screen shows the IoT Rules section proposing tiles for defining Rules and Rule Contexts.
You start with a virtual thing that you want to monitor, pick the properties to be observed, and associate them both by creating a Rule Context. Then you define the threshold values to be checked for each property and what shall happen if that particular value is observed in the stream of incoming real time data or over a certain period of time with a Rule.
In the particular case shown here we are creating a streaming rule on the Thing Type “SMBRobot”, we check if the luminosity property is lower than 50 lux and if the condition is met then we fire an IoT Event with the name “LowLumEvent”. Once this event fired we can define an action that will react on that event, check next section for more details on Actions.
Actions
Once an event is fired by a Rule we can define the Actions that will be executed. In the SAP IoT we have 2 tiles associated to Actions: Actions and Notifications & Email Templates. We will concentrate in the Actions, the Notifications & Email Setup will help you prepare Actions templates.
Once you select the Actions tile you can define your action based on your Rules.
In order to call your backend you need to define a Destination in your BTP account specifying the URL to be called when the action is executed, in our case we call the Messages oData service:
If your backend requires the Xsrf token to be fetched you can add the properties:
- sap.iot.fetchXcsrf - true
- sap.iot.XcsrfURL - pointing to your oData service URL (a get request will fetch the token before the POST request is sent)
Query Service
Query Models are structured like OLAP views and provide extensible metadata, such as labels, units of measures, and hierarchical relations. Both the content and the metadata are provided through standard interfaces so that you can analyze and visualize data with the analytics tools of your choice, for example, SAP Analytics Cloud.
To create a Query Model you can use the Query Modeler tile of the SAP IoT services:
Query Models are defined by:
- Measures that define the properties to be tracked and for which aggregation method.
- Dimensions that define the criteria for which the aggregated measures are displayed like for example per location or specific time or date.
Here you can see the definition of a Query Model based on the Luminosity Average measure aggregation having the Hour and ItemCode of the Thing as Dimensions:
Check the Creating a Query Model help documentation for more details on how to create a Query Model.
If a user is assigned to this particular role collection, then the user is able to consume the query model as for instance in SAP Analytics Cloud.
Tenant Administration
Tenant Administration is of a group of apps covering organizational tasks to be done during onboarding of companies and persons to the SAP IoT platform. A tenant is a technical entity offered by the platform and is used by administrators (or automatically in the background by the Companies app) to establish a 1:1 relationship between a tenant and a company. For SAP IoT being a multi tenancy platform where many companies are hosted on the same hardware simultaneously, it is crucial to strictly separate the data that belongs to different companies. This is accomplished with the tenant concept. The tenant serves as a container for company-specific data and isolates this data from the data of other companies.
The Tenant Administration apps comprise the following:
- Onboarding and maintaining companies
- Onboarding and maintaining persons (with a person always seen as a company member)
- Object authorizations, which define subsets of the company-specific objects that a particular user may access
For more details on Tenant Administration please check Tenant Administration help documentation.
In addition to these organizational aspects, there is a second area of system administration: The definition and modeling of things, that is, the digital twins of real-world objects that a company wants to manage. For this type of administrative tasks, SAP offers a second group of apps, namely the Thing Modeler apps that we already covered in the blog SAP IoT – Digital Twin.
APIs
In the blog SAP IoT – How to define your Device Model and start persisting measures I already shared some documentation and samples on how to define your Device Model and how to retrieve measures via Internet of Things APIs, please check that blog to refresh your knowledge and get all documentation links.
Then the API Reference for SAP IoT services compiles a set of REST-based as well as OData-based services to store and retrieve data efficiently like for example:
- Create a Thing
- Get the list of defined Things
- Get the list of Events
- Get the last data snapshot
- …
Several videos are available in the SAP Internet of Things Youtube academy:
- SAP HANA Academy - SAP IoT: IoT App Enablement 4of8 : Postman Collections
- SAP HANA Academy - SAP IoT: IoT App Enablement 5of8 : Updating Location
The blog Access the SAP IoT Application Enablement APIs using Postman shows how to obtain the token required to do further API calls.
Also important to note, is the availability of the SAP IoT SDK for Node.js in GitHub. The SAP IoT SDK for Node.js implements the consumer side usage of the most frequent used APIs of SAP IoT. All API calls will be automatically enriched with authorization information to ensure a simple consumption of the SAP IoT services. The GitHub repository includes the SDK documentation, how to install it and several showcases on how to implement apps on top of the SAP IoT SDK for Node.js.
More resources
Check the full set SAP IoT blog series:
- SAP IoT for SMBs
- SAP IoT – How to define your Device Model and start persisting measures
- SAP IoT – Digital Twin
- SAP IoT – Services
- SAP IoT – WebIDE templates and controls
- SMB.io: IoT mobile app prototype
Along the blog I provide many pointers to already available documents and tutorials covering the different topics I talk about; my goal is to compile all the links together on these blogs and not to repeat the materials already available. Please play the game and click on the different links not to miss any detail!
Happy IoT!
- SAP Managed Tags:
2 Comments
