Deliver real-life use cases with SAP BTP - Sustain...

MostafaSharaf · ‎12-08-2023

This blog post belongs to a series of technical enablements on SAP BTP for Industries. It has two parts: Part1 (overview) and Part2 (deep-dive).

Please check the full calendar here to watch the recordings of all program sessions.

Authors: yatsea.li, thiago, mostafa.sharaf.

In this second part, we will mainly deep dive into the implementation of the proposed healthcare waste management solution in order to understand more in details how we built the prototype empowered by SAP BTP.

Data Federation of Waste Data and Cost Data into SAP Datasphere

We’ll begin with Data Federation, which, in this proof of concept, is demonstrated conceptually without actual implementation. As mentioned earlier, this process involves federating Waste Data from Smart Bins and Cost Data from SAP S/4HANA Cloud into SAP Datasphere. This integration aims to facilitate comprehensive and informed analytics and forecast for Bob the sustainability analyst on sustainability metrics of medicate waste, and John the financial analyst on financial metrics of medical waste.

If you are interested in learning more about IoT integrated with SAP BTP, please refer to our session on the Flexible Energy Grid to explore further insights into the IoT solution approach.

The waste data comes from Smart Bin look like this;

The waste cost data comes from SAP S/4HANA Cloud look like this;

Since the data federation isn’t implemented in our PoC, we’ll stimulate the dataset in csv and import into SAP Datasphere, which look like;

Healthcare Waste transactions data have combined both waste data from smart bin with the waste sustainability metrics, encompassing factors like Waste Volume in kilograms, greenhouse gas emissions produced by the waste, and waste cost data from SAP S/4HANA Cloud, indicating the cost in euros for waste processing services by the service provider. This combined dataset serves as the foundational data for forecasting waste sustainability and financial metrics.

The waste transaction is also enriched with associations to another two master data datasets (Medical Facility & Waste Category).

Waste Metrics Forecasting with Federated Machine Learning using SAP Datasphere and & Hyperscaler AI Platform

To help Bob the sustainability analyst on keep tracked waste sustainability KPIs such as Waste Volume in kilogram and Greenhouse Gas Emission produced by the waste, John the financial analyst on better budget allocation of the cost for processing the waste for the coming months, machine learning is very helpful in forecasting the waste metrics for the next 12 months based on the historical data.

Next, we’ll move on the SAP Federation Machine Learning, learning how to build a medical waste forecasting model on Amazon SageMaker with the help of SAP FedML.

Before delving into how-to, you are recommended to read this blog post about an overview introduction to SAP Federation Machine Learning(FedML) by sangyrak1 for why and what it is about SAP FedML, and Federated Machine Learning using SAP Datasphere and Amazon SageMaker 2.0by karishma_kapur

Amazon SageMaker offers a broad selection of purpose-built tools that cover every step in ML development, encompassing data preparation, model building, training, deployment, and model management. However, our focus remains on the integration of SAP FedML with Amazon SageMaker.

Now let’s have a look at the process illustration of the Waste Metrics Forecast using Federated Machine Learning onSAP Datasphere and Amazon SageMaker.

The training data, an analytics model within SAP Datasphere about Waste Transactions, is retrieved into Amazon SageMaker via SAP FedML. Subsequently, it undergoes data pre-processing and applies time series forecasting on waste sustainability and cost metrics for the next 12 months, categorized by Facility and Waste Category.

Once the forecasts are complete, they will be written back to the open schema of SAP Datasphere. This data will be accessible to Bob and John for forecasting sustainability metrics and allocating budgets for waste in the upcoming months.

The input training data and output forecast

Next, we are going through the end-to-end process of building a medical waste forecast model with Amazon SageMaker using SAP Federation Machine Learning library.

Step-0 Configuration:

We'll start with some initial configurations, such as the creation of a technical data user in the designated space of SAP Datasphere. This user will be granted appropriate read-write privileges, enabling SAP FedML to retrieve and write data seamlessly from and to SAP Datasphere.

Step-1 Data Retrieval from SAP Datasphere

In this step, let’s see how to retrieve the data from SAP Datasphere into Amazon SageMaker via SAP FedML to prepare for the training.

Step-2 Data Preprocessing

The Waste Transaction data has been successfully loaded into Amazon SageMaker as a pandas dataframe. Given the goal of forecasting waste sustainability and financial metrics for the next 12 months, it becomes essential to preprocess the waste transaction data by aggregating the target metrics based on calendar months, facilities, and waste categories.

Step-3 Time Series Forecast

Now, the waste data has been aggregated by calendar month. Next, we’ll apply time series forecast on it to forecast for next 12 months.

Step-4 Check the forecast result

The forecast has been complete, next let’s check it by plotting it.

Step-5 Write back the forecast result to SAP Datasphere

We are happy the forecast result, let’s write them back to SAP Datasphere.

Step-6 Check forecast result within SAP Datasphere

Now the forecast results have been written back to the open schema of SAP Datasphere, let’s see how to use it for data modeling.

Data Modeling based on SAP Datasphere

Well! Through the previous sections, we have seen the vital role of SAP Datasphere in the data ingestion & federation across multiple data sources, as well we have explored how-to leverage Daatsphere, FedML lib & Hyperscaler AI Platform to source medical waste metrics data, build, train and deploy ML models its forecasting.

Below is our solution architecture again, where we will particularly focus on Data Modeling in Datasphere;

Datasphere offers multiple modeling capabilities for everybody which can address different personas from Business Analysts with deep business understanding to tech-savvy Developers and Power users, providing powerful NCLC built-in graphical editors for modeling.

In our use case, we will leverage Datasphere Data Builder since it is the central place for data modeling, whereby you can find various editors to create artifacts in the data layer.

SAP Datasphere helps in the semantic modeling across SAP & non- SAP data, that’s coming in from multiple sources, it is indeed the right unified business data layer which realizes a true implementation of the data fabric architecture, where the data fabric federates various data sources, such as involved medical facilities, their waste disposal orders, federated IoT waste metrics, as well the forecasting waste metrics, into a single virtual layer.

Now let’s explore the right steps to ultimately build the analytical models of the actual & the forecasting medical waste metrics which are federated from multiple data sources representing the medical waste disposal orders along with their IoT based waste metrics as well the ML based waste data predictions.

In step1, we have created two-dimension tables to represent the attributes of the master tables - waste categories & medical facilities.

In addition, we created a fact table to represent the attributes of the transactional table - waste disposal orders along with its relevant IoT federated waste measurements such as CO2 emissions, Weight, location, incurred cost…etc, then on top of our fact table we did build a fact view to project the waste transactions and its sustainability & cost metrics, afterwards we expose it for consumption.

We have also created another fact view, to project the waste metrics forecasting for weights, CO2 emissions and costs. This view is actually based on the Open SQL Schema which was created in our Datasphere space and finally populated to carry the waste metrics forecasting data for the upcoming 12 months as a result of inferencing the waste forecasting ML model (which was previously trained on a historical data of last 5 years), that was indeed mentioned earlier.

In step2, we finally built one analytical semantic model based on the previously created fact view of the waste transactions along with its sustainability & cost metrics, to represent the actual sustainability & cost waste metrics.

We did also build a second analytical semantic model to represent the sustainability & cost forecasting of waste metrics, which are based on the fact view - waste metrics forecasting.

Keep in mind that, the Analytic Model is the foundation to make data ready for data consumption with SAP Analytics Cloud. It offers analytical capabilities & calculations, as well help to answer.

In step3, after the analytic models have been successfully built, the data can be previewed to verify the models quickly & easily in Datasphere before having it displayed & visualized later by the consumption tools & Apps.

In our use case scenario, the consumption will be through SAP Build app & SAP Analytics Cloud as well!

In Step4: we built the stories in SAP Analytics Cloud based on the analytic models as described above. This part will be explained in details in the following section. Let's watch the full demo here;

Healthcare Waste Metrics Dashboards based on SAP Analytics Cloud

Next, let’s move to the implementation of the analytics part.

Here is our solution architecture again, where we will particularly focus on Data Analytics topic with SAC!

When it comes to our prototype – the two main personas: Bob: Sustainability Expert & John: Financial Analyst, will efficiently meet their business requirements, and so resolve current waste management challenges.

So, let’s see together how the Analytical Stories can visualize the actual financial & sustainability waste metrics, which in return will help Bob & Jon to derive unique insights through comparing historical medical waste metrics vs the actual ones, while predicting for the upcoming 12 months. Therefore, this will empower them to perform the right actions and appropriate planning accordingly.

So now let us focus on the SAP Analytics Cloud part and the significant role it plays in the overall architecture of our solution!

In Step4 - that's final one, whereby the respective artifacts have already been exposed for consumption by the SAP Analytics Cloud.

This is where the SAP Analytics Cloud Stories will be built on top of our analytic models for the Medical Actual Waste Metrics as well the Forecasting data.

In our use case we did create a live data connectivity towards SAP Datasphere from SAP Analytics Cloud in order to retrieve live the waste related metrics data which we will analyze in a demo video later on.

Based on the live connection we just configured, let’s look at this demo for how we built the analytical story of the actual & forecasting waste sustainability metrics to realize how we can help Bob to do his job better!

Finally, Let’s look at the final demo on how we built the analytical story of the actual & forecasting waste financial metrics to realize how we can help John to do his job better!

To conclude, SAP Analytics Cloud will enable healthcare analysts for predicting budgetary & sustainability - related metrics, perform comparative cost performance analysis and derive insights so that they can make informed decisions without doubts & without any data duplication across the engaged multiple systems.

Waste Management Prototype based on SAP Build App

It’s time to pack all of those nice features together in the Business User App that we designed with Build Apps and Cloud Application Programming Model. This is the scope of the implementation for the app:

Ok, so let’s get started with the first leg, it is, binding the Datasphere view containing the smart bins released to disposal to the App itself.

Right, so we get the smart bins listed already, know how do we create Disposal Requests – in the form of S/4HANA Cloud Purchase Orders of type service - so the third-party company can come collect them?

Ok, so the transactional part is covered, let’s move to the analytical part, it is, how to embed the cool analytical stories into our Build App?

Finally, It is worth to hihglight here that in such use case scenarios you can also use the SAP Analytics Cloud Embedded Edition (SACEE).

With the SAP Analytics Cloud, embedded edition, you can build and embed reports, dashboards, and visuals into your business application to make confident decisions. You can explore your business data via live connection between your SAP Analytics Cloud tenant and the remote SAP HANA database on SAP BTP.

It is available as a service on top of BTP under CPEA. This variant is meant for the application developers who would embed SAC and makes the analytics available for the end users within the context of the business application’s UI. From features point of view, it only offers the BI capabilities (no planning, predictive and analytics designer capabilities available with this variant).

While on the other side, SAC Enterprise Edition is the complete license of analytics that supports the 360-degree analytics with full capabilities like BI, planning, predictive and Analytics designer with both live and import data connection.

Wrap-Up

To conclude, the proposed solution is a full stack app based on Build & CAP, which leverages FedML Lib to create an end-to-end automated solution for ML based scenarios.