Technology Blogs by SAP
Learn how to extend and personalize SAP applications. Follow the SAP technology blog for insights into SAP BTP, ABAP, SAP Analytics Cloud, SAP HANA, and more.
cancel
Turn on suggestions
Auto-suggest helps you quickly narrow down your search results by suggesting possible matches as you type.
Showing results for 
Search instead for 
Did you mean: 

Embedding Intelligence and Sustainability into Custom Applications on SAP BTP

YatseaLi
Product and Topic Expert
Product and Topic Expert
‎07-14-2022 11:05 AM
This is the 5th blog post of the blog series about Building Intelligent and Sustainability Scenarios on SAP BTP with AI and Planning by gianluigi.bagnoli, yatsea.li, amagnani, cesare.calabria, stuart.clarke2, dayanand.karalkar2 and jacobtan, which showcases our SAP Partners how to create industry cloud solutions for end-to-end industry-specific business processes that embeds intelligence and sustainability.

Previously, in the first blog post of this series, my colleague gianluigi.bagnoli has explained you the storyline about a traditional Milan-based Light Guide Plates (LGP) manufacturer BAGNOLI & CO was transformed into a sustainable smart factory with the help of SAP AI Core, SAP AI Launchpad and SAP Analytics Cloud for Planning. Using Artificial Intelligence deep learning for computer vision based automatic defect detection, and condition (sound) monitoring based predictive maintenance in production lines, the company has improved the efficiency of production and energy, and reduced waste.

In the last blog posts written by  cesare.calabria and amagnani, we have learnt about the end-to-end MLops with SAP AI Core, and how to build your own model with TensorFlow about product image segmentation for auto. defect detection and equipment sound anomaly classification for predictive maintenance with SAP AI Core and SAP AI Launchpad.

Overview


Now, in this blog post, let us put the pieces together into a custom app named Sustainable Smart Factory App, which can:

  • Monitor the plant daily operation in real-time about plant status, production yield, defective rate, equipment conditions, as well as a new sustainability KPIs (Key Performance Indicators) about energy consumption and calculated CO2 emission.

  • Ingest and score the real-time IoT (Internet of Things) streaming data against the AI Models

    • Image Segmentation model on LGP product images streaming from camera for automated defect detection

    • Sound Anomaly Classification model on equipment audio data from microphones.



  • Trigger a proactive maintenance order in SAP S/4HANA Cloud once the occurrences of a certain type of anomalies detected exceed the threshold within a period, which extends the Maintenance Management of SAP S/4HANA Cloud with Predictive Maintenance.



Solution Architecture of Sustainable Smart Factory Solution



Introduction to the Sustainable Smart Factory App


Now, let us have a look at the Sustainable Smart Factory App from a functional perspective, which consists of the following four modules as listed below.


User interface of Sustainable Smart Factory App




  1. My Home (Target Persona: Plant Manager)





    • Tile to navigate to Plant 360 Analytics story in SAP Analytics Cloud.

    • Tile to navigate to Maintenance Order & Sustainability Planning story in SAP Analytics Cloud.





Plant 360 Analytics story in SAP Analytics Cloud




  1. Auto. Defect Detection (Target Persona: Quality Inspector or Quality Manager)





    • Quality Record via Computer Vision: A product quality record with its image and quality classification predicted by AI Model.

    • Manage Defective Product Price: Define the pricing rule for the defective product based on the size of the defective area.





Auto. Defect Detection




  1. Plant Master Data (Target Persona: Plant Manager)





    • Manage Equipment: Equipment master data maintained in the Sustainable Smart Factory App, which is synchronized as Technical Object in SAP S/4HANA Cloud, including general data such as equipment id, name, description, class etc., organization data such as company code, plant id, function location etc.

    • Configure Plant Status: Configuration of plant status, such as Breakdown, Fault, Maintenance, and Normal etc.

    • Configure of Anomaly Types: An Anomaly type is mapped into an Anomaly Class in the AI model, such as Slow Sound or Damage Noise.





  1. Predictive Maintenance (Target Persona: Plant Manager or Maintenance Technician/Manager)





    • Plant Condition Visualization: Real-time Monitoring of plant operation namely plant condition explained below, as well as Visualization of Plant Floor Plan about equipment conditions.

    • Manage Plant Condition: A plant condition is a historical snapshot of the plant condition including production yield, defective product number, energy consumption and its equipment conditions within a time period, such as a morning shift of today starting from 0:00 am to 7:59 am.

    • Manage Equipment Condition: Equipment Condition record within time period of a shift, including a list of detected anomalies associated to the equipment. Trigger the creation Maintenance order in SAP S/4HANA Cloud based on rule configuration.

    • Manage Equipment Anomalies: When an anomaly is detected on an equipment, an Anomaly record will be created including equipment, detected At, sound data, and sound anomaly classification by inferencing the AI model etc.





Sound-based Predictive Maintenance


Now, you may be interested on how we built the Sustainable Smart Factory App. Let us have a look at its technical details.

Technical Details of Sustainable Smart Factory App


High-level Solution Architecture


The high-level solution architecture of the Sustainable Smart Factory App as below.


Now let us zoom into the architecture of Bagnoli&Co’s Sustainable Smart Factory App, which is built on top of SAP Cloud Application Programming model (CAP) with

  • SAP Fiori Element as its main UI technology

  • NodeJS as its application and service layer

  • SAP HANA Cloud as its database for persistence.


The app glues all the pieces of Bagnoli&Co’s Sustainable Smart Factory into an end-to-end solution to achieve its sustainable goals.

As you have already seen in screenshot of the app above, it provides an user interface as a  SAP Fiori app for the end users such as Quality Inspector, Quality Manager, Plant Manager, Maintenance Manager, Maintenance Technician etc.

It inferences the AI Models with real-time IoT data ingestion for Defect Detection and Sound Anomaly Detection through REST API deployed in SAP AI Core, which automates the quality inspection, and eliminates the human errors.

Furthermore, it extends the Maintenance Management of SAP S/4HANA Cloud side by side with sound-based Predictive Management by rule-based trigger of proactive maintenance order in SAP S/4HANA Cloud via SAP Cloud SDK when the occurrences of anomaly detected on an equipment exceed the configured threshold, which can minimize the disruption of production line, and maximize the lifespan and efficiency of the equipment, and reduce the maintenance cost.

And, all the API integrations above are through the destination and xsuaa service in SAP BTP for security reasons.

With data persistence in SAP HANA Cloud, some calculation views are custom developed for plant-condition and equipment-insight queries, which provide live connection to SAP Analytics Cloud for a Plant 360 Analytics story in real-time.

 

Detail Design


#1: Auto. Defect Detection


Data Model (sqi-data-model.cds)

  • CVQualityRecords: Quality Record by Computer Vision

  • DefectiveProductPrices: Pricing rules of defective products based on the defective area percentage range.



Data Model of Defect Detection


Code snippets

  1. Connect to SAP BTP destination service, and authenticate to inference API of AI Models in SAP AI Core – getDestination(dest)
    https://github.com/SAP-samples/btp-ai-sustainability-bootcamp/blob/main/src/sustainable-smart-factor...

  2. Inference product image segmentation AI Model for Auto. Defect Detection - inferenceImageCV()
    https://github.com/SAP-samples/btp-ai-sustainability-bootcamp/blob/main/src/sustainable-smart-factor...



 

#2: Predictive Maintenance


Data Model (pdm-data-model.cds)

  • Equipments: Equipment Master Data synchronized as technical objects in SAP S/4HANA Cloud

  • AnomalyType: An anomaly type is mapped as a sound anomaly class in AI model.

  • PlantConditions: A historical record about the plant daily operation by shift within a time period, such as plant status, production yield, number of defective products and energy consumption, and its associated EquipmentConditions records within the same time period.

  • EquipmentConditions: A historical record about the condition of the equipment within a time period, such as the associated equipment, equipment status, the detected anomalies by the AI model. Creation of proactive maintenance order will be triggered in SAP S/4HANA Cloud if the occurrences of the detected anomalies in the time period exceed the threshold defined in the anomaly type. The associated maintenance order number and cost will also be recorded in the equipment condition record if any.

  • Anomalies: An anomaly is detected by the AI Model on about its associated equipment, including when it is detected, what is source type such as sound, image temperature, vibration etc, which can be used for any type of anomaly detected by different sensors. The raw value and raw measure unit depend on the source type as, for example, if the source type is sound, then raw value is the path to sound clip in cloud storage; if the source type is temperature, the raw value could the decimal string about the temperature captured by the sensor, and the raw measure unit could be Celsius to Fahrenheit.



Data Model of Predictive Maintenance


Code snippets

  1. Inference sound anomaly classification AI model for Predictive Maintenance: inferenceSoundAnomaly()
    https://github.com/SAP-samples/btp-ai-sustainability-bootcamp/blob/main/src/sustainable-smart-factor...

  2. Trigger of proactive maintenance order creation in SAP S/4HANA Cloud based on rule: createMO()
    https://github.com/SAP-samples/btp-ai-sustainability-bootcamp/blob/main/src/sustainable-smart-factor... 



#3: Plant 360 Analytics


To enable real-time plant 360 analytics, the following calculation views have been created.


Deployment


Please follow this manual to deploy the Sustainable Smart Factory App (sample app).

More materials


If you would like to learn more about this topic, please check out materials below

Summary


Throughout this sample, we have seen how an SAP partner can embed the AI models and sustainability dimension data (energy consumption and CO2 emission etc.) into end-to-end cloud solution on SAP BTP for Bagnoli&Co., which automates defect defection and employs sound-based predictive maintenance. As a result, Bagnoli &Co has improved the efficiency of production and the safety of workplace and reduced the waste.

What’s next in this blog series


In the next blog of this series, we will see how to perform an end-to-end planning for maintenance cost and sustainability for Bagnoli&Co with SAP Analytics Cloud for Planning. As a wrap up, here are all the episodes of our blog series:
2 Comments
Labels in this area
Popular Blog Posts