Skip to Content
Technical Articles
Author's profile photo Vinay Bhatt

Visual Inspection Prototype : SAP AppGyver & Kyma Runtime

 

 

 

Introduction

SAP AppGyver provides a great opportunity to quickly build mobile & desktop applications which can be used in field supplementing the business processes provided by core ERP/Transactional applications. In this blog post you will learn how I developed a prototype using AppGyver to count similar objects in a warehouse.(counting the packaging boxes in this case).

Project uses an open source object detection model (YoloV3 tiny Cpu), custom trained in Google Colab (GPU) and orchestrated using SAP BTP Kyma runtime & SAP AppGyver.

What are we learning here ?

  • Quickly put your ML models in action using SAP AppGyver & Kyma runtime.
  • SAP Kyma rumtime is a fully managed Kubernetes runtime which lets you deploy your micro apps within minutes. Learn how to deploy in Kyma
  • Utilize the Kyma ML deployment API to perform a http request in AppGyver.
  • Take an Image from AppGyver App, compress it & push it to an Object Store.
  • Pass the Image Object ID (or URL) to the deployment API as POST request and then the Python function in Kyma runs the model and writes back a new image with bounded boxes in the object store.
  • Make a GET call to read back the predicted Image and the object count.

Architecture

 

Project Code

GITHUB Here you would find the app which can be deployed in Kyma.

 

App Demo Video

Detection takes a while as I have only used a small model on CPU (need GPU for quicker inference) for a quick prototype ,video shows the actual duration without any edits.

AppGyver Low code development 

You easily sign up for AppGyver trial read here 

I’m sharing the most essential parts of this Project as following :

  1. Take an Image , resize it (I used factor of “1” which is the smallest size possible) & Store its properties (name,CreatedAt, local app file path, size) to a page variable of type “Object” ( a complex data type in AppGyver allowing keys value pairs of different types).

         A little trick here is to to add an additional key : UploadURL. This enables us to generate a                 unique object store upload path/ID for the image we take from the App and we can then                     directly use this object to perform a POST request to save the image to the object store.

         Below you can notice the image compression :

      Below you see the Page variable to store the image properties:

An additional key storing the uploadURL. The structure of the variable is defined seperately in the page variable section.

  I’m storing the images in Google Firebase store and here you see that the image file name is coming from the output of the “Resize/compress image” operator

 

"https://firebasestorage.googleapis.com/v0/b/<project-name>/o/"+outputs["Resize/compress image"].imageFile.name

        2. Upload Image : Use a HTTP POST request to save the image, since our page variable is already defined with uploadURL we simply point to the object under “Files to upload”

 

3. Once the Image is uploaded call Kyma deployed ML model function using HTTP POST request.

Notice the request body contains the unique path of the image in the object store.

{ filename: pageVars.PAGE_FileListObj.name,  url: pageVars.PAGE_FileListObj.uploadUrl+"?alt=media&token=<auth token>"}

         4. As soon as the POST call returns (takes a while I have only used CPU node), read the image uploaded by the python function (refer the github code app/main.py), this image is overwritten after every prediction ,output file name is always predictions.jpg.

 

After this you can easily preview and test your app using SAP AppGyver preview App available in Apple playstore.

 

Conclusion

I would mention here that i have tried to showcase an open source object detection model (Yolo) here as an example, look in the link for more information on it, there maybe several other suitable ways of acheiving the results of the scenario discussed in this blog post.

I had performed custom training on the images of a few boxes so as to detect and count them. This is just an attempt to showcase possible scenarios to use your prebuilt ML models in the SAP Business Technology Platform.

Please reach out to me in comments if you would like to collaborate, I’m happy to help out in anyway I can and also willing to learn from everyone.

Thank you!

 

#SAPLowCodeNoCodeChallenge

Assigned Tags

      6 Comments
      You must be Logged on to comment or reply to a post.
      Author's profile photo Thomas Jung
      Thomas Jung

      There is the AppGyver No-Code Challenge going on right now. Have you considered entering this great sample/showcase in that challenge? This would make an excellent addition to the contest. No-Code Challenge: 0% Code, 100% App (and hoodies!) | SAP Blogs

      Author's profile photo Vinay Bhatt
      Vinay Bhatt
      Blog Post Author

      Thanks Thomas, I will definetly add it. I will also try another version by 13th June which I can deploy to the com domain using a public API.

      Author's profile photo Basar Ozgur Kahraman
      Basar Ozgur Kahraman

      Hello Vinay,

      That is so amazing to see such kind of a content on SCN, thank you so much.

      I would be interested in solution rather than technology. Attached video is so helpful to understand the accuracy and performance. As you said that is a prototype but is it possible to improve it a little bit? For example is it possible to use YOLOR as algorithm. They claim to be better in both performance and accuracy. I found also some more image datasets on roboflow, medium, github . Maybe you can add them onto your training dataset to improve the object count accuracy.

      I'm looking forward to seeing the results 🙂

      Best Regards

      Özgür

      Author's profile photo Vinay Bhatt
      Vinay Bhatt
      Blog Post Author

      Thanks Özgür for the links to the dataset and the YOLOR algorithim.
      I will try and update the blog post soon.

       

      Author's profile photo Daniel Wroblewski
      Daniel Wroblewski

      Thanks for the great submission 😀

      Can you add the #SAPLowCodeNoCodeChallenge tag to the blog?

      Author's profile photo Pierre COL
      Pierre COL

      Great use case Vinay Bhatt, thanks!