Visual Inspection Prototype : SAP AppGyver & Kyma Runtime

[VinayBhatt]

 

 








 

Introduction

SAP AppGyver provides a great opportunity to quickly build mobile & desktop applications which can be used in field supplementing the business processes provided by core ERP/Transactional applications. In this blog post you will learn how I developed a prototype using AppGyver to count similar objects in a warehouse.(counting the packaging boxes in this case).

Project uses an open source object detection model (YoloV3 tiny Cpu), custom trained in Google Colab (GPU) and orchestrated using SAP BTP Kyma runtime & SAP AppGyver.

What are we learning here ?

• Quickly put your ML models in action using SAP AppGyver & Kyma runtime.


• SAP Kyma rumtime is a fully managed Kubernetes runtime which lets you deploy your micro apps within minutes. Learn how to deploy in Kyma


• Utilize the Kyma ML deployment API to perform a http request in AppGyver.


• Take an Image from AppGyver App, compress it & push it to an Object Store.


• Pass the Image Object ID (or URL) to the deployment API as POST request and then the Python function in Kyma runs the model and writes back a new image with bounded boxes in the object store.


• Make a GET call to read back the predicted Image and the object count.

Architecture

 

Project Code

GITHUB Here you would find the app which can be deployed in Kyma.

 

App Demo Video

Detection takes a while as I have only used a small model on CPU (need GPU for quicker inference) for a quick prototype ,video shows the actual duration without any edits.

AppGyver Low code development 

You easily sign up for AppGyver trial read here 

I'm sharing the most essential parts of this Project as following :

1. Take an Image , resize it (I used factor of "1" which is the smallest size possible) & Store its properties (name,CreatedAt, local app file path, size) to a page variable of type "Object" ( a complex data type in AppGyver allowing keys value pairs of different types).

         A little trick here is to to add an additional key : UploadURL. This enables us to generate a                 unique object store upload path/ID for the image we take from the App and we can then                     directly use this object to perform a POST request to save the image to the object store.

         Below you can notice the image compression :

      Below you see the Page variable to store the image properties:

An additional key storing the uploadURL. The structure of the variable is defined seperately in the page variable section.

  I'm storing the images in Google Firebase store and here you see that the image file name is coming from the output of the "Resize/compress image" operator

 

"https://firebasestorage.googleapis.com/v0/b/<project-name>/o/"+outputs["Resize/compress image"].imageFile.name

        2. Upload Image : Use a HTTP POST request to save the image, since our page variable is already defined with uploadURL we simply point to the object under "Files to upload"

 

3. Once the Image is uploaded call Kyma deployed ML model function using HTTP POST request.

Notice the request body contains the unique path of the image in the object store.

{ filename: pageVars.PAGE_FileListObj.name,  url: pageVars.PAGE_FileListObj.uploadUrl+"?alt=media&token=<auth token>"}

         4. As soon as the POST call returns (takes a while I have only used CPU node), read the image uploaded by the python function (refer the github code app/main.py), this image is overwritten after every prediction ,output file name is always predictions.jpg.

 

After this you can easily preview and test your app using SAP AppGyver preview App available in Apple playstore.

 

Learn More with SAP Learning Journey

To see LCNC in action, check out the SAP Low-Code/No-Code Learning Journey - designed to increase low-code/no-code skills and teach citizen developers the basic concepts of software development including SAP AppGyver, and learn how to build mobile apps for free. Check out LCNC and plus much more free learning at SAP Learning site.

Conclusion

I would mention here that i have tried to showcase an open source object detection model (Yolo) here as an example, look in the link for more information on it, there maybe several other suitable ways of acheiving the results of the scenario discussed in this blog post.

I had performed custom training on the images of a few boxes so as to detect and count them. This is just an attempt to showcase possible scenarios to use your prebuilt ML models in the SAP Business Technology Platform.

Please reach out to me in comments if you would like to collaborate, I'm happy to help out in anyway I can and also willing to learn from everyone.

Thank you!