SAPUI5 Image Background Removal

[ferrygun18]

In this tutorial, we will build a simple SAPUI5 image background removal. User will take a photo of image they wish to remove the background and the result will be displayed after processing is done. If you have been using https://www.remove.bg/, we will be doing the same thing but with the minimum version.

There are three modules we are going build:

• An SAPUI5 App
  A simple UI5 app as a front-end.


• Background Removal Server
  A Python Flask server to remove the background.


• Image Processing Server
  A NodeJS server as an interface between UI5 app and Background Removal Server

 

An SAPUI5 App

For UI5 app, I have modified the app from this blog, https://blogs.sap.com/2018/03/06/capturing-camera-photos-in-a-ui5-application-without-containers/ and added a SocketIO.

Change the IMAGE_PROCESSING_SRV with your Image Processing Server URL.

sap.ui.define([

    "sap/ui/core/mvc/Controller",

    "sap/ui/model/json/JSONModel",

    "sap/m/MessageToast"

], function(Controller, JSONModel, MessageToast) {

    jQuery.sap.require("Camera.lib.socketio");

    "use strict";

    var socket;

	var oModel;



	var IMAGE_PROCESSING_SRV = 'http://localhost:3000';

    socket = io(IMAGE_PROCESSING_SRV, { transport : ['websocket'] });



    socket.on('req', function(data) {

        sap.m.MessageToast.show("The image is ready now");

		console.log(data.base64);



		var aPhotosres = [];

		aPhotosres.push({

			src: data.base64

        });

		oModel.setProperty("/photosres", aPhotosres);

        oModel.refresh(true);

    });





    return Controller.extend("Camera.controller.Home", {

        onInit: function() {

            this.getView().setModel(new JSONModel({

                photos: []

            }));

        },



        /////////////////////////////////////////////

        //  EVENTS

        /////////////////////////////////////////////

        onSnapshot: function(oEvent) {

            // The image is inside oEvent, on the image parameter,

            // let's grab it.

            oModel = this.getView().getModel();

            var aPhotos = oModel.getProperty("/photos");



            socket.emit('client_data', {

                'base64': oEvent.getParameter("image")

            });



            aPhotos.push({

                src: oEvent.getParameter("image")

            });

            oModel.setProperty("/photos", aPhotos);

            oModel.refresh(true);

        },



        /**

         * Stop the camera when the tab is not visible.

         * @param {object} name

         * @returns {object}

         */

        onTabSelect: function(oEvent) {

            var oTab = oEvent.getParameter("key");

            var oCamera = this.getView().byId("idCamera");

            if (oTab !== "demo") {

                oCamera.stopCamera();

            } else {

                oCamera.rerender();

            }

        }

    });

});

 

Background Removal Server

Clone the source code from https://github.com/ferrygun/SAPUI5BackgroundRemoval.

Copy u2net.pth from https://drive.google.com/drive/folders/1759QAWIBkfYrB6gLlkLLGrp7qJkOo5UC

Put that file in \models\u2net\

Copy fba_matting.pth from https://drive.google.com/drive/folders/1cwwdjNb3pZEWkeActUvlxnL6fOBQwECC

And put that file in \models\fba_matting

 

Create the virtual environment with Python and Install the required libraries:

virtualenv -p python3.7 venv

source venv/bin/activate

pip install -r requirements.txt

Run the server:

python backgroundremoval.py

Test the server: http://localhost:8082/, check if you see the message "Background Image Removal".

Image Processing Server

Install the required libraries:

npm install fs http express request image-to-base64 body-parser cors socket.io

Run the server:

node imageprocessing.js URL_OF_BACKGROUND_REMOVAL_SERVER



example:

node imageprocessing.js http://localhost:8082

Test the server: http://localhost:3000/, check if you see the message "Image Processing Server"

Run the App

Open the UI5 app http://localhost/openui5-camera-master/test/demo/index.html and select Demo tab. Click on video capture.

Wait until you get the message "The image is ready now":

Select Result tab to check the output:

 

Demo Video

I am running the app with CPU on my notebook. If you have GPU, it will be much faster.

References:

• https://blogs.sap.com/2018/03/06/capturing-camera-photos-in-a-ui5-application-without-containers/


• https://github.com/OPHoperHPO/image-background-remove-tool


• https://github.com/ferrygun/SAPUI5BackgroundRemoval