SAPUI5 Image Background Removal

In this tutorial, we will build a simple SAPUI5 image background removal. User will take a photo of image they wish to remove the background and the result will be displayed after processing is done. If you have been using https://www.remove.bg/, we will be doing the same thing but with the minimum version.

There are three modules we are going build:

• An SAPUI5 App
  A simple UI5 app as a front-end.
• Background Removal Server
  A Python Flask server to remove the background.
• Image Processing Server
  A NodeJS server as an interface between UI5 app and Background Removal Server

An SAPUI5 App

For UI5 app, I have modified the app from this blog, https://blogs.sap.com/2018/03/06/capturing-camera-photos-in-a-ui5-application-without-containers/ and added a SocketIO.

Change the IMAGE_PROCESSING_SRV with your Image Processing Server URL.

sap.ui.define([
    "sap/ui/core/mvc/Controller",
    "sap/ui/model/json/JSONModel",
    "sap/m/MessageToast"
], function(Controller, JSONModel, MessageToast) {
    jQuery.sap.require("Camera.lib.socketio");
    "use strict";
    var socket;
	var oModel;

	var IMAGE_PROCESSING_SRV = 'http://localhost:3000';
    socket = io(IMAGE_PROCESSING_SRV, { transport : ['websocket'] });

    socket.on('req', function(data) {
        sap.m.MessageToast.show("The image is ready now");
		console.log(data.base64);

		var aPhotosres = [];
		aPhotosres.push({
			src: data.base64
        });
		oModel.setProperty("/photosres", aPhotosres);
        oModel.refresh(true);
    });


    return Controller.extend("Camera.controller.Home", {
        onInit: function() {
            this.getView().setModel(new JSONModel({
                photos: []
            }));
        },

        /////////////////////////////////////////////
        //  EVENTS
        /////////////////////////////////////////////
        onSnapshot: function(oEvent) {
            // The image is inside oEvent, on the image parameter,
            // let's grab it.
            oModel = this.getView().getModel();
            var aPhotos = oModel.getProperty("/photos");

            socket.emit('client_data', {
                'base64': oEvent.getParameter("image")
            });

            aPhotos.push({
                src: oEvent.getParameter("image")
            });
            oModel.setProperty("/photos", aPhotos);
            oModel.refresh(true);
        },

        /**
         * Stop the camera when the tab is not visible.
         * @param {object} name
         * @returns {object}
         */
        onTabSelect: function(oEvent) {
            var oTab = oEvent.getParameter("key");
            var oCamera = this.getView().byId("idCamera");
            if (oTab !== "demo") {
                oCamera.stopCamera();
            } else {
                oCamera.rerender();
            }
        }
    });
});

Background Removal Server

Clone the source code from https://github.com/ferrygun/SAPUI5BackgroundRemoval.

Copy u2net.pth from https://drive.google.com/drive/folders/1759QAWIBkfYrB6gLlkLLGrp7qJkOo5UC

Put that file in \models\u2net\

Copy fba_matting.pth from https://drive.google.com/drive/folders/1cwwdjNb3pZEWkeActUvlxnL6fOBQwECC

And put that file in \models\fba_matting

Create the virtual environment with Python and Install the required libraries:

virtualenv -p python3.7 venv
source venv/bin/activate
pip install -r requirements.txt

Run the server:

python backgroundremoval.py

Test the server: http://localhost:8082/, check if you see the message “Background Image Removal”.

Image Processing Server

Install the required libraries:

npm install fs http express request image-to-base64 body-parser cors socket.io

Run the server:

node imageprocessing.js URL_OF_BACKGROUND_REMOVAL_SERVER

example:
node imageprocessing.js http://localhost:8082

Test the server: http://localhost:3000/, check if you see the message “Image Processing Server”

Run the App

Open the UI5 app http://localhost/openui5-camera-master/test/demo/index.html and select Demo tab. Click on video capture.

Wait until you get the message “The image is ready now“:

Select Result tab to check the output:

Demo Video

I am running the app with CPU on my notebook. If you have GPU, it will be much faster.

References:

• https://blogs.sap.com/2018/03/06/capturing-camera-photos-in-a-ui5-application-without-containers/
• https://github.com/OPHoperHPO/image-background-remove-tool
• https://github.com/ferrygun/SAPUI5BackgroundRemoval