Consume Machine Learning API in SAPUI5

[zhaoyo]

Introduction

After SAP Leonardo Portfolio was announced on 2017, several SAP Leonardo Machine Learning APIs were published on SAP API Management (SAP API Business Hub).

We can consume those published APIs in despite of we don’t need to know the essential part of machine learning algorithm.

Here is an example to show how to consume “Image Classification API” in SAPUI5 application.




It is possible to pickup a local image and send this image to machine learning API. Then showing image classification result in the application.

Prerequisites

In order to consume APIs on SAP API Hub, you should register your SAP Cloud Platform user account.

You have knowledge of SAP UI5 development.

Reference

There is very detailed tutorial to call ML APIs in a SAPUI5 application. You can direct check this tutorial in the GitHub (link).

Different from Git tutorial, the JS HTML DOM objects are used in my example to display preview of images.

Technical Detail

API Hub

First of all, you need to log into SAP API hub: https://api.sap.com

In the API discover you will see function services on SAP Leonardo Machine Learning:




Go to detail page of “SAP Leonardo Machine Learning - Functional Services”. There are several artifacts available:




In our example, we will utilize “Image Classification API”. Before consuming it in UI5 application, we can test it directly in the API hub. Clicking this API and it will navigate to the detail page:




There is general information listed in the overview section.

Going to the resource section, we can test this API by posting data to API service:




Here you can see the technical definition of API. In the “Parameters” section you can directly select image file from local driver




Then click the button “Try it out!” to test this API:




After clicking “Try it out” button, API returns response in predefined format: pairs with label and its score.

Note: there is size limitation of image file. The image file should not exceed 2M and pixels should be less than one million. Otherwise, API will return error code back.

Another great feature on API hub is that we can generate sample code to trigger this API in different language. This can be simply achieved by clicking link “Generate Code”:




The sample code can be generated in different style depends on usage. Here we are using generated JavaScript code in our application:

SAP UI5 Application

In this example, we are only testing the ML service to classify images. There is no necessary to upload and store images somewhere. So the images will be loaded into the web browser and then pass to ML services.

In above consideration, the HTML DOM Input FileUpload object is used. You can get more detail about FileUpload object here: https://www.w3schools.com/jsref/dom_obj_fileupload.asp

Here are sample code for your reference.

Main.view.xml:

<mvc:View 

	controllerName="xx.test.pictureClassify.controller.Main" 

	xmlns:html="http://www.w3.org/1999/xhtml" 

	xmlns:mvc="sap.ui.core.mvc"

	displayBlock="true" 

	xmlns:l="sap.ui.layout"

	xmlns:u="sap.ui.unified"

	xmlns:core="sap.ui.core"

	xmlns="sap.m">

	<App id="idAppControl">

		<pages>

			<Page title="{i18n>title}">

				<content>

					<l:VerticalLayout>

						<core:HTML 

							id="fileHTML"

							content='&lt;div&gt;&lt;input type=&quot;file&quot; id=&quot;previewImg&quot;&gt;&lt;br&gt;&lt;img id=&quot;image&quot;&gt;&lt;/div&gt;'/>

						<Button

							text="Classify Image"

							press="onClassify"/>

						<Table 

							id="resultTable" 

							items="{

								path: '/results',

								sorter: {

									path: 'score',

									descending: 'true'

								}

							}">

							<headerToolbar>

								<Toolbar>

									<Title text="Image Classification Result"/>

								</Toolbar>

							</headerToolbar>

							<columns>

								<Column>

									<Text text="Label"/>

								</Column>

								<Column>

									<Text text="Score"/>

								</Column>

							</columns>

							<items>

								<ColumnListItem>

									<cells>

										<ObjectIdentifier title="{label}"/>

										<Text text="{score}"/>

									</cells>

								</ColumnListItem>

							</items>

						</Table>

					</l:VerticalLayout>

				</content>

			</Page>

		</pages>

	</App>

</mvc:View>

Main.controller.js

sap.ui.define([

	"sap/ui/core/mvc/Controller"

], function(Controller) {

	"use strict";



	return Controller.extend("xxx.test.pictureClassify.controller.Main", {

		onInit: function(){

			this.oFileUpload = this.getView().byId("fileUploader");

			this.oImage = this.getView().byId("imageId");

		},

		

		onAfterRendering: function(){

			//console.log("onAfterRending is called");

			// get DOM element of file input

			var fileInput = $("#previewImg")[0];		//var fileInput = document.getElementById("previewImg");

			// attach event of onchange

			fileInput.addEventListener("change", this.onChange);

		},

		

		onBeforeRendering: function(){

			//console.log("onBeforeRending is called");

		},

		

		// handle the image file change event

		onChange: function(oEvent){

			//console.log("onChange is called");

			var input = oEvent.target;

			var reader = new FileReader();

			

			// get file content

			reader.onload = function(){

				var dataUrl = reader.result;

				// set image area

				var image = $("#image")[0];			//var image =  document.getElementById("image");

				image.src = dataUrl;

			};

			// read content

			reader.readAsDataURL(input.files[0]);

		},

		

		// call ML API to classify the image

		onClassify: function(oEvent){

			//get file content

			var image = $("#image")[0];

			

			//get input file information

			var input = $("#previewImg")[0];

			

			//prepare file for api call

			var data = new FormData();

			data.append("files", input.files[0], input.files[0].name);

			

			var xhr = new XMLHttpRequest();

			xhr.withCredentials = false;

			

			var that = this;

			

			xhr.addEventListener("readystatechange", function () {

			  if (this.readyState === this.DONE) {

			  	//release busy indicator

			    that.getView().byId("resultTable").setBusy(false);

			    // convert response text to json format

			    var json = JSON.parse(this.response);

			    var oModel = new sap.ui.model.json.JSONModel();

			    oModel.setData(json.predictions[0]);

			    that.getView().setModel(oModel);

			  }

			});

			

			//setting request method

			xhr.open("POST", "/ml/imageclassifier/inference_sync", false);

			

			

			//adding request headers

			//xhr.setRequestHeader("Content-Type", "multipart/form-data");	//this will lead to 400 Bad Request error

			xhr.setRequestHeader("Accept", "application/json");

			//API Key for API Sandbox

			xhr.setRequestHeader("APIKey", "AYhSnclqxJyRJxbb9oNBxHGECp21uyFa");

			

			//set busy indicator before send request to ML API

			this.getView().byId("resultTable").setBusy(true);

			

			//sending request

			$.sap.delayedCall(2000, this, function(){

				xhr.send(data);

			});

			

		}

	});

});

Thanks for reading this!