SAP Leonardo Machine Learning API with a UI5 Application

[Former Member]

SAP Leonardo Machine Learning Foundation, on SAP Cloud Platform, provides advanced machine learning capabilities in the form of easily consumable intelligent functional services which can be integrated into an enterprise application without worrying about the complexity and the framework behind these microservices.

Leonardo ML foundation portfolio, has a wide range of functional services which analyzes and draw  meanings out of the image, videos, languages etc, and can be quickly consumed to build an application having ML capabilities.

In this blog, will try to show how we can integrate ‘Product Image Classification’ ML API into a custom-built SAP UI5 application.

Scenario - Upload image into a UI5 application and then using SAP Leonardo ML api get the classifications of the submitted image and eventually use the classification which has the highest probability to load an online shopping web page having filtered products (according to the image).

So let's get started...

 

1) Create a UI5 application in a SAP WEBIDE



 

2) Update View file according to the below code.

Quick explanation... 'FileUploader' UI5 element will generate an input field along with a browse function to select and hold the file. Along with that a 'Table' control has been added to display the product classifications result on the screen.

<mvc:View controllerName="com.sap.mlProductClassificationML.controller.View" xmlns:l="sap.ui.layout" xmlns:mvc="sap.ui.core.mvc"

	xmlns:semantic="sap.m.semantic" xmlns:smart="sap.ui.comp.smartfield" xmlns="sap.m" xmlns:u="sap.ui.unified" xmlns:f="sap.ui.layout.form"

	xmlns:core="sap.ui.core" class="viewPadding">

	<App>

		<pages>

			<Page id="mainPageId" titleLevel="Auto" title="{i18n>createTitle}">

				<content>

					<VBox class="sapUiSmallMargin">

						<f:SimpleForm id="newEntitySimpleForm" minWidth="1024" maxContainerCols="2" editable="true" layout="ResponsiveGridLayout" labelSpanL="3"

							labelSpanM="3" emptySpanL="4" emptySpanM="4" columnsL="1" columnsM="1" class="editableForm">

							<f:content>

								<Label id="fileNameId" text="Upload Image"/>

								<u:FileUploader id="fileUploader" name="FILENAME" placeholder="Upload image to search a product" change="processFile"

									sameFilenameAllowed="false" liveChange="onSelectInput" width="350px"/>

							</f:content>

						</f:SimpleForm>

					</VBox>

					<FlexBox width="70%" id="__box2" justifyContent="Center">

						<items>

							<Button visible="true" text="Route to Shopping" enabled="true" press="handleUploadPress" width="200px"/>

						</items>

					</FlexBox>

				</content>

				<Table id="idProductsTable" inset="false" items="{ path: '/results', sorter: { path: 'score', descending: 'true' } }">

					<headerToolbar>

						<Toolbar>

							<Title text="Image Classification Probablities" level="H2"/>

						</Toolbar>

					</headerToolbar>

					<columns>

						<Column >

							<Text text="Label"/>

						</Column>

						<Column >

							<Text text="Score"/>

						</Column>

					</columns>

					<items>

						<ColumnListItem>

							<cells>

								<ObjectIdentifier title="{label}"/>

								<Text text="{score}"/>

							</cells>

						</ColumnListItem>

					</items>

				</Table>

			</Page>

		</pages>

	</App>

</mvc:View>

Note - Controller file will be updated later.

 

3) Take Product Image Classification 'key' from SAP API Hub (Cloud Platform)

SAP API Hub -> Browse -> Search "SAP SAP Leonardo Machine Learning Functional Services" -> Artifacts -> Product Image Classification API



 

4) Now, update the controller code which will read the uploaded file and push the same to ML API.

Little explanation...FormData object has been used which will append file to the form interface using key/value pairs and then can be easily submitted to the api using the XMLHttpRequest.send() post request.

var data = new FormData();

//add a 'files' key inside a FormData object

data.append('files', file, filename);​

Add 'Handle Upload Press' method logic in the controller file using below code-

	handleUploadPress: function(oEvent) {

			var oFileUploader = this.getView().byId("fileUploader");

			if (!oFileUploader.getValue()) {

				MessageToast.show("Choose a file first");

				return;

			}

			var that = this;

			var f = document.querySelector('input[type="file"]').files[0];

			//Create form object and append file to the same

			var data = new FormData();

			data.append('files', document.getElementById("__xmlview0--fileUploader-fu").files[0], document.getElementById(

				"__xmlview0--fileUploader-fu").files[0].name);

			var xhr = new XMLHttpRequest();



			xhr.addEventListener("readystatechange", function() {

				if (this.readyState === 4) {

					var vjson = JSON.parse(this.responseText);

					//set the model for the UI5 table to populate product classification results

					var oModel = new sap.ui.model.json.JSONModel();

					oModel.setData(vjson.predictions[0]);

					that.getView().setModel(oModel);



					var label = vjson.predictions[0].results[0].label;

					var score = vjson.predictions[0].results[0].score;

				}

			});



			xhr.open("POST", "https://sandbox.api.sap.com/ml/prodimgclassifier/inference_sync");

			xhr.setRequestHeader("apikey", "xxxxxxxxxxxxxxxxxxx");

			xhr.setRequestHeader("accept", "application/json");

			xhr.send(data);

		}

 

5) Now, we are done with the coding stuff so let's test UI5 application by uploading an image, in my case it's a image of a 'Smart TV'.

 





'Route to Shopping' button will make a call to 'Product Image Classification' ML api and returns the image classifications.



 

6) As a last part of this scenario, have added the functionality to route the screen to open an online shopping web page which will display the list of the filtered products according to the uploaded image - ML API calculates and returns a list of classifications along with the probabilities for a given image, and the highest probability has been used as the searching criteria to redirect to a web page.

var label = vjson.predictions[0].results[0].label;

var score = vjson.predictions[0].results[0].score;

sap.m.URLHelper.redirect("http://www.site.com/search?Ntt="+ vjson.predictions[0].results[0].label +"&N=", true);