In this tutorial, I would like to go through on how to build the text scanner (OCR) app with Google ML and SAP Web IDE. The app that we are going to build is the Android Cordova SAPUI5 that has two functions: Take Photo and Pick from Gallery.
Take Photo
To snap a photo and feed into the text recognizer and get the OCR result.
Pick from Gallery
To feed the text recognizer with the available existing images from the phone gallery and get the OCR result.
Once it get recognized, the dialog box with the OCR result will pop-up. We can then use this information to proceed further like getting the information from the SAP back-end through oData, etc.
The final app will look like this:
Let's go through the following steps.
Configure Local HAT installation by following this blog: Check my previous blog on how to install SAP HAT on your machine: https://blogs.sap.com/2019/01/07/build-android-sapui5-ocr-scanner-with-sap-web-ide-and-anyline-ocr-s....
Download all plugins to local custom plugin folder.
Since the Cordova ML Text plugin requires cordova 7.1.0+ , cordova android 6.4.0+, we need to modify the local SAP HAT installation to meet this requirement.
On the SAP HAT Local folder, go to ./setup/server/routes/check.js and modify line 302 to use cordova android 6.4.0:
Now run HAT setup from command line and make sure status is passed.
Once the setup is completed, run the HAT server from command line.
Open SAP Web IDE Full Stack and go to Preferences > Hybrid Application Toolkit. Click Test Connection and make sure the connection is available.
Create a new project from template.
Go to Project Settings > Hybrid App Toolkit > App Configuration
Provide the app name Scanner and app id com.demo.scanner and select Android since we are going to build the android app.
Under Cordova plugin, select camera.
Under Custom plugin, select Google Firebase and Mltext and click Save.
Deploy the app. Right click on the app that we created and select Deploy > Prepare Hybrid Project.
Preparation in progress. It is also installing the custom plugins.
Once it is done, go to C:\Users\<user>\SAPHybrid folder. The project Scanner is created there.
The created cordova project is still using version 6.2.3 and it will not meet the Cordova ML Text plugin (6.4.0). Let's update it to 6.4.0.
Go to C:\Users\<user>\SAPHybrid\Scanner\hybrid\platforms\android and run this command:
I checked C:\Users\<user>\SAPHybrid\Scanner\hybrid\platforms\android\res\xml\config.xml and it looks like the Cordova ML Text plugin was not added successfully.
Let's add it manually then, run this command:
Now we got the plugin installed:
We also need to remove the cordova-plugin-compat otherwise we will have an error during build process. Refer to this https://stackoverflow.com/questions/46562289/multiple-dex-files-define-lorg-apache-cordova-buildhelp...
And this would be the final plugin list:
Go to Firebase console https://console.firebase.google.com and create a new project.
Go to Project settings and click Add Firebase to your Android App.
Add android package name com.demo.scanner and click Register app.
Download google-services.json.
And place it under C:\Users\<user>\SAPHybrid\Scanner\hybrid\platforms\android
Open Android Studio and import the project C:\Users\<user>\SAPHybrid\Scanner\hybrid\platforms\android
Update the build.gradle: Android and CordovaLib project
Update build.gradle: Android:
Update build.gradle: CordovaLib:
And now you can build the app without error.
Take a look at the function below from the controller from the asset folder.
mltext.getText was called to recognize the text in the image and will get the result from recognizedText.
Complete code is here.
And finally that's all that I would like to cover.
Take Photo
To snap a photo and feed into the text recognizer and get the OCR result.
Pick from Gallery
To feed the text recognizer with the available existing images from the phone gallery and get the OCR result.
Once it get recognized, the dialog box with the OCR result will pop-up. We can then use this information to proceed further like getting the information from the SAP back-end through oData, etc.
The final app will look like this:
Let's go through the following steps.
Configure Local HAT
Configure Local HAT installation by following this blog: Check my previous blog on how to install SAP HAT on your machine: https://blogs.sap.com/2019/01/07/build-android-sapui5-ocr-scanner-with-sap-web-ide-and-anyline-ocr-s....
Download Plugins
Download all plugins to local custom plugin folder.
- Cordova ML Text Plugin
https://bitbucket.org/bhivedevs/cordova-plugin-ml-text.git. - Cordova Plugin Firebase
https://www.npmjs.com/package/cordova-plugin-firebase - Cordova Plugin Camera
We will configure in Web IDE
Update Local HAT Installation
Since the Cordova ML Text plugin requires cordova 7.1.0+ , cordova android 6.4.0+, we need to modify the local SAP HAT installation to meet this requirement.
On the SAP HAT Local folder, go to ./setup/server/routes/check.js and modify line 302 to use cordova android 6.4.0:
cmd: 'cordova platform add ' + platform + (platform==="android"?"@6.4.0":""), // for android sdk tools > 26.0.1Run HAT
Now run HAT setup from command line and make sure status is passed.
Once the setup is completed, run the HAT server from command line.
Configure SAP Web IDE
Open SAP Web IDE Full Stack and go to Preferences > Hybrid Application Toolkit. Click Test Connection and make sure the connection is available.
Create a new project from template.
Go to Project Settings > Hybrid App Toolkit > App Configuration
Provide the app name Scanner and app id com.demo.scanner and select Android since we are going to build the android app.
Under Cordova plugin, select camera.
Under Custom plugin, select Google Firebase and Mltext and click Save.
Deploy the app. Right click on the app that we created and select Deploy > Prepare Hybrid Project.
Preparation in progress. It is also installing the custom plugins.
Modify the Plugins
Once it is done, go to C:\Users\<user>\SAPHybrid folder. The project Scanner is created there.
The created cordova project is still using version 6.2.3 and it will not meet the Cordova ML Text plugin (6.4.0). Let's update it to 6.4.0.
Go to C:\Users\<user>\SAPHybrid\Scanner\hybrid\platforms\android and run this command:
I checked C:\Users\<user>\SAPHybrid\Scanner\hybrid\platforms\android\res\xml\config.xml and it looks like the Cordova ML Text plugin was not added successfully.
Let's add it manually then, run this command:
cordova plugin add https://bitbucket.org/bhivedevs/cordova-plugin-ml-text.gitNow we got the plugin installed:
We also need to remove the cordova-plugin-compat otherwise we will have an error during build process. Refer to this https://stackoverflow.com/questions/46562289/multiple-dex-files-define-lorg-apache-cordova-buildhelp...
And this would be the final plugin list:
Create the Firebase Config File
Go to Firebase console https://console.firebase.google.com and create a new project.
Go to Project settings and click Add Firebase to your Android App.
Add android package name com.demo.scanner and click Register app.
Download google-services.json.
And place it under C:\Users\<user>\SAPHybrid\Scanner\hybrid\platforms\android
Import to Android Studio
Open Android Studio and import the project C:\Users\<user>\SAPHybrid\Scanner\hybrid\platforms\android
Update the build.gradle: Android and CordovaLib project
Update build.gradle: Android:
Update build.gradle: CordovaLib:
And now you can build the app without error.
Main Program
Take a look at the function below from the controller from the asset folder.
mltext.getText was called to recognize the text in the image and will get the result from recognizedText.
return Controller.extend("GPS.GPS.controller.GPS", {
oView: null,
onInit: function() {
oView = this.getView();
},
onPhotoDataSuccess: function(imageData) {
var myImage = oView.byId("myImage");
myImage.setSrc("data:image/jpeg;base64," + imageData);
mltext.getText(onSuccess, onFail, {
imgType: 4,
imgSrc: imageData
});
// for imgType Use 0,1,2,3 or 4
function onSuccess(recognizedText) {
//var element = document.getElementById('pp');
//element.innerHTML=recognizedText.blocks.blocktext;
//Use above two lines to show recognizedText in html
console.log(recognizedText);
alert(recognizedText.blocks.blocktext);
}
function onFail(message) {
alert('Failed because: ' + message);
}
},
onPhotoURISuccess: function(imageURI) {
var myImage = oView.byId("myImage");
myImage.setSrc(imageURI);
mltext.getText(onSuccess, onFail, {
imgType: 0,
imgSrc: imageURI
});
// for imgType Use 0,1,2,3 or 4
function onSuccess(recognizedText) {
//var element = document.getElementById('pp');
//element.innerHTML=recognizedText.blocks.blocktext;
//Use above two lines to show recognizedText in html
console.log(recognizedText);
alert(recognizedText.blocks.blocktext);
}
function onFail(message) {
alert('Failed because: ' + message);
}
},
onFail: function(message) {
console.log("Failed because: " + message);
},
getPhoto: function() {
var oNav = navigator.camera;
oNav.getPicture(this.onPhotoURISuccess, this.onFail, {
quality: 100,
destinationType: oNav.DestinationType.FILE_URI,
sourceType: oNav.PictureSourceType.PHOTOLIBRARY
});
},
capturePhoto: function() {
var oNav = navigator.camera;
oNav.getPicture(this.onPhotoDataSuccess, this.onFail, {
quality: 100,
destinationType: oNav.DestinationType.DATA_URL
});
}
});Complete code is here.
And finally that's all that I would like to cover.
- SAP Managed Tags:
2 Comments
