Technical Articles
SAP Data Intelligence: ML – Bring Your Own Model
Pre-Requisities: Familiarity with SAP Data Intelligence; Modeler and ML-Scenario Manager
We recently came across a case wherein a we wanted to train a Machine Learning model externally and use Data Intelligence for Inference and Consumption. The below steps outlines a ‘Bring your own model’ use-case with SAP Data Intelligence used for hosting and consumption
Train Model Externally
We used Google Cloud Platform to train our model. We created an AUTO-ML Vision for training our image classification model as out lined here.
The usecase trains the model on about 60 Cloud images to classify the clouds as cumulus, cumulonimbus or cirrus
- Upload images to GCP
uploadimages2gcpstorage
2. Navigate to the Vision Dashboard and Create a DataSet with your images.
3. Once your import is done you will be able to see your images
inspectuploadedimages
4. Click on Train button to create your model. Since you want to download and use the trained model externally – choose ‘Edge’ as the deployment option
chooseedgemodel
5. Follow the GCP tutorial link to test your trained model on GCP itself. Once you are satisfied download the model as a container. Note that you will get this option as shown in the screenshot only if you selected the ‘Edge’ option while creating your model in step 4. You can export to a cloud storage and then download it to your local disc.
exportmodelascontainer
6. You can check the exported model by browsing to the bucket. It is downloaded as “saved_model.pb”. Download the pb file to your local machine for consumption with DataIntelligence or on your local desktop
exportedmodel
Consuming Exported Model on your Local Desktop
To consume your exported model on your local machine follow the steps outlined here
Mount the model locally on your docker image using
docker run --rm --name <any_unused_container_name> -p 8501:8501 -v C:\<path>\ML:/tmp/mounted_model/0001 -t gcr.io/cloud-devrel-public-resources/gcloud-container-1.14.0:latest
\<path>\ML = location of the downloaded .pb model file
runninglocallyondocker
Now you are ready to test the exported model. Convert your test image to b64 encoding. I used the base 64 encode app from microsoft for this purpose – can be dowloaded here and save the results as a .json file
Test your model as
curl -X POST -d @resp1.json http://localhost:8501/v1/models/default:predict
output
testinglocally
You can also use postman to test this. Submit your input as a binary file then
Consuming Exported Model using SAP DataIntelligence
DI:2006 supports only exported models in the container form (.pb). It does not yet support tensorflow.js format. The default tensorflow run time is 1.15 and the Tensorflow 2 is supported with the Model Serving operator. Details here
Step 1: Zip and upload your model
Zip your saved_model.pb as saved_model.zip
Login to your DI instance’s Launchpad and click on ‘System Management’
Navigate to the Files tab. Create a new folder called ‘mlup’ under the Files folder and upload the saved_model.zip file in this folder
uploadmodeltoDI
Step2: Create a ML Scenario and Register the Model as an Artifact in SAP DataIntelligence
Login to your SAP DI instance’s Launchpad and click on ML Scenario Manager
Create a new scenario. We are naming it ‘gcpconsume’
Create a New Notebook with the kernel as ‘python3’. Enter the following code
import os
targetdir = '/vrep/mlup'
files = os.listdir(targetdir)
print(files)
This should output ‘saved_model.zip’
Now register saved_model.zip with the following code
import sapdi
from sapdi.artifact.artifact import Artifact, ArtifactKind, ArtifactFileType
zip_file_path = "/vrep/mlup/saved_model.zip"
artifact = sapdi.create_artifact(
file_type=ArtifactFileType.FILE,
artifact_kind=ArtifactKind.MODEL,
description="Cloud Model pb",
artifact_name="cloudpb",
file_name=os.path.basename('/vrep/mlup/saved_model.zip'),
upload_content=zip_file_path
)
print('Model artifact id {}, file {} registered successfully at {} \n'.format(artifact.artifact_id, zip_file_path,artifact.get_uri()))
output:
Model artifact id b3c65b93-c010-487e-bdc6-243884f110a6, file /vrep/mlup/saved_model.zip registered successfully
at dh-dl://DI_DATA_LAKE/shared/ml/artifacts/notebooks/exportmodel.ipynb/0d66f14c-c5bc-4c9e-aeed-d0389d1d649f/saved_model.zip
You should also see the model registered in your ML SCENARIO
Step 3: Create an Inference Pipeline with the template ‘Tensorflow Serving Pipeline’
Tensorflow Consumer pipeline
On clicking ‘Create’ the modeler application opens. Do not change anything. Make a note of the the various configuration parameters for the Model Serving Operator
Pipeline
Go back to the ML Scenario Manager.
select the pipeline and click Deploy
Enter the parameters in Step 4 as shown in the screenshot
ModelRuntimeId : tf-1.14 or the default of tf-1.1.5
Artifact Model: Select the model from the list
pipeline parameters
Click Save. Navigate back to the Modeler and monitor the pipeline Run.
Open the Wiretap once the pipleine shows running. The wiretap should show output as shown in the screenshot
wiretap
Step4: Consuming the hosted model
Navigate back to the ML Scenario Manager and grab the Deployment URL
DeploymentURL
Now open up Postman
Create a POST request with the URL as is. Nothing needs to be prefixed
Use the Basic Authorization and provide your
Username as Instance\<UID>
Password
Headers
key : X-Requested-With Value: Fetch
The image to be tested should be passed with base64 encoding. Convert this as in the earlier step and pass it to postman in RAW – JSON format
postman_DI
If you Navigate back to the Wiretap UI – you can see the time taken for the inference request
Time to Infer
Thank you Asim for this informative article & detailing the steps ... very helpful.
good one Asim ! Also with changes in DI , the validation of rest apis for image recognition needs to be an encoded image input which is corect. But couldnt validate it via postman . We keep getting the utf-8 error.