Off-the-shelf Object Detection for Intelligent Enterprise

[YatseaLi]

In the previous blog about Digital Transformation for SMBs – the Intelligent Enterprise, we have seen the Intelligent Enterprise in SMB context, and how SMB customers and partners(SAP Business One and SAP Business ByDesign) could gain the competitive advantage with intelligent solution by leveraging SAP Leonardo. SAP Leonardo Machine Learning Foundation has offered ready-to-use Functional Services and Business Services. In addition, you can bring your own TensorFlow model(a blog by f.lehmann) or your own data for re-training on SAP Leonardo, and provisioning as web service.

My blog series of Object Detection for Intelligent Enterprise:

• Off-the-shelf Object Detection for Intelligent Enterprise(this blog)


• Object Detection with Tensorflow for Intelligent Enterprise


• Object Detection with YOLO for Intelligent Enterprise

In this blog, we will have a look at some other off-the-shelf machine learning framework for building intelligent solution in conjunction with SAP Leonardo, Machine Learning Foundation, and running on SAP Cloud Platform, Cloud Foundry. We will take object detection for example in SMB Market Place PoC demo about Digital Transformation for online shopping experience (demo video) with Machine Learning and Chatbot mentioned in the Digital Transformation for SMB. Learning how to train and provision your custom object detection model with your own data for building intelligent solutions. Especially we will focus the most popular object detection algorithm YOLO(You Only Look Once) with Darknet and SSD(Single Shot Detection)/SSD Lite with Tensorflow Object Detection API. If you are new to object detection, it is recommended to have a look at the following introductory blogs:

• Zero to Hero: Guide to Object Detection using Deep Learning: Faster R-CNN,YOLO,SSD by ANKIT SACHAN


• Review of Deep Learning Algorithms for Object Detection by Arthur Ouaknine

What can we use object detection in enterprise context?

There are plenty of use cases for objection detection.

• In retail, face recognition can be used to identify customer in store, and empowering the store staff to gain customer insight and offer more personalised service, resulting in higher customer satisfaction and more revenue. Please refer to Face Recognition with in-Store Analytics by SAP SMB Innovation Lab. And a more business contextual article on Forbe about Face Recognition Technology Set To Transform Retail


• In marketing advisement, SAP Brand Impact measures and understands the value of your sponsorship campaigns with advanced computer-vision techniques for automatic logo recognition in videos that give you accurate, timely insights into marketing ROI. (Demo video)


• In security field, object detection can fuel the intelligence to the camera for surveillance. Trigger an alarm or send an alert for any emergent or dangerous occasion where need human interaction. (Demo video by Bosch Security)


• In healthcare, object detection can help the doctors for disease detection on CT/XRay images. Even further, bio-inspired camera and advanced object detection could be used in operation, or real-time disease detection in a human body, just like surveillance monitoring against disease inside your body. (Reference)


• In automobile, objection detection is used in autonomous car, which can recognise the pedestrians, other cars, traffic lights etc on the road.


• ...

Why, and what do I use object detection in SMB Market Place PoC Solution?

In v1.0 of the SMB Market Place Solution, the following two exception cases return some false results of matching the shoe product in image scoring similarity with SAP Leonardo.

1. No shoe in the image.
   However, it still proposes some shoe with matched feature vectors.
   The reason is that Similarity Scoring API in SAP Leonardo ML compares vectors using a similarity score (cosine distance) on the image, which has no clue if there is a shoe in the image.With object detection introduced on ver 1.1, the image will be pre-processed with shoe detector before handing it to SAP Leonardo for Image Similarity Scoring API. In this case, no shoe is detected by the shoe detector. Hereby, it returns the message saying "No shoe detected..." without further invoking SAP Leonardo for Image Similarity Scoring API.
   False Result without object detection:                Correct Result with object detection:
                                       


2. A shoe with a massive surrounding background which will affect the accuracy of the shoe by matched feature vectors.
   The reason is that the background introduces the chaos in the vector features on the shoe image, more chaos by background, less accuracy of matched shoe by feature vector. Which should be eliminated as much as possible.With object detection introduced v1.1, the object detection is used to detect the bounding box of shoes in the image, then the image is pre-processed to be cropped with the exact bounding box of the detected shoe with highest detection score. Afterwards, it sends the cropped image with shoe to Image Similarity Scoring API in SAP Leonardo for matching. In this way, the affect of background on the image is minimised or eliminated, resulting in a higher accuracy of shoe matching.
   False Result without object detection:               Correct Result with object detection:
                                         

In conclusion, object detection is used for the pre-processing of the product(shoe) image before passing to image similarity scoring api in SAP Leonardo.

1. If no shoes detected by shoe detector, then go to no shoe routine bypassing further calling image similarity scoring api in SAP Leonardo with false result.


2. In the case of at least one shoe detected, then cropping the image with the bounding box of the detected shoe with highest detection score, and passing to image similarity scoring api in SAP Leonardo, which results in higher accuracy of shoe matching.
   1).Shoe Detection.     ->     2).Image Preprocess.     ->        3).Similarity Scoring
    .           .                              

How do I implement the shoe detectors with YOLO and SSD lite?

The diagram below illustrates on the architecture. Two shoe detectors have been implemented for your reference, the source code of shoe detectors is published with MIT license here.

1. shoe detector#1(shoe-detector-yolo😞 Based on YOLO v2 or v3 with Darknet(An Open Source Neural Networks in C by Joseph Redmon), and Darkent.js(A NodeJS wrapper for Darknet by bennetthardwick). By default, it is using YOLO v2. Please refer to this blog about how to train my shoe detection model and provision it as web service on SAP Cloud Platform, Cloud Foundry


2. shoe detector#2(shoe-detector-tf😞 Based on SSD Lite v1 or v2 with Tensorflow Object Detection API. By default it  is using SSD Lite V2. Please refer to this blog about how to train my shoe detection model and provision it as web service on SAP Cloud Platform, Cloud Foundry

Now let us have a look at something tangible. A web demo kit has been included for you to play with object detection.

The demo kit includes the datasets and detection algorithm below, by default, the demo kit is initialised with MS COCO dataset and SSD Lite V2 with tensorflow.

Datasets:

1. MS COCO: including 80 object categories, such as person, cat, dog, dinning table, chair, bicycle, car etc...


2. Shoe: My custom trained model for shoe detection.

Detection Algorithms:

1. YOLO v2/v3


2. SSD Lite v1/v2

Play with the demo kit of object detection:

1. [Optional] Select the target dataset and detection algorithm, then click initialise the detector. The default dataset is MS COCO, and the default algorithm as SSD Lite V2 with tensorflow.


2. Copy and paste a image url that you want to detect.


3. Click Detect button. It will draw the bounding box in red rectangle with object name and confidence score, as well as the detection result in json.