Introduction

The SAP Leonardo Machine Learning service provides many API endpoints to easily integrate Machine Learning in your apps. I was looking in the list of services for a Face Recognition API. I found many services regarding Human, face and object detection + a Face Feature extraction API. All these API’s can detect faces and even extract the features of a face into a vector. I played around with these services and concluded that none of these API’s have the capability to compare and match faces…

I looked further in the list for other Machine Learning possibilities and came across the Similarity scoring API. This API offers the possibility to compare vectors.

The combination of the Face Feature Extraction API with the Similarity Scoring API would give the possibility to do face recognition.

• The Face Feature Extraction API searches for faces on an image and returns the features of the faces as vectors.
• Similarity Scoring API can compare vectors and so thus faces.

The above lists of Machine Learning API’s are coming from my Cloud Foundry Trial instance. Here you can create a trial Machine Learning instance and try the API’s. Next to that, you can also use the API hub of SAP. This is like a sandbox and provides these API’s without creating an instance on your trial account. You can find the Machine Learning API’s on the API hub here:

https://api.sap.com/package/SAPLeonardoMLFunctionalServices?section=Artifacts

Here you’ll find the Face Feature Extraction service a Similarity Scoring.

With the combination of these two Machine Learning API’s we should be able to do Face Recognition. Let me show it to you in the following steps.

Get started

I use Postman for testing the Machine Learning API’s. I suggest you use this tool too if you want to follow the next steps.

Before we can start using the API’s we need to get an authorization token, also known as Bearer Token.

To fetch this token, we need more information about the Machine Learning instance. You can find this information on you Cloud Foundry Trial account. Open your Machine Learning instance and go to the Service Keys. There, you’ll find everything you need.

Authenticate to Machine Learning

For authenticating to the Machine Learning instance, you need to fetch the bearer token.

First, look for the authentication url in the Service Keys. Normally, you’ll find this at the bottom:

Open Postman:

• Set the request method to GET
• Use the authentication URL from the Service Keys and add “/oauth/token?grant_type=client_credentials”

https://s0007914286trial.authentication.eu10.hana.ondemand.com/oauth/token?grant_type=client_credentials

• Choose “Basic Auth” as authorization type

• Fill in the clientid (from Service Keys”) as username and clientsecret as password

You will get the following result when you click on send. Copy the access_token, you’ll need it in the following requests

Face Feature Extraction

Next step is to convert several images to vectors. Therefore, we use the Face Feature Extraction API like this:

• Set the request method to POST
• Fill in the FACE_FEATURE_EXTRACTION_API_URL

https://mlftrial-face-feature-extractor.cfapps.eu10.hana.ondemand.com/api/v2alpha1/image/face-feature-extraction

• Set authentication type to Bearer Token and provide the “access_token” from previous request in the Token field

• To add the image that you want to convert, go to “Body” -> select “form-data” and add the key “files”. Change the type to “File” and upload your first image.

You’ll see the following result when you click on send. The “face_feature” array is your vector.

Do this now for several images to start comparing and finding matching faces.

Face Recognition

The final step to do face recognition!

This time, we’ll use the Similarity scoring api. This API will compare and match multiple vectors. You can use it in different ways. You could provide one set of vectors and it will compare each vector to the other vectors in the set. If you provide two sets of vectors, it will compare the vectors of set one with the vectors of the second set.

For Face Recognition, I use two sets. In the first set, I provide the vector of the image with the face of the person that I want to search for (based on his face). In the second set, I provide the vectors of faces (from other images) where I want to search in for similar faces.

To do this, you need to build the request like this:

• Set the request method to POST
• Fill in the SIMILARITY_SCORING_URL

https://mlftrial-similarity-scoring.cfapps.eu10.hana.ondemand.com/api/v2/similarity-scoring/

• Again, set authentication type to Bearer Token and provide the “access_token” from this first request in the Token field

• Then add the following key-values to the body in “form-data”
  • options
    • {“numSimilarVectors”:1}
    • We use “1” because we only want to find one match
  • texts
    • the json string like below
    • array “0” should contain the vector object of the image that you want to use to search with
    • array “1” should contain the list of vector objects that you want to use to search in
    • I use the “id”’s to know the name of the face behind the vector and I use “find” for the face that I use to search with (assuming that I don’t know who it is.. )

Full example of the “texts” part with only the first 3 values of each vector:

{
	"0": [{
		"id": "find",
		"vector": [
			-0.10909909754991531,
			0.1488579660654068,
			-0.0329568013548851,
			…
		]
	}],
	"1": [{
			"id": "DanielCraig",
			"vector ": [-0.0876927599310875,
				0.04249120503664017,
				0.09243552386760712,
				…
			]
		},
		{
			"id": "JamesBond",
			"vector": [
				-0.067575603723526,
				0.13878396153450012,
				0.11513494700193405,
				…
			]
		},
		{
			"id": "WouterLemaire",
			"vector": [
				-0.06013941019773483,
				0.17372670769691467,
				-0.011754469014704227,
				…
			]
		}
	]
}

I used two images of different James Bond actors and one of myself in array “1”. In Array “0”, I’ve added a vector of another image of myself.

The images of my second set in the same sequence:

The image in the first set, is just one I took at the time of testing from my laptop webcam.

If everything went well, it should find the vector of my picture in the second set for the vector in the first set ?

And that’s how Face Recognition is done with SAP Machine Learning Foundation services!