Technical Articles
Call Leonardo ML Foundation API from my SmartPhone via SAP Cloud Platform Functions.
The original post is here.(Japanese language)
I tried creating SNS chatbot using SAP Cloud Platform Functions and SAP Leonardo ML Foundation API, so I will show you how to make it.
You will know..
- How to use SAP Leonardo ML Foundation API.
- You can call SAP Leonardo ML Foundation API from Node.js.
Estimated time
- 90 minutes
Appendix: About LINE
LINE is a popular SNS tool in the Asian region, especially in Japan.
The number of users in Japan is about 76 million. The daily active user (DAU) boasts a very high figure of 85%, both number of users and frequency of use.
Appendix: What is the LINE Messaging API?
It is an API that realizes two-way communication with users through LINE’s account.
It is possible to develop interactive Bot application using LINE talk screen.
This API has already been used in many corporate LINE accounts, such as courier delivery redelivery acceptance and new year greeting creation.
Appendix: What is SAP Cloud Platform Functions?
It is a program execution environment of serverless architecture previously announced by SAP.
You can run the program created with Node.js.
Outline of the procedure
- Try Leonardo ML Foundation API with SAP API Business Hub.
- Create bot channel (bot account).
Log in with a LINE account and create and set up a BOT account. - Activate service of Functions.
Access the SAP Cloud Platform CF Trial Europe (Frankfurt) environment and make initial settings for using Functions. - Create a Node.js program.
We will create a program to receive image messages from SNS and reply. - Test on actual smartphone.
1.Try Leonardo Foundation ML API with SAP API Business Hub.
With the SAP API Business Hub, it is possible to easily try the Leonardo Machine Learning API.
In Leonardo ML Foundation, you can try
- Use pre-trained AI.
- Use the AI that the user has re-trained.
In this time we will call up the pre-trained image classification API.
Open the API Business Hub Image Classification API page in your browser and click the Try out button of the API you want to try.
Select an image file and try it.
The result of AI judging the image will be returned.
By pressing the Code Snippet button on the API Business Hub, you can refer to the code snippet that calls the API from various programming languages.
In the following procedure, API Key is used when calling Leonardo ML API from Node.js. Look at the screen of the snippet and let’s keep an API Key.
2.Create bot channel (bot account).
We will create a channel as per the official page of LINE.
Please refer to the my previous article for setting method and setting place.
I created a channel named “画像認識くん (means Image recognition man)” this time.
Perform basic setting of the channel. We change the setting in red.
3.Activate service of Functions.
Functions are only available in the SAP CP CF Trial Europe (Frankfurt).
You also need to activate the beta service.
First, create a new subaccount that the Beta service can use.
Next, activate the Functions service of the trial2 subaccount.
Create an instance of Functions.
4.Create a Node.js program.
Access the Functions dashboard and create a program.
I made the program name “firstmlbot”.
Copy the following code and use it.
Please change ********** place to access token for the channel of LINE BOT.
Please change YOUR_API_KEY place to API Key of Leonardo ML API.
■index.js
var request = require("request");
var https = require("https");
module.exports = {
handler: function(event, context) {
if (event.data) {
if (event.data.events[0].type == "message") {
if (event.data.events[0].message.type == "image") {
var imageid = event.data.events[0].message.id;
// get image from below link
//https://api.line.me/v2/bot/message/{imageid}/content
var options = {
method: 'GET',
uri: 'https://api.line.me/v2/bot/message/' + imageid + '/content',
encoding: null,
auth: {
bearer: "*********************" // LINE BOT Access Token
}
};
request(options, function(error, response, body) {
// get binary image
var binaryimage = new Buffer(body);
// request to SAP ML Server
var boundary = createBoundary();
let optionstosap = {
host: "sandbox.api.sap.com",
port: 443,
path: "/ml/imageclassification/classification",
method: "POST",
headers: {
"APIKey": "YOUR_API_KEY",
"Content-Type": "multipart/form-data; boundary=" + boundary
}
};
var reqtosap = https.request(optionstosap, function(res) {
var data2 = '';
res.setEncoding("utf8");
res.on("data", (chunk) => {
data2 += chunk;
var resultarr = JSON.parse(data2).predictions[0].results;
var columns = [];
for (var k = 0; k < resultarr.length; k++) {
var column = {};
column.title = resultarr[k].label.substring(0, 35);
column.text = Math.round(resultarr[k].score * 100 * 10) / 10 + "%";
var actions = [];
var action = {};
action.type = "uri";
action.label = "Google翻訳";
action.uri = "https://translate.google.co.jp/?hl=ja#en/ja/" + encodeURIComponent(resultarr[k].label);
actions.push(action);
column.actions = actions;
columns.push(column);
}
var options2 = {
method: 'POST',
uri: 'https://api.line.me/v2/bot/message/reply',
body: {
replyToken: event.data.events[0].replyToken,
messages: [{
type: "template",
altText: "Image classification result.",
template: {
type: "carousel",
columns: columns
}
}]
},
auth: {
bearer: "*********************" // LINE BOT Access Token
},
json: true
};
request(options2, function(err, res, body) {
console.log(JSON.stringify(res));
});
});
reqtosap.on('end', function() {
console.log('data;', data2);
res.end();
});
});
reqtosap.on("error", function(e) {
console.error(e.message);
});
var buffer = unicode2buffer(
'--' + boundary + '\r\n' + 'Content-Disposition: form-data; name="files"; filename="myimage.png"\r\n' +
'Content-Type: image/png\r\n\r\n'
);
var buffer = appendBuffer(buffer,
binaryimage
);
var buffer = appendBuffer(buffer,
unicode2buffer(
'\r\n' + '--' + boundary + '--'
)
);
reqtosap.write(Buffer.from(buffer));
reqtosap.end();
});
}
}
}
console.log('event data ' + JSON.stringify(event.data));
return 'hello world from a function!';
}
}
function createBoundary() {
var multipartChars = "-_1234567890abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ";
var length = 30 + Math.floor(Math.random() * 10);
var boundary = "---------------------------";
for (var i = 0; i < length; i++) {
boundary += multipartChars.charAt(Math.floor(Math.random() * multipartChars.length));
}
return boundary;
}
function unicode2buffer(str) {
var n = str.length,
idx = -1,
byteLength = 512,
bytes = new Uint8Array(byteLength),
i, c, _bytes;
for (i = 0; i < n; ++i) {
c = str.charCodeAt(i);
if (c <= 0x7F) {
bytes[++idx] = c;
} else if (c <= 0x7FF) {
bytes[++idx] = 0xC0 | (c >>> 6);
bytes[++idx] = 0x80 | (c & 0x3F);
} else if (c <= 0xFFFF) {
bytes[++idx] = 0xE0 | (c >>> 12);
bytes[++idx] = 0x80 | ((c >>> 6) & 0x3F);
bytes[++idx] = 0x80 | (c & 0x3F);
} else {
bytes[++idx] = 0xF0 | (c >>> 18);
bytes[++idx] = 0x80 | ((c >>> 12) & 0x3F);
bytes[++idx] = 0x80 | ((c >>> 6) & 0x3F);
bytes[++idx] = 0x80 | (c & 0x3F);
}
if (byteLength - idx <= 4) {
_bytes = bytes;
byteLength *= 2;
bytes = new Uint8Array(byteLength);
bytes.set(_bytes);
}
}
idx++;
var result = new Uint8Array(idx);
result.set(bytes.subarray(0, idx), 0);
return result.buffer;
}
function appendBuffer(buf1, buf2) {
var uint8array = new Uint8Array(buf1.byteLength + buf2.byteLength);
uint8array.set(new Uint8Array(buf1), 0);
uint8array.set(new Uint8Array(buf2), buf1.byteLength);
return uint8array.buffer;
}
■Dependencies
{
"dependencies": {
"request": "*"
}
}
(Remind) The access token can be confirmed on the basic setting screen of the LINE BOT channel.
(Remind) API Key can be confirmed in Code Snippet of SAP API Business Hub.
After editing the source code, press the Save and Deploy button in the upper right corner of the dashboard to save.
Next, the trigger setting. This time, we use HTTP trigger.
Appendix: Triggers available in Functions
Trigger | Description |
HTTP | It issues a URL and fires when an HTTP request is made for that URL. |
Timer | Processing can be executed at regular intervals. |
Event | Combined with SAP Enterprise Messaging, you can operate the application with the event sent from outside as a trigger. |
Copy the Trigger URL of Functions and paste it in the Webhook URL on the channel setting screen of LINE Developers.
This completes the development. We will do a test.
5.Test on actual smartphone
Read the QR code of the channel setting screen of LINE Developers on the smartphone and add account.
Sending an image to BOT will determine what image it is.
Almost OK.
OK!!
No.. it’s my room.
Nooooo, it’s my room!!
By loading the following QR code with your smartphone, you can easily experience the LINE BOT created this time.(If you have LINE account.)
You can also invite this LINE BOT to group chat for use.
Please invite them to LINE group chat and play together with people you know.
When using Leonardo ML image classification function in your business, you should use AI that has been re-training, not trained AI.
As for Leonardo ML’s re-training, I would like to write it as an article.
Cool share Yasayuki! Thanks for taking the time to write it both in Japanese and English!
We're tweeting it out via @sapcp - feel free to tag us with future posts, and thanks for contributing!
Hi yasuyuki-san,
Thanks for an informative blog about integrating Line and Leonardo ML with SCP Function.
Just wonder if you have tried SCP function for Facebook Messenger webhook other than Line.
I have tried it with no luck.
No query parameters(token etc) for validation from facebook are received in the event(event.data) object of the function, therefore the webhook validation doesn’t go through.
Btw, FB messenger webhook works fine with AWS Lambda, the validation token available as event.params.querystring
Kind Regards, Yatsea
Hello Yatsea,
Since I saw your comment, I just created Facebook Bot for the first time.
Although I was having difficulty, I was able to develop it in the end.
Here is the source code. ( Please change according to your setting at "// PLEASE_CHANGE_HERE" commented line.)
Sorry for un-beautiful source code.
Hope your help,
Yasuyuki
Hi Yasuyuki,
Excellent. So we can have access to the request and response object through event.extensions.
There is little document about this on the official online help document.
I tried to look into event object with console.log on execution log, but having some problem to retrieve the execution logs of the functions on the tool.
Thank you very much for sharing.
Kind Regards, Yatsea
Yes, we can full access to the request and response objects through event.extensions.
Attention: If you want to get the request.body informations….
Hope your help,
Yasuyuki
Hi Yasuyuki,
I have your code ported for Facebook Messenger as below. Thanks for sharing again.
Kind Regards, Yatsea
It looks like this on FB messenger.
Hi Yatsea,
It's Great job!!
Regards, Yasuyuki
Great! Thank you for writing the article a lot!