Skip to Content
Technical Articles
Author's profile photo Satheesh Ilu

Train Data Attribute Recommendation Service Machine Learning Model by Connecting to SFTP using SAP Cloud Platform Integration (Part 2)

In our previous part 1 blog post, we looked at how to setup Data Attribute Recommendation Service, SFTP and initial configuration of SAP Cloud Integration flows.  In this current blog post, we will look at each of the SAP Cloud integration flows in detail and how to configure and use them.

Integration Flows for Data Attribute Recommendation

Below are the three integration flows that are part of the Data Attribute Recommendation integration package. You would see these under the artifacts tab as shown below.

  • Create Dataset Schema
  • Create Dataset and Upload Data
  • Create Job and Deploy Model.

All%20the%20iFlows%20related%20to%20Data%20Attribute%20Recommendation%20Service

All the iFlows related to Data Attribute Recommendation Service

Let us look at each of the above now in detail

How to Configure iFlow: Create Dataset Schema

The iFlow “Create Dataset Schema” creates a Dataset schema based on a JSON input file. Below are the steps to configure the same.

1) Select option “Configure” by clicking on “Actions” icon next to the iFlow as in below screenshot.

Configure%20Option%20for%20iFlow

Configure Option for iFlow

2) You would get a popup screen to enter parameters of SFTP Sender and Receiver.  Let’s start with Sender. You would need to configure SFTP_Sender by providing the required information. The Directory property specifies in which directory the JSON Files are located. In the address field the address of the SFTP-Server must be set. As a proxy type you can either use On-Premise and provide the Location ID if necessary or use Internet. For Authentication you can reuse the Name of SFTP User created in Step 4 of User Credential Setup in the previous blog  Also, Post-Processing can be configured to specify the operation after a file has been processed by the Integration Flow.

SFTP%20Sender%20Info

SFTP Sender Info

3) You also need to configure the Receiver section. To configure the SFTP_Receiver,  you can enter the same data as in the previous step. Only the directory must be different from the one specified in the SFTP_Sender.

SFTP%20Receiver%20Info

 

SFTP Receiver Info

4) Next, we need to configure the OAuth_Server info. For that, you need to select the drop-down in Receiver as shown in the below screenshot.

OAuth_Server%20Selection

OAuth_Server Selection

After selection, now update the parameters for OAuth_Server

  • Input Address as url from uaa section from Document Attribute Recommendation service key JSON file
  • Input Credential Name with Data Attribute Recommendation User, the credentials name created in  Step 3 of User Credential Setup in the previous blog

OAuth_Server%20Config%20Info

OAuth_Server Config Info

5) Lastly, in the more section, you need to configure the parameter DAR Base URL as url from Document Attribute Recommendation service key JSON file

DAR%20Base%20URL%20Info

DAR Base URL Info

How to Configure iFlow: Create Dataset and Upload Data

The iFlow “Create Dataset and Upload Data” creates a Dataset and uploads the Data based on a CSV input file with training data.  Below are the steps to configure the same.

1) Similar to above, first you need to select the Configure option. See above section for the screenshot.

2) Similar to above, you need to enter parameters of SFTP Sender and Receiver.  You would need to configure SFTP_Sender by providing the required information. The Directory property specifies in which directory the JSON Files are located. In the address field the address of the SFTP-Server must be set. As a proxy type you can either use On-Premise and provide the Location ID if necessary or use Internet. For Authentication you can reuse the Name of SFTP User created in Step 4 of User Credential Setup in the previous blog  Also, Post-Processing can be configured to specify the operation after a file has been processed by the Integration Flow.

Create%20Dataset%20-%20SFTP%20Sender%20Info

Create Dataset – SFTP Sender Info

3) Similar to above, you also need to configure the Receiver section . To configure the SFTP_Receiver,  you can enter the same data as in the above step. Only the directory must be different from the one specified in the SFTP_Sender.

Create%20Dataset%20SFTP%20Receiver%20Info

Create Dataset SFTP Receiver Info

4) Next, we need to configure the SFTP_Receiver_Error. You can select the SFTP_Receiver_Error from the dropdown as shown below.

Select%20SFTP%20Receiver%20Error

Select SFTP Receiver Error

After the SFTP_Receiver_Error is selected, you can enter the same data as in the above step. Only the directory must be different from the one specified in the SFTP_Sender. 

SFTP%20Receiver%20Error

SFTP Receiver Error

5) Next, we need to configure the OAuth_Server info. For that, you need to select the drop-down in Receiver as shown in the above step. After selection, update the parameters for OAuth_Server

  • Input Address as url from uaa section from Document Attribute Recommendation service key JSON file
  • Input Credential Name with Data Attribute Recommendation User, the credentials name created in  Step 3 of User Credential Setup in the previous blog

OAuth_Server%20Config%20Info

OAuth_Server Config Info

6) Lastly, in the more section, you need to configure the parameter DAR Base URL as url from Document Attribute Recommendation service key JSON file

DAR%20Base%20URL%20Info

DAR Base URL Info

How to Configure iFlow: Create Job and Deploy Model

The iFlow “Create Job and Deploy Model” creates the training job and delpoys the model based on the JSON input file. Below are the steps to configure the same.

1) Similar to above, first you need to select the Configure option. See above sections for the screenshot.

2) Similar to above, you need to enter parameters of SFTP Sender and Receiver.  You would need to configure SFTP_Sender by providing the required information. The Directory property specifies in which directory the JSON Files are located. In the address field the address of the SFTP-Server must be set. As a proxy type you can either use On-Premise and provide the Location ID if necessary or use Internet. For Authentication you can reuse the Name of SFTP User created in Step 4 of User Credential Setup in the previous blog  Also, Post-Processing can be configured to specify the operation after a file has been processed by the Integration Flow.

Create%20Job%20and%20Deploy%20Model%20Sender%20Info

Create Job and Deploy Model Sender Info

3) Similar to above, you also need to configure the Receiver section . To configure the SFTP_Receiver,  you can enter the same data as in the above step. Only the directory must be different from the one specified in the SFTP_Sender.

Create%20Job%20and%20Deploy%20-%20SFTP%20Receiver%20Info

Create Job and Deploy – SFTP Receiver Info

4) Next, we need to configure the SFTP_Receiver_Error. Similar to the above, you need to select it from the dropdown. After the SFTP_Receiver_Error is selected, you can enter the same data as in the above step. Only the directory must be different from the one specified in the SFTP_Sender. 

Create%20Job%20and%20Deploy%20-%20SFTP%20Receiver%20Error

Create Job and Deploy – SFTP Receiver Error

5) Next, we need to configure the OAuth_Server info. For that, you need to select the drop-down in Receiver as shown in the above step. After selection, update the parameters for OAuth_Server

  • Input Address as url from uaa section from Document Attribute Recommendation service key JSON file
  • Input Credential Name with Data Attribute Recommendation User, the credentials name created in  Step 3 of User Credential Setup in the previous blog

Create%20Job%20and%20Deploy%20Model%20-%20OAuth%20Server

Create Job and Deploy Model – OAuth Server

6) Lastly, in the more section, you need to configure the parameter DAR Base URL as url from Document Attribute Recommendation service key JSON file

DAR%20Base%20URL%20Info

DAR Base URL Info

Integration Flows Deployment

After the configuration of all the integration flows, now it is time for deployment of the same.  All you need to do is click on the “Deploy” Option as shown below, which is under ‘Actions‘. You need to do this for all the three iFlows.

Deploy%20the%20iFlows

Deploy the iFlows

Now that we have configured all the iFlows and deployed them, let’s look at the processing of files and how it works.

Processing of files by iFlow

In this section, we review how the files related to each of the iFlows are handled, where to upload, and how it works.

How files are processed in “Create Dataset Schema” iFlow 

You need to upload the JSON file to the directory of the SFTP_Sender specified in Step 2 of the above configuration section of ‘Create Dataset Schema’. The Integration Flow picks up the files and processes them. The JSON-File must be in the following structure (as per the /datasetSchemas API)

{
  "features": [
    {
      "label": "string",
      "type": "TEXT"
    }
  ],
  "labels": [
    {
      "label": "string",
      "type": "TEXT"
    }
  ],
  "name": "string"
}

How files are processed in “Create Dataset and Upload Data ” iFlow 

In order to create Dataset and upload Data, a CSV-File has to be uploaded to the directory of the SFTP_Sender specified above in Step 2 of the above configuration section of ‘Create Dataset and Upload Data’. The Integration Flow picks up the files and processes the same.

The header information of the Dataset that is in the CSV file should match the feature and label names provided in the dataset schema. The Dataset CSV file uses only commas (,) as separators and should contain only UTF-8 encoding data. Any other encoding type is not supported by Data Attribute Recommendation.

The Filename of the CSV must start with the name of the dataset, then a dot as a separator followed by the id of the dataset schema (example: “my-dataset-name.datasetSchemaId.csv”)

How files are processed in “Create Job and Deploy Model ” iFlow 

In order to create a Training Job and Deploy a Model a JSON file has to be uploaded to the directory of the SFTP_Sender specified in Step 2 of above configuration section of ‘Create Job and Deploy Model’. The Integration Flow picks up the files and processes the same.

The JSON File must be in the following structure

{
    "datasetId": "String",
    "modelTemplateId": "String",
    "modelName": "String"
}

Monitor Integration Flows

After you have deployed everything, you can monitor the integration flows and the messages by going to the operations view. You can select the tile with ‘started‘ status under “Manage Integration Content” as shown below to see the status of each of them.

Operations%20view%20of%20Cloud%20Integration

Operations view of Cloud Integration

As you go to the detail view, you can see the status of the iFlow and when the “Latest Poll” happened. You can even go to the details of messages by selecting the “Monitor Message Processing” link under Artifact Details. See below screenshot.

Status%20of%20the%20iFlows

Status of the iFlows

 

Hope these two blog posts gave you an idea of how to configure and deploy all the iFlows related to Data Attribute Recommendation.  You may also read how to integrate your enterprise email inbox with the Document Information Extraction service via SAP Cloud Integration

For more information on SAP AI Business Services:

Explore: SAP Community Page

Dive deeper: Open SAP Course

Get an overview: Blogpost part I  | Blogpost part II

Exchange Knowledge:

Document Classification Questions | Document Information Extraction Questions

Business Entity Recognition Questions | Service Ticket Intelligence Questions

Data Attribute Recommendation Questions | Invoice Object Recommendation Questions

Assigned tags

      1 Comment
      You must be Logged on to comment or reply to a post.
      Author's profile photo Michael Haas
      Michael Haas

      Thanks, this is a great way to integrate Data Attribute Recommendation!