Ten Days with SAP BTP, AWS, a Third-party API, and...

former_member127591 · ‎04-21-2021

Follow along as the team from Campton Hills Industrial builds a multi-could, AI application that exploits SAP BTP to convert their enterprise product data into business value.

All code, integration flows, configurations, and resources can be found here. Implementation instructions are provided in Part II.

Despite her teams’ often-used battle cry of “Omni-channel, What?, What?!”, Bridget still enjoys working as a solution architect at Campton Hills Industrial. She really likes her team and the company seems to be on the right track. They’re a “general line” distributor that has managed to do well in a market seeing increased competition, thinning margins, and looming giants. The key to her company's success: using innovative technologies to efficiently integrate new acquisitions and improve their respective ROI. In turn, this allows the organization to build significant market share, while still improving commercial and operational capabilities. On top of all of that, from an IT perspective, it’s great a company for Bridget. The corporate culture fully embraces cloud computing and mobile, which keeps her well versed in the latest technologies. It also provides many interesting and challenging opportunities, like the one that just arrived in her inbox:

Hi Bridget,

The recent acquisition of South Side Supply (SSS) should provide a great opportunity to grow market share. To ensure this, we need to “close the loop” on several open issues. We are requesting your immediate help with an issue that requires a proof-of-concept (POC). Details follow. The internal demo is in ten days.

QR Codes hold more data than barcodes. For that reason, South Side Supply (SSS) recently started shipping products, from their “general industrial” line, with these improved labels. Ever since the switch, a key customer had continually requested an application to help identify and label the older, unlabeled products. This application was never delivered.

We need to demonstrate that we can quickly and effectively build this application in a short time frame. This will help ensure this customer of our capabilities and our commitment. In the end, the application should improve the customer’s procurement process and ultimately, helps us to become more ingrained within their supply chain.

Please review the following requirements and submit an architecture proposal, a UI mock-up, and resource estimates by EOD.

Purpose:

Search and identify older, unlabeled products and give the user the ability to generate a QR Code Label.

Key Requirements:

Users can search by Product ID.

Users can search by Product Description (Freestyle / no wild-cards)

Users can search by a picture of a product (image recognition / drag & drop)

Return a list of found products with basic information (ID, Description, UOM)

Users should be able to select a product to view the details, including a picture of the product (limited set of pictures for POC)

Users should be able to select a product to view the details, including a QR Code Label.

Future Design Considerations:

The QR Code Label cannot be static. The customer wants to be able to add their own text to the label.

Mobile with camera integration likely key UI platform

Integration with our Distribution Platform

Hooks for customer’s purchasing system

Thanks,
Ollie

Chief Technology Officer
Campton Hills Industrial, Inc.
4N498 Town Hall Rd, Campton Hills, IL 60175

Day 1:

After reviewing the CTO’s message, Bridget clears her calendar and quickly assembles a team to help design the POC. Late in the night, she submits the day’s effort:

Architecture Proposal

Besides the UI code, the full logic of the application will be contained within two integration flows or the services called by them. The integration flows (iFlows) will be built using SAP Cloud Integration, which is a key component of the SAP Business Technology Platform (BTP). Very little coding will be required to build these iFlows. The majority of the logic will be wrapped in components called “steps”, which you drag and drop onto a “canvas” and then configure as needed. There are steps for mapping messages from one format to another. There are steps for modifying the message, as it proceeds through the flow. There are additional steps for calling other systems, for logically changing the path of the message, for events, and so on. When a configurable step is not enough, you just drag a “script” step onto the canvas and manipulate the message as needed, using JavaScript or Groovy Script. This application will require the creation of two iFlows: “Products” and “Details”.

Products Integration Flow

The “Products” iFlow will return a list of products, filtered by product ID, product description, or an uploaded image. If an image is passed to the iFlow, it first passes this image to a custom connector, built within SAP Open Connectors. This connector reduces the amount of coding required to authenticate and connect to the Amazon Web Services (AWS) image recognition service, called Rekognition. The identified “labels”, returned from this service, are used to construct a query string, based on the product description. For example, if the following image was sent to the AWS Rekognition service, the followings labels would be returned:

Label	Confidence
Bird	99.9%
Animal	99.9%
Hummingbird	94.8%
Bee Eater	77.2%

These labels would then be used to construct a query string that searches for products with a description containing any or all of these words. If no image is passed, it will construct a query string based on the passed-in product ID and description. Passing both headers and the body will result in an amalgamated query string. With or without a query string, it will then call the “Product” API, on SAP API Business Hub, to find and return a list of products.

SAP API Business Hub

The main purpose of the API Business Hub is to provide a central catalog of SAP and selected partner APIs, for application developers to search, discover, and test. For this application, the API Business Hub is used to mimic the product service of Campton Hills Industrial

Details Integration Flow

The “Details” iFlow will return some product attributes, a URL to image of the product (if available), and a generated QR Code similar to:

The iFlow requires the product ID as input and calls the same “Product” API on SAP API Business Hub to return the details of a single product. These product details are sent to the QR Code Generator API to generate and return a QR Code. The returned image URL is a combination of the product ID and the address of a bucket, created with Amazon’s Simple Storage Service (S3).

These iFlows are designed to be called via HTTP, so when deployed, Cloud Integration will provide an endpoint. We could call these endpoints directly, from our user interface, but exposing the endpoint through SAP API Management will provide a number of advantages, including:

Simplified API consumption

Centralized API governance

Omni-channel experience creation

Configurable security and traffic management

Monetization of digital assets

Finally, the user interface will be built using Node.js and will utilize the following modules:

Comp.	Module	Purpose
Client	React	React is an open-source JavaScript library used for building user interfaces specifically for single-page applications
Client	React-simple-dropzone	Simple, easy to use dropzone with preview for react.
Client	React Google Charts	React Google Charts offers a declarative API to make rendering charts fun and easy
Client	Ant Design	A design system for enterprise-level products. Create an efficient and enjoyable work experience.
Server	dotenv	Storing configuration in the environment separate from code is based on The Twelve-Factor App methodology.
Server	Express	Fast, unopinionated, minimalist web framework for Node.js
Server	Axios	Promise based HTTP client for the browser and node.js
Server	axios-oauth-client	OAuth 2.0 client utils for Axios

User Interface Mock-up

The user clicks the “Search” button, after entering all or none of the filters, which include a product ID, a product description, and an image.

The products are returned in a table. If an image was uploaded, the “Image Recognition Labels” tab is visible. Clicking on this tab will show the labels returned by the image recognition service and the amount of confidence for each label.

Clicking on the details link will display a product card, which includes a product image (if it exists) and a generated QR Code.

Development Estimates

Resources Required	Task	Effort (Days)
4	Design	1
1	AWS Administration	.5
2	SAP BTP Development	5
1	UI Development	5
4	Testing	2
4	Internal Demonstration	1
Total Effort		14.5
Concurrency Adjustment		-4.5
Total Days		10

Day 2:

After receiving an early morning approval, the SAP BTP development is started. Meanwhile, the remaining team selects the five products to be used for the demonstration:

Later in the day, the AWS Administrator uploads the product images into an Amazon S3 bucket and enables public access.

Amazon recommends that buckets should only be publicly accessible for
verified uses cases (e.g. static web pages). You can learn more here.

Day 3:

The user interface development begins. In the afternoon, Bridget joins the AWS administrator to test the product images in the Amazon Rekognition Console:

She wants to be sure that labels, returned by the image recognition service, will find valid products. She documents the labels returned in the Rekognition console. Later, she tests the “Products” API and documents the results.

Image	Labels Returned (>= 80% Confidence)	# Products Returned
	Door	22
	Light Lightbulb Lamp	25*
	PC Computer Electronics Laptop	25*
	Machine Gear Cross Symbol	25*
	Beer Beverage Alcohol Drink	6

*The query string requests that only the top 25 products are returned.

Day 4:

Midday, the SAP BTP development team releases the new, custom connector for testing. The connector was created in SAP Open Connectors. It’s helpful because it simplifies the effort to connect from the iFlow to the AWS Rekognition service. Another advantage of the connector is that it can be tested independently of the iFlow:

Day 5:

At the end of the fifth day, the SAP BTP Development team deploy their iFlows for a final time and release them for testing.

Products Integration Flow

The HTTP message can come into the flow with the product ID and description in the request header. The request body may contain the base64 encoded image data.

The “Initialize” content modifier takes the externalized parameters (configurable properties) and sets them as values in the message header and properties.

If there is an image in the message body, the “Determine Route” script step sets the message route to “2”. Otherwise, the default route remains “1”.

The “Router” step directs the message to the correct route.

If the route has been set to “2”, then custom open connector is called to get the labels from the AWS Rekognition service. Otherwise, the “Set Product Filter” script is called.

The “Set Product Filter” script constructs the filter string of the query, using the values of product ID, product description, and any labels returned from the image recognition server. For example:
$filter=ProductID eq ‘112334’ and ( substringof(‘light’,Description) or ( substringof(‘lamp’,Description) )

The Product API is called on SAP API Business Hub to retrieve the selected products.

The products are returned in the body of the response message.

Details Integration Flow

The HTTP message comes into the flow with the product ID in the request header.

The “Initialize” content modifier takes the externalized parameters (configurable properties) and sets them as values in the message header and properties.

The “Get Product Info” is a request/reply step that calls the Product API, on SAP API Busines Hub, to retrieve the selected product.

The “Product Info – JSON to UTF-8” script step builds the value that will be used to generate the QR Code:

  String cr = "%0A";

  String sp = "%20";

  String tb = "%09";

  

  String line1 = "Campton Hills Industrial";

  String line2 = "ID:"+tb+properties.ProductID[0];

  String line3 = "Desc:"+tb+properties.Description[0];

  String line4 = "UOM:"+tb+properties.BaseUOM[0]+sp+'-'

                       +sp+properties.BaseUOMText[0];



  String qr_string = line1+cr+line2+cr+line3+cr+line4;



  message.setProperty("qr_string", qr_string);

The “Get QR Code” request/reply step calls the QR Code Generator API to get the QR Code.

The “finalize JSON” step encodes the QR Code in the message using Base64. It also manipulates the message body to return the product attributes, the image URL, and the base64 encoded QR Code.

The JSON is returned in the body of the response message.

Day 6:

The BTP team expose the iFlow endpoints in SAP API Management. It is a simple process and only takes a few minutes.

Day 7:

The UI developer has been working on the user interface for the last four days. Now that the BTP development is complete, it is simply a matter of replacing mock data with actual API calls and performing some additional unit testing. In the afternoon, the user interface is released for testing.

The user interface was built using Visual Studio 2019, with the code stored in a repository on GitHub. For the POC, the user interface is not deployed to any cloud platform.

Client Component

The client component of the user interface, uses React to provide the framework for rendering the HTML. React provides state management, a virtual DOM and other features. Within that framework, Ant Design components (buttons, forms, etc.) are used to provide a consistent “look and feel”. Additional modules are used for the drag-and-drop control and the chart of returned labels. Finally, two “fetch” calls are made to routes of the server component. When a user clicks the “Search” button, a fetch is made to the “/products” route on the server. Likewise, a click on the “details” link will initiate a fetch to the “/details” route.

Server Component

The sever component of the user interface, uses the Express module to handle routing. The Axios module simplifies making API calls and the “axios-oauth-client” module makes performing 0Auth2 authorization easier with Axios. The “dotenv” module is used to store authorization values and other configurable properties in a “.env” file. Finally, the “.gitignore” file is configured to ignore the “.env” file, so sensitive information is not stored in the git repository.

Days 8 & 9:

Several rounds of testing and revisions occur over the course of two days.

Day 10:

Bridget and team present an internal demonstration to members of upper management and the customer's sales team:

https://youtu.be/-KQAxt5-mCk

Overall, the feedback from the demonstration was positive. They were pleased with the results and impressed by the short amount of time in which Bridget and her team were able to construct the application. They expressed that using Node.js, to build the user interface, should provide flexibility when it comes time to determine a deployment target. Finally, they were appreciative that solution was built using SAP BTP because it makes it simple to integrate complex systems like Amazon Web Services.

All code, integration flows, configurations, and resources can be found here. Implementation instructions are provided in Part II.