Technical Articles
SAP B-LAB : Combining Integration Suite and LLMs to generate integration mappings
With the rise of LLMs (large-language-models), many business and technical users have seen their day-to-day work revolutionised.
One role has been left out; that of the integration specialist.
Here, we explore the potential of generative and interactive AI for integration scenarios
Who is “we”? The B-LAB!
We are a team of SAP BELUX customer advisors, passionate about tech.
We prototype with customers and explore the possibilities of SAP.
SAP Integration Suite already provides the infrastructure, tools and business content to orchestrate a vast amount of integration scenarios.
There are now thousands of standard integrations, but we cannot possibly cover all customer solutions. In these cases, you need to develop a custom integration.
The exact mapping from the fields of an API to another can happen in various ways, among which:
In javascript or groovy in SAP Integration Suite, in a node.js microservice, and more.
Your choice of the mapping technology will depend on the nature of the integration, time constraints, etc.
This task requires a solid understanding of both input and output, as well as quite some time.
A colleague and I wanted to try to automate this mapping process, using LLMs.
We tested this concept with two, rather standard, APIs:
- The northwind order table as “input” with 20 fields and 10 example lines
- The SAP S/4HANA Cloud Freight Order (A2X) as “output”, using its first 42 fields and 10 example llines
The real game-changer in our tests was to include example lines.
Headers were simply not enough to get close to a match. Giving only a dozen examples lead to much better results. With larger and larger model inputs, this could even scale to hundreds of example lines, for a more complete understanding and to include edge-cases.
Here are the steps we followed:
- Pick our 2 interfaces
- Ask in the prompt (available below) to generate a javascript mapping from one to the other
- implement this mapping in a standard node.js code
- Start this instance on SAP BTP Cloud Foundry Runtime
- Test this service: accuracy, latency, etc.
You could in theory automate pretty much all of these steps after the first one, using the SAP BTP Command Line Interface.
Our prompt to GPT 4.0 (similar result with the Falcon open source model):
“Generate a mapping in javascript interface between the following APIs:
The input (northwind orders) has the following format:
10 lines of northwind orders
The output (sap freight orders) needs to be in this format:
10 lines of sap freight orders
Do as many transformation mappings from input to output as possible.
Also Tell me how many mappings you successfully established.
Act step by step and make sure the code is rigorous.
If you are not certain of a mapping, don’t do it.”
Here is the output, with our node.js mapping:
const express = require('express');
const uuid = require('uuid');
const app = express();
app.use(express.json());
app.post('/api/transform', (req, res) => {
const input = req.body;
const output = input.map(orderData => { order = orderData.split(",");
return {
TransportationOrderUUID: uuid.v4(),
TransportationOrder: order[0],
Carrier: order[8],
Shipper: order[1],
ShipperAddressID: order[9],
Consignee: order[1],
ConsigneeAddressID: order[9],
TranspOrdPartnerReference: order[0],
TranspOrdResponsiblePerson: order[2],
TranspOrdHasMltplExectgPties: false,
TranspOrdExecutionIsBlocked: false,
}
});
res.send(output);
});
app.listen(3000, () => console.log('Server running on port 3000'));
module.exports = app;
GPT-4 Managed to find 11 mappings out of the 20, which is as much as there is in this scenario!
We further tested with fake customer data, and achieved high-quality mappings as well.
Broader testing and further research needs to be done.
I see this as a game-changing new chapter to build integrations.
The speed at which we were able to build integrations without previous knowledge of the input and output is truly staggering.
Developing our very own purpose-built LLM for integrations to SAP could be a very beneficial step SAP could take. Otherwise adding a layer of understanding and sanity checks could help scale this concept to more lines of business (finance, hr, etc).
The full power of SAP Integration Suite is revealed: if it’s not standard content, integration can largely be automated with AI.
It’s the story of the tower of Babel backwards; our systems just learned all the API languages of the world!
What do you think of this concept, its extensions and potential?
Hello Raphael,
Its very cool.Above project B-Lab is productized or kind of available as trial.
Major issue for us would be calling GPT-4 as it requires exposure of our data and we cant always mask data or our specific rules for mappings.
In case above solution or idea comes as extension where SAP embedds GPT-4 or equivalent AI tech as part of SAP Integration suite then we dont have any concerns for leveraging it or use or evaluate B-Lab.
Kudos to your team for initiating B-Lab and taking this to community.Good luck!
Regards,
Sri
Raphael Caillon could you please provide more details on how you submitted the examples for the input format and desired output format?
Also how did ChatGPT received feedback if the transformation was a correct one? Did you validate each trasnformation and provided feedback ?
Hello Eurico,
Thank you for your question.
1 - We extracted the 10 lines in a csv. It looks something like:
2 - We only ran our tests "once": giving it input and working with the output.
It would indeed be more realistic to have a short discussion with it.
We could give feedback like "ShipCountry abbreviations are not properly mapped; Use ISO 3166 instead."