Technical Articles
How to merge mass BP data with Postman automatically
Problem:
We had almost 12.000 duplicate individual customer in C4C and we had to merge this individual customer data with their master data. (On the solution part I will tell you each customer has just one duplicate customer scenario. But if you prepare the right data set, you can use this solution for scenario about each customer have 1 or 2 duplicate customer. ) For the merge of 2 or 3 customers in C4C you have to go to Customer Merge workcenter and click create button and have to progress the merge process manually. But with this scenario we had 12.000 customers and if you do it manually, it will be almost insoluble process. For this reason we had to create a new solution.
Solution :
For this mass duplicate data you could create many solutions. For example you could change duplicate customer’s status to obsolete and take them from your client’s OWL. (Because default filter listing all data except obsolete.) Maybe it can be a simple and effortless solution but let’s say we have 100 individual customer in C4C and 40 percent of this customers are duplicate and these duplicate customers assign their visits. If you change this customer’s status to obsolete, these visit documents may become corrupted. In merge process, if you combine duplicate and master customer, the system will write the master customer for all data records inside.
In our scenario first we thought use dataworkbench. But we can’t use dataworkbench for mass BP data merge. After we examined Odata services from here and found “BusinessPartnerMergeCollection” service. We had Odata service but the payload look like below;
{
"Name": "Documentation Merge 02",
"MasterBusinessPartnerID": "1056644",
"FirstDuplicateBusinessPartnerID": "1056643",
"InitiateMerge": true
}In this payload if you put right data, you can successfully save your customer merge document and the merge process will automatically complete. But for mass data you have to change payload for each request manually. And again this scenario goes to insoluble process.
At this point, we came up with our new solution. We thought we could set variables to this payload for name, masterbusinesspartnerid and firstduplicatebusinesspartnerid fields. And thought change these variable values with our individual customer numbers for each request when we trigger payload from Postman. But even the payload will change again, you have to trigger payload manually from Postman with click to send button. We searched on Google and found Postman Loop collections I will tell you these processes but first of all let’s say we do all these processes I have written about, If we send all this data to C4C in one time we thought it can be a security problem possibility and we thought we have to write a delay.
Let’s get on the road,
First, I pulled data into Excel to find out which customers are duplicated with 12.000 customers, and I prepared datasets with the help of formulas such as vlookup and countif.
After that I found a collection from here and change this collection’s body with writing many different scripts.
Below you can find how I define variable to payload body;
{
"Name": "{{mergename}}",
"MasterBusinessPartnerID": "{{master}}",
"FirstDuplicateBusinessPartnerID": "{{first}}",
"InitiateMerge": true
}The important point in here we defined variables to payload like above, and I will tell you how I wrote the script to change these variable values for each request.
let masters = pm.collectionVariables.get("masters");
if(!masters || masters.length == 0) {
masters = ["1006577", "1004524"];
}
let currentmaster = masters.shift();
pm.collectionVariables.set("master", currentmaster);
pm.collectionVariables.set("masters", masters);
let firsts = pm.collectionVariables.get("firsts");
if(!firsts || firsts.length == 0) {
firsts = ["1008504", "1008506"];
}
let currentfirst = firsts.shift();
pm.collectionVariables.set("first", currentfirst);
pm.collectionVariables.set("firsts", firsts);
let mergenames = pm.collectionVariables.get("mergenames");
if(!mergenames || mergenames.length == 0) {
mergenames = ["burn_baby_burn12", "burn_baby_burn11"];
}
let currentmergename = mergenames.shift();
pm.collectionVariables.set("mergename", currentmergename);
pm.collectionVariables.set("mergenames", mergenames);With the script above, first I defined the master variable and put to payload the dataset that this can use. And every next step, I shift one right into the dataset to get the other data.
And then I repeat it for my first and mergename variables.
Note**: Here, expressions such as master, masters, first, firsts, may seem meaningless, but my reason for adding the letter s to their end was just for the blended prevention. Also, for this article, I just added 4 example customer on script.
Then on the Test tab, I write the other script as follows:
var i=0;
while (i<1)
{
postman.setNextRequest ("Loop Post Parameter");
i++;
}
pm.test("Successful POST request", function () {
pm.expect(pm.response.code).to.be.oneOf([200,201]);
});With the script above, I’m actually going to loop through it, and I’m going to make sure it recalls my Loop Post parameter Report at the next step of the cycle. Every time you send a request, I also get a message to me that you’re successful, and I get it back on the next code block.
Here, in the number of iterations that will be completed within the while (12.000 iterations are required for our example.) if you type the number of iterations in the shape of n-1 (because we have i=0 you will do i=1 you can go straight to n). that’s how many cycles it goes into.
Also, Postman has a function to put a delay between both of our other problems, the request.
SetTimeout (function () {}, [number]);
but instead of using function here, I will show you another solution.
Note** : I used these functions after reading here. And the document says “Make sure to wrap setNextRequest in some additional logic so the request doesn’t loop indefinitely. For example, you might exit the loop after a certain number of iterations or when another condition is met. Otherwise you will need to force close the Collection Runner to end the loop.”
I give iterations to runner for make sure but you have to be very careful at this point for endless loop risk.
When all of this is over, I read that using Postman’s Runner, it will be better, and once I have completed the scripts, I will now come to the run step;
When you click the Runner button in the lower right corner in Postman, we drag and drop the corresponding Collection to the left on the following screen. Here we will only use the Post. But let’s say you have to get a token before the Post. If you check the request and put it in the top row. (You should also, of course, add a code block to your scripts that you can take and use this token.) First you take the token with get and then you throw the Post away.
Another feature here is the iterations and Delay features you see on the right. If we write to 2000 in this Delay field. It puts 2 seconds between both requests.
Finally, press the Run button;
Note** Probably, you have to use token for Post. For this reason first you have to send GET to C4C for something with header value x-csrf-token: fetch. After this step token will return to response header and you can use this token for post header. But also you have to do this process for use token just one time.
The image in the C4C will be completed as shown above. Column 3 includes master customers. (the code didn’t write name to 71 when I was taking screenshot, I fixed it later and you can use above codes.)
I hope it helps to you!
Thanks Mustafa for the detailed post.
I had a similar requirement which I fixed in a simpler (but less interesting) manner. It is good to see you accepted the challenge of scripting a loop in postman :-). I didn't dare...
I activated communication arrangement 'External Business Partner Merge', allowing me to use the https://my******.crm.ondemand.com/sap/bc/srt/scs/sap/managebusinesspartnermergein soap service.
This service allows you to call interface MaintainBundle to create merge cases and interface Initiate to initiate them.
Both can be added in batch, so Maintainbundle can consume for instance the following where 2 mergecases are created simultaneously using a single call:
and Initiate can consume where 2 mergecases are initiated using a single call:
This allows you to create the XML's in for instance Excel and then process them using for instance SOAPUI without any scripting (and infinate loop risks).
Note that you should limit the number of records you process as a timeout might occur if the message body is too big.
Note also that in the initiate process you should allow the system some time to actually process the mergecases in order to not overload the system. Overloading will result in a degraded performance in the merge process. So don't fire messages too soon after eachother.
Regards,
Pieter
Hi Pieter,
Thanks for sharing your experience for a similar requirement. Your solution also good and functional and I will note it. Actually as you say I just accepted challenge and wanted to create different solution. 🙂 All solutions has pros and cons. I just wanted to fix every point in one time e.g. (timeout, message body size)
Hi Peter,
That's a good approach. However, it promotes obsolete/deprecated SOAP service. That, in fact, is obsolete/deprecated for quite some time already. Please check here (OBSOLETE) Web Service APIs in SAP Cloud for Customer and here 3035375 - End-of-Life of OData API v1 and Web Services (SOAP, A2X) in SAP Cloud for Customer
Hi Andrei,
You are right. Thanks for pointing that out. It has indeed been deprecated for 2 years already, yet I found that it is still working, no side effects noted.
That said, I agree this points out that Mustafa's solution is the one to go for for anyone facing a similar challenge.
Maybe we can also conclude that the 'object merge case' would be most welcome in the data workbench?
Regards,
Pieter
Superb Mustafa. A really nice trick of using postman to play with the OData API. It is a pity that SAP hasn't provided an option for customer merging usecase via Data Workbench. But in any case, your solution will also do the job for now.
Thanks
Saurabh