Increasing performance of high-volume data loads via the Migration Workbench
Motivation
When uploading a high volume of data into C4C via the Migration Workbench (MWB) performance is always an important aspect. The MWB supports various ways to support high-volume data loads, the most relevant ones being:
- Bundling: Multiple instances of an object (e.g. multiple customer records) can be uploaded with 1 webservice call
- Parallelization: Multiple parallel webservice calls can be performed at the same time
While the bundling factor is hard-coded per migration object (e.g. for accounts it is 10), the parallelization factor can be set by the key user within the MWB before the actual migration run. If it is not set explicitly, the default is 4. The below section describes how to adjust this factor which can result in a significant performance boost.
Configuration
1) After accessing the staging area of the MWB, click on “You Can Also” in the top right corner. There you will find the option “Adjust Settings for Parallel Processing”
2) In the following popup you can adjust the parallelization factor. If you open it the first time you will see the global default, which is 4.
Raise this factor to increase the number of webservice calls which are executed by the MWB in parallel at any point in time. Note that for the customer upload internal tests have shown that a parallelization factor of 30 results in the best migration performance.
Hello Marcus,
Thanks for sharing this. 🙂
Regards,
Chandan
Great blog Marcus. So the No. of Parallel processes is actually the number of multiple WS calls scheduled for run in Parallel Processing. Can it be assumed, that this number does not contribute to the number of parallel processors that can be consumed for this migration. Is it possible to know what are the number of active parallel processors available at the time the migration activity is performed. BR Dedeepya
Hi Dedeepya,
yes, the parallelization factor (i.e. number of parallel webservice calls) is limited by the maximum possible number of parallel processes reserved for migration. The actual number depends on the system configuration, there is no way to know beforehand how big this number is.
Regards, Marcus
Hi Marcus.
Thanks for sharing.
Is there way to know the bundling factor for business object by any means? Is there any document or through SDK or any other means ?
Thanks
Apoorva
Hi Apoorva,
unfortunately not. The bundling factor is only visible in the backend. I can share it for some common objects though:
BTW: Some objects do not offer bundling or parallelization at all (e.g. Employee)!
Regards, Marcus
hi - Can you please let me know where to configure bundling factor. We have integrated C4C via HCI & webdispatcher. When we do replication of data from C4C to ERP it is occupying large number of dialog work processes. Where do we configure Bundling factor in backend or how do we restrict number of processes used.
Thanks,