Skip to Content
Author's profile photo Marcus Echter

Increasing performance of high-volume data loads via the Migration Workbench

Motivation

When uploading a high volume of data into C4C via the Migration Workbench (MWB) performance is always an important aspect. The MWB supports various ways to support high-volume data loads, the most relevant ones being:

  • Bundling: Multiple instances of an object (e.g. multiple customer records) can be uploaded with 1 webservice call
  • Parallelization: Multiple parallel webservice calls can be performed at the same time

While the bundling factor is hard-coded per migration object (e.g. for accounts it is 10), the parallelization factor can be set by the key user within the MWB before the actual migration run. If it is not set explicitly, the default is 4. The below section describes how to adjust this factor which can result in a significant performance boost.

Configuration

1) After accessing the staging area of the MWB, click on “You Can Also” in the top right corner. There you will find the option “Adjust Settings for Parallel Processing”

MWB_Parallel1.png

2) In the following popup you can adjust the parallelization factor. If you open it the first time you will see the global default, which is 4.

mwb_Parallel2.png

Raise this factor to increase the number of webservice calls which are executed by the MWB in parallel at any point in time. Note that for the customer upload internal tests have shown that a parallelization factor of 30 results in the best migration performance.

Assigned Tags

      6 Comments
      You must be Logged on to comment or reply to a post.
      Author's profile photo Chandan Bankar
      Chandan Bankar

      Hello Marcus,

      Thanks for sharing this. 🙂

      Regards,

      Chandan

      Author's profile photo Dedeepya Reddy
      Dedeepya Reddy

      Great blog Marcus. So the No. of Parallel processes is actually the number of multiple WS calls scheduled for run in Parallel Processing. Can it be assumed, that this number does not contribute to the number of parallel processors that can be consumed for this migration. Is it possible to know what are the number of active parallel processors available at the time the migration activity is performed. BR Dedeepya

      Author's profile photo Marcus Echter
      Marcus Echter
      Blog Post Author

      Hi Dedeepya,

      yes, the parallelization factor (i.e. number of parallel webservice calls) is limited by the maximum possible number of parallel processes reserved for migration. The actual number depends on the system configuration, there is no way to know beforehand how big this number is.

      Regards, Marcus

      Author's profile photo Apoorva Singh
      Apoorva Singh

      Hi Marcus.

      Thanks for sharing.

      Is there way to know the bundling factor for business object by any means? Is there any document or through SDK or any other means ?

      Thanks

      Apoorva

      Author's profile photo Marcus Echter
      Marcus Echter
      Blog Post Author

      Hi Apoorva,

      unfortunately not. The bundling factor is only visible in the backend. I can share it for some common objects though:

      • Activities - 25
      • Customers - 10
      • Materials - 10
      • Leads - 25
      • Opportunities - 25
      • Quotes - 25
      • Tickets - 25

      BTW: Some objects do not offer bundling or parallelization at all (e.g. Employee)!

      Regards, Marcus

      Author's profile photo Sriram Parvataneni
      Sriram Parvataneni

      hi - Can you please let me know where to configure bundling factor. We have integrated C4C via HCI & webdispatcher. When we do replication of data from C4C to ERP it is occupying large number of dialog work processes. Where do we configure Bundling factor in backend or how do we restrict number of processes used.

      Thanks,