Skip to Content
Product Information
Author's profile photo Zsombor Danka

Migration Cockpit – Parallelization of XML file loads

Hello SAP S/4HANA Cloud Community,

this would be my first blog here, when I would like share some insight on how to simultaneously transfer multiple XML files with the S/4HANA Migration Cockpit. It should improve the throughput of the data load during migration.

Loading one large XML file may take up too much time and the system doesn’t parallelize the processing by default. Therefore If you would like to achieve parallel processing keep reading! To introduce this functionality for your migration object, perform the following steps:

1. Instead of creating one big file, split your data in several (n) files.
2. Open fiori app “Migrate Your Data” and the object where you would like to introduce parallelization
3. Upload all (n) files and set the status to β€œactive”
4. Click “Edit” and in the field “Max. Data Transfer Jobs” enter a number of jobs (m) higher than 1 in order to transfer (n) files in parallel. Note: m <= n.

The number entered here will allow the system to trigger multiple jobs for data transfer. Therefore if several files are being transferred at the same time, those will be processed by the system in parallel if this attribute (“Max. Data Transfer Jobs”) is higher than 1. Otherwise only a single load job will be used to process all files.

Please note that your system is configured with a certain number of available batch job processes. For S/4HANA Cloud, the migration cockpit will only allow up to 80% of these processes (number m) to be used for the migration object.

Avoid running parallel processing using several migration projects and objects at the same time if the total number of batch jobs exceeds the available system resources.

Example:
You have split one large file into three smaller files, and you have set the “Max. Transfer data jobs” to value ‘3’. The SAP S/4HANA migration cockpit will try to migrate all three files in parallel using background jobs.

Restrictions:

  • This feature is not supported for migration object “Cost Center” because of the hierarchy group creations.
  • When splitting the data to multiple files, note the requirements detailed in KBA 2719524 for the XML size limit.
  • It is not recommended to migrate multiple migration objects simultaneously using several projects as this will only occupy job resources.
  • In SAP S/4HANA Cloud the number of available batch processes is normally 10 per instance (AppServer).
  • Usually, for quality environments, there is one AppServer (10 jobs available) and for productive environment 2 Application Servers (20 Jobs available). The migration cockpit will only allow up to 80% of these processes to be used for the migration object.
  • If you try to migrate several migration objects in parallel by using multiple migration projects, the system could run out of batch job processes and will queue the jobs. The result will be that the migration remains at a low percentage level because jobs are waiting in the queue to be executed.
  • We therefore recommend that you avoid running too many batch jobs in parallel – this will not improve the performance of the data migration.

Further Recommendations:

  • Do not start the migration of all objects in a migration project in parallel. There is a sequence for migrating objects (because of the dependencies between the migration objects). The sequence of the migration objects can be found in the migration object documentation.
  • Parallelization is meant for migration objects for which high data volumes need to be migrated.
  • We recommend preventing a situation where many people are working in parallel on separate migration projects, or where one user triggers multiple activities for several projects in same data migration context. Uploading multiple objects in parallel can flood the job queue. In our experience, it delivers better results to migrate one object with full number of jobs, complete the data migration and then start with the migration of the next object. Do this instead of distributing the number of free jobs and start several objects with high data volumes in parallel.
  • Distributing the migration objects into different migration projects will not help regarding performance. On the contrary, you will lose the advantage of the central cross-object value mapping that the migration cockpit provides.
  • Even if you assign a higher number of data transfer jobs that could run in parallel, the effect in the performance depends on the overall load of the system. If you have leveraged parallelization and have considered the recommendations above and still run into performance issues, you should know how many additional activities are going on in the system that also fills the process queue.
  • If you experience performance problems, consider executing the data migration in a time period when the system has a low workload.

Kind Regards,
Zsombor Danka
SAP S/4HANA Cloud Product Support

Assigned tags

      17 Comments
      You must be Logged on to comment or reply to a post.
      Author's profile photo Sandeep Kumar
      Sandeep Kumar

      Zsombor Danka very useful information and can be used where performance issues are in place for data migration.

      Author's profile photo Kelly Hannel
      Kelly Hannel

      Thank you Zsombor! Very good and useful information.

      Author's profile photo Owen Liu
      Owen Liu

      Thanks for the sharing! Zsombor Danka @zsombor

      Author's profile photo Norbert Birtalan
      Norbert Birtalan

      Very useful, thanks for the detailed information.

      Author's profile photo Former Member
      Former Member

      Thank you Migration master!;)

      Author's profile photo Former Member
      Former Member

      Highly appreciate the above information, thanks a lot.

      Author's profile photo Feras Al-Basha
      Feras Al-Basha

      Great blog ! thank Zsombor Danka

      Author's profile photo Katalin Csengodi
      Katalin Csengodi

      Good to know! Thank you for sharing, Zsombor πŸ™‚

      Author's profile photo Eric Yu
      Eric Yu

      Thanks for sharing this useful information. Zsombor.

      Author's profile photo Scarlett Wang
      Scarlett Wang

      Good to know πŸ™‚ Thanks Zsombor Danka

      Author's profile photo Sai Giridhar Kasturi
      Sai Giridhar Kasturi

      thanks for posting. very good share.

      Author's profile photo Varun Agarwal
      Varun Agarwal

      Great Blog !

      Author's profile photo Marissa Ren
      Marissa Ren

      Nice blog! β€Ž#Data_Migration_Cockpit

      Author's profile photo Vijayendra Tiwari
      Vijayendra Tiwari

      Good one! Thanks

      Author's profile photo Priyank Kumar Jain
      Priyank Kumar Jain

      Extremely useful..thanks for sharing!!

      Author's profile photo Gabriel Rossano
      Gabriel Rossano

      Thanks a lot for sharing!!

      Kind regards.

      Author's profile photo Former Member
      Former Member

      Thanks!!!