SAP Business ByDesign – How to improve Performance of data migration in Migration Workbench
During implementation process of a new Enterprise Resource Planning (ERP) there are plenty of steps involved, the most important initial step is Data migration. The transfer of data from one storage type, format, or previous computer system to the new ERP solution is the most time consuming of all implementation activities. Having a well-defined data migration approach ensures you’ll keep to the implementation timeline, avoid costly budget overruns, and go-live with clean, meaningful data.
SAP Business ByDesign (ByD) is a cloud ERP suite, do provide option to migrate data using Migration Workbench, you are migrating high volume of data into Business ByDesign and you want to improve the current performance data migration. So, you want to know if it’s possible to improve the performance of data migration using Migration Workbench ?
This blog post provides you with an overview on ByD Migration Workbench used for data migration and considers frequently asked question How to improve the performance of data migration in Migration Workbench. By using Parallel Processing feature in migration workbench.
For more information, refer to Guide to the Data Migration Tool in ByDesign Help Center.
Steps followed to in Business ByDesign Migration Workbench:
- Make sure you have created a valid source file using an up-to-date migration template filled with legacy data.
- Go to the Business Configuration -> Implementation Projects view. Select your implementation project and click Open Activity List.
- Select the Integrate and Extend phase, then click the relevant activity (Migration of …) from the Migrate Master Data group or Migrate Transaction Data group.
- Open the respective migration activity and open the migration tool
- Note: user can select Master data like Migration of Material , Service or Transaction data option like Migration of Sales Order, Service Order …
- Note: that data migration requires special user authorization. Your administrator can grant the required authorization (Business Configuration including the Data Migration view) in the Application and User Management work center.
- Upload the filled data migration template file using option Upload -> Data from migration template, user can upload even from text file, or manually fill using the option Actions -> Create.
- select the file entry in table Source Files and click on option You Can Also -> Adjust Settings for Parallel Processing option
- Click on Save and click on Execute Migration
- Note: you can optimize the system capacity during data import by executing different set of data in parallel by entering optimal value in Number of parallel Work Process field. depending on the number of entries in the payload migration input file the values could differ, entering very high value like 1000 will not improve performance by 1000 times, user has to enter optimal value.
- Example use case, in the input migration excel if there are 500 entries , without Parallel Processing option if system take 500 seconds to import data , and user enters Number of parallel Work Process as 5 , then the system will now parallel process data with 5 work process and performance will increase by many folds, the performance should be better compared to previous use case.
- Note: there will be resource limitation in the system with regards to work process and entering the optimal Number of parallel Work Process would improve performance by many folds.
- Note: The optimal value depends on the object and on the actual structure of the data and how large is the data you are migrating.
We recommend you set the value to 16 or 18. This should fit for most of the cases.
Also, only some objects do allow the parallelization, you will find the menu item Adjust Settings for Parallel Processing option disabled if an migration object does not support parallelization.
Important information:
Number of parallel Work Process settings is not used in simulation mode, its only used during actual data migration step. during simulation mode the default value 004 is used and during non-simulation migration step the user defined Number of parallel Work Process is used.
Due to above reason, in case user changes the default value of Number of parallel Work Process, the simulation mode will still use the default value and will be slower compared to non-simulation migration step, which means non-simulation step is faster than simulation step.
In case of high volume data migration , if customer thinks simulation step is slowing down the actual data migration process, then following Data Migration Strategy could help.
- Perform simulation with small data sets, to catch generic data related issues.
- Once the data quality used in migration payload is good
- Make sure you use Small package of data sets with less number of records
- Skip simulation step and migration process will be faster, since simulation step is skipped and productive run ( Immediate import) is using Parallelization with user defined Number of parallel Work Process
- Number of parallel Work Process value depends on lot of factors
- Depth of data in each record, which depends on number of sub-nodes and amount of data in each record
- Number of parallel tasks being executed in the system during migration process
- To come up with optimal Number of parallel Work Process, user could do the following
- Perform migration in to test system with different number of records in each run
- Find the optimal number by trial method in test system and use the same number in productive run in Production system
- Note: The performance improvement achieved by changing the Number of parallel Work Process is logarithmic in nature.
- Note: Delta File will have records which failed to migrate into ByD system( All error records are moved to delta file, delta file can be accessed in the last step of migration ,after clicking on the Finish action) and all records with good data is migrated into system.
For Demo video please refer to Business ByDesign – Parallalization in Migration Workbench.
Thank you Murthy for this post but if I' m honest, it dors not fit my expectations by reading the headline.
I know the feature of parallel provessing for DM but more interesting would be which concrete values have to be set for best performance.
If 16-18 is useful in most case, when I have than to select 18? For 1000 accounts, 10000 or 10000?
I think best would be having a recomendation from SAP like this:
Accounts
<=1000 -> 16
>10000 -> 17
>100.000 -> 18
Sales Orders
<=10000 -> 16
> 50000 -> 17
...
That would be better as that every partner try by its own which values are best.
If you not have all these values you could also work together with partners to get these.
Best Regards
Andreas
Hi Andreas,
very good question, we have enhanced the blog with more details which will address the question.
Please go through the "Important information" section in the blog for more detailed answer.
Regards
Murthy
Hi Andreas,
Thanks a lot for the response and valuable input .
The optimal value of Number of parallel Work Process depends on the object and on the actual structure of the data and how large is the data you are migrating.
I like the recommendation from a consumer perspective is that list of Object types ( Accounts, Sales Orders, Purchase Order, Invoice Request … ) and Proposed number of parallel work process.
Note: the work process is a resource available in the system and most optimal number could only be a recommendation, but the performance of migration depends on the parallel process/activities running at the time of data migration execution. the challenge is coming up with the right number
but very valid and valuable recommendation , we would like to take this point into consideration. thanks for your reply and suggestion.
Regards
Murthy
mr murthy
i'm new comer and I'm new sap by design I used to use SAP business one before and often upload data using template tools Data transfer workbench ( DTW )
my question how to upload in SAP by design and what tools? I see there is Demo data I want to delete it how to do SAP ByD?
Where can I find a reference book for it? Or I appreciate if someone could offer a program sample
thanks Regards ,