Product Information
Data Migration Steps
Data migration is one of the key processes in an SAP implementation. Identifying the right dataset or even identifying unusable dataset, transforming the data into desired format, extracting them from the source system and then finally loading into the SAP system is a long, cumbersome, and error-prone process which requires not only the knowledge of SAP, relevant process and data structure but it also requires extensive knowledge and experience as data architect. Additionally, in today’s world where data is available electronically in the form of many source system be it an Excel-based legacy system or an old ERP implementation or even a modern-day implementation of business functions, it is critical to understand the approach of data migration and even more critical is the buy-in from sponsor and the key stakeholders of the project.
In this blog, we will look into the different steps of data migration. The purpose of this blog is not to drill upon individual steps rather it will identify the necessary steps, any pre-requisites or dependencies with the other steps, ownership and the phase in which the given activity will be carried out. The blog can be used as a playbook for the data migration from one source system to S4HANA. Below diagram identify the source and target system.
The migration steps can be broadly categorized into
- Migration Approach
- Planning
- Execution
- Testing & Validation
- Closure
Migration Approach
The first step in any data migration initiative, irrespective of the driving force of the migration, is deciding the approach. The approach can be either a big-bang or incremental. There are advantages & disadvantages of each approach. Any discussion related to the approach is beyond the scope of this blog however, it is imperative to note the following differences between the two approaches.
Big Bang Data Migration | Incremental |
· Complete migration in one go · Requires downtime window · High Risk · Migration can complete within a short amount of time · Cost effective* · Strong verification & validation strategy · No Parallel run needed |
· Phased approach based on the need/priority · It doesn’t require any downtime · Low Risk · Long timeline to complete the migration · Costly* as compared to big-bang
· Strong verification & validation strategy · Two system will run in parallel |
*The comparison is made only for the direct cost of migration.
Planning
Now, when we know about the source system we can start planning the approach to the data migration. The data migration approach and planning are the key to success. Depending upon the number of factors the approach for the migration will different. However, from the planning purpose, the timeline and milestones for the migration would be critical. The simplest scenario is a greenfield implementation. Another scenario is when the client is retiring another ERP system say a financial system and moving into SAP S4HANA. In yet another scenario, when company A, which prefers SAP S4HANA, acquires company B and retires the company B’s old but functional systems. In all of these scenarios, data migration is a must.
As in the diagram below, the data migration planning requires many inputs; they are
- Project Management Plan
- Enterprise Constraints
- Risk Management Plan
- Project Charter
- Project Scope Statement
The outputs of this planning process are:
- Data Migration Plan
- Updated Project Management Plan
- Updated Risk Management Plan
- Updated Risk Register
Execution
During this part of the process, the experts will identify the data from the source system that needs to be migrated. It is important that the experts from the target system must review those datasets to ensure that their needs are met. Often times, I have seen that the review process is not robust costing the client heartache and headache at the 11th hour. Additionally, this review process will help the team migrate only the required data and in the right format. For one of the major Canadian bank, the review team ignored the column header that was part of the migrated data. At another bank, the date was provided in the YYYYMMDD format instead of MMDDYYYY.
It is mandatory to do the gap analysis at this phase of the migration. The purpose of this analysis is to identify the gaps between what can be delivered and what is desired. Those gaps either can be filled or should be accepted as a gap.
Testing & Validation
The date format or the row header issues, as mentioned above, were identified at this phase. At yet another bank, we faced a major challenge in terms of date formatting. Data from some of the fields were extracted in MM/DD/YYYY format while others were in DD/MM/YYYY format. What made the situation complex was the fact that the test data was provided only for the 1st 10 days of the month and the test load was successful. The lesson learned was that the test data must be random and should have sufficient breadth.
I also believe that the data migration process should be executed as a Financial System where at every step the auditors can audit. What I mean here is several levels of reconciliation should happen as a part of the plan. The first level of recon should be a simple sum of a numeric field, a number of rows, minimum/maximum of a date field etc. It should be compared every time the dataset is extracted to a file, transfer the file from one server to another, loaded the data in the staging area and when the data is pulled from the staging area into the target system. The second set of recon should more complex such as executing a complete workflow or a report and comparing the output from the source system as well as the target system. The final level of recon should be more at the system level by executing a complete function such as MRP and comparing the results.
Closure
Most of the experts feel that this part of the process is not required. However, in this part of the process, you ask for signoff from the clients, in writing. I believe this is the most critical aspect of data migration.
Below is the list of key steps along with the owner of those steps are identified.
Steps | Primary Owner | Secondary Owner | Phase |
High Level data Migration Plan | Vendor | Client | Prepare |
Detail Data Migration Plan | Vendor | Client | Explore |
Approve Data Migration Plan | Client | NA | Explore |
Data Migration Templates for Master & Transaction Data | Vendor | Client | Realize |
Identify Data Conversion Fields | Vendor | Client | Realize |
Field Mapping | Vendor | Client | Realize/Deploy |
Gap Analysis (Gap Report) | Vendor | Client | Realize/Deploy |
Dependency Analysis (Dependency Report) | Vendor | Client | Realize/Deploy |
Plan to populate Missing Information | Client | Vendor | Realize/Deploy |
Provide data in the specified format | Client | Vendor | Deploy |
Mock Data Load | Vendor | Client | Deploy |
Identify Errors in data load | Vendor | Client | Deploy |
Fix the errors | Client | Vendor | Deploy |
Exception Report | Vendor | Client | Deploy |
Repeat Data Load & error fixing | Respective | Respective | Respective |
Provide Final Data | Client | Vendor | Run |
Load Final Data | Vendor | Client | Run |
Provide Delta Data (in case of the parallel run) | Client | Vendor | Run |
Load Delta Data (in case of the parallel run) | Vendor | Client | Run |
Provide data load confirmation | Client | Vendor | Run |
Irrespective of the approach – Big bang or Iterative – the steps within are repetitive as shown below.
In nutshell, data migration is a clumsy process but has useful results if done with the right approach and with proper planning.
Nice Blog Mr.Aditya Lal.
Thanks for sharing Data Migration steps.
Mr. Aditya Lal - another good blog from you. I would be interested to see an article on data load process. Thanks
Magnificient!!!!
Detailed steps are great. Do you have any templates for the steps or plan?
Adi - wow - great write up!!!
Very well written steps. Do you have the template for Gap Analysis? Is it possible to share with me?
Thank you, a blast article. Cheers
That's a nice and informative blog. Looking forward for more such blogs. Great to have you in SAP Community.
Mr. Lal, Thank yo for this another article. It’s a great one. I will look forward for your articles
I have little bit confusion about data migration steps but after going through your detailed explanation about data migration, now it is very clear.
Tq for such a nice blog.
Very nice and informative blog.
Again Good One.
I like the way you describe SAP.
Thanks for this blog. I would like to see more articles from you.
HI Stephanie you can see his more blogs at Aditya Lal
Nice one Aditya!!!
This one is also good Aditya. Can you post more blogs related to SAP along with more information.
Hi Lily please visit at https://people.sap.com/aditya.nlal05#content Thanks 🙂
Adi great writeup again.
I got some great points from this article. Thanks
Thanks for sharing this article.
An awesome blog Mr. Lal! Its very well written and wonderfully explained.
Very Well done and keep it up
Good article on Data Migration.
I got some great points from this article. Thanks Aditya
Mr. Lal, Thank yo for the article. It’s a great one.
Do you have more blogs?
Thank You Aditya for this aricle. It’s a great one.
I was looking these kind of blogs. I read your two blogs and both are well written and well explained blogs.
I have checked lots of blogs related to Data Migration but those are not well explained but your all blogs are well explained and step by step. Thanks and keep it up.
Good One Aditya.
Great Mr. Aditya ji. Its really help me. Can you please write more blogs.
For other posts at Aditya Lal. Please use this for more information.
Amazing Mr. Lal. I got complete information from this post. Very clear information. I have shared this post to my friends and colleagues as well.
Really great post. It was a very clear and detailed post.
Great post. They have other posts at Aditya Lal. Please use this for more information.
Very awesome article. I like the way you explained each and everything.
Excellent article Mr. Aditya. Keep it up
Very Nice Post Mr. Aditya. Its really help me in project.
Very nice article on Data Migration.
Awesome write up. I liked your post.
wow nice Article.
Amazing writing skill Aditya. I like the way you explained your knowledge.
Very nice Aditya.
Awesome post Aditya Ji. I like it.
Very well detailed post.
Hi Aditya,
Really Nice Blog with lot of information. Besides I need to know that there is a scenario where Hardware change has to be done. Below are the points identified :-
My concern here is about the adaption of methodology
1. Can we go with File system copy from the source to the target because here only one time copy and paste will be done and bit change is profile parameter about system name and server name change. if this approach is applicable then will the HP-UX kernel binaries will work on Linux. or while copying the kernel binaries we should copy the linux version binaries in the target system.
2. or we have to go with SWPM tool system import and export because here we have a system downtime more if the DB size is more than 10TB than a first process mentioned to you will be acceptable by the clients as they don’t want to loose business.
Your suggestion is required in this regard.
Best Regards,
Pankaj Kumar
<personal info removed by moderator>
Hi Aditya,
Excellent blog & nice overview.