Planning the Data Migration Track of an SAP Project
This document is created with an intention to provide all the details possible for planning the data migration track of an SAP project. While it has been written specifically for SAP projects, projects using the information here for other technologies can adapt the information accordingly.
The intended audience of this document is SAP Program managers, Project managers, Data managers who are involved in planning the data conversion track in a project.
Need for Data Migration
Any SAP Project will broadly consist of the following phases
1. Requirement gathering / Blueprinting
2. Configuration & Unit Testing
3. Data Migration
4. Integrated System Testing
5. User Acceptance Testing
6. Go / No Go Decision
7. Go Live
The Step.3 (Data Migration) is for moving data from the source system (legacy system) onto the target system (which is newly being configured). The movement of data from the source system(s) to the target system(s) involves
1. At a higher level, understanding how data drives processes in the legacy and target systems.
2. At a more details level, field mappings, value mappings, value transformation, and validations.
This section describes some of the commonly used terms in data migration. Understanding these terms is key to understanding the document:
Field mapping: Mapping the table name – field name of target system to table name – field name of source system. e.g.
TargetTablename – Fieldname = MARA – MATNR
Source Tablename – Fieldname = pt_mstr – pt_part
The above basically means that in source system, the values in table pt_mstr field pt_part will move to target table MARA field MATNR
Value mapping: Once we have the field mapping, then we need to do value mapping. The below example will make things clear:
For the field Location, Value A in source system becomes Value X in Target system
Many times you can have a same value mapping in which case whatever is the value in the source field becomes the value in the target field.
Transformation rules: The sum of field mapping and value mapping is usually the transformation rule. Although, transformation rules can be more complex. For e.g. you can have a rule that target value = source value * 2 / 100. Or you can have a transformation rule like Location = W if Plant = A and so on.
Mock Run: An exercise of data conversion using the agreed upon mapping and transformation rules. Mock runs are usually done in development and/or quality systems, and are not done on the production system. A mock run will convert a limited agreed upon set of objects and volume of data.
Mock Cutover: This is the mock run just before the actual final cutover. It should contain the full volume of all objects that are going to be converted for the final cutover. The mock cutover is usually done in a client as close as possible to the production server. It is not done on the production server
Final Cutover: This is the data conversion done in the production server. It is the final conversion which utilizes all the conversion rules, for all the agreed upon objects and for the full volume of data. A final cutover usually involves much more than just data conversion. It will include all the business activities and SAP activities that need to be performed for the Go-Live. Examples of business activities that usually form part of the cutover are:
1. Stop receiving goods (GR’s).
2. Perform an inventory count
3. Reconcile physical inventory count against inventory in system
4. Provide a list of the reconciled inventory to the project team
Similar to this, there will be many activities performed in each of the departments in order to ensure that the data in the new system matches the physical data.
Examples of SAP activities that are usually performed are:
1. Take down the system and do a backup of the server before Transport Requests are moved
2. Block all users from performing any activities.
3. Stop all background jobs
4. Stop all interfaces.
5. Move all transport requests to production server
6. Restart all background jobs
7. and so on….
Different Types of Data
The 3 different types of data are given below. The project will have to decide which master data, transaction data and historical data are required to be migrated
This is data that is created once and does not change very often. Business processes depend on master data and transaction data are created based on the master data. Master data is usually shared by multiples users across an organization. Examples are Material Master, Customer Master, Vendor master, Bill-of-Materials, etc
These data get created when a business user performs a business transaction. The data usually does not get changed once it is created. Transactions can be reversed or cancelled, but usually not changeable. Transactional data depends on master data. Before a transaction data is executed, proper master data validations are carried out in the background. Examples are Sales order, Purchase orders, AR open items, AP open items, etc
These are transactional data that are old and closed in the source system. Usually data migrations will convert open transactions into the new system. Businesses will have to take decisions on converting historical data on a case to case basis. Examples are closed Sales order, closed Purchase orders, etc. It could also include the Open items (AR open items, AP open items).
Planning the Data Track
The data conversion track needs to be planned in context with the overall project plan. Data conversions should be planned such that the output of each of the conversion runs is used for the functional integration testing.
The mock runs must be planned so as to complete just around when the System Testing begin. This is to ensure that converted data is used in testing as much as possible. This will make testing easy, as well as highlight the correctness of the data conversion. Between 2 mock runs, there should be a reasonable period to allow for defects identification and resolution, as also time to incorporate any change requests.
Project System Landscape
The landscape of the project is crucial to the data migration track. Project landscapes will usually have a development , quality and a production environment.
The development environment (which contains new configuration) is used for build and unit test of data conversions.
The quality landscapes are usually used for the various mock runs. These landscapes should contain configuration of the target system. The conversion of data for mock runs should be from data in the production server.
The production server is where the final cutover is performed.
It is important to understand and agree upon which data conversions are done in which client and in which system.
A migration strategy (or migration methodology) is defined and agreed upon at the start of the project. This methodology is then adhered to throughout the project. A typical migration strategy document will talk about:
1. Data cleansing within the legacy system before it is extracted
2. The migration processes like extraction, cleansing, transformation, validation, loading, etc.
3. The sequence in which these processes are performed.
4. The source systems involved, the target system involved.
5. The roles and responsibilities of various stakeholders for each of the activities.
6. The tools used in migration
7. The list of migration objects along with business and project ownership
8. The project plan showing the migration activities in context
The migration team will have to determine the migration scope of the project and document it. The scope will detail out which all objects are to be migrated along with an estimate of the volume of records for each object
Once the scope is determined, then the migration sequence will have to be established. The migration sequence takes into account the dependancies of each of the object, and determines a sequence in which the objects will be migrated. For e.g., you cannot migrate material master before profit centers are created. A sample sequence would look something like this:
1. GL Accounts
2. Cost Center Groups
3. Cost Centers
4. Profit center groups
5. Profit centers
6. Bank Masters
7. Material Master
8. Customer Master
9. Vendor master
10. Sales pricing records
11. Purchase Info Records
12. Source lists
13. Customer Material Info records…. And so on
Migration Rules document
Each migration object will require a rules document. This can be thought of as a functional specifications document. This rules document will specify among other things:
1. The name of the object (e.g. GL Accounts)
2. The source system
3. The target system
4. The document version (creator, date, version number, reviewer, approver, etc)
5. The migration scope for that particular object
6. The logic for extracting data from the source. For e.g. all materials created in last 6 months or all materials used in material documents witha document date in last 1 year, and so on
7. Data that will be excluded in extraction from the source
8. The transaction code used for the migration of the object
9. The field mapping, value mapping and transformation rules of each table-field. An object can have many tables involved (e.g. Material master involves tables MARA, MARC, MBEW, MARM, etc) and all the applicable fields of these tables must be specified in the mapping
10. The method used to load the object (e.g. BDC recording, Direct Input, BAPI, Idocs, etc)
Validation is a very important step in the whole data migration process. There are usually 2 points when validations are done:
1. Preload validations: Data to be loaded is validated before it is uploaded
2. Postload validations: Data that got loaded is validated after it is uploaded
In both the cases, validators need to be provided reports of the data that is going to be uploaded / has been uploaded. It is important to receive a sign-off on each validation before the next object is loaded. This is especially important in the final cutover where migrations become irreversible or very time consuming / tedious to reverse.