Planning the Data Migration Track of an SAP Project

[Former Member]

Introduction

This document is created with an intention to provide all the details possible for planning the data migration track of an SAP project. While it has been written specifically for SAP projects, projects using the information here for other technologies can adapt the information accordingly.

The intended audience of this document is SAP Program managers, Project managers, Data managers who are involved in planning the data conversion track in a project.

Need for Data Migration

Any SAP Project will broadly consist of the following phases

1.       Requirement gathering / Blueprinting

2.       Configuration & Unit Testing

3.       Data Migration

4.       Integrated System Testing

5.       User Acceptance Testing

6.       Go / No Go Decision

7.       Go Live

8.       Hypercare

The Step.3 (Data Migration) is for moving data from the source system (legacy system) onto the target system (which is newly being configured).  The movement of data from the source system(s) to the target system(s) involves

1.       At a higher level, understanding how data drives processes in the legacy and target systems.

2.       At a more details level, field mappings, value mappings, value transformation, and validations.

Terminologies

This section describes some of the commonly used terms in data migration. Understanding these terms is key to understanding the document:

Field mapping:  Mapping the table name – field name of target system to table name – field name of source system.  e.g.

TargetTablename – Fieldname  =   MARA – MATNR

Source Tablename – Fieldname =  pt_mstr – pt_part

The above basically  means that in source system, the values in table pt_mstr field pt_part will move to target table MARA field MATNR

Value mapping:  Once we have the field mapping, then we need to do value mapping.  The below example will make things clear:

    For the field  Location, Value  A   in source system  becomes   Value  X  in Target system

Many times you can have a same value mapping in which case whatever is the value in the source field becomes the value in the target field.

Transformation rules:  The sum of field mapping and value mapping is usually the transformation rule. Although, transformation rules can be more complex.  For e.g. you can have a rule that target value = source value * 2 / 100.  Or you can have a transformation rule like   Location = W if Plant = A and so on.

Mock Run: An exercise of data conversion using the agreed upon mapping and transformation rules. Mock runs are usually done in development and/or quality systems, and are not done on the production system. A mock run will convert a limited agreed upon set of objects and volume of data.

Mock Cutover:  This is the mock run just before the actual final cutover. It should contain the full volume of all objects that are going to be converted for the final cutover. The mock cutover is usually done in a client as close as possible to the production server. It is not done on the production server

Final Cutover: This is the data conversion done in the production server. It is the final conversion which utilizes all the conversion rules, for all the agreed upon objects and for the full volume of data. A final cutover usually involves much more than just data conversion. It will include all the business activities and SAP activities that need to be performed for the Go-Live. Examples of business activities that usually form part of the cutover are:

1.       Stop receiving goods (GR’s).

2.       Perform an inventory count

3.       Reconcile physical inventory count against inventory in system

4.       Provide a list of the reconciled inventory to the project team

Similar to this, there will be many activities performed in each of the departments in order to ensure that the data in the new system matches the physical data.

Examples of SAP activities that are usually performed are:

1.       Take down the system and do a backup of the server before Transport Requests are moved

2.       Block all users from performing any activities.

3.       Stop all background jobs

4.       Stop all interfaces.

5.       Move all transport requests to production server

6.       Restart all background jobs

7.       and so on….

Different Types of Data

The 3 different types of data are given below. The project will have to decide which master data, transaction data and historical data are required to be migrated

Master Data

   This is data that is created once and does not change very often. Business processes depend on master data and transaction data are created based on the master data. Master data is usually shared by multiples users across an organization. Examples are Material Master, Customer Master, Vendor master, Bill-of-Materials, etc

Transaction Data

  These data get created when a business user performs a business transaction. The data usually does not get changed once it is created. Transactions can be reversed or cancelled, but usually not changeable. Transactional data depends on master data. Before a transaction data is executed, proper master data validations are carried out in the background.  Examples are Sales order, Purchase orders, AR open items, AP open items, etc

Historical Data

These are transactional data that are old and closed in the source system. Usually data migrations will convert open transactions into the new system. Businesses will have to take decisions on converting historical data on a case to case basis.  Examples are closed Sales order, closed Purchase orders, etc. It could also include the Open items (AR open items, AP open items).

Planning the Data Track

The data conversion track needs to be planned in context with the overall project plan. Data conversions should be planned such that the output of each of the conversion runs is used for the functional integration testing. 

The mock runs must be planned so as to complete just around when the System Testing begin. This is to ensure that converted data is used in testing as much as possible. This will make testing easy, as well as highlight the correctness of the data conversion. Between 2 mock runs, there should be a reasonable period to allow for defects identification and resolution, as also time to incorporate any change requests.

Project System Landscape

The landscape of the project is crucial to the data migration track. Project landscapes will usually have a development , quality and a production environment.

The development environment (which contains new configuration) is used for build and unit test of data conversions.

The quality landscapes are usually used for the various mock runs. These landscapes should contain configuration of the target system.  The conversion of data for mock runs should be from data in the production server.

The production server is where the final cutover is performed.

It is important to understand and agree upon which data conversions are done in which client and in which system.

Migration Strategy

A migration strategy (or migration methodology) is defined and agreed upon at the start of the project. This methodology is then adhered to throughout the project. A typical migration strategy document will talk about:

1.       Data cleansing within the legacy system before it is extracted

2.       The migration processes like extraction, cleansing, transformation, validation, loading, etc.

3.       The sequence in which these processes are performed.

4.       The source systems involved, the target system involved.

5.       The roles and responsibilities of various stakeholders for each of the activities.

6.       The tools used in migration

7.       The list of migration objects along with business and project ownership

8.       The project plan showing the migration activities in context

Migration Scope

The migration team will have to determine the migration scope of the project and document it. The scope will detail out which all objects are to be migrated along with an estimate of the volume of records for each object

Migration Sequence

Once the scope is determined, then the migration sequence will have to be established. The migration sequence takes into account the dependancies of each of the object, and determines a sequence in which the objects will be migrated. For e.g., you cannot migrate material master before profit centers are created. A sample sequence would look something like this:

1.       GL Accounts

2.       Cost Center Groups

3.       Cost Centers

4.       Profit center groups

5.       Profit centers

6.       Bank Masters

7.       Material Master

8.       Customer Master

9.       Vendor master

10.   Sales pricing records

11.   Purchase Info Records

12.   Source lists

13.   Customer Material Info records…. And so on

Migration Rules document

Each migration object will require a rules document. This can be thought of as a functional specifications document. This rules document will specify among other things:

1.       The name of the object (e.g.  GL Accounts)

2.       The source system

3.       The target system

4.       The document version (creator, date, version number, reviewer, approver, etc)

5.       The migration scope for that particular object

6.       The logic for extracting data from the source. For e.g.  all materials created in last 6 months or all materials used in material documents witha document date in last 1 year, and so on

7.       Data that will be excluded in extraction from the source

8.       The transaction code used for the migration of the object

9.       The field mapping, value mapping and transformation rules of each table-field. An object can have many tables involved (e.g. Material master involves tables MARA, MARC, MBEW, MARM, etc) and all the applicable fields of these tables must be specified in the mapping

10.   The method used to load the object (e.g. BDC recording, Direct Input, BAPI, Idocs, etc)

Validations

Validation is a very important step in the whole data migration process. There are usually 2 points when validations are done:

1.       Preload validations:   Data to be loaded is validated before it is uploaded

2.       Postload validations: Data that got loaded is validated after it is uploaded

In both the cases, validators need to be provided reports of the data that is going to be uploaded / has been uploaded.  It is important to receive a sign-off on each validation before the next object is loaded. This is especially important in the final cutover where migrations become irreversible or very time consuming / tedious to reverse.