Data Migration for an Auto OEM in a complex IT landscape
In any implementation or a roll-out of an SAP template, one of the first and the most critical activity is the ‘data migration’. Data Migration is the process of transferring the business critical data from a legacy system to the ERP system (SAP system is taken as a case in point in this document), which is imperative for the continuation of the business once the legacy system is down and the SAP system starts getting used in the day-to-day business activities. The data needs to be migrated to the SAP system in such a way that it is exactly in a desired state as it was in the legacy system at the time of migration, and which could be processed later once the SAP system goes live. This document basically covers one of the migration approaches that could be followed for an automotive OEM at the time of an SAP implementation or a roll-out in a particular country or an NSC (national sales center). The basic assumption is that typically in the automotive space, the landscape is quite complex. Typically, the automotive industry was amongst the early ones to adopt IT. Hence, there could be many IT systems in their landscape which might not be as integrated as desired, which means when transitioning to a more integrated system like SAP, the data source is disparate.Hence, the challenge at the time of migration is to ensure that the data is effectively fetched from the external systems to the SAP system and the data sanctity is maintained.
In the automotive industry, the IT landscape might typically look as below:
- Ø Legacy System, which is the main source of the vehicle data. This is being replaced by the SAP system. It is assumed that the legacy system is an ordering system which is dependent on the factory systems to book the orders in the plant.
- Ø Financial System, which is assumed to communicate with the SAP system once the legacy system is replaced. It is assumed that the financial system is the source of the customer and vendor master data, in addition to the vehicle accounts data. There is also a possibility that other external systems could act as master data sources.
- Ø Master data sources, which provide the material master data or configuration related data. This can exist in case there is a centralized dedicated system to supply the master data related to the vehicle models for application in the IT landscape.
- Ø Factory Systems (which are the manufacturing systems), which are the source for the latest production status related data. In automotive space, typically the factory systems register the data from the ordering system and provide real time production status related data back to the ordering system. The ordering system communicates with the factory systems on a regular basis to book the vehicles for production and receives updates on the order back from the manufacturing system.
- Ø Local systems, which exchange data with the legacy system to address the country specific needs.
During the implementation or roll-out of the SAP template, the legacy system is being replaced by the SAP system. The legacy system is the source of all the order data which needs to be transferred to the SAP system. Also, in addition to this, data needs to be fetched from the different sources shown to ensure that sanctity is maintained. The complexity might lie in the fact that data needs to be fetched from the different sources into the SAP system, staged, validated and then processed to ensure it is in sync with the different sources.
Data Exchange between systems in the landscape
As part of migration, there can be 2 types of data that need to be migrated or maintained in the SAP system – Master Data and Transaction Data. The sources for these could be different. There might be a need for the master data to be migrated before the transaction data.
- Ø Legacy system to SAP system: Typically, the legacy systems provide the data in the form of external files which are then manually uploaded into the SAP system using custom programs or LSMW. The files could be in any desired format such as text files, excel files, etc. Alternatively, a communication channel between the legacy system and SAP system could be established in order to transmit the data. The uploaded or transmitted data in SAP is first stored in intermediary staging tables, checked for the sanctity using validation programs, and subsequently used in the data processing.
- Ø Factory system to SAP system: The factory system, which could be a non-SAP system, typically interacts with the ordering system via PI. In order to receive the latest information from the factory, the messages sent by the factory are converted into idocs and then transmitted to SAP. This data could be stored in the staging tables to be accessed by the migration programs during data processing or the migration programs could directly retrieve the data from the idocs, and use it in the data processing.
- Ø Master Data source to SAP system: In case there are multiple master data sources in the landscape, communication with the SAP system could be through web service calls for instant exchange, and also idocs via PI in order to exchange the master data. In certain cases, the vehicle master data is received from these systems against which the material master is created in SAP. This is as part of the master data set up before the migration run.
- Ø Financial system to SAP system: It is possible in certain scenarios that the debtor and creditor master data is sent from the finance system to the SAP system. This is as part of the master data set up. This could be done using the CREMAS/DEBMAS structures, and any additional information for customers or vendors could be uploaded using the LSMW objects.
Data Migration Process
The whole data migration process can be split into the 4 mentioned stages:
- Ø Pre-Migration: As part of this stage, the data migration check list is verified and the readiness is gauged, both from the legacy system and SAP system point of view.
- Ø Preparation: All the necessary data from different sources is loaded into the staging tables in SAP and validated against a pre-defined set of rules.
- Ø Execution: This is when the data which is staged is actually processed and the order data is taken to a point in SAP where it is in sync with that of the legacy system.
- Ø Post Run: The post data migration reporting is done as part of this stage.
Steps in Data Migration
The above figure shows the modularized phases in the data migration activity, both from the legacy system data preparation point of view and the SAP system migration activity point of view.
Ø Legacy System
- Data Extraction: All the data that is in scope for migration is finalized and the data is extracted in the desired format. This could be in the form of text files, excel files or any other data format that is agreed.
- Basic Validations: Once the data is extracted, there is a sanity check that is done in order to ensure that the data being provided to the SAP system is correct. This could be done by certain rule based programs in the legacy system.
Ø SAP System
- Data Staging: As part of this step, data from multiple sources (legacy system, factory systems, etc) is fed into the SAP system. This is just the staging of data before migration where in the data is stored in intermediary staging tables. This data will include all the latest order and distribution related information (including documents related data), which is subsequently used for the next steps.
- Data Validation: Once the complete data which is in scope of migration is staged in the SAP system, it needs to be validated to ensure that the data meets the criteria for the migration process to happen smoothly, and also post-migration, the data could be processed further. The validation could be in the form of certain custom programs with pre-defined rule sets. This is a very critical step in the migration process as data sanctity is dependent on this step.
- Error Handling after Validation: After data validation, errors in the data provided by the legacy system could be reported back and correct data could be re-provided after fresh data extraction.
- Migration Run: Once the necessary data is all staged and sanity checks done, the migration activity could be initiated. The creation of all the data in SAP could be done by using custom programs or by using LSMW objects.
- Results and Error Reporting: Once the migration activity is complete, the results and the error list could be checked using custom reports or job spools.
- Post Migration activities: This step involves all the post migration activities, like output type processing, idoc processing, etc.
It is imperative to follow a structured way with different process steps as the data which is to be migrated could be huge. Checklist could be prepared and this needs to be monitored to ensure sequential execution of steps and hassle free data migration process. Also, an optimal number of test-migration runs, involving all the systems in the landscape, need to be conducted in order to ensure the below:
- Data which is being provided by the different systems in the landscape is correct.
- Sequence of steps which are to be executed are finalized for the main production activity.
- All the custom programs and objects which are being used at various stages for data validation, data creation and data processing are correct and the final data which is being stored in the SAP system is as desired.
As part of the main migration cut-over activity, some of the challenges that could crop up are:
- The first challenge could be related to finalizing the scope of the migration. It needs to be done in such a way that it is optimal enough to ensure the business continuity, and at the same time not large enough to ensure a smooth cut-over.
- With many systems that could be involved in the landscape during the cut-over/migration activity, it needs to be ensured that the interfacing with the systems is properly set-up an the data being provided by the external systems is as desired.
- One of the main challenges could be the system downtime. The production migration activity begins once the existing legacy system is brought down (with no further ordering process), data is extracted and provided to the SAP system. It needs to be ensured that the time frame between bringing down the legacy system and bringing up the SAP system for business is as short as possible.
One of the ways to address the challenges mentioned above could be to have enough number of pre-production migration runs (based on the project timelines). This will ensure that all the sanctity of all the necessary data being provided by the external systems is maintained and the interfacing with all the external systems is tested. Also, it is imperative that after every pre-production run, the migrated data needs to be processed and taken forward further in the life cycle. This will ensure that the data is being migrated correctly and will ensure business continuity once the SAP system goes live.
About the Author
Vinod Sripada is an SAP SD Consultant in Infosys Ltd. He has 2 years of domain experience in sales and marketing vertical of an Indian automotive major, and 6 years of SAP SD experience in the automotive vertical. He has worked extensively on industry specific SAP solution of IS-Auto. He holds a B.Tech degree in Computer Science from JNTU, Hyderabad followed by MBA from Nirma University, Ahmedabad.