Migrating data to your (new) SAP S/4HANA

This blog discusses the ways in which your business data can be migrated to your newly deployed SAP S/4HANA system (on premise or cloud).

This is not the first blog about the topic, but I hope it offers slightly different approach/perspective than the existing ones hence decided to post it. Researching for it has certainly helped me understand this space and I hope it will also help you. Please refer to the end of the article for links to other blogs and information sources about this topic.

The blog will be primarily focusing on New System Implementation transition scenario as illustrated:

In this scenario, you decided to deploy new SAP S/4HANA system and migrate selected data from your legacy system. By the way – that legacy system could be an SAP R/3 or ERP system or any 3^rd party system. If it is an SAP ERP system, then you could transition to SAP S/4HANA (on premise) using Conversion scenario, but you have decided not to.

If the above sounds like your case, read on…

In the context of this article, we focus on migration of business data and not system data, like for example configuration.

The New System Implementation scenario supports two distinct targets – SAP S/4HANA (lack of any suffix indicates on premise edition) and SAP S/4HANA Cloud.

Information Management

But we are getting ahead of ourselves – before talking about how the migration can be done, it is prudent to discuss how data should be prepared for the migration. Broadly, the data in your current environment can be classified into following buckets:

1. Good quality data which must be migrated to the new transactional system,
2. Good quality data which should be retained for analytical and/or auditing requirements.
3. Poor quality data which must be migrated to the new transactional system,
4. Poor quality data which should be retained for analytical and/or auditing requirements.
5. Data which is no longer required.

In this article, we are focusing on data which must be migrated to the new SAP S/4HANA transactional system – represented by buckets 1 and 3 above. So, before we move on (and after discarding bucket 5), what options do we have for buckets 2 & 4? Following could be considered:

• Use of corporate memory concept with analytical capabilities of SAP HANA and/or SAP BW (on HANA or BW/4HANA) combined with data management and tiering capabilities like Data Tiering Optimization, Native Storage Extension, Near-Line Storage, Dynamic Tiering and integration with Hadoop or other repositories.
• Use of SAP Information Lifecycle Management (ILM) and the retention warehouse – refer to https://www.sap.com/products/information-lifecycle-management.html for more information.

In essence, what you should have in place prior to undertaking major migration project should be a strategy or at least a vision for all kinds of information your organization manages. If you aren’t sure where to start, check the blog on Information Governance Model.

Now, we are left with buckets 1 & 3 to deal with. The only difference between them is the quality of the data – the poorer quality, the better tooling and more effort is required to cleanse and/or enrich the data and it may impact the decision regarding choice of migration tools.

The Options

The matrix below shows tools available in various data migration scenarios:

Let’s get the elephant out of the room straight away – the notable absence of Legacy System Migration Workbench (LSMW) from the matrix above. As per note 2287723 – LSMW in SAP S/4HANA on premise edition:

The use of LSMW for data load to SAP S/4HANA is not recommended and at the customer’s own responsibility. Instead, the SAP S/4HANA migration cockpit is SAP’s solution for data migration to SAP S/4HANA. You should always check if the object is available with the SAP S/4HANA migration cockpit before using the LSMW.

If you still use the LSMW, you have to carefully test the processes so that you can ensure that it is actually working for you. It might not work for you and in any case.

Expect restrictions around transaction recording (as this is not possible with the new SAP Fiori screens) and changed interfaces (for instance the Business Partner CVI). Standard Batch Input programs may also no longer work as transactions may have changed functionality or may be completely removed within some areas. For example, due to security reasons batch import has been limited for RCCLBI02 program. Also, transactions for customer master data (FD*/XD*) and vendor master data (FK*/XK*) cannot be used anymore due to the change to the Business Partner data model in SAP S/4HANA.

Needless to say that LSMW is not available at all in Cloud edition.

For the reasons above, we will not be discussing LSMW in this article. We will be discussing two set of tools and approaches – using built-in SAP S/4HANA Migration Cockpit and Object Modeler and rapid data migration content for SAP Data Services.

Please also refer to the following blog: Comparison: Migration Cockpit, Rapid Data Migration (with Data Services) and LSMW.

SAP S/4HANA Migration Cockpit (MC) and Migration Object Modeler (MOM)

For an easy introduction to SAP S/4HANA MC refer to this video (also check links at the bottom of this blog for more videos).

SAP S/4HANA Migration Cockpit (referred to as MC hereafter) is a new migration tool that is shipped exclusively with SAP S/4HANA – it was initially available for Cloud edition, but since SAP S/4HANA 1610 it is also shipped with on premise edition.

It is accompanied by Migration Object Modeler (referred to as MOM hereafter), which is a design tool for enhancements and modifications of pre-defined migration objects. We will get back to it a little bit later.

Since both of these tools are new, let’s elaborate a bit on their structure and capabilities to give you a better idea on what to expect of them.

MC is delivered with standard deployment of SAP S/4HANA – that is no additional add-ons or special UI activation activities need to take place after you have installed, upgraded or converted to SAP S/4HANA 1610 or higher. It can be launched using Manage Your Solution Launchpad in case of the Cloud edition or using tx. LTMC in case of On-Premise. The MC itself has a browser based (WebDynpro) interface. There are following concepts employed within MC:

• Migration Object Migration Objects are defined and delivered by SAP, and describe how to migrate data from the source system (which tables are needed and the relationships between the tables) to SAP S/4HANA. Custom Migration Objects are not yet supported, but on the roadmap – refer to section on Migration Object Modeler further in this blog.
• Migration Project Migration Projects are used to facilitate the transfer of data from a source system to SAP S/4HANA. In order to migrate data to SAP S/4HANA, you must first create a migration project. You use a migration project to specify the source system and the data that you want to transfer, and to monitor the status of the migration.

Migration Project provides what is based described as “organizational layer” – it allows for grouping of the migration activities to suit project needs. Examples of criteria used to define separate projects (examples only, can be combined):

• source systems,
• company organizational structure (like company code),
• languages,
• phases of deployment, etc.

Migration Project has a transfer ID associated with it – concept taken from System Landscape Transformation capabilities. The transfer ID acts as unique identifier per project in order to facilitate transfer of project specific settings (including value mappings) between environments in the landscape – for example between QA and Production systems (which is performed using the Export/Import Content function and not using Change & Transport System (CTS)).

Once Migration Project has been created, you can choose which Migration Objects will be utilized within that Project. On the Project’s overview screen, all available Objects are presented with following information:

• object Name,
• Progress – this at the start shows 0%. If the value is higher, it is an indication that particular object(s) has been copied to the Project’s specific repository and potentially other activities have been started.
• Documentation – link to object’s specific documentation,
• Dependent Migration Object – other migration object which should be loaded first or already present in the system. Depending on version of your S/4HANA system, the completeness of that information may vary – that is not all dependent objects may be listed, especially in older versions.

Once you pick particular object for the first time, the associated object template will be copied to the Migration Project – what that means is that this object’s standard template and transfer rules are copied. It should be noted that the template is only applied at the time of copy. This has following consequences:

• Changes to object template (for example delivered with new software update) do not affect Migration Objects within existing projects. Only newly created Migration Objects based on updated template will inherit changes in the new version of the template.
• Changes to Migration Objects within existing projects do not affect the object template.

Now, that you have selected your Migration Object(s) within the Migration Project, you can do the following:

• Access Migration Object documentation.
• Download template – the file, which you will need to populate with your legacy data.
• Upload file – the template above filled with data from your legacy system(s). You can upload several files at the same time, which allows you to control scope of migration on the file level. This action will load the data from the file – as-is, without any transformations – to staging area within the SAP S/4HANA system (tables DMC_FILE_HDR, DMC_FILE_T and DMC_FILE_STORE).
• View and Edit (only on-premise) – you can view data uploaded to staging area and in case of on-premise edition, you can also directly edit the uploaded data. This changes values stored in the staging area (and not in the source file).
• Activate or deactivate file – only files in Active state can be taken to the next stage, that is initiating transfer process. This allows you to work with multiple files and choose scope of migration every time you initiate the transfer.
• Start the transfer – this will initiate the guided process for transfer of data from selected (active) files. There are following stages in the transfer process:
  • Data validation (can be sent for execution in background) – data stored in the staging area is subjected to technical validation for issues like lack of data in mandatory field, wrong data length or type, lack of mapping of some key master/config data like country codes or units of measure.
  • Value conversion – you can define mapping rules to convert input values where validation encountered an error.
  • Import simulation (can be sent for execution in background) – in this step, the data stored in staging area is processed using actual BAPIs which are used during “real” run of the import – the only difference is that the changes are not committed to the database. This allows for much deeper validation – functional validation of interdependencies like existence of material master in case of purchase or sales order.
  • Import execution – the data in staging area is submitted for posting in the SAP S/4HANA system using relevant BAPIs. Successfully imported records are persisted in the database.

It is important to understand that APIs used to post data to the target SAP S/4HANA system only support INSERT actions. In other words, there are no UPSERT or UPDATE actions supported. Thus, any data already loaded to the target system cannot be re-loaded. Similarly, if particular data file has been partially successful, you should filter only failed records for subsequent re-run to avoid errors associated with attempts to create duplicate records (for those which were successful during first run).

To summarise key aspects of the SAP S/4HANA Migration Cockpit:

• It is available for both Cloud and ON-premise editions (as of 1610 for the latter), but may vary in some capabilities (like ability to edit data in staging area).
• It is built into the SAP S/4HANA system and does not require any additional deployments or configurations. The same applies to pre-delivered Migration Objects.
• It uses flat files to load legacy data, but there is a plan to provide capability to extract data from ABAP based SAP source systems directly.
• Supported Migration Objects are defined by SAP and at present can only be enhanced (for on-premise edition) using MOM. Creation of new custom objects is planned – refer to details in MOM section below.

Documentation of the SAP S/4HANA Migration Cockpit can be found here:

• For On-premise edition:
  • Online help for SAP S/4HANA
  • Available migration objects for SAP S/4HANA
  • SAP Best Practices for SAP S/4HANA > Solution Scope > Database and Data Management > Enterprise Information Management
  • Note 2537549 – Collective note and FAQ for SAP S/4HANA Migration cockpit (on-premise)
  • Note 2481235 – SAP S/4HANA Migration Cockpit (on-premise) – restrictions and extensibility of pre-delivered migration objects
• For Cloud edition:
  • Online help for SAP S/4HANA Cloud
  • Available migration objects for SAP S/4HANA Cloud 1905
  • SAP Best Practices for SAP S/4HANA Cloud for Enterprise Management > Solution Scope > Data Management > Enterprise Information Management
  • Note 2538700 – Collective note and FAQ for SAP S/4HANA Migration Cockpit (Cloud)
  • Note 2470789 – SAP S/4HANA Migration Cockpit – Cloud data migration template samples

—

SAP S/4HANA Migration Object Modeler (MOM) is a tool delivered in On-premise edition only and can be seen as a design-time tool for definition of the Migration Objects. Its enhancement capabilities are evolving and depending on which SAP S/4HANA version you are working with.

In essence, following activities can be performed using MOM in S/4HANA 1610 up to and including FPS01:

• Display Target Structures You can get an overview of the target structures/fields – these are dictated by the APIs used by the Migration Object to post data to SAP S/4HANA, thus cannot be changed here.
• Edit Source Structure This function allows you to adjust source structure and add a field as required.
• Edit Field Mapping This function allows you to map new or modified fields from source to target structure.

In SAP S/4HANA 1610 FPS02 new features have been introduced:

• Integrate newly created objects – custom ones as well as SAP standard objects which have not yet been delivered as ready-to-use templates in Migration Cockpit.
• Enhance standard delivered objects – for example add new fields.

So, with enhancements introduced recently, you are also able to migrate data into custom built apps (within SAP S/4HANA) as well – as long as there is a suitable API to post to particular data object.

Documentation for the SAP S/4HANA Migration Object Modeler can be found here:

• SAP S/4HANA
• SAP S/4HANA Cloud

SAP Data Services and associated best practice

Prior to introduction of SAP S/4HANA Migration Cockpit to on-premise world, SAP Data Services were the only tool recommended and fully supported for the purposes of data migration to SAP S/4HANA (on-premise). Furthermore, we have also built and been delivering associated accelerators in the form of Best Practice for “rapid data migration to SAP S/4HANA (on premise)” available at https://rapid.sap.com/bp/RDM_S4H_OP (always check for latest version of the BP).

This best practice is built on the capabilities of SAP Data Services, the market leading data integration tool with full data quality capabilities. The logical architecture and scope of this solution is depicted on the diagram below:

The content delivered with Best Practice includes:

• Detailed documentation for technical set-up, preparation and execution of migration for each supported object and extensibility guide.
• SAP Data Services (DS) files, including IDoc status check and Reconciliation jobs.
• IDoc mapping templates for SAP S/4HANA (in MS Excel).
• Lookup files for Legacy to SAP S/4HANA value mapping.
• Migration Services Tool for value mapping and lookup files management.
• WebI Reporting content for SAP BusinessObjects BI platform.

And before we proceed, it is worth distinguishing two use cases of this best practice depending on your requirements and license in place. As per Note 2239701 – SAP Rapid Data Migration for SAP S/4HANA, on premise edition:

If you own either a runtime (REAB) or full use SAP HANA license, this includes a limited use license of SAP Data Services software restricted to loading data into SAP HANA (called Data Integrator license). This fills the minimum requirement for the SAP Rapid Data Migration to SAP S/4HANA content which includes full ETL (Extract, Transform, and Load) used to extract data from heterogeneous source systems, the transformation and mapping, the validation and the data load.

In other words, the limited use license allows you to take advantage of all scope items in the diagram above. Additional licenses may be useful for two specific extensions to the core functionality – advanced data profiling and advanced data cleansing.
With regards to advanced data cleansing there is a dedicated job to standardise, cleanse, match and de-duplicate Business Partner names and addresses, which uses Data Quality transforms and thus requires the full Data Services license.
The SAP Information Steward comes into the picture to provide advanced data profiling capabilities based on its Data Insight that includes the ability to run extensive types of profiling like:

• Columns – used to examine the values and characteristics of data elements such as minimum, maximum, median, distribution of words, and so on.
• Address – used to determine the quality of an address. This sends data through the address cleansing transforms in Data Services to identify those that are valid, correctable, or invalid
• Dependency – used to identify attribute-level relationships in the data. For example, for each state you can find the corresponding city name
• Redundancy – used to determine the degree of overlapping data values or duplication between two sets of columns
• Uniqueness – returns the count and percentage of rows that contain nonunique data for the set of columns selected.

To take advantage of the capabilities listed above, respective license is required. Henceforward we will focus on capabilities included in the Data Integrator package.

SAP Data Services use IDocs to post data into SAP S/4HANA, therefore respective configuration on the SAP S/4HANA target system is a common requirement for all migration objects. There is a custom program delivered to create required partner profiles for each required message type (refer to building block “Data Migration IDoc Config Guide (W01)”).

Now, we do obviously need to deploy SAP Data Services and associated requirements as well as pre-delivered migration content – this is documented in “RDM_S4H_OP_DS42V2_Quick_Guide_EN_XX” attached to Note 2239701. Worth noting that there are two separate versions of this manual – one for Windows and one for Linux based deployment with support for underlying repository database platforms varying between these deployments. Also, the document may only detail set-up steps for one selected DB – for other supported DB platforms, refer to standard SAP Data Services documentation.

Once standard set-up is complete, delivered content needs to be applied – from that point forward, the platform is ready for preparation and execution of data migration for selected objects.

When sourcing data from your legacy system, you will have a choice of using flat files (text or XLS) as intermediary or to connect directly to your source system or database.

With the documentation and content delivered via referenced Note, you can deliver your data migration project yourself. But, SAP does have packaged service to deliver the described scope. You can get more details from https://rapid.sap.com/bp/#/browse/packageversions/RDM_S4H_OP > Accelerators > Customer presentation. The service has flexible scope and experienced team to deliver. This can be particularly interesting when you do not operate SAP Data Services in your environment and do not have the necessary skills available.

Closing remarks and considerations

The two toolsets and methods for migrating your data described above are quite different when it comes to the tooling, capabilities and associated effort. The matrix below attempts to summarise and compare key aspects of each.

The decision, dear reader, is yours.

Other useful links and references

• SAP knowledge articles:
  • Note 2537549 – Collective SAP Note and FAQ for SAP S/4HANA Migration cockpit (on premise)
  • Note 2538700 – Collective SAP Note and FAQ for SAP S/4HANA Migration Cockpit (Cloud)
• Videos:
  • SAP HANA Academy – S/4HANA RIG: Migration Cockpit
  • SAP HANA Academy – S/4HANA RIG: Migration Cockpit and Migration Object Modeler
  • SAP S/4HANA Migration Cockpit: How to Migrate your SAP Data in S/4HANA
  • SAP S/4HANA Migration Cockpit: Migrate Data Directly from SAP System SAP S/4HANA 2020
  • SAP S/4HANA Migration Cockpit: Migrate Data Using Staging Tables, SAP S/4HANA Cloud 2008 and SAP S/4HANA 2020
  • SAP S/4HANA Migration Cockpit: Transferring Data to SAP S/4HANA Using Files SAP S/4HANA 1909
  • SAP S/4HANA Migration Object Modeler: Create an own mapping rule SAP S/4HANA 2020 FPS00
  • SAP S/4HANA Migration Object Modeler: Migrate Data Directly from SAP System – Creating a new migration object SAP S/4HANA 2020 FPS00
  • SAP S/4HANA Migration Cockpit: Enhanced error analysis capabilities SAP S/4HANA 2020 FPS01
  • SAP S/4HANA Cloud – Data Migration Status – Overview
  • SAP Rapid Data Migration with SAP Data Services (please note this video does not explicitly feature SAP S/4HANA, but the concepts presented are common for various data migration target solutions)
• Other informative blogs:
  • Comparison: Migration Cockpit, Rapid Data Migration (with Data Services) and LSMW
  • How do you Migrate to SAP S/4HANA
  • Starter Blog for SAP S/4HANA Migration Cockpit
  • Data Migration with RDS
  • Watch the Demo: Migration to the Cloud
  • SAP S/4HANA Migration Cockpit – tips & tricks
• Presentations:
  • SAP S/4HANA Migration Cockpit – Migrate Your Data to SAP S/4HANA Cloud
  • SAP S/4HANA Migration Cockpit – Migrate your Data to SAP S/4HANA (on-premise)
• Best Practice “Rapid data migration to SAP S/4HANA (on premise)”
• Blog “Introducing The Data Migration Guys”
• The SAP Press book “Migrating to SAP S/4HANA”