Technical Articles
Working with Staging Tables in S/4HANA Migration Cockpit
S/4HANA migration cockpit is the migration tool which was initially designed for S/4HANA cloud edition but now it is also available for 1709 S/4HANA FPS1 on premises and later versions.
S/4 HANA Migration Cockpit is browser based (WordPro) interface. No additional setup or activation is required once we setup SAP S/4HANA system.
Key Features of Migration Cockpit
· This tool is embedded and delivered with S/4HANA system.
· No programming is required by the customer.
· As the name suggests, this tool is used for migrating data from SAP or Non SAP system to S/4HANA.
· This tool has predefined migration object which contains the mapping for all master and transactional data. It reduces migration cost and time.
· Migration activities are predefined and easy to use.
Using Staging tables in Migration Cockpit we can use Database Tables as a source for your Migration Project. These are called as ‘Staging Tables’, so you extract the data from the source system into these staging tables and import the data from there with the S/4HANA Migration Cockpit.
The Advantages are:
- faster overall process (export/import), less clicks
- Performance
- Use Database Tools to Extract and Transform
Method to populate Staging Tables:
- SAP Data Services
- HANA Studio
- Using Excel as Source in S/4 and populate Staging tables programmatically.
This document will use SAP Data Services as a method to populate Staging Database.
- Create a Database Connection:
The first step in using Staging table is to create a Database Connection between S/4 and the schema where the Staging tables will reside .The Staging tables can exist in a remote database or in the target S/4HANA database (but in a separate Schema) .
Go to transaction DBCO to create the database connection ( or let the Basis team do their job and ask them).
Once it is saved, the connection looks like this.
Important tip:
Just by Creating a Database Connection does not mean you can use it as a Staging Area for Migration Cockpit. Once the connection is available in DBCO, you have to white-list this connection in table DMC_C_WL_DBCO_OP using transaction SM30:
Once the entry is maintained, the Database Connection is ready for use externally.
Creating a New Project for Data Migration
Go to Transaction LTMC – Migration Cockpit.
Click on Create New Migration Object and Select the Transfer Option – Transfer Data from Staging Tables.
When you select the Database Connection, you can see the Database connection created in DBCO in previous Step.
Once you Select Click on Create. Select it and Save and the Migration Object is ready for use. I have named it as STAGING_DEMO .
Click on STAGING_DEMO and it will enter the Project with all Migration Objects available. Migration object comes with the following information:
- Status– Object can be active – available for migration, deactivate – not available for migration and started – migration has started on this object.
- Object name– It is migration object name which is relevant to customer master or transnational data.
- Documentation– Once we click on documentation it will open a new window where we can find all information on migration object like required structure, fields, uses.
- Dependent migration object– It shows a list of migration object that must be loaded first or already present in the system.
You can see all Conversion Objects are in Status Not Started. For this Demo I am using BANK Master Conversion Object. When you Click on BANK Conversion Object, a popup comes asking for copying the Conversion Object and Creating Staging tables.
Click OK.
Few things that you can see here:
- Status – Synchronized
- Structure – S_BNKA
- Staging Table – /1LT/DSR10000003
Click on the table /1LT/DSR10000003 and you can see there is no records right now.
Now we will validate the table name in S/4 and HANA Studio.
Go to SE11 and enter the table name –
Click Display and you can see Bank Master Transparent Table .
In HANA Studio also you can see the table under the correct schema:
Filling the table with Data to be used for Cockpit Load.
As mentioned earlier we will use Data Services to populate the Staging Table .
Log into Data Services and map the HANA Database.
Once it is mapped. Right Click and Select Import by Name.
Once the table is Imported you can see it under Datastore tab and under Tables tab under the STG Datastore configured earlier.
Create a job to pass data to the Staging Table.
I have created a Basic Job to pass 1 record to the table, which will use for the demo.
The name of the job is – JB_STAGING_AREA and it has one Data Flow – DF_STAGING_AREA.
The Data Flow in details looks like this:
In Query Data Query, I am passing Default values for load.
And as we have to pass only First row, the filter is in where Clause.
The job looks like this:
Once you execute the Job.
1 record is passed to the Target table:
Select the table preview of the table:
And we can see the same in HANA now once the job is processed:
Load the data using Migration Cockpit
The last and main Step is to Migrate this Data into S/4 using the Migration Cockpit .
Once you Click Start, the migration starts and progress bar will show the
The Validate Data Step will come with Errors and Warnings .
Click Next and you see the Mapping errors. These are technically not errors, but S/4 just validates the value we are passing matches the help values for that field.
Click on each field and just Select and Press the key
The field turns Green Status. DO this for the next 2 fields.
Once the Open Task area is Blank, Click Next to go to next Step (Convert Values)
Once the task is finished and you get the message below for Conversion. Click Next to Simulate Import.
Simulate Import will do the test but will not Commit to save any values.
The simulate gives a Success message that the Data is good for Commit. You get the message at the Top: All data have been processed; choose “Finish” to complete the transfer
Click on Finish to Finish the Load and Commit the values.The load is finished and you can see the processed records in Migration Cockpit.
Go to S/4 and check the table BNKA.
Go to Transaction – FI03 , to see the Data in Transaction. You can see the Bank Master Data.
With this document you will be able to import data into Staging tables and then into S/4 HANA SYSTEM USING Migration Cockpit. This document will be helpful at the time of data migration where Data is huge and coming and being Transformed in Data Services .Please note the staging concept in SAP S/4HANA Migration Cockpit is only valid for HANA Databases.
You can leverage the same steps for other Conversion objects ( Vendors, Customers, Equipment, etc ).The templates for each Conversion Object is different would be different, but the fundamentals remain the same.
References:
SAP Notes and Blogs
- https://blogs.sap.com/2017/12/04/sap-s4hana-migration-cockpit-tipstricks/
- https://launchpad.support.sap.com/#/notes/2733253
- 2596411 – SLT / NZDT / S4HANA Migration Cockpit (DMIS2011 SP11-SP15; DMIS2018; S/4HANA 1610, 1709 & 1809) – Note Analyzer
- https://launchpad.support.sap.com/#/notes/2587257
Hello Vijender Sangwan ,
Good examination for Master Data Services. Thanks for sharing.
Brgds
M.Ozgur Unal
Thanks Mehmet.
Hello Vijender,
Nice document ...thanks for sharing
Thanks Vijender .. Very helpful post.
Nice Document.. Very helpful.
Thank you so much for sharing such interesting facts.
Good examination for S/4HANA Migration Cockpit.
Useful document.
Thanks for sharing it.
Hi Vijendar, Nice Document? But I have few doubts raised while reading about S/4 HANA Migration Cockpit?
Transfer data by File is only for Insert and for small amount of data ? It cannot handle delta Update? suppose 1 material migrated today can with some delta update tomorrow then can S/4 HANA Migration Cockpit handle that delta update in Target system. Which means 1 entry for that material with latest values and no duplicate right?
2nd Doubt is when you transfer by staging tables is the staging table is truncate and load every time? if any delta update here then do we need to handle that in BODS level only right???
please clear me these doubts.
Hi,
Ever got an an answer for this?
Would it be possible to do a delta load (insert/update) from the staging tables at project go-live/cutover from ECC to S/4?
Hi Narasingha and Adriaan ,
Migration Cockpit cna handle and load deltas during Cutover and Trials.
When you load the data and its start the load process , Cockpit has an auto mechanism of validating data against config and existing data .
It so happens many times when some records fails during initial validation and are left out . You can either fix them there or do a delta load with only those records with errors fixed.
Same happens for Delta loads . It just leaves the records out as error which are already created .
Let me know if you have queries.
Thanks,
Vijender
Dear Vijender,
Thank you for the detailed explanation. Even though I've seen blogs on using LTMC via files, there hasn't been much info on the usage of staging tables and your blog has come in handy.
However, I have a query. I have a scenario wherein data extraction is a customer responsibility and they give me data in the Migration templates provided (XML files). There are objects where the file size crosses the 100MB limit and I may have to consider using staging tables for having these loaded. How do we upload data from an excel/xml file into a staging table. Any tips? Can we have the ABAP team create an upload program for having the data loaded to the staging table and will it be similar to having data loaded in a normal table.
Appreciate if you could give me some pointers.
Thanks and Regards,
Rakesh
Hi,
The size limit for each file is 200 MB for SAP S/4HANA migration cockpit.
Note that for on-premise systems, parameter icm/HTTP/max_request_size_KB controls the size of the http request.
The value of this parameter is the maximum size (in KB) of an HTTP request.
This parameter has the standard value 102400 kb (100MB) but can be changed if required.
For more information,
https://help.sap.com/viewer/fe1dfb6e6c551014b6f1befabc903d5d/7.02.19/en-US/483e87b3ca6b72d0e10000000a42189c.html
Beside above actions, you can also do below steps to achieve parallel load:
1. Split your xml template into multiple files
2. Create multiple identical projects
3. Upload your files among the projects
4. Start transfer for each migration object in parallel
Note: You still need to make sure enough background jobs are available.
Hope this helps!
Thank you, Prithvi. That helps, for now at least. Will get back in case I have further questions.
Hi Prithvi and Rakesh,
Is there any sizing limit for Staging tables as well?
I have a scenario where the load volume for Material Master data is huge. Around 3.5 million records for each load cycle.
Any suggestions on this can be handled through LTMC?
Thanks and Regards
Kashish Gulhar
It was very informative.
Can somebody help me in this situation.We are in S/4 HANA cloud and want to use staging Table approach. I see the per-requisite is to establish a db connection to HDB where the staging tables reside.
How can we do this in cloud? Do we have any apps for this purpose similar to tcodes in on premise?
can we access staging tables by any chance the way we do in OP system?
Thanks in advance
Hi Sam,
Thanks for reading the blog.
Yes,it is possible to Migrate Data using Staging tables in the Cloud.
There are some basic requirements specific to Transferring Data using Staging Tables in the cloud .
The prerequisites are:
Communication Arrangement (Service)
− SAP Cloud Platform Account
− Customer must register for a DBaaS HANA Instance on SAP Cloud Platform
− Customer has to request the activation via ticket on component XX_S4C-OPR-SRV
If you need to review in details, all details are available in the SAP NOTE - 2538700 (Link Below)
https://launchpad.support.sap.com/#/notes/2538700
Let me know if you have any queries.
Thanks,
Vijender
Thanks for the link. That helps.
Hi Vijender,
Very informative post. I have issues loading the data from staging table in HANA SAP table. I inserted one record in staging table in the backend HANA DB. But the thing is we are not able to transfer the data from staging tables into HANA SAP table like BNKA. It just gets stuck with the message "validate data is running in the background (5% complete)" but it doesnt get processed and finally we need to reset it. Any idea on this. Any setting we are missing. Could you please help us here?
Regards,
Sarath
Hi Sarath,
can you please check if you implemented latest notes?
Run Report DMC_NOTE_ANALYZER
Title: Note Analyzer - SLT / NZDT / S4HANA Migration Cockpit (DMIS2011;DMIS2018;S/4HANA)
Link: https://launchpad.support.sap.com/#/notes/2596411
regards,
Sybille Lackermeier
Thanks Sybille for pointing that report !
Excellent Detailed Document.Thanks for Sharing.
Hi,
I tried to use Staging tables for Bank Master and when i click on after selecting DB connection , I am getting below error. Can you please advise?
"Error while copying the migration object Z_BANK_MASTER_008"
Thanks
Kris
For more information about SAP Data Management and Landscape Transformation (DMLT) see this page https://www.sap.com/services/implementation/data-mgmt-landscape-transformation.html or read the solution brief and the SAPPI Success Story. You can contact the global SAP DMLT team by email sap_dmlt_gce@sap.com. They provides services for Selective Data Transition, New Implementation and System Conversion that can also be delivered remotely.
Thank you, Mr. Vijender
This article help a lots ?
Vijendar,
Really big help. article is very clear and give’s us in depth step by step information.
Thanks!