SAP BW ODP – Performance improvements and benefits
This is a follow-on from the creation of an ODP ECC Source System article. In this article I’ll be covering an overview of the ODP datasources along with the extraction performance improvements and benefits of ODP.
Operational Data Provisioning (ODP) provides a technical infrastructure that you can use to support two different application scenarios. The first of these is Operational Analytics for decision making in operative business processes. The other is data extraction and replication. Please note, this article primarily targets the “Data Extraction and Replication” use case for Operational Data Provisioning (ODP).
The following are the main benefits:
- Can load the data directly into the BW InfoProviders, bypassing the Persistent Staging Area (PSA) layer by using a Data Transfer Processes (DTP)
- The ODP infrastructure (with delta queues) takes over important services such as monitoring data requests.
- The data is stored in a compressed state in the delta queue. A delta request transfers data records from the queue to the subscriber (target system).
- The data changes to a queue can also be requested by more than one subscriber (target system).
- The data is retained in the delta queue for a specified time for recovery purposes.
The pre-requisites to use ODP for data transfer are
- SAP_BASIS < 730 – SAP Note – 1521883 – ODP Replication API 1.0
- SAP_BASIS >= 730 – SAP Note 1931427 – ODP Replication API 2.0
- Recommended starting release with BW 7.40 SP5 and supported for all databases.
Operational data provisioning supports extraction and replication scenarios for various target applications and supports delta mechanisms in these scenarios. Besides SAP BW/4HANA and SAP BW, Operational Data Provisioning can be used to provide data to other SAP Products such as SAP Data Services or SAP HANA Smart Data Integration. The following are possible source and target systems for ODP datasources:
- Data transfer from SAP DataSources (Extractors)
- Data transfer from ABAP CDS Views
- Data transfer from SAP BW or SAP BW/4HANA systems
- Real-time replication of Tables and DB-Views from SAP Source System via SAP SLT
- Data transfer SAP HANA Information Views in SAP ABAP based SourcesTarget Systems
- Data transfer to SAP BW or SAP BW/4HANA
- Data transfer to SAP DataServices
- Data transfer to the “ABAP Adapter” in SAP HANA Smart Data Integration (SDI)
The following are general points for ODP’s
- Most Business Content DataSource (Extractors) can easily get released for Operational Data Provisioning.
- Since SAP BW >= 7.4, ODP is the strategic relevant source system connection to SAP Sources and with SAP BW/4HANA, only ODP source systems are available.
- ODP doesn’t change the implementation of application extractors, all the features and capabilities are the same.
- Consumption of generic DataSources created in SAP ABAP Systems in transaction RSO2 is possible in SAP BW/4HANA using the ODP-SAP source system connection type.
- ODP can be deployed in parallel with the traditional delta queue approach but it will multiply the data.
- If the extractor logic is the ‘bottle neck’ the throughput won’t change for an ODP datasource compared to a 7.x datasource.
The ODP performance scenario that we’ve completed was on the GL Line item datasource (0FI_GL_14) from ECC system (source) into a BW 7.4 system (target). We created the ODP source system as per the steps in my last article and activated the ODP FI GL line item datasource (0FI_GL_14). Once activated, we mapped the ODP FI GL line item datasource to a new ADSO object (write only) with the same mappings of the existing 7.x FI GL line item data flow:
The following are the performance improvements for each system in the landscape (ODP vs 7.x DS)
- Prototype – 25% performance improvement (982k records – 675 seconds (0FI_GL_14 ODP) / 909 seconds (0FI_GL_14 7.x DS)).
- Development – 18% performance improvement (1.6 million records – 1263 seconds (0FI_GL_14 ODP) / 1533 seconds (0FI_GL_14 7.x DS)).
- Test system 1 – 40% performance improvement (511k records – 753 seconds (0FI_GL_14 ODP) / 1251 seconds (0FI_GL_14 7.x DS)).
- Test system 2 – 20% performance improvement (54k records – 223 seconds (0FI_GL_14 ODP) / 277 seconds (0FI_GL_14 7.x DS)).
- Production – 39% performance improvement (2.6 million records – 61 mins (0FI_GL_14 ODP) / 100 mins (0FI_GL_14 7.x DS)).
All the above extractions for both 7.x and ODP datasources was based off full loads (except for the 2.6 million extraction on 7.x DS in production).
After deploying the ODP datasource to PRD, we executed the delta DTP off this ODP datasource with No Data Transfer; Delta Status in Source: Fetched (Delta Init). We then done two delta extractions and the following are the performance benefits:
- Production – 86% performance improvement (5.2 million records – 28 mins (0FI_GL_14 ODP) / 2.6 million records – 100 mins (0FI_GL_14 7.x DS)).
- Production – 74% performance improvement (1.3 million records – 13 mins (0FI_GL_14 ODP) / 2.6 million records – 100 mins (0FI_GL_14 7.x DS)).
The performance benefit of a delta execution of the 0FI_GL_14 ODP is from 74% to 86%. This increase in performance compared to a full data extraction on the same ODP datasource is due to the extraction not have to read the underlying tables (BSEG/BKPF).
ODP functionality brings many benefits – bypassing the Persistent Staging Area(PSA), data changes requested by more than one subscriber, compression of the data and data retained in the delta queue for recovery purposes. Along with this there is a substantial performance benefit. SAP lab results have shown a reduction in runtime by more than 40% for an ODP datasource compared to a 7.x datasource. From the delta tests that we’ve executed in production system we’ve evidence that the benefit is much greater than this – 74% to 86%. Also, it’s important to note that 7.x datasource are not supported on BW/4HANA, only ODP source systems are available. There is a BW/4HANA Transfer Tool (transaction RSB4HTRF) available that will automatically copy your 7.x DataSource to a corresponding ODP datasource.
Useful ODP SAP Notes
- 2232584 – Release of SAP extractors for ODP replication (ODP SAPI)
- 2407906 – Additional information about SAP extractors released to operational data provisioning (ODP)
- 2368268 – Export DataSources not exposed to ODP
- 2433354 – Missing Business Content DataSources or Transformations when using ODP framework
- 2500202 – S4TWL – BW Extractors in SAP S/4HANA
- 2481315 – Operational Data Provisioning (ODP): Extracting from SAP Systems to SAP BW or SAP BW/4HANA
- 2480284 – BW4SL – Hierarchy DataSources
- 2464541 – BW4SL – Data Transfer Processes
- 2300483 – ODQ fetch performance ODQDATA_V
It is important to not get straight into this concept of source and target. There a subscriber and a provider, this concept makes life easier.. it can be BW to BW, DataServices to BW and back and forth. It is a context for communication 🙂
The benefits of time can't be taken for granted before a proper analysis of your system , it is made to run with Hana! If you have Hana DB, then get ready to fly. Otherwise, remember that the data compression is something huge and very well achieved by ODP. Of course this take some time to be done 😉
Thanks for sharing your knowledge Donal!
Thanks Donal for an amazing post.
I had a question regarding how the data is moved from provider(considering ECC) to subscriber(BW) in case of ODP framework. How different it is in terms of data movement when compared to SAPI adapters?
I came to know that ODP framework does not make use of IDOCs. Can you tell me how exactly it operates then.
Any info on this will help.
Thanks in advance.
Thanks Donal for sharing this info.
Can you please tell me when I compare the Delta Queue of 7.x DS vs ODP (RSA7 vs ODQMON).
For LIS Extractors we can check the delta data directly in RSA7 which is already pushed to delta queue when the LIS job runs.
How is the delta data pushed to ODQMON and where I can see it before pulling it to BW.
Donal, thank for sharing.
But PSA allows you to cleanse the data that fails a data load and also gives you a good idea of what data is extracted by the extractor.
How do you do that in the new ODP/DTP architecture? I know DTP has the error stack, but I thought PSA is easier to operate. I understand odqmon has some capability to monitor the actual data extracted. But I found it not user friendly at all since you can directly query the PSA table. Also, you are still duplicating the data in odqmon on the ERP side which I argue it's worse.
There is an ability to modify data in the inbound ADSO table I believe, that was created to replace manual changes in PSA.
Inspiring! Oh, there is a Like button there.