CDS based data extraction – Part III Miscellaneous
Back to the CDS view extraction..
The topic of hierarchy extraction deserves a small section, as some things have changed here in contrast to the classical extractors. You will need to make some changes to your hierachy bearing characteristics in order to work with the hierarchy CDS views. This is due to the fact that the node texts are not delivered by the actual hiercharchy CDS view, but a separate one.
We’ll use the Cost Center hierarchy as example. You will need the following three CDS views for getting the complete picture for a cost center hierarchy across:
- Cost Center Hierarchy Node (I_COSTCENTERHIERARCHYNODE)
- Cost Center Hierarchy Node Text (I_COSTCENTERHIERARCHYNODET)
- Cost Center Text (I_COSTCENTERTEXT)
You will need to equip a characteristic Cost Center, ZCOSTCTR in our case, with two additional external characteristics in the hierarchy:
One characteristic for the Cost Center Hierarchy ID (ZCOCTRHID) type CHAR40.
One characteristic for the Cost Center Hierarchy Node Text (ZCOCTRHNT) type CHAR50, compounded to the Cost Center Hierarchy ID
The hierarchy transformation between a DataSource based on CDS view Cost Center Hierarchy Node (I_COSTCENTERHIERARCHYNODE) and the cost center characteristic needs to be mapped as follows (Segment 0003):
The hierarchy node texts will be delivered via a DataSource based on CDS view Cost Center Hierarchy Node – Text (I_COSTCENTERHIERARCHYNODET) and loaded into the characteristic Cost Center Hierarchy Node Text (ZCOCTRHNT)
The cost center texts will be delivered via a DataSource based on CDS view Cost Center Text (I_COSTCENTERTEXT) directly into the cost center characteristic as usual:
Just for completenes, please note that the two additional characteristics needed as external hierarchy characteristics can have different lengths according to a particular hierarchy characteristic (e.g. GL Accounts, Profit Centers etc.). Please make sure to check them in the respective hierarchy node CDS view first.
Testing CDS view extraction
With classic extractors the Extractor Checker S-API (transaction RSA3) is your friend on source system side. No need replicate a DataSource to BW side etc. Bad news, you cannot use RSA3 to test your extraction enabled CDS views.
How can you test CDS view extraction? The good news, you can use report RODPS_REPL_TEST.
Before sharing the details, please consider the warning on top (Warning: Not Simulated): Initializing a delta and/or running a delta extraction with the report will trigger a real subscription in the ODQ frame work. This is no dummy run.
When starting the report RODPS_REPL_TEST you will need to fill the fields as shown in the screen shot below.
The ODP Name equals the @AbapCatalog.sqlViewName again and is suffixed by the data category classification in the CDS view:
- $P for master data attributes
- $T for texts
- $H for hierarchies
- $F for facts/transaction data
- $E for other/without dataCategory
After you are done with testing please make sure to press Reset Delta so your ODQ subscription can be cleaned.
One speciality for the hierarchy extraction test. For being able to see the extracted hierarchy data you will need to check the hierarchy segement that you would like to see, i.e. use F4 value help for Projections
While in RSA3 you can navigate to the different segements via a drop-down
in RODPS_REPL_TEST you can double-click on the figures to jump to these segment entries.
Units, Currencies, Fiscal Year Variants via CDS
In SAP BW and SAP BW/4HANA you will most likely also want to work with the units of measurement, currencies, currency conversion rates and fiscal year variants as they are maintained in your SAP S/4HANA system. In the on-premise world, this means a right click on your source system, choosing options
- Transfer Exchange Rates
- Transfer Global Settings
This is the same for a SAP S/4HANA Cloud source sytem in the ODP_CDS context:
In case you do not see these menu entries for your SAP S/4HANA Cloud source system, please check SAP note 2792996.
Alternatively after implementing the SAP note you can call up the reports RSIMPCUST (currencies, units of measurement and fiscal year variants) and RSIMPCURR (exchange rates) and trigger the replication manually or scheduled via process chain.
BW/4HANA Content based on CDS extraction
As mentioned in the last blog already, SAP has started delivering content based on CDS view extraction.
So far the Sales and Distribution (Sales Documents, Billing Documents, Conditions) and Plant Maintenance areas have been covered. You can find them in the documentation.
For the upcoming areas please check the road map slide in our BW/4HANA Content presentation here.
As usual taking over the content is done via transaction RSORBCT, selecting the data flows or objects that you require. As it sometimes leads to confusion about the content being missing, please make sure to check the steps described in the following SAP note, especially point 4 and 5
Q: Is there a Mapping of classic extractors to CDS views?
A: Unfortunately you will not find a mapping between classic extractors and CDS views provided by SAP; mainly due to the fact that data structures in the SAP S/4HANA have and keep changing as well as the extractor logic that can be expressed in a classical ABAP based extractor vs a CDS view.
Q: Where can I find a list of available extraction enabled CDS views delivered by SAP?
A: Please have a look the blog of my colleague Ranjitha Balaraman here
Additionally starting from SAP S/4HANA 2020 as well as in the current SAP S/4HANA Cloud Edition you can use the CDS view I_DataExtractionEnabledView to identify CDS views available for data extraction. The view also allows for identifying CDD delta enabled DataSources selecting ‘X’ for field DELTACHGDATACAPTUREISSUPPORTED.
Q: I a missing a certain CDS view for extraction, where can I provide feedback to SAP to request it?
A: Please go to the customer influence site and provide your request here.
Q: What about global field names in CDS views and BW naming?
A: You can find the details on the CDS naming conventions in the SAP help documentation. Due to their descriptive nature, CDS view field names are sometimes quite lengthy clashing with the SAP BW limits, like DataSource Field names, Unit field names in field-based DataStoreObjects (CHAR 25). So please keep your eyes open to not lose the semantics of a field.
Q: Which delta method should I use?
A: hm, it depends. If possible from a CDS point of view (projection or a left_outer_to_one_join) go for the Change Data Capture (CDC), escpecially if you are after deletions as well.
Q: Formerly an application has worked with before- and after-images in the classic extractor. Is this still supported in the CDS view based extraction?
A: CDS view based extraction will only deliver after-images, i.e. the status of the record at the point of time it was added to the ODQ. This results in mode Overwrite (MOV) for key figures on BW side. Hence if you are working with InfoCubes (in SAP BW) or Data Mart DataStore Object (SAP BW/4HANA) you will need a Standard DataStore Object as first target in your data flow before.
Q: Can I switch between full/delta methods?
A: Changing a C1 released CDS view from full data extraction to the CDC delta extraction is considered a compatible change. Hence adding the annotation for CDC delta into an already C1 released full-extraction CDS view is possible.
Changing the delta method from generic to CDC delta is considered an incompatible change as you will be forced to re-initialize your data load. If reloading the data is not an issue for you, you can neglect this point.
Q: How do classic extractors from the Logistic cockpit compare against CDS views?
A: The good news, with CDS view extraction you can skip the whole process of Set-up tables, system lock, but start loading data right away. Also SAP is focusing on delivering the data via denormalised header-item CDS views, so no more header-item consolidation on BW side.
Q: What about filters on non-key fields using the CDC delta?
A: Please do not use any filters on non-key fields (i.e. as part of the WHERE condition of a joined view). An update on the table level might lead to a change of the cardinality of the join.
Q: Using the CDC delta I receive delta records for records that supposedly do not have changed.
A: Due to the nature of the CDC delta you will always receive a delta record in case some field value has changed in the underlying tables of your CDS view, no matter if this field is exposed as a view element in the CDS view. In case the field in question is not exposed as view element, your delta record will look the same as the one you already have. As the records are all after-images, no harm to your existing data, but good to keep this in mind in case you wonder.
Q: Are there any CDS annotations that do not go well together with extraction annotations?
A: Yes there are, please make sure to avoid the following annotations in a CDS view that you want to use for extraction:
- @Odata.publish : true
- @Analytics:dataCategory : AGGREGATIONLEVEL
- @ObjectModel.transactionalProcessingEnabled : true
- @ObjectModel.transactionalProcessingDelegated : true
Q: After having changed the CDC annotation I receive the following message during data loads “Change data capture of is broken“.
A: To properly work with your updated CDC annotation you will need to reset/cancel all subscriptions for this particular view. From BW side this means deleting the respective data load requests. For other requests you can also directly cancel them in transaction ODQMON. You can find more details in note 2930269.
Last but not least, if something goes wrong and the above answers do not help you …
Should you experience any issues with CDS extraction to BW please open a ticket on the component Operational Data Provider for ABAP CDS, HANA & BW (BW-WHM-DBA-ODA).
Should you experience any issues with CDS extraction using the Change Data Capture (CDC) delta, please open a ticket on component Subscription-based Change Data Capture Mechanism (EIM-DH-ABA-CDC).
Well, I guess this is it… for now…, thanks for reading up until here. As things might change and new features appear, I will update this blog accordingly, so feel free to come back any time.
Any feedback, questions let me know; happy to answer your questions and to extend the Q&A part.