Delta Extraction and Real Time Streaming ODP-CDS to BW

There has been a shift in the way business applications are developed with the availability of SAP HANA. Idea is to move logic (code) as much as possible to the Database so as to get performance benefits. To simplify this code push down to database, SAP has provided new data modelling technique called Core Data Services (CDS).

SQL based data definition is used to define CDS views with additional concepts of associations (similar to joins) and annotations. To know more about CDS annotations click here.

CDS has become the keystone for application development as this can be used to build core transactional functionality as well as analytical content for reporting.

In this Document we will see how CDS views can be used for Delta Data Extraction or Real Time Streaming to BW

Pre-requisites

BW data extraction is not supported by S4HANA 1511. Minimum system requirement is S4HANA 1610 or Higher. With 1610 SAP has introduced some new annotations to facilitate data extraction to BW using CDS views.

Source Requirements

Full Data Extraction or Direct Access: Minimum of NetWeaver 7.5 SP04 or equivalent.

Delta Extraction: Minimum of NetWeaver 7.5 SP05 or SAP BW/4HANA SP01

Target Requirements

Minimum of NetWeaver 7.3 SP08 or equivalent (ODP recommendation)

For the extraction of data into BW, ensure an ODP source system of type ABAP_CDS is available to connect to the source system.

Extraction shown in this document is done on system on S4HANA 1610 with embedded BW components.

Delta Extraction of CDS-ODP source to BW

In this example Delta extraction will be shown for changes in the FI documents. CDS view to fetch the delta records for any changes in the FI document with object class ‘BELEG’ or ‘BELEGMASS’ or ‘BELEGR’ will be created and used for the same.

Create Delta enabled CDS View

• Go to the Eclipse editor and create a CDS View.

• Provide CDS View details

• Select The Transport number

• CDS view with blank template will be generated

• Provide the SQL View name (with this name view will be created in back-end system) and write the view query logic. Make sure to use the Analytics annotations for the element/field to be considered for delta identification (it could be Date or timestamp type filed)

• As we need timestamp field (for multiple time execution in a day) for delta extraction and we don’t have the same in CDHDR table, convert the change date and change time into time stamp and cast it to timestamp format (By default the converted field is created of type Char). Also use Semantics annotation to enable the filed for delta extraction as shown below.

• Activate the CDS View. It will generate the View in backend system with the name provide in step 5.

• The timestamp field we created in our view will be created with data type DEC. But nothing to worry, it will be work as timestamp only.
• Execute the View to see the content and verify the generated timestamp field

• With this, we have created our delta enabled CDS view.

Create a BW data source

• Open the BW perspective in eclipse and create a BW Project. Under the Data Sources tab in left hand side window, you should see your source Logical system under ODP-ABAP CDS views node

• Under the Logic system you will see the node for different application areas. Select the node under which you want to create your Data Source. Right click on the node->New->Data Source

• You will get the below pop up window with default values for BW project and Source System. Click Next.

• In the next popup window select the “Proposal from operation Data Provider” radio button and click Next.

• In the next popup window, select the CDS View created in earlier steps and Click Next.

• Enter the Data Source description and click Finish

• Data Source created as shown below. Click on Activate Icon to activate the Data Source.         

• Once Activated, click on “Extraction” tab highlighted in above step. You can the Data Source is created with Delta Process “After Images via Delta Queue (FI-AP/AR for example).

• Right click on the Data Source in the light side window and click Replicate.

• Select your Data Source and click Next

• Click Finish

• Job with Successful replication will be displayed. Refresh the screen if job is not yet finished.

• We have successfully created a DataSource for our CDS View.

Create Advanced Data Store Object

• GoTo BW Repository Node in the Left side window. Select a Repository and right click on it to create New Data Store Object

• Enter the Details as shown below and Click Finish

• Advanced Data Store Object is created. Click on Active to activate the same.

Create Transformation

• Right Click on Data Store Object and click on create Transformation

• Provide the details of the Data Source in the next window

• Transformation is created with Proposed mapping between Data Source and DSO. Activate the Transformation by clicking on the Activate Icon.

• Transformation is activated Successfully as is shown in Version

Create Data Transfer Process (DTP)

• Right Click on DSO and click on New->Data Transfer Process

• On the next screen details are be filled by default. Click Continue.

• DTP is created. Activate the same.

• DTP must be activated Successfully.

Create Infopackage

• Open TCODE RSA1 in backend system and under the Data Source Node select the Data Source “ZDELTA_ACDOCA” we created.

• Right click on the Data Source and Click Create Infopackage

• Enter the InfoPackage description, select the Data Source and click Save.

• Open The Update Tab and select the Full Load Radio Button and start the extraction from schedule tab.

• Go to the update tab and select Initialize the delta process

• GoTo Schedule tab and start the extraction

• Monitor the extraction. All the records are fetched and the delta processed is analyzed. This means now with Delta run only delta records should be fetched.

• Now let’s make a change in one the document. Doc Header text is changes as shown below.

• Go to the Update tab in the infopackage and select the Delta radio button

• GoTo Schedule Tab and start the extraction.

• Monitor the Extraction. We can see there are 21 records are extracted.

• GoTo TCODE ODQMON and see the Documents which are changes to verify if the extract is corresponding to the changes we made. As seen in screen shot below all the 21 records are for the documents we changed.

Enabling Real-Time Streaming

• Go to SAP Gui and open TCODE RSPC

• Go to Process chain, right click on the top node and click create Process chain

• Provide the Process chain name and long description

• Click on Create

• Provide Process variant name and description

• Maintain Start process screen will appear. Click on Save.

• Add the DTP and aDSO activation processes from the Left side window to the Process Chain by dragging and dropping to the process chain highlighted below. The connect the Variant to DTP by dragging Variant to DTP

• Click on streaming button.

• Select the ‘Process Chain Runs in Streaming Mode’ check box. Set the Error threshold and Log retention according to requirements. Click OK

• Activate the process chain by clicking on Activate icon

• Process chain is activated successfully and the Events are created for the same as highlighted below

• With this we Real time streaming is enabled for the Data flow and we see a Data extraction job is triggered and waiting for any changes. Now let’s check if data changes are getting reflected at real time.

• Let’s make the changes in same Document header text again and save.

• Check in tcode ODQMON. As we can see these is one Real time extraction request. Double click on the same

• There are 105 rows extracted. Double click on the row.

• We can see the entries for the change we made are there in the extracted records

With this we have completed the Real Time Data Streaming to BW using the CDS view.