Skip to Content
Author's profile photo Maksim Alyapyshev

How to create delta-enabled BW DataSource based ABAP CDS views

Hi!

In this post I would like to consider creation of generic data extractor with delta supported option. Logic of data preparation will be in CDS view with special annotations.

I am using S/4HANA 1610 with NW 7.51 for creation a demonstration examples.

Step-by-step procedure:

I.I am using standard CDS I_SalesDocument of SD application component as a starting point. Analyzing it in HANA Studio ABAP perspective we could see main source table is VBAK.

II. Create new CDS view for data extraction to SAP BW (ODP_CDS source system). I am not included all standard fields here, just some for demonstration.

@AbapCatalog.sqlViewName: 'ZDS_ISALESDOC'
@AbapCatalog.compiler.compareFilter: true
@AccessControl.authorizationCheck: #CHECK
@EndUserText.label: 'CDS for Extraction I_SalesDocument'
@Analytics:{dataCategory:#DIMENSION ,
            dataExtraction.enabled:true}
@Analytics.dataExtraction.delta.byElement.name:'LastChangeDateTime'
@Analytics.dataExtraction.delta.byElement.maxDelayInSeconds: 1800
            

define view Zds_I_Salesdocument as select from I_SalesDocument {
key SalesDocument,

      //Category
      SDDocumentCategory,
      SalesDocumentType,
      SalesDocumentProcessingType,

      CreationDate,
      CreationTime,
      LastChangeDate,
     @Semantics.systemDate.lastChangedAt: true
     LastChangeDateTime,

      //Organization
      SalesOrganization,
      DistributionChannel,
      OrganizationDivision,
      SalesGroup,
      SalesOffice,
      
      //Pricing
      TotalNetAmount,
      TransactionCurrency,
      PricingDate,
      RetailPromotion,
      PriceDetnExchangeRate,
      SalesDocumentCondition
    
}   
 

Notes:

  1. For extraction we need annotation @Analytics.dataExtraction.enabled: true. Prerequisites are annotations @Analytics.dataCategory #CUBE, #FACTS, or #DIMENSION (our case) or @ObjectModel.dataCategory #TEXT, or #HIERARCHY.
  2. For making delta option works we also need some annotations:@Analytics.dataExtraction.delta.byElement.name This is the element that should be used for filtering during generic delta extraction.
    This element can either be a date (ABAP type DATS) or an UTC time stamp.

    @Analytics.dataExtraction.delta.byElement.maxDelayInSeconds

    There is always a time delay between taking an UTC time stamp and the database commit. This annotation specifies the maximum possible delay in seconds. The default is 1800 seconds.

    For more details about these annotations see documentation here.

    An additional annotation to support the detection of deleted records is planned, but not yet available.

  3. In our case delta field is LastChangeDateTime which is originally located in VBAK and have UTC time stamp data type.

 

III. That’s it for CDS view part. Now go to SAP HANA BWMT perspective, create a DataSource in ODP_CDS source system, check that Extraction tab contains delta option AIM. Activate it and test data extraction.

IV. For demonstration of delta work I will use familiar method of loading with PSA and data packages: FULL, INIT with data load, DELTA.

Testing procedure will be following:

1. Start INIT package with data transfer. Check PSA.

Number of record is the same as number of record in CDS view data preview.

2. Check initialization in Scheduler – Initialization Options for Source System on Info-package selection.

3. Go to VA02 in S/4 HANA to change information in Sales Order (SO). I changed quantity and should see difference in Net price of SO.

Before 300 EUR

After 350 EUR

4. Start delta package in BW. Check PSA, at least 1 record should be there in second data request (delta). Do not forget that our delta process is AIM.

Only one delta record is there with Net Value is 350 EUR of SO Number equals 2.

5. You could optionally go to ODQMON t-code in S/4 HANA system and check ODP queue here.

We have 2 Requests here: first for initialization, second for delta extraction.

Conclusion

We have walked through a step-by-step path from new CDS view creation to delta extraction to SAP BW system.

 

Thank you for attention!

Assigned tags

      14 Comments
      You must be Logged on to comment or reply to a post.
      Author's profile photo Githen R
      Githen R

      Nice Article ... Very relevant for S/4 HANA - BW combination

      Author's profile photo Former Member
      Former Member

      Hello Maksim, Great work, thanks for the effort. But I would say that, by using a time stamp mechanism, if you extract a document 1234 once to BW and you delete it afterwards in the source, the "event" of deletion would not be forwarded (differing from the "old" LIS extractors). Did you test it? Kind regards, Philipp

      Author's profile photo Eduardo Provenzano
      Eduardo Provenzano

      Hi Philipp,

      I understand that such a logic for deleted records is not yet in place for this timestamp case.
      As mentioned in the document, an additional annotation to support the detection of deleted records is planned, but not yet available.

      Kind regards,
      Eduardo

       

      Author's profile photo Sergii Zaitsev
      Sergii Zaitsev

      Hi, Maksim!

      Is it really need the annotation – “@Semantics.systemDate.lastChangedAt: true”  for this  delta process?

      Sergii.

       

      Author's profile photo Maksim Alyapyshev
      Maksim Alyapyshev
      Blog Post Author

      Hi, Sergii! I didn't test, but think that this annotation is not mandatory.

      Author's profile photo Former Member
      Former Member

      Hi Maksim,

      Really Informative article. I am trying to do the same thing in one of my CDS views but somehow I am not getting the delta option in annotation. It is just showing one option that is

      @Analytics.dataExtraction.enabled: true

      I tried on both S/4HANA 1510 and 1610 versions. Do you have any idea what the issue could be?

      Thanks

      Hassaan

       

       

      Author's profile photo Maksim Alyapyshev
      Maksim Alyapyshev
      Blog Post Author

      Hi Hassaan!

      I used 1610 and all was ok. May be problem could be with old version of HANA Studio and/or ABAP Development Tool Add-In. Try to update ADT.

      BR, Maksim

      Author's profile photo Former Member
      Former Member

       

      Hi Maksim Alyapyshev  - Thank you for this detailed post.

      I can see that SAP has released the delta deletion annotation (at least in their documentation), but I am not able to get it working on my system. Have you tried the new annotation with any success?

       

      Author's profile photo Maksim Alyapyshev
      Maksim Alyapyshev
      Blog Post Author

      Hi Former Member, I didn't try yet. Could you please share documentation links?

      Author's profile photo Simon Kranig
      Simon Kranig

      Hi Maksim,

      check here:

      Help Page for current SP

       

      Using

      Analytics.dataExtraction.delta.byElement.detectDeletedRecords

      and

      Analytics.dataExtraction.delta.byElement.ignoreDeletionAfterDays

       

      Author's profile photo Bodhisattva Chattopadhyay
      Bodhisattva Chattopadhyay

      Hello Maksin,

      We tried the exact process end to end as suggested by you here...

      creating a data source (ZISDSALESDOC_P) on the CDS View ZI_SalesDocument (which again is a copy of I_SalesDocument ) . The Real-Time Data Replication is not enabled .

      Is there any patch missing in our system or we are doing something wrong ?

      Please find reference screenshot showing the Real-Time Data Replication option is disabled...

      We are now stuck in a bit of pressure, so your suggestion here is highly needed...

       

      Author's profile photo Maksim Alyapyshev
      Maksim Alyapyshev
      Blog Post Author

      I suppose that this option is not availible. Use batch load through process chains with DTP, monitor records in ODQMON in source system. Intialize delta and then load delta periodically.

      Br, Maksim

      Author's profile photo Bodhisattva Chattopadhyay
      Bodhisattva Chattopadhyay

       

      Thanks Maksim for your prompt reply.

      Use batch load through process chains with DTP, monitor records in ODQMON in source system. Intialize delta and then load delta periodically- Does that means we can not replicate ABAP CDS view on Real time basis and do we need to schedule on batches?

      Just to give you more details on requirement, we created CDS view with Delta enable annotations

      and while creating Datasource in BW Modelling, Real-Time Data Replication is not Supported (by default it is disabled )

      However, in step III mentioned in your blog, it is showing as Real Time Acquisition is Supported; but for us it s disabled with Real Time Acquisition is Not supported (refer screen shot above)

      Could you please suggest, is there any patch/note missing in our system or any annotation missing? Please advise.

      Author's profile photo Debdeep Ray
      Debdeep Ray

      Hello Maksim

      Many thanks for the useful document

      Are we able to process that data package wise?

      I know we can maintain packagesize in DTP settings, but is it actually being used while the data is being fetched through view from database?

      If you compare traditional BW Function Module based generic extractor, where we used to mention maxsize along with cursor in the ABAP code, and that actually directly relates to infopackage package size.

      In case join output from CDS view is huge and it tries to process the data at once, then there is a chance of memory dump

       

      Thanks

      Deb Deep Ray