Streaming Process Chains

Context

To update and ADSO in near-realtime, one has to consider the Process Chain for Streaming (available since BW 7.5 SP03; information can be found at https://help.sap.com/viewer/2e90b26cf7484203a523bf0f4b1bc137/7.5.4/en-US/ac029de05e164a12ac1ce08d16180f05.html).

Using RDA (realtime data acquisition) does not support updating ADSO’s (only cubes, classic DSO’s, and alike), and has thus become yet another obsolete BW-artifact.

This blog shows the setup in a nutshell in context of both ODP-SAPI and ODP-SLT datasources.

ODP-SAPI

In this case, the basis are the well-known (sometimes rich in business-logic) extractors.

In this example, I’ve used the 2LIS_11_VAITM “Sales Document Item Data”, for which I’ve set the Update Mode to “Direct Update” in the Logistics Workbench in the source system (this aspect is covered sufficiently elsewhere).

The source-system connection is of type ODP-SAPI. This implies that after applying an initial load in BW, there is no more usage of the qRFC-mechanism (so no more RSA7), but now we have to take a look at the queues and subscriptions via at ODQMON (on the system where ODQ has been implemented) :

Remark that there is also no need to use PSA-level due to the usage of ODQ; so both qRFC as PSA level is being replaced via one ODQ-queue.

Due to the usage of Direct Update, any change in the transactions (in this case VA01 or VA02, or alike) will immediately insert the change-records into ODQMON-queue; very alike the behavior in RSA7, actually.

One can use this part for further batch-processing, with whatever target-BW-object.

The transformation is just a straightforward 1:1; the ADSO is purely field-based (just had to indicate the right key-elements and some key-figure aggregation behavior). All this can actually be performed in a couple of minutes.

Now the fun starts : we want to use this setup to update the ADSO as soon as possible.

Trying to create a DTP of type “RealTime” is not possible : it is removed from the drop-down list as soon as the target is and ADSO !!

And actually it is no longer necessary to do so. Just use a regular delta-DTP, and put it into a Process Chain. As soon as you tag the flag “streaming” in that process chain, and schedule it, the according DTP’s will behave as “realtime” :

Once (and as long as) this Streaming Process Chain is scheduled, you will find one request  in ODQMON which has status “Extraction Running”; as soon as the Streaming Process Chain is out of scheduling this request receives the status “Confirmed”.

As a consequence, the process chain is running with the indicated frequency (pull). Herewith the resulting view of updates of the ADSO.

ODP-SLT

In case it is necessary to replicate a table, or if a delta is not easily available via the above described extractor technique, one can use SLT. To update the SLT-queues towards BW yet another ODQ-layer can be introduced.

To activate the SLT/ODQ setup and subscription, it is sufficient to first create a new datasource in BW for the table needed, and load a first delta-DTP. The example we are showing here is VBAP :

Then I’ve created a field-based ADSO using this datasource as a template, and a 1:1 transformation between the datasource and this ADSO. This is all depending on the use-case at hand.

Once done, a normal delta-DTP can be created.

The first execution of this might take a while, as this is setting up the SLT-specific delta-capturing mechanism in the source-system; it will finally also result in an initial load data-set (and an additional initial line in SLT/ODQMON).

Again : you can use the above for batch-purposes.

To now start streaming the changes towards the ADSO, put the delta-DTP in a process-chain, flag “streaming”, and schedule. Remark : if you schedule according to ‘API’, the process chain will be triggered event-based via ODQ !! Meaning : this is a real-time push-mechanism.

As such chain will only process based on an event coming from the SLT/ODQ, the number of update-requests in the ADSO is largely reduced : only changes are resulting in an update request (in real-time)

Conclusion

Although SLT is purely table-based, so it doesn’t contain any business logic in itself, it it a very powerful delta-enabling technique, and it is capable of pushing changes in real-time to BW’s ADSO’s.