LOGISTIC COCKPIT DELTA MECHANISM – Episode three: the new update methods

Brief summary of previous episodes…

In the beginning it was Serialized V3 Update.

After examining the conception and technical background of delta extraction in Logistic Cockpit by using this method (Episode one: V3 Update, the ‘serializer’ ), we also examined all peculiar restrictions and problems related to its usage (LOGISTIC COCKPIT DELTA MECHANISM – Episode two: V3 Update, when some problems can occur…).

Since performance and data consistency are very critical issues for a datawarehouse and, from this point of view, the ‘serializer’ started showing its not so reliable face,

with advent of PI 2002.1 (or PI-A 2002.1) three new update methods came upand, as of PI 2003.1 (or PI-A 2003.1), these methods have been completely replaced Serialized V3 update method, that it’s no longer offered.

Fig.1</b>: LBWE, Update Mode Selection Screen as PI.2002.1

Fig.2</b>: LBWE, Update Mode Selection Screen as PI.2003.1

Now let’s meet our new guests and try to discover when it’s better to make use of one than another one!

h4. 1. New “direct delta” update method: when in R/3, life is not so berserk…

With this update mode, extraction data is transferred directly to the BW delta queues with each document posting.

As a consequence, each document posted with delta extraction is converted to exactly one LUW in the related BW delta queues.

Just to remember that ‘LUW’ stands for Logical Unit of Work and it can be considered as an inseparable sequence of database operations that ends with a database commit (or a roll-back if an error occurs).

Fig.3</b>: Direct Delta update mechanism

Starting from the definition of this method we can see at once what are advantages and disadvantages resulting from direct delta usage.

BENEFITS AND…

As we can see from the picture above, there’s

no need to schedule a job at regular intervals(through LBWE “Job control”) in order to transfer the data to the BW delta queues; thus, additional monitoring of update data or extraction queue is not require.

Logically, restrictions and problems described in relation to the “Serialized V3 update” and its collective run do not apply to this method: by writing in the delta queue within the V1 update process, the

serialization of documents is ensured by using the enqueue conceptfor applications and, above all, extraction is independent of V2 update result.

…LIMITATIONS !

The number of LUWs per datasource in the BW delta queues increases significantly because different document changes are not summarized into one LUW in the BW delta queues (as was previously for V3 update).

Therefore this update method is

recommended only for customers with a low occurrence of documents(a maximum of 10000 document changes – creating, changing or deleting – between two delta extractions) for the relevant application.

Otherwise, a larger number of LUWs can cause dumps during extraction process and, anyway, V1 updatewould be too much heavily burdenedby this process.

Besides, note that

no documents can be posted during delta initialization procedurefrom the start of the recompilation run in R/3 (setup tables filling job) until all records have been successfully updated in BW: every document posted in the meantime is irrecoverably lost.

(Remember that stopping the posting of documents always applies to the entire client).

h4. 2. New “queued delta” update method: how to easily forget our old ‘serializer’…

With queued delta update mode,

the extraction data(for the relevant application) is written in an extraction queue (instead of in the update data as in V3)and can be transferred to the BW delta queues by an update collective run, as previously executed during the V3 update.

After activating this method, up to 10000 document delta/changes to one LUW are cumulated per datasource in the BW delta queues.

Fig.4</b>: Queued Delta update mechanism

BENEFITS, BUT…

When you need to perform a delta initialization in the OLTP, thanks to the logic of this method, the

document postings(relevant for the involved application) can be opened again</b> as soon as the execution of the recompilation run (or runs, if several and running in parallel) ends, that is when setup tables are filled, and a delta init request is posted in BW, because the system is able to collect new document data during the delta init uploading too (with a deeply felt recommendation: remember to avoid update collective run before all delta init requests have been successfully updated in your BW!).

By writing in the extraction queue within the V1 update process (that is more burdened than by using V3), the

serialization is ensured by using the enqueue concept, but collective run clearly performs better than the serialized V3and especially slowing-down due to documents posted in multiple languages does not apply in this method.

On the contrary of direct delta, this process is especially

recommended for customers with a high occurrence of documents(more than 10,000 document changes – creation, change or deletion – performed each day for the application in question.

In contrast to the V3 collective run (see OSS Note 409239 ‘Automatically trigger BW loads upon end of V3 updates’ in which this scenario is described), an

event handling is possible</b> here, because a definite end for the collective run is identifiable: in fact, when the collective run for an application ends, an event (&MCEX_nn, where nn is the number of the application) is automatically triggered and, thus, it can be used to start a subsequent job.

Besides, don’t omit that queued delta process extraction is

independent of success of the V2 update.

+…REMEMBER TO EMPTY THE QUEUE ! +

The ‘queued delta’ is a good friend, but some care is required to avoid any trouble.

First of all, if you want to take a look to the data of all extract structures queues in Logistic Cockpit, use transaction

LBWQor “Log queue overview”function in LBWE (but here you can see only the queues currently containing extraction data).

In the posting-free phase

before a new init runin OLTP, you should always execute (as with the old V3) the update collective run once to make sure to empty the extraction queue from any old delta records (especially if you are already using the extractor) that, otherwise, can cause serious inconsistencies in your data.

Then,

if you want to do some change(through LBWE or RSA6) to the extract structuresof an application (for which you selected this update method), you have to be absolutely sure that no data is in the extraction queue before executing these changes in the affected systems (and especially before importing these changes in production environment !).

To perform a check when the V3 update is already in use, you can run in the target system the RMCSBWCC check report.

The extraction queues should never contain any data immediately before to:

R/3 or a plug-in upgrade

2. import an

R/3 or a plug-in support packages

h4. 3. New “unserialized V3 update” update method: when a right delta sequence is not a requirement…

With this update mode, that we can consider as the serializer’s brother, the extraction data continues to be written to the update tables using a V3 update module and then is read and processed by a collective update run (through LBWE).

But, as the name of this method suggests, the V3 unserialized delta disowns the main characteristic of his brother:

data is read in the update collective run without taking the sequence into accountand then transferred to the BW delta queues.

Fig.5</b>: (Unserialized) V3 Delta update mechanism

It’s pleonastic to say that

all (performance) problems related to the serialized V3 update can’t applyto the unserialized one !

However, already known

V2 dependencykeep on subsist.

When this method can be used ?

Only if it’s irrelevant whether or not the extraction data is transferred to BW in exactly the same sequence (serialization) in which the data was generated in R/3 (thanks to a specific design of data targets in BW and/or because functional data flow doesn’t require a correct temporal sequence).

!https://weblogs.sdn.sap.com/weblogs/images/21406/CallSequence.gif|height=378|alt=image|width=576|src=https://weblogs.sdn.sap.com/weblogs/images/21406/CallSequence.gif|border=0!

Fig.6</b>: Comparison among call function hierarchies involved in different update methods

h4. Little recap of essential points to consider in migration process

If you want to select a new update method, you have to implement specific OSS notes, otherwise, even if you have selected another update method, data will still be written to the V3 update and it can no longer be processed !

Here is an OSS collection related to update method switch divided for application:

1. PURCHASING (02) -> OSS note 500736
3. INVENTORY MANAGEMENT (03) -> OSS note 486784
5. PRODUCTION PLANNING AND CONTROL (04) -> OSS note 491382
7. AGENCY BUSINESS (45) -> OSS note 507357

If the new update method of an application is the queued delta, it’s better to have the latest qRFC version installed.

However, before changing, you must make sure that there are no pending V3 updates (as suggested before, run the RMCSBWCC during a document posting free phase and switch the update method if this program doesn’t return any open V3 updates).

h4. Early delta initialization in the logistics extraction and final considerations

Not always on our systems an important downtime is possible in the initialization process during the reconstruction and the delta init request (just thinking when you need to ask a billing stop period…a real nightmare in some company !)

For this reason, as of PI 2002.1 and BW Release 3.0B, you can use the

early delta initializationto perform the initialization for selected datasources (just checking in infopackage update mode tab if this function is available): in this way you can readmit document postings in the OLTP system as early as possible during the initialization procedure.

In fact, if an early delta initialization infopackage was started in BW, data may be written immediately to the delta queue.

But, if you are working with the queued delta method, using early delta initialization function doesn’t make sense: as described before, it’s the same method definition that permits to reduce downtime phase.

But leaving aside that, don’t forget that, regardless of the update method selected, it’s ALWAYS necessary to stop any document postings (for the relevant application) during setup tables recompilation run !

h5. Final considerations

In the end, just some little personal thoughts…

After concluding this delta methods overview, it’s clear that

queued delta will be very probably the most used and popular delta method in Logistic Cockpit: if we consider direct and unserialized ones as exploitable only in specific and not so frequent situations (low delta document occurrences or no serialization needed), queued delta comes as legitimate heir to the throne before occupied by the old ‘serializer’.

Ok, all the elements are now available: it’s up to you making a right choice of delta method taking in consideration your specific scenario !