Customizing Cockpit: Managing delta queues when structure changes

Customizing Cockpit: Managing delta queues when structure changes

Overview

As we know the Logistics Customizing Cockpit enables you to enhance the Datasources of the logistics listed there from a pool of predefined fields and before changing the extract structures in Customizing cockpit (transaction LBWE).

We must carry out several tasks to avoid inconsistencies into the system because when you change a DataSource or their extract structure, the data that is saved in the old form of the structure is no longer interpreted correctly.

This guide will give you some steps to help you to avoid inconsistencies problems.

Reasons of these inconsistencies

When we change a structure, the data which is stored in the old form of the structure can no longer be interpreted correctly.

This can happen during following standard processes:

Update methods

1) Concept

There are three different ways to handle how the data can come from the document into the delta queue.

2) Types

These are the different types:

With PI 2002.1 the following new update methods for logistics extraction will be offered:

2.1) Direct Delta:

With this update mode, the extraction data is transferred with each document posting directly into the BW delta queue. In doing so, each document posting with delta extraction is posted for exactly one LUW in the respective BW delta queues. When the document is created or modified, it is transferred to RSA7 directly.

SAP recommends to use this update mode, when the number of the delta records between 2 delta loads are less then 100.000 records!

2.2) Queued Delta:

With this update mode, the extraction data is collected for the affected application instead of being collected in an extraction queue, and can be transferred as usual with the V3 update by means of an updating collective run into the BW delta queue. In doing so, up to 10000 delta extractions of documents for an LUW are compressed for each DataSource into the BW delta queue, depending on the application.

When the document is created or modified, it is transferred to LBWQ and after executing the collective run this information is transferred to RSA7.

2.3)Non-serialized V3 Update:

This update mode is recommended ONLY for the application “03”, the extraction data for the application considered is written as before into the update tables with the help of a V3 update module. They are kept there as long as the data is selected through an updating collective run and are processed. However, in contrast to the current default settings (serialized V3 update), the data in the updating collective run are thereby read without regard to sequence from the update tables and are transferred to the BW delta queue.

When the document is created or modified, it is transferred to SM13 and after executing the collective run this information is transferred to RSA7.

For additional info, refer to  note 505700 When using the method “Queued Delta”, it is strongly recommended, according to SAP Note 438015 that you use the newest qRFC version. (QRFC-Version 6.20.045, Supplement 9)

When using the new update methods “Direct Delta” and “Queued Delta”, the extraction with the new V1 update modules, such as MCEX_UPDATE_11_V1, is updated. In the case of “Queued Delta”, the storage of data occurs from there in the extraction queue with the RFC MCEX_UPDATE_11_QRFC.

3) Why is it necessary to know these methods?

It is important to know these because we will be able to understand the data flows to enable you to check the correct data containers before extract structure changes are made and to get a better explanation of which data containers must be emptied and at what time.

When a document is created or changed, the data is written in the document tables, and they are used in parallel for the update into the Business Information Warehouse (BW). Transaction data from the Logistics Customizing Cockpit is collected for transferring into BW in the delta queue (transactions RSA7 or SMQ1). Please refer to document in link: http://service.sap.com/~sapidb/011000358700005772172002 “contains the technical details about the new update methods.

Recommended steps

These are the steps that we need to follow before executing previous mentioned processes (plug-in upgrade, plug-in patch upgrade, R/3 upgrade, support package upgrade, cockpit datasource transports or manual changes and Unicode conversion. Please notice that the structure changes are client Independent, so in all clients has to be checked the extraction and delta queues before the changes!

They will be different in each case:

1) Only for R/3 upgrade:

Regarding the delta queues generated during the “online” phases of upgrade, as mentioned in note 328181 we always recommend to transfer all the data from the delta queue into the BW system before the upgrade and after that lock all the users and processes that can lead to a creation of data into the delta queue. Now the upgrade can be started and once that it has finished the users can be unlocked again. Anyhow, as this is a productive system and in order to have a lower downtime period, you can do the following:

1. Lock users out of R/3.

2. Deletion of the contents of set-up tables for specific applications in transaction LBWG.

3. Run the collective run so all the data are sent into the delta queue.

The collective run will execute according to the selected update method:

4. Empty BW delta queues on R/3 by extraction to BW:

5. Start prepare phase on R/3 including generation of shadow instance.

6. Return R/3 productive instance to users.

7. Recommence normal extraction into BW from R/3 while upgrading shadow instance.

8. Lock users out of R/3.

9. Run the collective run so all the data are sent into the delta queue.

10. Empty BW delta queues on R/3 by extraction to BW.

11. Final upgrade of productive instance.

12. Return upgraded system to users.

13. Re-commence extraction into BW from upgraded R/3.

Refer to SAP Note 1081287.

2) For the rest processes like plug-in upgrade or plug-in patch upgrade; support package upgrade; transports; manual changes; Unicode conversion.

1. Lock users out of R/3.

2. Deletion of the contents of set-up tables for specific applications in transaction LBWG.

3. Run the collective run so all the data are sent into the delta queue.

The collective run will execute according to the selected update method:

4. Empty BW delta queues on R/3 by extraction to BW.

5. Apply one of these processes: plug-in upgrade or plug-in patch upgrade; support package upgrade; transports; manual changes; Unicode conversion.

6. Unlock users out of R/3.

7. Run the collective run so all the data are sent into the delta queue.

8. Recommence extraction into BW from modified R/3.

The SAP Note 328181 describes the different errors when doing upgrades, importing support packages, transports, or making manual changes… And SAP Note 602260 also explains correct procedure for performing the statistical setup (reconstruction) and the delta initialization. Regarding to import support packages, consider the SAP Note 1083709.

3) Another issue to take into in account

If, despite an empty SM13, there is not possibility to make a structure change in the Logistics Customizing Cockpit, inconsistencies may occur in your system: There are records in the VBMOD update table, for which there is no longer an entry in the VBHDR table. To solve this, refer to SAP Notes 652310 and 396647(question 10). To solve the inconsistencies, follow the instructions in the Solution section of note 67014.

If you forget to process the V3 update before a change

Before deleting the delta data by customer, there can be an alternative solution (HASH Solution) or the logistic safety belt (see http://www.sdn.sap.com/irj/scn/weblogs;jsessionid=%28J2EE3417500%29ID0294196150DB01660827622326273616End?blog=/pub/wlg/5316))

If the above solution does not work, at the last solution will be to delete the delta data. And for this case it means that this data is lost. So, to do this,

1) For V3 update methods:

Go to the update overview (Transaction SM13), select the records (user *, V2 executed, set date), and select Update records -> Delete -> All records. Do not forget to deactivate the flag “Only incorrect”.

2) For update method “Delta Queued” (as of PI(-A) 2002.1):

Go to the logistics queue overview (Transaction LBWQ). Mark the queues for the applications concerned one after the other (MCEXnn, nn = number of the logistics application, see Transaction LBWE), and choose the display function. Then choose the display function again to list all entries of the queue. Then select and delete all entries that had been stored before the structure change. In case of a very large number of entries in a queue it may be required to delete the entire contents of the queue. This is possible already on the initial screen of Transaction LBWQ.

Guideline for structure changes in LWBE

We have a guideline as overview about structure changes in LBWE into the customizing of our R/3 system:

The available fields by default in LBWE are automatically filled during the extraction. Fields that are not available in LBWE can also be extended (user-defined fields, for example). We must then fill these fields in the exit user SAP Note 576886 describes the enhancement of extract structures with separate (user-defined) fields or fields that are not available in LBWE.

The changes to structures are frequently made in the development system and then these should be transported into the production system.

How to detect errors

Usually, when we did not follow the steps aforementioned to avoid inconsistencies, the main effect is regarding to NO Delta works for implicated cockpit datasources. There are several cases which are collected in our messages:

1. The V3 control jobs LIS-BW-VB_APPLICATION* finished with errors (transaction code SM37), usually the dump arises, for instance

The job of collective run uses the same reports as before (RMBWV3<Appl.-No)

2. For the rest cases like upgrades, applying patches or similar, the system shows that the structure was changed into the log.

Solutions:

To solve the dump problem  we can use the hash solution from the notes 835466 and 834897. The Hash-Solution enables you to redo the extractor structure change and load the “old” data up to the last old record, than switch to the new version of the extractor structure and start the normal data loading.

Otherwise another alternative is to reinitialize the involved application.

Please remeber that it is very important to clear all of the delta queues before an upgrade or changing of structure to avoid such kind of problems.
Please take a detailed look at the note 1081287 as well.

More relevant Transaction codes and Reports

LBWE: to display all datasources of LO Data extraction: Customizing Cockpit.

LBWF: Log for logistics Extract Structures

LBWG: Deletion of Setup Data. The deletion will be executed by component,for instance for “04” component, all data from these setup tables will be deleted.

LBWQ: Logistics Queue Overview

SM13: Administrate Update Records

RSA7: BW Delta Queue Monitor

NPRT: Log for Setup of Statistical Data: Selection

To fill the set up tables of customizing cockpit we have the following transaction codes / reports:

1. Inventory Management (03):

2. Purchasing (02) ->OLI3BW /RMCENEUA

3. Production (04):

4. – Quality Management (05) ->OLIQBW / RMCQNEBW

5. – Plant Maintenance (17) ->OLIIBW / RMCINEBW

6. – Service Management (18) ->OLISBW / RMCSNEBW

7. – SD-Sales-Orders (11) ->OLI7BW / RMCVNEUA

8. – LE-Deliveries (12) ->OLI8BW / RMCVNEUL

9. – SD – Billing Documents (13) -> OLI9BW / RMCVNEUF

10. – LES-Shipment Costs (08) -> VIFBW / VTRBWVIFBW

11. – LES-Transport (08) ->VTBW / VTRBWVTBWNEW

12. – Retail (40) ->ORISBW / RMCBWEXUSEG

13. – Agency business (45) ->OLIABW / RMCENEUB

14. – Invoice Verification (06) ->OLI6BW / RMCENEUR

Relevant SAP Notes

1081287 Data extraction orders block the upgrade process

1083709 Error when you import Support Packages

328181 Changes to extract structures in Customizing Cockpit

762951 Organization of structure changes in the LBWE

834897 Avoiding inconsistent data with extract structure

380078 FAQ: BW delta queue (RSA7): Questions and answers

396647 FAQ: V3 update: Questions and answers

835323 Hash solution – Application 11

835457 Hash solution – application 13

835466 Using the repair mode of the hash solution

602260 Procedure for reconstructing data for BW

67014 Reorganizing update requests

652310 LBWE: Message MCEX 140 with extract structure change

Your comments and experiences about the contents are very welcome.