Archiving & Deletion of messages in PI - Part 3

[deepak_shah]

          Part 3 of the blog series explains the steps to be performed for deletion of xml messages.

          XML messages processed by the Integration Engine are saved in the multiple database tables when persisted. XML messages are saved in a hierarchy.  Each XML message has a master entry that contains the most important data, such as the message ID, QualityOfService or date of execution.  This data is also used as search criteria in dynamic searches for XML messages. All additional table entries are attached to this master entry. For each message version, the header and body of the XML message and the message payload are saved in a separate table for each version, along with a version record.

The XML messages are not automatically deleted from the database tables. You can determine whether an XML message is to be archived or deleted, according to the interface. The deletion and archiving jobs are scheduled in cycles using the administration interface.

If a message is to be deleted, the first step is to set the flag Flagged for Deletion in the master entry. The second step is then to delete all dependent table entries.

If a message is to be archived, the first step is to set the flag Flagged for Archiving.  You can then archive all dependent table entries. Finally, you can delete all archived table entries.

Below are two different procedures for deleting XML messages from the database tables:

Simple Deletion Procedure:

The simple deletion procedure deletes all XML messages flagged for deletion or archiving from the database tables in records. This procedure is recommended when small volumes of XML messages are to be processed by the Integration Engine. However, large data volumes have an adverse effect on performance during the deletion process.  For this reason, the switch procedure is recommended when processing large volumes of messages. To schedule delete jobs, proceed as follows:

1.        In the Integration Engine-->Administration menu, choose +Schedule Delete Jobs.

Table Switch Procedure: The switch procedure is based on the fact that an identical table copy exists for each persistence layer table which is shipped by SAP. To begin with, the original tables are the active tables. All XML messages are saved in these tables. When the deletion job is started, the table entries are not physically deleted from the database tables as in the procedure above, instead the flag Deleted is set in the master entry. The monitors then no longer display this XML message. When a certain fill level is reached (currently 90% as shown in screen shot below), the deletion job recognizes that a reorganization (or switch) is required. The table copies that were inactive before now become active tables. All new XML messages are written to the table copies. For all existing XML messages, the system checks whether the delete flag in the master entry is set or not.  If it is not set, all corresponding table entries from the original tables are copied to the table copies. Once the all the table entries have been copied, the original tables in the database are deleted and then recreated again immediately.

Set parameter "DROP_MAX_TABLE_LOAD" to determine levels at which Switch procedure is required as shown in screenshot below.

To Activate Table Switch procedures go to transaction SXMB_ADM -->Configure Delete Procedure.

Select the check box to activate switch procedure and click on Save.

Now schedule delete jobs as per steps defined above. 

In above case switch procedure was not required as the tables were only 1% filled as against 90% configured in SXMB_ADM. You can reduce the table fill level as per your needs so that the switch process is carried out.

Choosing a Deletion Procedure:

The simple deletion procedure is set as the default. No steps are required to activate the deletion procedure. It is easy to activate the switch procedure. To do so, select the Switch Procedure Active checkbox. You can do this at any point. The deletion jobs react automatically.

You can only deactivate the switch procedure when the original tables are active and the counter for the number of deleted records in the original tables is at zero. If this is not the case at the time of configuration, this is noted by the system and the switch procedure is deactivated the next time that table entries are copied from the table copies to the original table.

You can refer to other parts of the blog series at following links:

[Part1: Introduction | Archiving & Deletion of messages in PI - Part 1]

[Part 2: Archiving of xml messages | Archiving & Deletion of messages in PI - Part 2]

[Part 4: XML messages in AE & BPE Workitems | Archiving & Deletion of messages in PI - Part 4]

[Part 5: Performance header tables & TIPS | Archiving & Deletion of messages in PI - Part 5]

[Part 6: Transport Archive configurations | Archiving & Deletion of messages in PI - Part 6]