Introduction and Use Case
AIF provides an own persistence called xml persistence to store and keep messages. Thereby each message gets transformed in an xml document and stored in a single field as a byte-string/blob on db level. Dedicated keyfields are used to ensure fast searching and selections on messages. Many features in AIF rely on having chosen the right keyfields (e.g. filtering in /AIF/ERR). In general this leads to the following limitations:
- All keyfields have to be defined in advance. Adding a keyfield on existing messages is nearly impossible but sometimes necessary (e.g. when having many messages with errors related to a specific value of a non-keyfield-field).
- Adding a keyfield requires changes on the corresponding index table or even the creation of a new index table.
- Redundant data and an additional overhead due to storage in xml documents.
- Especially when using HANA, using the messages' fields in CDS views, joins or calculations is not possible.
The newly released AIF SP 5 now offers a new persistence implementation called structured persistence as an alternative to the existing xml persistence to finally solve the mentioned issues. This how-to-guide explains how to activate/use the structured persistence for an interface.
How does the Structured Persistence work?
Overview
Instead of storing a message as an xml document the messages get persisted in a structured and atomic way. Therefore a generator tool creates and deploys a relational database schema especially designed for the raw structure of a given interface. All messages of this interface can then be stored in the generated tables. Because the tables are generated to fit to the message’s raw structure, every field in the message is represented by a column in the persistence table. In contrast to the xml persistence now every field can be used as a (multi-index-)keyfield at any time without any additional changes or impacts (when using HANA – on non-HANA databases it might be useful to manually add additonal indexes). Furthermore there is no additional overhead or any redundant data anymore.
The structured persistence is implemented as a set of new AIF engines. It can be activated in the AIF customizing for each interface individually (/AIF/CUST -> Define Interfaces -> Specify Interface Engines). The structured persistence works completely transparent. Referring to the application help, the structured persistence is recommended for new interfaces ( ℹ ). Unfortunately in SP 5 there is no standard way to migrate existing messages of an interface.
XML Persistence vs. Structured Persistence
The following table gives an overview on the differences of both persistence layers:
| XML Persistence | Structured Persistence | |
|---|---|---|
| Fully and seemly integrated in AIF | ➕ | ➕ |
| Works transparently | ➕ | ➕ |
| Method of storing messages | Messages are stored as xml documents in blop-columns | Messages are stored in a relational db schema |
| Available since | AIF 1.0 | AIF 3.0 SP 5 |
| No installation needed | ➕ | ➖ |
| Single-index-keyfields w/o theneed of redundant data | ➖ | ➖ (currently not supported, but theoretically possible with some manual adjustments) |
| Multi-index-keyfields (on raw structure) w/o the need of redundant data | ➖ | ➕ |
| Compression | ( ➕ ) In application layer available since SP 4 | ( ➕ ) Depends on the db (e.g. HANA automatically chooses and performs different compression algorithms) |
| Keeps multiple versions of a message | Yes (since SP 4 only the original and the latest message version are kept) | No (only the most recent message version is kept, but still manual updates are recorded by the error handling) |
| Use fields of messages in native db features (e.g. joins, CDS views, etc.) | ➖ | ➕ |
Preparation
For this how-to-guide we need a configured AIF interface. Feel free to use an own interface or copy an existing one. As preparation I created an own namespace called Z_FZ10 and copied the standard FLBOOKING interface of namespace /AIF/ via transaction /AIF/CUST_SMAP_COPY.
Furthermore I decided to change the keyfield settings in order to better demonstrate the structured persistence. Therefore I went to transaction /AIF/CUST and executed Error Handling -> Define Interface-Specific Features. I deleted some keyfields and switched the origin of some others to “Source Structure”. Please note that this change also affects the configured selection screen in the error handling transaction. I did not modify the selection screen and therefore not all parameters/fields are working now. But it is just a demonstration :wink:
The final keyfield configuration should look like the following:
In addition I adjusted the interface determination, in order to receive messages on this interface.
How to Configure
The configuration can be done in three steps:
- Generate the db schema/tables by using the generator report
- Switch the engine configuration in the interface customizing to enable the structure persistence
- Adapt the keyfield settings to benefit from the structured persistence
These steps will be explained in the following sections in more detail.
Step 1: Generate DB Schema
The db schema contains all db tables, which are necessary to store messages. The tables can be automatically generated with the generator report by using transaction /AIF/PERS_TBL_GEN. The report needs an interface as input but works on the raw structure. This means, when having multiple interfaces with the same raw structure you only have to generate the db schema once for one interface.
To execute the report, you have to enter your interface (Z_FZ10 / FLBOOKING / 1). The name of the RAW structure will be filled automatically when pressing the enter-key. Then enter a package name (you also can use the local package by entering $TMP) and a prefix (e.g. ZFZ10_). The prefix is used for all generated db tables and has to be unique for a single interface. In case you do not use the local package, you also have to provide a transport request. All generated tables and db records will be recorded there (no customizing request is needed).
The report also provides a settings-button (
), which navigates to a maintenance view. The maintenance view can be used to define special parameters to influence the generator (e.g. to limit the max. number of columns per db table). I kindly refer to the report’s application help for further details.
To generate and calculate the db schema simply press the execute-button. As result the next screen appears, which displays a preview of the db schema and thederived mapping. The mapping defines which parts of the raw structure map to which parts in the db schema (and has nothing to do with the structure mapping :wink: ). The left column shows the components of the raw structure. The empty value (first row) refers to the structure’s root. In our example the structure’s root maps to the table ZFZ10_ROOT. The field BOOKING-CUSTOMER of a message will be stored in the column CUSTOMER of the same table. The right columns define the type of the columns.
The generator report uses an optimization logic to reduce the number of needed tables and to avoid collisions. Aside from the fact that our example uses a rather simple raw structure, the generator report only uses a single db table for the raw structure including the BOOKING-substructure.
To confirm and finally generate/deploy the db tables to the DDic and DB simply press the execute button. The result will be displayed in the last screen, which contains the written application log.
Please note: The generated db tables do not specify any additional indexes (only the primary key). This is optimal when using HANA. In case you are using any other db, it is recommended to manually add an index for every multi-index-keyfield.
Step 2: Switch Engine Configuration
As next step, you have to change the interface customizing to enable the structured persistence for the interface to finally use the generated db tables.
Therefore go to /AIF/CUST and execute Interface Development -> Additional Interface Properties -> Specify Interface Engines. Enter your interface (Z_FZ10 / FLBOOKING / 1). Then select ‘Structured Persistence’ as Application, Persistence and Selection engine. Save your changes. Now you all new messages will be persisted in the generated db tables by using the structured persistence.
Step 3: Adapt Keyfields
One benefit of the structured persistence is the keyfield handling. The generated db tables directly can be used as multi index tables for multiple search keyfields. Therefore you do not have to maintain custom index tables and there is no redundancy for these keyfields anymore. Now we want to adapt the existing keyfields to use the generated persistence tables.
Therefore go to transaction /AIF/CUST and execute Error Handling -> Define Interface-Specific Features and enter your interface (Z_FZ10 / FLBOOKING / 1). The FLBOOKING interface has several keyfields configured (I changed them as described in the Preparation-chapter). To best leverage the structured persistence we will change the following:
- Switch to Multi-Index-Keyfields (if possible): Currently all keyfields of the FLBOOKING-interface are using a single index table. With SP 5 the benefits of the structured persistence can only be leveraged with multi-index-keyfields. Therefore we will switch all keyfields to multiple search keyfields – except of the first one with Field Sequence = 1, because this one is used to structure the ALV-tree in the error handling transaction, which is only supported for single index keyfields.
- Change index table: Furthermore we change the “Message Index Table Name” to the corresponding persistence tables and adjust the keyfield name to match the column in the persistence table. In our example, the multiindex table is always the ZFZ10_ROOT table. In more complex examples, you have to check the mapping to determine the correct table (you can display the mapping at any time by executing the first step of the generator report again).
- Adjust keyfield names (if necessary): The keyfield name has to match the name of the corresponding column of the multi-index-table/persistence-table. In our example the names are already correct.
The result should look like the following:
Test the Configuration
In order to test the example, we will send some messages to AIF by using the test tool. Therefore call transaction /AIF/IFTEST and proceed like the following:
- Press the Search-button
- Create a new test file by pressing the “New Data”-button and providing the interface (Z_FZ10 / FLBOOKING / 1).
- Enter a description into field “File Description” and press the Save-button.
- Select the newly created test-file and press the “Read Data”-button.
- Enter some test data into the raw structure (upper section). I used a valid flight from the SFLIGHT table, but changed the carrier to produce an error. A valid customer can be found in table SCUSTOM and a valid agency in table STRAVELAG. (In general you also could use some arbitrary data.)
- Save the test-file and press the “Process in XML Runtime”-button. This directly transfers the message to AIF.
To view the message go to transaction /AIF/ERR. Enter the interface (Z_FZ10 / FLBOOKING / 1) and press the execute-button. In my example, I can see one erroneous message. You also can see the message’s payload as entered in the test-tool:
This data now is retrieved from the generated persistence table. You can check the generated table ZFZ10_ROOT in transaction se16 and there should be one record containing the message’s payload:
Now we are going to edit the message in the error handling tool. I just changed the carrier from “LH” to “AA” and saved the changes. Then I reloaded the record and the field changed:
And I was also able to search for the message by using the CONNID-keyfield and to restart the message, which finally was processed successfully:
So the interface is still working and the structured persistence is used transparently.
Summary
This blog post gave an introduction on how to use the structured persistence. The structured persistence is a new implementation of the AIF’s integrated persistence layer. It allows storing messages in its natural figure by using a generated relational database schema. Once generated the db schema and switching the engines of the interface, the structured persistence works transparently. One of the biggest benefit is to use the persistence tables as multi-index-keyfield (on the raw structure). Furthermore you also can add further multi-index-keyfields at any given time and they will work also on existing messages out of the box.
PS: I am happy to receive your feedback to this blog post.
- SAP Managed Tags:
1 Comment
