Last Changed: 18th of April 2023

SAP ILM with SAP IQ Database

Blog Content

• SAP ILM – Important Questions and Answers
• SAP ILM – Documentation and Information
• SAP IQ Database Setup for ILM
• SAP IQ Monitoring with DBACOCKPIT
• Setup SAP Information Lifecycle Management
• Configuration of the SAP ILM Framework
• Define the Origin Concept for SAP ILM
• Setup and Publishing of the SAP ILM Store
• Testing the SAP ILM Functionality with IQ
• SAP ILM – Next Steps …

---

Manage information through its lifecycle while balancing cost, risk, and compliance.

Streamline your IT infrastructure and protect the privacy rights of your consumer data with the SAP Information Lifecycle Management (SAP ILM) component. You can automate data archiving and retention, as well as the decommissioning of legacy systems, while balancing the total cost of ownership, risk, and legal compliance.

 

While the nearline storage functionality for the SAP Analytical (OLAP) based Systems like SAP BW and BW/4 with SAP IQ Database is well established, the usage of the SAP IQ Database for the general storage of BLOB, ILM-controlled archive files and archive indexes from transactional (OLTP) Systems like R/3, ERP or S/4 Data is also suitable option to use.

• SAP Commitment to On-Premise SAP IQ Customers (May 2020)

So let’s optimize the SAP ILM Store Setup with the experiences and strengths of the SAP-NLS Implementation known from SAP BW and BW/4.

Keep in Mind that SAP IQ is not the primary Database here, where the Application Data is stored, it is the secondary Database where the mentioned Objects from above stored for anytime fast access outside the primary Database for shrinking the main Database, e.g. SAP HANA. SAP IQ can compress the SAP HANA Data up to 80% in size due to the column based compression logic.

---

Important Questions and Answers

Blog – Should you archive both BW (or BW/4HANA) and ERP (pre-S/4HANA) on the same instance of SAP IQ? – from Mark Mumy (SAP)
No – Note 3085053 – Landscape Considerations for NLS and ILM

Question – Is an ILM store mandatory to use the ILM Retention management feature in ILM?
Yes, to fully utilize the functionality of ILM an ILM Store is needed, which meets the BC-ILM 3.1 certification.

Question – Is the SAP HANA Datalake ready to handle the ILM/NLS capabilities?
No, see the comment of the online Document – Top 5 Reasons to Migrate SAP IQ Systems to SAP HANA Cloud, Data Lake

 

---

SAP ILM – Documentation and Information

Blog – Mastering the Data Deluge: A Comprehensive Guide to SAP’s Volume Management, Archiving, and Disposition

SAP ILM Detailed Presentation (PDF)

SAP ILM Archiving Client Options

SAP HELP Documentation

– SAP ILM Help
– SAP ILM Implementation Guide
– NW ILM Store
– Quick Configuration Guide for the ILM Store (SAP IQ)
– SAP Information Lifecycle Management in SAP S/4HANA (7.55)

SAP Customer Activity Repository (CAR) – Install and Set Up Integration with SAP IQ

SAP Solution Brief – SAP ILM – ILM Store Configuration –

Note 2222074 – Collective information regarding ADK and AIS (Archive Development Kit and Archive Information System)
Note 2705663 – Collective information regarding ILM (Information Lifecycle Management)

The IQ implementation is the same as for NLS, so the following Notes are relevant for ILM as well.
Note 2780668 – SAP First Guidance – BW NLS Implementation with SAP IQ
Note 3017355 – SAP IQ 16.1 SP04 PLx – correct SAPIQDB.cfg settings
Note 3094594 – Enable SAP IQ for DBACOCKPIT
Note 3085053 – Landscape Considerations for NLS and ILM
Note 3119008 – Configure SAP IQ and HANA for SDA/ODBC
Note 3196447 – troubleshooting an existing SAP IQ 16.x Implementation for NLS/DTO and ILM

despite of SAP ILM or SAP-NLS, it is all about the SAP IQ Database Setup
getting a working ILM License for IQ needs the following options (separately purchased)

Note 1985209 – How to enable a IQ_UDA license – SAP IQ
Note 2110827 – Error 131275 ‘IQ functionality that requires the IQ_LOB license will be disabled’ – SAP IQ
Note 2183758 – Error ‘Insufficient quantity authorization available for IQ_VLDBMGMT license’
Note 2372218 – Requisite IQ licenses for SAP-NLS solution – SAP BW with SAP IQ Nearline-Storage
Note 3159879 – ILM licence information and validity

– IQ_CORE             SAP IQ Enterprise Edition/AS Platform Edition (avail. CPU Core)
– SY_IQUDA           SAP IQ Unstructured Data Analytics Option (avail. CPU core) IQ_UDA/IQ_LOB
– IQ_VLDBMGMT   SAP IQ Very Large Database Management Option (avail. CPU Core)

 

---

Just for the Record: the following SAP Notes are obsolete/not necessary, when the SAP IQ Database is installed in the mentioned way above. (Components BC-SYB-IQ,  BC-ILM-STO)

Note 2161052 – ILM Store: Free connection types
Note 2257318 – ILM Store: Lock Exception using IQ
Note 2284492 – ILM Store: Message “Connection limit exceeded” on IQ
Note 2414314 – IQ: PUT succeeds, GET fails with Row not found
Note 2478703 – ILM Store – IQ: Put Succeeds, GET fails with Row not found for the Streaming Case
Note 2634053 – SAP ILM and Sybase IQ storage
Note 2705663 – Collective information regarding ILM (Information Lifecycle Management)
Note 2748058 – During ILM storage “DETERMINE_PUT_TARGET” error occurs in SLG1
Note 2785785 – DAP ends with IQ Error SQL-131 Syntax error near ‘(end of line)’
Note 2806115 – Error binary data not supported on data longer than 32767 Bind host variable – SAP ILM and IQ
Note 2942648 – ILM store: error during storage of files with size greater than the streaming size
Note 2969212 – ILM : Error while deleting documents from ILM store set up as remote content repository

these are some corrections in relation to SAP IQ (Components BC-CCM-ADK, BC-ILM)

Note 3010127 – Dump CX_ILM_STOR_EXCEPTION during report RILM_STOR_TEST_PF_SINGLE
Note 3010305 – IQ: Structure tables are not filled on SAP IQ during setup of inactive infostructures
Note 3017579 – DBACOCKPIT: Monitoring multi sys using the SQL Statements cockpit action

Update June 2021 – Improvements when testing the ILM Functionality

Note 3020979 – ILM Store : Performance improvements during the archive file read process in ILM store

Note 3064301 – ILM Store : Runtime errors occur when modifying an operational origin

Note 3061532 – LHM : Runtime errors when adding objects to a legal case using E-Discovery report

Note 3060009 – ILM Store : Report RILM_STOR_TEST_PF_REPEAT_SMALL contains incorrect description for blob size

Note 3055028 – ILM store: Report RILM_STOR_TEST_PF_REPEAT_BIG considering incorrect BLOB size

 

---

SAP IQ Database Setup for ILM

Consolidate Your Archived Data on SAP Sybase IQ

Blog – Q – the easy Installer for SAP IQ (for the Installer contact roland.kramer@sap.com)
Installation of the SAP IQ Database with three Values, only SDA/ODBC Setup needed.

Note 3017355 – SAP IQ 16.1 SP04 PLx – correct SAPIQDB.cfg settings

Q – the easy SAP IQ database installer

smart data access (SDA) for SAP ILM

Report ADBC_TEST_CONNECTION

there is one little addition to the DBCO setup of the ILM Store:

DBCO Values for ILM

SYBASE_SERVER=hostname.fqdn SYBASE_PORT=3<nr>38 SYBASE_IQ_ENGINE=hostname_<SID>_<nr> 
SYBASE_DBNAME=SAPIQDB SYBASE_IQ_LOAD_SIDE_FILES=1 SYBASE_IQ_LOAD_DIR=/sapmnt/IQ 
SYBASE_CONTYPE=IQ SYBASE_IQ_LOAD=1 SYBASE_IQ_CESU-8=1 SIQ_BULK_OP=1 
SYBASE_IQ_LOCKWAIT=600 SYBASE_IQ_BUFFER_SIZE=500000

 

sybase@server:/usr/sap/<SID>/data/db>
iqdsn -y -w "hostname_<SID>_<nr>" -c "UID=DBA;PWD=Sybase4me;ServerName=hostname_<SID>_<nr>; 
LINKS=tcpip(host=hostname.fqdn;port=3<nr>38)"
Configuration "hostname_<SID>_<nr>" written to file /usr/sap/<SID>/home/.odbc.ini
sybase@server:/usr/sap/<SID>/data/db> dbping -d -c dsn=hostname_<SID>_<nr>
Connected to SQL Anywhere 16.1.50.1949 server
Ping database successful.
sybase@server:/usr/sap/<SID>/data/db>
dbisql -c "uid=SAP<SID>USR;pwd=Sybase4me;dsn=hostname_<SID>_<nr>" -nogui
(SAP<SID>USR>) 
sp_iqlmconfig;
sp_iqstatus;
sp_iqcheckoptions;
sp_displayroles;
sp_iqdbspaceobjectinfo USER0001;

 

---

SAP IQ Monitoring with DBACOCKPIT

Blog – How to keep Enterprise Flash Applications accessible in 2021
Blog – enable SAP IQ for DBACOCKPIT

Note 3094594 – Enable SAP IQ for DBACOCKPIT
Note 3017355 – SAP IQ 16.1 SP04 PLx – correct SAPIQDB.cfg settings

tx. DBACOCKPIT – define the SAP IQ Database (ILM)

tx. DBACOCKPIT – overview SAPILM05 IQ Database

tx. DBACOCKPIT – SAP IQ Database Options

tx. DBACOCKPIT – SAPILM05 – IQ Database Tables

 

---

Setup SAP Information Lifecycle Management

First of all, you have to activate the SAP ILM Business Function with tx. SFW5 or maintain with tx. SFW2

SAP Help – Prerequisites for Using the ILM Store

Please Note, you have to use a Web based Method to do changes in the Business Framework. you can use the WebGUI for that task.

Note 2900689 – SAPGUI for HTML. WebGUI logon not working on S/4 Hana 1909

Business function FIN_ACC_ILM is prerequisite for business function ILM_CDE_CONT_SAP_APPL

the activation of the Business Functions is mandatory to execute the ILM Basis Customizing.

activate the ILM Database Store (ILM_STOR)

For the ILM Store usage there are several SAP Roles available. For a detailed description and the use of the authorization objects, you can refer to the ILM Implementation Guide.
In a new System, these (and many more other Roles) are not active by default.
You have to enable them via tx. SU24 and PFCG first.

Roles for the usage of ILM

Once one of these ILM Roles is activated or pronounced, you have to check the individual Values as well. For this the following Example the Role SAP_BC_ILM_ADMIN_RW_V3 is taken.

Role maintenance with tx. PFCG

Depending on the Organizational Structure of the Company were the S/4 System is used, these Settings can have a massive Influence on the existing Processes and Authorizations.
Please Consider carefully, the activation of the needed Roles for the ILM usage.
Check the completeness of the necessary Role(s) carefully with the  SAP ILM Implementation Guide => Chapter 3.1

 

---

Configuration of the SAP ILM Framework

as already mentioned, we are focusing here only on the usage of the SAP IQ Database together with ILM (no Azure BLOB, Hadoop or NFS will be covered).

nevertheless, some of the existing entries in tx. FILE have to be validated as well, before we can continue with the SAP ILM Framework Settings.
There is a logical connection between Path and File Settings defined in tx. FILE

ARCHIVE_DATA_FILE and ARCHIVE_GOBAL_PATH 
ILM_STOR_FILE and ILM_STOR_FOLDER
ILM_STOR_REQ_SHARED_FILE and ILM_STOR_REQ_SHARED_FOLDER

tx. File – Logical File Path Definition

tx. File – Logical File Name Definition, Cross-Client

 

---

Note 2563024 – ILM Store – Collection of Notes
Note 2243603 – ILM Store: Utility for adding Connection IDs to Class Factory

create an Entry (per system) in the Table TILMSTOR_CF for the created SAP IQ Database with the
Report RILM_STOR_CF_DBCON

Table: TILMSTOR_CF
Interface: IF_ILM_STOR_DB_OPERATOR
for SAP_SYS_FILE: CL_ILM_STOR_DB_OPERATOR_FS
for SAP IQ: CL_ILM_STOR_DB_OPERATOR_IQ
for SAP HANA: CL_ILM_STOR_DB_OPERATOR_HDB

Report RILM_STOR_CF_DBCON for DBCON

tx. SM30 – Table TILMSTOR_CF

Please Note, the Customizing Settings for the tx. SARI and SARA are only available, when the activation of the SAP ILM Business Function was successfully.

customizing tx. SARI

customizing tx. SARA – 1-2

customizing tx. SARA – 2-2

Connection Test to the ILM store on IQ

optional: switch to the local SRS service in SM59

tx. SARA – Remote SRS Client is local

Note 2544517 – Archive administration: Direct navigation to Archive File Browser
Note 2607963 – How to find the relationship between table and archive object

 

---

Define the Origin Concept for SAP ILM

ILM Origin Concept

The Origin supports the ILM Store in determining the data origin and is a central element of the configuration of the ILM Store. Basically you define in this two Transaction from were the Data is coming from, and were it will be stored. In our Case of course the defined SAP IQ Database.

ILM Table distribution to different systems

Details of creation the Origin can be found in the ILM Implementation Guide. Additional hints can be found in the Blog  – ILM Store Configuration – from Neenu Susan George (SAP)

Note 3068508 – Simplified transaction (ILM_STOR_GEN_CUST) for ILM Store customizing

• ILM_STOR_ADM_CUST (ILM Store Administrative Customizing)
• ILM_STOR_OPR_CUST (ILM Store Operational Customizing)
• Report RILM_STOR_GENERATE_CUSTOMIZING (ILM_STOR_GEN_CUST)

tx. ILM_STOR_GEN_CUST

tx. ILM_STOR_ADM_CUST

tx. ILM_STOR_ADM_CUST

review the settings in Table TILM_STOR_CUS for your origins and make sure you executed at least once the Report RILM_STOR_TEST_SM69 to create the missing SM69 commands for ILM. Please Note Customer Command’s are starting with X,Y,Z*

Report RILM_STOR_TEST_SM69

tx SM30 – Table TILM_STOR_CUS

tx. SM30 – Table TILM_STOR_CUS_O

tx. SM30 – TILM_STOR_O_ROUT

options in Table TILM_STOR_O_ROUT

---

Setup and Publishing of the SAP ILM Store

Details of the Publishing of the ILM Store  can be found in the ILM Implementation Guide.
Additional hints can also be found in the Blog – ILM Store Configuration – from Neenu Susan George (SAP)

• tx. SICF (HTTP Service Hierarchy)
• tx. SM59 (RFC Destination)
• tx. ILMSTOREADM (access with WebGUI)

Note 3027603 – Error “401 Unauthorized” in ILMSTOREADM while adding an ILM Store
Note 3044246 – “500 Internal server error” in ILMSTOREADM while creating ILM Store

to publish the storage system using an ICF node, the SAP Backend must be correctly configured for the HTTP/S usage as described in the following Documents.

• SAP First Guidance – SAP BW on HANA – Edition 2021
• SAP First Guidance – complete functional scope (CFS) for SAP BW 7.50

tx. SMICM

Note 1936501 – ESI – Transaction SOAMANAGER does not work
Note 2033765 – HTTPURLLOC entries are ignored when you test a service in SICF
Note 2483210 – HTTP 500 error occurs when calling SAPGUI transactions – the call is stuck in the /sap/public/myssocntl service
Note 3261819 – Downloading attachments in Fiori apps is launching HTTP port instead of HTTPS port

tx. SM30 – Table HTTPURLLOC

Note 2704178 – The error HTTP 500 “Redirect is not possible” occurs in /sap/public/myssocntl or in /sap/public/bc/icf/logoff
Note 2846151 – WEBGUI: “No OK code 200 in response from server.” when uploading large files

In newer ABAP System adapt the SICF setting systemloginjs
Note 2900689 – SAPGUI for HTML. WebGUI logon not working on S/4 Hana 1909

tx. UCON_CHW

tx. SICF – 1-2

tx. SICF – 2-2

from Version SAP S/4 2021 and onwards, the SICF Services is already defined as ilmstore and srs and no additional customizing is necessary to create the service.

tx. SM59 – S4H_ILM_STORE

tx. SM59 – test the RFC connection

tx. ILMSTOREADM (Web Dynpro Application)

 

If the ILM User lacks on the RFC permission, then the creation of the ILM Store will fail with the following message:
Note 3115295 – In ILMSTOREADM error “597: Inconsistent Configuration” occurs

597: Inconsistent Configuration (CX_XADK_DAS_SERVER)

 

If this error arise, you can already run the mentioned check Reports RILM_STOR_TEST_AT and RILM_STOR_TEST_PF_REPEAT_SMALL (the message might be misleading)

598: Input or output error (triggered during access to file system, WebDAV or data stream)

 

Due to the complexity of the ILM Solution, it is hard to assign directly a solution to an error message, like with NLS. So the careful review of the settings is mandatory to guarantee a working System.

SQL message: [IQ Error SQL-141:42S02][SAP][ODBC Driver][SAP IQ]Table 'TILM_STOR' not found

597: Inconsistent configuration
CX_XADK_DAS_SERVER
DA Service Message: _DEFINE_ARCHIVE_STORES: 
Store S4H_400_ILM_IQ Does Not Correspond to ILM Conformance 2

598: Input or output error 
(triggered during access to file system, WebDAV, or data stream; see long text)
CX_XADK_DAS_SERVER
DA Service Message: _DEFINE_ARCHIVE_STORES: Store SAP_ILM_STORE Does Not Support the WebDAV Log

598: Input or output error CX_XADK_DAS_SERVER DA Service Message: 
_DEFINE_ARCHIVE_STORES: Error While Testing Store ...; Cause: 500 Internal server error
Component: BC-ILM-STO

Status Code: 598 Reason Phrase: IO Error                                                                                       
GET_WEBDAV_STORE_META_DATA: Error While Executing the OPTIONS Request to Store ZILMSTORE; 
Cause: 15 HTTPIO_ERROR_PLG_CREATE-Fehlermeldung beim Senden der Daten

Error Message: 598: Input or output error (triggered during access to file system, WebDAV, or data stream; see long text
CX_XADK_DAS_SERVER
DA Service Message: _DEFINE_ARCHIVE_STORES: Error While Testing Store OPENTEXT STORAGE; 
Cause: 500

500: Internal error in data archiving service or in lower-level server components
DA Service Message: Problem occurred while inserting data via facade

500 Error 99 while attempting to lock table ZILMSTORRDE87D70 for key 5CF3FCDCB65C1EDDADA581A2B9953FAB
500 An exception was raised
500 Problem occurred while inserting data via facade

Cause: 500 Internal server error
500 Logical file name ILM_STOR_ADK_ROOT_FOLDER does not exist
500 Exception while writing data to INSERT_SQL (open for write)
500 An exception was raised
500 Problem occurred while inserting data via facade

Parameter HANDLE wurde nicht bereitgestellt => tx. file
Parameter HANDLE was not provided

 

Some of the used ILM Transactions are only available as WebDynpro Application.
You can also access them via the WebGUI alternatively.

### used Services in the HTTP hierarchy - tx. SICF
/sap/bc/webdynpro/sap/irm_policies
/sap/bc/webdynpro/sap/ilm_audit_area
/sap/bc/webdynpro/sap/das_store_browser
/sap/bc/webdynpro/sap/ilm_store_admin
/sap/bc/webdynpro/sap/wdr_test_input

Note 2970651 – Transaction ILM_SB does not work (but calling ILM Store Browser in different ways is successful)

ILM – selected access via WebGUI

ILM Work Center

ILM Store Browser (tx. ILM_SB)

 

---

Testing the ILM Functionality with IQ

Blog – demystifying NLS/ILM error messages …

for the testing of the ILM functionality, there are various Report’s available, which more less show a successful result or errors which are not related to the Solution. Additional checks can also be found in the Blog from Neenu Susan George – ILM Store Configuration and – ILM Store Troubleshooting

Blog – How to analyze error „expiration_date409 null” during ILM Store phase (Report RSARCH_STORE_FILE) – from Gergely Juhasz (SAP)

Blog – How to solve “expiration_date403 Forbidden” error during ILM store phase – from Gergely Juhasz (SAP)

• tx. SLG1 => Object ILM_STOR
• RILM_STOR_TEST_HC_SERVER
• RILM_STOR_TEST_HC_CLIENT (ILM Store: Health Check for Client Configuration)
• RILM_STOR_TEST_AT (Validation of BC-ILM 3.0)
• RILM_STOR_TEST_PF_SINGLE (only for file based archiving)
• RILM_STOR_TEST_PF_REPEAT_SMALL
• RILM_STOR_TEST_PF_REPEAT_BIG
• RILM_STOR_TEST_CLEAR (needed to reset the Test Loop)
• RILM_STOR_WD_DEMO (test of the WebDAV Layer)
• RILM_STOR_TEST_PET (Performance Evaluation Tool)
• RSARCH_CHECK_CUSTOMIZING

---

tx. SLG1 => Object ILM_STOR

tx. SLG1 => Table created in IQ

the message below is not an error (bug in the report)

Report RILM_STOR_TEST_HC_SERVER

Report RILM_STOR_TEST_HC_CLIENT

Report RILM_STOR_TEST_AT

Report RILM_STOR_TEST_CLEAR

Report RILM_STOR_TEST_PF_SINGLE

you can use the URI from the Report RILM_STOR_TEST_PF_SINGLE to feed the Report RILM_STOR_WD_DEMO.
Make sure that the Table ZILMSTOR* can be accessed and is not locked

Report RILM_STOR_WD_DEMO

RILM_STOR_TEST_PET 1-2

RILM_STOR_TEST_PET 2-2

Report RSARCH_CHECK_CUSTOMIZING

 

---

SAP ILM – 1^st Step Done … 

Ok, that was a Hell of a Ride.

Nevertheless, this is another proved Example of SAP (Sybase) IQ Multi Purpose Database.
It is true: SAP (Sybase) IQ – the hidden treasure …

 

Next Steps

Connect the eBW to IQ

 

Blog – SAP S/4HANA Data Management – from Rami Kandimalla (SAP)
talks about Data Volume Management in SAP S/4HANA

Blog – SAP Data Archiving Principals & Legal Consolidations
talks about SAP Data Archiving SARI/SARA

Blog – ILM: Time definitions and processes in a single picture – from Revathi M P (SAP)
talks about different sections/processes during the life cycle of data

Blog – SAP Archiving Process and Configuration Steps
nice Overview how the SAP Archiving Process works and now with SAP IQ Database as Target

Blog – SAP Archiving Step by Step Guide for Beginners
nice Overview including how the Table Analysis (tx. TAANA) can be used

Blog – SAP Data Archiving Basic Guide for Beginner’s
similar Overview based on SAP Data Archiving Basic Training Material

Blog – Data Archive – Archive Material Documents – MM_MATBEL
nice “Tutorial” to understand the basic concepts of SAP Data Archiving

Blog – Data Decommissioning and Data Aging for SAP S/4HANA – from Klaus Roehler genannt Riemer (SAP)
when really everything comes to an end …

---

Roland Kramer, SAP Platform Architect for Intelligent Data & Analytics
@RolandKramer

 

“I have no special talent, I am only passionately curious.”