Remediation of uncalled active ICF services

Web applications and web services to rule them all

Updates:
2022-11-07: Added hint for new secure-by-default logs.

The time of SAP GUI as a proprietary interface for every user interaction seems to be over soon. SAP is moving towards serving more and more applications as web applications or web services – be it SAP Fiori, be it the Webgui utilising Screen Personas or OData or REST APIs.

While this change counts in for sure to better user experience or using open standards for interfaces (e.g. towards cloud services), it also effects the SAP Basis Administrator:
A SAP basis administrator nowadays need to have a deep understanding of web technologies (additionally to understanding the SAP application server, the database as well as to the operating system like linux).

I will not go into detail about the journey from the integration of Internet Communication Manager (ICM) as web server into SAP NW AS ABAP/Java or what to consider in general when operating a web server within your ERP system – maybe even exposed to the internet.
In this blogpost I would like to catch up a topic which is more related to the content served by the Internet Communication Framework (ICF) through the ICM.

“Please activate one or the other ICF service – it is urgent!”

In the early days – before “Secure by Default” was a claim – SAP NetWeaver AS ABAP based systems were shipped with certain ICF services in status “activated”. In addition to that, SAP Basis Administrators often where asked on short notice to quickly activate one or the other service in transaction SICF until some urgent needed functionality works. So it happened that there were more ICF service activated then ever needed, especially in the production system.

While in the meantime SAP released many security notes to fix vulnerabilities in the SAP standard ICF services, in many systems a number of 500 up to more than 3000 ICF services are still enabled although they are not needed nor called.

Bring some light into the darkness

Please note: As of SAP S/HANA-Version 2022 / ABAP Platform 2022 the so called secure-by-default logs for ICF calls are collected. These logs can be evaluated in transaction SBDLOG. Those can be used to activate all ICF services that have ever been logged or deactivate services that were not logged by choosing ‘Menu’ -> ‘ICF Integration’ in transaction SBDLOG. With this, the manual steps described below can be omitted.

At first lets export a list of the active ICF services of all of your systems (Dev, Int, Prod) by the following steps:

1. Go to transaction SE38.
2. Execute report RS_ICF_SERV_ADMIN_TASKS.
3. Choose “Export of Active Services to CSV File” and execute to save the file, e.g. as <SID>_active_services.csv
   
4. Again. Repeat these steps for every system in your landscape.

To give yourself an overview about the active ICF services in your systems take a look at the exported lists. By screening every list you will figure out that the active services in the various systems of a system landscape will be different from system to system.

Please be aware for some services there is a good reason to be active in development or integration system only! There are services used for development purposes or there might be services for features which are not yet released to production. Besides that, there are web applications used only for one time tasks or initial setup. Or active services differ simply due to configuration drift.

The main purpose of this lists is to use it a) as a template for the deactivation of uncalled ICF services in the next steps and b) as a backup to quickly mass re-activate all former active services in emergency case.

What about service usage?

It is strictly advisable to activate ICF services only if they are necessary for business needs in order to reduce the attack surface. Often there is no documentation which services are involved in what business functionality, some of them are even involved in more than one since they provide a framework  functionality (e.g. for Webdynpro ABAP or BSP). So it might be hard for you to determine which service is truly needed in the first.

As a quick win you could analyse how often each service were called in each of your systems. To do so there are at least three methods:

1. extract the relevant info from the HTTP access log of the ICM (parameter icm/HTTP/logging_<xx>).
2. enable and evaluate the ICF trace using report RS_ICF_SERV_ADMIN_TASKS as mentioned in SAP note 2430473.
3. evaluate the Web Statistics.

Method 1 might be time consuming to gather all log files and grep and normalize the relevant parts.

Method 2 could have a performance impact as of SAP note 1611713.

Since Web Statistics are collected by default I recommend to use method 3. However, I like to advise to prolong the retention period of Web Statistics for this purpose in advance by following these steps:

1. Go to transaction ST03.
2. Go to Collector and Performance DB -> Performance Database -> Workload Collector Database -> Reorganization -> Control.
3. Adjust retention period for the WEB statistics, e.g. TOTAL Monthly Aggregates Retention Period for `VC` and `VD`.
4. Again: Repeat these steps for every system in your landscape.

After a reasonable time frame in which the Web Statistics for relevant phases (month end closing or alike) have been collected, the statistics can be exported by following these steps:

1. Go to transaction ST03.
2. Expand Workload -> Total -> Month -> and perform for each available month the following steps.
3. Expand Analysis Views -> Web Statistics -> WEB Server Statistics.
4. Switch to tab “URL”.Note: The Web Statistics may contain entries for services which are not even present in the system but are requested for example by a scanning server.
   Hint: You may have noticed the column “Protocol” indicates if a service is called via HTTP or HTTPS.
5. Export the list of URLs as spreadsheet (“Excel (in Office 2007 XLSX Format)“) using the “Export” button. (Note: available as of SAP_BASIS 7.02).
6. Again: Repeat these steps for every system in your landscape.

Put the puzzle together

Now it is time to compare the list of active services with the list of called services to prepare the cleanup.

You can also think of comparing the lists across your system landscape. As already stated, for your development and integration system you should consider that there are indeed services active which are necessary for development purposes only or services which are in acceptance test and not yet set productive. These should be evaluated for the relevant systems. Talk to the developer team!

Besides that there may also be web applications activated which are used only for initial setup tasks, e.g. SAML2 configuration or troubleshooting e.g. sec_diag_tool. You should consider to activate these only at configuration time or as long as needed.

There may also be services which are actively used in the production system but are not activated in one or another system of the system landscape. It’s time to eliminate this configuration drift.

Ok, lets start the preparation:

1. Copy the export of active services CSV file as a template.
2. Remove all lines which have an entry in column REF_PATH. These are aliases which point to a service node. Aliases can not be deactivated, but they will stop working if the original service is disabled.
3. Remove all lines for called services and their parent paths from the list so that you end up with a list of ICF services which can be deactivated in the relevant system.
4. Make sure no parent node stays in the list to be deactivated while it has sub nodes which should remain active. This would lead to so called “stale nodes” and non-working services. For example if /sap/bc/echo should stay active remove /sap, /sap/bc and /sap/bc/echo from the list.
5. Save this list of your to-be-deactivated services as CSV file.
6. Again: Repeat these steps for every system in your landscape.

Time to cleanup

Now you can start deactivating unused services in each system with your processed templates by following these steps:

1. Open the list of active ICF services, e.g. <SID>_acvite_services.csv and doublecheck if it holds only to-be-deactivated ICF services.
2. Go to transaction SE38.
3. Execute report RS_ICF_SERV_MASS_PROCESSING.
4. Select “Deactivate ICF Services for All Virtual Hosts Listed in CSV File” and execute.
5. Confirm the popup and upload the list to deactivate all services in the list.
6. Again: Repeat these steps for every system in your landscape.

Did you reveal a configuration drift? Activate the missing services in the relevant systems either manually in transaction SICF or by preparing a CSV file for mass activation.

Maybe you have overlooked some parent paths during creation of the list and they have now been deactivated accidentally. This is not that fatal. You simply can perform a fresh “Export of Active Services to CSV File” and use this to “Activate ICF Services for All Virtual Hosts Listed in CSV File”. The activation logic of the report will then take care of activating also all necessary parent nodes.

Keep in mind if something goes totally wrong you have also the initial export of active services as a backup which can be used to mass activate everything again as it was before.

Just as a side note:
The UCON HTTP whitelist scenario or former HTTP_WHITELIST does not serve for the same purpose as the UCON RFC Basic scenario. Meaning it does not affect the general reachability of certain paths through ICM! But this is a topic for another blogpost.

Whats next?

After finishing the deactivation in all systems, I recommend to “Export of Active Services to CSV File” once again. With this list you should start documenting your web services and their business need.

I started to collect information about SAP standard ICF services in https://github.com/J-Goerlich/Standard_ICF_services which may give you a hint if a service is needed in your scenario.