Technical Articles

Johannes Goerlich

February 11, 2022 3 minute read

Remediation of CVE-2022-22536 Request smuggling and request concatenation in SAP NetWeaver, SAP Content Server and SAP Web Dispatcher

Tuesday Mar 22, 2022: added info about emergency SP Stack Kernel 7.22.

Thursday Feb 24, 2022: added details to the section describing the impact of the workaround to AS Java systems.

Tuesday Feb 17, 2022: added section describing the impact of the workaround.

Wednesday Feb 16, 2022: updated section For JAVA systems behind SAP Web Dispatcher.

Monday Feb 14, 2022: updated section For ABAP or JAVA systems systems or SAP Content Server behind load balancer / reverse proxy from other vendors.

On 8th of February 2022, SAP Security Patch Day a vulnerability in the Internet communication manager (ICM) was disclosed.

SAP released the security note 3123396 and later on the FAQ note 3148968. The workarounds are described in note 3137885.

EDIT: On March 22, 2022 SAP released the emergency SP Stack Kernel 7.22 PL 1101.

After gathering and evaluating all current available information, I came to the following recommendation for remediating this vulnerability in the various affected scenarios:

For ABAP systems or SAP Content Server behind SAP Web Dispatcher:

1. SAP Web Dispatcher has to be patched and the parameter wdisp/additional_conn_close=TRUE has to be set in the SAP Web Dispatcher.

If a request passes through multiple SAP Web Dispatchers on its way, then this must be implemented in all SAP Web Dispatchers of this chain.

2. The SAP Kernel in all application servers and SAP Content Server has to be patched to the minimum required patch level.

3. After patching the SAP Kernel in all application servers, the parameter wdisp/additional_conn in the SAP Web Dispatcher is no longer necessary must be reverted.

For ABAP systems with integrated SAP Web Dispatcher:

1. The SAP Kernel of all application servers has to be updated. No workaround available!

For JAVA systems behind SAP Web Dispatcher:

1. When using the SAP Web Dispatcher version of SAP Note 3147927 the workaround as described in section “For ABAP systems or SAP Content Server behind SAP Web Dispatcher” can be used. With this, the same workaround applies to AS ABAP and AS Java (and DUAL stack).

No differentiation necessary any longer, which makes it easier for you if you are just about to start working on this incident after February 16th.

If you already implemented the workaround as mentioned below, go forward – patch the SAP Kernel of AS Java and remove the workaround afterwards. There is no need to switch to the same workaround procedure as used for ABAP!

1. SAP Web Dispatcher has to be patched and the request modification rules have to be configured in the SAP Web Dispatcher in the file which is defined by parameter icm/HTTP/mod_<x> as described in SAP note 3137885 and the parameter icm/HTTP/support_http2=FALSE has to be set.

~~Please note: Do not set wdisp/additional_conn_close=TRUE !~~

If a request passes through multiple SAP Web Dispatchers on its way, then this must be implemented in all SAP Web Dispatchers of this chain.

2. The SAP Kernel in all application servers has to be patched to the minimum required patch level.

3. After patching the SAP Kernel in all application servers, both the request modification rules and the parameter icm/HTTP/support_http2 in the SAP Web Dispatcher are no longer necessary and must be reverted.

For ABAP or JAVA systems systems or SAP Content Server behind load balancer / reverse proxy from other vendors:

Note: The workaround of this scenario requires SAP Kernel versions >= 7.53 PL 30 or 7.49 PL >= 410 in the ABAP or Java system.
The workaround cannot be implemented in configurations where profile parameter icm/HTTP/auth_<n> with sub-parameter AUTHFILE is configured.
For SAP Kernel versions 7.49 and 7.53, the following statements must not be present in any icm/HTTP/mod_<n> files: RegRedirectUrl, RegIRedirectUrl, RegGoneURL, RegIGoneURL.

1. The SAP Kernel in all application servers and SAP Content Server has to be patched to the minimum required patch level (in this case steps 2. and 3. are to be omitted) or

the request modification rules have to be configured in the ABAP/JAVA system and SAP Content Server in the file which is defined by parameter icm/HTTP/mod_<x> as described in SAP note 3137885.

2. The SAP Kernel in all application servers and SAP Content Server has to be patched to the minimum required patch level.

3. After patching the SAP Kernel in all application servers and SAP Content Server, the request modification rules are no longer necessary and must be reverted.

You may consider to apply the last one also to directly accessible systems, depending on your risk appetite.

For each scenario, point 1. should be performed as soon as possible.

Impact of the workaround to AS Java:

By default, http 1.1 keeps connections between client and server alive so that they can be reused for further requests.

In contrast to that, http 1.0 closes connections after each request. With this, each request needs a new 3-way handshake for TLS which adds some overhead.

The parameter wdisp/additional_conn_close=TRUE seems to change the behavior of http 1.1 to close connections after requests with a tampered content-length header. For AS Java, this may create a performance impact for POST requests or cause ephemeral TCP port exhaustion. So please make sure to remove the parameter after patching the SAP Web Dispatcher and the SAP Kernel of all backend systems.

28 Comments

You must be Logged on to comment or reply to a post.

Vijay Krishna Varma

February 14, 2022 at 10:36 pm

Johannes,

Thanks for the detailed steps and it really helped.

One question, what Web Dispatcher parameters do you recommend to implement if we have a WebDispatcher which routes to both ABAP & JAVA backend systems? I assume we need to implement both the below parameters, but wanted to confirm.

wdisp/additional_conn_close=TRUE

icm/HTTP/support_http2=FALSE

Regards,

Vijay

Johannes Goerlich

Blog Post Author

February 15, 2022 at 10:06 am

Hello Vijay,

If you operate Application Server ABAP and Application Server Java systems or dual-stack system behind one Sap Web Dispatcher, do not set SAP Web Dispatcher profile parameter "wdisp/additional_conn_close=TRUE". Otherwise you may end with issues described in SAP note 3147927.

Instead, configure the request modification rules and set the parameter icm/HTTP/support_http2=FALSE. Remove both after patching the SAP Web Dispatcher and all backends.

Best regards,

Johannes

Vijay Krishna Varma

February 15, 2022 at 5:33 pm

SAP released a new patch for WebDispatcher today and as per the OSS#3147927, this new version overcomes the performance issues with the ABAP parameter (wdisp/additional_conn_close=TRUE). Based on OSS notes I read, below is what I think works best assuming we have the latest WebDisp patch released today.

WebDisp (ABAP) - Patch WebDisp & implement parameter (wdisp/additional_conn_close=TRUE)

WebDisp (Java) - Patch WebDisp, implement parameter (icm/HTTP/support_http2=FALSE) and add request modification rules as per OSS#3137885

WebDisp (ABAP+JAVA) - Patch WebDisp, implement parameter (icm/HTTP/support_http2=FALSE) and add request modification rules as per OSS#3137885

I am also reading (OSS#3137885) that these temporary work-arounds with WebDisp parameters and request modification rules are no longer needed if you can update the WebDispatcher with the patch level released on 02/15/2022. A blurb from the note is as below.

"We are currently working on a solution that will make replace this special solution. It will be shipped in SAP note 3147927. You can monitor the note to determine when it is available. We expect this by February 18, 2022. At that time, we recommend to switch to the new SAP Web Dispatcher version of note 3147927 and remove this special workaround again."

Regards,

Vijay

Johannes Goerlich

Blog Post Author

February 16, 2022 at 4:22 pm

SAP published the update for the workaround for AS Java backend systems with respect to the new SAP Web Dispatcher Patches. I updated the blogpost accordingly.

Jens Gleichmann

February 16, 2022 at 12:10 pm

Hi Johannes,

thanks for writing about the current webdisp situation! I really appreciate to have all details on one page filtered / focused only on the main topic.

One aspect which is also missing in all SAP notes is the fact that also the HANA and XSA components are affected, because they are also running a webdisp. Here you have to use the template to adjust parameters. You may be able to add this procedure as well to work around this issue in this context as well.

Regards,

Jens

Johannes Goerlich

Blog Post Author

February 16, 2022 at 1:32 pm

Hello Jens Gleichmann ,

in the FAQ note 3148968 it is stated:

1.3 Are SAP HANA XS Classic or SAP HANA XS Advanced affected?

No, these are not affected because they are configured in a specific way that does not expose the vulnerability.

Do you have different information?

Regards,

Johannes

Jens Gleichmann

February 17, 2022 at 9:44 am

Hi Johannes,

the statement is just another way to tell you: hey in the default configuration your XSC/XSA systems are not affected, but if you have some customized settings we don't tell you if you are affected or not.

I would apreciate what exactly means "in a specific way" from SAP because in the end the default configuration differs not really from a normal webdisp configuration. You can run nearly the same config. There is already a rewriterules.txt which is a difference by default:

SetHeader default_port 443
if %{SERVER_PROTOCOL} strcmp http
begin
SetHeader default_port 80
end
SetHeader tmp_host %{HTTP_HOST}
RegRewriteHeader tmp_host ([^:]*)(:([0-9]*))? $3
SetHeader tmp_port %{SERVER_PORT}
if %{HTTP_HOST} regimatch .+
begin
SetHeader tmp_port %{HEADER:default_port}
end
if %{HEADER:tmp_host} regimatch "^[0-9]+$"
begin
SetHeader tmp_port %{HEADER:tmp_host}
end
SetHeaderIfEmpty X-Forwarded-Port %{HEADER:tmp_port}
RemoveHeader tmp_host
RemoveHeader tmp_port
SetHeaderIfEmpty X-Forwarded-Proto %{SERVER_PROTOCOL}
SetHeaderIfEmpty clientprotocol %{SERVER_PROTOCOL}
if %{SERVER_PORT} = 51000
RegIForbiddenUrl !^\/sap\/wdisp\/admin\/publicicp\/show_latest_update\.icp$ -

But this has nothing to do with the workaround:

if %{HEADER:Content-Length} !strcmp "" [AND]
if %{HEADER:Content-Length} !strcmp 0
begin
    SetHeader Connection "close"
    SetResponseHeader Connection "close"
end
if %{HEADER:Transfer-Encoding} !strcmp ""
begin
    SetHeader Connection "close"
    SetResponseHeader Connection "close"
end

As fas as I know it is a memory pipes (MPI) desynchronization vulnerability depending on thea header and content length of the HTTP request. The workaround parameter "wdisp/additional_conn_close=TRUE" ist not included by default in XSC/XSA configuration. It will take some research for me what exactly is the stated out "specific configuration".

May be you can add some more details about the vulnerability itself from onapsis or request smuggling itself. The idea beyond request smuggling "request desynchonization" is quite old (2005).

Regards,

Jens

Jens Gleichmann

February 17, 2022 at 12:58 pm

Update: It seems that the included webdispatcher in XSC/XSA has another memory pipelining configuration and is not directly affected by CVE-2022-22536, but it is affected by CVE-2021-38162, but I will double check this.

Basis User

February 23, 2022 at 12:57 pm

Hi,

We are using below methods , so can you please guide me clearly , what I have to do ? what to check and all?

For ABAP or JAVA systems systems or SAP Content Server behind load balancer / reverse proxy from other vendors:

Johannes Goerlich

Blog Post Author

February 24, 2022 at 1:21 pm

Hello Basis User, please respect the rules of engagement.

Ashu Adhana

February 23, 2022 at 2:00 pm

Hi Johannes,

Thanks for sharing such a precise analysis, it is very helpful and to the point. Usually SAP Infrastructure is present inside the corporate Network, which obviously eliminates most of the attack vectors,coming directly from the Internet. Could you advise if this vulnerability is still relevant even if the dispatchers and SAP AS are inside such Intranet/ Corporate Network ? Since we're already planning the Kernel upgrades later this year, this info would help us to speed up or delay the same

Regards,

Adrea

Johannes Goerlich

Blog Post Author

February 23, 2022 at 3:00 pm

Hi Adrea,

sorry I cannot advise to delay patching of a CVSS 10.0 vulnerability at all. That's against the codex of honour of my profession 🙂

Best regards

Joe

Ashu Adhana

March 24, 2022 at 11:13 am

Hi Johannes,

Thank you Noted. I had one more query If I see the official note 3123396 , the minimum required PL let's say for "sap kernel 7.53 64-bit unicode" is mentioned as 915 , if we check the latest PL released it is PL 1000 , it was released 2 days back. prior to that PL 900 was released on 29.10.2021 for kernel 753. If we get to PL 900 is our system still vulnerable ?

Regards,

Adrea

Johannes Goerlich

Blog Post Author

March 24, 2022 at 1:01 pm

Hi Adrea,

please have a look at https://support.sap.com/deployment-strategies-kernel-abap.pdf. This should perfectly answer your question.

BR,

Johannes

Basis User

February 23, 2022 at 3:33 pm

Hi Johannes

We are using below methods , so can you please guide me clearly , what I have to do ? what to check and all?

For ABAP or JAVA systems systems or SAP Content Server behind load balancer / reverse proxy from other vendors:

Jerry Janda

February 25, 2022 at 2:47 am

Basis User

It isn't clear why you posted this comment twice on this post, but as Johannes mentioned above, you should familiarize yourself with the rules of engagement. Please don't ask such broad questions to a blog post author. If you have a specific comment or question about the post, please be clearer in your comments.

If you are trying to ask a question beyond the scope of what's covered here, please do so via https://answers.sap.com/questions/ask.html. I recommend that you familiarize yourself with https://community.sap.com/resources/questions-and-answers, as it provides tips for preparing questions that draw responses from our members. Feel free to take our Q&A tutorial at https://developers.sap.com/tutorials/community-qa.html as well, as that will also help you when preparing questions for the community.

Kind regards,

--Jerry

Bilal Daud

February 23, 2022 at 5:41 pm

Hi,

Thank you for giving a precise detail about CVE-2022-22536.

could you please let me know that what is the level of security risk we can have if we do not consider to use a parameter wdisp/additional_conn_close=TRUE in our SAP Web Dispatcher that we recently upgraded to version 7.85.0 Kernel release 785 PL74. This is pointing to our two back-end systems CRM and ECC and they are on Kernel Version Level 745 PL301 and 753 PL910 respectively. I am asking this because if we go down the path of putting the above parameter in then we have to go through intensive testing and also a downtime for our business critical production systems.

Regards,

Bilal

Johannes Goerlich

Blog Post Author

February 24, 2022 at 1:48 pm

Hi Bilal Daud,

Please check SAP note 3123396 if your systems run a vulnerable SAP kernel version.

Please read the prerequisites and circumstances under which the vulnerability can be exploited. With this you could analyze your landscape, setup, network architecture, etc. if there are mitigating measures in place. Then you can define what the residual risk looks like and decide to either implement the workaround asap or reduce or take the residual risk until you finally patch the kernel. Patching is inevitable to fix the vulnerability.

I appreciate that you perform intensive testing before applying configuration changes in your production systems. Since such a vulnerability may require expedited patching you could also think about to reduce the test cases to the components or business processes which are very likely to be affected and take the residual risk of such a change.

BR,

Johannes

Bilal Daud

February 24, 2022 at 2:56 pm

Thank you for your reply and explanation Johannes!

I also read about this parameter in your blog under information about the work around that...

The parameter wdisp/additional_conn_close=TRUE changes the behavior of http 1.1 to close connections after each request. As said before, this adds overhead and creates the performance impact. So please make sure to remove the parameter after patching the SAP Web Dispatcher and the SAP Kernel of all backend systems.

Since we have several connectivity happening through our SAP Web Dispatchers so that slowness will be deal breaker if we implement this parameter.

So I am guessing only expedited patching will be the only option?

BR,

Bilal

Johannes Goerlich

Blog Post Author

February 24, 2022 at 10:24 pm

Hi Bilal,

as of SAP a performance impact may show up for POST requests towards AS Java systems in high volume scenarios only. I edited the section in my blogpost to reflect the available info from SAP more precisely. A general performance impact shouldn’t be expected nor should it deter you from implementing the workaround.

BR,

Johannes

Bilal Daud

February 25, 2022 at 5:47 pm

Hello, Thank you for your reply.

So after talking to our network team, they mentioned that our CRM and ECC Web Dispatchers have additional protection as they are white listed to only the SAP C4C cloud for the connectivity with on premise CRM and ECC. So the average hacker sitting on the Internet will get dropped by our firewall if attempting to access these dispatchers. Given that, do we still require to update ur SAP WD with the parameter wdisp/additional_conn_close=TRUE can the vulnerability still be be exploited even if do not use that parameter and do not upgrade out back-end system Kernels?

Regards,

Bilal

Johannes Goerlich

Blog Post Author

February 25, 2022 at 9:57 pm

For this HTTP request smuggling attack to be exploited, the attacker hast to send requests over the very same reverse proxy or load balancer as the victim.

BR,

Johannes

Charles Muthu

February 24, 2022 at 4:38 pm

Hi Johannes,

We are using SAP content server Version="753";serverPatch="832", database MAXDB and OS -

Linux. I have gone through this note. I couldn't find any action items about SAP content server. It is

pointing only Kernel upgrade for Application systems, web dispatcher parameters change and

version upgrade. Please suggest is there any action required for SAP content server 753.

Regards,

Charles

Jonathan Hoare

February 24, 2022 at 5:10 pm

Hi Charles,

As per the note 3123396 you should go to SAP Kernel version="753";serverPatch="915" and if you access the SAP Content server via a web dispatcher also patch that.

SAP strongly recommends patching all components (SAP Kernel and SAP Web Dispatcher) even in scenarios in which they are not vulnerable

Regards

Jon

Kedalene Chong

August 29, 2022 at 12:48 am

Hi Johannes

This is not relevant to SAP Business One right?

Kedalene Chong

Isaias Freitas

August 29, 2022 at 12:55 pm

Hello Kedalene,

This is only relevant for SAP systems based on NetWeaver or ABAP Platform technology.

As far as I am aware, SAP Business One does not run on top of NetWeaver / ABAP.

In addition, also as far as I am aware :-), SAP Business One cannot be accessed using a browser, so it cannot have an SAP Web Dispatcher in front of it.

Thus, this is not relevant for SAP Business One ;-).

Regards,

Isaías

Kedalene Chong

August 29, 2022 at 1:57 pm

Thanks Isaias for the detail explanation!

Kedalene

Isaias Freitas

August 29, 2022 at 5:00 pm

You are welcome!

Technical Articles

Remediation of CVE-2022-22536 Request smuggling and request concatenation in SAP NetWeaver, SAP Content Server and SAP Web Dispatcher

Tuesday Mar 22, 2022: added info about emergency SP Stack Kernel 7.22.

Thursday Feb 24, 2022: added details to the section describing the impact of the workaround to AS Java systems.

Tuesday Feb 17, 2022: added section describing the impact of the workaround.

Wednesday Feb 16, 2022: updated section For JAVA systems behind SAP Web Dispatcher.

Monday Feb 14, 2022: updated section For ABAP or JAVA systems systems or SAP Content Server behind load balancer / reverse proxy from other vendors.

For ABAP systems or SAP Content Server behind SAP Web Dispatcher:

For ABAP systems with integrated SAP Web Dispatcher:

For JAVA systems behind SAP Web Dispatcher:

For ABAP or JAVA systems systems or SAP Content Server behind load balancer / reverse proxy from other vendors:

Impact of the workaround to AS Java:

Assigned Tags

For ABAP or JAVA systems systems or SAP Content Server behind load balancer / reverse proxy from other vendors:

For ABAP or JAVA systems systems or SAP Content Server behind load balancer / reverse proxy from other vendors: