In this blog post I will quickly share an error I was facing when retrieving data from SAP Ariba, when using SAP Cloud Integration, the reason for the error and the solution to prevent it.
TLDR; Check the number of HTTP headers sent in the request to the SAP Ariba APIs. If sending 55+ HTTP headers an HTTP 502 error will be raised. There is no real need to send more than 10 HTTP headers, check the code on the client side and only send the necessary HTTP headers.
From Mozilla Developer Network - https://developer.mozilla.org/en-US/docs/Glossary/502
An HTTP error code meaning "Bad Gateway". A server can act as a gateway or proxy (go-between) between a client (like your Web browser) and another, upstream server. When you request to access a URL, the gateway server can relay your request to the upstream server. "502" means that the upstream server has returned an invalid response.
Ok, this is not telling us much on what the problem might be when communicating with the SAP Ariba APIs.
I was building an integration flow, in SAP Cloud Integration, for a customer to move data from SAP Ariba to SAP BW/4HANA 2.0. The integration ran successfully most of the time but every now and then it was raising an HTTP 502 error. To give you an idea of the integration flow... the integration flow receives a JSON payload as input, extracts data from SAP Ariba, transforms it and sends it to SAP BW/4HANA. If a pageToken is included in the SAP Ariba response, it will proceed to call the SAP Ariba API again to extract more data and so on until there are no page tokens in the response.
When was it working?
When was it not working?
So, I knew that the request URL was correct, as I test it in an external tool and I was retrieving the data. The pagination mechanism was working but fails when running on a schedule.I needed to figure out what was different in the first and subsequent requests to the SAP Ariba API. Debug mode in the integration flow to the rescue 🦸♀️.
When checking the run, in debug mode, I noticed that the number of exchange headers in the second request were significantly more than in the first request. Given how Apache Camel (what runs SAP Cloud Integration under the hood) works, the exchange headers are included in the request sent to an HTTP system. Within the exchange headers you can find response headers from the first call to the SAP Ariba API, response headers from the call to SAP BW/4HANA, scheduling specific headers, and more. I initially thought that what was causing the 502 error was one of the headers sent in the request. To test this, I end up copying all headers available, before the HTTP request, in the exchange and try sending a request to the SAP Ariba API.
When I first tried sending the request with all the exchange headers (73 in total) to the SAP Ariba API, the connection was being terminated by the API - ECONNRESET. A new error but that gave me the idea that the problem could be that there were too many headers in the request.
Then, I decided to start removing unnecessary headers in the request. Headers that I know are not relevant to the SAP Ariba API, e.g. all the CamelHttp*, X-Ariba-*, X-RateLimit-*, X-Kong-*, scheduling headers. In the end, a request from Postman to the SAP Ariba API will only contain around 10 HTTP headers, included apiKey and Authorization. After removing / commenting quite a few headers I avoided the ECONNRESET error and started getting the 502 error.
Now that I was able to replicate the error, I kept removing headers to see if I get a successful response. A minute later.... a successful response 🙌 🎉.
The request sent from SAP Cloud Integration contains way too many HTTP headers, hence why the 502 error is raised. To prevent it, we can remove unnecessary headers using a Groovy script like the one below.
I hope this blog post gives you a good understanding of why SAP Cloud Integration includes so many headers in the HTTP request, why the SAP Ariba API is raising an HTTP 502 error and how we can prevent it by removing them before the HTTP request.
TLDR; Check the number of HTTP headers sent in the request to the SAP Ariba APIs. If sending 55+ HTTP headers an HTTP 502 error will be raised. There is no real need to send more than 10 HTTP headers, check the code on the client side and only send the necessary HTTP headers.
What is an HTTP 502 error?
From Mozilla Developer Network - https://developer.mozilla.org/en-US/docs/Glossary/502
An HTTP error code meaning "Bad Gateway". A server can act as a gateway or proxy (go-between) between a client (like your Web browser) and another, upstream server. When you request to access a URL, the gateway server can relay your request to the upstream server. "502" means that the upstream server has returned an invalid response.
Ok, this is not telling us much on what the problem might be when communicating with the SAP Ariba APIs.
The problem
I was building an integration flow, in SAP Cloud Integration, for a customer to move data from SAP Ariba to SAP BW/4HANA 2.0. The integration ran successfully most of the time but every now and then it was raising an HTTP 502 error. To give you an idea of the integration flow... the integration flow receives a JSON payload as input, extracts data from SAP Ariba, transforms it and sends it to SAP BW/4HANA. If a pageToken is included in the SAP Ariba response, it will proceed to call the SAP Ariba API again to extract more data and so on until there are no page tokens in the response.
When was it working?
- When posting the request directly from an external tool, e.g. Postman. Pagination was working fine in this scenario.
- When running on a schedule 🕒 and no pagination taking place.
When was it not working?
- When running on a schedule and needing to paginate the SAP Ariba response.
The HTTP 502 error mentioned here was received when consuming the Operational Reporting API and the External Approval API.
Error in SAP Cloud Integration
So, I knew that the request URL was correct, as I test it in an external tool and I was retrieving the data. The pagination mechanism was working but fails when running on a schedule.I needed to figure out what was different in the first and subsequent requests to the SAP Ariba API. Debug mode in the integration flow to the rescue 🦸♀️.
When checking the run, in debug mode, I noticed that the number of exchange headers in the second request were significantly more than in the first request. Given how Apache Camel (what runs SAP Cloud Integration under the hood) works, the exchange headers are included in the request sent to an HTTP system. Within the exchange headers you can find response headers from the first call to the SAP Ariba API, response headers from the call to SAP BW/4HANA, scheduling specific headers, and more. I initially thought that what was causing the 502 error was one of the headers sent in the request. To test this, I end up copying all headers available, before the HTTP request, in the exchange and try sending a request to the SAP Ariba API.
The integration flow development only sets the apiKey header, any "variables" required for the functionality of the integration flow are set as properties, not headers.
When I first tried sending the request with all the exchange headers (73 in total) to the SAP Ariba API, the connection was being terminated by the API - ECONNRESET. A new error but that gave me the idea that the problem could be that there were too many headers in the request.
ECONNRESET from SAP Ariba API
Then, I decided to start removing unnecessary headers in the request. Headers that I know are not relevant to the SAP Ariba API, e.g. all the CamelHttp*, X-Ariba-*, X-RateLimit-*, X-Kong-*, scheduling headers. In the end, a request from Postman to the SAP Ariba API will only contain around 10 HTTP headers, included apiKey and Authorization. After removing / commenting quite a few headers I avoided the ECONNRESET error and started getting the 502 error.
Bingo, I managed to replicate the error in Postman
Now that I was able to replicate the error, I kept removing headers to see if I get a successful response. A minute later.... a successful response 🙌 🎉.
So, what's the problem and how can this be prevented?
The request sent from SAP Cloud Integration contains way too many HTTP headers, hence why the 502 error is raised. To prevent it, we can remove unnecessary headers using a Groovy script like the one below.
map = message.getHeaders()
new_headers = [:]
remove_list = [
'Access-Control-Allow-Credentials',
'CamelServletContextPath','Content-Length',
'Content-Type','Date','functionName',
'Transfer-Encoding','tenant_alias','Vary'
]
for (h in map) {
if (!h.key.startsWith('X-Kong-')
&& !h.key.startsWith('X-RateLimit-')
&& !h.key.startsWith('X-Ariba-')
&& !h.key.startsWith('Access-Control-')
&& !h.key.startsWith('CamelHttp')
&& !h.key.startsWith('scriptFile')
&& !h.key.contains('fire')
&& !(h.key in remove_list)) {
new_headers << h
}
}
message.setHeaders(new_headers)I hope this blog post gives you a good understanding of why SAP Cloud Integration includes so many headers in the HTTP request, why the SAP Ariba API is raising an HTTP 502 error and how we can prevent it by removing them before the HTTP request.
- SAP Managed Tags:
1 Comment
