Understanding performance statistics for SAP HANA Cloud Platfom, OData provisioning

[CarlosRoggan]

In SAP HANA Cloud Platform (HCP) it is possible to expose OData services which are based on business data residing in an SAP backend. This is done with:
SAP HANA Cloud Platform, OData provisionig

For more information, you may refer to this introduction  in the SAP community or to the official documentation for SAP HANA Cloud Platform, OData provisioning . See here for a nice comparison.

In the present blog, we’re going to have a closer look into performance statistics.

 

Overview

But first, let’s have a quick look at the scenario and the involved components:



In SAP HANA Cloud Platform (HCP), the OData provisioning is enabled and exposes an OData service.
The OData service is used to provide data residing in an SAP backend like SAP Business Suite.
As required for OData exposure, the backend implements a SAP Gateway service.
In order to connect the cloud with the backend, a SAP HANA Cloud Connector (HCC) is used .
The OData service can be consumed by a Fiori application in the cloud, in our example we're using a browser or REST client.

 

Performance statistics

When executing an HTTP request to an OData service, you might be interested in analyzing the performance of the service call.
A quick and easy step in this direction is to have a look at the SAP performance statistics which can be obtained for each service call.
Simply append the following URL parameter to the end of your service-request-URL:

 

 sap-statistics=true

 

Example URL:

https://gwaas-<yourUser>trial.hanatrial.ondemand.com/odata/IWBEP/GWSAMPLE_BASIC;v=1/ProductSet?sap-s...

Other example:

https://gwaas-<yourUser>trial.hanatrial.ondemand.com/odata/IWBEP/GWSAMPLE_BASIC;v=1/SalesOrderSet?$e...

See the Appendix section below for alternatives.

Note:
Depending on your URL, it might be required to add a ? or a & like shown in the examples above

The performance statistics are then returned in the response of the service call.
They are returned as header.
There are 2 headers in the response and each header has several metrics passed as values.
The names of the 2 headers in the response are:

 

sap-statistics-hciodp
sap-statistics-scc

 

The first one is meant for the HCP OData provisioning, the second one for the SAP HANA Cloud Connector.

For example, the performance statistics, provided in the response headers, can look like this



Please refer to the Appendix section below for instructions on how to access the performance statistics.

Before we go through each of the metrics, I’d like to introduce a little diagram which attempts to visualize the flow in a simplified manner.
It shows the involved components and the communication between them.
The horizontal lines in the bottom indicate how to correlate each metric to the corresponding component, like a timeline.



 

 

In the following sections we’re going to describe each of the returned metrics.
The example values of the descriptions are based on the screenshot above.

 

SAP HCP OData provisioning statistics

These statistics are contained in the value of the header sap-statistics-hciodp in the response of your odata service call.
The following metrics are relevant for us:

 


hciodp
total
nwoh
fw
gwbe
gwapp


 

In the following sections,  we'll have a closer look.

 

hciodp

This metric provides the time that is spent in the cloud for exposing the backend data as OData service. With other words, the time that is consumed by the OData provisioning service in HCP.
The following screenshot shows the cloud cockpit to illustrate what I mean with “OData provisioning”:



OData provisioning is a piece of software running in HCP, which exposes the backend data as OData service.
It analyzes the incoming OData request, retrieves the requested data from the backend and exposes the received data as OData response.
The time consumed inside OData provisioning is provided in the hciodp metric.
This means that the metric doesn’t contain the time during which the OData provisioning is “waiting” for the data to be returned from the backend.
After the OData provisioning has sent a request to the backend, some time is spent on the network. This duration is captured in the following metric.

 

nwoh

Network overhead. This figure identifies the time which is lost on the network.
To be more precise:
The OData provisioning talks to the backend and all the time that is lost in between, is specified in the nwoh metrics.
There might still remain an open question: usually, there’s a SAP HANA Cloud Connector (HCC) between the cloud and the backend.
The answer is that the HCC in mid of the communication counts towards the communication, means it is part of the nwoh figure.

 

gwbe

This parameter describes the time it takes for processing the request in the SAP Gateway framework in the backend.
The OData provisioning talks to the SAP Gateway component in the backend which processes the request and asks the data provider class for the data.
This sentence also makes clear that the backend-duration is split into 2 components:
the SAP Gateway framework, which handles the call for data in a generic way
and the data provider class which is implemented by the Gateway service developer and which has the knowledge about which data has to be delivered and how.

 

gwapp

Also referred to as app
The OData provisioning in HCP exposes backend-data as OData service. When talking about “backend-data”, we aren’t referring to arbitrary raw data. OData provisioning requires the SAP Gateway component in the backend. At the end of the day, the ABAP developer implements a so-called data provider class, which has the task to provide the raw data as expected by the Gateway framework. The SAP Gateway framework is contained in the IWBEP component in the SAP backend.
The implementation of the data provider class is responsible for fetching and providing the data that is afterwards handed over to the framework(s). In that regard, the gwapp metric might be very interesting when analyzing the performance statistics.
For more details on the performance statistics in SAP Gateway, please follow this link

 

fw

This figure is intended to allow to separate the time spent in the framework(s) from the time consumed by developer implementation and other components.
The fw number is calculated by adding the hciodp (the framework in the cloud) to the gwbe (the gateway backend, meaning the SAP Gateway framework without data provider implementation) and to the nwoh the time lost while the 2 framework-components talk to each other

In our example:

hciodp + gwbe + nwoh = fw
8         + 11      + 48      = 67

 

total

Also referred to as gwtotal.
The “official” description, as stated in the docu, is the following:
“Total processing time for executing an OData request between OData provisioning and back-end.”

The number is calculated as follows:

hciodp + nwoh + gwbe + gwapp = gwtotal

This means, the number simply sums all existing values and ignores whether they are spent in the framework, by developer or lost in the network.

In our example

hciodp + nwoh + gwbe + gwapp = gwtotal
8          + 48     + 11      + 4         = 71

 

gwbewait, gwnongw, gwappsum

These statistics are provided by the SAP Gateway backend, like gwbe and gwapp, and they are explained here.

The diagram below shows the example values in the timeline.



 

 

SAP HANA Cloud Connector statistics

These statistics are contained in the value of the header sap-statistics-scc in the response of your OData service call.

 

openRemoteConn

As the name indicates, this parameter informs about the time used for opening a connection to the configured backend.

 

ext

The job which has to be done by the cloud connector is to forward the incoming request to the connected backend.
The cloud connector itself doesn’t do much and thus doesn’t waste much time. The time spent inside the cloud connector is not provided in the header value.
But the time that is spent after the request leaves the cloud connector, which is much bigger, is specified in the metrics ext.
The duration which is measured is all the time spent in the backend and in the communication between HCC and backend.
Note:
the ext number contains durations that are already known to us: the time that is consumed in the backend is measured and provided within the sap-statistics-hciodp

 

total

The total figure sums all the cloud connector statistics.
However, if we look at our example, we find that the calculation is not correct:

In our example:

openRemoteConn + ext  ≠ total
2                            + 53   ≠ 56

The reason is that a number is missing, which counts to the total duration. This number (“Internal”) can be viewed in the HCC itself.
It is the duration which the cloud connector itself needs internally to process the request. Usually a very small number.
Please refer to the Appendix for an explanation about how to find the performance view in the HCC.

So now let’s find out how to find the missing number to get our calculation correct.

Open the HCC and go to the performance monitor view.
Find the matching request in the list, select it and view the details:



Apart from other details, you can see the performance numbers in a horizontal chart.
The numbers should be identical with those that are given in the response header of the OData request.
We can compare the Duration, Open Connection and External (Back-end) (from the screenshot above) to the metrics in the response of our OData requests, the total and openRemoteConn and ext.
In addition to this, we can see the number that is missing in our header: the Internal (SCC)

So finally, at the end of the day, our calculation is correct.

In our example:

internal + openRemoteConn + ext = total
1          +                2              + 53 = 56

The below diagram tries to visualize our findings. It tries to visualize how the different metrics fit together and how the nwoh correlates to the other metrics.



Note:
The example values are based on a configuration using Basic Authentication.
In case of more sophisticated authentication settings, the figures in HCC would look a bit different.

 

Examples

So far, all has been pure theory – abbreviations, numbers, descriptions... all boring.
It’s time to get our fingers dirty and try out a couple of examples.
Frankly speaking – it will still be boring... 😉

OK, let’s have a look at a few examples:

 

Example 1: normal QUERY

The following is a normal call to retrieve the collection of sales orders

https://gwaas-<yourUser>trial.hanatrial.ondemand.com/odata/IWBEP/GWSAMPLE_BASIC;v=1/SalesOrderSet?sa...

Let’s take a note:
The framework in the cloud takes around 40 ms and the service implementation in the backend needs 5 ms to provide all sales orders.

 

Example 2: time consuming QUERY

The following request contains 3 nested expands, which is very time consuming and should never be used in that way:

https://gwaas-<yourUser>trial.hanatrial.ondemand.com/odata/IWBEP/GWSAMPLE_BASIC;v=1/SalesOrderSet?$e...


As expected, this call takes much more time (1198 ms).
The service implementation in the backend needs 980 ms, which is much more than the 5 ms of the previous example.
However, interestingly, we can observe that the time spent in the OData provisioning has not changed, it is again around 40 ms

Side-remark
If for a minute we assume that we have a performance problem... a service call which takes around 1200 ms...
What would we do?
We would check the statistics and see that all involved components are acceptable (41 ms, 100 ms), even the network doesn’t take too long (71 ms).
And we would easily identify that it is the service which consumes most of the time and might be not properly implemented (980 ms) in the backend and the performance issue could be solved there.
...But of course... in reality, the service implementer would answer that it is up to us, that we shouldn’t fire such call with 3 nested expands on the full query, without any filter or select statement...
End of side-remark

 

Example 3: QUERY without backend

The following call is handled by the OData provisioning in the cloud, it doesn't "leave" the cloud:

https://gwaas-<yourUser>trial.hanatrial.ondemand.com/CATALOGSERVICE;v=2/ServiceCollection?sap-statis...

The service call returns the list of registered services, which are persisted and maintained in the cloud, so no call to the backend is required.
Accordingly, the screenshot below shows that the Cloud Connector time consumption is 0.
All figures that are sent by the backend, like gwbe, gwapp, are empty.
The OData provisioning consumes only few milliseconds to read the list of registered services (I have to confess that in my example trial account I have only very small amount of registered services)



 

Example 4: QUERY with backend but without service implementation

This example can only be followed if there’s a so-called "standalone annotation model" available in the backend.
Here you can find a tutorial about how to create an annotation model.

An example URL could look like this:

https://gwaas-<yourUser>trial.hanatrial.ondemand.com/CATALOGSERVICE;v=2/Annotations(TechnicalName='ZCR_ANNOTATIONS_TEST1_ANNO_MDL',Version='1')/$value?sap-statistics=true

Such URL requires a call to the backend for retrieving the annotation model, which is implemented and registered in SAP Gateway in the backend.
However, there’s no service implementation done by the developer. More concrete: there’s no data provider implementation, only annotation provider which is not time-consuming. Therefore, no gwapp value is returned for this call.
However, the SAP Gateway framework delivers the information about the annotation model, that’s why the value for gwbe (6 ms) is available.
And also the involved Cloud Connector takes a little time, since the call goes to the backend.



 

Example 5: Working with metadata cache

The previous examples were recorded with enabled metadata cache.
Now let’s see how it looks like when the cache is disabled.
BTW, if you aren’t familiar with it, you may refer to the Appendix section.

So first let’s disable the metadata cache.
This means, that for every OData service call, the OData provisioning has to first execute a call to the backend to fetch the metadata document.
Only after that call, the actual user request is executed. These additionla calls are done internally and user won’t notice.
Usually, the metadata of a service don’t chance once the service is developed and ready. But in case the metadata do change (in the backend), then the admin has to clear the cache (in the cloud), such that the metadata can be fetched in its new version from the backend.
Furthermore, the OData provisioning helps to deal with such issues by performing a so-called "metadata handshake". This means that is checks if the cached metadata are equal to the backend-metadata and informs in case that they aren’t equal.

So after the cache is disabled, we execute the request to the sales orders again (like in example 1) and check the statistics.

https://gwaas-trial.hanatrial.ondemand.com/odata/IWBEP/GWSAMPLE_BASIC;v=1/SalesOrderSet?sap-statisti...

As a result, we can see that the total duration is much higher (506 ms compared to 123 ms in example 1).
However, the duration of the data provider (5 ms) has not changed at all, which is expected.


It is worth the time to have a closer look to understand the reason of the difference.
The truth is in the cloud connector
It shows that 3 requests have been done at the same time:


We sum up the durations of the 3 requests:
83 + 225 + 91 = 399

The result is exactly the value that we have in the sap-statistics-scc in the screenshot above.
The same calculation can be done with the other values.
In brief: the numbers that we – as users of the OData service – get from the statistics headers is the sum of the 3 calls that OData provisioning executes towards the backend.
Yes, WE fire only ONE call to the OData service, but the framework internally fires 3 calls to the backend.
Why 3 calls?
One is for fetching the metadata document, another one is for the texts and finally the one that fetches the requested data.

In example 1 we had nwoh=66, now we have 253 , which is nearly the 66 multiplied with 3

So this last example has clearly shown that using the metadata cache improves the performance.
Therefore, as last step of this blog, let’s re-activate the cache...

 

Appendix

1) Alternatives for requesting the performance statistics

• URL parameter:
  The following parameter has to be added at the end of the URL
  sap-statistics=true
  Like a query string, it has to be separated from the service URI with a ?
  If there are more query options, it has to be appended with a &


• URL parameter
  The following parameter can be used as well:
  ?sap-statistics-hciodp=true


• Header
  instead of appending a parameter to the URL, you can alternatively sent a header in the request
  This is only possible if you use a REST client
  You have to enter a header with name sap-statistics
  And value: true


• Or
  a header with name sap-statistics-hciodp
  And value: true

 

2) How to get performance statistics

If you want to request and read the performance statistics, you have at least 2 possible ways to achieve it

 

Alternative 1 : using Chrome browser

Open your Chrome browser and invoke a URL of a service exposed by OData provisioning, with the sap-statistics parameter appended

For example:

https://gwaas-<yourUser>trial.hanatrial.ondemand.com/odata/IWBEP/GWSAMPLE_BASIC;v=1/SalesOrderSet?sa...

The statistics are returned in a header value.
Headers cannot be viewed in a browser window, but if your browser contains developer tools (like Chrome, Firefox, Internet Explorer etc), you can view that information there.
Open the developer tools via menu or simply press F12
The toolset is usually opened in the bottom of the browser.
Click on the “Network” tab

Then fire the request again.
The request is listed in the developer tools
Click on the request to view the details.
There you are.



 

Alternative 2 : using REST client

Install a REST client of your choice, e.g. “Postman”.
In the request section, enter your cloud user credentials, enter the sap-statistics request header and fire the request.
Then view the header section of the response



 

3) Performance monitor in HCC

When analyzing performance of OData requests, it might be worth having a closer look at the Administration Frontend of the SAP HANA Cloud Connector.
The HCC frontend can be reached at port 8443.
For a locally installed HCC, the URL is:

https://localhost:8443

(The default credentials for login are: Administrator/manage, but they must be changed during initial configuration)
In order to access the performance information, proceed as described below:
On the left pane, select the desired account from the drop-down box.
Note that otherwise you would be monitoring the wrong requests!
Then expand the entry “Monitoring” and click on “Performance” to get the "overview" screen



Now select “Most Recent Requests” from the left navigation pane.
On the right pane, select your desired backend system from the drop-down box.
The below list gets populated with the most recent requests.
Click into the first line.
The “Request Details” section gets visible below.

The detailed documentation of the performance monitor in the HCC can be found here:

 

Links

Official documentation
https://help.hana.ondemand.com/hci_odata_provisioning/frameset.htm?f8ac4c9fd77d4db4b15cf5522597054b....

SAP Community blog clarifying statistics in SAP Gateway
https://blogs.sap.com/2016/01/28/understanding-sap-performance-statistics-for-sap-netweaver-gateway-...

Cloud Connector docu about performance monitoring
https://help.hana.ondemand.com/help/frameset.htm?6d9c937dd35344bca3eb61ebf34a5c1d.html#loio6d9c937dd...

SAP Community blog about SAP Gateway deployment options in SAP backend
https://blogs.sap.com/2013/05/27/sap-netweaver-gateway-deployment-options-in-a-nutshell/