In this how-to guide, you will learn how to implement Delta Token capabilities with SAP NetWeaver Gateway to enable offline functionality. This how-to guide assumes that you have already worked your way through the following how-to guides:
- “How to Develop a Gateway Service using Code based Implementation”
- "How to Develop Query Options for an OData Service Using Code-Based Implementation"
These guides describe the basic implementation steps of the service that we are now going to enhance. The sample service uses demo data from the Enterprise Procurement Model, which is part of every SAP NetWeaver ABAP server as of 7.02.
After completing these steps, you will be able to:
• Enhance the GET_ENTITYSET and GET_ENTITYSET_DELTA methods of an entity set to support the delta query functionality.
Business Example
You want to build an application that shows a list of products. The application shall be able to retrieve only those products that have been newly created, changed or deleted after having retrieved a complete list of products as an initial load.
Short Introduction to Delta Token Functionality
OData exposes collections of records as EntitySets. These EntitySets support various options like sorting, filtering or paging to minimize the amount of data that is sent accross the wire.
Paging is a very powerful option to reduce the size of the payload and improve performance. To use paging on client side, the OData request needs to specify the $skip and $top tokens. For server-driven paging you have to use the $skiptoken.
Some applications, however, prefer to use client side caching rather than paging. In such cases, SAP NetWeaver Gateway provides Delta Tokens that help to reduce resource consumption on the server and the wire when the same request is sent several times from the same client. The OData Delta Token enables clients to use caching when retrieving entity sets. Consequently, the client gets only the changed records after an initial retrieval of all records. Therefore, the initial request will return all records and a Delta Token. The subsequent request (with exactly the same query options) sends the Delta Token from the first request. The response will then contain only the changed records. If no records have been changed since the first request then the response will not contain any records at all.
You can implement Delta Token in several ways and these options can be grouped into two main approaches:
- The first approach, which is based on Syclo Exchange framework, calculates deltas at modification time: The ABAP system tracks relevant changes when they occur. At request time, the deltas are already prepared and thus available. On the one hand this approach requires more development. On the other hand it’s more scalable and has an optimized overall performance.
- The second approach is based on delta determination at request time where the system compares old and new state to find out which records have been changed/deleted. The implementation effort is rather small but it does not optimize the performance of the backend. That means, the more records you have in the full collection, the longer the response time of the request.
With both approaches, the payload of the response is reduced but as described only the first approach is also able to optimize the performance in the backend.
In this guide we will look at the basic implementation, using the Delta Request Log Component. The Delta Request Log Component is based on the second approach. Therefore the implementation effort is rather small but the performance of the backend is not optimized.
Prerequisites
You have implemented an OData service as described in the two how-to guides mentioned above.
If you are using NetWeaver 7.02, then you need at least software component IW_BEP Support Package 7 and software component IW_FND Support Package 7.
If you are on NetWeaver 7.40, then you need at least software component SAP_GWFND Support Package 4.
You have to implement following SAP notes: 1926946, 1932247 and 1970229.
Implementation Overview
In this how-to guide, you will create a new Entity Set Products_DQ that shall support delta queries in the existing sample service that you have created earlier. You will redefine the methods GET_ENTITYSET and GET_ENTITYSET_DELTA to support the delta query functionality by calling method /IWBEP/CL_QUERY_RESULT_LOG-> CREATE_UPDATE_LOG_ENTRY_HASH.
Task 1: Create an additional Entity Set Products_DQ
We will enhance the service you created earlier to support delta queries for product data. For this we create a new entity set called Products_DQ where we implement the delta query specific coding while leaving the original Products Entity Set unchanged.
Task 2: Enhance method ZCL_GW_PRODUCT_DPC_EXT->PRODUCTS_DQ_GET_ENTITYSET to support the delta query functionality
In this task you will enhance the existing GET_ENTITYSET method of the newly created entity set Products_DQ. Like the existing GET_ENTITYSET method of the Products entity set it should retrieve the application data containing product entries.
First you need to create a local data provider facade which will be needed later by the Delta Token functionality.
In the next step you will need to call method /IWBEP/CL_QUERY_RESULT_LOG-> CREATE_UPDATE_LOG_ENTRY_HASH for the delta query functionality by passing the local table containing the product entries and the local data provider facade. The method is the central method for the Delta Token functionality. It calculates a hash value for each product and stores the hash values in some application tables, so the entries can be later checked for changes. The method will also return a delta token which you need to export.
Step 1: Get the data provider facade (you need to pass the facade later to the Delta Token method)
* get the data provider facade
TRY.
lo_dp_facade = /iwbep/if_mgw_conv_srv_runtime~get_dp_facade( ).
CATCH /iwbep/cx_mgw_tech_exception.
RAISE EXCEPTION TYPE /iwbep/cx_mgw_tech_exception.
ENDTRY.
Step 2: Call method /IWBEP/CL_QUERY_RESULT_LOG-> CREATE_UPDATE_LOG_ENTRY_HASH for the delta query functionality by passing local table et_entityset containing the product entries
* call the delta token functionality
TRY.
CALL METHOD /iwbep/cl_query_result_log=>create_update_log_entry_hash
EXPORTING
io_tech_request_context = io_tech_request_context
io_dp_facade = lo_dp_facade
ir_service_document_name = mr_service_document_name
ir_service_version = mr_service_version
it_entityset = et_entityset
CHANGING
ev_delta_token = lv_delta_token.
CATCH /iwbep/cx_qrl_locked.
RAISE EXCEPTION TYPE /iwbep/cx_qrl_locked.
CATCH /iwbep/cx_qrl_delta_unavailabl.
RAISE EXCEPTION TYPE /iwbep/cx_qrl_delta_unavailabl.
ENDTRY.
Step 3: Export the delta token which was calculated by the method in step (2)
es_response_context-deltatoken = lv_delta_token.
Task 2: Re-define method ZCL_GW_PRODUCT_DPC_EXT-> /IWBEP/IF_MGW_APPL_SRV_RUNTIME~GET_ENTITYSET_DELTA to support the delta query functionality
In this task you will re-define method GET_ENTITYSET_DELTA. First, you need to retrieve the application data containing product entries, just like in GET_ENTITYSET method.
Then you need to create a local data provider facade which will be needed later by the Delta Token functionality.
In the next step you will need to call method /IWBEP/CL_QUERY_RESULT_LOG-> CREATE_UPDATE_LOG_ENTRY_HASH for the delta query functionality by passing the local table containing the product entries and the local data provider facade.
It calculates a hash value for each product. Those hash values are then compared with the hash values of the prior call (related to the same delta token ID) to determine which values have changed in the meantime. The method will return a new delta token which you need to export. The method will also return any entries that were changed in the meantime (please store these in local table lt_product) and any entries that ware deleted (please store these in local table lt_deleted_product).
Step 1: Re-define method /IWBEP/IF_MGW_APPL_SRV_RUNTIME~GET_ENTITYSET_DELTA
Step 2: Retrieve all relevant entries for this entity set (like in GET_ENTITYSET above)
Step 3: Get the data provider facade (you need to pass the facade later to the Delta Token method)
* get the data provider facade
TRY.
lo_dp_facade = /iwbep/if_mgw_conv_srv_runtime~get_dp_facade( ).
CATCH /iwbep/cx_mgw_tech_exception.
RAISE EXCEPTION TYPE /iwbep/cx_mgw_tech_exception.
ENDTRY.
Step 4: Call method /IWBEP/CL_QUERY_RESULT_LOG-> CREATE_UPDATE_LOG_ENTRY_HASH
Call method /IWBEP/CL_QUERY_RESULT_LOG-> CREATE_UPDATE_LOG_ENTRY_HASH for the delta query functionality by passing local table et_entityset containing the product entries – just like in task 1, step 2 but this time take over the changed objects (and store them in local table lt_product) as well as the deleted objects (and store them in local table lt_deleted_product):
* call the delta token functionality
TRY.
CALL METHOD /iwbep/cl_query_result_log=>create_update_log_entry_hash
EXPORTING
io_tech_request_context = io_tech_request_context
io_dp_facade = lo_dp_facade
ir_service_document_name = mr_service_document_name
ir_service_version = mr_service_version
it_entityset = lt_product
IMPORTING
et_deleted_entityset = lt_deleted_product
et_entityset = lt_product
CHANGING
ev_delta_token = lv_delta_token.
CATCH /iwbep/cx_qrl_locked.
RAISE EXCEPTION TYPE /iwbep/cx_qrl_locked.
CATCH /iwbep/cx_qrl_delta_unavailabl.
RAISE EXCEPTION TYPE /iwbep/cx_qrl_delta_unavailabl.
ENDTRY.
Step 5: Export the delta token which was calculated by the method in step (4)
es_response_context-deltatoken = lv_delta_token.
Step 6: Export the deleted entity set
Export the deleted entity set - add deleted records (from table lt_deleted_product) to the deleted entries list er_deleted_entityset
* export the deleted entity set
copy_data_to_ref( EXPORTING
is_data = lt_deleted_product
CHANGING
cr_data = er_deleted_entityset ).
Step 7: Export the changed entity set - add changed records from lt_product to er_entityset
* export the changed entity set
copy_data_to_ref( EXPORTING
is_data = lt_product
CHANGING
cr_data = er_entityset ).
Implementation
ZCL_GW_PRODUCT_DPC_EXT->PRODUCTS_DQ_GET_ENTITYSET
The coding of the method:
DATA:
lt_return TYPE TABLE OF bapiret2,
lv_delta_token TYPE string,
lo_dp_facade TYPE REF TO /iwbep/if_mgw_dp_facade.
* retrieve all relevant entries for this entityset
CALL FUNCTION 'BAPI_EPM_PRODUCT_GET_LIST'
TABLES
headerdata = et_entityset
return = lt_return.
* get the data provider facade
TRY.
lo_dp_facade = /iwbep/if_mgw_conv_srv_runtime~get_dp_facade( ).
CATCH /iwbep/cx_mgw_tech_exception.
RAISE EXCEPTION TYPE /iwbep/cx_mgw_tech_exception.
ENDTRY.
* call the delta token functionality
TRY.
CALL METHOD /iwbep/cl_query_result_log=>create_update_log_entry_hash
EXPORTING
io_tech_request_context = io_tech_request_context
io_dp_facade = lo_dp_facade
ir_service_document_name = mr_service_document_name
ir_service_version = mr_service_version
it_entityset = et_entityset
CHANGING
ev_delta_token = lv_delta_token.
CATCH /iwbep/cx_qrl_locked.
RAISE EXCEPTION TYPE /iwbep/cx_qrl_locked.
CATCH /iwbep/cx_qrl_delta_unavailabl.
RAISE EXCEPTION TYPE /iwbep/cx_qrl_delta_unavailabl.
ENDTRY.
* export the delta token
es_response_context-deltatoken = lv_delta_token.
ZCL_GW_PRODUCT_DPC_EXT-> /IWBEP/IF_MGW_APPL_SRV_RUNTIME~GET_ENTITYSET_DELTA
The coding of the method:
DATA:
lt_return TYPE TABLE OF bapiret2,
lv_entity_set_name TYPE string,
lt_product TYPE zcl_gw_product_mpc=>tt_product,
ls_product TYPE zcl_gw_product_mpc=>ts_product,
lt_deleted_product TYPE zcl_gw_product_mpc=>tt_product,
ls_deleted_product TYPE zcl_gw_product_mpc=>ts_product,
lv_delta_token TYPE string,
lo_dp_facade TYPE REF TO /iwbep/if_mgw_dp_facade.
FIELD-SYMBOLS:
<lt_deleted_product> TYPE zcl_gw_product_mpc=>tt_product,
<lt_product> TYPE zcl_gw_product_mpc=>tt_product.lv_entity_set_name
= io_tech_request_context->get_entity_set_name( ).
IF lv_entity_set_name = 'Products_DQ'.
* retrieve all relevant entries for this entityset
CALL FUNCTION 'BAPI_EPM_PRODUCT_GET_LIST'
TABLES
headerdata = lt_product
return = lt_return.
* get the data provider facade
TRY.
lo_dp_facade = /iwbep/if_mgw_conv_srv_runtime~get_dp_facade( ).
CATCH /iwbep/cx_mgw_tech_exception.
RAISE EXCEPTION TYPE /iwbep/cx_mgw_tech_exception.
ENDTRY.
* call the delta token functionality
TRY.
CALL METHOD /iwbep/cl_query_result_log=>create_update_log_entry_hash
EXPORTING
io_tech_request_context = io_tech_request_context
io_dp_facade = lo_dp_facade
ir_service_document_name = mr_service_document_name
ir_service_version = mr_service_version
it_entityset = lt_product
IMPORTING
et_deleted_entityset = lt_deleted_product
et_entityset = lt_product
CHANGING
ev_delta_token = lv_delta_token.
CATCH /iwbep/cx_qrl_locked.
RAISE EXCEPTION TYPE /iwbep/cx_qrl_locked.
CATCH /iwbep/cx_qrl_delta_unavailabl.
RAISE EXCEPTION TYPE /iwbep/cx_qrl_delta_unavailabl.
ENDTRY.
* export the delta token
es_response_context-deltatoken = lv_delta_token.
* export the deleted entity set
copy_data_to_ref( EXPORTING
is_data = lt_deleted_product
CHANGING
cr_data = er_deleted_entityset ).
* export the changed entity set
copy_data_to_ref( EXPORTING
is_data = lt_product
CHANGING
cr_data = er_entityset ).
ENDIF.
Test the service
- Start the Gateway Client by calling transaction /IWFND/GW_CLIENT
- Enter the following URI to test your implementation: /sap/opu/odata/sap/ZGW_PRODUCT_SRV/Products_DQ
- You should get a response containing all products. Almost at the end of the XML
document, you will get your delta token (of course in your response you will
get a different ID) <linkrel="delta" href="Products_DQ?!deltatoken= '005056A2025C1EE2BFE687AFDC2FAAF4_20130807073741'" />
- Now enter the URI from above again, but this time add your delta token (copy and paste it from your response). You URI should look something like this (but will have a different ID):/sap/opu/odata/sap/ZGW_PRODUCT_SRV/Products_DQ?!deltatoken='005056A2025C1EE2BFE687AFDC2FAAF4_20130807073741'
- After you click on execute, you will notice that the response does not contain any products at all. This is because the Delta Token functionality returns only changed records and no products were changed in the meantime.
- You could now change a product and execute exactly the same URI as in step (4). The response should contain exactly one product.
Technical Restrictions
Key Fields of Entity Type
There are strict restrictions regarding the key fields of Entity Types that are used with the basic delta query functionality:
- The maximum legth (output length) of a key field must not exceed 200 characters
- If several key fields were defined for one Entity Type then the combined legth (output length) of those key fields must not exceed 200 characters
- The key field type must have a fixed legth; therefore types of variable legth (e.g. type String) are not permitted
JSON Format
Currently the JSON format is not supported in combination with the delta query functionality.
Expand System Query Option ($expand)
Currently the Expand System Query Option ($expand) is not supported in combination with the delta query functionality.
Paging
Both the server-driven paging and the client-driven paging are not supported in combination with the delta query functionality. As a result, the following query options are not supported in combination with the delta query functionality:
- Skip Token System Query Option ($skiptoken)
- Top System Query Option ($top)
- Skip System Query Option ($skip)
Further Notes
Clean-Up Report
The Delta Token functionality based on Delta Request Log Component uses two database tables to store hash values: /IWBEP/D_QRL_HDR and /IWBEP/D_QRL_ITM. Once this offline functionality is used productively the size of these tables will increase with each request. We offer areport that can be run to clean up outdated entries: /WBEP/R_CLEAN_UP_QRL. We suggest that you schedule a job for this report so that it is executed periodically, for instance on a daily basis.
