Skip to Content

/wp-content/uploads/2015/02/skiptoken1_678923.png

With the SMP OData SDK, we can implement offline scenario with simple and easy API – if you’re not yet familiar with the SDK, there’re blogs and how-to-guides explaining practical details.

During the offline store creation, you might encounter the runtime error, simply because the OData collection you’re trying to fetch is too big – you can copy & paste the OData endpoint URL in the web browser and see if it shows 500 timeout error. The issue in this case is a problem on the server-side.  The HTTP requests for data were timing out because the backend producer was taking too long to query the data and encode it as OData.

In this case we should consider using server side paging on OData producer. There are two types of paging in OData – server side and client side.

Client side paging is driven by having a $top value and then incrementing the $skip values for each request (e.g. $top=100&$skip=0 for first request, then $top=100&$skip=100 for second request etc.) This type of paging will not help with populating the database in the offline store because the defining requests are fixed. (Of course, you can use $top or $skip to make requests for small amounts of data to the local store after the store has been populated.)

/wp-content/uploads/2015/02/skiptoken_678264.png

In server side paging, even if you request the entire big OData collection, the server will only return a portion of the results, but provide a “next link” info, which can be followed to get the next portion and so on. When the MobiLink is populating the local store it knows how to follow these next links. Server side paging via next links is OData Version 2.0 feature. The $skiptoken query is seen in OData URL if it is implemented.

This H2G explains how to build OData Query service which supports $skiptoken with SAP Gateway.

Prerequisites

You should have done the following steps:


#1 – OData CRUD Crash Course – Query

Implementing $skiptoken logic

Essentially the $skiptoken implementation is fairly simple. Once you have a table data, here’s the basic logic flow:

/wp-content/uploads/2015/02/split_675742.png

  1. Obtain the $skiptoken value if it exists in the URL.
  2. Measure the size of the table.
  3. If the size is beyond the predefined chunk size (ex. 1000 entities for each $skiptoken page), cut the whole table into pieces (= pages) with the chunk size.
  4. Set both the last index of the entity for the next $skiptoken call and one piece of the table data as the return table (= a page) of the get_entityset.


That’s all. Here’s the code implementing the steps above:


Note: Appendix has the complete $skiptoken implementation code you can run by copy & paste in get_entityset method section.

01 * a) Obtain the $skiptoken value if it exists in the URL
02   lv_skiptoken = io_tech_request_context->get_skiptoken( ).
03
04 * b) Measure the size of the table
05   DESCRIBE TABLE lt_entityset LINES lv_table_size.
06   IF lv_table_size IS INITIAL.
07     CLEAR es_response_context-skiptoken.
08     EXIT.
09   ENDIF.
10
11 * c) If the size is beyond the predefined chunk size,
12 *    cut the whole table into pieces with the chunk size
13   IF lv_table_size < lv_paging_max_chunk.
14     lv_index_beg = 1.
15     lv_index_end = lv_table_size + 1.
16   ELSE.
17
18     IF lv_skiptoken IS NOT INITIAL.
19       lv_token_len = STRLEN( lv_skiptoken ) - 1.
20       IF lv_token_len > 0.
21         lv_token_beg = lv_skiptoken(1).
22         lv_token_end = lv_skiptoken+lv_token_len(1).
23         IF lv_token_beg = '''' AND lv_token_end = ''''.
24           lv_token_len = lv_token_len - 1.
25           lv_skiptoken = lv_skiptoken+1(lv_token_len).
26         ENDIF.
27       ENDIF.
28 * Determine indices for BEG and END
29       lv_token_len = STRLEN( lv_skiptoken ).
30       IF lv_token_len > 0   AND
31          lv_token_len < 10  AND
32          lv_skiptoken CO '0123456789'.
33         lv_index_beg = lv_skiptoken.
34       ELSE.
35         lv_index_beg = 1.
36       ENDIF.
37     ELSE.
38       lv_index_beg = 1.
39     ENDIF.
40     lv_index_end = lv_index_beg + lv_paging_max_chunk.
41   ENDIF.
42 * d) Set both the last index of the entity for the next $skiptoken call
43 *    and one piece of the table data as the return table of the get_entityset
44   IF lv_index_end <= lv_table_size.
45     es_response_context-skiptoken = lv_index_end.
46   ENDIF.
47   CONDENSE es_response_context-skiptoken.
48
49   LOOP AT lt_entityset INTO ls_entity.
50     IF sy-tabix <  lv_index_beg. CONTINUE. ENDIF.
51     IF sy-tabix >= lv_index_end. EXIT. ENDIF.
52     INSERT ls_entity INTO TABLE et_entityset.
53   ENDLOOP.

In the line #02, “io_tech_request_context->get_skiptoken( )” method obtains the $skiptoken value if it exists in the URL.

#11 – #41 are the core trick algorithm to split one table into pieces, basically you can copy & paste for any single table – they are all about to calculate where is “begin” and “end” row of a table with a specific chunk size.

#44 – #46 set the last entity index number as a return parameter “es_response_context-skiptoken” so that the subsequent OData Query call can fetch the continuous collection data.

#49 – #53 extracts the entities from begin to end and set them as a returning OData entityset.


You should wonder how those variables are declared, so here’s the one:

   Naming convention:
   l - local scope
   t - table
   s - structure
   v - variable

01   DATA: lt_entityset TYPE TABLE OF stravelag,
02 * variables for $skiptoken
03         ls_entity           TYPE stravelag,
04         lv_paging_max_chunk TYPE i VALUE 50,
05         lv_index_beg        TYPE i,
06         lv_index_end        TYPE i,
07         lv_skiptoken        TYPE string,
08         lv_token_len        TYPE i,
09         lv_token_beg        TYPE c,
10         lv_token_end        TYPE c,
11         lv_table_size       TYPE i.

As you see this sample code has chunk size “50”, which indicates each page should have 50 entities with this $skiptoken implementation.

Now you can activate it and test it.


Testing the $skiptoken

Try running the service from either the browser or REST Client. Have a look at the <link> data at the bottom of the OData payload – this is the $skiptoken we have implemented. You can try copy this generated $skiptoken URL as the OData URL and see how the next $skiptoken data looks like – for this example it continues to show the next chunk starting from the 51st entity.

/wp-content/uploads/2015/02/token_675795.png

Your $skiptoken implementation runs successfully!


Note: Once you implement it, in general, you should experiment a bit with the page sizes until you find a size that balances the backend performance and the response size of offline store. (For example, overall offline store initial load performance with a chunk size 1000 in one page might be slower than 5000, and so on.)



And – have you read this yet? – SMP3 OData SDK – Performance Tuning with Offline Store

List of H2Gs

Appendix – Query operation with $skiptoken

method TRAVELAGENCYSET_GET_ENTITYSET.

  DATA: lt_entityset TYPE TABLE OF stravelag,
* variables for $skiptoken
        ls_entity           TYPE stravelag,
        lv_paging_max_chunk TYPE i VALUE 50,
        lv_index_beg        TYPE i,
        lv_index_end        TYPE i,
        lv_skiptoken        TYPE string,
        lv_token_len        TYPE i,
        lv_token_beg        TYPE c,
        lv_token_end        TYPE c,
        lv_table_size       TYPE i.

  SELECT * FROM stravelag INTO TABLE lt_entityset.

* a) Obtain the $skiptoken value if it exists in the URL
  lv_skiptoken = io_tech_request_context->get_skiptoken( ).

* b) Measure the size of the table
  DESCRIBE TABLE lt_entityset LINES lv_table_size.
  IF lv_table_size IS INITIAL.
    CLEAR es_response_context-skiptoken.
    EXIT.
  ENDIF.

* c) If the size is beyond the predefined chunk size,
*    cut the whole table into pieces with the chunk size
  IF lv_table_size < lv_paging_max_chunk.
    lv_index_beg = 1.
    lv_index_end = lv_table_size + 1.
  ELSE.

    IF lv_skiptoken IS NOT INITIAL.
      lv_token_len = STRLEN( lv_skiptoken ) - 1.
      IF lv_token_len > 0.
        lv_token_beg = lv_skiptoken(1).
        lv_token_end = lv_skiptoken+lv_token_len(1).
        IF lv_token_beg = '''' AND lv_token_end = ''''.
          lv_token_len = lv_token_len - 1.
          lv_skiptoken = lv_skiptoken+1(lv_token_len).
        ENDIF.
      ENDIF.
* Determine indices for BEG and END
      lv_token_len = STRLEN( lv_skiptoken ).
      IF lv_token_len > 0   AND
         lv_token_len < 10  AND
         lv_skiptoken CO '0123456789'.
        lv_index_beg = lv_skiptoken.
      ELSE.
        lv_index_beg = 1.
      ENDIF.
    ELSE.
      lv_index_beg = 1.
    ENDIF.
    lv_index_end = lv_index_beg + lv_paging_max_chunk.
  ENDIF.
* d) Set both the last index of the entity for the next $skiptoken call
*    and one piece of the table data as the return table of the get_entityset
  IF lv_index_end <= lv_table_size.
    es_response_context-skiptoken = lv_index_end.
  ENDIF.
  CONDENSE es_response_context-skiptoken.

  LOOP AT lt_entityset INTO ls_entity.
    IF sy-tabix <  lv_index_beg. CONTINUE. ENDIF.
    IF sy-tabix >= lv_index_end. EXIT. ENDIF.
    INSERT ls_entity INTO TABLE et_entityset.
  ENDLOOP.

endmethod.
To report this post you need to login first.

Be the first to leave a comment

You must be Logged on to comment or reply to a post.

Leave a Reply