Skip to Content

At TechEd 2017, I attended a presentation on DX113, presented by Matt Borges . In particular, it was clear to me that there are quite SAP Mobile offline capabilities and limitations that users are not aware of and I’d like to highlight them here.

 

This blogs focuses on my experience implementing offline mobile , offline SDK evolution , best practices and improvements needed.

 

Design consideration plays very important role while designing offline apps , i have included more details to what was presented in TechEd while designing offline app.

  •  OData Modelling
  •  Performance improvement
    • Structuring Download / Upload calls
    • Delta calls
    • Using  Batch instead of single call.
    • Repeatable requests
    • Shared Data
    • Offline Relationship
    • Reading Data in pages
    • Repeatable request
  •  Conflict Resolution on mobile – Using ETags.
  •  Conflict Resolution on server ( who wins when two user update same record )
  •  Error handling on mobile.
  •  Error handling on server ( Reprocessing errors in backend using idocs)

Offline SDK Evolution

Below are the new updates in offline SDK i learnt from from TechEd session by Matt Borges

  • Entity Store and Request queue DB
  • Schema upgrades
  • Ability to create new entities when offline

Before we get into the details of design considerations lets first lets discuss about the Architecture and features of SMP/CPms and Mobile SDK for offline.

 

SAP Cloud Platform – Mobile Services / SMP  Architecture

Picture shows high level architecture to connect CPms to on-premise system.

 

SAP Cloud Platform requires setup mentioned in the picture

  1. Cloud Connector to connect On-Premise system such as CRM , ECC and non SAP system.
  2. CPms – It stores minimal information about the app.

 

 

SAP is now mainly focusing on CPms rather than On-Premise version of SMP.

SMP On-Premise version will go end of maintenance in 2020.

 

Challenges customer are facing today with on premise setup.

  1. Time required to setup on-premise version of SMP
  2. Load balancing and DMZ nuisances
  3. Maintenance and patch level, version upgrade.
  4. Server outage due to incorrect sizing.
  5. Increase in number of users

With CPms all above challenges are addressed and customer can focus on app development rather than worrying about infrastructure needs.

 

Features of SAP Cloud Platform

Below picture shows the features of SAP Cloud Platform.

 

Now going back to design consideration and best practices when defining offline mobile app.

OData modelling plays a very important role when designing mobile application.

Data model and mapping of backend call (RFC) is very important , lot of customers and partners design backend and then they adjust the mobile design,  instead first data model should be created and then RFC should be mapped to the data model. Designing the data model requires understanding of backend as well as understanding of offline store in mobile.

Offline store in mobile is created looking at the oData model and metadata , all the tables (keys and relationship) in offline store are created from metadata.

.ini configuration in CPms/SMP helps to define indexes, delta token lifetime to optimize the performance.

 

Performance Improvement 

Upload & Download

Its very important to understand the sequence for synchronization first the upload need to performed then download , if done vice versa  user changes in offline can get lost.

Upload of information is done by Flush method in SDK which sends the information from request queue and Download by Refresh method which downloads the information to entity store.

 

Upload

It is important to understand what happens when data is created and updated in offline store. During Uploads( flush method in SDK) the operations in the request queue are sent to the server. New records which are created with local IDs are resolved with real IDs.

Records which failed to execute in backend for some reason, offline store keep track of error information and puts affected entities into the error state, user can query the errors from error archive to show the reason for the failed records.

Upload is the frequent activity so users can upload information several times in day.

Download 

Download also resolved local entities/relationships with backend data.

Download activity is not frequent , it is required during the start of the day or couple of times depending on scenario. Delta plays very important role during download of information which i have highlighted below and various mechanism to implement delta.

During initial download when user is onboarding all the data need to be downloaded which includes master data , configuration data and transactions. Transaction data is needed be downloaded frequently when on field, so its very important to separate out  master data and transaction data when defining offline store or defining request.

Several offline stores can be created depending on scenarios.

Defining request is which tell what should entities should participate in synchronization during download.

 

Deltas 

The data which is changed should only be downloaded and sync to mobile.

Delta can be implemented in backend by Syclo Exchange Framework or by Gateway.

Determining delta on gateway is more expensive call compared to Syclo Exchange object.

There are two places where deltas come into play:

  • Between the SMP/CPms server and the client device (MobiLink protocol)
  • Between the OData backend and the SMP/CPms server (OData delta protocol)

To maximize performance, the OData delta protocol SHOULD be implemented by the OData backend to ensure only the changes are sent to the SMP/CPms server.

By default, if no OData deltas are provided by the OData backend, SMP/CPms will compute the delta to be sent to the client.

 

There are two main approaches to implement Delta Token functionality in backend

  1. Syclo Exchange Framework based approach

It calculates deltas at modification time, wherein the ABAP system tracks relevant changes, as they occur. At request time, the deltas are already prepared, making them available. Though this approach needs higher development efforts, however, it’s more scalable and ensures an optimized overall performance.

 

  1. Delta determination at request time approach

In this type of approach, the system compares old and new states to check which records have been modified/deleted. Here, though the implementation effort is small, it however does not optimize the backend performance. This implies, if you have more records in the full collection, the response time of the request tends to get longer.
However, with both the approaches, the payload of the response is minimized, though the first approach seems to be more prudent, as it ensures optimization of the backend performance.

 

Batch Calls

Using batch in offline store is extremely important , it reduces the number of roundtrips between the end-user and the SAP NetWeaver Gateway server. As a result, network time is reduced.

It helps to reduce CPU time of server in case of update calls with single atomic unit of work.

Always consider to use batch calls against the single call to improve performance of server and mobile client.

Shared Data 

Its very important to understand use of Shared data , if same data needed to be downloaded for group of client should be made as shared.

Shared defining queries are cached in the SMP/CPms server so that all clients can retrieve the shared data from the cache instead of all going to the backend.

Using shared data will reduce the load on backend and help to improve performance

 

Offline relationships without using $expand

Maintaining cardinality is equally important to define offline store to improve performance. Associations in the $metadata (except many-to-many) can be exposed as referential constraints.  Offline OData can build the relationships for the Offline store using referential constraint.

$expand is only supported for ODataV4 ( NOT OData V2) but implementation can get very complex.

 

Example

Original defining query

  • Customers?$expand=Orders/OrderItems/Product

Separated after exposing referential constraint

  • Customers
  • Orders
  • OrderItems
  • Products

 

<Association Name="Product">     
       <End Multiplicity="*" Role="ItemEnd" 
Type="SAP.OrderItem"/>     
       <End Multiplicity="1" Role="ProductEnd" 
Type="SAP.Product"/>     
       <ReferentialConstraint>         
          <Principal Role="ProductEnd">             
            <PropertyRef Name="ID"/>         
          </Principal>         
          <Dependent Role="ItemEnd">             
            <PropertyRef Name="ProductID"/>         
          </Dependent>     
   </ReferentialConstraint> 
</Association> 
 

 

Reading data in pages

 

Reading the data in pages allows users to display complete data using client and server driven paging. Client-driven paging; add $top and $skip to the local queries .

For server-driven paging; specify the page size (using a Prefer odata.maxpagesize header or store option) and the SDK will return a next link if there is more data

 

Creating and updating records with relational entities in offline store

When working with parent/child objects in the offline data store,  updates which is  performed are done against the “__metadata/uri” field of the object.

SAP Offline OData store works as below:

  1. When new data is posted to the offline store, it creates in internal ID/URI which represents the location of this data in the offline store (“__metadata/uri” of the object/response).
  2. Any updates must be made to this object or else the context may get lost when flush happens. PATCH/MERGE/PUT to the metadata uri.
  3. Include the etag as a parameter (this is also in the __metadata).

Example using batch:

batchChanges.push(oModel.createBatchOperation(uri, “PATCH”, oEntry, {sETag: etag}));

  1. Keeping the keys blank is the essential part (if the key is going to be generated on the backend at sync time).  To perform any updates to the record  use the record locator url and etag to reference the item to update instead. (example “/Entity(lodata_sys_eid = X’047AD786A01011E4800095AE655F2D3900000000′)” .  If there is need to display a temporary key field in the offline app until sync happens, I would recommend creating a secondary field to store the temporary display name so you can the leave the key field blank.

 

 

 

Repeatable Requests 

The client can make the request multiple times with the same RequestID and RepeatabilityCreation header values and get back an appropriate response without the server executing the request multiple times

When an upload is called with failed requests in the queue, a merge algorithm is run to merge the failed requests with new requests referencing the same entity to attempt to create requests without the errors.

 

Repeatable requests is a feature that OData servers can support to prevent the same operation (for example, a POST) from being executed more than once.

Repeatable requests is important for all OData backends, but in particular for those used with Offline OData.

 

New features which were demonstrated in TechEd Session

 

Offline Store and Request queue

With the new version two databases are created Entity store and Request queue, this is helpful when any schema changes are done.

Mobilink protocol is used for communication between SMP/CPms and mobile.

Entity store contains defining requests and if delta is enabled,  delta link information from response of entities so that mobile can use delta queries to optimize download. It stores backend data which ultralite DB.

 

Request queue contains create , update and delete information and response of CUD operations. It also stores error information and temporary local key and real key mapping.

Picture besides is picked up from TechEd session and shows the high level details about offline store.

 

 

 

 

 

 

Schema Upgrades

Schema changes are done frequently even after the roll out, user have to upload the data to get the new updates but it is not possible with several 1000 devices in the field .

This is new update i learned from SAP TechEd where schema changes does not impact offline store as long as OData standard are followed.

Changes are automatically updated to offline store without recreating it.

 

Operations in the request queue that had not been uploaded to the backend should not be affected by the schema change.

 

Ability to create new entities while offline

Often there is need to store local data when offline , cool feature was added to SDK to create new entities while offline and entities can be modified and read.

Rather than assign arbitrary key values (or rely on values that could be changed), generate a unique entity ID for newly created entities and use that ID in the edit/read link returned in responses until the real ID/keys become available

Rather than construct URLs from key properties (which may not exist or could be changed), simply use the URLs that are returned

Below is the JSON Example to demonstrate how to use it.

{     
      "__metadata": {         
           "uri": "Customers(113)",         
           "type": "SAP.Customers”     
      },     
      "ID": 113,     
      "Surname": "XXX",     
      "GivenName": "XXXX", 
}, 
{     "__metadata": {         
        "uri": "Customers(lodata_sys_edt=X'5CD949C1F0F11343443AAA7C200000000')",         
        "type": "SAP.Customers",     
     },     
     "@com.sap.vocabularies.Offline.v1.islocal": true,     
     "ID": null,     
     "Surname": "QWER",     
     "GivenName": "XXX", 
} 

 

Conflict Resolution & Error Handling 

Using etags helps to handle conflict when several users are modifying the same records. Etags uses timestamps to compare the server data with client data to determine the conflict. There are weak and strong etags can be implemented depending on usecase.

Data related error returns response as 200 which is considered successful by mobile client and removes it from request queue , backend need mechanism either manually fix the data and logical error in the backend  or IDOC’s to reprocess the errors.

There are several uses to do Conflict Resolution on server ( who wins when two user update same record ) and users on server has ability to decide and overide the changes.

 

Limitations .

  1. Offline SDK does allow condition based synchronization for upload meaning synchronize when certain condition are met- Example upload only when order is completed in offline.
  2. Does not support 1: N relationship for upload for offline , SDK support deep insert with 1:1 relationship. Limitation results in creation of Staging table in backend which unnecessarily increases overhead on development.
  3. SDK does not allow to batch multiple entities in single batch.
  4. OData V2 does not support deltas when $expand are used in defining queries.
  5. Mobile app connects to backend system to sychronize data , it has to go through severals systems like SAP Gateway, CPms, Cloud Connector and Private mobile network and so forth its very hard to find out the network lag during the peak hour. A tool to trace the network hop will help customer to trace the network issues.

Limitations which we see today hopefully Mobile product development team  Matt Borges  addresses it in future release or suggest better options to implement.

 

To report this post you need to login first.

2 Comments

You must be Logged on to comment or reply to a post.

  1. Filippo Consonni

    Hi Manish,

    I’m trying to implement a Windows Application that connect to HCPMs to download data from OData backend and to store this data on offline database (mobilink sync). I’m using SMP SDK for Windows 3.16.3
    I have 2 different scenarios:
    – UWP App: all works well, I can import necessary package from the SDK
    – Desktop (WPF) application: I can’t import the SAP.Data.OData.Offline.Store dll, then I can interact only in online mode

    The Offline capabilities in SDK for Windows are compatible with desktop application?

    Thanks

     

     

    (0) 
    1. Manish Meshram Post author

      Hi Filippo,

      Earlier version of SMP i.e 2.3 with SUP allowed to create offline capabilities for windows desktop but now sure if this version allows to create offline on Windows desktop we have not tried it.

      I will request Matt Borges  to reply on this.

       

      Regards

      Manish

       

      (0) 

Leave a Reply