New features for Windows in SAP Mobile Platform SDK – Part 6 (Offline support)
Update: The code for the sample application is now published.
In this blog, I will be talking about offline support for Windows. If you have been following my other blogs (or corresponding blogs for Android and iOS), you are already aware of the concept of a Store. You are probably aware that there is an Online Store and by extension, an Offline Store. Although I haven’t talked about an Offline Store explicitly, I have made a few subtle references to its existence in my previous blogs. An astute reader might have already guessed that an Online Store can be used when the device has network connection. So what about the Offline Store? A common mistake most people make, is that they only associate Offline Store with no network connection. While this is true (Offline Store is used with no network connection), an Offline Store can also be used when there is network connection. In fact, I would recommend using the Offline Store in all cases (if you want offline support) – and not use the Online Store at all. Of course, this would also depend on the business needs. But using the Offline Store regardless of network connectivity reduces the complexity of switching back and forth between Online and Offline Stores. The caveat is that the user might be working with stale data.
Comparing Online and Offline Stores
In the table below, I roughly compare the features of an Online Store and an Offline Store.
Online Store |
Offline Store |
Can work with network connection only |
Can work regardless of network connectivity |
Opening Online Store – straightforward |
Opening Offline Store – requires more options |
HTTP request – roundtrip to server |
HTTP request – reads from local ultralite database |
Very basic persistence using Technical Cache |
Full-fledged persistence using ultralite database |
CRUD operations – accomplished by OData calls |
CRUD operations – OData calls are intercepted and converted to SQL calls and executed against ultralite database |
No data latency |
Data latency exists |
No concept of flush |
Flush method pushes changes made locally to the backend |
No concept of refresh |
Refresh method pushes changes made on the backend to the local ultralite database |
Data conflicts – very minimal |
Data conflicts – a real possibility |
HTTP(S) protocol to transfer data between SMP Server and device |
Mobilink protocol to transfer data between SMP Server and device |
What happens when I open an Offline Store?
The first time an Offline Store is opened, a lot of things happen. The Offline Store connects to the SMP Server and sends it the defining request. It is very critical to understand what constitutes a defining request. A defining request is an URL representing data fetched by an HTTP GET method that needs to be persisted. An application can have multiple defining requests.
Examples of defining requests… Note that each defining request corresponds to a single GET request.
- Request 1 = BusinessPartners?$expand=SalesOrders/Items
- Request 2 = SalesOrders(‘0500000002’)?expand=Items
- Request 3 = Products?$expand=ProductDetails
Request 1 |
BusinessPartners
SalesOrders
Items
|
Data persisted on device BusinessPartners
SalesOrders
Items
Products
ProductDetails
|
Request 2 |
SalesOrders
Items
|
|
Request 3 |
Products
ProductDetails
|
SMP Server queries the backend using the defining requests. The union of the data retrieved by all the defining requests is used to populate the database on the SMP Server. This prepopulated database is then sent to the device. Note that data is not duplicated on the device – just the union of the data retrieved by all the defining requests is persisted. For efficiency reasons, it is recommended not to have overlapping defining requests. However, even if you have overlapping defining requests, data will not be duplicated on the device.
Lot less happens during subsequent openings of the Offline Store. The application does not connect to SMP Server. Therefore application does not require network connection. Application simply opens the existing database on the device.
Store |
Action |
Offline Store (first time opening) |
Needs network connection Defining request sent to SMP Server SMP Server queries backend using defining requests Creates a new database with backend response Sends newly created database to device |
Offline Store (Subsequent openings) |
Does not need network connection Simply opens existing database on device Does not connect to SMP Server |
How do I create and open an Offline Store?
Creating an Offline Store is straightforward. The constructor does not take any parameters.
this.Store = new SAP.Data.OData.Offline.Store.ODataOfflineStore(); |
Opening the Offline Store takes ODataOfflineStoreOptions as a parameter
await this.Store.OpenAsync(options); |
So, how do I create this “options” parameter? For this, you need to create the ODataOfflineStoreOptions object with the proper values. Thankfully, most values are intuitive. The following code snippet creates the ODataOfflineStoreOptions and populates it with the proper values.
var options = new SAP.Data.OData.Offline.Store.ODataOfflineStoreOptions();
var client = new SAP.Net.Http.HttpClient(new System.Net.Http.HttpClientHandler() {Credentials = new System.Net.NetworkCredential(“user”, “password”)}, true); // will be disposed by the store! client.DefaultRequestHeaders.TryAddWithoutValidation(“X-SMP-APPCID”, connectionId); client.DefaultRequestHeaders.TryAddWithoutValidation(“X-SUP-APPCID”, connectionId); client.ShouldHandleXcsrfToken = true; options.ConversationManager = client;
options.Host = “10.4.64.212”; options.Port = 8080; options.ServiceRoot = “com.sap.flight”; options.EnableHttps = false; options.StoreName = “OfflineStore”; options.StoreEncryptionKey = “SuperSecretEncryptionKey”; options.URLSuffix = “”; options.AddDefiningRequest(“TravelagencyDR”“TravelagencyCollection”false); options.EnableRepeatableRequests = true; |
Note: For operations that change state (for example, inserting a new resource) re-issuing the request may result in an undesired state (for example, two orders placed). Set EnableRepeatableRequests property to true to avoid such situations. Note that the backend OData Service must support this feature. SAP Gateway supports this feature.
CRUD operations on Offline Store after opening
CRUD operations can be performed on the Offline Store after successfully opening the Offline Store. The syntax for CRUD operations on an Offline Store is identical to the syntax for an Online Store. The only difference is that the data is retrieved from the locally persisted ultralite database instead of from the backend.
CREATE
var execution = store.ScheduleCreateEntity(entity, collectionName); var response = await execution.Response; |
READ
var execution = store.ScheduleReadEntitySet(collectionName); var response = await execution.Response; |
UPDATE
var execution = store.ScheduleUpdateEntity(copiedEntity); var response = await execution.Response; |
DELETE
var execution = store.ScheduleDeleteEntity(entity); var response = await execution.Response; |
Understanding Flush and Refresh
Flush and refresh allows the locally persisted data to be synchronized with the backend data. Both Flush and Refresh methods require network connection. Flush allows changes made locally on the device to be applied on the backend in an asynchronous fashion. Refresh on the other hand, allows changes made on the backend to be downloaded to the device. An important thing to note when calling the Flush method is that all the changes made locally on the device are submitted. The SDK currently does not support sending part of the changes. However, Refresh method has the option of downloading part of the backend changes based on defining request.
Flush method |
Refresh method |
Requires network connection |
Requires network connection |
Changes made locally submitted to backend |
Changes made on backend downloaded to device |
This call is asynchronous |
This call is asynchronous |
All changes are submitted – Cannot submit part of the changes |
Developer has the option of downloading part of backend changes based on defining request |
Call Flush before calling Refresh |
Call Flush before calling Refresh |
FLUSH
await this.Store.ScheduleFlushQueuedRequestsAsync(); |
REFRESH
await this.Store.ScheduleRefreshAsync(); OR await this.Store.ScheduleRefreshAsync(definingrequest); |
Some important considerations
- Offline support for Windows is only available for Windows Store applications and Windows Phone Store applications. Offline support is not available for Windows desktop .NET applications.
- The local database is created in the location given by: Windows.Storage.ApplicationData.Current.LocalFolder. It is essentially the application data directory. iLoData.exe is an utility that is shipped with SMP SDK SP11 onwards. This command line utility can be used to open the local ultralite database and view the contents. This is a great tool for troubleshooting purposes.
- The Offline Store takes care of getting and setting the XCSRF token in the MobiLink server component. Nothing additional needs to be done on the client application.
- Batch processing is supported in the Offline Store as well.
- In the event, the SMP Server is configured with multiple endpoints, then your application can have multiple Offline Stores (one for each endpoint – make sure you supply a unique store name for each Offline Store). You can also have multiple Offline Stores open at the same time.
- You can also have Online Store and Offline Store (for the same endpoint) in the same application. Both these stores can be open at the same time. Depending on network availability, the developer can choose which store to use. A lot of times it is easier to only have the Offline Store and use it regardless of network connectivity. Periodically call flush and refresh to sync the data with the backend.
Please feel free to post any comments or questions that you might have. I will try to answer them as soon as I can. Hopefully, these blogs have given you enough insights into the Windows SDK to start building new mobile applications. Good luck !
Hi Milton,
I am developing Native Android App, consuming OData Services. I want to use offline store options, because most of the time my app will be in offline. But when i try to open offline store my app crashes "Null pointer Exception in gCtx = lgCore.getLogonContext();".
ODataOfflineStore.globalInit();
lgCtx = lgCore.getLogonContext();
String endPointURL = lgCtx.getAppEndPointUrl();
URL url = new URL(HOST_NAME);
// Define the offline store options.
// Connection parameter and credentials and
// the application connection id we got at the registration
ODataOfflineStoreOptions options = new ODataOfflineStoreOptions();
options.host = "hosturl";
options.port = "8080";
options.enableHTTPS = true;
options.serviceRoot= HOST_NAME;
//The logon configurator uses the information obtained in the registration
// (i.e endpoint URL, login, etc ) to configure the conversation manager
// It assumes you used MAF Logon component to on-board a user
IManagerConfigurator configurator = LogonUIFacade.getInstance().getLogonConfigurator(LogonActivity.this);
HttpConversationManager manager = new HttpConversationManager(LogonActivity.this);
configurator.configure(manager);
options.conversationManager = manager;
options.enableRepeatableRequests = false;
options.storeName = SERVICES_METHOD;
//This defines the oData collections which will be stored in the offline store
options.definingRequests.put("defreq1", SERVICES_METHOD);
//Open offline store synchronously
offlineStore = new ODataOfflineStore(LogonActivity.this);
offlineStore.openStoreAsync(options);
//A way to verify if the store opened successfully
Log.d("OpenOfflineStore","openOfflineStore: library version"+ ODataOfflineStore.libraryVersion());
Looks like your option.serviceRoot value is incorrect. The serviceRoot property should not be set as the HOST_NAME. In the Windows example, I have set serviceRoot to be the name of application (com.sap.flight)
options.ServiceRoot = "com.sap.flight";
If you have additional endpoints, then you will set the serviceRoot to the name of the endpoint.
Hope that helps !
Hi Milton,
I am working with Windows Phone 8.1 and SMP Native SDK SP 07.
Thanks for your blog, I am able to on-board the device and retrieve the sample data from SMP Server.
But when I was working on offline support it crashed on initializing ODataOfflineStore.
I am using Windows Phone 8.1 emulator with x86 arch.
Code:
var store = new SAP.Data.OData.Offline.Store.ODataOfflineStore();
var options = new SAP.Data.OData.Offline.Store.ODataOfflineStoreOptions();
..........
..........
..........
await store.OpenAsync(options);
Error:
An exception of type 'System.IO.FileNotFoundException' occurred in SAP.Data.OData.Offline.Store.DLL but was not handled in user code
Additional information: The specified module could not be found. (Exception from HRESULT: 0x8007007E)
Note:
I have added the all needed nupkg of SAP. Some times SAP.Data.OData.Offline.Store not show in references list but it available in build path.
Using offline nupkg : SAP.Data.OData.Offline.Store.16.5.3.1690.nupkg
I had not included a couple of mandatory calls that need to be made when the application starts and application is closed - for offline support. I was going to include them in the sample application and 'How To... Guide'. Perhaps that is causing the issue.
// This needs to be called when the application starts.
// I would add this line in app.xaml.cs in the OnLaunched event... (and Resuming event)
SAP.Data.OData.Offline.Store.ODataOfflineStore.GlobalInit();
// This needs to be called when the application is closed
// I would add this line in app.xaml.cs in the OnSuspending event...
SAP.Data.OData.Offline.Store.ODataOfflineStore.GlobalFini();
I believe that is causing this issue. Let me know if that helps. I will update the blog soon with these mandatory calls.
Thanks you.
I have added the global functions in app.xaml.cs.
I have fixed that issue by referring Visual C++ 2013 Redistribution extension that resolved missing DLL dependencies of SAP.Data.OData.Offline.Store.
Still I am facing some issue.
Error:
A first chance exception of type 'SAP.Data.OData.Store.ODataNetworkException' occurred in mscorlib.ni.dll
SAP.Data.OData.Store.ODataNetworkException: [-10210] The operation failed due to an error on the server. ---> SAP.Data.OData.Offline.Store.ODataOfflineException: [-10210] The operation failed due to an error on the server.
--- End of inner exception stack trace ---
at SAP.Data.OData.Offline.Store.ODataOfflineStore.<OpenAsync>d__6.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)
at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
at System.Runtime.CompilerServices.TaskAwaiter.GetResult()
at SAP.Data.OData.Offline.Store.ODataOfflineStore.<OpenAsync>d__3.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)
at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
at System.Runtime.CompilerServices.TaskAwaiter.GetResult()
at InspectSMP.MainPage.<<InitUI>b__6>d__e.MoveNext()\
My code snippet:
var options = new SAP.Data.OData.Offline.Store.ODataOfflineStoreOptions();
var client = new SAP.Net.Http.HttpClient(new System.Net.Http.HttpClientHandler() { Credentials = new System.Net.NetworkCredential("myusername", "mypassword") },
true); // will be disposed by the store!
client.DefaultRequestHeaders.TryAddWithoutValidation("X-SMP-APPCID", Windows.Storage.ApplicationData.Current.LocalSettings.Values["ConnectionID"].ToString());
client.DefaultRequestHeaders.TryAddWithoutValidation("X-SUP-APPCID", Windows.Storage.ApplicationData.Current.LocalSettings.Values["ConnectionID"].ToString());
client.ShouldHandleXcsrfToken = true;
options.ConversationManager = client;
options.Host = "xx.xx.xx.xx";
options.Port = 8080;
options.ServiceRoot = "http://xx.xx.xx.xx:8080/com.xxx.yyy";
options.EnableHttps = false;
options.StoreName = "OfflineStore";
options.DefiningRequests.Add("defreq1", "Customers");
options.EnableRepeatableRequests = false;
await store.OpenAsync(options);
Looks like your options.serviceRoot value is incorrect. The serviceRoot property should not be set as the full path. In the Windows example, I have set serviceRoot to be the name of application (com.sap.flight)
options.ServiceRoot = "com.sap.flight";
If you have additional endpoints, then you will set the serviceRoot to the name of the endpoint.
Hope that helps !
I have changed the options.ServiceRoot as you mentioned. Even though the same error is occurring.
And is it mandatory to have the same SMP SDK version as SMP Server version?
For example I am using SMP Server 3.0 SP5 PL0 and SMP SDK 3.0 SP07.
Sorry, I didn't get a chance to respond to your question. Is there a way I can look at your setup ?
Hi Milton,
I am trying to create the offline store by i am getting the error as
The specified module could not be found. (Exception from HRESULT: 0x8007007E)
I have added the offline store package.
I am using SMP 3.0 SDK SP9.
can you help me how i can solve this issues.
Here is code:
SAP.Data.OData.Offline.Store.ODataOfflineStore offileStore = new SAP.Data.OData.Offline.Store.ODataOfflineStore();
Is there a way I can look at your setup ?
There is a minor change in the SDK in SP09. So I can walk you through those changes as well.
How i can share the code to u privately?
Hi Milton,
Any Update on the issue.
I have tried SMP 3.0 and SP9 PL2 also i am phasing the same error.
I noticed that the defining request is commented out.
//options.AddDefiningRequest("TravelagencyDR", appSettings.ApplicationEndpointUrl.ToString(), false);
Without the defining request, the Offline Store is meaningless…
Maybe that’s why you are getting the error.
Hi Milton,
I am using SMP 3.0 and SP9 to create the offline store but i am getting error as
The specified module could not be found. (Exception from HRESULT: 0x8007007E)
Here is the code:
SAP.Data.OData.Offline.Store.ODataOfflineStore offileStore = new SAP.Data.OData.Offline.Store.ODataOfflineStore();
Hi Milton,
Its not going to options. it is showing the error at creation without any arguments
SAP.Data.OData.Offline.Store.ODataOfflineStore store = new SAP.Data.OData.Offline.Store.ODataOfflineStore();
I ran your project and was able to reproduce the issue. The problem is that you are not calling the method below... This needs to be called when the application starts up. You also need to call the corresponding GlobalFini() method when the application is suspended.
SAP.Data.OData.Offline.Store.
ODataOfflineStore
.GlobalInit();
SAP.Data.OData.Offline.Store.
ODataOfflineStore
.GlobalFini();
In App.xaml.cs, you can add the following snippets of code...
public App()
{
this.InitializeComponent();
this.Suspending += OnSuspending;
this.Resuming += this.OnResuming;
}
protected override void OnLaunched(LaunchActivatedEventArgs e)
{
// whole bunch of code here...
// Call GlobalInit when application launches
SAP.Data.OData.Offline.Store.ODataOfflineStore.GlobalInit();
// Ensure the current window is active
Window.Current.Activate();
}
private void OnSuspending(object sender, SuspendingEventArgs e)
{
var deferral = e.SuspendingOperation.GetDeferral();
//TODO: Save application state and stop any background activity
deferral.Complete();
// Call GlobalFini when application suspends
SAP.Data.OData.Offline.Store.ODataOfflineStore.GlobalFini();
}
private void OnResuming(object sender, object e)
{
// Call GlobalInit when application resumes
SAP.Data.OData.Offline.Store.ODataOfflineStore.GlobalInit();
}
And last but not least, make sure you add a reference to Microsoft Visual C++ 2013 Runtime Package for Windows
Problem:
Unable to open offline store
Details:
Error: The specified module could not be found. (Exception from HRESULT: 0x8007007E).
Code Used for Creating the Offline Store:
var client = new SAP.Net.Http.HttpClient();
await store.OpenAsync(options);
Version Used:
SMP3_SP9_PL1 and SMP3_SP9_PL2
I Have tried the below step also
SAP.Data.OData.Offline.Store.ODataOfflineStore.GlobalInit();
same error i am facing the same error.
have you seen references image which i attached in previous comment?
@Imp: Where ever i am using the ODataOfflineStore i am facing the error. I hope it might be SDK Issue.
I am using the project you sent me and I am able to run fine. Please see the bitmap below and my breakpoint is past the line that breaks.
Have you added the reference ?
Thanks Milton. Its working after adding Microsoft Visual C++ 2013 Runtime Package for Windows. But i am facing the problem while i am doing this step
For Creating this is the step i am using:
var offlineExecution =
SharedContext.Context.OfflineStore.store.ScheduleCreateEntity(SharedContext.Context.Entity, "CollectionName");
@ScheduleCreateEntity : It is taking only one collection(Entity) as input, but i want to insert a group of collections (EntirySet) as input.
Can you insert the collection of entities sequentially ? Or you could use a batch request to insert all the entities - it still inserts one entity at a time, but the change set can be treated as a transaction...
Hi Milton,
Is $filter working in offline in SP 10 PL01?
If works how we need to call filters?
$filter can be used to filter the number of rows. The defining request can include the $filter query option to filter the number of rows.
For example,
options.AddDefiningRequest("ProductsDefiningRequest", "Products?$filter=ProductID lt 10", false);
No Manikanta, from SP10 PL10 on wards it is working check with updated package.
Happy coding 🙂
HI Milton,
Your post for Offline was awesome, I am having some queries.
I am adding the
"options.AddDefiningRequest("req1", "NotificationCollection", false);"
for fetching all the NotificationCollection it is working fine, but I need to get a particular record as NotificationCollection('1110001234').
So how can achieve this using Offline.
Your help is very appreciable.
Hi Milton,
How to add custom header, I am using as following.
client.DefaultRequestHeaders.Cookie.Add(new HttpCookiePairHeaderValue("X-SMP-APPCID", SharedContext.Context.ApplicationConnectionId));
client.DefaultRequestHeaders.Cookie.Add(new HttpCookiePairHeaderValue("X-SUP-APPCID", SharedContext.Context.ApplicationConnectionId));
It is working fine but I want to add another header which is for localization purpose.like below,
client.DefaultRequestHeaders.Cookie.Add(new HttpCookiePairHeaderValue("langu", "FR"));
But it is not working. (here client is an HttpClient object which will be passing to online store object while login)
Can you please check this.
Hello Milton,
We are developing Windows 8.1 Store app and we are using offline store feature to download data and use it for offline purpose.
During synchronization of data, we are getting bulk of records (4k) delta coming from backend oData server during Refresh.
We have seen this delta is processed on backend server in around 10 minutes in access logs.
Unfortunately we get <a:DateTime>2016-07-22T11:30:08.0960521Z</a:DateTime><lid>SAP.Data.OData.Offline.Store</lid><msg>[5018] Finished refreshing the store.</msg>back from await this.Store.ScheduleRefreshAsync(); after 4 and half hours.
Please see below logs which are captured from client device.
Green:- Pushing Offline data from Client to Server
Yellow:- Fetching updates from Server to client DB
Red :- Delay of 4:30 hrs. is observed while updating client database.
<a:DateTime>2016-07-22T07:05:18.2804799Z</a:DateTime><msg>Calling to Flush the Offline Data </msg>
<a:DateTime>2016-07-22T07:05:18.2951162Z</a:DateTime><msg>Flushing the Offline Store</msg>
<a:DateTime>2016-07-22T07:05:18.2961168Z</a:DateTime><lid>SAP.Data.OData.Offline.Store</lid><msg>[5015] Flushed the request queue.</msg>
<a:DateTime>2016-07-22T07:05:18.6321261Z</a:DateTime><lid>SAP.Data.OData.Offline.Store</lid><msg>[5016] Finished flushing the request queue.</msg>
<a:DateTime>2016-07-22T07:05:18.6330777Z</a:DateTime><msg>Offline Store flushed</msg>
<a:DateTime>2016-07-22T07:05:18.634085Z</a:DateTime><msg>Finished to Flush the Offline Data </msg>
<a:DateTime>2016-07-22T07:05:18.6351157Z</a:DateTime><msg>Calling to Synchronize the Offline Data </msg>
<a:DateTime>2016-07-22T07:05:18.6723634Z</a:DateTime><msg>Refreshing the Offline Store for both master and user data</msg>
<a:DateTime>2016-07-22T07:05:18.6733656Z</a:DateTime><lid>SAP.Data.OData.Offline.Store</lid><msg>[5017] Refreshing the store.</msg>
<a:DateTime>2016-07-22T11:30:08.0960521Z</a:DateTime><lid>SAP.Data.OData.Offline.Store</lid><msg>[5018] Finished refreshing the store.</msg>
<a:DateTime>2016-07-22T11:30:08.0980388Z</a:DateTime><msg>Offline Store refreshed with master and user data details</msg>
<a:DateTime>2016-07-22T11:30:08.0990395Z</a:DateTime><msg>Finished synchronize the Offline Data </msg>
We are observing this delay whenever we are getting bulk data.
Can you help me understand how the CRUD operations which are coming from oData backend are executed on Client UDB.
is there any SMP Server performance involved as we do not ask SMP to track delta?
is there way to log all CRUD operations performed on UDB in client device logs?
I am using SMP SDK 10.5 and SMP Server 3.10.0.6.
Unless you are asking for clarification/correction of some part of the Document, please create a new Discussion marked as a Question. The Comments section of a Blog (or Document) is not the right vehicle for asking questions as the results are not easily searchable. Once your issue is solved, a Discussion with the solution (and marked with Correct Answer) makes the results visible to others experiencing a similar problem. If a blog or document is related, put in a link. Read the Getting Started documents (link at the top right) including the Rules of Engagement.
NOTE: Getting the link is easy enough for both the author and Blog. Simply MouseOver the item, Right Click, and select Copy Shortcut. Paste it into your Discussion. You can also click on the url after pasting. Click on the A to expand the options and select T (on the right) to Auto-Title the url.
Thanks, Mike (Moderator)
SAP Technology RIG
I am getting below error when instantiate odata offline store.:
"Requested Windows Runtime type 'lodatawinrt.RequestFailureContext' is not registered."
Getting error in following line of code:
this.Store = new SAP.Data.OData.Offline.Store.ODataOfflineStore();
my application is UWP app, this code works in Windows8.1 and windowsphone8.1, do you know why?
thanks,
yong.
Yong Hu, did you find the problem?
Hi Milton,
the link for code for the sample application is no more available. Is it possible to have it?
Thanks in advance
Best regards