Lessons learned from a SAP Work Manager implementation
Hi everyone,
As we are working towards the go-live of a huge SAP Work Manager implementation, I would like to share some of the lessons that we as a team learned out of this implementation.
This was(and continues to be) a very challenging implementation and the SCN community has been extremely helpful through out, this is a small way I can give back to the community. Although I would have preferred to write a technical blog(I already have a couple of ideas and will write later), but I hope these LL’s will help all of you in your Work Mgr. implementations and client conversations.
Before I start, I wanted to cover the skillsets that were part of our team, you will pretty much need all of these in your implementations:
1) SMP administrator/Mobile architect – This is first on the list as this was my role(just kidding, these roles are not listed in any particular order). You will need an SMP administrator who will be responsible for sizing, installations, upgrades, application deployments etc.-> this could be the BASIS guy at the client side. In our case, we had amazing BASIS support from client. You also need a person responsible for the overall system architecture, troubleshooting system issues, network issues, evaluating work manager sync performance and pretty much everything that is part of your mobile landscape. Below is a high level overview of our mobile landscape.
MDM = mobile device management solution like Afaria, Mobile Iron, Air Watch etc.
2) Syclo – configuration/functional – This person is responsible for all the configuration at the Syclo level. Technically, an experienced EAM functional consultant can upskill in this area as well, but you are better off having a dedicated resource for this.
3) Syclo technical, agentry developer + ABAP developer – Responsible for all development work, this is your development team. We had 2 developers on our team – one was an Agentry developer who could understand ABAP concepts and the other was an experienced ABAP developer who also had Agentry development experience. Having developers who can understand concepts on both sides is a HUGE advantage. It is difficult to find resources in this area.
4) iOS developer/platform specific developer – We deployed our application on iOS devices, so initially there was some work to include GIS libraries in the client. We were able to fill this role with the help of an off shore iOS developer. Based on the complexity and scope of your implementation and the platform, you will need more resources. Currently, we are working on integrating GIS with SAP Work Manager, and had to bring on additional resources. Here is my blog on “How to integrate GIS libraries in SAP Work Manager” – Integrate GIS libraries in SAP Work Mgr. – iOS – part 1
5) MDM administrator – Person responsible for MDM concepts of your implementation. We ended up working a LOT with this person. In our case, this role was from the client side.
I will add more to this list if I think I forgot any, but now for here are the lessons learned. Please note the following:
1) Some of these might be more specific to system architecture. I am sure that other team members would like to add challenges from their particular area – I will discuss with them and have them add lessons learned from their side as well.
2) These are not listed in any particular order
3) Not all of these are Agentry specific, some of these might apply to other mobility and user experience implementations as well
Lesson learned 1 – Ensure that you have client support across each technical component in your mobile architecture
As you can tell from above architecture diagram, we had 6 servers on the MDM side(including load balancer), 2 application servers and 2 database nodes + 1 Cisco load balancer on SMP side, SAP(8 application servers, and database nodes configured in active-standby mode). In addition, we had other servers like Document Management system, GIS server and the list continues. It is important for you to establish that you will need support from all involved parties if you want your implementation to succeed. We struggled with this initially, but eventually we had buy in from all sides.
Lesson learned 2 – Evaluate how mature your client is in terms of enterprise mobility, do they have other apps that work on the same platform?
Don’t assume anything, just because they have an MDM solution and a device policy in place. For example, in our case, this was one of the first enterprise iOS applications that was being rolled out so we found out that the infrastructure was not in place for Macs to connect to the client’s intranet. This of course meant that we could not test iOS application code on the simulator. So initially, we had to make small changes, package the application and send the ipa to the MDM administrator. They would upload it to the enterprise app store, we would download it and test and go over this process all over again for additional changes.
Lesson learned 3 – Make sure expectations are set for performance related metrics like initial and delta sync performance etc.
Our client had a LOT of asset data in their system, just crazy amounts. Of course, this had an impact on how long initial syncs took, in our case we started off at around 50 mins to an hour for an initial sync -> this was unacceptable. We had to go through a lot of performance optimization exercises, there were some issues at the code level, some at the DB level and some were product issues that were causing performance degradation. In the end we got this down to about 30 mins average for an initial sync.
Lesson learned 4 – Conduct enough proof of concepts and shadow your end users as much as possible
In the initial phases of the project, we conducted 4 POCs and solicited feedback from end users both from an application AND device perspective. iPad was the preferred device. We also got very good insights as to how the end users will actually use the application as we followed them during the day to maintenance sites as they were going about their days. This was critical for us to empathize and understand some of the pain points they experienced in the later stages of the project.
Lesson learned 5 – Don’t just listen to anecdotal feedback about sync performance, pull sync statistics to have the right data
In the course of the project, we started dreading the term “sync issue”. Don’t get me wrong, we had a LOT of these sync issues initially, some of them were due to poor network, some due to product issues, and some were triggered due to room for backend performance optimization. Regardless of the cause, everything was clubbed under the umbrella of “sync issues”. We resolved a lot of these sync issues, but there was still a negative perception about the solution.To get around this, we started pulling Agentry logs daily(server, events, messages log) etc. and started identifying patterns in these logs -> I cannot tell you how many server and messages logs I had to analyze. But the hard work paid off, we now have an automated excel sheet(and also an ABAP program), that takes agentry logs as inputs and gives us sync statistics as output. This gave us some data to turn around the “perception” around sync issues. See example below:
Lesson learned 6 – Use your application as your users would and do not underestimate the role of network, something as trivial as this can influence solution adoption
This in some way ties into lesson learned 4. As consultants, we were used to hooking devices to our mi-fi devices and working off of those. During pilot go live we realized that some of the client sites had network speeds of about 1 Mbps/second. Trying to perform an initial sync with a backend system with a lot of asset data is like trying to watch a 4 hr Netflix movie on a dial up connection. Moreover, we were told that the end users would come to the yard in the morning and sync their devices to get that day’s work on the device and then sync back again from the yard when they are back. We found out the end users were initiating syncs from the maintenance sites, some of these were in the middle of no where with bad network. All of these things should be cleared out before implementation. We need to understand that Agentry framework works in an offline setting, but needs good network for the syncs to go through without too many issues
Lesson learned 7 – Do not underestimate the impact of a major iOS upgrade
With iOS 9 being released recently, Apple has changed some of the network libraries on their side. We found out that there were compatibility issues with SAP Work Manager and iOS 9, the result of this was that “per app VPN” ended up not working on iOS 9 devices. This of course meant that the application could not be used on iOS 9 devices. This issue is actually pretty recent and we are working with all the vendors to resolve this ASAP(this issue is specific to our implementation)
Lesson learned 8 – Check firewalls before troubleshooting anything
This is a pretty straight forward one, we ended up troubleshooting something for a week and later realized that the firewall exception request was not in place. Not fun.
Lesson learned 9 – Configure load balancer for ip stickiness
If you have a clustered SMP set up and use a load balancer in front of the SMP nodes, make sure the load balancer is set up for “ip stickiness”. You don’t want the load balancer to change nodes on the work manager client in the middle of the sync.
Lesson learned 10 – (SMP specific) Have early conversations around performance testing and disaster recovery plan
We actually got great support from SAP to execute our Agentry performance test. I had to re-validate server sizing iteratively as we got more accurate data around actually go live volume. Disaster recovery was a bit of a disaster. The client had no process to restore Linux machines from a tape back up and we had to resort to some really creative ideas to have a good disaster recovery plan in place. We used global site selector to configure this, there is a regular TCP health check probe and as soon as it detects that prod. VIP is down, it routes request to SMP DR servers. Of course, the disadvantage of this is that we have to maintain DR and prod. SMP servers manually and keep them in sync, which is additional effort.
As I mentioned, these are just some of the lessons that come to my mind. I will update this blog after getting inputs from my colleagues who were playing different roles on the team to get different perspectives.
I am also interested in hearing what lessons you guys learned during your implementations. Hopefully, this helps you address some of the challenges you might face in your projects and have the right conversations with your client.
My next project is a SAP Fiori implementation, so I am going to be off Agentry for a while, only to come back to it.
Cheers,
Abhinav Gupta
Hi Abhinav,
Thanks for your valuable experience sharing with us and this blog is really helping me for a upcoming SAP Work Manager pilot project (definitely keep it mind). Besides, I got two questions and appreciated it if you could give me some feedback.
Q1. What is the version number of SAP Work Manager, SMP, Agentry, and Syclo you implemented in this project?
Q2. Does any built-in approval workflow for timesheet entries in SAP Work Manager solution or can only be done via implementing custom workflow(s) in ECC to approve the entries?
Saw your next project is an SAP Fiori implementation project and I have done several this year. Below are my some experience.
1. performance problem:
2. cache problem:
Regards,
Nick
Hi Nick,
Thanks for the comment and the lessons learned for the Fiori implementation, you should share your lessons and experience in the Fiori community.
Here are the answers to your questions:
Q1. What is the version number of SAP Work Manager, SMP, Agentry, and Syclo you implemented in this project?
We deployed SAP Work Mgr. 6.2 on SMP 3.0 SP07(will be upgrading to SP09 PL01), Agentry version 70.6.0.2. We built the iOS client with SMP SDK SP06. Back end component, SMERP 610_700 SP 04.
Q2. Does any built-in approval workflow for timesheet entries in SAP Work Manager solution or can only be done via implementing custom workflow(s) in ECC to approve the entries?
As far as I know, "Crew Mgt." add-on includes functionality for time approval, we did not have a time approval requirement as our application was only deployed to field technicians.
Good luck with your implementation!
Cheers,
Abhinav
Hi Abhinav,
Many thanks for sharing the useful information.
We also just completed our work manager 6.2 up gradation on SMP 3.0 from work manager 5.3.
We have also faced similar issues you have highlighted above. Below are the some of the major issues we are still facing.
1. Initial Sync issue
2. Issues when downloading the work order notification attachments
3. Opening the MS Word Doc in device iPad
For Issue 1:
a. We are not getting any sync errors, but the sync is getting struck at some point. So we are stopping it and syncing it again.
b. Another issue is the app is getting crashed while running some business processes from device iPad. What would be the root cause for this issue.
You have suggested that, you have an ABAP program to provide the sync statistics by taking the SMP & event logs as inputs. Could you please provide more info on this.
Just want to implement the same in our environment to analyze the sync issues..
For Issues 2 & 3:
We have raised these issues with SAP and they have suggested us to upgrade the work manager patch to SWM 6.2.1 which have lots of fixes for SWM 6.2 (See SAP Note 0002168629 ).
But, here we are struggling at one point to upgrade the work manager patch, because we have LOT of customization in front end Agentry application, java middle ware & back end ABAP code.
Could you please provide your suggestion on upgrading the work manager app to WM6.2.1 without disturbing the existing customization.
We thought, we can deploy the new SWM 6.2.1 standard java code into our custom app. To do this first we need to check the java custom classes code dependency. But we don't think there is dependency with Agentry front end app. Could you comment on this?
Thanks in advance,
Roopa
Hi Again,
Issue with Work Manager 6.2 app on iOS9 was resolved after upgarding the SMP 3.0 with below.
1. SMP 3.0 SDK SP10 PL01
2. SMP 3.0 Server SP09 PL01
Thanks & Regards,
Roopa
Hi Roopa,
Thanks for your comments. When your client gets stuck, are there any backend SAP processes still running? If yes, then you need to wait for SAP to process the request and send it back to the Agentry server(SMP). If the backend process is done and you don't see any activity in the messages log for a while(tail the messages.log file), then you have a problem.
What do you see in the messages log? Do the server logs give you more detail on what the server was doing when the sync got stuck? What network are you trying this on? Does this happen on a particular network or is this issue across networks?
For the app crash issue - are you able to reproduce this consistently? If yes, then I would recommend hooking your iPad up to a Mac and having an iOS developer look at the device logs.
More on your questions below.
Cheers,
Abhinav
Continuing from above....
I would have loved to share the ABAP program or the excel sheet that we have, but there is probably no way our scenario is similar to yours. But I can share how we went about this.
Every time we got a message that a user has had a "sync issue", we looked at the messages logs and server logs - we must have done this about a 1000 times. To make it worse, these issues were not reproducible on demand, after a while patterns started to emerge.
A very basic example is that we were seeing a bunch of "Client is not logged into the server(2)". We consistently saw a series of messages O(Outgoing), T(Trying), F(Failed) with code 5, exactly after 10 mins of the last request. The 10 mins timeout value was configured at 3 places:
- Inactivity timeout in Agentry Editor
- Inactive timeout in SMP under App. Specific settings
- Timeout when MDM solution would not see any traffic between client and SMP
We finally figured out this occurred when SAP took more time to send the data than the inactive value set in SMP -> which is why Agentry server issued a log out notice to the client. We figured out the exact pattern for this error and then with some excel magic we were able to partially automate this.
We had another issue when the client got stuck when uploading images on iOS 8.3 devices. The pattern for that was an abrupt S/R messages with code 200 after the login block(message 9) and initial exchange between client and server(202).
These are just two of about 15 cases that we were able to identify. Even after this much work, we have a category called "Other failed syncs", when it is impossible for us to figure out what happened -> I dread this category
I hope the approach helps, this is a very arduous process and I wish you luck. Glad to hear that your issues got resolved with the SDK and server upgrade, SAP had mentioned that SMP SDK10 has fixes for the iOS clients that will be used with iOS9.
Cheers,
Abhinav
Hi Abhinav
Great write up. This has prepared me for our development stage.
Are you able to provide a list of recommended enhancements? We are building our requirements at the moment and have identified some areas to enhance. Example
Thank you
Jeff
Jeff,
A lot of what you have listed is coming in the next version of Work Manager so depending on when you are looking to get started you might just want to wait and see how well the new functionality will meet your needs.
If you are starting the project now, then start with WM 6.2.1 and stay out of the box and upgrade when 6.3 comes out.
Just a thought.
--Bill
Jeff,
Thank you for the note, I am glad that this helped. We had added an enhancement for uploading image back to SAP for notifications created locally on the device.
For the other stuff, as Bill mentioned, if it is being included as part of WM 6.3, your best bet might be to start with 6.2.1 now and upgrade in the middle of the project.
What does your timeline look like?
Regards,
Abhinav
Very nice writeup Jeff!
At the beginning of this project did you consider using Fiori or something aside from WM / Agentry?
Now that things have progressed with HCPms and offline access, etc. would you consider that at this point?
The size & scale of all of the middleware seems a bit daunting to setup, and to maintain over the long run.
Kind regards,
Gavin
Hi Gavin
I did have a quick look a Fiori, but it was always pushed that it never worked offline.
I am very interested in this. So I will begin to google HCPms.
Can you provide any links where I can find more information.
Thanks
Jeff
You can use the Cordova based container to access some of the native device feature and host Fiori apps on SMP to make use of the offline features. Check out this blog:
SAP Fiori & SMP
If you have further questions around SAP Fiori and offline, I would recommend creating a new thread in the SAP Fiori community.
In our scenario, our client already had work manager licenses and we wanted to utilize the responsiveness that native WM app offers, so it was a no brainer.
Cheers,
Abhinav
Hi Abhinav,
Thanks for the nice article. I am more interested in understanding what you meant by
"2) Syclo - configuration/functional - This person is responsible for all the configuration at the Syclo level. Technically, an experienced EAM functional consultant can upskill in this area as well, but you are better off having a dedicated resource for this."
I am a SAP functional person and trying to learn Work Manager. Appreciate if you elaborate more on my questions and direct me to any documentation available.
Appreciate your help.
Thanks
Sanjay
Sanjay,
You can refer to the reference and configuration guide found at https://service.sap.com/instguides -> SAP Mobile -> Agentry.
That would be a good start. Apart from that, the best thing to do is to play around with the Syclo configuration panel if you have access to a sandbox system.
Hope this helps.
Cheers,
Abhinav
Thanks Abhinav for quick response.
Awesome summary of our journey Abhinav !
Thanks for the good support Donny!(and the awesome pool showdown)
Hi Abhinav,
first let me thank you for great post, it is and will be helpful.
Have you implemented any sort of Barcode and/or RFID integration with WM?
If you did, could you share some of your experience please?
Mario
Mario,
Apologies for the delayed response, have not been to the Work Mgr. side of things in a while as I am in the middle of a SAP Fiori implementation.
We did not enhance WM for Barcode/RFID. I think it is possible using openUI etc. I am pretty sure that I came across some good material around this. If it still does not help, I would recommend creating a separate post.
Cheers,
Abhinav
Hi Abhinav;
This is a great write up of lessons learned from Work Manager 6.2. We are currently working on an implementation on Work Manager 6.2 with SMP SP10 PL01, and we are seen a lot time taking on the initial sync (@45+ min). We have a huge amount of FLocs, Equipment, and Parts, and we are looking on ways to filtering more of this objects to reduce the sync time. However, I wanted to check with you if you could share if there was a specific SAP Notes that that helped reduce the initial sync or if performed specific development to for this. We will be having @1000 users, and we will be running all three main platforms (e.g. Android, iOS, and WPF).
Also let me know if you have any questions in regards Fiori implementation. I just finished a Fiori implementation that included Transactional, Factsheets, and Smart apps.
Regards, Eric
Hi Eric,
You can try following options to reduce the sync time:
1. You can analyze which Complex tables are not used for the functionalities delivered in current go live and disable those complex tables from the Syclo Config Panel.
2. You can use filters. We have faced a situation once for a customer where lakhs of the parts were downloading and hence more sync time. So, ABAP consultant has done filtering of parts based on the country of the logged in technician. That client had different parts for different countries. You can check the possible way of filtering data for your customer.
Regards,
Sahil Dudeja
Hi Sahil;
Thanks for your feedback. Filters is one thing that we are constantly re-visiting to see what other filter can be apply. I will post any new options we have apply to reduce sync time.
Regards, Eric.
Eric,
Apologies for the delayed response, you posted your question a couple of months back and I am hoping that your problem is already solved. Can you try the below suggestions:
1) Capture a performance trace in the backend and see if there are indexes you can activate/apply to reduce query times?
2) You could think of including the Agentry.db file in the client application, this completely eliminates the initial sync. When users are done with their work. they can initiate a "delta sync" which will take lesser time. The Syclo functional consultant on our project also worked with SAP to tweak some settings in the Syclo config. panel that controlled, when new Flocs etc. get pushed to the device. Let me know if you are still looking for a solution, I can get you some specific details.
Cheers,
Abhinav
Unless you are asking for clarification/correction of some part of the Document, please create a new Discussion marked as a Question. The Comments section of a Blog (or Document) is not the right vehicle for asking questions as the results are not easily searchable. Once your issue is solved, a Discussion with the solution (and marked with Correct Answer) makes the results visible to others experiencing a similar problem. If a blog or document is related, put in a link. Read the Getting Started documents (link at the top right) including the Rules of Engagement.
NOTE: Getting the link is easy enough for both the author and Blog. Simply MouseOver the item, Right Click, and select Copy Shortcut. Paste it into your Discussion. You can also click on the url after pasting. Click on the A to expand the options and select T (on the right) to Auto-Title the url.
Thanks, Mike (Moderator)
SAP Technology RIG