With a proliferation of (smart) mobile devices, today’s work place is allowing more and more people to bring their own devices to work (Bring Your Own Device). This naturally means companies have to manage a multitude of devices leading to what is now known as Mobile Device Management. However, it does not encompass everything there is about managing mobile devices. After all, device management at end of the day is mostly about procuring and managing the device. Enterprises need to look beyond the hardware aspect. In her blog SAP’s Milja Gillespie is calling this Enterprise Mobility Management. Milja notes that Enterprise mobility management is about the following-
- Mobile Inventory/Asset Management
- Mobile Application Management
- Mobile Security
- Wireless Expense Management
- Mobile Operations Management
- Mobile Help Desk
I envision that within Mobile Application Management, Testing and Validation will play a bigger role than it is currently. Companies are aware of the seismic shift that is happening in mobility arena but are not prepared for it to the full extent. According to current research by year 2012, companies will have, by a factor of 4 to 1 more mobility projects than the desktop variety. I think there is a gap when it comes to testing applications for mobile (regardless of the technical platform) – think about it this way– conventional testing methodologies rely upon harnessing the Graphic User Interface (GUI) for traditional applications (Windows, Mainframe, Portals) to “validate” the delta between desired and actual outcome. With the mobile devices what we have instead is Natural User Interface (NUI). If you have used Xbox Kinect (it uses spatial gestures instead of a hardware device to interact with the system), you would have a fairly good example of what NUI is. You swipe, You pinch and zoom, you tap, you get the idea.
Given the revolutionary way we interact with our mobile devices, any strategy we adopt to test the applications for mobile must also take into account these differences. Given below are some challenges that will have to be overcome when instituting a test harness for mobile applications-
What I just described represents challenges when creating a test plan or strategy for a Device Under Test (DUT). What comes next? Now that you have the BIG picture, the strategy taken care of, now is the time to focus on operational and tactical aspects of your test plan. Any test plan that aims to verify the integrity of a mobile device application should have a repository of test cases, design steps and scenarios that will rigorously put the application to work in below mentioned areas.
- Installation, Start up time – This should be a set of tests that ensure the app can be installed over the air (OTA). If the application is too large, it should notify the user that excessive data consumption could be possible leading to surprises in monthly mobile bill.
- Memory used – Memory consumption by the app should be measured. Ensure that memory is not used excessively, and that it does not lead to corrupt memory locations.
- User interface – This should be an array of tests to validate several aspects of user interaction with the interface. Ease of use, readability, consistency are some of the flavors you would include ion your test plan.
- Connections – Test to make sure app can function under variety of connectivity situations. It can handle broken connections, roaming etc.
- Performance – Resource consumption. Execute test cases to determine load factor and other metrics that ensure CPU utilization is minimal.
- Functionality – ensure that the app does not interfere with calls and other media messages.
- Media – test how the app functions with other settings tweaked. For instance, if you app is a media player, make sure it stops playing media when a headphone is ejected from the device.
- Data related – Run tests to make sure app handles data in consistent and safe manner. Check for insertion of new records, data deletion etc. Ensure you can clear temporary files and logs when indicated by the user.
- Security – Test for encryption, passwords. Make sure your app does not expose the device to security vulnerabilities.
I will conclude by noting that this field is in its infancy and there are miles to go before application testing for mobile reaches same maturity levels as testing processes for other mainstream platforms. There are new and exciting initiatives in this field and I have explored various vendors making testing software for mobile applications testing- many of these tools are building on VBA model or leveraging existing solutions such as QTP. It would be very interesting to hear readers thoughts and experiences how their companies are meeting requirements for mobile application testing.