Several colleagues asked me what I think about Selenium. Selenium has, similar to many tools, advantages and disadvantages and any advice can only be given context specific.
Jonathan Benn has compiled the following list of good reasons for using selenium for testing in an internal blog:
- Selenium is an excellent library for UI testing. It's succinct, easy to use, well-thought-out and reliable.
- Assuming that the UI is easily testable, meaning that every UI element has a unique ID, tests are easy to write and maintain, and are reliable.
- When the UI is less testable, e.g. because UI elements have repeated IDs or IDs that change over time, Selenium still makes it possible to test. Development time increases, and reliability and maintainability decrease, but at least it's still possible to test.
- Thanks to its simulation of user interaction, Selenium enables a complete top-to-bottom integration test of the app. After running a Selenium test on a feature, you can feel very confident that the feature works as prescribed and that users will not run into problems.
- Similarly, Selenium offers a lot of support during refactoring, since it's easy to tell if you broke any part of the app while making changes.
- If you stick to testing the major use-cases of the app, Selenium tests will not require a lot of maintenance and will provide very good value for money/time spent.
These are all valid reasons for using selenium, especially for end-to-end testing scenarios. However, the authors of How Google Tests Software state: “But overinvesting in end-to-end test automation often ties you to a product’s specific design and isn’t particularly useful until the entire product is built and in stable form. And it’s still browser testing. It’s slow, you need hooks in the API, and tests are pretty remote from the thing being tested.” [How Google Tests Software]. Furthermore we’ve observed that it’s rather inefficient if there are people that write only tests while others are doing the coding without eating their own dog food. Therefore we thought about an alternative, avoiding these disadvantages.
Testing apps with JavaScript based tools, on the other hand, has benefits and this brings OPA5 into play: OPA5 is an extension for SAPUI5 controls developed by Tobias Oetzel. It hides asynchronity and eases access to SAPUI5 elements. This makes OPA5 especially helpful for testing user interactions, integration with SAPUI5, navigation, and data binding:
The OPA5 library is JavaScript based, meaning that you can write your tests in the same language in which your app is written. This has the following advantages:
- Quick and easy access to JavaScript functions
- Easy ramp-up as it can be used with any JavaScript unit test framework (e.g. QUnit, Jasmine)
- For Jasmine the adapter is currently missing. If interested you can contribute on the GitHub project. The implemetation should be straightforward.
- Using the same runtime enables debugging
- Good SAPUI5 integration
- Feedback within seconds makes it possible to execute tests directly after a change
- Asynchronity is handled with polling instead of timeouts, which makes it faster
- Enables test-driven development (TDD)
- Easy TestDouble Usage: Can be combined with MockServer to isolate from the backend. This is also the recommended way to test your app, to avoid fragile test data and fragile backend systems.
Developer writes tests along with developing the app, thus ensuring from the beginning that the app is testable. This is important from a testability point of view, whereas having people writing tests while others do the coding without writing automatic tests is rather inefficient. TDD results in less fragile tests, because the app is better isolated and supports less fragile APIs for testing.
- Follows the arrange act assert pattern (corresponds to given when then), which improves readability and understanding of the test cases
- Easy to run on mobile devices, no plugins/apps are needed but you can just run it in the browser.
- Saves time for the developer since regressions decrease
It is very easy to write acceptance tests with OPA5 – Give it a try! It is already used by many developers inside of SAP.
What you currently should not or cannot do with OPA5:
- Screen capturing
- Logging console errors
- Testing across more apps
- Remote test execution
How Google Tests Software, James Whittaker, Jason Arbin, Jeff Carollo, 2014
The documentation visible outside of SAP will be improve, so stay tuned. https://openui5beta.hana.ondemand.com/docs/guide/22f175e7084247bc896c15280af9d1dc.html
MockServer: https://help.sap.com/saphelp_uiaddon10/helpdata/en/41/5d1fc8cc154cb189809d3cb9747165/content.htm
Arrange Act Assert and Given When Then:
http://martinfowler.com/bliki/GivenWhenThen.html
http://xp123.com/articles/3a-arrange-act-assert/
