In this blog-post i wanted to point out, why and when we need manual and automated testing. Also automated testing is getting more and more popular, manual testing has still some benefits, which are hard to achieve with automated testing.
The benefits of manual testing
Automated tests are build with determinism in mind. So they are mostly static without the random events and states, that occur in the real world. The dependencies are replaced by test-doubles, which don't necessarly mock all the states and events, that occur in the real dependencies.
Let's take an example. Recently i had to develop an application, that books consumption quantities, surcharges and production quantities related to a process order. After each booking step, the SAP-LUW needs to be closed due to the restrictions for booking material documents. If one step failed, the process could be repeated without booking the consumption or the surcharges twice. My solution was to set the withdrawn sign in the reservation and only book the consumption when this withdrawn sign wasn't set. When writing the automated test, i created process orders and used the real dependencies (BAPI_GOODSMVT_CREATE) to book the consumption and the production. Every test-case showed green, but in production it fails. What was the reason? The process orders were confirmed before the consumption and the production was booked. The confirmation has set the withdrawn sign. So the consumption step was never booked in the production environment. Why we didn't found that bug in the tests? Because the tests were static and we didn't consider the events like the confirmation, that happened on the process order in the production environment.
Through random tests it's possible to discover bugs, which slipped through the automated tests like in the example above. Manual testing brings new variations and helps to randomize the tests.
Why automated tests are still needed
Although determinism is some kind of drawback, when it comes to new variations, it's a benefit on the other site. With determinism it's possible to execute the test-cases whenever needed, but they hardly have any variation. So it's very important to design the test-doubles and the database entities used for testing accurate, that they act like the real dependencies.
