eCatts is a tool which is used for testing an ABAP application. I am sure every ABAP developer would have used or atleast heard of this tool. But when it comes to a non-ABAP application in the SAP World, the tools we have for automated testing are very few or less commonly known. This makes us wonder - why is it so?
There are many pros and cons for both manual testing as well as automated testing. In most cases, we often find that in the initial stages of development of a product, manual test is done first (to identify the bugs) and then the automated test is done. But when the product matures or is no longer in the development phase, it will be the automated testing first and manual test later (if required). But there is a certain debate about whether we should automate testing at all (especially in the non-ABAP application world).
The following are some of the advantages of manual and automated testing -
Manual testing
1. Flexibility - especially when it comes to data issues. For example, if a particular set of master data does not work, with manual testing the tester can change the master data and check.
2. Ability to identify a "real" bug - with manual testing the tester can immediately identify whether something is not working because it is a bug or whether it is just a temporary problem (maybe connection was lost temporarily)
Automated testing
1. Ease of repitive testing - automated testing definitely has an advantage in cases where the tests have to conducted at regular intervals of time (for example, a daily check on an application).
2. Eliminates human "carelessness" - automated script checks each and every step which needs to be tested. There is no chance of a human error (for example, a certain step being overlooked)
If testing of a certain non-ABAP application is being automated, there are some questions which immediately pop up. The questions could be -
1. How long does it to automate a test scenario? What is the effort involved?
The effort to prepare automation scripts should be reasonably less. Otherwise the effort spent in creating automation scripts might turn out to be greater than the effort required for manual testing. From a pure effort perspective, automation could become a pain. This depends to a large extent on the automation tool and how easy it is to use the tool.
2. How frequently is the testing done?
Some applications are tested daily and some are tested less frequently. It makes more sense to automate testing of an application wherever the frequence of testing is very high. It reduces the manual effort a lot. Also, in a majority of cases, if an application is being tested very frequently, then it means that it is a very critical application and it is better to avoid human error in testing as much as possible by resorting to automation.
3. What is the test coverage expected?
It is obvious that automation can test more lines of code than manual testing in the same amount of time. But one has to check whether it is really required to have a tremendously high code coverage. There could be certain functionalities which are very rarely used or used only once (maybe during the setting up of the system). There is no point in automating such functionalities. A compromise has to be made between code coverage by an automation tool and the importance of functionality to be tested.
4. How adaptable is it?
This question basically arises from the fact that there might be multiple test systems in the landscape (for example, multiple test Portals) and the master data in these Portals could be different. In this scenario, it should be possible to use the same script to test all the test Portals and the option to change the master data easily and with minimal effort should be available (like using placeholders in the script instead of actual master data)
I think the decision to automate testing of a non-ABAP application or not has to be taken purely in the context of that application. There cannot be a general set of rules which specifies that manual testing is better or automated testing is better. It is this aspect which really makes any discussion on the topic "Manual vs Automated Testing" very interesting.
