Skip to Content

In this article I will try to compare provided tools for SAP applications testing. This analisys were made for one of SAP customers.

There are three known options for SAP products testing:

  1.       Using SAP Solution Manager Test Workbench
  2.       Using HP tools (SAP Quality Center by HP)
  3.       Using IBM Rational tools

At the same time, much attention is paid to testing automation which helps to solve such problems as:

  • A large number of the tested business processes
  • High labor input and cost of manual testing
  • A human factor when testing
  • Availability of necessary human resources for carrying out testing
  • Complexity of scoping of testing
  • Increase in complexity of testing in case of increase in number of business processes

However in the automated testing not everything is so simple.

High requirements of qualification of employees during creation of test scripts are imposed. It is also rather difficult to support the existing scripts in an actual condition in case of frequent change of business processes.

For the solution of similar problems there was developed so-called “Component-Based testing”

Component-Based testing of business processes allows to simplify creation and support of the automated test scripts for the users who are responsible for testing.

  1. Instead of working directly with a script, the tester can build the scenario, using ready components of the test, each of them represents the written-down script.
  2. The instrument of component testing scans the application and generates the automated script consisting of a set of components of the test.
  3. This script can be used further for testing. Testers can independently carry out such scanning since knowledge of programming of scripts isn’t required.
  4. The generated scenario can be united in an independent business component which can be used further for creation of other scripts

Time required for manual testing are the greatest for an overall picture of testing. However it is the most economic option for the scenarios tested once or very rare. Despite a possibility of reuse of the written-down automated scenarios, costs for their support are big since scripts improve manually. The smallest labor costs on support are required for component testing since there is a possibility of repeated scanning of the tested process without their rewriting.

The following evaluation criteria have been developed for further assessment, taking into account the aforesaid

Functional capabilities

Name of evaluation criterion

Description of evaluation criterion

Test Plans generation

Possibility of drawing up plans of testing on the basis of the generated scripts. Possibility of use by the subsequent scripts of results previous.

Existence of the tool for record of a test script

Existence of mechanisms for recording of the user actions  made in system with a possibility to use this record at the subsequent testing.

Existence of additional tools of component testing for optimization of works on generation and support of a test repository

Availability of additional instruments of record of the user actions facilitating initial generation of scripts with a possibility of their automated updating in case of change of the corresponding business process, for example SAP TAO

Possibility of the automated testing of ABAP developments

Support of test automation of ABAP developments

Possibility of the automated testing of Java developments

Support of automation of Java developments for the SAP applications

Possibility of the automated testing of developments of SAP Webdynpro for Java

Support of automation of testing of developments of Webdynpro for Java

Возможность автоматизированного тестирования разработок SAP Webdynpro for ABAP

Possibility of the automated testing of developments of SAP Webdynpro for ABAP

Possibility of the automated testing of developments of SAP CRM Web UI

Support of automation of testing of developments of Web UI

Analysis and Reporting

Possibility of generation of reports on testing and simplicity of their generation

Support of regular load testing of SAP systems

Possibility of regular carrying out and comparison of several load tests on the chosen system in a uniform window with comparison of trends on the executed transactions

Possibility of automation of testing of mobile platforms for SAP systems

Possibility of automation of functional regression testing of mobile applications on various devices

Existence of a possibility of adjustment of loading at load testing

Adjustment of loading when conducting load testing

Integration with application lifecycle management processes

Name of evaluation criterion

Description of evaluation criterion

Integration with Project Documentation

Support of business processes structure. Possibility of transfer of structure of the project from SAP Solution Manager. Updating of structure of the SAP Solution Manager project based on results of accomplishment of scripts. Determination of a covering test scripts of specific elements of structure of business processes

Integration with SAP Solution Manager ITSM scenarios

Possibility of maintaining errors of testing in Solution Manager. Existence of automatic communication of an incident and test unit

Integration with Change Management of SAP SM

Possibility of use of the tool for testing of change operations of Solution Manager in a test landscape. Transfer of test results in change operation

Integration with BPCA

Possibility to use of results of work of SAP Solution Manager Business Process Change Analyser (BPCA) intended for the analysis of changes in system with the subsequent generation and optimization of test plans

Ease of use

Name of evaluation criterion

Description of evaluation criterion

Simplicity of a script recording

Efforts on initial record of a script

Simplicity of debugging of a script

Efforts for debugging of a script (adding of breakpoints, computing logic and etc)

Simplicity of updating of a script

Efforts for updating of a script in case of change of business process in change transaction

Simplicity of support of input data

Complexity of definition/updatingof input data of a script

Usability

Simplicity of the interface and its clearness for the ultimate user

Simplicity and convenience of administration

Simplicity and convenience of administration

Existence of the built-in means of the organization of a role model

Existence of model of differentiations of powers in a product.

On the basis of the criteria stated above, assessment of the offered tools has been carried out. Prioritezation of criteria has been executed by a customer on the basis of his requirements from 1 – not important, to 5 – very important. Opportunity/extent of realization can be expressed or as parameter realizable/unrealizable (1/0) or as extent of realization from 1 to 5

Functional capabilities analysis

Name of evaluation criterion

Priority

SAP SM

SAP QC by HP

IBM Rational

Test Plans generation

4

1

1

1

Existence of the tool for record of a test script

4

1

1

1

Existence of additional tools of component testing for optimization of works on generation and support of a test repository

4

1

1

0

Possibility of the automated testing of ABAP developments

5

1

1

1

Possibility of the automated testing of Java developments

5

1

1

1

Possibility of the automated testing of developments of SAP Webdynpro for Java

5

1

1

1

Возможность автоматизированного тестирования разработок SAP Webdynpro for ABAP

5

1

1

1

Possibility of the automated testing of developments of SAP CRM Web UI

5

1

1

1

Analysis and Reporting

5

1

1

1

Support of regular load testing of SAP systems

5

0

1*

1*

Possibility of automation of testing of mobile platforms for SAP systems

5

1

1

0

Existence of a possibility of adjustment of loading at load testing

5

0

1

1


Integration with application lifecycle management processes analisys

Name of evaluation criterion

Priority

SAP SM

SAP QC by HP

IBM Rational

Integration with Project Documentation

4

1

1

1

Integration with SAP Solution Manager ITSM scenarios

3

1

1

1

Integration with Change Management of SAP SM

3

1

0

0

Integration with BPCA

4

1

1

1

Ease of use analysis

Name of evaluation criterion

Priority

SAP SM

SAP QC by HP

IBM Rational

Simplicity of a script recording

5

5

5

5

Simplicity of debugging of a script

4

5

5

3

Simplicity of updating of a script

3

5

5

3

Simplicity of support of input data

4

5

5

4

Usability

5

5

5

5

Simplicity and convenience of administration

3

5

5

5

Existence of the built-in means of the organization of a role model

4

5

1

1

Tool cost

5

5

3

4


Total:

• On the SAP Solution Manager 226 points

• On the HP 207 points

• On the IBM Rational 185 points

By results of assessment the SAP Solution Manager tools and HP tools are approximately identical by opportunities. Besides it is possible to carry to pluses of HP that this tool is long enough used in the market and is suitable for testing not only the SAP applications. But the SAP Solution Manager tools are free in the presence for Enterprise Support clients.

Further we will in detail consider the offered tools

To report this post you need to login first.

Be the first to leave a comment

You must be Logged on to comment or reply to a post.

Leave a Reply