Advanced Test Orchestration using Test Plans in SAP Cloud ALM
In this blog post I would like to provide some details about using test plans for advanced test orchestration in SAP Cloud ALM. The blog will address the following questions:
- When do you need test plans in SAP Cloud ALM?
- How to use test plans in SAP Cloud ALM?
1. When do you need test plans in SAP Cloud ALM ?
In larger implementation or transformation projects you might want to run your test activities in iterations and so need setup multiple test cycles re-using the same or a similar set of test cases.
On top, you might want to roll-out the implemented solution or a global template to multiple organizations like subsidiaries, plants or markets which require dedicated tests per roll-out. Thus, you might want to re-use most of the test cases.
Although it was possible without test plans to run the same test case again while creating new test runs, the overall test status was always set to the status of the last test run.
With the following illustrations I would like to explain the underlying concept with the existing approach without test plans and the new approach while using test plans:
1.1. Lean Test Orchestration Without Test Plans
In the following example, the first test run was set to ‘Passed’ after all test actions were completed successfully and the test run was set to ‘Finished’. Then a second test run was started and test action 1.1 was set to ‘Fail’. In consequence, the overall test status is displayed as ‘Failed’.
After introducing test plans, this lean approach is still available, so you could continue without any overhead. In most cases, this is sufficient for smaller projects and public cloud implementations with an agile or lean implementation approach. Therefore, we still support executing test cases without the need for test plans.
1.2. Advanced Test Orchestration with Test Plans
For more complex projects, customers asked for the possibility to have a dedicated execution context for each test cycle.
To achieve this, you could create a test plan and assign relevant test cases
As the test plan status is still ‘In Preparation’, no occurrences of the assigned test cases are displayed in the Test Execution app yet.
After switching the test plan status to ‘In Testing’, all assigned test cases in status ‘Prepared’ get a new occurrence (execution context) and the test execution can be started.
The next picture shows the situation after Test Case 3 has been set to ‘Prepared’ and Test Plan B has been set to ‘In Testing’.
How to use test plans in SAP Cloud ALM?
New Test Plan App
There is a new tile in the launch pad called Test Plans:
When you open the Test Plan app, you will see a list of existing test plans with their status, time frame, and the aggregated preparation status and execution status of assigned test cases:
You can create new test plans choosing ‘Create’ to get this detail screen in draft mode:
To assign test cases choose ‘Assign Test Cases’ to get this simple assignment dialog:
You can either start typing in the field to get a prefiltered list of test cases, or you could open the value help to get an advanced assignment dialog with search and filter options:
After confirming the selection and the assignment dialog, you will come back to the test plan details in draft mode, showing the assigned test cases:
In draft mode, the test plan is stored but not yet available to work with. Only when you choose ‘Create’, there will be a ‘saved version’ that can be used for processing.
Whenever you open the test plan in edit mode, a draft version is created, which contains all your changes without saving. You can switch between the draft and saved version of your test plans next to the test plan title:
You could even close your browser without losing your changes.You can find more details on draft handling here.
Besides the test plan statuses ‘In Preparation’ and ‘In Testing’, there is a status called ‘Finished’, which indicates that the test phase / test cycle is finished. This status does not have any impact on the visibility of related test cases occurrences in the Test Execution app.
Enhancements in the Test Execution App
In the Test Execution app, you will find a new filter for test plans and the occurrences of test cases are grouped by test plans.
Without setting the filter for test plans, you will see at least one entry per prepared test case belonging to the group ‘(None)’ . In addition, you’ll get another entry for each test case per test plan that is in status ‘In Testing’ or ‘Finished’.
When planning to work with test plans, you could simplify the entry for your testers by setting the filter for the right test plan(s) and then sharing the URL.
Enhancements in the Test Preparation App
In the Test Preparation app, we have also added the filter for test plans to be able to reduce the list of test cases of the whole project to test cases that are not assigned yet to any test plan, or to list all test cases already assigned to a specific test plan.
Outlook on Planned Enhancements*
We plan to enhance the test plan capabilities in several iterations. In one of the next iterations we plan* to provide the following enhancements:
Filter on test plans in following dashboards:
- Test Execution Analysis and Defects Reporting
- Solution Process Traceability
- Requirements Tracebility
- User Story Tracebility
- Test-related cards in Overview app
Test Case Assignment to Test Plans
Advanced test case selection
- with additional filters in the Test Plan app
- by levaraging the Traceability apps to select related test cases for assignment to a test plan
Assign testers to test cases
- Default assignment without test plan
- Dedicated assignment per test plan
Copy of Test Plans
Define the sequences of test cases within a test plan
Roadmap for Further Planned Innovations:
You can find the latest updates on our planning in the Roadmap Explorer for SAP Cloud ALM
* The information in this presentation is confidential and proprietary to SAP and may not be disclosed without the permission of SAP. Except for your obligation to protect confidential information, this presentation is not subject to your license agreement or any other service or subscription agreement with SAP. SAP has no obligation to pursue any course of business outlined in this presentation or any related document, or to develop or release any functionality mentioned therein. This presentation, or any related document and SAP’s strategy and possible future developments, products and or platforms directions and functionality are all subject to change and may be changed by SAP at any time for any reason without notice. The information in this presentation is not a commitment, promise or legal obligation to deliver any material, code or functionality. This presentation is provided without a warranty of any kind, either express or implied, including but not limited to, the implied warranties of merchantability, fitness for a particular purpose, or non-infringement. This presentation is for informational purposes and may not be incorporated into a contract. SAP assumes no responsibility for errors or omissions in this presentation, except if such damages were caused by SAP’s intentional or gross negligence. All forward-looking statements are subject to various risks and uncertainties that could cause actual results to differ materially from expectations. Readers are cautioned not to place undue reliance on these forward-looking statements, which speak only as of their dates, and they should not be relied upon in making purchasing decisions.