4-Eyes-Principle vs. End-to-End: Different Testing Modes with Focused Build Test Steps
With Focused Build – Test Steps, designing and executing manual tests with SAP Solution Manager is simplified and accelerated. Many features like content-driven test case design, guided test execution or in-depth reporting capabilities enable a testing experience which was not possible before in SAP Solution Manager.
In my previous blog post I gave an overview on the most important capabilities & features of Focused Build – Test Steps. In this article I want to focus on a specific functionality of Test Steps that gives you the chance to execute manual tests in different testing modes. At design time you can already define the Testing Mode on header tab of your test case. You can choose between “Shared Test Results” and “Tester Individual Results”. But what does it actually mean when it comes to test execution?
Shared Test Results – a Testing Mode for End-to-End Test Cases
When it comes to large End-to-End Test Cases where you want to run a test across several processes, usually different organizational units are involved. For instance, in a typical Make-to-Order process units like Sales, Billing, Accounts Receivable and Production are taking part. While running the End-to-End test, these units need to share information such as Sales Order or Invoice IDs.
The testing mode “Shared Test Results” is the right choice in this case. Each step is performed in sequence (see option “Strict Step Sequence”), the different testers can document status and results in a single document. With mandatory result attributes you can even force testers to add information like Sales Order ID at specific steps. This ensures that other units which take over this test execution at a later point in time have all information available to proceed with the test.
To ease the execution of a large test case you can add the organizational unit to each step that belongs to this unit. In the column “Partner” you can find all business partners maintained in your SAP Solution Manager system. Please note: this is “information only” for the testers and does not imply any technical check during test execution.
After preparing the test case in Test Steps Designer you only have to assign it to a Test Package via Test Plan Management. As soon as the relevant organizational units are added to this Test Package the testers can start working on the test case.
Tester Individual Results – a Testing Mode for Four-Eyes-Principle Test Cases
In certain cases the availability, performance and reliability of a particular business process is of high importance for an organization. In such a situation you want to make sure that a positive test result is not a matter of coincidence but in fact reproducible independent from the individual tester. Especially for customers in regulated environments (e.g., FDA regulated), the so-called Four-Eyes-Principle plays an important role also in Test Management.
The basic idea is that four eyes see more than two eyes. When executing a test case, Tester A will work differently to create a Sales Order compared to Tester B. Hence, the chance is much higher to find a software defect if two different testers execute the very same test case individually.
With the testing mode “Tester Individual Results” you can exactly achieve that. The relevant test case and the testers need to be assigned to a Test Package. Afterwards each tester can work on a separate test execution without sharing any status or test results. When both testers have completed their tests the worst status wins and is automatically aggregated to the Test Plan level. That is, if Tester A completes the test successfully while Tester B finds a defect, the overall status is aggregated to error.
Combining both Testing Modes
Of course you could also end up in a situation where you want to run a large end-to-end test case in a four-eyes-principle approach. But how would you do this with Focused Build Test Steps when having only two different testing modes available?
In fact it is quite easy: in Test Steps Designer you create your end-to-end test case and set the testing mode to “Tester Individual Results”. In Test Plan Management you prepare your Test Package and assign the test case. The difference now is that you do not assign individual testers but groups of testers to this Test Package (see maintenance of business partners of type group). Each group consisting of testers from different organizational units can now perform an individual test execution of the very same test case and may come to different results.
With Focused Build Test Steps you have a powerful tool at hand which allows you to perform tests in different modes. Whether you have a large end-to-end process like Make-to-Order or a single functionality which is of significant importance for your upper management – the different Testing Modes help you to run the test in the right way.
Thank you for sharing this information.
I have a query though about this part "Each step is performed in sequence (see option “Strict Step Sequence”), the different testers can document status and results in a single document. "
As you know we can only assign testers at test case level but not at the test step level, how do we then control the execution of the test steps like which user need to execute which step in the sequence?
Currently we have various business end-to-end scenarios that require multiple test cases (with test steps) stringed into a sequence using the test sequence concept. In such cases, what is the best option to share some test notes or test results attributes to next tester without asking them to navigate into previous test cases, which consumes some time and also it requires the testers to be knowing upfront which test case they need to view for this data to use for their testing.
if you design an end-to-end test case you can use the "partner" column to describe which org unit or group should execute the referring step. During test execution testers can then see whether they should execute the step or not.
Please note: this partner attribute is "information only" and does not imply any auth. check or the like.
The alternative to that is the test sequence concept which you already described. We are aware of the usability limitations in terms of shared test results between test cases in a sequence and we will try to improve it in a future release.
Thank you for sharing this. It is very informative.
Aravind Kumar Jagannathan
Thanks Tobias for details,
Please confirm if we can have Approval process, eSignature, Export feature (test results having eSignature), Audit Trail ? at the time Test Case designing/ Test Execution/ Defect raising or resolution. Would really appreciate a quick response as we are deciding which Tool to be used for Test Management.
eSignature/Approval process is supported as follows:
I hope this answers your question?
Dear Tobias Meinzer ,
Thank you for the testing methods explained her using test steps. Very helpful.
We have a case where same test case (test steps) like QM Inspection is repeated multiple times within a test package. But, there is also a sequence to be followed. Currently, we end up creating multiple copies of the same test case - but this is not preferred.
Is there a recommendation how this can be handled via test steps without creating multiple copies of the same test case and yet be able to reuse the sam test case more than once in a test package with a sequence.
thank you for your positive feedback.
The recommendation in your case would be to create references to the original Test Case in Solution Documentation via context menu -> New -> Test Cases -> Test Steps (assign). These references don't have own attributes or content but just refer to the original test case. They can be used in Test Plans/Test Packages like the original test case itself.
Assuming you talk about an end-to-end testing use case, you could further create dedicated folders for end to end tests in Solution Documentation and organize the references there in the right sequence. You might also want to apply a meaningful naming convention to ease follow-up Test Plan Management and Test Execution activities.
I hope this helps?
Dear Tobias Meinzer
This again helped us a lot . Thank you very much 🙂
We were able to reference same test step based test case multiple times under the same business process and reuse more than once in a package.