SAP S/4HANA Cloud Test Automation Tool – Our Approach to Test Automation
Automation has been a buzzword in the domains of testing and process automation. It necessarily has not seen the widespread adoption which was predicted earlier due to multiple reasons. In this blog we will briefly cover some of the challenges faced in test automation, and how we plan to overcome these with the features being delivered as part of the SAP S/4HANA Cloud Test Automation Tool.
The expectation from automation solutions has long been – “plan, execute and forget”. The maintenance effort involved in running automation scripts as part of a constantly evolving software landscape is largely forgotten. Most implementations start off with automation as a central theme for testing strategy but give it up because the automates need to be reviewed with every software update. To truly realize the ROI from automation we need to bring down the maintenance effort and work towards being able to re-use automates across different test cycles.
The key factors for any automate to function consistently across software upgrade cycles are:
- Resilient: Automates which are based on non-static identifiers tend to fare worse across software releases and upgrades. Static identifiers ensure that the automates are more resilient to smaller changes
- Reuse: Automate design should ensure that we are able to reuse a single automate across multiple scenarios and multiple test cycles
- Test Data Consistency: Choosing the right set of data which works across multiple scenarios. This leads to the requirement of centralizing the distribution and maintenance of test data
- Understanding and actioning Errors: Not all errors are made equal, the ability to make the decision on which script error to prioritize and which to ignore is important as well
The SAP S/4HANA Cloud Test Automation Tool provides our SAP S/4HANA Cloud Essential customers the necessary feature set to create an automated regression suite for their testing needs. The tool is a no-code, no-installation, cloud native solution. The solution comes bundled with more than 300+ standard pre-delivered test automates based on the SAP Best Practice Processes. Customers generally use the pre-delivered automates as a starting point and adapt them as per their business needs. There is a built-in recording engine which enables end users to record their own automation scripts as well. Furthermore, test script executions are done in an SAP managed infrastructure which utilizes Hyperscalers and is able to scale with increasing demand.
The tool offers a very rich toolset, but still has to overcome the test automation challenges mentioned above. We have over the course of the past year focused on this aspect and introduced new features which make it easier to work with test scripts while making them more resilient to changes.
UI Control Repository
The first feature is the UI Control Repository, which achieves the modularization and reusability of UI Controls. The standard pre-delivered scripts are maintained by SAP, and hence any changes are updated and shipped as part of the software release. Customers utilizing the standard scripts did not have to review the scripts again, but customers using custom scripts still needed to review and incorporate these changes into their scripts. UI Control Repository moves the level of standardization from the script to individual UI controls (Button Click, Input field etc.), thus ensuring that custom scripts are more resilient to the changes due to the evolving software. More details about the UI Control Repository can be read here.
API Automation Framework
API Automation Framework provides the feature of creating automates based on both OData and SOAP API’s available in the SAP API Hub. It provides a smart and user-friendly interface to create API payloads for business users not very accustomed to working with API’s. While UI Automation ensures that we can test the functional correctness of the business process under test, API Automation provides the ability to test integration end points. API automation also enables running different test scenarios via a single script while making the necessary changes to the payload. There is also built-in integration between the API framework and Test Data Container application.
Both UI Control Repository and API Test Framework, are part of the broader approach we are taking towards building zero maintenance automates. Currently the features are enabled for Sales and Finance areas, with additional areas being included every release. More details on API Test Framework can be found here.
Test Data Container (TDC)
When we look for re-use of automates it is important to have a rich repository of test data to fall back on to test the various scenarios. This is where the Test Data Container (TDC) app comes into picture, it provides an easy and logical way of grouping relevant data for a test plan. TDC provides a central repository with standard test data templates based on business functional areas. Multiple data sets can then be assigned to a single TDC, which then as per need be used in the execution of multiple test plans.
Data templates aggregate all fields at an area level, and further logical classification is given based on the business flows. We also deliver Standard TDC’s which are grouped as per business scenarios. TDC provides the end users the capability to reflect data changes across multiple Test Plans by making a single change in the assigned data variants. More details on TDC can be found here.
Improved Error Reporting
The Analyze Automated Test Results (AATR) app now provides a “Failure Category” field which enables end users to quickly understand the reason for the script failure and action on them. The failure category, along with the failure reason and screenshots captured during the automate execution, provide a powerful tool to ascertain whether the error can be ignored or needs to be reported to development teams for further analysis. In most cases the user can themselves get the issue resolved by providing the right data or correcting configuration.
Moving ahead the reporting will be further enhanced with recommendations for corrective actions based on the “Failure Category”. This would enable users to quickly action on the script errors.
The above innovations enable our customers to get maximum value from their automated test regression suite while also bringing down the overall maintenance needed. Reduced maintenance is just one part of the automation tool, below are some additional facets of the tool which make it a compelling use case for all S/4HANA Cloud customers.
- Integration with SAP Cloud ALM solution
SAP Cloud ALM is our next generation lifecycle management tool which enables our customers to get a centralized view on their implementation working environment. The test tool is also integrated with the SAP Cloud ALM solution and provides a great way to execute and track progress of the automation suite.
- Post Upgrade Tests (PUT)
The test tool also enables SAP and customers to run post upgrade tests after every software upgrade release in the customer Q-tenants. PUT helps verify basic standard features by picking latest data, and executing scripts in Q-systems after SAP S/4HANA Cloud upgrade
- Pre-delivered SAP Best Practice Process Automates
The tool is also bundled with 300+ pre-delivered automates based on SAP Best Practice processes. This enables our customers to kick start their automation journey by using these automates as a starting point
The links below provide an overview of the feature enhancement made over the past releases:
- 2302 release highlights
- 2208 release highlights
- 2202 release highlights
- 2111 release highlights
- 2108 release highlights
- 2105 release highlights
All blogs related to the Test Automation Tool can be accessed here. We encourage you to make use of the customer community to ask any question relevant to the Test Automation Tool.
The test automation community page on SAP.com is live, visit the page to get the latest updates and upcoming sessions in the Test Automation space.
Thanks for the blog, Anurag. This is very helpful. Can you please advise if SAP S/4 HANA Cloud Test automation can be used to perform testing post upgrade impacts related to SAP Successfactors?
Hi Sriramakrishna Rekapalli ,
The tool currently is only available within the S/4HANA Public cloud solution.
Is it possible to use the Test Automation Tool with S/4HANA on-premise?
If not, will it be made available for on-premise in future or is there another strategy for that?
Hi Jack Alan Reynolds ,
The test automation tool currently is only available with the S/4HANA Cloud Public solution.
yes this is the main entry point to do it onPrem in the most cases.
Please see further notices here: https://support.sap.com/en/alm/partners/test-automation.html
If we have copied a standard script and added few steps to add our own custom fields and in future if the standard script has changed by SAP in a release upgrade with some additional standard fields, do we need to adjust our custom script or it will it adopted automatically from the standard?
Hi Karanam Guru ,
No changes done in a standard process will flow into a custom process. The updates in the standard process will have to be evaluated and as per requirement changes done to the custom script.
There is one exception though, if you are still using fields which are part of the UI Control Repository, then any changes in the field properties updated by SAP will also reflect in the custom script.