A key component to any project implementation is the testing aspect to ensure all test cases, plans, and other key areas are working effectively and as expected during multiple phases of the project. In the SAP Activate Roadmap Viewer implementation roadmaps, we have a designated section for the Testing workstream. Let's take a look at SAP S/4HANA Cloud implementation roadmap and take a deeper dive into the Testing workstream. Keeping in mind, the Testing workstream is an essential workstream that is covered in all solutions - not just SAP S/4HANA Cloud.
Although SAP Activate spans from Discover through Run with many phases in between, we start the Testing workstream in the second phase: the Prepare phase. This is because we start the enablement of the testing team to understand which testing tool(s) need to be utilized and how to best utilize them during their project deployment. The Self-Enablement on Test Management and Tools task has accelerators ready for the project team which helps accelerate the learning on testing tools with tutorials and implementation guidance. The team has the option to use tools such as SAP Cloud ALM which has been recently introduced, the Test Automation Tool (built into the solution), and any other testing tools which are desired.
Note: SAP strongly recommends using SAP Cloud ALM and the Test Automation Tool for all testing activities. Any third-part testing tools are not supported by SAP but can be used if desired.
Within the Explore phase, we dive more into the test planning for the project. We are now looking more closely to set up and create our strategy, and test plan up for the team. Keep in mind, the more preparation and focus which is dedicated in this phase of the project, will yield better results overall since the processes have been thought through and documented.
Outlining the strategy, approach, roles and responsibilities, testing tools, defect management, and other topics in detail and how the testing activities will be executed and by whom, sets the project up for success. This step is very important as it defines how testing will be performed and completed in the project. All project members and stakeholders must agree and align to the agreed approach before testing starts.
Creating and scheduling the test plan requires defining the test scope and getting sign off from all parties (including the customer, project implementation teams, business process experts, and stakeholders). If using SAP Cloud ALM, this can be managed by creating the project scope and adding the relevant business processes to it.
Now that we have gone through the planning and documentation, this brings us to the Realize phase of the project.
In this phase, we have the bulk of the testing activities. We dive into the test preparation activity which focuses in on ensuring the availability of the testers and perform knowledge transfers to get everyone up to speed. In addition, the testing tool(s) (including third party automated testing tools - if desired) are setup and a kickoff meeting occurs. All users are set up in the testing tools at this point with the respective business roles assigned to perform testing activities.
End-to-end business flows are defined by reconfirming or adapting the best practice process flows (SAP Best Practices). SAP delivers the best practice business process flows which are available via the SAP Best Practices Explorer. Defining the test data for the project is done at this point. A test process can be tested with several test data variants. Where possible, we recommend that the master data within your test data be data sets which are also available in your productive environment (e.g. material master). You may wonder why regression testing is found at this stage in the Realize phase - the reason why is releases will most typically occur during the implementation timeline and testing those new enhancements is needed.
Additionally, preparing the automated test scripts in the Test Automation Tool is performed. Standard automated test scripts are delivered (which cover many of the SAP Best Practices processes) and should be reused as much as possible or by adaptation to the business processes. Within the Test Automation Tool, custom test cases can be created using the recording functionality where steps are captured. Keeping in mind, staying to the standard is recommended versus creating manual test cases. Manual test cases should only be created if the business process does not yet exist for an automated test case. Then, the test data will have to be defined for this manual test case. If third party automated test tools are used, any necessary automated test cases are created.
Lastly, under the Test Preparation for SAP S/4HANA Cloud deliverable, edit action data and change visibility of test processes is done. This entails modifying actions in customer specific automated test scripts, and to manage the visibility of a test case in the Manage Your Test Processes app.
Now, moving to the test execution activities, we create the test plan by including the standard and custom test processes in the sequence that defines the end-to-end flow. Test data using the data variant is changed, as needed. Then, the test plan which was established earlier is executed where the statuses are displayed in the form of "success," "with errors," and "failed" along with a detailed log and screenshots of each step. The first “failed” test process step in the test plan will have one “failed” action followed by “cancelled” actions when an action fails, the following actions do not execute, hence their statuses become “cancelled.”
Any errors which were identified in the test execution, need to be reviewed along with all corresponding documentation which was captured in the log. Defect management is critical during this stage to resolve the identified test issues. The testing team should resolve all failed actions and fix them per the detailed action log and re-run the test plan to ensure they are run successfully. Then, test execution analysis and reporting is performed to generate, review, and evaluate the test results report utilizing the Test Result Dashboard. The Executed Test Processes report is available on a bar chart, and the Post Upgrade Test Plans and Customer Test Plans reports are available on an interactive visual report showing the various test statuses.
The next phase in SAP Activate is the Deploy phase. However, Testing activities are not performed in this phase. We move to the final phase - the Run phase to close out the Testing workstream.
The Testing workstream brings us to the Run phase at this point in SAP Activate and your project implementation.
Preparing the Testing team for the solution upgrades helps to ensure new solution functionality is executed smoothly for the business processes. Project teams also need to be aware of the upgrade dates as they are not flexible and cannot be shifted. The upgrade for the Q-System will be first, followed by the P-System two weeks later. During the time between the Q and P upgrades, regression testing occurs. You can refer to the SAP S/4HANA Release/Upgrade Schedule and Prepare for Quarterly Upgrade task for additional details. Additionally, the Testing team will prepare for post upgrade testing (PUT) by SAP (relevant only for customers that are live in the production system).
The Post Upgrade Tests are smoke tests performed by SAP and does not replace the Regression Testing activity. Regression Testing is the responsibility of the customer, and the customer needs to test their solution, based on custom processes and custom test data. Consent needs to be provided for SAP for the PUT at least two weeks prior to the upgrade - this can be done directly in the tool.
The PUT scripts are standard scripts delivered by SAP. The scripts are built with data picking logic which picks the latest transactional data needed to run those test scripts in the Quality system. The customer needs to ensure that the prerequisites are met. All PUT results can be accessed and viewed within the Analyze Your Test Result app along with the respective logs and screenshots for the test scripts.
Testing teams should check, confirm, and/or create master and transactional data in the system. Additionally, the maintenance of test user credentials and applicable roles needs to be reviewed and updated as needed to ensure team members have the correct roles and access to their profiles to execute the testing activities during the project. Test users in the solution need to review and update their password whenever the password is changed or expired.
The testing team needs to perform and view the post upgrade test results and statuses in the solution and all logs/screenshots (i.e. failed, success, in process, and cancelled). Tests which need to be re-executed due to failing must be re-executed.
As performed in the Realize phase, executing regression testing in the Quality System is done to make sure the functionality works as expected after the quarterly release upgrade. The Test Automation Tool is strongly recommended however, other third party test automation tools may be used. Note, SAP does not manage any third party testing tools and processes.
Lastly, the final testing activity of test execution analysis and reporting is performed again (first performed in the Realize phase) to generate, review, and evaluate the test results report utilizing the Test Result Dashboard.
Additionally, check out the additional resources and content below to expand your knowledge on SAP Activate and SAP S/4HANA:
Inside SAP S/4HANA Podcast
As always, I would like to encourage you to subscribe to our new podcast “Inside SAP S/4HANA.” This podcast leverages the unique knowledge and expertise of SAP S/4HANA product experts, partners and customers to address your needs by sharing product insights and project best practice. There is no customer success without product success and project success; we help you get to the next level and make your SAP S/4HANA projects a success. Subscribe now and benefit from the shared knowledge.
openSAP Microlearnings for SAP S/4HANA
Just a couple of months ago, we launched openSAP microlearnings for SAP S/4HANA as an exciting new learning opportunity. What began with a small batch of 20 videos, has now become a channel with more than 50 microlearnings that have generated over 20,000 views since then. Today we cover multiple lines of business, like Finance and Manufacturing & Supply Chain, and key technology topics like Master Data Management, Extensibility, User Experience, and Upgrade Management. We are continuously adding new microlearnings to the SAP S/4HANA channel, so make sure you check them out.
Your voice matters!
If you want to learn more and actively engage with SAP subject matter experts on SAP S/4HANA Cloud, join our SAP S/4HANA Cloud Customer Community. This platform which is available to SAP S/4HANA Cloud customers and partners has a clear mission: deliver an interactive community to engage with one another about best practices and product solutions.
For more information on SAP S/4HANA Cloud, check out the following links:
Follow us via @Sap and #SAPS4HANAValueSeries
Although SAP Activate spans from Discover through Run with many phases in between, we start the Testing workstream in the second phase: the Prepare phase. This is because we start the enablement of the testing team to understand which testing tool(s) need to be utilized and how to best utilize them during their project deployment. The Self-Enablement on Test Management and Tools task has accelerators ready for the project team which helps accelerate the learning on testing tools with tutorials and implementation guidance. The team has the option to use tools such as SAP Cloud ALM which has been recently introduced, the Test Automation Tool (built into the solution), and any other testing tools which are desired.
Note: SAP strongly recommends using SAP Cloud ALM and the Test Automation Tool for all testing activities. Any third-part testing tools are not supported by SAP but can be used if desired.
Within the Explore phase, we dive more into the test planning for the project. We are now looking more closely to set up and create our strategy, and test plan up for the team. Keep in mind, the more preparation and focus which is dedicated in this phase of the project, will yield better results overall since the processes have been thought through and documented.
Outlining the strategy, approach, roles and responsibilities, testing tools, defect management, and other topics in detail and how the testing activities will be executed and by whom, sets the project up for success. This step is very important as it defines how testing will be performed and completed in the project. All project members and stakeholders must agree and align to the agreed approach before testing starts.
Creating and scheduling the test plan requires defining the test scope and getting sign off from all parties (including the customer, project implementation teams, business process experts, and stakeholders). If using SAP Cloud ALM, this can be managed by creating the project scope and adding the relevant business processes to it.
Now that we have gone through the planning and documentation, this brings us to the Realize phase of the project.
In this phase, we have the bulk of the testing activities. We dive into the test preparation activity which focuses in on ensuring the availability of the testers and perform knowledge transfers to get everyone up to speed. In addition, the testing tool(s) (including third party automated testing tools - if desired) are setup and a kickoff meeting occurs. All users are set up in the testing tools at this point with the respective business roles assigned to perform testing activities.
End-to-end business flows are defined by reconfirming or adapting the best practice process flows (SAP Best Practices). SAP delivers the best practice business process flows which are available via the SAP Best Practices Explorer. Defining the test data for the project is done at this point. A test process can be tested with several test data variants. Where possible, we recommend that the master data within your test data be data sets which are also available in your productive environment (e.g. material master). You may wonder why regression testing is found at this stage in the Realize phase - the reason why is releases will most typically occur during the implementation timeline and testing those new enhancements is needed.
Additionally, preparing the automated test scripts in the Test Automation Tool is performed. Standard automated test scripts are delivered (which cover many of the SAP Best Practices processes) and should be reused as much as possible or by adaptation to the business processes. Within the Test Automation Tool, custom test cases can be created using the recording functionality where steps are captured. Keeping in mind, staying to the standard is recommended versus creating manual test cases. Manual test cases should only be created if the business process does not yet exist for an automated test case. Then, the test data will have to be defined for this manual test case. If third party automated test tools are used, any necessary automated test cases are created.
Lastly, under the Test Preparation for SAP S/4HANA Cloud deliverable, edit action data and change visibility of test processes is done. This entails modifying actions in customer specific automated test scripts, and to manage the visibility of a test case in the Manage Your Test Processes app.
Now, moving to the test execution activities, we create the test plan by including the standard and custom test processes in the sequence that defines the end-to-end flow. Test data using the data variant is changed, as needed. Then, the test plan which was established earlier is executed where the statuses are displayed in the form of "success," "with errors," and "failed" along with a detailed log and screenshots of each step. The first “failed” test process step in the test plan will have one “failed” action followed by “cancelled” actions when an action fails, the following actions do not execute, hence their statuses become “cancelled.”
Any errors which were identified in the test execution, need to be reviewed along with all corresponding documentation which was captured in the log. Defect management is critical during this stage to resolve the identified test issues. The testing team should resolve all failed actions and fix them per the detailed action log and re-run the test plan to ensure they are run successfully. Then, test execution analysis and reporting is performed to generate, review, and evaluate the test results report utilizing the Test Result Dashboard. The Executed Test Processes report is available on a bar chart, and the Post Upgrade Test Plans and Customer Test Plans reports are available on an interactive visual report showing the various test statuses.
The next phase in SAP Activate is the Deploy phase. However, Testing activities are not performed in this phase. We move to the final phase - the Run phase to close out the Testing workstream.
The Testing workstream brings us to the Run phase at this point in SAP Activate and your project implementation.
Preparing the Testing team for the solution upgrades helps to ensure new solution functionality is executed smoothly for the business processes. Project teams also need to be aware of the upgrade dates as they are not flexible and cannot be shifted. The upgrade for the Q-System will be first, followed by the P-System two weeks later. During the time between the Q and P upgrades, regression testing occurs. You can refer to the SAP S/4HANA Release/Upgrade Schedule and Prepare for Quarterly Upgrade task for additional details. Additionally, the Testing team will prepare for post upgrade testing (PUT) by SAP (relevant only for customers that are live in the production system).
The Post Upgrade Tests are smoke tests performed by SAP and does not replace the Regression Testing activity. Regression Testing is the responsibility of the customer, and the customer needs to test their solution, based on custom processes and custom test data. Consent needs to be provided for SAP for the PUT at least two weeks prior to the upgrade - this can be done directly in the tool.
The PUT scripts are standard scripts delivered by SAP. The scripts are built with data picking logic which picks the latest transactional data needed to run those test scripts in the Quality system. The customer needs to ensure that the prerequisites are met. All PUT results can be accessed and viewed within the Analyze Your Test Result app along with the respective logs and screenshots for the test scripts.
Testing teams should check, confirm, and/or create master and transactional data in the system. Additionally, the maintenance of test user credentials and applicable roles needs to be reviewed and updated as needed to ensure team members have the correct roles and access to their profiles to execute the testing activities during the project. Test users in the solution need to review and update their password whenever the password is changed or expired.
The testing team needs to perform and view the post upgrade test results and statuses in the solution and all logs/screenshots (i.e. failed, success, in process, and cancelled). Tests which need to be re-executed due to failing must be re-executed.
As performed in the Realize phase, executing regression testing in the Quality System is done to make sure the functionality works as expected after the quarterly release upgrade. The Test Automation Tool is strongly recommended however, other third party test automation tools may be used. Note, SAP does not manage any third party testing tools and processes.
Lastly, the final testing activity of test execution analysis and reporting is performed again (first performed in the Realize phase) to generate, review, and evaluate the test results report utilizing the Test Result Dashboard.
Additionally, check out the additional resources and content below to expand your knowledge on SAP Activate and SAP S/4HANA:
Inside SAP S/4HANA Podcast
As always, I would like to encourage you to subscribe to our new podcast “Inside SAP S/4HANA.” This podcast leverages the unique knowledge and expertise of SAP S/4HANA product experts, partners and customers to address your needs by sharing product insights and project best practice. There is no customer success without product success and project success; we help you get to the next level and make your SAP S/4HANA projects a success. Subscribe now and benefit from the shared knowledge.
openSAP Microlearnings for SAP S/4HANA
Just a couple of months ago, we launched openSAP microlearnings for SAP S/4HANA as an exciting new learning opportunity. What began with a small batch of 20 videos, has now become a channel with more than 50 microlearnings that have generated over 20,000 views since then. Today we cover multiple lines of business, like Finance and Manufacturing & Supply Chain, and key technology topics like Master Data Management, Extensibility, User Experience, and Upgrade Management. We are continuously adding new microlearnings to the SAP S/4HANA channel, so make sure you check them out.
Your voice matters!
If you want to learn more and actively engage with SAP subject matter experts on SAP S/4HANA Cloud, join our SAP S/4HANA Cloud Customer Community. This platform which is available to SAP S/4HANA Cloud customers and partners has a clear mission: deliver an interactive community to engage with one another about best practices and product solutions.
For more information on SAP S/4HANA Cloud, check out the following links:
- SAP S/4HANA Cloud release info: http://www.sap.com/s4-cloudrelease
- SAP S/4HANA Cloud Release Blogs here
- SAP S/4HANA PSCC Digital Enablement Wheel here
- Inside SAP S/4HANA Podcast here
- Product videos on our SAP S/4HANA Cloud and SAP S/4HANA YouTube playlist
- Best practices for SAP S/4HANA Cloud here
- SAP S/4HANA Cloud Customer Community: register here
- SAP Activate Community here
- Feature Scope Description here
- What’s New here
- Help Portal Product Page here
- Implementation Portal here
Follow us via @Sap and #SAPS4HANAValueSeries
- SAP Managed Tags:
