Introduction: In the ever-evolving landscape of software development and quality assurance, we introduced a game-changing solution: Intelligent Test Scoper (i-TS). This next-generation tool brings real-time Test scope proposal and prioritization to the forefront, harnessing the power of code change analysis in SAP S/4HANA Cloud Release and SAP S/4HANA Cloud Continuous Feature delivery (CFD). The outcome? Optimized regression tests that are not just precise and relevant but also reduce test cycle time, effort, and costs for SAP customers. In this blog post, we will take you through the journey of i-TS, its key benefits, its vital role in ensuring high-quality cloud software delivery through Intelligent Scoping, Pilot results with SAP Customers, and GA timelines.
In-house innovation and its viral adoption: It all started as an idea back in 2017 to solve a teething problem – Test scoping for regression tests was static and not scientific or intelligent! Worse of all in the world of cloud and automation, one couldn’t afford infinite resources to run the entire suite of regression tests for every test phase/ release! This brought us to the drawing table where we mooted the idea of a scientific scoping solution that is based on code changes (coming into the test system), Customer usage (info coming from Cloud reporting) and Error-proneness. This is best illustrated in Figure-1 below where you see the Venn Diagram showcasing the intersection point-1 where the Test Cases that touch the code having undergone a change is all that needs to be tested. We christened it i-ScOper.
Figure-1: Intelligent Test Scoping - The Concept
Being just an idea, and a path breaking one at that, the Team had to do several Proofs of Concept and Pilots to win stakeholder confidence (not to mention further innovations to address the feedback coming). The i-ScOper team meticulously tailored its approach, considering detailed user profiling. What followed has been nothing short of a transformative odyssey. We were able to go productive as the tool of choice for On-Premise test scoping in 2019 and soon after, in 2021 the tool of choice for Cloud Release regression test scoping. It would be an understatement to say that every step of the way was a trial by fire for the Team to prove the relevance and accuracy and optimizations of the solution offered by us!
Accelerated growth and adoption internally: In subsequent years, i-ScOper underwent remarkable advancements. The team established a Central Repository, introduced innovative UI5 code change tracing, and integrated an automated test plan creation feature, expanding the product's scope to encompass cloud regression testing. The year 2021 marked a significant milestone with the launch of i-ScOper 2.0 with a new FIORI UI and based on SAP Cloud Platform. This version evolved into a versatile Test scheduling and Scoping tool also for the occasional Emergency/ Verification Patch tests. Figure-2 and Figure-3 below illustrate the Journey of i-ScOper maturing into i-TS and Thought leadership footprints, along the way, in internal and external forums.
Journey of i-ScOper maturing into i-TS and the Thought leadership footprints
The impact of i-ScOper has been remarkable, as evident from the following key benefits:
- 60% Reduction in On-Premises Test Scope with no Defect leakage
- 90% Effort Savings for Test Managers with Automatic Test Plan creation solution
- 75% increase in Defect yield in regression tests, by focussing on the right scope
- 87% Reduction in NetWeaver Inbound Qualification Test Scope
- Increased Efficiency in scope for the occasional Emergency/Verification Patch tests
Motivation for i-TS (intelligent Test Scoper) for Customer: Post-upgrade Test is a strategic offering for SAP Cloud Customers, to build upgrade confidence by automatically triggering customer chosen regression test suite soon after an upgrade. This helps identify any regression issues and thereby build Customer confidence post upgrade. Scope of these tests however must strike a balance between effectiveness and efficiency. With i-TS having insights into the code changes of a SAP S/4HANA Cloud main release or Continuous Feature Delivery (CFD), the idea is to optimize also for the Customers the number of standard scripts delivered (via SAP S/4HANA Cloud Test Automation Tool) for Post Upgrade Tests. This is illustrated in Figure-4 and Figure-5 below.
i-TS - High Level Overview and as viewed in Test Automation Tool for S/4HANA Cloud
Phased Roll-out under Toggle in CFD 2308.1: i-TS was introduced through a phased rollout in Continuous Feature Delivery No.1 of 2308, as part of Test Automation Tool for S/4HANA Cloud. Refer following Help Portal Documentation to learn more about i-TS (Intelligent test Scoper)
Customer Pilots: In this rollout, the i-TS App was enabled only in the systems of 6 Pilot Customers. This involved the activation of the feature toggle "CFD_FT_ITS_SCOPING" and assigning the catalogue "SAP_CA_BC_SCOPE_TEST_PC" to the business role "SAP_BR_ADMIN_TEST_AUTOMATION". As part of the Pilot:
- Customers pushed the Optimized scope from i-TS to “Manage Upgrade Tests” App and triggered it in ‘Test Your Process’ App (of Test Automation Tool for S/4HANA Cloud) and were able to visualize the gains of consuming optimized scope.
- Customers compared the 2308 originally executed Test Scope with i-TS proposed Test Scope and noticed efficiency gains of ~10-12% for a full release and ~ 60% for a Continuous Feature Delivery.
Customers were convinced this is the right way forward and were keen to use i-TS for CFD/Release Upgrade Test Scoping. A summary of the feedback from Pilots is captured in Figure-6 below.
Figure-6: i-TS - Customer Pilots Feedback Summary
General Availability: i-TS shall be GA in SAP S/4HANA Cloud Release 2402, Stay tuned....
Conclusion: In conclusion, i-TS is a testament to SAP's commitment to enhancing software quality and efficiency. It has transformed the landscape of test scoping and regression testing, resulting in significant benefits for SAP internally and for its Customers. The journey of i-TS, from its inception internally as i-ScOper, to successful Customer pilot deployments and positive feedback showcases a dedication to continuous improvement and innovation. As software development continues to evolve, solutions like i-TS will play a crucial role in ensuring high-quality software delivery to customers.
It has been an absolute pleasure to work on this in-House Test Engineering innovation from its inception to release to SAP Customers for productive use. The Pride and pleasure of the Team (that grew from just one developer to two fully staffed SCRUM Teams in the process of working on this innovation) knows no bounds! 😊 This innovation journey catapulted those involved into a journey of exploration, innovation and steep learnings in engineering excellence thereby transforming Careers!
- SAP Managed Tags:
2 Comments
