Minimize the risk in the regression test for a sFIN 3.0 EhP 8 upgrade by using BPCA
The advantages of the Business Process Change Analyzer (BPCA) as one of many capabilities of the SAP Solution Manager are very well specified in the blogs BPCA – Powerful Risk Eliminator and Business process variants and change impact analysis with BPCA.
A customer project that included an upgrade of the ERP solution from sFIN 2.0 to sFIN 3.0 EhP8 without any business driven or functional changes sounded like a very suitable starting point for the usage of BPCA.
Therefore the customer started a parallel project for the conceptual design and implementation of SAP Solution Manager 7.1 Business Process Change Analyzer. The concept included the automatic generation of semi-dynamic Technical Bill of Materials (TBOMs) based on the Usage Procedure Logging (UPL) in the productive system of the ERP.
The concept for the BPCA implementation sounded simple and included the steps of the reliable implementation guide by SAP for BPCA. So what was the catch?
The first challenge was the prioritization for all main requirements for the planned Go Live including the sFIN upgrade. The realization of BPCA as one of these main requirements had a lower priority than 2-3 other topics. This meant that if resource conflict showed up then BPCA had to wait until the other 2-3 topics freed the needed resources.
The second challenge was the strict requirement for the implementation of SAP authorizations in the form of roles. It wasn’t allowed to use the delivered SAP standard roles nor some critical authorization objects like S_RFCACL.
These two challenges resulted in the third challenge. A lot of the higher prioritized requirements led to a high consumption of the existing SAP authorization resources. Therefore we hadn’t enough SAP authorization support for the planned concept and implementation steps and the BPCA project took longer than planned. The test planning for the sFIN upgrade was finished and the regression test started. A second aspect in the area of short resources was the recording of dynamic TBOMs. It wasn’t feasible due to the tight upgrade schedule of the sFIN upgrade and due to the limited availability of business process experts.
The exciting questions were:
- Is BPCA able help the project at this late point in time?
- How can it be avoided that the BPCA generated test plans include test case duplicates of the existing test plans?
- How could we minimize the additional test scope by BPCA since
- many test cases started with the same executable and
- most of the test cases included more than one executable in their test case steps?
Approach and Solution
A positive aspect of the long running implementation phase for the BPCA project was that the UPL was running for long period (more than 3 months). So in the SAP Solution Manager 7.1 there were saved a lot of information about the used executables and related objects in the productive ERP.
For the configuration of UPL we could follow the standard process based on the SAP implementation guide for BPCA and Custom Code Management. The exception was the authorization customizing for the role to generate semi-dynamic TBOMs. We tried to customize a role without the critical authorization object S_RFCACL, however the validation with the “BPCA Check” from the “Test Management Administration” in the Solution Manager Workcenter as well as the execution of the report for the generation of semi-dynamic TBOMs showed issues. The same result was shown by the test with tcode SM59 for the trusted RFC to the ERP test system.
Therefore we customized a special role only for the semi-dynamicTBOM generation, assigned it to a temporary dialog user and executed the “BPCA Check” in addition to the report for the semi-dynamic TBOM generation. It led to the expected result and semi-dynamic TBOMs were generated.
Then we customized a batch user, assigned the proven role for the semi-dynamic TBOMs generation and configured a daily job for the semi-dynamic TBOM generation with this batch user.
For other scenarios like dynamic TBOM generation or change impact analysis this authorization object is not needed and therefore we could follow the instructions of the SAP implementation guide for BPCA plus the SAP security guide for SAP Solution Manager. Anyway the recording of dynamic TBOMs was planned for a later phase due to the above mentioned limited resources on business side.
After the first creation of semi-dynamic TBOMs we started the first change impact analysis.
Change Impact Analysis
Then we started the next step – the test scope definition in the form of a test plan. But this was in the middle of the planned regression testing time frame. At first sight it shouldn’t be an issue but it was. There was a regression test plan in place plus current test results in form of test notes and test case status. Thus a test plan update with BPCA was omitted.
Hence we generated a new test plan with BPCA including the Test Scope Optimization (TSO). For both test plans – the regression test plan and the BPCA test plan – we exported the content via tcode STWB_INFO and the status detail analysis including the test object column (for transactions as test cases). Then we compared both tables with excel to prevent duplicate test cases (test documents and transactions).
Test Plan Comparison
A still open issue was the fact that a lot of the customers test cases for the process steps included more than one tcode/executable for the different test steps. Unfortunately, BPCA only compares the related test object (e.g. a single transaction) from the test case with the result of the EhP analysis. The test preparation process in the customer project provided that all executables of a test case had to be included in the solution documentation (tab “transactions” in the SOLAR02).
As result, the change impact analysis also showed affected transactions that were not assigned to a test case as a test object and as a result we could not see the relation to the given test case.
For that challenge we have not found an easy or efficient solution yet. Therefore we manually compared the executables with the content of the test cases and eliminated all duplicates of executables, included in test steps of the test cases in the regression as well as in the BPCA test plan.
Optimized BPCA test plan
The optimized BPCA test plan only included 70 test cases and transactions. Because of time constraints we prioritized all test packages. All test cases that produced a cumulated test coverage of 80% got the priority ‘Very High’. Test cases that produced the missing 10 % to 90% cumulated test coverage got the priority ‘High’. The subsequent test cases that produced a 95% cumulated test coverage got the priority ‘Medium’ and the remaining test cases the priority ‘Low’. Thus the tester had the opportunity to prioritize their test execution by the test case (means test package) priority and the existing regression testing time frame.
With SAP Solution Manager BPCA it was possible to increase both the test coverage as well as the protection for affected SAP and Custom Code during a running regression test. Higher security requirements could be met with BPCA.
Providing test case duplicates between an existing test plan and the BPCA generated test plan was possible but there was no efficient solution in place to eliminate the duplicates. Here we are currently searching for better approaches as well as waiting for new capabilities in SAP Solution Manager 7.2.
Our process for the reduction of duplicate transactions in the BPCA generated test plan has potential for optimizations and I’m very exciting for the response to our experiences made.