In this series, I will explain common methodologies in vogue when it comes to testing SAP solutions, their merits and disadvantages. In this first part installment I will describe how QA teams have a choice when it comes to selecting the right methodology and then describe in detail the V Model.
As I have always maintained in my presentations that testing is not only imperative, an absolute must but it is also essential that it is done right. A lot of testing projects fail to take off or do not bring the desired value to the table because they are either not started right or somehow in mid-flight take a different course than intended. As a QA manager one might often think “Do I plan for Unit, Integration testing or both?”, “How can I handle so many duplicate bugs?”, or “How to make my tests more business relevant?”. Does this sound familiar to you? If so, read on.
A typical testing & QA project has three main drivers:
- Test faster with more iterations
- Reduce the risk but with less cost to company
- Create more value and (you guessed it) with reduced costs
At times these goals seem unachievable, in fact even at loggerheads with each another. The good news is – the objective is achievable. Even better news is- there is an approach available using which you can achieve the seemingly impossible objective.
At the heart of these requirements, once you boil it down to essentials, it becomes clear that there are 3 core-principles that you can leverage to attain these results:
- Methodology (or processes)
Let us first talk about the methodology. In fact let us take an example where the QA manager has no methodology whatsoever so we can draw a contrast between a project that starts with a well defined method versus one that has no such process in place. If you have not defined a methodology to test with, you would have considerable challenges when it comes to following tasks:
- How to define and manage testing requirements: Requirements are the core of any testing project using any methodology
- What about traceability: How do I know what the gaps in my approach are?
- Mitigating Risks- You cannot measure what you cannot define. Without Risk Based testing, how do you know what your risks are?
- What to test and when to test: Unit, Integration, Regression, String or all? Which cycle to test and when?
- Root cause analysis: how to determine what is underlying cause of errors
Now I am a big fan of re-usability, parallel testing and multiple iterations. A testing methodology that emphasizes on these very principles is called V methodology. I would not go into the basics of V Model here assuming most know what it stands for. If interested, you can read more about V testing methodology here and here.
Following is a high level depiction of a V model. As you will notice, the testing activities are bent upwards to the right in this diagram. This is to make the m correspond to their counterpart development activities. For instance, you have Unit testing stage corresponding to Unit design. It all converges upon development or coding. The x-axis is representative of time elapsed. The vertical axis denotes the granularity or level of advancement in project.
Figure 1: V Model of Testing (Image courtesy @ www.compensationanalytics.com)
The rationale behind V model is to highlight the relationship between various stages of development and testing. Traditional waterfall method is a linear process. You have various milestones and activities arranged on a horizontal timeline. It is not a nimble process. When you do arrive at the testing stage, often pretty late in the cycle, it is difficult to go back. For instance, in a conventional implementation, Integration testing is part of the realization phase. When you do arrive at Integration testing phase, there is often a mad scramble to assemble so called Integration test scripts. I have lost count of many a sleepless nights getting test scripts ready and loaded into the test tool to pass the next toll gate, entry criteria and what not.
Realizing well that I might cause quite a few raised eyebrows, I would nevertheless state – “Test cases and scripts are the least important assets in a testing process”. Don’t get me wrong- we need test cases but there is an overemphasis on their importance. I have frequently heard QA personnel declare emphatically, “We are going in this cycle with 2000 test cases!”. The numbers do not matter- what matters is coverage. What is more important in my opinion is the test objective or requirements. If you get the requirements right, it is highly likely you would get the test plans right. Unfortunately conventional testing operates in a completely different context. Test scenarios and steps dictate testing. In fact I have seen projects where defects drive testing. It is ridiculous and dangerous at the same time.
Lest I digress, the V Model ensures early on that that the project team is aware different types of testing that will be required in order to go-live. It represents a virtuous circle where continuous feedback and iterations ensure high quality software is delivered. It also ensures that with each development stage, you are able to “extract” the testing criteria. For instance, very early on during requirements specifications, you would be able to have a set of requirements that will lead to acceptance criteria just prior to go-live.
I would invite the readers to share what testing methodologies they have implemented and have been part of. It would make for an interesting read. In my next blog, I will do a deeper comparison of more traditional, waterfall models with newer testing methodologies and conclude the series with why I think SAP testing projects require a hybrid testing approach that includes the best of both worlds (Agile and traditional).