Best Practice approach for Test case writing
Hello,
If there exists multiple variants of the same test case for example if the same test case had be executed for different countries or different lines of business where the input and output varies but overall steps of execution remains same then below approach i feel would be the better approaches can be used based on the underlying logic or the called programs.
Approach 1 – If the underlying Logic/program is same
Would be a better approach to write single test case if the steps planned to be executed are same with multiple variants with different inputs and outputs for each variant.
Since the underlying logic is same accross different variants just the inputs and outputs vary , if one variant fails other variants also fail so test case status will be same accross all variants so is the reporting of defect for that particular Test case
Approach 2 – If the underlying Logic/program is different
If the underlying logic/program is different though the steps are similar for different variants with different inputs and outputs better approach would be to split it as seperate test case.
Reason for this Logic A/Program A might fail but not Logic B/Program B then we would have better clarity on Test status reporting and Defect Management.
If we club all the variants into one then though one variant might pass due to failure of other variant overall test case status had be set to failure which is not the case which will give wrong indication
Hope it helps.
Thanks,
Aj
I have read the content couple of times but i fail to understand what is it that you want to convey. Are you talking about developer unit testing or functional testing?
- Suhas
I am talking Integration Functional Test case and not Unit test case.
For that matter it can apply for Unit test case aswell
Consider example:
input A is used for a particular transaction of functionality executed for country India
input B is used for the same transaction of same functionality executed for country Singapore
wherein if the input A to a particular field may trigger certain underlying program A/loop A/path A where as input B to the same filed of the same transaction may trigger different programB/loop B/path B , here there may be a error in program A but not in program B having single test case for this particular transaction with India and Singapore as two different variants of the same test may not show right status in terms of Test Management & Defect Management wherein we have to show as the entire test case has failed which is not the case only for India it has failed but not for Singapore.
Hope its clear now.
Thanks,
Aj
Hi,
Nice graphs which simply illustrate the idea. I agree that this can be one approach. However, I would rather not recommend to write different scenarios if they test exactly same logic path.
Example?
test_divide(10, 2) = 5
test_divide(100, 5) = 20
It is simple but shows my intent - it is enough with one example for logic. But of course we may have cases when showing different input/output values will help to understand method even if there is same flow - I see your point.
I agree that it is good to write separate tests for different flow logic - boundary case with exception handling for example above:
test_divide(10, 0) -> exception thrown or handled inside?
With your approach of splitting test by different logic paths you help to detect what is wrong just by looking into failed test method name. And that is good.
If I talk about unit tests, sometimes I simplify tests strategy. Due to limitation of 30 characters for method name, I may just test different variants (input/output) of one method in the same test case method. Why? Because method should be simple and unit tests should test small pieces (units) of code. So if method test_divide fails, I know that division fails although from test name I do not know whether it was division by 0 or wrong result. I must go into details and check.
In addition it is easier to make one initialization for same method execution in test_method rather than in setup or class_setup. Data preparation is placed closer to test scenario and in setup there should be more general initialization not only for for individual test method I think.
So I choose compromise between more informative tests results just by looking into test methods name or simpler implementation of test methods.
Regards,
Adam
Effort estimation for test case writing and test execution will be increase as i need to execute with different variants before i write test case and also i need re-execute gain entirely for different variants during test execution if the underlying flow logic is different.
If i write same test case with different variants with underlying flow logic being different then we will not have clarity overall effort estimation for test case writing and execution if there are many test cases like this to be written and executed then even the resource estimation will also not be correct. I am talking here from the Test Management prespective not for just single test case.
Thanks,
Aj
Hi,
If you are talking about Integration Functional Tests then I agree - it is easier with separated tests for each different flow scenario. The point is that you cannot easily see the code to know what happened but you need to rely on method name and test scenario. Like black box testing.
With Unit Tests it is simpler - even if I have 10 verification in one test methods, I can quickly find line which fails. In addition I can give descriptive output for failing assertion (this can be done as well in functional tests). So in test_divide I may give error message '10 / 2 must be 5' or '10 / 0' must throw exception'. From message I know what actually failed when test_divide is red.
The difference is that if I put many different verification into one test method I will see only first failure, others will be ignored. I just know that test_divide is wrong, but I do not know how many scenarios are not working. If methods are split, then I may know that standard division is working but only division by 0 fails for example so that is the benefit.
Regards,
Adam
Agreed Adam.. finding out the underlying code or flow logic is a challenge and it requires Techno Functional experience.
Thanks,
Aj