Quality Assurance – A Complementary Approach

ABSTRACT:

Real-time versatile and stable product is the need of the current business world, in which very huge number of commercial transactions are triggered across the global business sectors.

To fulfil the same, a complementary approach of both Functional and Performance Testing is the most reliable / proven path where this combination will add as a major / primary tool for the IT organizations who are supporting the Global Corporate Business.

Functional testing is one face of Quality Assurance process whereas the desired output will be validated for the given input data.

But, whereas the Performance Testing is the other face in which the output is measured w.r.t the response time for the functionally successful business scenarios.

In a nutshell, it is a must to have both Functional & Performance Testing because the Complementary Features of both will act as a tool towards the evolution of real time stable product.

INTRODUCTION:

While developing a product (be it on premise / Client-Server / Cloud), on an average it is observed that around 50% of time & cost will be consumed for the Quality Control (QC) and Quality Assurance (QA) activities.

In order to meet the desired quality for a Product, it is required that an elaborated testing approach addressing technical, functional and non-functional needs to be carried out.

In this regard, both Functional & Performance are the major test types that will help to determine the quality of a Product.

Both the testing types complementing in nature so that successful execution and completion of both will provide the needed confidence for the End User to use the Product.

Testing is the primary activity / approach to effectively remove the Defects (Bugs) which gets created/induced during the construction (development) of the product.

Testing can be categorized in different types across various criteria. Primarily, one can make two distinct types in Testing as Functional & Performance based on the nature of testing.

In Functional Testing one will validate the desired functionality, which is a group of Test Cases designed, analyzed & executed so that the results will confirm to User expectations / product specifications.

This test will have Input data and Output Results will be validated as per the input.

However, only by having functional testing will not provide the complete efficiency as proper functionality alone cannot hold the stability of the product.

To make Complete testing proof, one need to consider few critical nonfunctional parameters tested through different type of Testing such as Performance testing which will provide higher efficiency in Testing.

Some of the key factors verified in this testing are, Benchmarking (comparing with a standard accepted by majority), Response time for the given input workload and configuration conditions.

Functionality testing involves the following:

• Unit testing
• Smoke/Sanity testing
• New feature testing
• Retesting
• Module Integration testing
• Scenarios Integration Testing
• Regression testing
• Adhoc / Freestyle testing
• Process flow testing

Performance Testing involves the following :

• Load testing – This testing checks the ability of application / product to perform under anticipated user loads.

• Stress testing – This testing involves testing an application / product under extreme workloads to see how it handles high traffic or data processing. The objective of this test is to identify breaking point of an application.
• Spike testing – This testing will test the software’s reaction to sudden large spikes in the load generated by users.
• Volume testing – Under Volume Testing large number of Data is populated in database and the overall system behavior is monitored. The objective of this test is to check performance of S/W product /application under varying database volumes.
  • Scalability testing – Scalability testing will determine the software application’s effectiveness in “scaling up” to support an increase in user load. It helps plan capacity addition to your software application / system.
• Pre-Requisites to be considered before the start of actual testing:
  • Identify the User: Have the user with the required Role which is essential to execute test case.
  • Set-up the Test Data: Determine whether the data needs to be created or whether it used by another tester or whether the data is locked & cannot be modified.
  • Review Test cases: Before the start of Actual testing, the testcases to be reviewed from the required project stakeholders and hence will have baseline (detailed) test case document.
  • Test Planning: Prepare Test Plan document having the complete details about Testing viz., Testing Duration, Area to be Tested, Area NOT to be tested, Entry Criteria, Exit Criteria Acceptance Criteria, Test Strategies, Test Data Required, Third Party Tools / SWs required (OS / Middleware / Database), Necessity of Simulation (if any), etc., This Test Plan should be reviewed and baselined (with the consent from all the required project stakeholders) before starting the actual testing activity.
  • Defect identifying / addressing through Static Testing: Identifying the Defects (Bugs) and fixing them within short time without execution of any code, only by just by reviewing the Testing Docs.
  • Test System Set-up under the given conditions with all the required software and other necessary tools.
  • Test Scoping: Define the Test Cases that needs to be tested for a system / application for the given Release.
  • End User / Product Expectations:(from Functionality and Performance Check) :
  • Requirements Coverage
  • User Experience
  • Response time
  • Ease of Operation
  • Uninterrupted Interactions
  • Consistent Performance at all time Different scenarios of Functional and Performance Testing :
    1. Both Functional & Performance Testing Fails.
    2. Functional Testing Fails and Performance Testing Succeeds.
    3. Functional Testing Succeeds and Performance Testing Fails.
    4.  Both Functional & Performance Testing Succeeds.

   ***   Example : Online Shopping / Banking / Ticket Reservation System ****

• 1. Both Functional & Performance Testing Fails and
  2. Functional Testing Fails and Performance Testing Succeeds.

In the above mentioned both scenarios (1) & (2), there is absolutely no need to go ahead and invest time, money & energy (effort) on carrying out Performance testing becoz when Functional testing Fails then there is no necessity to carry out Performance Testing as the Product will defeat the very purpose of creating it.

When a User logs in into Railway Reservation System, it is expected that the System need to provide the output which has passed all the required Functional tests successfully in all respects.

If there is failure in the Functional Tests, then the system will fail to deliver the intended functionalities leading to the collapse of the system. In such case, there is no need of any kind of Performance Testing.

Functional Testing Succeeds and Performance Testing Fails.In the above-mentioned scenario (3), it is very much needed to carry out both Functional & Performance testing because when Functional testing passes in all respects, then one needs to ensure that all the Users who are logged into the Reservation System will get the responses from the system in decent time as accepted by Product Owner & Users.

If Performance Testing fails, then there will be issues like long delay in getting response and other associated problems, leading to failure of the system in delivering the desired expectations. In this Scenario also, Product will not be viable for the general public use and hence the product collapse.

1. Both Functional & Performance Testing Succeeds

In the above-mentioned scenario (d), there is very much required to carry out both Functional & Performance testing because when Functional testing Pass in all aspects,then one need to ensure that all the Users who are logged into the Reservation System will get the responses from the system in decent time as accepted by Product Owner & Users.

If Performance Testing also succeeds in all respects, then only one can certify the Product as fit for the use of general public. Users will find complete satisfaction both in terms of functionality and performance leading to the success of the product.

Standard Project Life Cycle

Performance and Functional Testing (Complementing Nature) :

It is more often observed that testers will emphasize more on functional testing as compared to performance testing. Many a times, functional testing will be progressing good but performance will lag as non-functional aspects are ignored while functional requirements will be the prime deliverables for most of the Systems / Products.

It is a pure myth that functional testing has an advantage over the performance testing. Most of the time, the non-functional aspects are ignored which affects software quality. Moreover, if the software / product is performing well and there is no complaint from the client regarding the non-functional aspects, it is really not required.

Let us consider some of the working models of a software system as mentioned below.

• Response–reply models
• Transaction models
• Load based models
• Data replication models

Most of the times, we see that all the Software Systems will be developed and tested by different organizations. The business expectations will remain the same irrespective of the organizations working on that Software System / Module. New Functionalities (Features) will be delivered across Iterations / Releases. It is naturally expected that the System will have improved quality per Iteration / Release. Automation will help a lot in this regard.

In spite of a thorough Testing, some of the products will fail at to exhibit the minimum quality at Client Location as there will be a number of defects. So, one need to find out “Where does the Problem lie”?

There could be a number of reasons. Some of them are listed as below.

• Problem in understanding / analyzing requirements.
• Development / Test Environments are not proper.
• Incomplete test data / inputs.
• The software was weak in terms of robustness.
• Lack of Synchronization etc.,

Upon the detailed investigation, some of the possible restorative actions suggested are,

• There must be more frequent interactions between the Dev & Test teams.
• The functional testing must include all dependent applications interconnected with each other.
• The functional testing must take different inputs ranging from simple to complex
• Thorough Performance testing should be carried out.
• Testing must be done on the integrated system.
• Traced bugs must be re-tested after some fixed time-interval.
• The current iteration / release must fix the previous open bugs.
• Testing on a Customer -like test environment.

Conclusion:

These remedies will make the Product stronger by exposing more defects in less time. Moreover, in a nutshell there will be a positive impact on the system overall.

• The software quality improved significantly with optimized test-cycle time
• Resulted in fewer issues post install / implementation
• Rework got replaced by iterations, which kept on improving the product’s quality.