Skip to Content
Technical Articles

Tune and Scale while you Develop

Introduction

As a solutions architect, I have been doing application performance analysis for several years. Slow performing application is always a disruption to the business growth. I have seen many projects focus on application functionality in the build phase (development phase) and application performance is measured only in the performance test cycle. This approach will result in many performance issues in performance testing, and puts the development teams to do rework in the performance test phase.

I have been part of many war room sessions to resolve production performance issues, I hope its not a new thing for may of fellow architects.

This situation can be avoided by following the Left Shift Performance Tuning Paradigm.

Define, Measure and Scale

The build (or) development phase is the right time to define, measure, and scale application performance.

In SAP commerce applications, the below factors plays a major role in application performance and scalability

  • Tuning Database queries
  • Application Scalability

Note: Caching, infrastructure, configurations are also driving factors for application tuning, this article is focused on database usage and application scalability

How to tune database queries

In my experience, additional infrastructure may not help the application performance if the database queries are not properly handled. Improper use of read, update, insert queries will slow down the application on a peak load (promotional) day. Some times system can be unresponsive and can not be recovered. I have observed additional infrastructure puts the systems in unrecoverable state if the system is struggling with improper use of the database.

To avoid this situation, make sure to monitor the database query usage in the application development phase.

 

Perform the below tasks in the application development phase

  • Baseline database query count

This activity is to set the targeted number of queries in each flow (or) page.

How can I define a target? – It’s simple, compare it with accelerator (B2B/B2C) database query count (or) based on your experience from other implementations. This number is to guide the development cycle.

  • Define Load profile

Load profile depicts the customer behavior on the site. Identify the critical areas of the application. Define a load profile based on the customer behavior. Especially consider the most used customer flows on a promotional day (peak load day). Create automated test scripts to execute the load profile.

  • Execute Test

Test the load profile with each build. Compare the database queries from each test with the baseline

Identify the difference between baseline and application queries and log a defect accordingly. This whole process to be automated so that every build is monitoring the database queries.

  • Tune Database Queries

Fix the issues which are identified as part of the automated test process.

This complete process to be automated and executed with each build. This process can be treated as a quality gate in the development cycle.

Application scalability

Application scalability is a potential factor for application growth. Application scalability check is one more major tuning exercise to measure the database tuning. This also helps the team to validate the database query target.

 

 

Steps to measure the application scalability

  • Define scalability SLAs based on application non-functional requirements

 

  • Benchmark single node test

Perform the actual system performance with a single application node and capture the SLAs of a single node. Capture concurrent users, transactions placed in a duration, response time of the site, memory consumption of the server.

Note: If the system is not meeting the SLAs, then reduce the concurrent users. When I define the benchmark for a single node I always check for a healthy system.

Recommendations to perform single node test:

    • Remove the external system calls and perform the system test in an isolated mode. Implement stubs in the application for all external interfaces
    • Stop batch jobs and backend processes
    • Ramp up users based on system memory condition, response times. Always benchmark the concurrent users per node based on the healthy state of the application
    • The test scenarios should reflect the customer behavior of a peak load. Ex: Promotional day the customer load is high

Sample matrix

Nodes
Concurrent users
Transactions
Response Time
Stub Enabled
Duration
CPU/Node
Heap Memory/Node
Errors
1 100 2000 2 sec Yes 60 mins 4GB 2 GB No

 

  • Scalability test

Perform the scalability test with more nodes. perform the test with 2, 4, 6, and more.

How to measure the successful scalability test? Transaction count and Concurrent user count should be increased proportional to the number of nodes.  Response time, memory usage, and duration should stay constant. The numbers may not be proportional, but they should be close

Sample matrix with scalability test

Nodes
Concurrent users
Transactions
Response Time
Stub Enabled
Duration
CPU/Node
Heap Memory/Node
Errors
1 100 2000 2 sec Yes 60 mins 4GB 2 GB No
2 200 3950 2 sec Yes 60 mins 4GB 2GB No
4 400 7900 2 sec Yes 60 mins 4GB 2GB Np

 

  • Tune system

If the system is not meeting the scalability criteria then we need to tune the system. Consider the below aspects of the tuning based on application performance monitoring (APM) tools

    • Validate Database calls
    • Validate cache usage
    • Code bottlenecks
    • Repeat the cycle till we meet the benchmark

Conclusion

Application tuning and scalability is a critical part of the development. Many performance issues can be easily avoided in the performance test phase and in production if the teams understand the importance of this activity. Make sure to perform the below activities in the application development phase.

  • Define database query benchmarks for each application page and flow
  • Perform the scalability test to make sure the benchmarks are met
  • Tune the system based on benchmark

 

 

3 Comments
You must be Logged on to comment or reply to a post.
  • Thanks Akhil for the article, it covers so many important aspects of performance tuning. The idea of doing performance tuning while doing implementation is really helpful.

  • Good Coverage of the most important aspects of the performance tuning exercise. This helps a lot for the performance benchmark exercise and gives the direction to tackle the pain points first and get to the required degree of tuning. Liked the process defined to tune on single nodes and scale up the application further. Great explanation and very useful and perfect to share across the projects.