Skip to Content

SAP Benchmark Performance – Linux vs. Windows

 
Performance can be measured using very different tools and techniques. There are the well known hardware and software providers delivering a tools or toolsets which shows the performance of a certain product or technology. The same counts for SAP providing a benchmark tool kit to benchmark different core enterprise scenarios of an SAP system. Such standard application benchmarks are Sales&Distribution (SD), Financials (FI), Human Resources (HR), Portal and many more. Out of these the SAP SD-Benchmark became a de facto industry standard for enterprise business benchmarks. Therefore I’d like to choose the SD-Benchmark as reference for the performance measurement described in this SDN blog.
 

SD-Benchmark

 
Taken from the SD-Benchmark documentation:
 
The Sales and Distribution (SD) Benchmark consists of the following transactions:
 

  • Create an order with five line items. (VA01)
  • Create a delivery for this order. (VL01N)
  • Display the customer order. (VA03)
  • Change the delivery (VL02N) and post goods issue.
  • List 40 orders for one sold-to party. (VA05)
  • Create an invoice. (VF01)

 
Note: Each benchmark user has his or her own master data, such as material, vendor, or customer master data to avoid data-locking situations.
 

DB/OS Platforms?

 
SAP software runs on several hardware architectures like Sparc, Power, Itanium and commodity x86_64 from different hardware vendors, on several operating systems like HP-UX, AIX, Solaris, Windows and Linux and on several databases like Oracle, DB2 or MaxDB, the amount of combinations is very huge. I’d like to choose Linux on commodity hardware as platform to show you the performance of such an SAP Linux SD-Benchmark and compare it to other benchmarks on other platforms like Windows on the same hardware.
 

Round 1 – Fight!

 
The first round of Linux versus Windows was fought on an HP ProLiant DL380 G5. The machine specs are as follows: 2 Processors / 8 Cores / 8 Threads, Quad-Core Intel Xeon Processor X5355, 2.66 GHz, 64 KB L1 cache per core and 4 MB L2 cache per 2 cores.  

Operating System  SuSE Linux Enterprise Server 10 Windows Server 2003 Enterprise Edition
Database Oracle 10g SQL Server 2005
Benchmark Certificate  2007028 2006079
Number of SD-Users  1795 1790
ERP Release 6.0 (SAP ERP 2005) 6.0 (SAP ERP 2005)
Dialog Response times 1.96 sec 1.98 sec

The Linux SAP SD Standard Application Benchmark got five more users. What a tense victory. Let’s see what the second round with actual hardware says
 

Round 2 – Fight!

 
The second round of Linux versus Windows was fought on an HP ProLiant DL380 G5. The machine specs are as follows: 2 Processors / 8 Cores / 8 Threads, Quad-Core Intel Xeon Processor X5460, 3.16 GHz, 64 KB L1 cache per core and 6 MB L2 cache per 2 cores. The software configuration, the amount of SD-Users and the official SAP SD-Benchmark certificate number are:

Operating System  SuSE Linux Enterprise Server 10  Windows Server 2003 Enterprise Edition
Database MaxDB 7.7 SQL Server 2005
Benchmark Certificate 2008025 2007064
Number of SD-Users  2275 2436
ERP Release 6.0 (SAP ERP 2005) 6.0 (SAP ERP 2005)
Dialog Response times 1.98 sec 1.99 sec
Database Request time (dia/upd) 0.005 sec / 0.023 sec 0.030 sec / 0.016 sec

 
Wow, Microsoft did some really nice performance improvements since the last benchmark. Comparing the database request times (dia), MaxDB is more then 6 times faster than SQL Server 2005! But as the number of users shows, there is still room for improvement.

The SAP LinuxLab is currently evaluating and considering several performance options which are going to push the Linux benchmark up to the level of the Windows one. Such investigations won’t finish in a matter of weeks, so it may take some time to get the full performance potential incl. a certified SAP SD-Benchmark. Stay tuned…
 

To report this post you need to login first.

12 Comments

You must be Logged on to comment or reply to a post.

  1. Bernd Eckenfels
    (disclaimer: the below observations are for sure not important on production or benchmark sap instances)

    BTW: for some situations Oracle on Windows is performing better than on Linux/Unix. The main two reasons I see here are the Connect Times (20ms vs. 100ms) to dedicated servers. The Windows Threads are just more leightweight. (But of course your application should not connect so often to the DB).

    And a second thing is that some workloads can greatly be speed up by a filesystem buffer cache underneath the data files. (especially small full table scans and some non-cached LOB operations). So in a typical unix scenario where you have raw partitions or directio the Oracle DB might be more efficient, but slower.

    Gruss
    Bernd

    (0) 
  2. Fidel Vales
    Hi,
    I do not get it.
    If you really want to compare only the OS (as the title suggest) why do you use different RDBMs?

    You are introducing a new variable on the equation that could bring Linux/Windows up or down but it would be more accurate for a comparative point of view

    (0) 
    1. Hannes Kuehnemund Post author
      Hi Fidel,

      Currently, the Microsoft SQL Server is not available on Linux. According to the official SAP  SD-Benchmark results page, there are 249 2-tier SD-Benchmarks with “Windows Server 2003” as operating system. Out of these 249, 218 were made using SQL Server as database. This means, only 31 of 249 were made using an other database then SQL Server. Taking into account, that you need some constant for comparision (which is the hardware), it is very hard to find exactly the same hardware to compare these 31 non SQL Server benchmarks with other ones.

      Additionally, the database usage pattern of an SD-Benchmark does not exceed 10% of the system resources. Even if any RDMS is 10% faster over another one, the increase in the overall performance wouldn’t exceed 1%. The same is true for the operating system, but having a simple view on the different parts is not enough. Only the big picture, having all of them working together is the one of interest.

      Last but no least, the only chance to compare such benchmarks it to take what is in the list.

      Nevertheless, there are two possibilities to overcome the database issue:

      1a) The hardware vendor, who is conducting these benchmarks, could perform official benchmarks on Windows using another database. Is there any reason why 87% of all Windows Benchmarks use SQL Server?

      1b) Microsoft could do a port of SQL Server to Linux. But this is very unlikely to happen ๐Ÿ˜‰

      Thanks,
        Hannes

      (0) 
  3. Puru Govind
    Hi,
    I was looking at benchmark results and observed that in Round 1 fight, the CPU utilization for Windows server is pretty low(only 94%). I am clueless about the reason. Is that hardware made to run on Linux? Clueless!

    Regards
    Puru

    (0) 
  4. Ekrem Geรงim
    Firstly, thanks for this blog.
    I just want to know, Why did you use Suse? What about Redhat? Is there any performance differences between them? Which one do you suggest?
    (0) 
    1. Hannes Kuehnemund Post author
      Hi Ekrem,

      actually, I didn’t use SUSE, it was HP because HP did the benchmark ๐Ÿ˜‰

      From my experience, both SLES and RHEL show the same performance when it comes to SAP SD-Benchmarks. There isn’t any notable difference.

      The SAP LinuxLab is neutral, so you question which I’d suggest has to be left unanswered.

      (0) 
  5. William Blanchard
    Does this test actually prove that Windows is getting faster?  You are comparing apples to oranges.  Looking at the results, I would have to say that MaxDB slowed the Linux test down.  Put Oracle on Suse and run the test again.  You can’t make a change like that and claim it to be a legitimate test.
    (0) 
    1. Hannes Kuehnemund Post author
      Hi William,

      obviously, it wasn’t me who changed the DBMS and I never claimed that this is a legitimate test … ๐Ÿ™‚

      Seen from Linux perspecive, these two Linux benchmarks are the only ones which have conterparts on another platform (Windows) using exactly the same hardware. I’d love to compare apples and apples, but then we first need a Windows benchmark with another DBMS then MYSQL. And this is most unlikely to happen.

      Additionally, I’m really proud that there was a benchmark with MaxDB. Even if it was not the best, it shows that MaxDB is already settled in this market. I’ve worked closely with the colleages perfoming both Linux benchmarks, and you can believe me when I say that MaxDB (alone) outperforms other database systems. The work left is the optimization of the interaction of all parts. On Windows, the OS, DBMS and SAP are already in the benchmark circus for years. Now it’s time for MaxDB, Linux and SAP to catch up.

      Said this, I’d love to see more MaxDB benchmarks in the future. I’d like to say a huge thanks to HP who did this test, knowing that the numbers aren’t as good as on Windows. Thanks a lot!

      (0) 
    1. Hannes Kuehnemund Post author
      Kumar,
      you can check http://www.sap.com/benchmark and search the listed 2-tier benchmarks for comparable Windows/Linux benchmarks.
      I’d keep in mind, that a platform decision should not be based on sth like a 5% performance difference. Other factors like
      * education and skills of the own IT stuff
      * future stability of a company
      * company regulations (open source/closed source)
      * …
      are way more important for decision making.
      Hannes
      (0) 

Leave a Reply