Skip to Content
Author's profile photo Adam Gaspar

Performance impact of disabling NFS attribute caching

In some situations it might be recommended to disable NFS attribute caching, to avoid stale NFS handle errors in SAP systems. There are multiple ways to achieve, but usually it is done by adding the ‘noac’ option to mount. Setting this parameter however carries some possible performance degradation.

Description taken from: http://linux.die.net/man/5/nfs

ac / noac

To improve performance, NFS clients cache file attributes. Every few seconds, an NFS client checks the server’s version of each file’s attributes for updates. Changes that occur on the server in those small intervals remain undetected until the client checks the server again. The noac option prevents clients from caching file attributes so that applications can more quickly detect file changes on the server.

Using the noac option provides greater cache coherence among NFS clients accessing the same files, but it extracts a significant performance penalty.

There is an alternate options which also disables attribute caching:

actimeo=n

Using actimeo sets all of acregmin, acregmax, acdirmin, and acdirmax to the same value. If this option is not specified, the NFS client uses the defaults for each of these options listed above.

acregmin=n The minimum time (in seconds) that the NFS client caches attributes of a regular file before it requests fresh attribute information from a server. If this option is not specified, the NFS client uses a 3-second minimum.

acregmax=n The maximum time (in seconds) that the NFS client caches attributes of a regular file before it requests fresh attribute information from a server. If this option is not specified, the NFS client uses a 60-second maximum.

acdirmin=n The minimum time (in seconds) that the NFS client caches attributes of a directory before it requests fresh attribute information from a server. If this option is not specified, the NFS client uses a 30-second minimum.

acdirmax=n The maximum time (in seconds) that the NFS client caches attributes of a directory before it requests fresh attribute information from a server. If this option is not specified, the NFS client uses a 60-second maximum.

When setting the actimeo=0 all attribute caches will time out in 0 seconds, technically achieving what noac does, to not use caching. It is reasonable to expect it would inflict the same performance penalty, but testing shows that impact is much less.

I’ve set up a test environment to measure read and write on NFS, with the different caching options. Two SLES 11 SP3 servers, the first exporting a directory on NFS to the second. The test ran on the second server, and include copying a 1GB file, and 100 1MB files from local filesystem to NFS, and copying the same files from NFS to another local filesystem. The mount options used were: “rw,bg,hard,rsize=32768,wsize=32768,tcp,vers=3,timeo=600” with the first round, the second round with noac, and the thrid with actimeo=0. Each test were repeated 5 times, and the results were averaged:

Caching is on noac actimeo=0
1x1GB to NFS 18,8s 4m 1s 20,6s
100x1MB to NFS 5,6s 24,4s 6,2s
1x1GB from NFS 13,2s 22,8s 23,2s
100x1MB to NFS 2,4s 4,4s 4,6s

These numbers are only from one test setup, and performance will depend on the network and use case, but it shows the actimeo=0 provides comparable results to when caching is on, and much better througput than noac.

Assigned Tags

      2 Comments
      You must be Logged on to comment or reply to a post.
      Author's profile photo Adam Csaba Goetz
      Adam Csaba Goetz

      Thank you Adam for this brilliant summary!

      Author's profile photo Soeren Schmidt
      Soeren Schmidt

      Hi,

      just one comment, because a customer run into a problem with actimeo=0.

      An installation routine called the glibc function realpath() for every path and realpath() calls lstat() for every path component.
      Each path consisted of roughly 5 to 10 subdirectories, therefore the same amount of lstat() calls.
      Because there is no attribute caching every lstat ended up in a time consuming RPC call to the NFS server.

      This accumulated in a way, that the installation routine terminated with an error. In case of the customer the missing caching
      slowed the installation down by the factor 90.

      Before setting everything to zero, may be a very small value is a better approach.