Performance impact of disabling NFS attribute caching

In some situations it might be recommended to disable NFS attribute caching, to avoid stale NFS handle errors in SAP systems. There are multiple ways to achieve, but usually it is done by adding the ‘noac’ option to mount. Setting this parameter however carries some possible performance degradation.

Description taken from: http://linux.die.net/man/5/nfs

ac / noac

To improve performance, NFS clients cache file attributes. Every few seconds, an NFS client checks the server’s version of each file’s attributes for updates. Changes that occur on the server in those small intervals remain undetected until the client checks the server again. The noac option prevents clients from caching file attributes so that applications can more quickly detect file changes on the server.

…

Using the noac option provides greater cache coherence among NFS clients accessing the same files, but it extracts a significant performance penalty.

There is an alternate options which also disables attribute caching:

actimeo=n

Using actimeo sets all of acregmin, acregmax, acdirmin, and acdirmax to the same value. If this option is not specified, the NFS client uses the defaults for each of these options listed above.

acregmin=n The minimum time (in seconds) that the NFS client caches attributes of a regular file before it requests fresh attribute information from a server. If this option is not specified, the NFS client uses a 3-second minimum.

acregmax=n The maximum time (in seconds) that the NFS client caches attributes of a regular file before it requests fresh attribute information from a server. If this option is not specified, the NFS client uses a 60-second maximum.

acdirmin=n The minimum time (in seconds) that the NFS client caches attributes of a directory before it requests fresh attribute information from a server. If this option is not specified, the NFS client uses a 30-second minimum.

acdirmax=n The maximum time (in seconds) that the NFS client caches attributes of a directory before it requests fresh attribute information from a server. If this option is not specified, the NFS client uses a 60-second maximum.

When setting the actimeo=0 all attribute caches will time out in 0 seconds, technically achieving what noac does, to not use caching. It is reasonable to expect it would inflict the same performance penalty, but testing shows that impact is much less.

I’ve set up a test environment to measure read and write on NFS, with the different caching options. Two SLES 11 SP3 servers, the first exporting a directory on NFS to the second. The test ran on the second server, and include copying a 1GB file, and 100 1MB files from local filesystem to NFS, and copying the same files from NFS to another local filesystem. The mount options used were: “rw,bg,hard,rsize=32768,wsize=32768,tcp,vers=3,timeo=600” with the first round, the second round with noac, and the thrid with actimeo=0. Each test were repeated 5 times, and the results were averaged:

These numbers are only from one test setup, and performance will depend on the network and use case, but it shows the actimeo=0 provides comparable results to when caching is on, and much better througput than noac.