November 2022 update
- SAP-certified Oracle on EC2 using FSx for ONTAP. Check note 2358420
September 2022 update
- SAP-certified Sybase ASE on EC2 using FSx for ONTAP. Check note 1656250
- Scale-out is supported. Note 1656250
- SAP HANA Host Auto-Failover (HAF) is supported. Note 1656250
Intro
ONTAP is NetApp’s flagship NFS technology for storage. In September 2021, Amazon released Amazon FSx for NetApp ONTAP. Later in May 2022, AWS and NetApp certified this service for SAP HANA workloads running on AWS. Some essential benefits and limitations exist. Since it is super interesting, let’s explore it in this blog.
While other traditional NAS products such as Dell EMC PowerScale, Hitachi NAS, and IBM Spectrum Scale can migrate data to the public cloud, NetApp has gone more profound in its cloud integration. NetApp sells Cloud Volumes ONTAP (CVO) and Cloud Volumes Service (CVS) through some public clouds. CVO is a virtual ONTAP instance that runs in the cloud, while CVS is a file service that NetApp manages.
Amazon FSx represents the most profound integration for ONTAP with a public cloud, because AWS sells FSx for NetApp ONTAP as a fully managed service for file systems, featuring data management capabilities that support various use cases; FSx for Lustre is Linux-only and uses a performance-optimized Portable OS Interface file protocol. Windows uses the SMB protocol, OpenZFS uses NFS, and ONTAP supports NFS and SMB file protocols plus iSCSI for shared block storage.
Amazon FSx for NetApp ONTAP supports all the features NetApp delivers with its on-premises systems, so SAP on AWS users can also get features such as SnapMirror, SnapVault, and FlexCache, which is the added value of this offering.
Since FSx for ONTAP has only been certified for HANA (and Sybase), I will only compare Amazon FSx for NetApp ONTAP with the other solutions currently available for SAP HANA, excluding other database options for SAP. However, the same features exist within Amazon FSx for NetApp ONTAP, regardless of the database technology employed.
SAP HANA’s peculiarity is that it stores and processes its data in memory; the attached disk protects against data loss by saving it in persistent storage locations. To achieve optimal performance, the attached storage solution used for SAP HANA DATA and LOG volumes must meet SAP’s storage KPI.
With SAP on AWS, we have two main storage options, Amazon EFS and EBS. Elastic File System (EFS) is an OS-agnostic network file system based on NFS. SAP customers are familiar with EFS for shared filesystems, but it can’t be used for performance-sensitive DATA and LOG volumes. EBS is Amazon’s block-level storage, attached to EC2 instances, suitable for DATA and LOG. FSx for NetApp ONTAP covers all filesystem usages; DATA, LOG, and shared volumes.
Amazon FSx for NetApp ONTAP, SAP Highlights
- FSx for NetApp ONTAP must use Single-AZ deployment type
- No Multi AZ for FSx for ONTAP for DATA or LOG volumes
- Multi-AZ can be used for any of the other file systems to provide replication of data outside of the DB between AZs
- /hana/data, /hana/log, /hana/shared, and /usr/sap must have their own volumes
- List of HANA-supported instances in this link
- Rapid, 0 impact database backups, regardless of the size
- “DR lite” options to reduce costs or optimize TCO/RPO
- Production HANA workloads must have a throughput capacity of at least 1,024 MB/s
- Scaling can be done via the throughput capacity or via additional FSx filesystems. Read/Write performance will be doubled through the addition of a second FSx filesystem. Only read performance is doubled by using a throughput capacity of 2048 instead of 1024 MiB/s.
- Storage Efficiency is not supported for SAP HANA and must be disabled.
- Capacity Pool Tiering is not supported for SAP HANA and must be set to none.
- 100% of data in SSD
- Daily automatic backups should be disabled for HANA data/log volumes. Application-consistent backups and replication can be done with SnapCenter.
- Volume-based snapshot policy to none for DATA/LOG volumes; application consistent snapshots with SnapCenter for all databases.
- NFS 4.1 is highly recommended
The following table lists the most common use cases for instances and file-sharing solutions for SAP HANA landscapes on AWS and the solutions that support the use cases.
| EFS | EBS | FSx for Ontap | |
| Storage type | NFS v4 | SSD gp2, gp3, io1, io2 | NFS v4, SMB/CIFS |
| Cost* | $0.3 per GB/month and $6 per MB/month for provisioned throughput | $0.125 - $.045per GB/month, depending on volume type | $0.125 - $.055per GB-month, plus additional options |
| Availability and Accessibility | 99.99% EFS Standard Accessed by up to 1k instances from multiple AZs | 99.99% to 99.999% durability Accessed via a single EC2 instance | 99.99% for Single AZ, 99.99 for Multi-AZ, though not suitable for HANA on AWS |
| Storage and File Limits | Unlimited storage size 47.9 TB file size | 16 TB storage per volume with up to 27 volumes. File size limited by volume size | Up to 192 TB per filesystem. Up to 16TB per file. |
| Disaster Recovery | EFS replication | DRS – old CloudEndure- suitable for SAP systems on AWS or SAP HANA System Replication | Snapmirror replication |
| High Availability (for SAP) | EFS Standard is Multi-AZ | SAP HANA System Replication | Single AZ, replicated via SnapMirror/SnapVault to a second Single AZ instance. Multi-AZ for everything but the data/log volumes. |
| Snapshots | N/A | EBS snapshots. See SAP note 2039883 for limitations | SAP HANA application-aware snapshots via SnapCenter. |
| Backups | EFS Backup | backint to S3 with AWS backint agent or 3rd party tools | Backup and replication via SnapCenter for database volumes, |
| Data Tiering | N/A | N/A | N/A |
| Migration Services | AWS DataSync, excluding HANA files | HSR works on DB level. No storage replication for EBS Migrate using AWS MGN for HANA, previously known as Cloudendure Migration | Migrate On-prem ONTAP to cloud using SnapMirror/vault replication Xpack Cloudsync |
| Performance | 35kIOPS 500 MB/s throughput check | Up to 256k IOPS 4GB/s throughput check | +100k Network IOPS 80k SSD IOPS 2 GB/s throughput |
| Suitable for SAP HANA | /usr/sap/<SID> HANA Shared and Backup | HANA volumes - DATA and LOG /usr/sap/<SID> HANA Shared and Backup | All Volumes |
| AZ Failure | Redundant, it survives an AZ failure | Only through data replication or EBS snapshot | Cold standby based on replicated database snapshots. |
Netapp Ontap Enciclopedia
SnapMirror. This replication technology is the alternative to HANA System Replication with the benefit of not having an EC2 instance attached. FSx for ONTAP cant be implemented in Multi-AZ so that SnapMirror could be used for DR and Cloud Migration purposes.
SnapVault. Disk-based storage backup. Snapvault is a technology that could be used in combination with SnapCenter.
FlexCache. It's an inter-cluster caching for FSx and can work for Multi-region or from On-prem to AWS.
FlexClone. Cloning mechanism for volumes. FlexClone can be used for HANA refreshes.
Cloud Manager. A centralized user interface to manage, monitor, and automate ONTAP deployments in AWS. SnapMirror replication can be controlled from Cloud Manager.
Pricing for FSx for ONTAP
Users are billed for file systems based on the following categories:
- SSD storage capacity (per GB-month)
- SSD IOPS over 3 IOPS/GB (per IOPS-month)
- Throughput capacity (per megabytes per second [MBps]-month). 1024 MBps for SAP production needed
- Capacity pool storage consumption (per GB-month)
- Capacity pool requests (per read and write)
- Backup storage consumption (per GB-month)
Price for Single AZ comparison of FSx for Ontap vs. EBS gp3
Assume we want to store 10 TB of data in the US East (N. Virginia) Region for a production SAP HANA DB where 100% of the data is stored in SSD, single Az.
By default, 3k SSD IOPS are included for every GB of SSD storage in FSx for Ontap and gp3.
Assume to provision 1024 MBps of throughput capacity and have no backup data during the month, we would use backint or SnapCenter.
gp3 volume throughput can not exceed 1000 MBps, so we would need to stripe at least two volumes of 512 MBps to meet 1024 MBps on DATA or LOG
- Total gp3 cost 973 USD
- Total FSx for Ontap cost 2017 USD
PROs and benefits of ONTAP
- NetApp on-premises users can efficiently replicate HANA volumes to FSx for ONTAP to support disaster recovery without dedicated application servers.
- NetApp cloning features can be used for DR testing without interruption to verify that the defined Recovery Point Objective (RPO) can be met.
- Eliminates risks by performing efficient DR testing using native cloning technology without breaking the mirror.
- Leverages DR replication image for additional development/test system copies
- If appropriately configured, Application-consistent HANA backups leveraging snapshot copies create zero performance load.
- Use FlexClone for quick snapshots and QA Refresh.
- The combination of multiple SAP systems accessing the same NFS file share and the Host Auto Failover (HAF) makes this a compelling solution for HANA Scale-Out deployment scenario. Single AZ, Multi-Node
- In a classic EBS (non-NFS) Single AZ, Multi-Node deployment, each one of the EC2 will have its own dedicated storage for DATA and LOG volumes; with ONTAP, DATA and LOG can be shared between every EC2 node. This helps to considerably reduce the need for redundant EBS storage.
- For SAP HANA scale-out with Host Auto-Failover, NFSv4 locking is an excellent alternative to a server-specific STONITH (SAP HANA HA/DR) implementation.
- HANA Auto-Failover and EC2 auto-recovery should not be mixed. HAF is worth exploring for scale-out with ONTAP scenarios; its application-aware setup can be explored in the document linked in the last part of this blog
- HAF used nameserver heartbeat to check the status of the nodes, EC2 auto recovery uses cloudwatch
For Multi-region or hybrid deployments, by using Cloud Manager, we can simplify the data replication process by mirroring data between FSx for ONTAP file systems in different regions or replicating data from an on-prem ONTAP-based system to FSx for ONTAP using SnapMirror.
SnapMirror technology can be used for On-Prem ONTAP to Cloud ONTAP HANA migration.
Links
https://docs.aws.amazon.com/sap/latest/sap-hana/sap-hana-amazon-fsx.html
https://docs.aws.amazon.com/fsx/latest/ONTAPGuide/what-is-fsx-ontap.html
https://docs.netapp.com/us-en/ontap/pdfs/fullsite-sidebar/ONTAP_9_Documentation.pdf
https://fsxontap.calculator.aws/
SAP HANA Host Auto Failover configuration
- SAP Managed Tags:
