SAP On Azure : SAP Netweaver 7.5 on MS SQL Server ...

former_member642947 · ‎10-20-2020

Introduction

When we deploy SAP production environment in Azure, we need to address the Single Point of Failure, High Availability, Disaster Recovery and SLAs to make sure SAP systems meets the business requirements on these aspects.

This blog provides the guidance for setting up SAP Netweaver on MS SQL Server with stringent High Available and Disaster Recovery requirement for Production Environment. Blog is focused on MS SQL Server Database HA and DR setup and will not cover the details of SAP application layer setup. Though including references for SAP Application layer HA/DR options.

System Design

Following is the overall high-level design of the SAP Netweaver Production environment with HA & DR which is described in this blog.

Above design is based on Availability Set and provide VM level SLA of 99.95% for the pair. We need to have Database and SAP ASCS HA solution to automate the failover of these services between the pair of VMs using the AlwaysOn for DB and WSFC clustering for ASCS. Same design can also be applied in Availability Zones which provides VM level availability SLA of 99.99%. Refer the Azure VM SLA and Main page for Azure SLA.

Setup is extended to Secondary Region in Azure for Disaster Recovery and will also have full HA capabilities for DR environment to achieve 99.95% VM SLA. If the above DR design in deployed in AZ then we can achieve 99.99% of VM availability SLA.

This setup is based on the master SAP Note 1772688 for SQL Server AlwaysOn Availability Group for SAP Netweaver and need to be referred for latest updates.

Following are the details for each of the components of the setup:

MS SQL Server Database

HA – MS SQL Server AlwaysOn setup with 2 DB Nodes on WSFC cluster is deployed in Av Set to have HA for the DB of SAP Netweaver. Sync replication mode is used to achieve Zero data loss. Both of the VMs have same size SKU and disk layout for DATA and LOG files. An Internal Load Balancer(standard) in the primary region is used to define the listener Virtual IP of the AlwaysOn Availability Group.

DR – DR deployment done as an extension of the existing WSFC cluster in secondary region (in different vNet) with 2 additional DR nodes. All the 4 DB Nodes are part of AlwaysOn Availability group and replication mode should be ‘async’ for the DR DB node. STAR topology is used for replication so Primary DB will be responsible for shipping the data and keeping remaining 3 nodes up to date. An Internal Load Balancer (standard) in the secondary region is used for the listener Virtual IP.

Cloud witness for the WSFC cluster is created in the nearest Third azure region in a LRS storage account. This will ensure that cloud witness will still be available in case primary region failure.

We need to create 2 ILBs, one each in each region to define the virtual IP of the SQL AlwaysOn Listener which will be used in SAP profile parameters to redirect the connection from SAP application servers to Primary database. When DR systems is activated as Primary, SAP application servers in the DR region will use the ILB IP of DR region which is attached to Listener to connect to DB. Configuration of multiple listener IP are part of the AlwaysOn setup and no manual action will be required during DR failover. Refer to the related docs link.

SAP ASCS/ERS Layer

HA – SAP ASCS/ERS is deployed with WSFC cluster and ILB is used for defining Virtual IP/hostname of ASCS.

DR – Azure Site Replication (ASR) is used to replicate the VM to secondary region. ILB needs to be defined in the DR region to define the VIP/hostname of ASCS. During DR activation, WFCS Role for SAP need to be updated with new VIP before starting the services.

SAPMNT Layer

SAP system HA setup requires highly available ‘sapmnt’ which is mounted across ASCS/ERS and application server VMs. This setup is using Azure Netapp Files (ANF) which is PaaS fileshare service in Azure and very convenient to deploy. ANF Cross Region Replication (CRR) to be used to replication of ANF fileshare across regions.

Azure provides multiple other options to setup the ‘sapmnt’ fileshare like SOFS cluster, SIOS Datakeeper Fileshare, Azure File Share(premium). Azure Shared Disk is another option to setup the ASCS/ERS cluster with shared disk for ‘sapmnt’.

SAP Application Servers Layer

Multiple SAP Application server are deployed in Av Set for load balancing users and proving high availability using SAP logon groups. ASR is used for replication of VMs across region. Proximity Placement Group (PPG) is defined to co-locate SAP application and Database layer and minimize the latency.

Overview of Deployment Steps

Following are the high-level steps which are described in the blog for HA & DR setup of SAP NetWeaver with MS SQL Server.

Preparations

ANF SMB File Share for ‘sapmnt’

SAP ASCS/ERS HA Setup 4

MS SQL Server Database Setup

SAP PAS and AAS Installation

MS SQL Server AlwaysOn Cluster for HA & DR

Database High Availability Test

Disaster Recovery Test for the SQL Server DB

Preparations

Read the required Installation Guide, SAP Notes, SAP on Azure docs and download the installation media.

Deploy the VM in Availability Sets and include them in PPG as per the system architecture and Choose Operating System as Windows Server 2019.

Add data and log disks to SQL Server VMs and ‘\usr\sap’ disk to application servers(if needed).

Join the VMs to the Domain.

Define Page File in Temp Disk (D Drive).

Check that necessary Ports (including ILB Probe ports) are open in Windows firewall.

Disable the Continuous Availability Feature in Windows using the instructions in this link.

ANF SMB File Share for ‘sapmnt’

Follow the section ‘Create and mount SMB volume for Azure Netapp Files’ in the link to create the ‘sapmnt’ share in ANF.

ANF path(auto generated) in my setup is “\\contoso555-4b60.contoso555.com\sapmnt” which is having longer host name than permissible, so created an alias in DNS as ‘t01anfsapmnt’ and used this path(“\\t01anfsapmnt\sapmnt”) for SAP ACS, PAS and DB setup .

SAP ASCS/ERS HA Setup

Please follow the section “SAP ASCS/ERS HA Setup with Fileshare” from my previous blog and then return back to this blog for next actions.

In SWPM, Specify the Fileshare hostname as ‘t01anfsapmnt’ which is the DNS alias for the ANF fileshare hostname.

MS SQL Server Database Setup

In this section, we will install the MS SQL DB for the HA & DR setup and Create the DB Instances.

Perform the preparations steps in all the four VMs for the databases as mentioned in ‘prepare’ section above.

Install MS SQL DB software on all the 4 VMs in primary and DR region.
- Login to Node1(azwint01sqdb1) of SQL Server DB VM as administrator.
- Start the SQL DB install on VM1 and select the features.
- Keep the default Instance name

Alternatively, you can also create a named instance.

Define users and password for SQL DB services

Select the collation for SAP installation

Chosen the SQL admin user and Windows authentication mode.

Go to tab ‘Data Directories’ and Select the path for DATA, LOG and Backup.

Complete the SQL Server Installation.

In SQL Server Configuration Manager, Enable TCP/IP connection within SQL server Network Configuration.

Repeat the SQL server installation in all the remaining 3 DB VMs.

DB Instance Installation

Perform the database instance installation on VM1(azwint01sqdb1).

- Run SWPM to Install the DB Instance.
- Enter the Profile Directory location. In this case, its \\t01anfsapmnt\sapmnt\T01\SYS\profile.
- Enter the DB server hostname.
- Enter the SQL Server DB DATA & LOG file location. Its recommended to have DATA and LOG files in different disks.
- Complete the DB Instance installation.

Perform the Full database backup of Primary DB instance(T01) and restore the backup to all the 3 database hosts (azwint01sqdb2, azwint01sqdb3 and azwint01sqdb4) with ‘RESTORE WITH NORECOVERY’ option.

SAP PAS and AAS Installation

Perform Installation of the SAP Primary Application server and Additional Application servers(if required) in the respective VMs.

MS SQL Server AlwaysOn Cluster for HA & DR

Define the Internal Load balancer (standard) to define AlwaysOn Listener for HA and DR setup.

Create front-end IP, backend-pool, health probe, and loadbalancing rule for cluster roles.

Front-end IP	Backend Pool	Health probe port	Load balancing rule
10.20.10.145 (Listener Virtual IP - HA)	azwint01sqldb1 and azwint01sqldb2	64830	Enable HA Port, Enable Floating IP, Idle Timeout (30 Minutes)
10.40.20.145 (Listener Virtual IP - DR)	azwint01sqldb3 and azwint01sqldb4	64860	Enable HA Port, Enable Floating IP, Idle Timeout (30 Minutes)

Install Failover Clustering and .Net Framework 3.5 features on all the four cluster Nodes. This can be done in Server Manager -> Manage -> Add Roles and Features.

In DB VM1, Create the Failover Cluster and add Node1 (azwint01sqdb1) to the cluster.

If its Windows Server 2016, then update the cluster IP and start the cluster services.

Add the Cloud Witness.

Add all the remaining three Nodes to the cluster.

Perform the cluster validation test while adding each node. Also Remove the checkbox for ‘Add all eligible storage’ to cluster.

Enable AlwaysOn High Availability on all the nodes of SQL Servers from Config Manager.

Define AlwaysOn Availability group from SQL Studio.
- Add the HA nodes(Node1 & Node2) at first.
- Select the backup preferences.
- Choose the option to not creating the Listener for now as we setup it later.
- Choose the ‘join only’ option as primary db backup is already restored on secondary db.
- AlwaysOn Availability Group configuration is completed.
- Continue to Add Node3 and Node4 to the AlwaysOn group by selecting option ‘Add Replica’ in the SQL Studio AlwaysOn Availability Group. Make sure availability mode is set as ‘asynchronous’.
- We can the AlwaysOn status in the dashboard in SQL Studio.

Define the SQL Listener.
- Create a DNS entry for the listener hostname which needs to be mapped to frontend IP of the Internal Load Balancer.
- In the Failover Cluster manager, go to the Roles section. right click on defined Role -> Add Resource -> Client Access Point.
- Define the listener name which should be same as the DNS hostname of the listener.
- Update the static IP as ILB frontend IP.
- Add dependency by right-clicking on the Other Resources -> properties.
- Verify the SQL AlwaysOn Listener in SQL Studio. We should be able to see both the virtual IP defined in different regions. Else we can click on ‘add’ to include the entry for the listener IP in secondary region.

Update the Probe port of the cluster as its defined in the ILB.
- Update for Primary region Virtual IP

Check the parameter value of Probeport in the cluster resource. Its yet to be defined.

Update the probe port using the PowerShell commands.

Verify the parameter value of Probeport.

- Update for Secondary region virtual IP. Below screen shows the updated values.
- Below are the PowerShell commands to update the Probeport. Update the values accordingly and refer to the above screenshots for the values used in this case.

$ClusterNetworkName = “<value>"

$IPResourceName = "<value>"

$ILBIP = "<value>"

Import-Module FailoverClusters

Get-ClusterResource $IPResourceName | Set-ClusterParameter -Multiple @{Address=$ILBIP;ProbePort=<value>;SubnetMask="255.255.255.255";Network=$ClusterNetworkName;EnableDhcp=0}

- Restart the Cluster Role to activate the parameters.

Configure additional settings in AlwaysOn Cluster for SAP Application.
- These steps need to be performed on all the DB VM nodes when its acting as Primary. So ensure that failover is performed to make the DB Node as Primary and remaining Nodes as Secondary and then run the below steps.
- Start SWPM and Go to ‘Generic Options’ -> ‘MS SQL Server’ -> ‘Database Tools’ -> ‘Configure Additional AlwaysOn Node’.
- In next screens, provide the SAP Profile path, Domain Name and <sid>adm password and continue with the steps.

Update SAP Parameters

In DEFAULT.pfl file update/add the following parameters so that SAP connects to Primary DB using Listener virtual hostname/IP.

SAPDBHOST = azwinsqllsnrt01

dbs/mss/server = azwinsqllsnrt01

dbs/mss/conn_opts = MultiSubnetFailover=yes

In <sid>adm user environment, update/add the following parameters so that SAP connects to Primary DB using Listener virtual hostname/IP.

MSSQL_SERVER = azwinsqllsnrt01

MSSQL_CONNOPTS=MultiSubnetFailover=yes

Restart the Application Servers to activate the parameters.

Database High Availability Test

SQL DB HA can be tested by performing the failover from Primary to Secondary Node in the Primary region. SAP application will continue to run as usual.

Planned Failover

In SQL Studio, right-click on Primary Node of AlwaysOn Availability Group and click on ‘Failover’. Select the DB Node2 (azwint01sqdb2) and continue with the failover. Once failover is complete, SQL AlwaysOn dashboard will show the updated status.

Unplanned Failover

This can be done by crashing the Primary DB using NotmyFault tool. DB will failover to secondary node which was replicated using ‘sync’ replication.

Once VM is online and DB is up, Node will join back the availability group on its own.

Disaster Recovery Test for the SQL Server DB

DR will be invoked, when we failover the DB to Node3 or Node4 in another region due to outage in Primary region.

DR test Failover

In SQL Studio, right-click on Primary Node of AlwaysOn Availability Group and click on ‘Failover’. Select the DB Node3/4 and continue with the failover.

In SQL Studio, AlwaysOn Dashboard will show the below status. DB replication to other nodes will not be available.

To start the replication to other Nodes from the new primary node, we need to update the ‘Availability Mode’ and ‘Failover Mode’, so that same region nodes will have ‘sync’ replication and ‘automatic’ failover. Nodes in other region need to be replicated using ‘async’ mode and it will be ‘manual’ failover.

Click on OK to continue.

Now resume the data movement for all the secondary databases.

Click OK to continue.

In SQL studio, AlwaysOn dashboard will display all green status.

Note : There are some instances when SAP application server failed to connect to DB after 'async' failover to DR region and I found bellow messages in dev_w0 file.

C ERROR: -1 in function StartSelect (execute) [line 17558]
C (208) [42S02] [Microsoft][ODBC Driver 17 for SQL Server][SQL Server]Invalid object name 'SVERS'.
C ERROR: -1 in function StartSelect (execute) [line 17558]
C (8180) [42000] [Microsoft][ODBC Driver 17 for SQL Server][SQL Server]Statement(s) could not be prepared.

To fix the issue, Start SWPM on the new primary DB and Go to ‘Generic Options’ -> ‘MS SQL Server’ -> ‘Database Tools’ -> ‘Configure Additional AlwaysOn Node’.

For Failback of DB to Primary Region, we need to repeat the steps mentioned in this section to perform reverse failover of DB to previous Primary region.

DR test for SAP Application layer needs to be performed using Azure Site Recovery (ASR). ASR also provides an option to create DR test environment without disturbing the ASR replication to Secondary region.