SAP on Azure: SAP Netweaver 7.5 with ASE 16 on Lin...

former_member642947 · ‎11-30-2021

Introduction

SAP ASE database now supports Floating IP and Azure Standard Internal Load Balancer(ILB). High Availability(HA) setup of SAP ASE DB can now utilize this feature for SAP NetWeaver applications and non-SAP applications deployed in Azure.

This blog provides guidance on the design and deployment of HA setup of SAP ASE DB using SAP ASE replication server (REP) with Fault Manager for automatic failover for SAP Netweaver environment. SAP Application servers will be connecting to the SAP ASE DB using Virtual hostname/IP (floating IP) as defined in the standard load balancer. This blog describes the solution in Linux environment, but the ASE Floating IP with ILB solution is also available on Windows based setup. SAP ASE DB Always-On HA setup can be used for SAP Netweaver ABAP or JAVA based systems. Automatic Failover is only supported for a two node SAP ASE configuration.

Blog focuses on SAP ASE DB HA setup with ILB including Fault Manager and provides references to other parts of the overall setup like SAP ASCS/ERS HA, SAP Application Servers and fileshare for ‘sapmnt’ & ‘trans’.

System Design

The below diagram provides overview of the HA setup of SAP Netweaver with SAP ASE DB on Azure Cloud. Deployment is done across Availability Zones to achieve availability SLA of 99.99%. Same setup can also be performed using Availability Set which provides 99.95% of availability.

Following are the major components of the setup.

SAP ASE DB - ASE DB will be running be on 2 separate VMs with role as Primary DB and Secondary DB and deployed in different AZ. DBs replication (synchronous replication method) will be running from primary to secondary DB. DBs can automatically switch roles in the event of failover.

SAP application will connect to primary/active DB using the floating IP assigned to the Azure Internal Load Balancer(ILB). SAP ASE DB HA setup is independent of Operating System HA software such as Windows cluster and Pacemaker and require the Fault Manager performing the failover. Its also highly recommended to use latest version of ASE DB for HA setup and saphostcontrol for all the components.

Fault Manager - FM will monitor the availability of DB nodes and trigger switching of roles of DB from Secondary to Primary in the event of unavailability of Primary DB. An enhancement to Fault Manager configures and monitors the ILB IP and probe port assignment. The Fault Manager binary must be version 126 or higher. FM should be installed on ASCS cluster nodes and it will be highly available within the ASCS Cluster.

SAP ASCS/ERS Cluster – SAP ASCS/ERS is deployed on 2 nodes with Linux pacemaker cluster. Azure Fencing agent is the preferred option for fencing mechanism though SBD device can also be used as alternate option on SLES. Pacemaker will monitor the SAP ASCS/ERS and perform auto-failover of services in case of failure. We need to use Standard Load Balancer (ILB) to define the virtual IP addresses of ASCS and ERS.

Another important component is highly available ‘sapmnt’ and ‘trans’ filesystems. There are several options in Azure to configure it and related information are provided in the detailed setup section.

Summary of Deployment Steps

High level Installation process for HA setup SAP Netweaver 7.5 with ASE DB are following:

Preform the Preparatory Steps.

Install the ASCS/ERS on first and second Node of the Cluster including configuration of HA fileshare.

Install the SAP ASE Database (primary).

Install the SAP Primary Application Server and AAS.

Perform the installation of ASE Replication Server on the Primary DB.

Install the SAP ASE DB (Secondary).

Perform the installation of ASE Replication Server on the Secondary DB.

Post DB HADR Setup actions.

Deploy the Fault Manager and configure DB listener

Test the ASE DB failover (planned & unplanned).

Activate DBA Cockpit and confirm DB Server to Application Server Latency

This blog is focusing on setting up highly available SAP ASE DB and Fault Manager Installation with ILB configuration for DB listener and going to skip the detailed procedure to perform the SAP ASCS/ERS Cluster and Primary Application Server (links to existing documents are there in the relevant section).

Preparations

Read the required Installation Guide, SAP Notes, SAP on Azure docs and download the installation media.

Choose the Operating System image from Azure Marketplace and Deploy the VM in Availability Zone/Sets as per the system architecture.

Add data and log disks to ASE DB Server VMs and ‘/usr/sap/<SID>’ disk to application servers (if needed). Follow the disk layout recommendations for ASE DB in the link.

Define the swap space in the temp disk by configuring it /etc/waagent.conf. Refer to the link for details.

Follow the instructions in SAP Note 3013481, make changes to the tcp parameters at OS level. Also ensure the SAP ASE SRS version (SRS 16.0 SP03 PL07 and higher) after the deployment.

SAP ASCS/ERS HA Setup

Please follow the below Microsoft documentation to deploy SAP ASCS/ERS HA setup. Referring SUSE Linux specific docs links and similar solution for RHEL Linux is also available in Microsoft documentation.

Basic Pacemaker setup

Setting up Pacemaker on SLES in Azure - Azure Virtual Machines | Microsoft Docs

Choose one of the solutions based on your design.
- With NFS Cluster for sapmnt

High availability for NFS on Azure VMs on SLES - Azure Virtual Machines | Microsoft Docs

Azure VMs high availability for SAP NetWeaver on SLES - Azure Virtual Machines | Microsoft Docs

- With ANF for sapmnt

Azure VMs high availability for SAP NetWeaver on SLES - Azure Virtual Machines | Microsoft Docs

- With Azure Files NFS – now fully released and generally available

Deploy SAP ASCS/ERS with Azure Files NFS v4.1 shares - Microsoft Tech Community

For Disaster Recovery setup of ASCS/ERS, you may refer to this blog.

SAP ASE DB (Primary) Setup

Install the below linux packages.

sudo zypper install glibc-32bit

sudo zypper install libgcc_s1-32bit

Mount the ‘sapmnt’ filesystem on the VM.

Proceed with SWPM based deployment of ASE DB.

sudo <swpm>/sapinst SAPINST_REMOTE_ACCESS_USER=<username>

SAP Application Server Setup

Mount the ‘sapmnt’ filesystem on the VM.

Proceed with SWPM based deployment of SAP PAS & AAS.

sudo <swpm>/sapinst SAPINST_REMOTE_ACCESS_USER=<username>

DB Replication Setup on Primary DB

Login as OS user ‘syb<sid>’

Create/update the DMA & setuphadr files. Sample file are attached.

Link to download sample file 'sybdb1_dma.rs' and 'sybdb1_setup_hadr.rs'

Update SID, hostname, userid, passwords,port numbers and path in the sample files.

Execute the below command with complete path of SAP ASE DVD folder to setup the DMA.

setup.bin -f /sybase/sybdb1_dma.rs -i silent

Unlock the user

isql -Usapsso -P<password> -SP<SID> -X

>sp_locklogin sa, unlock

>go

Setup the HADR

cd /sybase/<SID>/ASE-16_0/bin

setuphadr /Sybase/sybdb1_setup_hadr.rs

Setup up ASE HA on Secondary Node

Install the below linux packages.

sudo zypper install glibc-32bit

sudo zypper install libgcc_s1-32bit

Mount the ‘sapmnt’ filesystem on the VM.

Start SWPM to setup the replication environment. Select ‘SAP Netweaver’ > ‘SAP ASE’ > ‘Database Replication’ > ‘Setup of Replication Environment’

Provide the ASE DB details and select ‘Setup a secondary database instance’

If you get the below error, rename the DEFAULT.PFL till you move to next screen in SWPM.

Continue with SWPM installation steps to complete the instance setup.

Login as user syb<sid>

Create/update the DMA & setuphadr files. Sample file are attached.

Link to download sample file 'sybdb2_dma.rs' and 'sybdb2_setup_hadr.rs'

Update SID, hostname, userid, passwords, port numbers and path in the sample files.

Execute the below command with complete path of SAP ASE DVD folder to setup the DMA

setup.bin -f /sybase/sybdb2_dma.rs -i silent

Unlock the user

isql -Usapsso -P<Password> -S<SID> -X

>sp_locklogin sa, unlock

>go

Setup the HADR

cd /sybase/<SID>/ASE-16_0/bin

setuphadr /Sybase/sybdb2_setup_hadr.rs

Post Setup Actions for ASE DB HA

Update the DR_Admin password in Secure Store

In both the DB Nodes. Login as <sid>adm user and run the following command.

rsecssfx put DB_CONNECT/SYB/DR_USER DR_admin -plain

rsecssfx put DB_CONNECT/SYB/DR_PASSWORD <password of DR_admin>

Configure Replication Server with sap_tune_rs

Perform these steps on the Primary and Secondary DB servers to configure Replication Server to specify the maximum number of the CPUs and the maximum size of the memory for the Replication Server instance.

In DB Node 1 & 2, Login as syb<sid> and run the following command

- Login to RMA

isql -UDR_admin -P<password> -S<hostname>:4909

- Run the Following Command (choose the suitable values)

sap_tune_rs Site1, 6, 2 (In Node 1)

sap_tune_rs Site2, 6, 2 (in Node 2)

Check the Status of DB Replication

/usr/sap/hostctrl/exe/saphostctrl -user sapadm <password of sapadm> - -function LiveDatabaseUpdate -dbname <SID> -dbtype syb -updatemethod Check -updateoption TASK=REPLICATION_STATUS

Deploy the Fault Manager

Fault manager needs to be deployed in active ASCS Node. Once successfully installed, it will bind itself with active ASCS node and can failover between ASCS cluster nodes.

Deploy the Standard Load Balancer for virtual IP/hostname of DB listener.

(Below table is for reference)

Front-end IP	Backend Pool	Health probe port	Load balancing rule
10.20.10.145 (Listener Virtual IP - HA)	azlnxasedb01 and azlnxasedb02	63500	Enable HA Port, Enable Floating IP, Select Session persistence as Client IP & protocol, Idle Timeout (30 Minutes), TCP Reset Disabled

Assign the hostname to Virtual IP in DNS or in /etc/hosts.

Additional Notes about Standard Load Balancer :

- Do not enable TCP timestamps on Azure VMs placed behind Azure Load Balancer. Enabling TCP timestamps will cause the health probes to fail. Set parameter ipv4.tcp_timestampsto 0. For details see Load Balancer health probes.
- Floating IP is not supported on a NIC secondary IP configuration in load-balancing scenarios. For details see Azure Load balancer Limitations. If you need additional IP address for the VM, deploy a second NIC.
- When VMs without public IP addresses are placed in the backend pool of internal (no public IP address) Standard Azure load balancer, there will be no outbound internet connectivity, unless additional configuration is performed to allow routing to public end points. For details on how to achieve outbound connectivity see Public endpoint connectivity for Virtual Machines using Azure Standard Load Balancer in SAP high-ava....

Download the latest version of ASE DB kernel SAR File SYBCTRL.SAR and extract it. Copy the extracted files to the /sapmnt/<SID>/exe/uc/linuxx86_64 directory. Make sure ‘sybdbfm’ version is 126 or higher.

Login to ASCS Node as <sid>adm and install the FM.

>>cdexe

>>./sybdbfm install

FM installation updates the ASCS Instance profile and create a FM profile file SYBHA.pfl in the profile directory.

Update the SYBHA.pfl in the profile directory (/sapmnt/<SID>/profile) by adding the following parameters.

ha/syb/support_floating_ip = 1

ha/syb/vdbhost = <vip ip address or vhostname>

ha/syb/az_ilb_port = <probe port of ILB>

ha/syb/failover_if_unresponsive = 1

Restart the Fault Manager by executing command ‘sybdbfm restart’ from ASCS’s ‘work’ directory. Restart can also be achieved performing ASCS failover to node2. This way we can also ensure that FM runs on either of the ASCS/ERS nodes along with ASCS.

Check the version of sybdbfm in following folders are up to date (version 126 or higher) in ASCS and DB nodes.

/sapmnt/<SID>/exe/uc/linuxx86_64

/usr/sap/<SID>/ASCSxx/exe/

/sybase/<SID>/sapdbctrl-config/hb (in both of the DB nodes)

We can run the command ‘sybdbfm version’ to check the version of the file in these folders.

Check the status of the FM.

cd /usr/sap/<SID>ASCSxx/work

sybdbfm status

Ensure that VIPA(virtual IP assignment) status is OK.

Update the Default profile and environment parameters of Application servers
- Updates to DEFAULT.PFL

dbs/syb/server = <virtual hostname of DB>

j2ee/dbhost = <virtual hostname of DB>

- Environment parameters

dbs_syb_server=<virtual hostname of DB>

dbs_syb_ha = 1

Test the ASE DB failover

Planned failover

Login to one of the DB Node as user syb<sid> and run the following command

/usr/sap/hostctrl/exe/saphostctrl -user sapadm <password of sapadm> - -function LiveDatabaseUpdate -dbname <SID> -dbtype syb -updatemethod Execute -updateoption TASK=FAILOVER -updateoption FAILOVER_FORCE=1 -updateoption FAILOVER_TIME=30

FM will detect the failover and show the status as Primary role switches to DB Node2. FM will also move the floating/virtual IP (ha/syb/vdbhost) to the new primary.

Unplanned Failover
- We can crash the ASE DB (Primary) by running the OS command ‘ifdown eth0’. FM will detect the unavailability of Primary DB and activate the secondary DB as Primary. SAP application servers will auto connect to the newly active DB using ILB based virtual IP which will now get attached to the new Primary DB.

FM will show following status.

Once the old Primary VM is restarted. Start the ASE DB and Replication server Services.

/usr/sap/hostctrl/exe/saphostctrl -user sapadm <password of sapadm> -function StartDatabase -dbname <SID> -dbtype syb

/usr/sap/hostctrl/exe/saphostctrl -user sapadm <password of sapadm> -function StartDatabase -dbname <SID>_REP -dbtype syb

Login to OS user syb<sid>.

Open the command prompt and login to RMA

isql -UDR_admin -P<password of DR_admin user> -S<hostname>:4909

and run the command

>>sap_host_available <site name>

>>go

Once the command is completed, FM will show that DB sync is OK.

- Another test is the kill the ‘dataserver’ process in the primary DB node. FM will trigger the failover of the DB to the secondary node and also move/attach the virtual IP to secondary node. SAP Applications will not require to be restarted. In the Old primary, we need to start the ASE DB, rejoin the replication as secondary as explained in the previous test scenario.

References

3086679 - SYB: Fault Manager: floating IP address on Azure

1959660 - SYB: Database Fault Management

2906375 - SYB: Database Fault Management: testing the Floating IP

SAP ASE Azure Virtual Machines DBMS deployment for SAP workload - Azure Virtual Machines | Microsoft...

SAP Note 2912040 - SAP ASE on Microsoft Azure: Documentation by Microsoft

SAP Note 2134316 - Can SAP ASE run in a cloud environment? - SAP ASE

SAP Note 1941500 - Certification information for Linux and other Operating Systems - SAP ASE

SAP Note 1891560 - SYB: Disaster Recovery Setup with SAP Replication Server

SAP Note 1928533 - SAP Applications on Azure: Supported Products and Azure VM types