Hi again,

My name is Man-Ted Chan and I’m from the SAP HANA product support team. Recently I’ve been seeing a few issues in regards to High Availability (HA) environment using system replication so I’m writing this piece on setting up the HA along with some troubleshooting tips, and SAP notes.

To avoid confusion with the terminology I will refer to another posting on the SCN:

http://scn.sap.com/docs/DOC-52345

  • System Replication is NOT Host Auto-Failover
  • System Replication is NOT Scale Out
  • System Replication is Disaster Tolerance (DT) / Disaster Recovery (DR)
  • System Replication synchronizes data between two data centers (Site A and Site B)
  • There is always one (logical) primary and one secondary system, e.g. site A is primary and site B is secondary. After a takeover, site B is (logically) primary system. Thus, primary and secondary changes, whereas site A and B will refer to a physical instance.
  • A takeover is making a secondary system functioning as primary system. Note that this explicitly does not include changing the state of the primary (in exceptional/disaster situations, the secondary must not depend on having access to the primary site to be able to change the state)
  • Failback: back to original setup, e.g. a takeover from the backup site to the preferred site: the preferred site may have a better internet connectivity, better reachable by clients, etc.

Also I’ve had to break up this blog into two parts as I hit a limit on the number of images that can be in a single blog posting.

Pre-requisites

  • Have separate primary and secondary server with HANA installed with equal number of services and nodes. The revision of HANA on the secondary server has to be equal to or new than the primary.
  • Secondary system has the same SAP system ID and instance number.
  • Ports 3<instance number>15 and 3<instance number + 1>15 must be available
  • The primary server must have a backup available

Setting up System Replication

These are steps from, but done in an SP09 environment:

http://scn.sap.com/docs/DOC-47702

I have included screen caps, tests, and log snippets

Setting up primary
When setting up the system replication a backup needs to exists, as a test I will show what happens when there is no backup:

/wp-content/uploads/2015/09/1_794433.png

/wp-content/uploads/2015/09/2_794434.png

Right click on your primary system and select ‘Configure System Replication…’

/wp-content/uploads/2015/09/3_794456.png

/wp-content/uploads/2015/09/4_794457.png

/wp-content/uploads/2015/09/5_794458.png

/wp-content/uploads/2015/09/6_794459.png

As we can see we cannot proceed with the replication as there is no backup. In the next few images we will create the backup.

/wp-content/uploads/2015/09/7_794460.png

/wp-content/uploads/2015/09/8_794461.png

/wp-content/uploads/2015/09/9_796172.png

Afterwards try and create the replication again. Please note that field ‘Primary System Logical Name’ can be whatever you want, but I chose the name ‘primary’.

/wp-content/uploads/2015/09/10_794463.png

After this is ran the following can be found in the nameserver trace

==== Starting hdbnsutil, version 1.00.090.00.1416514886 (fa/newdb100_rel),

i Basis            TraceStream.cpp(00469) : MaxOpenFiles: 1048576

i Basis            TraceStream.cpp(00472) : Server Mode: L2 Delta

i Basis            ProcessorInfo.cpp(00713) : Using GDT segment limit to determine current CPU ID

i Basis            Timer.cpp(00650) : Using RDTSC for HR timer

i Memory          AllocatorImpl.cpp(01326) : Allocators activated

i Memory          AllocatorImpl.cpp(01342) : Using big block segment size 8388608

i Basis            TopologyUtil.cpp(03894) : command: hdbnsutil -sr_enable –name=primary –sapcontrol=1

w Environment      Environment.cpp(00295) : Changing environment set SSL_WITH_OPENSSL=0

i sr_nameserver TopologyUtil.cpp(02581) : successfully enabled system as system replication source site

If you wanted to use the command line to create this replication run the following:

hdbnsutil -sr_enable –name=< Primary System Logical Name>

After this your system is now enabled for system replication

Setting up the secondary node

You will have to stop the HANA servers on the secondary server prior to setting up the replication. Right click on the server and select ‘Configuration and Monitoring’->’Configure System Replication…’ again, please note that the SID is the same.

/wp-content/uploads/2015/09/11_794465.png

/wp-content/uploads/2015/09/12_794466.png

/wp-content/uploads/2015/09/13_794467.png

/wp-content/uploads/2015/09/14_794468.png

At this step you will name the replication in the ‘Secondary System Logical Name’ and enter the host from the above (note that the Instance number is non-editable)

Replication mode options that are available are the following:

  • Synchronous with full sync option (mode=sync. Full sync is configured with the parameter [system_replication]/enable_full_sync) means that log write is successful when the log buffer has been written to the logfile of the primary and the secondary instance. In addition, when the secondary system is disconnected (for example, because of network failure) the primary systems suspends transaction processing until the connection to the secondary system is re-established. No data loss occurs in this scenario.
  • Synchronous (mode=sync) means the log write is considered as successful when the log entry has been written to the log volume of the primary and the secondary instance.
  • Synchronous in memory (mode=syncmem) means the log write is considered as successful, when the log entry has been written to the log volume of the primary and sending the log has been acknowledged by the secondary instance after copying to memory.
  • Asynchronous (mode=async): The primary system sends redo log buffers to the secondary system asynchronously. The primary system commits a transaction when it has been written to the log file of the primary system and sent to the secondary system through the network. It does not wait for confirmation from the secondary system. This option provides better performance because it is not necessary to wait for log I/O on the secondary system. Database consistency across all services on the secondary system is guaranteed. However, it is more vulnerable to data loss. Data changes may be lost on takeover.

The above is from the SAP HANA Admin guide:

http://help.sap.com/saphelp_hanaplatform/helpdata/en/54/01f498b2c84fb5b3bcdcbda948d991/content.htm?frameset=/en/05/caef5b24794dc2accc4fb6561e26fa/frameset.htm&current_toc=/en/00/0ca1e3486640ef8b884cdf1a050fbb/plain.htm&node_id=440

This can be done via the command line

hdbnsutil -sr_register –remoteHost=<primary hostname> –remoteInstance= –mode=<sync/syncmem/async> – -name=< Secondary System Logical Name>

During this registration I ran into the following error

/wp-content/uploads/2015/09/15_794469.png

I then ran it via the command line to show the error

/wp-content/uploads/2015/09/16_794470.png

/wp-content/uploads/2015/09/17_794471.png

I checked the listed nameserver trace to see if there is any other information

==== Starting hdbnsutil, version 1.00.090.00.1416514886 (fa/newdb100_rel),

e Configuration    ConfigStoreManager.cpp(00693) : Configuration directory does not exist.

e Configuration   

  1. TopologyUtil.cpp(03894) : command: hdbnsutil -sr_register –remoteHost=xxxxx509 –remoteInstance=00 –mode=sync -name=sec

e sr_nameserver    TNSClient.cpp(06778) : remoteHost does not match with any host of the source site. all hosts of source and target site must be able to resolve all hostnames of both sites correctly

From this error we can see that the landscape between the two do not match. I checked the landscape in the primary and secondary

/wp-content/uploads/2015/09/18_794472.png

/wp-content/uploads/2015/09/19_794473.png

Here we can see that in secondary server there is the ‘sapstartsrv’ process. After this is resolved re-run the wizard or enter in the hdbnsutil command

/wp-content/uploads/2015/09/20_794474.png

‘Initial full data shipping’ is the equivalent to running hdbnsutil –sr_register –force_full_replica

If parameter is set, a full data shipping is initiated. Otherwise a delta data shipping is attempted.

If you run this via command line you will have to manually start up the secondary server

For more information on the hdbnsutiloptions please refer to the following reference guide

http://help.sap.com/saphelp_hanaplatform/helpdata/en/4d/8f09ec3f2c49f593d415c78e924d9b/content.htm?frameset=/en/52/913ed4a8db41aebef3ce4563c6f089/frameset.htm&current_toc=/en/00/0ca1e3486640ef8b884cdf1a050fbb/plain.htm&node_id=443

Checking the status of the replication

You can check the status of the replication in studio and via command line. The below screen caps

/wp-content/uploads/2015/09/21_794475.png

/wp-content/uploads/2015/09/22_794476.png

/wp-content/uploads/2015/09/23_794477.png

Check the name server trace to see following success messages upon startup to see the replication:

TREXNameServer.cpp(12634) : called registerDatacenter from registrator=xxxx301545c

i sr_nameserver    TREXNameServer.cpp(12776) : registerDatacenter; new disaster recovery site id =2

i sr_nameserver    TREXNameServer.cpp(12864) : matched host xxxx509 to xxxx301545c

i sr_nameserver    TREXNameServer.cpp(15138) : volume 1 successfully initialized for system replication

i sr_nameserver    TREXNameServer.cpp(15138) : volume 2 successfully initialized for system replication

i sr_nameserver    TREXNameServer.cpp(15138) : volume 4 successfully initialized for system replication

i sr_nameserver TREXNameServer.cpp(15138) : volume 3 successfully initialized for system replication

/wp-content/uploads/2015/09/24_794478.png

Please note that when you add the replication server you’ll notice that you cannot open the Administration panel or run SQL queries. So you will not be able to check the data in the replication server.

Instead you are opening Diagnosis mode, below the screen cap shows the difference between the 2


Diagnosis Mode

/wp-content/uploads/2015/09/25_794479.png

Admin Panel

/wp-content/uploads/2015/09/26_794480.png

Click here for part 2

To report this post you need to login first.

3 Comments

You must be Logged on to comment or reply to a post.

  1. David Wouters

    Hi Man-Ted Chan,
    Nice blog!  Thanks for sharing.
    One question: How exactly did you resolve below error?  I have the same error now but I see no difference in processes on primary and secondary.

    “remoteHost does not match with any host of the source site. all hosts of source and target site must be able to resolve all hostnames of both sites correctly”

    Thanks in advance,
    David.

    (0) 
  2. Matt Ambagtsheer

    “remoteHost does not match with any host of the source site. all hosts of source and target site must be able to resolve all hostnames of both sites correctly”

     

    We found this was our issue it relates to using the short name or FQHN

    2050830 – Registering a secondary system via HANA Studio fails with error ‘remoteHost does not match with any host of the source site’

    (0) 

Leave a Reply