VMWare based VM online cloning of an SAP System with the help of SAP Netweaver Landscape Virtualization Management (SAP NW LVM) Software HowTo
During the last couple of weeks we have spent a significant amount of time to enable VM based online cloning
of SAP systems with the help of SAP Netweaver Landscape Virtualization Management and i'd like to share my experience with you.
What is VM based Online cloning ?
SAP NW LVM supports two ways of cloning an SAP system:
1.) Storage based. All the SAP components including the Database files are installed on shared filesystems.
SAP NW LVM connects to the Storage System and initiates a storage based copy of the Data and mounts the copy
to a new compute node (which can be a virtual machine), including isolation on OS level. To ensure consistency
of the copied Database, the Database can either be shutdown (offline), put into Hot Backup mode, or Suspend I/O
mode depending on the Database type. This is achieved by executing corresponding Database scripts.
2.) VM based. All the SAP components including the Database files are installed inside a virtual machine using
virtual disks (vmdk's not RDM). SAP NW LVM connects to the Hypervisor Manager (Virtual Center) and initiates
a full VM clone including isolation on OS level. To ensure consistency of the copied Database, the Database needs
to be shutdown (offline) by default.
So, by default, in the vm based clone case you would have to live with a shutdown of the source system Database.
To avoid this downtime, we have developed a procedure that allows to keep the source Database online while
doing a VM based clone and ensuring Database consistency.
The procedure consists of two components.
1.) You have to enable the "online cloning" in SAP NW LVM with the help of a configuration parameter
described in SAP Note 1808432
2.) You have to make sure that the VMware tools are installed inside the Guest OS
So, how is consistency of the Database achieved ?
The solution uses the so called VSS (Volume Shadow Service) of the Windows OS to ensure filesystem and Database consistency.
The installed VMware tools will automatically trigger the VSS procedure inside the Guest OS during the clone/copy process.
If you want to verify that VSS is triggered in your VM, you can verify this via the Windows event log under Application. You see
VSS entries like "VSS-04107: successfully backed up component Oracle Database"
Windows VSS ? What about Linux ?
Unfortunately, Linux can currently not be supported by this method, since there is no similar mechanism for Linux supported by VMWare.
We are working very close with VMWare and with the other Hypervisor vendors for SAP NW LVM 2.0 to enable a solution, but this will
take some time.
So, what are the prerequisites:
VMware Vsphere 4.x and higher
VMware tools installed in the VM Guest OS and on the latest patch level
Windows 2008 R2 and higher as the Guest OS (Windows 2003 should work, but not tested)
SQL Server 2008 and higher (SQL Server 2005 should work, but not tested)
Oracle 11 and higher (The VSS writer has to be installed and active)
LVM 1.0 SP 6 and higher
LVM 2.0 SP 1 and higher
System Rename 1.0 SP3 and higher
All other supported DB/OS combinations will not work, since the underlying technology is not implemented.
We have done extensive testing in our lab and a couple of pilot implementations at customers and below are
a couple of lessons learned that might be helpful in your environment.
1.) I get a warning message in the Parameter entry Roadmap in SAP NW LVM, stating that "online cloning is not supported"
This message is to ensure that you have understood the prerequisites and confirm that your environment meets
them. If this is the case, you can click on "Ignore warnings for this step" on the top right of the step. Make sure
that there are no other warnings, that are relevant for your environment.
2.) When cloning Windows based systems it can happen, that the drive letters on the cloned system will be changed from the original drive letters,
e.g. E: gets F: and so on, and the copy process fails.This is a known issue caused by sysprep.
There are a couple of known work-arounds:
Install everything on C: (only recommended if a single disk is able to sustain enough I/O for your requirements)
Use junctions on C: e.g. C:/oracle/SID/SAPDATA1 and mount your disk on this directory in Disk Manager
Manually change back the drive letters on the clone in Disk Manager after the failure, and execute a re-try from the SAP NW LVM activities tab
(meaningful Volume names are helpful)
Export the registry key "HKLM_SYSTEM_MountedDevices on the source system, delete it on the target system and import it fromthe source export.
(Only recommended for advanced Windows admins.This could also be achieved with the help of a custom hook script)
3.) During the User management phase you get the error : "The security database on the server does not have a computer account for this workstation trust relationship."
This can be caused by the source machine making changes in the Active Directory during the clone process.
There are two workarounds: Start all over with a new clone/copy, or log onto the cloned VM , remove it from the Domain, re-join the Domain manually , reboot and
after Windows is up and running again do a retry of the clone/copy operation on the SAP NW LVM activities tab.
4.) The clone phase of the VM fails during the VM snapshot phase:
Error message during the provision phase "An error occurred while quiescing the virtual machine. See the virtual machine's event log for details"
Update to the latest VMware tools version, in most of the cases you can do a retry on the activities tab of SAP NW LVM and it will work.
5.) Configuration suggestions for the VM: Make sure there is no CD/DVD image mounted on the source VM's DVD Drive (the clone will fail). We also recommend to
not configure a floppy drive on the VM.
6.) Before you initiate a clone/copy operation from the SAP NW LVM Console, do a testrun on VMWare VCenter level (clone VM), this makes potential troubleshooting easier.
Additional information about SAP NW LVM can be found here:
http://scn.sap.com/docs/DOC-25771
Additional information about VMware and VSS:
VMware VSS implementation
http://pubs.vmware.com/vsphere-51/index.jsp?topic=%2Fcom.vmware.vddk.pg.doc%2FvddkBkupVadp.9.6.html&...
Microsoft VSSADMIN tool:
http://technet.microsoft.com/en-us/library/cc754968%28WS.10%29.aspxhttp://technet.microsoft.com/en-us/library/cc754968%28WS.10%29.aspx
I have left out the SAP NW LVM configuration parameter on this Blog Post on purpose. Please read the SAP note 1808432 before proceeding, it may change over time.
Michael Hesse
