Using sap_vendor_cluster_connector for interaction between cluster framework and sapstartsrv

[fabian_herschel]

A Problem and its Solution

One of the classical problems running SAP instances in a high available environment is that if a SAP administrator changes the status (start/stop) of a SAP instance without using the interfaces provided by the cluster software than the cluster framework will detect that as an error status and will bring the SAP instance into the old status by either starting or stopping the SAP instance. This can result into very dangerous situations, if the cluster changes the status of a SAP instance during some SAP maintenance tasks.

The solution is that the central component sapstartsrv, which controls SAP instances since SAP Kernel versions 6.4, will be enabled to communicate the state change (start/stop) to the cluster software.

The manual page for the sap_vendor_cluster_conenctor could be found at: http://scn.sap.com/docs/DOC-54845

The Components of the Solution

To enable sapstartsrv to communicate with the cluster product the process needs an additional interface to ensure proper functionality.

The involved components

For a standard start or stop of a SAP instance two major components are needed: a client program requesting the start or stop and sapstartsrv to process the request.

In the cluster integrated scenario there are now five components working together:

• Client (responsible: SAP)
  The client like sapcontrol, startsap, ACC, SAP Management Console and others are the same as in the standard scenario. For all mentioned client programs SAP is responsible. All those clients are typically part of the SAP delivery. All the clients must use the web-service interface of sapstartsrv. No client is allowed to work around of this server process. In other case the complete solution would be broken.

• Server (responsible: SAP)
  The server process sapstartsrv is providing web-service interfaces for starting, stopping SAP instances and also for getting information about the status of a SAP instance. There are a lot of detailed functions which can be called. You can use sapcontrol to get a list of available functions. This program is also delivered by SAP.

• Library (responsible: SAP)
  The new library saphascriptco.so for Unix (saphascriptco.dll for Windows) is loaded by sapstartsrv, if the parameter service/halib is used in the SAP instance profile. More details about the needed SAP parameters will follow in this article. sapstartsrv uses the library as a generic interface to a set of cluster products. The library saphascriptco.so takes the calls by sapstartsrv and matches them to calls to an external helper program: the cluster connector.

• Cluster connector (responsible: Cluster Vendor)
  The cluster connector sap_vendor_cluster_connector ( in the following I will use sap_suse_cluster_connector as an example) is called by the library saphascriptco.so and is an abstraction layer between the SAP sapstartsrv and the cluster product. The cluster connector is responsible to answer questions like “is a SAP instance controlled by the cluster?” or “please stop cluster resource rsc_mySAP”. While sapstartsrv and sapscriptco.so do not need anything to know about the cluster product the cluster connector needs to answer all queries or to “fire” cluster commands to change the status of cluster resources.

• Cluster product (responsible: Cluster Vendor)
  The last piece of the solution is the cluster product of course. Discussing feature sets, architectures and responsibilities of the several cluster products is not in the scope of this document. In the scope of this solution the cluster product must be able to answer the queries of the cluster connector and to process the “fired” changes of cluster resources.

A typical call flow for SAP instance start

To understand the flow of a SAP instance start in a cluster context the following diagram gives you a first overview.

Abbildung : Flow diagram of a SAP instance start request

Description of the work flow including the cluster connector:

• The administrator uses a client like SAP Management Console or sapcontrol to start a SAP instance. This request is sent via the SOAP based Start-WebService to sapstartsrv.

• Now sapstartsrv calls function SAP_HA_FindSAPInstance in library saphascriptco.so, if the cluster is responsible for this SAP instance. The library saphascriptco.so itself uses the cluster connector function lsr (details for lsr see call below) to answer that query. The cluster connector is the external system command like sap_vendor_cluster_connector provided by the cluster vendor. The library returns with SAP_HA_OK, if the cluster connector has found the SAP instance. If the cluster is not responsible sapstartsrv starts the SAPinstance using its own functionality.

• If the cluster is responsible for that SAP instance sapstartsrv uses the library saphascriptco.so to ask the cluster, if the requested cluster action (here start) is already in progress. This query has to be answered by the external cluster connector using the function cpa. The library saphascriptco.so takes the answer and returns SAP_HA_OK or SAP_HA_START_IN_PROGRESS back to sapstartsrv.

• Depending on the return code sapstartsrv decides to either request the cluster to start the SAPinstance or to do the start action by it’s own:

• If sapstartsrv got the return code SAP_HA_OK it uses the library to tell the cluster to change the resource status. The library will call the function fra (action start) of the cluster connector.

• If sapstartsrv got the return code SAP_HA_START_IN_PROGRESS it will continue to start the SAP instance by its own. This means that SAP_HA_START_IN_PROGRESS is the return code to tell sapstartsrv that it’s now ok to start the SAP instance.

A typical call flow for SAP instance stop

For the stop of a SAP instance the flow is nearly the same of course with action “Stop” instead of “Start”.

• The administrator uses a client like SAP Management Console or sapcontrol to stop a SAP instance. This request is sent via the SOAP based Stop-WebService to sapstartsrv.

• Now sapstartsrv calls function SAP_HA_FindSAPInstance in library saphascriptco.so, if the cluster is responsible for this SAP instance. The library saphascriptco.so itself uses the cluster connector function lsr (details for lsr see call below) to answer that query. The library return will SAP_HA_OK, if the cluster connector has found the SAP instance. If the cluster is not responsible sapstartsrv stops the SAPinstance using its own functionality.

• If the cluster is responsible for that SAP instance sapstartsrv uses the library saphascriptco.so to ask the cluster, if the requested cluster action (here stop) is already in progress. This query has to be answered by the external cluster connector using the function cpa. The library saphascriptco.so takes the answer and returns SAP_HA_OK or SAP_HA_STOP_IN_PROGRESS back to sapstartsrv.

• Depending on the return code sapstartsrv decides to either request the cluster to stop the SAPinstance or to do the stop action by its own:

• If sapstartsrv got the return code SAP_HA_OK it uses the library to tell the cluster to change the resource status. The library will call the function fra (action stop) of the cluster connector.

• If sapstartsrv got the return code SAP_HA_STOP_IN_PROGRESS it will continue to stop the SAP instance by its own. This means that SAP_HA_STOP_IN_PROGRESS is the return code to tell sapstartsrv that its now ok to stop the SAP instance.

Mapping sapstartsrv-Calls to Library functions and cluster connector commands

The status of the implementation for saphascript.co and sap_suse_cluster_connector version 1 and 2 and the mapping between the different layers is show in the following table:

API	sapstartsrv-Call	Library function in saphascript.so	sap_suse_cluster_connector command	Remark
1	FindSAPInstance	SAP_HA_FindSAPInstance	lsr –out FILE –sid SID –nr NR	Is used to find the cluster resource name for a specific SAP instance, which is located on the actual (calling) host.
-	FindDB	SAP_HA_FindDB	Not implemented in version 1 and 2	Not used so far.
1	StopCluster	SAP_HA_StopCluster	cpa –res SAPRESID –act stop fra –res SAPRESID –act stop	First the library checks, if a cluster actions is already in progress. If not it sends a cluster command to stop the resource.
1	StartCluster	SAP_HA_StartCluster	cpa –res SAPRESID –act start fra –res SAPRESID –act start	First the library checks, if a cluster actions is already in progress. If not it sends a cluster command to start the resource.
2	MoveCluster	SAP_HA_MoveCluster	fra –res SAPRESID –act migrate \ –node NODE	Move the requested SAP instance to the requested cluster node
-	FailCluster	SAP_HA_FailCluster	Not implemented in version 1 and 2	Fail Cluster is currently not called in sapstartsrv.
2	GetClusterNodes	SAP_HA_GetClusterNodes	lsn –out FILE –res SAPRESID	Find actual node running a resource and possible takeover nodes as well as the name of the node the call was requested from.
2	CheckConfig	SAP_HA_CheckConfig	hcc –out FILE –sid SID –ino INSTNO	Run HA solution specific configuration and status checks.
2	FreeConfigCheck	SAP_HA_FreeConfigCheck	--	No external script call needed.
2	GetVersionInfo	SAP_HA_ GetVersionInfo	gvi –out FILE	Provide information about HA product, SAP interface version and documentation links.