SAP Landscape Management (LaMa) & Ansible Tower/AW...

naeem_maqsud · ‎04-12-2023

Introduction

In this blog I will cover the scenario of provisioning a new instance in a public cloud directly from LaMa and then using this new host to perform an operation (e.g. new HANA replication tier, add application server, system copy/clone).

I will use AWS (Amazon Web Services) as the public cloud and Ansible AWX (upstream open source project for Ansible Tower) to launch the playbook for creation of a new instance. The same approach also applies to Azure or GCP. The module used in the playbook can be replaced by those for the other cloud providers and some other minor adjustments.

In a previous blog I described the outbound REST API feature of LaMa that was first added in SP25.

The ansible playbook (for use in Ansible AWX or Tower) and the LaMa configuration (provider definitions, custom operations etc.) are available at GitHub .

Scenario Description

The REST provider that is part of automation studio will be used to trigger the new host provisioning via Ansible AWX, followed by a discovery and then using the new host to provision an additional application server.

It is also possible to use the newly added host for other provisioning tasks such as system copy or new replication tier.

The diagram below illustrates at a high-level how the flow looks like.

It should be noted that when using some cloud adapters in LaMa (such as the Cloud Manager for AWS), it is already possible to provision new cloud instances as part of tasks like "Install Application Server" or "System Copy". The choices presented are to use existing hosts (already discovered in LaMa as ready to use) or provision a new host. However, the built-in process does not always meet the requirements for some customers. For example there may be a need to customize the instance further before it is used. In this case this alternative approach with Ansible can be used.

Disclaimer

This blog is published “AS IS”. Any software coding and/or code lines / strings (“Code”) included are only examples and are not intended to be used in any productive system environment. The Code is only intended to better explain and visualize the features of the SAP Landscape Management Automation Studio. No effort has been made to make the code production quality (e.g. security, error handling, robustness, etc). If you use any of the code shown, you are doing it at your own risk.

Information in this blog could include technical inaccuracies or typographical errors. Changes may be periodically made.

Ansible Playbook

This is the playbook on GitHub.

The blog does not cover how to configure Ansible AWX.

The following key configuration steps need to be performed and relevant guides consulted. These are just the configuration steps performed for this scenario and not necessarily the only way to do it.

Add SSH credentials

Add AWS credentials

Add Inventory

Add host -- optional as the playbook itself will make the entries but you add an existing host to just have an entry in addition to localhost

Add Project (source control type = Git; Source control URL = url)

Add Job Template (assign above Project and Inventory; Also assign above credentials - SSH and AWS)
- A Survey also needs to be added and enabled - described further down

Create a new Application (Note down the Client ID and Client Secret - we will need it in LaMa)
- Authorization grant type: Resource owner password-based
- Client type: Confidential

Figure out your job template ID and note it down.

The playbook takes the following parameters and should be added as "Survey" in the job template:

i_name (name assigned to the instance as well as the hostname at OS level)

i_type (the AWS instance type that matches your requirements - e.g. t2.xlarge)

i_subnet (the subnet id where your SAP landscape is installed)

i_secgrp (the security group id applicable to your landscape)

i_keypair (the private key file that has also been stored in Ansible AWX to allow SSH)

i_ami (the image file to use - this can be an image that you have taken from an existing server to act as a starting point for further configurations as needed)

lama_ip (IP address of LaMa)

awx_username (credential for AWX/Tower)

awx_password (credential for AWX/Tower)

When calling from LaMa the initial authentication credentials needed to launch the playbook will also be passed automatically. So the input of credentials is only once.

The playbook has multiple plays within it and you can get the detailed description of each play at the GitHub site.

Prior to configuring LaMa, test the template directly in AWX/Tower web UI. If it successfully creates a new instance then move to the LaMa section. Remember to terminate the test instance, cleanup the host file of LaMa, and remove the DNS entries (if you kept that section of the playbook).

Provider Definitions (calls to AWX)

You can get the provider definitions xml files at GitHub

Retrieves the Ansible AWX token and acts as supplier to the main provider definition

Create a Provider Definition that retrieves a token
- Provider Type: REST
- Provider Subtype: Token Supplier
- URL: http://<AWX host>:<AWX port>/api/o/token/

- Method: POST

- Authentication: BASIC
  - User Name: <Client ID> -- previously created in AWX
  - Password: <Client Secret> -- previously created in AWX

- Request Configuration - Header Entry

Request Body:

grant_type=password&username=${PARAM_username}&password=${PARAM_password}&scope=write

Add a Response Parser

Obtaining the Ansible AWX job results

Create a Provider Definition that monitors the AWX job and knows when it has completed or if there was an error. It does this by making multiple GET calls with a defined interval.
- Provider Type: REST
- Provider Subtype: Result Retriever
- URL: http://<AWX host>:<AWX port>/api/<REF-jobLocation>

- Method: GET

- Authentication: DYNAMIC TOKEN

- Token Type: Bearer
  - When saved it will show as all CAPS but must be entered as shown above

- Token Supplier: Select the previously create provider definition for token retrieval

- Request Configuration - Default

- Add a Response Parser

Executing the ansible playbook

Create a Provider Definition to make REST API call to Ansible AWX
- URL: http://<AWX host>:<AWX port>/api/v2/job_templates/<template ID>/launch/
  - Get the template ID by clicking on the template in AWX and looking in the url bar. The ID is the number after the job template.

- Operation Uses: Instance

- Provider Type: REST

- Provider Subtype: Standalone

- Method: POST

- Authentication: DYNAMIC TOKEN

- Token Type: Bearer
  - When saved it will show as all CAPS but must be entered as shown above

- Token Supplier: Select the previously create provider definition for token retrieval

- Result Retriever: Select the previously create provider definition for obtaining AWX job results
  - Timeout (Milliseconds): 600000
  - Sleep Time (Milliseconds): 15000

- Request Configuration - Default

- Request Body:

{

    "extra_vars" :

    {

        "i_name": "${PARAM_hostname}",

        "i_type": "t2.xlarge",

        "i_ami": "ami-xyz",

        "i_subnet": "subnet-xyz",

        "i_secgrp": "sg-xyz",

        "i_keypair": "xyz",

        "lama_ip": "xyz",

        "awx_username": "${PARAM_username}",

        "awx_password": "${PARAM_password}",

        "awx_host_name":  "${PARAM_hostname}"

    }

}

- Add Response Parsers

- Add Parameters
  - username for AWX (in the example 'admin' is used but can be a different account that is authorized to run a job template
  - password is for the above account
  - hostname refers to the new cloud instance

Custom Operation: Create VM (call to AWX)

In order to execute the REST API call to Ansible AWX and execute the playbook, we need to create a custom operation from the provider definitions created earlier.

LaMa Provisioning Blueprint

The provisioning blueprint needs to be created and then parameterized. The blueprint is not saved to Github and should be created from scratch in your own environment. Read the relevant section in LaMa Guide. The steps are below.

Provision a new application server using an existing host. You can refer to this blog for the process
- At the end of the roadmap and just before executing, choose the option to create a blueprint
- Proceed with the execution to ensure that the blueprint is valid and leads to a successful application server installation

Now edit the blueprint
- Automation Studio -> Provisioning Blueprints and click edit on the above created blueprint
- Click on Parameters tab
- In the "Custom Placeholders section at the bottom you already have parameter listed - "ASInstallSapinstStep_abapSchemaUserPassword"
- Add the following additional parameters by first making changes
  - Name of new host -> "host" : "$[as_hostname]"

- - Name of virtual hostname to assign to the above host (this virtual hostname should already exist)

- - Click on Refresh and this will transfer the new parameters to the Custom Placeholders section

It is also recommended to parameterize the instance number so that you can use the same blueprint to create more application servers. Just find the entry and change the value to a parameter. I left it out as I'm only using it for demo purposes.

Provider Definitions (calls to LaMa)

This section covers the provider definitions for making calls from LaMa to LaMa (outbound and inbound rest APIs)

Obtaining the LaMa job results

Create a Provider Definition that monitors the LaMa activity and knows when it has completed. It does this by making multiple GET calls with a defined interval.
- Provider Type: REST
- Provider Subtype: Result Retriever
- URL: http://<lama host>:<lama port>/lama-api/<REF-jobLocation>

- Method: GET

- Authentication: BASIC

- Request Configuration - Header Entries
  - Content-Type: application/json
  - x-csrf-token (blank value)

- Add Response Parsers
  - Type "Recurring_Execution", name "status_executing" and JMES Path "status == 'executing'"
  - Type "Success_State", name "status_completed" and JMES Path "status == 'completed'"

Discovery of new VM

Create a Provider Definition to make REST API call to itself (LaMa to LaMa)
- URL: http://<lama host>:<lama port>/lama-api/discovery

- Operation Uses: Instance

- Provider Type: REST

- Provider Subtype: Standalone

- Method: POST

- Authentication: BASIC

- Result Retriever: None
  - As there is no activity id we cannot use it for this
  - Instead we will use a sleep step via script registered with host agent

- Request Configuration - Header Entries
  - Content-Type: application/json
  - x-csrf-token (blank value)

- Request Body:

{

  "entityDetectionType": "Hosts",

  "hostnames": [

    "${PARAM_i_name}"

  ],

  "detectInstancePhysicalHost": true,

  "connectionTimeout": 0,

  "socketTimeout": 0,

  "hostAgentSettings": {

    "useSecureCommunication": false,

    "port": 1128,

    "httpProxy": "",

    "authenticationType": "Basic",

    "userName": "sapadm",

    "password": "${PARAM_ha_password}"

  },

  "pool": "us-east-1d",

  "autoAssignInstances": true,

  "storeEntities": true

}

- Response Parsers - none

- Add Parameters
  - i_name - name of the new host
  - ha_password - sapadm password to be used for host agent authentication

Trigger Blueprint

Create a Provider Definition to make REST API call to itself (LaMa to LaMa)
- URL: http://<lama host>:<lama port>/lama-api/activities

- Operation Uses: Instance

- Provider Type: REST

- Provider Subtype: Standalone

- Method: POST

- Authentication: BASIC

- Result Retriever: LaMa Job Results
  - Timeout: 600000
  - Sleep Time: 15000

- Request Configuration - Header Entries
  - Content-Type: application/json
  - x-csrf-token (blank value)

- Request Body:

{

    "operation": "NM_Install_Application_Server",

    "validateOnly": false,

    "type": "blueprint",

    "ignoreWarnings": false,

    "parameters": [

        {

            "key": "ASInstallSapinstStep_abapSchemaUserPassword",

            "value": "${PARAM_schema_password}",

            "isSecure": true

        },

        {

            "key": "as_hostname",

            "value": "${PARAM_as_hostname}",

            "isSecure": false

        },

        {

            "key": "virt_hostname",

            "value": "${PARAM_virt_hostname}",

            "isSecure": false

        }

    ]

}

- Response Parsers

- Add Parameters
  - schema_password - ABAP schema password
  - as_hostname - new VM hostname
  - virt_hostname - virtual hostname to be attached to the new VM

Provider Definition for Hook: Sleep 120 Seconds

Create a Provider Definition using script registered with host agent to sleep for 120 seconds to give enough time for discovered host to be validated
- Specify a Central Host such as the central instance

Name:sleep

Command: /usr/bin/sleep 120

ResultConverter: flat

Platform: Unix

Custom Operation: Discover VM (call to LaMa)

Using the provider definition created earlier to discover the new host, create a custom operation

Custom Operation: Trigger Blueprint via REST (call to LaMa)

Using the provider definition created earlier to trigger the blueprint, create a custom operation

Custom Hook: Sleep 120 Seconds

Use the earlier provider definition for sleep, create a Post Hook operation. The hook applies to the earlier created operation for VM discovery.

LaMa Custom Process

Now that we have the custom operations created, we can join them together to create a custom process that allows us to perform an end-to-end execution.

Step 1 - Create VM Operation Parameters -- this is the custom operation that makes the REST API call to AWX
- All are set to global parameters for this step - will prompt at execution

Step 2 - Discover VM Operation Parameters -- this is the custom operation that makes a REST API call to itself (i.e. LaMa to LaMa) and triggers a discovery
- Hostname: Quick Binding to "hostname" - see step 1
  - Hostname = hostname (New VM hostname)
- host agent password: Global Parameter - will prompt at execution

Step 3 - Blueprint execution to install application server -- this is a REST API call to itself (LaMa to LaMa) that triggers the provisioning blueprint to perform an AS install using the newly provisioned VM
- Schema password: Global Parameter - will prompt at execution
- New VM Host: Quick Binding to "hostname" - see step 1
  - new_host = hostname (New VM hostname)
- Virtual Hostname: Global Parameter - will prompt at execution

Note that the Post Hook is inserted automatically between step 2 and step 3.

LaMa Custom Process Execution

We can now execute the custom process from any instance. It does not matter which instance is used to trigger it as none of the three steps require any info related to an instance or system.

The custom operations that were created can also be executed independently. For example if you only want to trigger a new VM provisioning, then you can just use the custom operation. The custom process combines the two custom operations (create VM, discover VM) with the operation that triggers the blueprint for application server install.

Conclusion

What has been shown is how you can integrate infrastructure provisioning via Ansible AWX/Tower to workflows in LaMa. This example can be extended to cover other scenarios that need an empty target host such as system clone/copy and HANA replication tier. Once the custom process is in place, it can be used multiple times.

With the XML files on GitHub, you can get started quickly and get a feel of the feature.

References

Samples on GitHub (for this blog)

LaMa Automation Studio Provider Definitions

Editing LaMa Blueprints

JMESPath

Ansible AWX and Tower API

Ansible AWX