Automating SAP BTP setup with Terraform – A workaround for Global Account, Directories and Subaccounts with btp CLI

[mauriciolauffer]

UPDATED: SAP has released a Terraform provider for SAP BTP! Terraform provider for SAP BTP now available for non-productive use. Follow this to learn how to use: Automating SAP BTP setup with the new Terraform provider for SAP BTP

 

 

In my previous blog, Automating SAP BTP setup with Terraform – Infrastructure as Code for Cloud Foundry and Kyma Environm..., I introduced Infrastructure as Code (IaC) and Terraform in SAP BTP context. I explained how Terraform Providers can automate pretty much everything in Cloud Foundry and Kyma (Kubernetes) environments. I described some use cases and showed a step-by-step on how to use Terraform to provision a CF space with services and users configured.

I also said that "At this moment, there isn’t any Terraform provider created specifically for SAP BTP. Which means everything above CF or Kyma isn’t supported, e.g., global account and directories", and that I'd create my own Terraform Provider for BTP. I still haven't got there yet, but I bring some good news.

Terraform allows you to run shell scripts, local and/or remote. This capability is called Provisioners and its use case is for anything not supported by Providers, but doable via any CLI. Provisioners don't work as Providers, they don't fully control a resource state, they have heaps of limitations and are the last resort for when you need to do something which is impossible via a Provider.

That said, I have managed to use Terraform Provisioners with btp CLI to control everything BTP related that is not supported by a Provider: global accounts, directories, subaccounts, entitlements, platform users, etc.

If you don't know what btp CLI is, here a brief description from the help page:

Use the SAP BTP command line interface (btp CLI) for all account administration tasks, such as creating or updating subaccounts, authorization management, and working with service brokers and platforms. It is an alternative to the SAP BTP cockpit for all users who like to work in a terminal or want to automate operations using scripts.

 

How to use it?

Using Terraform Provisioners is quite easy. As we don't have a proper Provider for SAP BTP, we'll use resource null_resource, and, because we're running btp CLI from our local computer, we'll use provisioner local-exec. The command property is the CLI command you want to execute.

resource "null_resource" "example" {

  provisioner "local-exec" {

    command = "echo 'You can execute any shell command here'"

  }

}

 

Why should I use it?

With this approach you can have everything in just one place. All your SAP BTP setup managed and versioned in a git repo. No need to have multiple tools nor perform heaps of manual tasks. You can have reusable recipes for common environments such as training space for solution XYZ, on-demand testing instances or a free-for-all development sandbox.

 

Demo time!

Make sure you have Terraform and btp CLI installed in the computer running the script.

Note that I'm using Terraform variables to make the examples more generic and closer to what we would see in the real world.

The first script will create a BTP subaccount.

variable "username" {}

variable "password" {}

variable "global_account_subdomain" {}

variable "subaccount_region" {}

variable "subaccount_subdomain" {}

variable "subaccount_name" {}



resource "null_resource" "btp_subaccount_creation" {

  provisioner "local-exec" {

    command = "btp login --url https://cpcli.cf.eu10.hana.ondemand.com --user ${var.username} --password ${var.password}"

  }



  provisioner "local-exec" {

    command = "btp target --global-account ${var.global_account_subdomain}"

  }



  provisioner "local-exec" {

    command = "btp create accounts/subaccount --display-name ${var.subaccount_name} --region ${var.subaccount_region} --subdomain ${var.subaccount_subdomain}"

  }

}

 

The second script assumes a subaccount already exists, it will change its entitlement adding Cloud Foundry to it and then provision the CF environment.

variable "username" {}

variable "password" {}

variable "global_account_subdomain" {}

variable "subaccount_id" {}

variable "cf_org_name" {}



resource "null_resource" "btp_cf_setup" {

  provisioner "local-exec" {

    command = "btp login --url https://cpcli.cf.eu10.hana.ondemand.com --user ${var.username} --password ${var.password}"

  }



  provisioner "local-exec" {

    command = "btp target --global-account ${var.global_account_subdomain}"

  }



  provisioner "local-exec" {

    command = "btp assign accounts/entitlement --to-subaccount ${var.subaccount_id} --for-service cloudfoundry --plan free --amount 1"

  }

  

  provisioner "local-exec" {

    command = "btp create accounts/environment-instance --subaccount ${var.subaccount_id} --display-name ${var.cf_org_name} --environment cloudfoundry --service cloudfoundry --plan free --parameters {\"instance_name\":\"${var.cf_org_name}\"}"

  }

}

 

The third script assumes subaccount and CF environment have been created, it will change subaccount entitlement and configure SAP Business Application Studio + Audit Log services.

variable "username" {}

variable "password" {}

variable "global_account_subdomain" {}

variable "subaccount_id" {}



resource "null_resource" "btp_subaccount_entitlement" {

  provisioner "local-exec" {

    command = "btp login --url https://cpcli.cf.eu10.hana.ondemand.com --user ${var.username} --password ${var.password}"

  }



  provisioner "local-exec" {

    command = "btp target --global-account ${var.global_account_subdomain}"

  }



  provisioner "local-exec" {

    command = "btp assign accounts/entitlement --to-subaccount ${var.subaccount_id} --for-service sapappstudio --plan free --amount 1"

  }



  provisioner "local-exec" {

    command = "btp assign accounts/entitlement --to-subaccount ${var.subaccount_id} --for-service auditlog-management --plan default --enable"

  }



  provisioner "local-exec" {

    command = "btp assign accounts/entitlement --to-subaccount ${var.subaccount_id} --for-service auditlog-viewer --plan free --enable"

  }

}



resource "null_resource" "btp_subaccount_app_subscription" {

  depends_on = [null_resource.btp_subaccount_entitlement]



  provisioner "local-exec" {

    command = "btp target --subaccount ${var.subaccount_id}"

  }



  provisioner "local-exec" {

    command = "btp subscribe accounts/subaccount --to-app sapappstudio --plan free"

  }



  provisioner "local-exec" {

    command = "btp subscribe accounts/subaccount --to-app auditlog-viewer --plan free"

  }

}

 

Last script shows everything together, CF Provider + Provisioners with btp CLI manipulating all aspects in your BTP account.

terraform {

  required_providers {

    cloudfoundry = {

      source  = "cloudfoundry-community/cloudfoundry"

      version = "0.50.4"

    }

  }

}



variable "user" {}

variable "password" {}

variable "global_account_subdomain" {}

variable "subaccount_id" {}

variable "cf_org_name" {}

variable "cf_space_name" {}

variable "cf_api_url" {}



resource "null_resource" "btp_subaccount_creation" {

  provisioner "local-exec" {

    command = "btp login --url https://cpcli.cf.eu10.hana.ondemand.com --user ${var.user} --password ${var.password}"

  }



  provisioner "local-exec" {

    command = "btp target --global-account ${var.global_account_subdomain}"

  }



  provisioner "local-exec" {

    command = "btp assign accounts/entitlement --to-subaccount ${var.subaccount_id} --for-service cloudfoundry --plan free --amount 1"

  }

  

  provisioner "local-exec" {

    command = "btp create accounts/environment-instance --subaccount ${var.subaccount_id} --display-name ${var.cf_org_name} --environment cloudfoundry --service cloudfoundry --plan free --parameters {\"instance_name\":\"${var.cf_org_name}\"}"

  }

}



resource "null_resource" "btp_subaccount_entitlement" {

  depends_on = [null_resource.btp_subaccount_creation]



  provisioner "local-exec" {

    command = "btp assign accounts/entitlement --to-subaccount ${var.subaccount_id} --for-service sapappstudio --plan free --amount 1"

  }



  provisioner "local-exec" {

    command = "btp assign accounts/entitlement --to-subaccount ${var.subaccount_id} --for-service auditlog-management --plan default --enable"

  }



  provisioner "local-exec" {

    command = "btp assign accounts/entitlement --to-subaccount ${var.subaccount_id} --for-service auditlog-viewer --plan free --enable"

  }

}



resource "null_resource" "btp_subaccount_app_subscription" {

  depends_on = [null_resource.btp_subaccount_entitlement]



  provisioner "local-exec" {

    command = "btp target --subaccount ${var.subaccount_id}"

  }



  provisioner "local-exec" {

    command = "btp subscribe accounts/subaccount --to-app sapappstudio --plan free"

  }



  provisioner "local-exec" {

    command = "btp subscribe accounts/subaccount --to-app auditlog-viewer --plan free"

  }

}



provider "cloudfoundry" {

  api_url  = var.cf_api_url

  user     = var.user

  password = var.password

}



resource "cloudfoundry_org" "org1" {

  name = var.cf_org_name

}



resource "cloudfoundry_space" "org1-space1" {

  name = var.cf_space_name

  org = cloudfoundry_org.org1.id

}



resource "cloudfoundry_space_users" "org1-space1-users" {

  space      = cloudfoundry_space.org1-space1.id

  managers   = [var.user]

  developers = [var.user]

  auditors   = [var.user]

}



data "cloudfoundry_service" "application-logs" {

  name = "application-logs"

}



resource "cloudfoundry_service_instance" "application-logs-srv" {

  name         = "app-logs-srv"

  space        = cloudfoundry_space.org1-space1.id

  service_plan = data.cloudfoundry_service.application-logs.service_plans["lite"]

  depends_on = [cloudfoundry_space_users.org1-space1-users]

}

 

As seen above, you can mix and match whatever you want to satisfy your requirements. You don't have to control everything in the account if you don't want to. You can alsogng have multiple .tf files: one with proper Providers managing resources and states, another just with Provisioners and btp CLI commands to handle the one-off tasks.

Remember, this is a workaround to be used until we have a proper Terraform Provider for SAP BTP.

 

What next?

Same as before... Still awaiting SAP to provide us an official Terraform Provider for SAP BTP. Meanwhile, learning the basics of Go 🙃

 

 

 

 

 

 

 

[MustafaBensan]

Hi Mauricio,

How would you compare the Terraform approach described in this post with BTP Setup Automator?  When would you use one over the other?

Regards,

Mustafa.

[mauriciolauffer]

Hi Mustafa,

Using Provisioners (not Providers) has the same effect as BTP Setup Automator as both are using btp CLI to get the job done. You can send commands, do everything btp CLI supports, but nothing else. However, Terraform is not limited to btp CLI only.

The following link lists most of the limitations in BTP Setup Automator and gives the answer straight away (https://github.com/SAP-samples/btp-setup-automator/discussions/368😞

"First things first: the btp-setup-automator is not aiming in providing a or competing with standards like Terraform or Pulumi. Although the btp-setup-automator can support you in automating your SAP BTP account setup it does not aim to catch up with these solutions."

IMO, the main difference between both is state management. BTP Setup Automator cannot manage your resources. It doesn't know what has changed, therefore you cannot rely on it for configuration drift.

But I will list other points for a general comparison:

• BTP Setup Automator is a newborn, it's limited to SAP BTP only;
  • Terraform is the best tool in this space, you can control not only your SAP BTP, but also your SAP servers, everything in AWS/Azure/GCP and much more;
• it's maintained by a couple of people from the SAP community: Rui Nogueira and Christian Lechner;
  • Terraform is maintained by a company called Hasicorp +1000s of open source developers;
• it's a wrapper for CLI tools such: btp CLI, cf CLI, kubectl, etc;
  • Terraform is much more than a wrapper for these CLI;
• it supports only 1 subaccount;
  • Terraform has no limits, same script can handle as many as you need;
• it requires a lot of work before get anything done (install docker, configure, get image, etc);
  • Terraform is simpler, just install and run;

• too many moving parts, seems to be more complicated, it's a script running inside a Docker container;
  • Terraform doesn't require any configuration;
• Docker license could be a problem for some companies;
  • Terraform is free, there's a paid version, but nobody is required to use it;
• it can be really hard to set this up in a CI/CD pipeline;
  • Terraform is easily integrated to CI/CD pipelines, there's even a github action ready to use: https://github.com/hashicorp/setup-terraform

It sounds biased, and it's. Terraform is in another league, there's no comparison.

But don't get me wrong. BTP Setup Automator is great and served its purpose when there was nothing else! Have a look at its git repo. It's a massive effort to maintain that, and considering it's maintained by the community makes that even more awesome. Kudos to the people involved in the project!

To close my comment, another quote from https://github.com/SAP-samples/btp-setup-automator/blob/main/docs/FAQ.md

"It's meant to be an inspiration for you to think of other ways to integrate SAP BTP into your development landscape or to simply use the tool as is."

[Rui-Nogueira]

You are spot on Mauricio. Thanks for the great explanation! Although btp-setup-automator and Terraform are serving the same purpose on BTP, Terraform is from my point of view THE industry standard for infrastructure-as-code.

Best,
Rui

[ikenna-okeke]

So while playing around with the terraform to enable cloud foundry,  I stumbled accross a problem that took me quite sometime to fix because I was working with a trial account and I believe most of us probably will be using trial accounts. So first of all, If you are using the global account of a trial account where you already have a subaccount and now trying to create another subaccount within, you might notice that your first subaccount has already occupied the 4 quota for Cloud foundry assigned to each trial global account so first of all start by reducing the quota of the first sub account by: clicking on the sub >etitlements >edit> reduce the quota by 1 or 2>save.  

1) Now if you are using this as provided in the blog post you might be getting erros like "Your global account is not entitled to XYZ"

provisioner "local-exec" {

command = "btp assign accounts/entitlement --to-subaccount ${var.subaccount_id} --for-service cloudfoundry --plan free --amount 1"

}

so instead use this  :

provisioner "local-exec" {

command = "btp assign accounts/entitlement --to-subaccount ${var.subaccount_id} --for-service APPLICATION_RUNTIME --plan MEMORY --amount 1"

}

2) instead of 

provisioner "local-exec" {

command = "btp create accounts/environment-instance --subaccount ${var.subaccount_id} --display-name ${var.cf_org_name} --environment cloudfoundry --service cloudfoundry --plan free --parameters {\"instance_name\":\"${var.cf_org_name}\"}"

}, use 

provisioner "local-exec" {

command = "btp create accounts/environment-instance --subaccount ${var.subaccount_id} --display-name ${var.display-name} --environment cloudfoundry --service cloudfoundry --plan trial --parameters '{\"instance_name\":\"${var.cf_org_name}\"}' "

},pay attention to the formatting here '{\"instance_name\":\"${var.cf_org_name}\"}' " and you will notice an additional single quotes at the start and end of the {}. These werer the points I battled errors all day. 

Thank you Mauricio for the post

[ikenna-okeke]

Also, I have a question as I have noticed that terraform destroy, returns true that it destroyed the resources but the resources are still in the SAP cockpit because apparently, the resources are created using the BTP CLI. so this means when we use provisioner "null_resource", terraform does not manage the state and is not incharge of the created resources right?