UPDATED: SAP has released a Terraform provider for SAP BTP! Terraform provider for SAP BTP now available for non-productive use. Follow this to learn how to use: Automating SAP BTP setup with the new Terraform provider for SAP BTP
In this blog, we'll see how to automate SAP Business Technology (BTP) provisioning and configuration using Terraform. But before get to it, we need to understand what is Terraform and Infrastructure as Code (IaC). What are the benefits and the problems it solves?
Microsoft defines it as:
Making it simpler: IaC means managing your IT infrastructure with configuration files hosted and versioned by your git repository of choice (GitHub, GitLab, etc). It means automation for provisioning, compliance, and management of cloud, datacenter, and service.
IaC increases speed, reduces costs and minimizes risks when deploying IT infrastructure. Automation is paramount! No more manual provisioning nor thousands of CLI commands to deploy and setup your servers. It helps taking your DevOps to the next level!
From Terraform website:
Making it simpler: Terraform is the tool to make IaC a reality in your team. Today, Terraform is the defacto standard in the IaC area, it's platform-agnostic, open source and free to use. Its open source community is massive, they have an active army of developers always extending Terraform capabilities through its plugins (called providers) ecosystem.
Pretty much everything Cloud Foundry (CF) and Kubernetes (Kyma) related. Manage service instances, applications, users, quotas, routes, etc. Think about SAP BTP Boosters on steroids.
Check the links for more details:
You manually run multiple SAP BTP Boosters for every new project/environment. Terraform can automate it.
You want to make sure nobody changes anything in production. Terraform can detect and manage configuration drift. All changes to the infrastructure outside of Terraform will be detected as deviation.
You run your SAP on-premise instances (ECC, S/4HANA, PI/PO, SolMan, SAP Cloud Connector, etc) on AWS, Azure or Google Cloud Platform. Automated server provisioning is Terraform's bread and butter!
You need disposable environments for testing or training, perhaps copying PRD into a new QAS. Terraform makes it easy and effortless.
You have advanced CI/CD pipelines and complex automated tests which require a new S4 instance and a new BTP CF space to be created just for it and to be deleted at the end. Terraform can coordinate everything based on a recipe you have defined. For instance, every time you push code to your git repo, Terraform will create all new instances and configure them, the tests run and Terraform delete them all (they exist just to run the tests). It could also run in a GitHub Action.
At this moment, there isn't any Terraform provider created specifically for SAP BTP. Which means everything above CF or Kyma isn't supported, e.g., global account and directories.
We do have some alternatives from SAP BTP Boosters embedded into the cockpit to SAP BTP Setup Automator an open source project maintained by the community (thanks Rui Nogueira).
Do they get the work done? Yes, sure! They may work just fine for your use case. However, none of them have the power, simplicity and flexibility of Terraform. Remember, Terraform is the defacto standard in this area, it's not a new and unproven tool and could be used for pretty much anything.
Enough said, let's see it in action. We're going to create a configuration file which will provision a new QAS CF space, some users and a couple of services.
Go ahead, download and install Terraform: https://developer.hashicorp.com/terraform/downloads;
Create a new directory. In the directory, create a new file main.tf with the content below. The configuration describes a Cloud Foundry environment, its ORG and SPACE, also the users and the services provided to the SPACE.
From a terminal, run $ terraform init command to initialize the project:
The command will create the file .terraform.lock.hcl and folder .terraform in the directory. This is the internal Terraform configuration for this project. You should NEVER hardcode your credentials and secrets as I did! Use input variables, environment variables and terraform.tfvars files to do it in a secure manner.
Because I'm using a SAP BTP trial account, I cannot create new CF ORGs, I need to use the one already provided, therefore I need to import its configuration into my Terraform project. The CF ORG GUID is required for that, it can be found on SAP BTP cockpit or from cf CLI.
Run $ terraform import command to import the CF ORG configuration. If I was creating a new CF ORG, I could skip it.
File terraform.tfstate will be created with the CF ORG configuration just imported.
Run $ terraform apply command to tell Terraform to execute the configuration defined in main.tf.
The execution plan will be displayed, it will list all actions to be performed. You need to confirm it by typing yes, as aforementioned in the screenshot.
Congratulations! You have just created a new space called "qas" with some users assigned to it, and 2 services activated (application logging and destination) 🎉🎉🎉🍻🍻🍻
That was quite simple, right? I believe this is a terrific way to automate most of the operations in SAP BTP, and it's not limited to BTP or SAP systems only, it could be used for pretty much anything. Imagine using Terraform to manage your SAP BTP accounts, SAP and non-SAP systems deployed to any cloud provider or even on-premise systems. One tool to rule them all!
How nice would be if SAP created an official Terraform provider for SAP BTP? There's also the opportunity for the SAP community to step in and create an open source project, as we have done so many times. I know, I know, it's a big ask...
I don't know Go, the programming language used to write Terraform providers. But, if I find the time, I'll learn the language and try to create an SAP BTP provider...
In this blog, we'll see how to automate SAP Business Technology (BTP) provisioning and configuration using Terraform. But before get to it, we need to understand what is Terraform and Infrastructure as Code (IaC). What are the benefits and the problems it solves?
What is Infrastructure as Code (IaC)?
Microsoft defines it as:
Infrastructure as Code (IaC) is the management of infrastructure (networks, virtual machines, load balancers, and connection topology) in a descriptive model, using the same versioning as DevOps team uses for source code. Like the principle that the same source code generates the same binary, an IaC model generates the same environment every time it is applied. IaC is a key DevOps practice and is used in conjunction with continuous delivery.
Making it simpler: IaC means managing your IT infrastructure with configuration files hosted and versioned by your git repository of choice (GitHub, GitLab, etc). It means automation for provisioning, compliance, and management of cloud, datacenter, and service.
IaC increases speed, reduces costs and minimizes risks when deploying IT infrastructure. Automation is paramount! No more manual provisioning nor thousands of CLI commands to deploy and setup your servers. It helps taking your DevOps to the next level!
What is Terraform?
From Terraform website:
HashiCorp Terraform is an infrastructure as code tool that lets you define both cloud and on-prem resources in human-readable configuration files that you can version, reuse, and share. You can then use a consistent workflow to provision and manage all of your infrastructure throughout its lifecycle. Terraform can manage low-level components like compute, storage, and networking resources, as well as high-level components like DNS entries and SaaS features.
Making it simpler: Terraform is the tool to make IaC a reality in your team. Today, Terraform is the defacto standard in the IaC area, it's platform-agnostic, open source and free to use. Its open source community is massive, they have an active army of developers always extending Terraform capabilities through its plugins (called providers) ecosystem.
What are the Terraform use cases for SAP BTP?
Pretty much everything Cloud Foundry (CF) and Kubernetes (Kyma) related. Manage service instances, applications, users, quotas, routes, etc. Think about SAP BTP Boosters on steroids.
Check the links for more details:
You manually run multiple SAP BTP Boosters for every new project/environment. Terraform can automate it.
You want to make sure nobody changes anything in production. Terraform can detect and manage configuration drift. All changes to the infrastructure outside of Terraform will be detected as deviation.
You run your SAP on-premise instances (ECC, S/4HANA, PI/PO, SolMan, SAP Cloud Connector, etc) on AWS, Azure or Google Cloud Platform. Automated server provisioning is Terraform's bread and butter!
You need disposable environments for testing or training, perhaps copying PRD into a new QAS. Terraform makes it easy and effortless.
You have advanced CI/CD pipelines and complex automated tests which require a new S4 instance and a new BTP CF space to be created just for it and to be deleted at the end. Terraform can coordinate everything based on a recipe you have defined. For instance, every time you push code to your git repo, Terraform will create all new instances and configure them, the tests run and Terraform delete them all (they exist just to run the tests). It could also run in a GitHub Action.
What about the Terraform limitations with SAP BTP?
At this moment, there isn't any Terraform provider created specifically for SAP BTP. Which means everything above CF or Kyma isn't supported, e.g., global account and directories.
What are the alternatives?
We do have some alternatives from SAP BTP Boosters embedded into the cockpit to SAP BTP Setup Automator an open source project maintained by the community (thanks Rui Nogueira).
Do they get the work done? Yes, sure! They may work just fine for your use case. However, none of them have the power, simplicity and flexibility of Terraform. Remember, Terraform is the defacto standard in this area, it's not a new and unproven tool and could be used for pretty much anything.
Demo time!
Enough said, let's see it in action. We're going to create a configuration file which will provision a new QAS CF space, some users and a couple of services.
Go ahead, download and install Terraform: https://developer.hashicorp.com/terraform/downloads;
Install Terraform
Create a new directory. In the directory, create a new file main.tf with the content below. The configuration describes a Cloud Foundry environment, its ORG and SPACE, also the users and the services provided to the SPACE.
//main.tf
terraform {
required_providers {
cloudfoundry = {
source = "cloudfoundry-community/cloudfoundry"
version = "0.50.4"
}
}
}
provider "cloudfoundry" {
api_url = "https://api.cf.us10.hana.ondemand.com" //BTP CF API Endpoint
user = "YOUR_USERNAME" //BTP Global Account Username
password = "YOUR_PASSWORD" //BTP Global Account Password
}
// CF Organization
resource "cloudfoundry_org" "org1" {
name = "YOUR_CF_ORG_NAME" //The Cloud Foundry Environment Org name
}
// CF Space
resource "cloudfoundry_space" "org1-qas" {
name = "qas"
org = cloudfoundry_org.org1.id
}
// CF Space Users
resource "cloudfoundry_space_users" "org1-qas-users" {
space = cloudfoundry_space.org1-qas.id
managers = ["USERNAME_A"]
developers = ["USERNAME_A", "USERNAME_B"]
auditors = []
}
// CF Services
data "cloudfoundry_service" "application-logs" {
name = "application-logs"
}
resource "cloudfoundry_service_instance" "application-logs-srv" {
name = "app-logs-srv"
space = cloudfoundry_space.org1-qas.id
service_plan = data.cloudfoundry_service.application-logs.service_plans["lite"]
depends_on = [cloudfoundry_space_users.org1-qas-users]
}
data "cloudfoundry_service" "destination" {
name = "destination"
}
resource "cloudfoundry_service_instance" "destination-srv" {
name = "destination-srv"
space = cloudfoundry_space.org1-qas.id
service_plan = data.cloudfoundry_service.destination.service_plans["lite"]
depends_on = [cloudfoundry_space_users.org1-qas-users]
}
SAP BTP cockpit
From a terminal, run $ terraform init command to initialize the project:
terraform initTerraform has been successfully initialized!
The command will create the file .terraform.lock.hcl and folder .terraform in the directory. This is the internal Terraform configuration for this project. You should NEVER hardcode your credentials and secrets as I did! Use input variables, environment variables and terraform.tfvars files to do it in a secure manner.
Because I'm using a SAP BTP trial account, I cannot create new CF ORGs, I need to use the one already provided, therefore I need to import its configuration into my Terraform project. The CF ORG GUID is required for that, it can be found on SAP BTP cockpit or from cf CLI.
Run $ terraform import command to import the CF ORG configuration. If I was creating a new CF ORG, I could skip it.
terraform import cloudfoundry_org.org1 YOUR_CF_ORG_GUIDCF ORG import successful!
File terraform.tfstate will be created with the CF ORG configuration just imported.
Run $ terraform apply command to tell Terraform to execute the configuration defined in main.tf.
terraform applyTerraform execution plan
The execution plan will be displayed, it will list all actions to be performed. You need to confirm it by typing yes, as aforementioned in the screenshot.
Apply complete!
Congratulations! You have just created a new space called "qas" with some users assigned to it, and 2 services activated (application logging and destination) 🎉🎉🎉🍻🍻🍻
CF Space QAS has been created, note the Service Instances number
What next?
That was quite simple, right? I believe this is a terrific way to automate most of the operations in SAP BTP, and it's not limited to BTP or SAP systems only, it could be used for pretty much anything. Imagine using Terraform to manage your SAP BTP accounts, SAP and non-SAP systems deployed to any cloud provider or even on-premise systems. One tool to rule them all!
How nice would be if SAP created an official Terraform provider for SAP BTP? There's also the opportunity for the SAP community to step in and create an open source project, as we have done so many times. I know, I know, it's a big ask...
I don't know Go, the programming language used to write Terraform providers. But, if I find the time, I'll learn the language and try to create an SAP BTP provider...
Bender Futurama meme
- SAP Managed Tags:
12 Comments
