Kubernetes service accounts are somewhat similar to a technical user concept, and are very useful in managing service to service automation and communication. For instance, one would rely on service accounts to manage CI/CD pipelines but there are other use-cases as well.

Putting it all together

My personal motivation behind this brief was to procure a service account based kubeconfig as to eliminate an OIDC-user browser redirect and make it suitable for use in headless and unmanned contexts.

Here goes the agenda for this brief.

Brief anatomy of a kyma cluster

Let’s assume you have got access to your SAP BTP, Kyma runtime cluster.

OIDC-user based kubeconfig

After you have provisioned your kyma cluster you can either download the OIDC-user based kubeconfig or directly launch a kyma dashboard from a BTP sub-account.

Worker nodes

Typically, a kyma cluster will have a number of worker nodes, possibly in different availability zones (in the same region/data center though), as depicted below:

SAP BTP, Kyma runtime cluster with AZURE with 4 nodes in 3 availability zones

SAP BTP Free Tier, Kyma runtime cluster with AWS with a single node in one availability zone

Each worker node features a certain CPU and Memory capacity that can be reserved by kyma workloads. Moreover, each node is mapped into specific hardware (machine type) from a cluster provider (AZURE, AWS, GCP).

Good to know:

• It is possible to dynamically change a worker node machine type without incurring any downtime.

Namespaces

On a logical level, each kyma cluster comes provisioned with two public namespaces, namely default and kube-public. The default namespace cannot be deleted and kube-public can be used to host kyma dashboard extensions.

Namespaces are very useful in organizing cluster resources.

Thus, indeed, most likely, you will end up creating several other namespaces to cater for different tasks or users profiles as a means of exercising some level of control over cluster resources.

Namespaced isolation. Keep your clutter at bay.

If we were to manage a population of several thousand or so kyma cluster named users as well as a number of service accounts, we could simply divide the entire user population into logical teams or tasks and then assign each team or task a separate namespace.

Subsequently, we would grant each team member a type of an access – a role binding – that is compatible with their duties. Furthermore, we’d create a service account for each team for a service access to a namespace.

To get you more insight about organizing with namespaces here goes a very good read:

• Kubernetes best practices: Organizing with Namespaces

Role bindings

Namespaced role bindings are created based on the existing roles. These roles can be either cluster wide or namespaced roles.

1. Let us start with a full cluster-admin access for all team members and then eventually narrow down the level of access according with each team member profile.
2. Let’s see how to create a service account with a never expiring token for a team namespace.

Service Accounts

One of the qualities of service accounts is that they are namespaced.

• How to use Service Accounts | Kubernetes
• Grant permissions to Service Accounts | Kubernetes

• Manually retrieve ServiceAccount credentials | Kubernetes

Service Accounts tokens.

Tokens are required to grant access to Kubernetes API server.

Nowadays, with Kubernetes versions 1.24.8 onwards, the secret-based tokens are no longer auto-generated by default for new service accounts. But there are other ways to get service accounts tokens, as described here:

• How to get a token | Kubernetes

bound tokens

Bound, short-lived tokens can be created with kubectl create token Alternatively, a kyma dashboard can be used to create bound tokens for a service account, as depicted below:

secret-based tokens

Albeit the short-lived bound token are recommended for workload automations, legacy secret-based tokens can still be useful if you want to gain access to a kyma namespace from a headless environment without incurring a browser redirect to localhost (or if the redirect port was already taken).

Let’s create a Makefile and a helm chart to automate the entire process, namely:

• create a helm chart to create a service account, service account secret and a role binding (see appendix)
• create a Makefile with the following targets: btp-easy-deploy and btp-easy-kubeconfig (see appendix)

Run the following make target:

$make btp-easy-deploy

$ make NAMESPACE=toto btp-easy-deploy
kubectl create ns toto --kubeconfig ~/.kube/kubeconfig--btp-easy--btp.yaml --dry-run=client -o yaml | kubectl apply --kubeconfig ~/.kube/kubeconfig--btp-easy--btp.yaml -f -
namespace/toto created
kubectl label namespace toto istio-injection=enabled --kubeconfig ~/.kube/kubeconfig--btp-easy--btp.yaml
namespace/toto labeled
helm upgrade --install --create-namespace btp-easy ./helm/btp-easy/ --namespace=toto --kubeconfig ~/.kube/kubeconfig--btp-easy--btp.yaml
Release "btp-easy" does not exist. Installing it now.
NAME: btp-easy
LAST DEPLOYED: 
NAMESPACE: toto
STATUS: deployed
REVISION: 1
TEST SUITE: None

A new namespace, a service account, a service account secret and a role binding will be created in the namespace, as follows:

service account	role binding

View service account based kubeconfig

Run the following make target:

$ make NAMESPACE=toto view-btp-easy-kubeconfig

$ make NAMESPACE=toto view-btp-easy-kubeconfig
kubectl-view_serviceaccount_kubeconfig toto-sa -n toto --kubeconfig ~/.kube/kubeconfig--btp-easy--btp.yaml

apiVersion: v1
clusters:
- cluster:
    certificate-authority-data: LS0tLS1CRUdJTiBDRVJUSUZJQ0FURS0tBtS05SbGo4TENFL2ZVTi8vbDFsZwotLS0tLUVORCBDRVJUSUZJQ0FURS0tLS0tCg==
    server: https://api.***.btp-easy.shoot.canary.k8s-hana.ondemand.com
  name: shoot--btp-easy--***
contexts:
- context:
    cluster: shoot--btp-easy--***
    namespace: toto
    user: toto-sa
  name: shoot--btp-easy--***
current-context: shoot--btp-easy--***
kind: Config
preferences: {}
users:
- name: toto-sa
  user:
    token: eyJhbGciOiJSUzI1NiIsImtpZCI6Im1yTGRIdW9EOFJLZ3RQaDh2ZGxqSEhpWnVwN3ZSQThxdmlWb1dlWnBid

This is using a third-party kubectl plugin called kubectl-view-serviceaccount-kubeconfig which allows to view a service account based kubeconfig…

Run $ make NAMESPACE=toto btp-easy-kubeconfig to create a kubeconfig file, namely

~/.kube/kubeconfig--btp-easy-toto--btp.yaml

Good to know:

• For enhanced security the file has both group and world read attributes stripped as follows:
  • chmod 600 ~/.kube/kubeconfig--btp-easy-$(NAMESPACE)--btp.yaml

Conclusion

Please continue reading through to the coffee corner as it gathers all the resources to help you create a Makefile and a helm chart to automate the creation of a Service Account.

Coffee corner

helm chart structure

 ┣ 📂helm
 ┃ ┗ 📂btp-easy
 ┃ ┃ ┣ 📂templates
 ┃ ┃ ┃ ┣ 📜clusteradmin-role-binding.yaml
 ┃ ┃ ┃ ┣ 📜sa-secret.yaml
 ┃ ┃ ┃ ┗ 📜sa.yaml
 ┃ ┃ ┗ 📜Chart.yaml
 ┣ 📜Makefile

1. create sa.yaml:

• create a service account and then a never expiring secret attached to the sa.

kind: ServiceAccount
apiVersion: v1
metadata:
  name: {{ .Release.Namespace }}-sa
  namespace: {{ .Release.Namespace }}
  labels:
    app.kubernetes.io/name: {{ .Release.Namespace }}-sa
automountServiceAccountToken: false
secrets:
  - name: {{ .Release.Namespace }}-sa-token

2. create sa-secret.yaml

• create a service token secret

apiVersion: v1
kind: Secret
type: kubernetes.io/service-account-token
metadata:
  annotations:
    kubernetes.io/service-account.name: {{ .Release.Namespace }}-sa
  name: {{ .Release.Namespace }}-sa-token
  labels:
    app.kubernetes.io/name: {{ .Release.Namespace }}-sa-token
data: {}

3. create clusteradmin-role-binding.yaml

• The new role binding will be created based on cluster-admin cluster role.

kind: RoleBinding
apiVersion: rbac.authorization.k8s.io/v1
metadata:
  name: clusteradmin
  namespace: {{ .Release.Namespace }}
subjects:
  - kind: User
    apiGroup: rbac.authorization.k8s.io
    name: foo.bar@acme.com
  - kind: ServiceAccount
    name: {{ .Release.Namespace }}-sa
    namespace: {{ .Release.Namespace }}
roleRef:
  apiGroup: rbac.authorization.k8s.io
  kind: ClusterRole
  name: cluster-admin

4. create Chart.yaml

apiVersion: v2
name:  btp-easy
description: Helm chart for btp-easy

# A chart can be either an 'application' or a 'library' chart.
type: application

# This is the chart version.
version: 0.0.1

# This is the version number of the application being deployed.
appVersion: 0.0.1

Makefile

5. create Makefile with a number of targets

.DEFAULT_GOAL := help

LOCAL_DIR = $(dir $(abspath $(lastword $(MAKEFILE_LIST))))

NAMESPACE=btp-easy
KUBECONFIG=~/.kube/kubeconfig--btp-easy--btp.yaml

help: ## Display this help.
	@awk 'BEGIN {FS = ":.*##"; printf "\nUsage:\n  make \033[36m<target>\033[0m\n"} /^[a-zA-Z_0-9-]+:.*?##/ { printf "  \033[36m%-15s\033[0m %s\n", $$1, $$2 } /^##@/ { printf "\n\033[1m%s\033[0m\n", substr($$0, 5) } ' $(MAKEFILE_LIST)
		echo $(KUBECONFIG) $(NAMESPACE)

.PHONY: btp-easy-deploy
btp-easy-deploy: ## btp-easy-deploy
	kubectl create ns $(NAMESPACE) --kubeconfig $(KUBECONFIG) --dry-run=client -o yaml | kubectl apply --kubeconfig $(KUBECONFIG) -f -
	kubectl label namespace $(NAMESPACE) istio-injection=enabled --kubeconfig $(KUBECONFIG)
	helm upgrade --install --create-namespace btp-easy ./helm/btp-easy/ --namespace=$(NAMESPACE) --kubeconfig $(KUBECONFIG)

.PHONY: btp-easy-undeploy
btp-easy-undeploy: ## btp-easy-undeploy
	helm uninstall btp-easy  --namespace=$(NAMESPACE) --kubeconfig $(KUBECONFIG)
	kubectl delete ns $(NAMESPACE) --kubeconfig $(KUBECONFIG) 

.PHONY: btp-easy-template
btp-easy-template: ## btp-easy-template
	helm template btp-easy ./helm/btp-easy/ --namespace=$(NAMESPACE) --kubeconfig $(KUBECONFIG)

.PHONY: btp-easy-kubeconfig
btp-easy-kubeconfig: ## btp-easy-kubeconfig
	kubectl-view_serviceaccount_kubeconfig $(NAMESPACE)-sa -n $(NAMESPACE) --kubeconfig $(KUBECONFIG) > ~/.kube/kubeconfig--btp-easy-$(NAMESPACE)--btp.yaml
    chmod go-r  ~/.kube/kubeconfig--btp-easy-$(NAMESPACE)--btp.yaml

.PHONY: view-btp-easy-kubeconfig
view-btp-easy-kubeconfig: ## view-btp-easy-kubeconfig
	kubectl-view_serviceaccount_kubeconfig $(NAMESPACE)-sa -n $(NAMESPACE) --kubeconfig $(KUBECONFIG)

helm template

helm template is very useful to dry run the chart without ever touching the cluster…

helm template btp-easy ./helm/btp-easy/ --namespace=toto > toto.yaml

---
# Source: btp-easy/templates/sa.yaml
kind: ServiceAccount
apiVersion: v1
metadata:
  name: toto-sa
  namespace: toto
  labels:
    app.kubernetes.io/name: toto-sa
automountServiceAccountToken: false
secrets:
  - name: toto-sa-token
---
# Source: btp-easy/templates/sa-secret.yaml
apiVersion: v1
kind: Secret
type: kubernetes.io/service-account-token
metadata:
  annotations:
    kubernetes.io/service-account.name: toto-sa
  name: toto-sa-token
  labels:
    app.kubernetes.io/name: toto-sa-token
data: {}
---
# Source: btp-easy/templates/clusteradmin-role-binding.yaml
kind: RoleBinding
apiVersion: rbac.authorization.k8s.io/v1
metadata:
  name: clusteradmin
subjects:
  - kind: User
    apiGroup: rbac.authorization.k8s.io
    name: foo.bar@acme.com
  - kind: ServiceAccount
    name: toto-sa
    namespace: toto
roleRef:
  apiGroup: rbac.authorization.k8s.io
  kind: ClusterRole
  name: cluster-admin

Troubleshooting and un-deploy

• https://help.sap.com/docs/btp/sap-business-technology-platform/getting-started-in-kyma-environment
• https://help.sap.com/docs/btp/sap-business-technology-platform/authentication-and-authorization-in-kyma-environment
• helm install in certain order
• helm doesn’t support managing namespaces directly
• https://helm.sh/docs/helm/helm_upgrade/
• https://github.com/helm/helm/issues/9115
• https://kubernetes.io/docs/reference/kubectl/cheatsheet/
• Authenticating service account credentials in your own code | Kubernetes

$ make NAMESPACE=toto btp-easy-undeploy
helm uninstall btp-easy  --namespace=toto --kubeconfig ~/.kube/kubeconfig--btp-easy--btp.yaml
release "btp-easy" uninstalled
kubectl delete ns toto --kubeconfig ~/.kube/kubeconfig--btp-easy--btp.yaml
namespace "toto" deleted

Tooling

$ kubectl plugin list
The following compatible plugins are available:

~/.krew/bin/kubectl-krew
~/.krew/bin/kubectl-oidc_login
~/.krew/bin/kubectl-view_serviceaccount_kubeconfig

$ kubectl krew list
PLUGIN                          VERSION
krew                            v0.4.3
oidc-login                      v1.27.0
view-serviceaccount-kubeconfig  v2.3.0

$ helm version
version.BuildInfo{Version:"v3.12.0", GitCommit:"c9f554d75773799f72ceef38c51210f1842a1dea", GitTreeState:"clean", GoVersion:"go1.20.3"}

SAP Kyma Community and SAP BTP, Kyma runtime Q&A Tags

Follow me in SAP Community: Piotr Tesny