Technical Articles
Setting up a private docker registry in SAP Cloud Platform Kyma (Kubernetes)
Kyma is a Kubernetes environment with extensions such as eventing and serverless functions to make it better suited for enterprise applications.
Users of the SAP Cloud Platform can provision a Kyma cluster with a productive or trial account.
To fully use Kubernetes features of the Kyma cluster, you’ll need a private software registry for docker images or python packages. Some hyperscalers offer a container registry service, however this has a few drawbacks:
- Forfeit a multi coud strategy
- Use as many different services as there are registry technology (docker, python, node, java)
This tutorial explains how to deploy and configure Nexus, a single server that acts a hosted or proxy repository for docker, python, helm charts and other artifacts. Nexus is just one of many free software with a readymade deployment solutions.
Pre requisites:
- A kyma cluster
- The kubectl command working with your cluster.
- The helm command installed.
Deploy Nexus from a helm chart
What’s a helm chart? A working application is more than a container, it is various artifacts bound together to define filesystems, network ports, SSL termination, routes, etc.
A helm chart defines all of those kubernetes artifacts and help automate applications deployments. You may have already used some in the Kyma samples.
Define sonatype as a trusted source of helm charts
#helm repo add sonatype https://sonatype.github.io/helm3-charts/ "sonatype" has been added to your repositories
Create a dedicated Namespace
It is good practive to group resources into namespace to avoid crowding the default one. Let’s create a namespace for various tools:
# kubectl create namespace tools namespace/tools created
Execute the chart
helm install mynexus-repo sonatype/nexus-repository-manager -n tools NAME: mynexus-repo LAST DEPLOYED: Mon Feb 1 13:35:07 2021 NAMESPACE: tools STATUS: deployed REVISION: 1 TEST SUITE: None NOTES: 1. Get the application URL by running these commands: export POD_NAME=$(kubectl get pods --namespace tools -l "app.kubernetes.io/name=nexus-repository-manager,app.kubernetes.io/instance=mynexus-repo" -o jsonpath="{.items[0].metadata.name}") echo "Visit http://127.0.0.1 to use your application" kubectl --namespace tools port-forward $POD_NAME 8081:80
Check the status
After a few minutes, open the Kyma web console, navigate to the namespace “tools” to check the status of the pods. You can also use the command line:
kubectl get pods,svc -n tools NAME READY STATUS RESTARTS AGE mynexus-repo-nexus-repository-manager-5d788cffcd-9g6jq 2/2 Running 0 6m2s NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE service/mynexus-repo-nexus-repository-manager ClusterIP 100.67.43.33 <none> 8081/TCP 6m3s
Here, we see 2 out 2 containers running for the pod, so we can assume all is ok.
Expose the service
The default service points to the main admin interface, it is currently ony reachable from inside the cluster. We’ll expose the service to the internet.
Default Nexus Service
Now click on the name to get more details. Click on the button “expose service”
Click on the URL to open the nexus user interface.
Configure Nexus – Change admin password
The administration password is generated automatically. To obtain it you need to use the exact name of the pod and execute the below command:
# kubectl exec -n tools -t mynexus-repo-nexus-repository-manager-5d788cffcd-9g6jq \ --container nexus-repository-manager -- cat /nexus-data/admin.password
This should print the existing password in the console.
My advice is to store a new password as a Kubernetes secret. If you are out of imagination and don’t have a tool to generate passwords, you can use the linux command “ps -ef | md5 | head -c 16” :
# kubectl exec -n tools -t mynexus-repo-nexus-repository-manager-5d788cffcd-9g6jq \ --container nexus-repository-manager -- ps -ef | md5 | head -c 16 299e228264c13fcf
And to store the future password inside Kubernetes:
kubectl create secret generic nexus-admin-password -n tools --from-literal=password="9e48753634d29568"
This should bring up a wizard that prompts you for a new password, and if you’d like to enable anonymous access.
Configure Nexus – Create the docker repository
When logged in as an admin, select settings / repositories
Choose docker (hosted), give it a name, set HTTP port to 8082 and create the new repository. Don’t worry, traffic will be encrypted up to the Kyma router, and unencrypted innside Kyma.
Configure Nexus – Create a role
Configure Nexus – Create a user
Configure Service – Add the port 8082
Select the service and click edit.
Configure API rule to expose new service port
Create a new API rule to expose the port 8082
You should now have 2 api rules:
The second URL is your private docker registry, if you open it in a web browser, you should get an error message “Not a Docker request”
Finally, push a docker image
Grab a small image
# docker pull hello-world Using default tag: latest latest: Pulling from library/hello-world 0e03bdcc26d7: Pull complete Digest: sha256:31b9c7d48790f0d8c50ab433d9c3b7e17666d6993084c002c2ff1ca09b96391d Status: Downloaded newer image for hello-world:latest docker.io/library/hello-world:latest
Retag it with your private url
# docker tag hello-world private-docker-nexus.c-xxxxx.kyma.shoot.live.k8s-hana.ondemand.com/hello-world
Login to your private url, use the user and password you’ve defined in Nexus.
# docker login -u remi https://private-docker-nexus.c-xxxxxx.kyma.shoot.live.k8s-hana.ondemand.com/ Login Succeeded
And finally push it !
# docker push private-docker-nexus.c-xxxxxxx.kyma.shoot.live.k8s-hana.ondemand.com/hello-world Using default tag: latest The push refers to repository [private-docker-nexus.c-xxxxxxx.kyma.shoot.live.k8s-hana.ondemand.com/hello-world] 9c27e219663c: Pushed latest: digest: sha256:90659bf80b44ce6be8234e6ff90a1ac34acbeb826903b02cfa0da11c82cbc042 size: 525
You could also use Nexus to browse the repo
Using the private registry in deployment
To make use of the registry in deployment recipes in other namespaces, you need to store the access credentials into a kubernetes secret in each namespace.
kubectl create secret docker-registry myprivatenexus --docker-server=private-docker-nexus.c-xxxxxxx.kyma.shoot.live.k8s-hana.ondemand.com --docker-username=remi --docker-password=<mysecretpassword> --docker-email=remi.astier@sap.com -n <namespace with your deployment>
Finally, use the secret name and docker tag in a deployment file:
Conclusion
A private docker registry, or any readily available helm chart can be deployed using similar steps of installing the chart and configuring the services and api rules.
Important note
After just two days of part time work, the filesystem has filled and the registry in read only mode. The steps to increase the filesystem are:
- Shut down the pod
kubectl patch deployment -n tools mynexus-repo-nexus-repository-manager -p '{ "spec": { "replicas": 0 }}'
- Increase the volume
kubectl patch -n tools pvc mynexus-repo-nexus-repository-manager-data -p '{"spec": { "resources": { "requests": { "storage": "41Gi" }}}}'
- Restart the pod
kubectl patch deployment -n tools mynexus-repo-nexus-repository-manager -p '{ "spec": { "replicas": 1 }}'
Great blog, Remi! Thanks!
And this document explains how to use such private registry for deployments :
https://kyma-project.io/docs/#details-deploy-with-a-private-docker-registry
Thanks Andrei Vishnevsky I edited the post.
Great post Remi. Your instructions worked perfectly for me.
Great post Remi! Thanks