Setting up a private docker registry in SAP Cloud Platform Kyma (Kubernetes)

Kyma is a Kubernetes environment with extensions such as eventing and serverless functions to make it better suited for enterprise applications.

Users of the SAP Cloud Platform can provision a Kyma cluster with a productive or trial account.

To fully use Kubernetes features of the Kyma cluster, you’ll need a private software registry for docker images or python packages. Some hyperscalers offer a container registry service, however this has a few drawbacks:

• Forfeit a multi coud strategy
• Use as many different services as there are registry technology (docker, python, node, java)

This tutorial explains how to deploy and configure Nexus, a single server that acts a hosted or proxy repository for docker, python, helm charts and other artifacts. Nexus is just one of many free software with a readymade deployment solutions.

Pre requisites:

• A kyma cluster
• The kubectl command working with your cluster.
• The helm command installed.

Deploy Nexus from a helm chart

What’s a helm chart? A working application is more than a container, it is various artifacts bound together to define filesystems, network ports, SSL termination, routes, etc.

A helm chart defines all of those kubernetes artifacts and help automate applications deployments. You may have already used some in the Kyma samples.

Define sonatype as a trusted source of helm charts

#helm repo add sonatype https://sonatype.github.io/helm3-charts/
"sonatype" has been added to your repositories

Create a dedicated Namespace

It is good practive to group resources into namespace to avoid crowding the default one. Let’s create a namespace for various tools:

# kubectl create namespace tools
namespace/tools created

Execute the chart

helm install mynexus-repo sonatype/nexus-repository-manager -n tools

NAME: mynexus-repo
LAST DEPLOYED: Mon Feb 1 13:35:07 2021
NAMESPACE: tools
STATUS: deployed
REVISION: 1
TEST SUITE: None
NOTES:
1. Get the application URL by running these commands:
export POD_NAME=$(kubectl get pods --namespace tools -l "app.kubernetes.io/name=nexus-repository-manager,app.kubernetes.io/instance=mynexus-repo" -o jsonpath="{.items[0].metadata.name}")
echo "Visit http://127.0.0.1 to use your application"
kubectl --namespace tools port-forward $POD_NAME 8081:80

Check the status

After a few minutes, open the Kyma web console, navigate to the namespace “tools” to check the status of the pods. You can also use the command line:

kubectl get pods,svc -n tools NAME READY STATUS RESTARTS AGE mynexus-repo-nexus-repository-manager-5d788cffcd-9g6jq 2/2 Running 0 6m2s

NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
service/mynexus-repo-nexus-repository-manager ClusterIP 100.67.43.33 <none> 8081/TCP 6m3s

Here, we see 2 out 2 containers running for the pod, so we can assume all is ok.

Expose the service

The default service points to the main admin interface, it is currently ony reachable from inside the cluster. We’ll expose the service to the internet.

Default Nexus Service

Now click on the name to get more details. Click on the button “expose service”

Set an API name, virtual hostname and select “Allow” as a access strategy.

After you click on “create”, you should see a API rule with the status “OK“

Click on the URL to open the nexus user interface.

Configure Nexus – Change admin password

The administration password is generated automatically. To obtain it you need to use the exact name of the pod and execute the below command:

# kubectl exec -n tools -t mynexus-repo-nexus-repository-manager-5d788cffcd-9g6jq \
     --container nexus-repository-manager -- cat /nexus-data/admin.password

This should print the existing password in the console.

My advice is to store a new password as a Kubernetes secret. If you are out of imagination and don’t have a tool to generate passwords, you can use the linux command “ps -ef | md5 | head -c 16” :

# kubectl exec -n tools -t mynexus-repo-nexus-repository-manager-5d788cffcd-9g6jq \
     --container nexus-repository-manager -- ps -ef | md5 | head -c 16
299e228264c13fcf

And to store the future password inside Kubernetes:

kubectl create secret generic nexus-admin-password -n tools --from-literal=password="9e48753634d29568"

Now that you have the existing and the new password, you can open the Nexus UI and sign in with user admin and the existing password.

This should bring up a wizard that prompts you for a new password, and if you’d like to enable anonymous access.

Configure Nexus – Create the docker repository

When logged in as an admin, select settings / repositories

Choose docker (hosted), give it a name, set HTTP port to 8082 and create the new repository. Don’t worry, traffic will be encrypted up to the Kyma router, and unencrypted innside Kyma.

Configure Nexus – Create a role

Configure Nexus – Create a user

Configure Service – Add the port 8082

Select the service and click edit.

In the yaml service definition, add the 4 lines selected lines to define a new port with name nexus-docker and number 8082:

Configure API rule to expose new service port

Create a new API rule to expose the port 8082

You should now have 2 api rules:

The second URL is your private docker registry, if you open it in a web browser, you should get an error message “Not a Docker request”

Finally, push a docker image

Grab a small image

# docker pull hello-world
Using default tag: latest
latest: Pulling from library/hello-world
0e03bdcc26d7: Pull complete
Digest: sha256:31b9c7d48790f0d8c50ab433d9c3b7e17666d6993084c002c2ff1ca09b96391d
Status: Downloaded newer image for hello-world:latest
docker.io/library/hello-world:latest

Retag it with your private url

# docker tag hello-world private-docker-nexus.c-xxxxx.kyma.shoot.live.k8s-hana.ondemand.com/hello-world

Login to your private url, use the user and password you’ve defined in Nexus.

# docker login -u remi https://private-docker-nexus.c-xxxxxx.kyma.shoot.live.k8s-hana.ondemand.com/
Login Succeeded

And finally push it !

# docker push private-docker-nexus.c-xxxxxxx.kyma.shoot.live.k8s-hana.ondemand.com/hello-world
Using default tag: latest
The push refers to repository [private-docker-nexus.c-xxxxxxx.kyma.shoot.live.k8s-hana.ondemand.com/hello-world]
9c27e219663c: Pushed 
latest: digest: sha256:90659bf80b44ce6be8234e6ff90a1ac34acbeb826903b02cfa0da11c82cbc042 size: 525

You could also use Nexus to browse the repo

Using the private registry in deployment

To make use of the registry in deployment recipes in other namespaces, you need to store the access credentials into a kubernetes secret in each namespace.

kubectl create secret docker-registry myprivatenexus 
   --docker-server=private-docker-nexus.c-xxxxxxx.kyma.shoot.live.k8s-hana.ondemand.com 
   --docker-username=remi 
   --docker-password=<mysecretpassword> 
   --docker-email=remi.astier@sap.com 
   -n <namespace with your deployment>

Finally, use the secret name and docker tag in a deployment file:

Conclusion

A private docker registry, or any readily available helm chart can be deployed using similar steps of installing the chart and configuring the services and api rules.

Important note

After just two days of part time work, the filesystem has filled and the registry in read only mode. The steps to increase the filesystem are:

1. Shut down the pod
   kubectl patch deployment -n tools mynexus-repo-nexus-repository-manager -p '{ "spec": { "replicas": 0 }}'​
2. Increase the volume
   kubectl patch -n tools pvc mynexus-repo-nexus-repository-manager-data -p '{"spec": { "resources": { "requests": { "storage": "41Gi" }}}}'
3. Restart the pod
   kubectl patch deployment -n tools mynexus-repo-nexus-repository-manager -p '{ "spec": { "replicas": 1 }}'​