My Kubernetes Journey
Being a software developer at heart, I am always interested and drawn towards new technologies and platforms. About 6 months back, I decided to switch from the on-premise world to the cloud world with respect to software development. I mean, cloud has been the new rage for quite a while now, especially for enterprises and startups.
I changed my product team and moved to a cloud based team, which incidentally was a new team which was just about to be created. Since I was from the “older” on-premise world, this was more of a challenge to me, but it was something that I was looking forward to.
As part of our strategy, we decided to go with Cloud Foundry (CF) as our Cloud Platform for our project. Now, you must be thinking, “The title of the blog talks about Kubernetes, but why on earth is this guy talking about on-premise world and Cloud Foundry?” Right? Just stay with me, as these experiences are an important factor which lead me to pick up on Kubernetes. Just stick with me for a minute or two, and you will fully understand what lead me to start my “R&D” on Kubernetes.
Ok, now going back to my original story, I started to go over the concepts and cloud architecture as I would be investing most of my time on this. We had our KTs(Knowledge Transfers) in terms of trainings, recordings, online material, etc which actually gave a pretty good idea about what CF could deliver and was capable of.
I am not sure if I had more interest in the backend-development or if I was way too much interested in the OS and filesystem, some of the aspects which CF dealt with was not too comfortable for me. Now before you say that I have no idea what I am talking about and that CF is amazing, I would like to add that this is my personal opinion and not something I am preaching around anywhere. I have nothing against CF, except for certain CF restrictions in terms of the control provided to developers. I was not comfortable with this. Then again, this was my personal opinion and wanted to explore around for a bit more.
It was at this time that I met a colleague of mine who had prior expertise with Kubernetes and Cloud Foundry. I was asking him on how I could migrate an internal project that I had developed to the Cloud and we had a few discussions. I mean, when you market you application as “on-premise”, people look at you with contempt and also ask questions like “Is this cloud native”, “Is it deployed on the cloud”, “Why is this application still running on-premise”, etc, etc,
My colleague suggested the concept of containers and this opened up my mind to a horizon of new ideas. He said that containers gave all the flexibilities that I need for my development and could scale, at the same time avoiding the age old developer problem “It works on my dev landscape, but doesn’t work on the production landscape”. He explained in brief the advantages of using containers and on the spot, I had made up my mind. For any future development involving my personal projects, “containers” was the way to go.
Containers – Kick-off:
Once I had set my mind on containers, I started my research and easily came to the conclusion that Docker was the easiest and the most reliable way to proceed ahead. I read up on the documentation, watched a few online recordings and started to dabble around a bit. As part of my learning and development, I used Docker to create an image for Jupyter notebooks as we had a few ML use cases to be evaluated as part of a POC (proof-of-concept). I am not elaborating on that right now, but if you are interested you can read about it @ https://blogs.sap.com/2018/10/28/jupyter-python-notebooks-on-docker/
But just Docker was fine, since this was a stand-alone scenario. But what happens when I need to link and orchestrate the communication between multiple apps running in their individual containers? What do I do in-case a container goes down? Do I restart it manually? Or maybe write a script to check and see if the app is running as expected and then restart the container? These
On further reading, I came to know that there are container orchestrators which exactly solves these issues. The first name I came across in my reading was Docker Swarm (that’s to be expected, since most of my reading material was from the Official Docker documentation.
At the same time, I started to see references to another orchestrator named Kubernetes (which was abbreviated as k8s). Upon further reading, I became aware of the fact that Kubernetes was built at Google, which was based on the Borg container technology which Google has been using for 10+ years. Moreover, Google had open sourced and handed it over to CNCF (Cloud Native Computing Foundation) community, from where k8s graduated as a top level project. For people who are new to CNCF and want to know how their app graduation process works, here is a short explanation. Apps graduate only once they are stable, voted and used by numerous users in the community. This made it ample clear that this platform had won the competition for the container orchestrator and like most people, I decided that k8s will become my Cloud Platform of choice.
I did spend the next few weeks reading about this new platform and was ready to dabble on the same with a developer’s curiosity. The more I read, the more I understood how massive this platform is. I was sure that the only way I could learn and understand about this was to try out implementations and practice. I started searching online for examples and found a few.
I started off with a simple “deployment” and a “service”, which created a container and exposed on the local node which was running on Kubernetes on Docker Desktop application.
It was at this point that I came across the “Minikube”, a single node Kubernetes cluster on the local system. This was a way to mimic the actual k8s implementation which are provided by hyperscalers like GCP(Google Cloud Platform), AWS(Amazon Web Services) and Azure. I did set up a Minikube cluster locally and tried to run some scenarios, but encountered multiple errors at runtime, which was making my life miserable. If you search for the errors, you could find a lot of content online, some of which are applicable, while a lot are deprecated.
This led me to write the following blog about setting up Kubernetes on Minikube, talking about the configurations and setup so that all the issues that I faced are resolved and no one else have to jump through hoops of fire to get it to work.
What I am also planning to achieve here (or try to 😉 ) is to write a simple and end to end setup of minikube and then write about K8S topics with which was initially confusing for me. I would like to help out people who want to kick-start their K8S development. Please note that this wont be an in-depth tutorial, but rather something that can be used to understand the basic concepts.
I will link and publish new pages to this blog as and when I complete them.
Stay tuned for updates 🙂