Learning: Kubernetes – Controller & Cloud Controller

Controller

k8s controller is like a thermostat in a room. It checks the current temperature and maintains the temperature by turning off & on the switch. Same here the k8s controller checks the current state and verify that if the current state is equal to the desired state or not. If the current state is not matched with the desired state it makes the necessary changes and brings to the desired state.

Types of the controller –

  • ReplicaSet – It is responsible for maintaining the set desired number of pods.
  • Deployment – Deployment is the most common way to get your app on Kubernetes. It maintains a ReplicaSet with the desired configuration.
  • StatefulSet – A StatefulSet is used to manage stateful applications with persistent storage.
  • Job – A Job creates one or more short-lived Pods and expects them to successfully terminate.
  • CronJob – A CronJob creates Jobs on a schedule.
  • DaemonSet – A DaemonSet ensures that all (or some) Nodes run a copy of a Pod. As nodes are added to the cluster, Pods are added to them.

Cloud Controller

The cloud controller manager lets you link your cluster into your cloud provider’s API, and separates out the components that interact with that cloud platform from components that only interact with your cluster.

  • Different functions
    • Node Controller –
      • It updates node objects when new servers are created in the cloud.
      • Annotating and labeling the node object with the cloud-specific information.
      • Obtain node hostname & network address.
      • Checks the node health. If the node has been deleted from the cloud then it also removes the node from the k8s cluster.
    • Route Controller – It configures the routes properly so the nodes on the k8s cluster can communicate with each other.
    • Service Controller – It interacts with the cloud provider’s API to set up a load balancer and other infrastructure components.
Advertisement

Learning: Kubernetes – ConfigMaps & Secrets

Every application has configuration data like API key, DB URL, DB user, DB password, etc. Yes, you can hardcode these data in your application but after some time it will be unmanageable. You need some kind of dynamic solution where you define all these data once and every component of your cluster can access these data.

Suppose our application DB URL changes for that we need to change every place where the URL is used. So, for this our application needs to be rebuilt again and we have to change the code inside the application. To encounter this situation we use ConfigMaps and Secrets for storing the application’s required data.

ConfigMaps – A ConfigMaps is an API object used to store non-confidential data in key-value pairs. Pods can consume ConfigMaps as environment variables, command-line arguments, or as configuration files in a volume.

Ex – ConfigMaps contains the URL of the database, user, and other non-credential data.

Secrets – A secret is similar to ConfigMaps, except a secret is used for sensitive information such as credentials. One of the main differences is that you have to explicitly tell kubectl to show you the contents of a secret.

  • It is encoded in base64.

Learning: Kubernetes – Labels & Selectors, Finalizers

Labels & Selectors

Labels – Labels are key-value pairs attached to pods, ReplicaSet & Services. They are used for identifying objects for pods and ReplicaSet. It can be added at the creation time or can be modified or added at the run time.

Some properties –

  • must be 63 characters or less (can be empty),
  • unless empty, must begin and end with an alphanumeric character ([a-z0-9A-Z]),
  • could contain dashes (“), underscores (_), dots (.), and alphanumeric between.

Selectors – Kubernetes API currently supports two type of selectors −

  • Equality-based selectors – We use = == != as equality selector.
environment = production
tier != frontend
  • Set-based selectors – It allows filtering of object in a set of values. There are three king of operator allowed in notin exists.
environment in (production, qa)
tier notin (frontend, backend)
partition
!partition

Finalizers

Deleting an object isn’t as simple as it looks because it involves a lot of conditional checks about the used resources. Finalizers are certain conditions be met before an object can be deleted.

When you run kubectl delete namespace/example k8s check the finalizers defined on that object.

  1. Issue a deletion command. – Kubernetes marks the object as pending deletion. This leaves the resource in the read-only “Terminating” state.
  2. Run each of the actions associated with the object’s Finalizers. – Each time a Finalizer action completes, that Finalizer is detached from the object, so it’ll no longer appear in the metadata.finalizers field.
  3. Kubernetes keeps monitoring the Finalizers attached to the object. – The object will be deleted once the metadata.finalizers field is empty, because all Finalizers were removed by the completion of their actions.

Learning: Kubernetes – Service, Scalability, Rolling Updates

Service

There multiple challenges with pods. Suppose we have two pods with one front-end and one back-end. Now we have couple of questions –

  • How does the front-end app expose to the outside world?
  • How the front-end app talks to the back-end app?
  • When a pod dies a new pod gets created and get assigned with a new IP Address. How to resolve Pod IP changes, when pod die?

So, the services are the way of grouping of pods in a cluster. We can have as many as services in cluster. There are mainly three type of services in k8s –

  1. ClusterIP – It actually deals with the pod IP change problem. It’s a static IP address that can be attached with each pod. So even the pod dies the service stays in place and don’t change. Exposes the Service on an internal IP in the cluster. Here the service only reachable within the cluster.
  2. NodePort – Makes a Service accessible from outside the cluster.
  3. Load Balancer – Creates an external load balancer in the current cloud (if supported) and assigns a fixed, external IP to the Service.
# Get the running services
$ kubectl get services

# Expose service to outside of the world
$ kubectl expose deployment/kubernetes-bootcamp --type="NodePort" --port 8080

# Delete a specific service
$ kubectl delete service -l app=kubernetes-bootcamp

Scalability

When we want to scale our app then we create multiple replica of the pods on the nodes and to balance the request on each pods we use load balancer service.

Rolling Update

K8s allows us to do rolling updates and let’s see how it does –

  • First it creates the new pod with updated config.
  • Then it replaces the new pods with old ones one by one and change the pod IP addresses.

It allows app to update in zero down time.

# Update image of the application
$ kubectl set image

# Get Rollout update status
$ kubectl rollout status <service_name>

# Get the service info
$ kubectl describe <service_name>

# Roll Back to the deployment to your last working version
$ kubectl rollout undo <deployment_name>

Learning: Kubernetes – Pods and ReplicaSet Simplified

Pods

A Pod is the smallest execution unit of a Kubernetes application. Each Pod represents a part of a workload that is running on your cluster.

We usually have one pod per application. Inside that pod we could have multiple container.

  • A Pod is a Kubernetes abstraction that represents a group of one or more application containers and some shared resources.
    • It has shared volumes.
    • Cluster IP(Every pod has unique IP even in same Node)
    • Info about how to run container.
  • We don’t deal with containers instead we work with pods.
  • If a container dies inside a pod it will be automatically restarted.
  • Each pod is tied to one node until termination.
  • Pods that are running inside k8s are only visible from other pods and services inside the k8s cluster.
  • We have to expose the app to outside the k8s.

Multiple Container Pods – The pods are always designed to support multiple correlated containers. The containers in a pod is automatically scheduled in same VM or physical machine in the cluster.

The containers can communicate to each other and share resources.

Pods Networking –

  • Each pod is assigned with a unique IP address.
  • Each container in pods share the network share the same IP address with port.
  • The containers inside a pod can communicate to each other with localhost.
  • The containers inside a pod can also communicate using Inter Process Communication.

Life Cycle of a Pod

  • A pod is said to be ephemeral.
  • A pod is never rescheduled to a different node Instead the pod is replaced by a new one.
  • If a node fails the pods assigned to it also fail.

Generally a pod has 5 phases –

  1. Pending – Pod has been accepted by the cluster but one or more container haven’t been setup.
  2. Running – A pod has been bound to a node and containers have started.
  3. Succeeded – All containers in the pod have been terminated successfully.
  4. Failed – At least one container have been terminated in failure.
  5. Unknown – For some reason the state of pod could not be obtained.
# Create a deployment
$ kubectl create deployment kubernetes-bootcamp --image=gcr.io/google-samples/kubernetes-bootcamp:v1

# Get deployment info
$ kubectl get deployments

# Get the list of pods running
$ kubectl get pods

# See which containers are running inside a pod
$ kubectl describe pods

# Run a command inside a container
$ kubectl exec $POD_NAME -- env

# Open bash inside a container
$ kubectl exec -it $POD_NAME -- bash

ReplicaSet

We don’t create the pods directly. The reason is suppose we need 4 pods in our deployment always and if we create the pods directly and the one pod goes down then we have to create the pods manually.

That’s why we use ReplicaSet. It is a management system that ensure that I have the desired set of pods in the k8s cluster. And the controller check the current state with the desired state and see if the current pod count match the ReplicaSet count or not. If not it creates or deletes pods.

# Get the replica set
$ kubectl get rs

# Scale up the app and change replicaset
$ kubectl scale deployments/kubernetes-bootcamp --replicas=4

# Scale down the app
$ kubectl scale deployments/kubernetes-bootcamp --replicas=2

# To see a pod in managed by ReplicaSet
$ kubectl get pods <pod_name> -o yaml

# Delete the ReplicaSet
$ kubectl delete rs <replica_name>

# Delete the replica set but keep the pods
$ kubectl delete rs <replica_name> --cascade=false

Learning: Kubernetes – Cluster, Control Plane, Nodes

Cluster

It is a set of machines connected and work together to run as a single unit. The idea is deploying the containerized application without tying them to a specific machine.

There are mainly two component of any k8s cluster –

  • Master Node or Control Plane
  • Worker Nodes
  • Virtual Network

Control Plane

In general it is the control panel of a cluster and manages the entire cluster.

  1. It runs the API server which works as the entry point for the Kubernetes cluster.
  2. It runs the controller manager which keeps an overview of the cluster and Maintain application desired state.
  3. It runs scheduler which is responsible for scheduling containers and pods for different nodes based on workload and available server resources on each node. After deciding which Node to use for pod or container creation it actually sends the requests to the particular node’s kubelet process and kubelet does the creation of the pod and containers.
  4. Another important thing that runs is ETCD Key-value storage which holds the current state of the cluster.

Nodes

It is a physical computer or VM that serves as a worker machine in a k8s cluster.

It is a physical computer or VM that serves as a worker machine in a k8s cluster.

  1. Each worker node have docker containers of different application deployed on it.
  2. The kubelet manage the node ****talk to the control plane.
  3. The node use Kubernetes API to communicate to the control plane.
  4. Two node can’t have the same name as name identifies the node.
# Start the cluster with minikube
$ minikube start

# Get the cluster info
$ kubectl cluster-info

# Get node information
$ kubectl get nodes

What is Kubernetes Cluster API and Setup a Local Cluster API using Docker

I have came across the term cluster API while I was contributing to Flatcar Linux. But I didn’t knew much about it then. In recent days I have been tinkering around the Kubernetes and started learning what cluster API is and what it does. So Cluster API or CAPI is a tool from the Kubernetes Special Interest Group(SIG) that uses Kubernetes-style APIs and patterns to automate cluster lifecycle management for platform operators.
In general term it is the project that helps manage your k8s cluster no matter where they are including various cloud providers. Because a k8s cluster include a lot of component from hardware, software, services, networking, storage and so on and so forth.

Motivation

I wrote this blog in the motivation of setting it up locally and contribute in this project. In recent days I have came across a lot of Computer Science core subjects like Computer Networking, Database Management System and really amazed to see the interconnection with the distributed systems.
I am still very new in the operation of various cloud provider but in the near future I am willing to learn those thing and apply Kubernetes over there.
I also want to participate in the GSoC and work in this particular project and Improve CAPG by adding more features and support GKE.

Setting up CAPI locally with Docker

Requirements : You need to have the following packages installed in your system before starting it –

Step 1 –

Infrastructure Provider – It is like a provider which is providing compute & resources in order to spin a cluster. We are going to use docker as our infrastructure here.

  • Create a kind config file for allowing the Docker provider to access Docker on the host:
cat > kind-cluster-with-extramounts.yaml <<EOF
kind: Cluster
apiVersion: kind.x-k8s.io/v1alpha4
nodes:
- role: control-plane
  extraMounts:
    - hostPath: /var/run/docker.sock
      containerPath: /var/run/docker.sock
EOF
  • Then I create a kind cluster using the following config file –
kind create cluster --config kind-cluster-with-extramounts.yaml

Step 2 –

Now installing the clusterctl tool to manage the lifecycle of a CAPI management cluster –

  • Installation in linux OS – (For other OS – ref)
$ curl -L https://github.com/kubernetes-sigs/cluster-api/releases/download/v0.4.0/clusterctl-linux-amd64 -o clusterctl
$ chmod +x ./clusterctl
$ sudo mv ./clusterctl /usr/local/bin/clusterctl
$ clusterctl version

Step 3 –

Now it’s time for use the clusterctl to transform the kind cluster to a management cluster by clusterctl init command. The command accepts a list of provider.

Management Cluster – A Management cluster is a Kubernetes cluster that manages the lifecycle of Workload Clusters. A Management Cluster is also where one or more Infrastructure Providers run, and where resources such as Machines are stored.

  • I am using docker as my infrastructure so I will use the command below –
clusterctl init --infrastructure docker

Step 4 –

Now it’s time for creating a workload cluster.

Workload Cluster – A workload cluster is a cluster created by a ClusterAPI controller, which is not a bootstrap cluster, and is meant to be used by end-users.

  • Now we use clusterctl generate cluster to generate a YAML file to create a workload cluster.
clusterctl generate cluster test-workload-cluster --flavor development \
--kubernetes-version v1.21.2 \
--control-plane-machine-count=3 \
--worker-machine-count=3 \
> test-workload-cluster.yaml
  • Now apply the file to create the workload cluster –
kubectl apply -f test-workload-cluster.yaml

Step 5 –

Now we verify our workload cluster and access it.

  • Get the status of the cluster
kubectl get cluster
  • View the cluster and it’s resources
clusterctl describe cluster test-workload-cluster
  • Check the status of the control plane
kubectl get kubeadmcontrolplane

Note – The controller plane won’t be ready untill the next step when I install the CNI (Container Network Interface).

Step 6 –

Now it’s the time to setup the CNI solution

  • First get the workload cluster kubeconfig
clusterctl get kubeconfig test-workload-cluster > test-workload-cluster.kubeconfig
  • It will use calico for an example.
kubectl --kubeconfig=./test-workload-cluster.kubeconfig apply -f https://docs.projectcalico.org/v3.18/manifests/calico.yaml
  • After some time the node should be up and running.
kubectl --kubeconfig=./test-workload-cluster.kubeconfig get nodes

Step 7 –

Now it’s the last phase to delete the resources –

  • Delete the workload cluster
kubectl delete cluster test-workload-cluster
  • Delete the management cluster
kind delete cluster

Stateful vs Stateless Application

State – It is the session data that is been generated at the time of a client connects to a server.

  • Stateful – In stateful architecture let’s imagine a scenario of a eCommerce website that has 3 servers and there are 3 user connected to three servers and all the user session is store in the servers accordingly. Now let’s imagine a scenario of one of the three server goes down and all the state information about the users that were associated with it also lost. Now all that users have to do all the process again after establishing connection to a new server. So that’s a big issue in scalability and availability.
  • Stateless – In stateless architecture all the state information about the client is been stored in the shared storage connected to all the servers. Now let’s suppose a server goes down and and because all the state information is shared so if a client establish a connection with new server all the state information can be restored from the shared storage.

In conceptual we are going stateless but basically the we cannot eliminate the state entirely. There must be some place we are storing the states. We call it shared storage.

To be continued