Learning: Kubernetes – Cluster, Control Plane, Nodes

Cluster

It is a set of machines connected and work together to run as a single unit. The idea is deploying the containerized application without tying them to a specific machine.

There are mainly two component of any k8s cluster –

  • Master Node or Control Plane
  • Worker Nodes
  • Virtual Network

Control Plane

In general it is the control panel of a cluster and manages the entire cluster.

  1. It runs the API server which works as the entry point for the Kubernetes cluster.
  2. It runs the controller manager which keeps an overview of the cluster and Maintain application desired state.
  3. It runs scheduler which is responsible for scheduling containers and pods for different nodes based on workload and available server resources on each node. After deciding which Node to use for pod or container creation it actually sends the requests to the particular node’s kubelet process and kubelet does the creation of the pod and containers.
  4. Another important thing that runs is ETCD Key-value storage which holds the current state of the cluster.

Nodes

It is a physical computer or VM that serves as a worker machine in a k8s cluster.

It is a physical computer or VM that serves as a worker machine in a k8s cluster.

  1. Each worker node have docker containers of different application deployed on it.
  2. The kubelet manage the node ****talk to the control plane.
  3. The node use Kubernetes API to communicate to the control plane.
  4. Two node can’t have the same name as name identifies the node.
# Start the cluster with minikube
$ minikube start

# Get the cluster info
$ kubectl cluster-info

# Get node information
$ kubectl get nodes
Advertisement

Daily Learning: Computer Networks – Securing TCP/IP

There are 5 points that to me mentioned while understanding security in TCP/IP –

  • Encryption – It’s the scramble of data such a manner that in between the data can’t be read. Also at the receiving end the the data must be descramble.
  • Integrity – It’s the guarantee that data that is received is same as originally sent.
  • Nonrepudiation A person cannot deny that they took a specific action.
  • Authentication – It means that who ever is accessing the data is the person you want to access the data. username & password defines the authentication.
  • Authorization – It means what an authenticated person do with the data.

Encryption

A packet of data on the Internet often comes with a port number encapsulated in the segment or datagram, for example, so a bad guy quickly knows what type of data he’s reading. All data starts as cleartext, which roughly means the data hasn’t been encrypted yet.

Here comes the encryption and we use cipher. A cipher is a general term for a way to encrypt data. An algorithm is the mathematical formula that underlies the cipher. When you run plaintext through a cipher algorithm using a key, you get the encrypted ciphertext.

Types of encryption –

  • Symmetric Encryption – It is the encryption algorithm which use single key to both encrypt and decrypt the data. So the key must be shared between the sender and receiver. There are major two kind of symmetric key –
    • Block cipher – Here it divide the data in blocks and encrypt each block. usually 128 bit block.
    • Stream cipher – Here the algorithm encrypt each bit coming from stream of bits.
  • Asymmetric Encryption – There was a major drawback in the symmetric encryption that the key is tampered then the communication is vulnerable. For that we use asymmetric encryption. Here suppose there is two people Alice & David and alice wants to send the data to the david then alice will create a key pair of public and private key. The public key is used for the encryption and the private key is used for the decryption. So the alice will give her public key to david and david will encrypt the data and send it to the david. Now alice will decrypt the data with the help of her private key.

Hash

  • It’s a mathematical function that we run in a string and get a fixed length of string(checksum or message digest).
  • The message digest will always be same length regardless of the input string length.
  • The hash is a one way function that we can’t get back the message from the hash.

Uses –

  • When we download any file from the internet the download provider also provide a message digest of the download file.
  • We first download the checksum and download the file.
  • Then we compare our checksum with the downloaded checksum.
  • If the checksum is not correct then the data has been tampered in the middle.

Digital Signature

It is the proof of truth about someone took some action in a network. and they can’t deny it.

  • First sender hashes the message and encrypt with sender’s private key.
  • Then send the data to the receiver.
  • Then the receiver decrypts the message with the help of sender’s public key.
  • If the hash match then it’s the proof that the sender has sent the message.

SSH

The invention of SSH was heavily related to the telnet protocol because the telnet protocol was completely unsecured that everything was transferred in plain text. Then a student from Helsinki University Of Technology Tatu Ylonen created another protocol called SSH after his network was breached because of telnet.

Working Principal

  • When a client wants to connect to the server for the first time the server sends it’s public key to the client.
  • Then the client create a session ID and encrypt it using the public key and sends it back to the server.
  • Then the server decrypt the session ID using it’s private key and use in all the data communication going forward.
  • Then the client server decides the type of encryption will be used for the session(Generally AES).

SSH can use public key for encryption and we can turn off the password based authentication.

Use Public/Private Key for authentication –

  • client first generate a key pair using ssh-keygen.
  • The the public key is sent to the server and the private key is kept safe in the client machine.
  • WHen you connect to the server the client create a signature using it’s private key and send to server.
  • The server check the signature using it’s public key and if everything matches you are authenticated to the server.

Create a key Value storage using Golang – Part 1

A few days back I gave a interview in a company for Golang developer intern. There they asked me about a lot of questions and some I cracked and some I couldn’t cracked. The things I failed miserably was Go interfaces and Go Concurrency(Channels & atomic package). I took it as a motivation and learnt the Go interface again and wrote a blog on that. Now It was time for me to develop a project so my knowledge become more concrete. Nabarun gave a idea to make a key-value storage using Go that will support multiple storage types.

So I started building the project. First create a directory api/models from the root of your directory. Then create a file called records.go and write the following code –

type InMemory struct {
	Data map[string]string `json:"data"`
}

This code will be responsible for in memory storage of the key – value storage. Now lets define some interface that will be help us when we will be enhancing our app to support more storage systems like file structure. Add the following code in records.go

type StorageSetter interface {
	Store(string, string)
}

type StorageGetter interface {
	List() map[string]string
	Get(string) (string, error)
}

type StorageDestroyer interface {
	Delete(string) error
}

type Storage interface {
	StorageSetter
	StorageGetter
	StorageDestroyer
}

Now let’s see why I have written multiple interface like setter and Getter? Because it is always best practice to keep the interface small. It actually helps increasing abstraction.

Now let’s define a helper function in records.go that will return the InMemory structs.

// Return In Memory struct
func NewCache() *InMemory {
	return &InMemory{Data: make(map[string]string, 2)}
}

Let’s now create the methods in records.go which will do operations on the struct –

func (r *InMemory) Store(key, val string) {
	r.Data[key] = val
}

func (r *InMemory) List() map[string]string {
	return r.Data
}

func (r *InMemory) Get(key string) (string, error) {
	val, ok := r.Data[key]
	if !ok {
		return "", errors.New("key not found")
	}
	return val, nil
}

func (r *InMemory) Delete(key string) error {
	_, ok := r.Data[key]
	if !ok {
		return errors.New("key not found")
	}
	delete(r.Data, key)
	return nil
}

Now let’s define our server so create a directory api/controllers and create base.go file inside it. And write the code inside it.

type Server struct {
	Router *http.ServeMux
	Cache  models.Storage
}

This Server struct will contain dependency of the server which is typically the router and the storage interface.

Now create a server.go inside the api directory and write the code –

package api

import (
	"flag"

	"github.com/aniruddha2000/goEtcd/api/controllers"
)

var server controllers.Server

// Initialize and run the server
func Run() {
	var storageType string

	flag.StringVar(&storageType, "storage-type", "in-memory",
		"Define the storage type that will be used in the server. By defaut the value is in-memory.")
	flag.Parse()

	server.Initialize(storageType)
	server.Run("8888")
}

Here you can see it is taking the flag from the command line and passing it in the Initialize method and calling Run method to run the server in the port 8888. Now let’s define these two Initialize & Run method in the base.go file –

func (s *Server) Initialize(storageType string) {
	s.Router = http.NewServeMux()

	switch storageType {
	case "in-memory":
		s.Cache = models.NewCache()
	case "disk":
		s.Cache = models.NewDisk()
	default:
		log.Fatal("Use flags `in-memory` or `disk`")
	}

	log.Printf("Starting server with %v storage", storageType)

	s.initializeRoutes()
}

// Run the server on desired port and logs the status
func (s *Server) Run(addr string) {
	cert, err := tls.LoadX509KeyPair("localhost.crt", "localhost.key")
	if err != nil {
		log.Fatalf("Couldn't load the certificate: %v", cert)
	}

	server := &http.Server{
		Addr:    ":" + addr,
		Handler: s.Router,
		TLSConfig: &tls.Config{
			Certificates: []tls.Certificate{cert},
		},
	}

	fmt.Println("Listenning to port", addr)
	log.Fatal(server.ListenAndServeTLS("", ""))
}

Here you can see Initialize method is setting the router for the server and then setting the storage for different storage and at the last it is initializing the routes.

In the Run method it is loading the certificate and setting up the server and running the server at the end.

Now let’s define initializeRoutes function that we saw in the last initialize method in the base.go. Create a routes.go file in side api/controllers

func (s *Server) initializeRoutes() {
	s.Router.HandleFunc("/record", s.Create)
	s.Router.HandleFunc("/records", s.List)
	s.Router.HandleFunc("/get/record", s.Get)
	s.Router.HandleFunc("/del/record", s.Delete)
}

Now we will see the implementation of the route controllers. Create a cache.go file inside the api/controllers and paste the below code –

package controllers

import (
	"log"
	"net/http"

	j "github.com/aniruddha2000/goEtcd/api/json"
)

func (s *Server) Create(w http.ResponseWriter, r *http.Request) {
	if r.Method == "POST" {
		r.ParseForm()
		key := r.Form["key"]
		val := r.Form["val"]

		for i := 0; i < len(key); i++ {
			s.Cache.Store(key[i], val[i])
		}

		j.JSON(w, r, http.StatusCreated, "Record created")
	} else {
		j.JSON(w, r, http.StatusBadRequest, "POST Request accepted")
	}
}

func (s *Server) List(w http.ResponseWriter, r *http.Request) {
	if r.Method == "GET" {
		records := s.Cache.List()
		j.JSON(w, r, http.StatusOK, records)
	} else {
		j.JSON(w, r, http.StatusBadRequest, "GET Request accepted")
	}
}

func (s *Server) Get(w http.ResponseWriter, r *http.Request) {
	if r.Method == "GET" {
		keys, ok := r.URL.Query()["key"]
		if !ok || len(keys[0]) < 1 {
			log.Println("Url Param 'key' is missing")
			return
		}
		key := keys[0]

		val, err := s.Cache.Get(key)
		if err != nil {
			j.JSON(w, r, http.StatusNotFound, err.Error())
			return
		}
		j.JSON(w, r, http.StatusOK, map[string]string{key: val})
	} else {
		j.JSON(w, r, http.StatusBadRequest, "POST Request accepted")
	}
}

func (s *Server) Delete(w http.ResponseWriter, r *http.Request) {
	if r.Method == "DELETE" {
		keys, ok := r.URL.Query()["key"]
		if !ok || len(keys[0]) < 1 {
			log.Println("Url Param 'key' is missing")
			return
		}
		key := keys[0]

		err := s.Cache.Delete(key)
		if err != nil {
			j.JSON(w, r, http.StatusNotFound, err.Error())
			return
		}
		j.JSON(w, r, http.StatusNoContent, map[string]string{"data": "delete"})
	} else {
		j.JSON(w, r, http.StatusBadRequest, "DELETE Request accepted")
	}
}

Here you can see the controller that will handle different route traffics and call the record methods and doing the operations.

I have creates a helper JSON method to reduce redundant code while writing the route controllers. create a api/json directory and crate a json.go file. Paste the code below –

func JSON(w http.ResponseWriter, r *http.Request, statusCode int, data interface{}) {
	w.Header().Set("Location", fmt.Sprintf("%s%s", r.Host, r.RequestURI))
	w.WriteHeader(statusCode)
	json.NewEncoder(w).Encode(data)
}

In the next part I will walk you though how to extend this application to support disk based storage along side with In-Memory storage.

What is Kubernetes Cluster API and Setup a Local Cluster API using Docker

I have came across the term cluster API while I was contributing to Flatcar Linux. But I didn’t knew much about it then. In recent days I have been tinkering around the Kubernetes and started learning what cluster API is and what it does. So Cluster API or CAPI is a tool from the Kubernetes Special Interest Group(SIG) that uses Kubernetes-style APIs and patterns to automate cluster lifecycle management for platform operators.
In general term it is the project that helps manage your k8s cluster no matter where they are including various cloud providers. Because a k8s cluster include a lot of component from hardware, software, services, networking, storage and so on and so forth.

Motivation

I wrote this blog in the motivation of setting it up locally and contribute in this project. In recent days I have came across a lot of Computer Science core subjects like Computer Networking, Database Management System and really amazed to see the interconnection with the distributed systems.
I am still very new in the operation of various cloud provider but in the near future I am willing to learn those thing and apply Kubernetes over there.
I also want to participate in the GSoC and work in this particular project and Improve CAPG by adding more features and support GKE.

Setting up CAPI locally with Docker

Requirements : You need to have the following packages installed in your system before starting it –

Step 1 –

Infrastructure Provider – It is like a provider which is providing compute & resources in order to spin a cluster. We are going to use docker as our infrastructure here.

  • Create a kind config file for allowing the Docker provider to access Docker on the host:
cat > kind-cluster-with-extramounts.yaml <<EOF
kind: Cluster
apiVersion: kind.x-k8s.io/v1alpha4
nodes:
- role: control-plane
  extraMounts:
    - hostPath: /var/run/docker.sock
      containerPath: /var/run/docker.sock
EOF
  • Then I create a kind cluster using the following config file –
kind create cluster --config kind-cluster-with-extramounts.yaml

Step 2 –

Now installing the clusterctl tool to manage the lifecycle of a CAPI management cluster –

  • Installation in linux OS – (For other OS – ref)
$ curl -L https://github.com/kubernetes-sigs/cluster-api/releases/download/v0.4.0/clusterctl-linux-amd64 -o clusterctl
$ chmod +x ./clusterctl
$ sudo mv ./clusterctl /usr/local/bin/clusterctl
$ clusterctl version

Step 3 –

Now it’s time for use the clusterctl to transform the kind cluster to a management cluster by clusterctl init command. The command accepts a list of provider.

Management Cluster – A Management cluster is a Kubernetes cluster that manages the lifecycle of Workload Clusters. A Management Cluster is also where one or more Infrastructure Providers run, and where resources such as Machines are stored.

  • I am using docker as my infrastructure so I will use the command below –
clusterctl init --infrastructure docker

Step 4 –

Now it’s time for creating a workload cluster.

Workload Cluster – A workload cluster is a cluster created by a ClusterAPI controller, which is not a bootstrap cluster, and is meant to be used by end-users.

  • Now we use clusterctl generate cluster to generate a YAML file to create a workload cluster.
clusterctl generate cluster test-workload-cluster --flavor development \
--kubernetes-version v1.21.2 \
--control-plane-machine-count=3 \
--worker-machine-count=3 \
> test-workload-cluster.yaml
  • Now apply the file to create the workload cluster –
kubectl apply -f test-workload-cluster.yaml

Step 5 –

Now we verify our workload cluster and access it.

  • Get the status of the cluster
kubectl get cluster
  • View the cluster and it’s resources
clusterctl describe cluster test-workload-cluster
  • Check the status of the control plane
kubectl get kubeadmcontrolplane

Note – The controller plane won’t be ready untill the next step when I install the CNI (Container Network Interface).

Step 6 –

Now it’s the time to setup the CNI solution

  • First get the workload cluster kubeconfig
clusterctl get kubeconfig test-workload-cluster > test-workload-cluster.kubeconfig
  • It will use calico for an example.
kubectl --kubeconfig=./test-workload-cluster.kubeconfig apply -f https://docs.projectcalico.org/v3.18/manifests/calico.yaml
  • After some time the node should be up and running.
kubectl --kubeconfig=./test-workload-cluster.kubeconfig get nodes

Step 7 –

Now it’s the last phase to delete the resources –

  • Delete the workload cluster
kubectl delete cluster test-workload-cluster
  • Delete the management cluster
kind delete cluster

Daily Learning: Computer Networks – Access Control Methods – CSMA(Carrier Sensing Multiple Access)/CD(Collision Detection)

Properties

  1. There can be multiple stations in a network.
  2. It will sense if the transmission line is busy or not. If it is not busy then it will transmit the data.
  3. It will also sense for any type of collision while sending the data.

Persistence Methods

  1. 1 Persistent – It sense the medium continuously. When the medium is free it sends the packet imediately.
  2. Non-Persistent – It first sense the medium then waits for a random amount of time then again sense.
  3. P Persistent – It fist sense the medium continuously then when the medium is free it generate a random number and check if the number is less than the probability of which host if the number is less than some host’s probability then that host will transmit the data.

CSMA

Vulnerable Time – It is the total propagation time Tp. If first bit of the packet reaches to the end of the medium then every station will heard of the transmission and no one will transmit.

CSMA/CD

The Carrier Sense Multiple Access/ Collision Detection protocol is used to detect a collision in the media access control (MAC) layer. Once the collision was detected, the CSMA CD immediately stopped the transmission by sending the signal so that the sender does not waste all the time to send the data packet. Suppose a collision is detected from each station while broadcasting the packets. In that case, the CSMA CD immediately sends a jam signal to stop transmission and waits for a random time context before transmitting another data packet. If the channel is found free, it immediately sends the data and returns it.

It is used in the wired medium and used by Ethernet.

Transmission Time – Tt = 2 * Tp

CSMA/CA

CSMA stands for Carrier Sense Multiple Access with Collision Avoidance. It means that it is a network protocol that uses to avoid a collision rather than allowing it to occur, and it does not deal with the recovery of packets after a collision. It is similar to the CSMA CD protocol that operates in the media access control layer. In CSMA CA, whenever a station sends a data frame to a channel, it checks whether it is in use. If the shared channel is busy, the station waits until the channel enters idle mode. Hence, we can say that it reduces the chances of collisions and makes better use of the medium to send data packets more efficiently.

It is used in the wireless interface

  • Interframe Space(IFS) = Collision are avoided by deferring transmission even if the channel is found idle. When an idle channel is found, the station does not send immediately. It waits for a period of time called the interframe space.
  • Contention Window – We divide the network into windows and if some signal collide the it has to wait for the next window which is 2^n [n = number of collision]

Minimum amount of data – L ≥ 2 * Tp * B

Efficiency(η) = Tt / (C * 2 * Tp) + Tt + Tp

Back off Algorithm – Wt – K * Tslwt [ k = 0 – 2^n-1 ] [ n = collision number ]

Daily Learning: Computer Networks – Access Control Methods – TDM, Polling, Token Passing, Aloha

Types of Communication Links

  • Point to Point Link
  • Broadcast Link – The connection is shared between all the stations.

Need Of Access Control

In the broadcast link if all stations are sending data simultaneously then there will be collision that’s why we implement the Access Control.

Types Of Access Control Method

1. TDM(Time Division Multiplexing) –

Divide the time into slots and assign each slot to one station.

Efficiency(η) = 1 / 1 + a [ a = Tp / Tt ]

2. Polling –

When a station wants to transmit the data then only we give the chance to that station to transmit the data.

Efficiency(η) = Tt / Tpoll + Tt + Tp [ Tt = Time taken for transmission, Tp = Time taken for propagation]

3. Token passing-

Token – A token is a small message composed of a special bit pattern.

Ring Latency – It is time taken by a bit to cover the entire ring and come back to the same point.

RL = d / v + N * b

[ d = length of the ring, v = velocity of data in ring, N = no. of stations in ring, b = time taken by each station to hold the bit before transmitting it (bit delay)]

Cycle Time – The time taken by the token to complete one revolution of the ring is known as cycle time.

CL – d / v + N * (THT)

[ d = length of the ring, v = velocity of data in ring, N = no. of stations in ring, THT = Token Holding Time ]

Strategies –

Delayed Token Reinsertion (DTR) –

Station keeps holding the token until the last bit of the data packet transmitted by it takes the complete revolution of the ring and comes back to it.

Working –

After a station acquires the token,

  • It transmits its data packet.
  • It holds the token until the data packet reaches back to it.
  • After data packet reaches to it, it discards its data packet as its journey is completed.
  • It releases the token.

Token Holding Time (THT) = Transmission delay + Ring Latency = Tt + Tp [ Tt = Transmission time, Tp = Propagation time ]

Ring Latency = Tp + N x bit delay = 0 [ bit delay = 0 ]

Early Token Reinsertion (ETR) –

Station releases the token immediately after putting its data packet to be transmitted on the ring.

Token Holding Time (THT) = Transmission delay of data packet = Tt

4. Aloha

Rules –

  1. Any station can transmit data to a channel at any time.
  2. No carrier sensing.
  3. There is no collision detection.
  4. It re-transmit the data after some time.(If acknowledgement don’t come)

There are mainly two type of aloha –

  • Pure Aloha –
  1. The total vulnerable time = 2 * Tfr [ Tfr = Average time required to send a packet ]
  2. Maximum throughput occurs when G = 1/ 2 that is 18.4%.
  3. Successful transmission of data frame is S = G * e ^ – 2 G.
  • Slotted Aloha –

We divide the process into slots and a host can only send packets at the beginning of any slot. If it comes after then it has to wait till next slot.

  1. Maximum throughput occurs in the slotted Aloha when G = 1 that is 37%.
  2. The probability of successfully transmitting the data frame in the slotted Aloha is S = G * e ^ – 2 G.
  3. The total vulnerable time required in slotted Aloha is Tfr.

5.

Golang Interface Simplified

What is Interface?

Interface is used for abstraction. It contains one or more method signatures. Below is an example of how we define interface.

type Human interface {
	speak()
}

Why we use interface?

In simple term interfaces are the contract for the methods for different structure type. To increase the code readability and maintenance we use interface. Let’s say there is Person datatype in my application and all the methods mentioned above actually implement the Person data type.

type Person struct {
	First string
	Last  string
}

Now let’s say the method mentioned in the interface actually implement the Person struct

func (p Person) speak() {
	fmt.Println("I am Person ", p.First)
}

Now the interesting part our software got a new requirement of adding another data type called SecretAgent.

type SecretAgent struct {
	Person Person
	Org    string
}

Now we define another method speak() for the SecretAgent data type.

func (s SecretAgent) speak() {
	fmt.Println("I am secret agent ", s.Person.First)
}

Now we can take the help of interface and the power of abstraction. We define a function that will take the interface and call the speak method.

func Earthling(h Human) {
	fmt.Println("Hey there I am from planet Earth")
	h.speak()
}

Understand what happened above? The Indian function take the human interface and call the speak method and we don’t have to specify for which data type the speak is going to work it will be managed by the go interfaces. So, it reduced a lot of hard coding and our design is future ready to accept more data type.

Let’s see the main function.

func main() {
	sa1 := SecretAgent{
		Person: Person{First: "James", Last: "Bond"},
		Org:    "MI6",
	}
	sa2 := SecretAgent{
		Person: Person{First: "Ajit", Last: "Doval"},
		Org:    "RAW",
	}
	p1 := Person{First: "Dr.", Last: "Strange"}

	Earthling(sa1)
	Earthling(sa2)
	Earthling(p1)
}

Daily Learning: Computer Networks – Sliding Window Protocol

Go Back N

If there is a packet lost in the receiver side then it is going to discard the subsequent packets and re transmit the the entire window means it is going back N values and re transmit the the packets

  • Sender window size is go back N is N
  • Receiver Window size is always 1.
  • Acknowledgement –
    • Cumulative – One acknowledgement is used for many packets.
      • Adv – Traffic is low
      • Dis Adv – Reliability low
    • Independent – One acknowledgement is used for one packet.
      • Adv – Reliability is high
      • Dis Adv – Traffic is high

Relationship Between Window Sizes and Sequence Numbers

Minimum sequence numbers required in GBN = N + 1

Bits Required in GBN = log2 (N + 1)

If the sequence number is N then Sender window size is N – 1. Receiver window size is 1.

If the bit is K then sequence number 2^k, Sender Window size is 2^k – 1. Receiver window size is 1.

Formula

  • Maximum window size = 1 + 2 * a [a=Tp/Tt]
  • Minimum sequence numbers required = 1 + 2 * a
  • Number of Bits in Sequence Number Field = log2(1+2a)

Selective Repeat

Selective Repeat

This protocol(SRP) is mostly identical to GBN protocol, except that buffers are used and the receiver, and the sender, each maintains a window of size. Here we only re transmit the lost packet not the entire window again.

  • Sender window size is greater than 1 (Ws > 1)
  • Receiver window size = sender window size (Wr == Ws)
  • Acknowledgements are independent. If bits in the packets are corrupted then SR will send a negative acknowledgement.

Note – If N is the size of sequence number field in the header in bits, then we can have 2N sequence numbers.

Window Size = min(1+2*a, 2^N)

Stop & WaitGBNSR
Efficiency1/1+2aN/1+2aN/1+2a
Buffer1+1N+1N+N
Segment Number1+1N+1N+N
Retransmission1N1
Bandwidthlowhighmedium
CPUlowmediumhigh
Implementationsimplemediumcomplex

How to get environment variable value with the help of struct tags?

Struct Tags

Go struct tags are annotations that appear after the type in a Go struct declaration. Each tag is composed of short strings associated with some corresponding value.

A struct tag looks like this, with the tag offset with backtick “ characters:

type User struct {
	Name string `example:"name"`
}

We can write other go code to examine the tags and do some cool stuff with it. The tags don’t change the main go struct behavior.

How to get environment variable value by the struct tags?

The classic example of populating the struct fields by the environment variable is by doing os.Getenv() again and again and manually populating the values. But in this case the work is to much repeat and error prone if the struct is too large.

type EmailConfig struct {
	Email string `env:"EMAIL"`
}

func main() {
	cfg := EmailConfig{
		Email: os.Getenv("EMAIL")
	}
}

But can we improve it? Yes we can by using the reflection

import (
	"fmt"
	"os"
	"reflect"
)

type Config struct {
	Email    string `env:"EMAIL"`
}

const tagName = "env"

func LoadConfig(q *Config) {
	v := reflect.ValueOf(q).Elem()
	if v.Kind() == reflect.Struct {
		val := reflect.TypeOf(q).Elem()
		for i := 0; i < val.NumField(); i++ {
			field := val.Field(i)
			tag := field.Tag.Get(tagName)
			fmt.Printf("field : %v | tagName : %v\n", field.Name, tag)
			envVal := os.Getenv(tag)
			reflect.ValueOf(q).Elem().FieldByName(field.Name).Set(reflect.ValueOf(envVal))
		}
	}
}

func main() {
	var cfg Config
	LoadConfig(&cfg)
	fmt.Println(cfg)
}
  • Here we have initiated a empty cfg structs and passed the reference in the LoadConfig function.
  • In the LoadConfig function we have iterated over the struct field.
  • we extract the tag fields and set the field values with the environment variable value extracted by the tag name.