Create a key Value storage using Golang – Part 1

A few days back I gave a interview in a company for Golang developer intern. There they asked me about a lot of questions and some I cracked and some I couldn’t cracked. The things I failed miserably was Go interfaces and Go Concurrency(Channels & atomic package). I took it as a motivation and learnt the Go interface again and wrote a blog on that. Now It was time for me to develop a project so my knowledge become more concrete. Nabarun gave a idea to make a key-value storage using Go that will support multiple storage types.

So I started building the project. First create a directory api/models from the root of your directory. Then create a file called records.go and write the following code –

type InMemory struct {
	Data map[string]string `json:"data"`
}

This code will be responsible for in memory storage of the key – value storage. Now lets define some interface that will be help us when we will be enhancing our app to support more storage systems like file structure. Add the following code in records.go –

type StorageSetter interface {
	Store(string, string)
}

type StorageGetter interface {
	List() map[string]string
	Get(string) (string, error)
}

type StorageDestroyer interface {
	Delete(string) error
}

type Storage interface {
	StorageSetter
	StorageGetter
	StorageDestroyer
}

Now let’s see why I have written multiple interface like setter and Getter? Because it is always best practice to keep the interface small. It actually helps increasing abstraction.

Now let’s define a helper function in records.go that will return the InMemory structs.

// Return In Memory struct
func NewCache() *InMemory {
	return &InMemory{Data: make(map[string]string, 2)}
}

Let’s now create the methods in records.go which will do operations on the struct –

func (r *InMemory) Store(key, val string) {
	r.Data[key] = val
}

func (r *InMemory) List() map[string]string {
	return r.Data
}

func (r *InMemory) Get(key string) (string, error) {
	val, ok := r.Data[key]
	if !ok {
		return "", errors.New("key not found")
	}
	return val, nil
}

func (r *InMemory) Delete(key string) error {
	_, ok := r.Data[key]
	if !ok {
		return errors.New("key not found")
	}
	delete(r.Data, key)
	return nil
}

Now let’s define our server so create a directory api/controllers and create base.go file inside it. And write the code inside it.

type Server struct {
	Router *http.ServeMux
	Cache  models.Storage
}

This Server struct will contain dependency of the server which is typically the router and the storage interface.

Now create a server.go inside the api directory and write the code –

package api

import (
	"flag"

	"github.com/aniruddha2000/goEtcd/api/controllers"
)

var server controllers.Server

// Initialize and run the server
func Run() {
	var storageType string

	flag.StringVar(&storageType, "storage-type", "in-memory",
		"Define the storage type that will be used in the server. By defaut the value is in-memory.")
	flag.Parse()

	server.Initialize(storageType)
	server.Run("8888")
}

Here you can see it is taking the flag from the command line and passing it in the Initialize method and calling Run method to run the server in the port 8888. Now let’s define these two Initialize & Run method in the base.go file –

func (s *Server) Initialize(storageType string) {
	s.Router = http.NewServeMux()

	switch storageType {
	case "in-memory":
		s.Cache = models.NewCache()
	case "disk":
		s.Cache = models.NewDisk()
	default:
		log.Fatal("Use flags `in-memory` or `disk`")
	}

	log.Printf("Starting server with %v storage", storageType)

	s.initializeRoutes()
}

// Run the server on desired port and logs the status
func (s *Server) Run(addr string) {
	cert, err := tls.LoadX509KeyPair("localhost.crt", "localhost.key")
	if err != nil {
		log.Fatalf("Couldn't load the certificate: %v", cert)
	}

	server := &http.Server{
		Addr:    ":" + addr,
		Handler: s.Router,
		TLSConfig: &tls.Config{
			Certificates: []tls.Certificate{cert},
		},
	}

	fmt.Println("Listenning to port", addr)
	log.Fatal(server.ListenAndServeTLS("", ""))
}

Here you can see Initialize method is setting the router for the server and then setting the storage for different storage and at the last it is initializing the routes.

In the Run method it is loading the certificate and setting up the server and running the server at the end.

Now let’s define initializeRoutes function that we saw in the last initialize method in the base.go. Create a routes.go file in side api/controllers –

func (s *Server) initializeRoutes() {
	s.Router.HandleFunc("/record", s.Create)
	s.Router.HandleFunc("/records", s.List)
	s.Router.HandleFunc("/get/record", s.Get)
	s.Router.HandleFunc("/del/record", s.Delete)
}

Now we will see the implementation of the route controllers. Create a cache.go file inside the api/controllers and paste the below code –

package controllers

import (
	"log"
	"net/http"

	j "github.com/aniruddha2000/goEtcd/api/json"
)

func (s *Server) Create(w http.ResponseWriter, r *http.Request) {
	if r.Method == "POST" {
		r.ParseForm()
		key := r.Form["key"]
		val := r.Form["val"]

		for i := 0; i < len(key); i++ {
			s.Cache.Store(key[i], val[i])
		}

		j.JSON(w, r, http.StatusCreated, "Record created")
	} else {
		j.JSON(w, r, http.StatusBadRequest, "POST Request accepted")
	}
}

func (s *Server) List(w http.ResponseWriter, r *http.Request) {
	if r.Method == "GET" {
		records := s.Cache.List()
		j.JSON(w, r, http.StatusOK, records)
	} else {
		j.JSON(w, r, http.StatusBadRequest, "GET Request accepted")
	}
}

func (s *Server) Get(w http.ResponseWriter, r *http.Request) {
	if r.Method == "GET" {
		keys, ok := r.URL.Query()["key"]
		if !ok || len(keys[0]) < 1 {
			log.Println("Url Param 'key' is missing")
			return
		}
		key := keys[0]

		val, err := s.Cache.Get(key)
		if err != nil {
			j.JSON(w, r, http.StatusNotFound, err.Error())
			return
		}
		j.JSON(w, r, http.StatusOK, map[string]string{key: val})
	} else {
		j.JSON(w, r, http.StatusBadRequest, "POST Request accepted")
	}
}

func (s *Server) Delete(w http.ResponseWriter, r *http.Request) {
	if r.Method == "DELETE" {
		keys, ok := r.URL.Query()["key"]
		if !ok || len(keys[0]) < 1 {
			log.Println("Url Param 'key' is missing")
			return
		}
		key := keys[0]

		err := s.Cache.Delete(key)
		if err != nil {
			j.JSON(w, r, http.StatusNotFound, err.Error())
			return
		}
		j.JSON(w, r, http.StatusNoContent, map[string]string{"data": "delete"})
	} else {
		j.JSON(w, r, http.StatusBadRequest, "DELETE Request accepted")
	}
}

Here you can see the controller that will handle different route traffics and call the record methods and doing the operations.

I have creates a helper JSON method to reduce redundant code while writing the route controllers. create a api/json directory and crate a json.go file. Paste the code below –

func JSON(w http.ResponseWriter, r *http.Request, statusCode int, data interface{}) {
	w.Header().Set("Location", fmt.Sprintf("%s%s", r.Host, r.RequestURI))
	w.WriteHeader(statusCode)
	json.NewEncoder(w).Encode(data)
}

In the next part I will walk you though how to extend this application to support disk based storage along side with In-Memory storage.