Daily Learning: Computer Networks – ARP & DHCP

ARP (Address Resolution Protocol)

How a computer gets another computer’s MAC address in a LAN?

If there is two computer in a LAN suppose Computer A and Computer B and A wants to sends some data to the computer B over the LAN then it needs computer B’s MAC address to send the data. Let’s see how it works –

  • Computer A sends ARP(Address Resolution Protocol) request to all the machines in a LAN with the IP address of computer B.
  • Computer B respond on the ARP and sends back the MAC address.

DHCP (Dynamic Host Control Protocol)

Ever imagined when you connect to any wifi and it gives you a unique IP address automatically, How?

It is the protocol which gives IP addresses to host that joins a network and also manage the given IP addresses to the hosts.

There are four part of any DHCP system

  1. DHCP Discover – When a new machine joins a LAN network it sends a broadcast message to to network to know the addresses of the DHCP server.
  2. DHCP offer – After it gets a DHCP discover request from a host it sends a DHCP offer to the host with one IP address from it’s pull.
  3. DHCP request – If the host accept the DHCP offer then it sends back a DHCP request back to the server.
  4. DHCP Acknowledgement – Once the DHCP server gets the DHCP request it gives back a acknowledgement to the host and store the information.

E.G. – A typical example is our home router.

DHCP Relay – So big techs wants a specific DHCP server rather than the router in that case the router works as a DHCP relay and forwards the DHCP requests to the DHCP server knowing the destination IP address.

Daily Learning: Computer Networks – TCP/IP Model

TCP/IP model stands for Transmission Control Protocol and Internet Protocol.

The TCP/IP model consists of five layers: the application layer, transport layer, network layer, data link layer and physical layer.

The first four layers provide physical standards, network interface, internetworking, and transport functions that correspond to the first four layers of the OSI model and these four layers are represented in TCP/IP model by a single layer called the application layer.

TCP/IP is a hierarchical protocol made up of interactive modules, and each of them provides specific functionality.

Below there is a table with all layers given with their individual task.

Layer 4ApplicationAllow access to network resources
Layer 3TransportProvides process to process message delivery and error handling
Layer 2InternetMove packets from source to destination
Layer 1Link/Network InterfaceResponsible for transmission of data between two devices in a same network.

Here below a visual difference between OSI and TCP/IP protocol.

Note : The higher you go up in the model the headers are stripped away until you have the main data left for your application.

Internet Layer

This layer is consist of different protocol like TCP or UDP. It has a thing called headers and the headers has certain properties as well –

  • Version – It describes the IP address type IPV4 or IPV6
  • Header Length – It specifies the size of the header. Usually it is 32 bit.
  • Differentiated Services Code Point (DSCP) – It contains data of bandwidth sensitive application like VOIP.
  • Time To Live (TTL) – Some time error happens and a Ip packets travels through the routers infinitely and to prevent we use this.
    • Protocol – It specifies the protocol type. E.G. – TCP or UDP.

Intro to GraphQL & How it differs from REST APIs

What is GraphQL?

It is a query language for the API in simple term. But what actually means and how it helps is a whole bunch of talk.

GraphQL is a syntax that describes how to ask for data, and is generally used to load data from a server to a client. GraphQL has three main characteristics:

  • It lets the client specify exactly what data it needs.
  • It makes it easier to aggregate data from multiple sources.
  • It uses a type system to describe data.

With GraphQL a user an make a single request and take all the data from the back-end rather than making a lot of request and it drastically reduce the server requests and improve performance. Below a sample query and the response.

# Query
{
  hero {
    name
  }
}

# Response
{
  "data": {
    "hero": {
      "name": "R2-D2"
    }
  }
}

Difference between GraphQL and REST API

For the past few decades we are using REST APIs and it was completely fine. But let’s see how it works.

In REST API there are specific endpoints for every resource that it needs and a client sometime have to make several requests to get a the actual data and also the data is sometime overloaded and sometime under loaded.

And the data is also store in different tables so you have to make different queries for different tables.

E.G. –

  1. The YouTube page there are whole bunch of videos that comes up and in a REST architecture the YouTube make a request to the server to get all the IDs.
  2. And then it again request back to get all the information about all the ID.
  3. So for a million user the REST architecture is sending 2 request for this simple task.

But in a GraphQL architecture the data will comeback in a single request and there will also be specific.

The GraphQL has only one endpoint “/graphql”.

Daily Learning: Computer Networks – OSI Model

OSI stands for Open System Interconnection is a reference model that describes how information from a software application in one computer moves through a physical medium to the software application in another computer.

OSI consists of seven layers, and each layer performs a particular network function.

OSI model divides the whole task into seven smaller and manageable tasks. Each layer is assigned a particular task.

Each layer is self-contained, so that task assigned to each layer can be performed independently.

Below there is a table with all layers given with their individual task.

Layer 7ApplicationNetwork services provided to end-user applications such as a web browser.
Layer 6PresentationThe presentation layer formats or translates data for the application layer based on the syntax or semantics that the application accepts.
Layer 5SessionThe session layer controls synchronization of communication between different computers.
Layer 4TransportBreaks the data in chunk and Transfer the data between systems and hosts.
Layer 3NetworkDelivering frames to their intended destinations among based on the addresses contained inside the frame.
Layer 2Data LinkDirectly connected nodes are used to perform node-to-node data transfer where data is packaged into frames. Also corrects Error.
Layer 1PhysicalTransmitting electrically or optically raw data from sender to receiver

Below I am going to give some idea about important terminologies regarding OSI model –

  • MAC Address (Media Access Control) – It is built into all the network devices.
    • It is a 48 bit address represented in hexadecimal form.
    • Each hex character contains 4 bit.
    • There are 12 hex characters
    • E.G. – 00–40–05–60–7D–49
    • The first six digit represents the NIC manufactures. Last six digit refers to the unique ID of each NIC.
  • Frames – The data transfer in network happens in frames.
    • Different network have different frame size.
    • All NIC in a same network must have same frame size.
    • Generally the frame size is 1500 bytes

Data Transmission in Hub – When a specific NIC sends the data to a specific host in a network it just sends the data to the hub and the hub sends the data to all the connected host and the host with the matching MAC address will receive and others will discard it.

Data Transmission in Switch – Later we use switch which filter the MAC address and sends the data to only the destination address.

FCS – It is 4 byte long.

  • Data Link Layer parts –
    • LLC(Logical Link Control) – NIC that talks to the system’s operating system (usually via device drivers). The LLC handles multiple network protocols and provides flow control.
    • MAC Address – which is discussed above.
  • Network Layer –

Here it gives a logical address called IP address. It is a 32 bit address and represented as dotted decimal. There is a device called router which sends data though IP address.

Daily Learning: Computer Networks – Network Topology

In real life there are many ways the hosts in a network is connected. Upon which the efficiency and other parameters depends and discussed topologies are been developed through the decades.

Bus & Ring Topology

  • A bus topology use a connection line that connects all computers.
  • Ring topology connects all the computer on a network with a ring of cable.
    In ring topology the data gets transferred in uni direction (clockwise or anti clockwise )

Problems – Both suffered from same problem with cable break down.

Star Topology

Here all machines are connected to a central node.

Pros – It is robust and if one connection goes down then entire network don’t get affected.

Cons – If the central node goes down then the entire network goes down.

Mesh Topology

In wireless network every machine is connected to every network then it is a mesh topology. There are mainly two type of topology is there-

  • Fully Mesh – All machines are connected to all other machines.
  • Total number of Links – n * (n – 1) / 2
  • Partial Mesh – At least two machines are not connected.

Hybrid Topology

Here in the central box all the machines are connected to a internal bus. It is often known as star-bus topology.

All the hosts are connected to a central node but inside the box every connection is actually bus topology.

Daily Learning: Computer Networks – Internet & Data Communication

Internet

It is a distributed vast network that connect all type of devices together.

Data Communication

There are total 5 fundamental parts of any data communication –

  • Message – For a data communication one of the most important thing is message or the data.
  • Sender – The second thing is sender there must be a person or application which is sending the data.
  • Receiver – The third part obviously is the receiver there also must be a receiver in order to receive the data.
  • Transmission Medium – It is the way by which the data is going to be transferred. It can be many types for example – Wire, wireless etc.
  • Protocol – It is similar to human conversation some device send some message/request to another machine and it sends back reply as another message/request. In general protocol is a set of rule upon which devices communicate with each other.

Connecting To Host In A Same Network(LAN) Via SSH

SSH(Secure Shell) is a cryptographic network protocol that allow you to operate network services securely over an unsecured network. There are multiple client in various OS to run ssh. For example, In Linux OpenSSH and in Windows PuTTY is there.
Now suppose you have a home router and there are multiple network connected to it and certainly you want to access a computer from another computer. Let’s take Comp1 wants to connect to Comp2. So, Now you have to create a SSH connection between the computers.

Setting up the SSH Server

So for connection you have to install and start a SSH server in the Comp2 . Let’s see the commands for installing SSH server.

$ sudo apt install openssh-server

Now you have to check SSH service is running or not if not you have to start it.

$ sudo systemctl status ssh

If the status is not showing active in your case the you have to execute the below command

$ sudo systemctl enable ssh

Get the IP of the Server

We connect to a server by $ ssh user@ip_address so you have figure out the ip address of the Comp2 . And for that you can type ifconfig . Your ip will be something like 192.168.x.y

Enabling SSH traffic on your firewall settings

If you are using UFW as a default firewall on your Ubuntu host, it is likely that you need to allow SSH connections on your host.

To enable SSH connections on your host, run the following command –

$ sudo ufw allow ssh

Then check for firewall has enabled or not –

$ sudo ufw status | grep 22

It should give something like this –

Note : SSH use port 22 by default.

Connecting via SSh

Now go to the Comp2 and type the ssh command followed by the user@ip_address. In my case bellow –

$ ssh aniruddha@192.168.1.5

Then it will prompt for the password and type the password for that user in the computer and here you go your Comp2 terminal will appear in the Comp1 terminal.

Encrypt your files by Ansible Vault

So, in previous blog I have given you a brief description about ansible roles. In this blog I am going to discuss about another very important topic that is ansible vault.

Why we need encryption?

We write our ansible playbook, jinja templates and other files in normal unencrypted format but we can’t keep our crucial information such as password in unencrypted format so that’s why we need encryption.

How we can encrypt?

Just like we use ansible-playbook to run our ansible playbooks we have a option called ansible-vault to do our encryption and decryption work. Here below some important and useful commands –

  • we use ansible-vault create a_file to create a encrypted file
$ ansible-vault create a_file
New Vault password: 
Confirm New Vault password:
  • we use ansible-vault encrypt a_existing_file to encrypt a existing file
$ ansible-vault encrypt a_existing_file 
New Vault password: 
Confirm New Vault password: 
Encryption successful
  • we use ansible-vault view a_file to view the encrypted file
$ ansible-vault view a_file 
Vault password: 
You are reading Aniruddha's Blog ;)
  • we use ansible-vault decrypt a_file to decrypt any encrypted file
$ ansible-vault decrypt a_file 
Vault password: 
Decryption successful

What is vault ID?

A vault ID is an identifier for one or more vault secrets. Vault IDs, you must provide an ID of your choosing and a source to obtain it’s password (either prompt or a file path). We use --vault-id to provide the ID. Here below examples how we do –

  • We encrypt a file with ID
$ ansible-vault encrypt --vault-id id1@prompt a_file 
New vault password (id1): 
Confirm vew vault password (id1): 
Encryption successful
  • we view a file with ID
$ ansible-vault view --vault-id id1@prompt a_file 
Vault password (id1): 
You are reading Aniruddha's Blog ;)
  • we decrypt a file with ID
$ ansible-vault decrypt --vault-id id1@prompt a_file 
Vault password (id1): 
Decryption successful

Thank you 🙂

REST API

Representational State Transfer (REST) is an architectural style that defines a set of constraints to be used for creating web services. A client requests some data from the server and the server returns the data(resource). There are 6 Constraints –

  1. Uniform Interface – It is a key constraint that differentiate between a REST API and Non-REST API. It suggests that there should be an uniform way of interacting with a given server irrespective of device or type of application (website, mobile app).
  2. Stateless – The server will not store anything about the latest HTTP request the client made. It will treat every request as new.
  3. Cacheable – Every response should include whether the response is cacheable or not and for how much duration responses can be cached at the client side.
    • Caching – Caching is the ability to store copies of frequently accessed data in several places along the request-response path. When a consumer requests a resource representation, the request goes through a cache or a series of caches (local cache, proxy cache, or reverse proxy) toward the service hosting the resource. If any of the caches along the request path has a fresh copy of the requested representation, it uses that copy to satisfy the request. If none of the caches can satisfy the request, the request travels all the way to the service (or origin server as it is formally known).
  4. Client-Server – A Client is someone who is requesting resources and are not concerned with data storage, which remains internal to each server, and server is someone who holds the resources and are not concerned with the user interface or user state.
  5. Layered system – REST allows you to use a layered system architecture where you deploy the APIs on server A, and store data on server B and authenticate requests in Server C.
  6. Code on demand (optional) – servers can also provide executable code to the client. The examples of code on demand may include the compiled components such as Java applets and client-side scripts such as JavaScript.

REST Resource naming guide

We can divide resource archetypes into four categories.

  • Document – A document resource is a singular concept that is akin to an object instance or database record.
    Use “singular” name to denote document resource archetype.
http://api.example.com/device-management/managed-devices/{device-id}
http://api.example.com/user-management/users/id
http://api.example.com/user-management/users/admin
  • Collection – A collection resource is a server-managed directory of resources. Clients may propose new resources to be added to a collection.
    Use “plural” name to denote collection resource archetype.
http://api.example.com/device-management/managed-devices
http://api.example.com/user-management/users
http://api.example.com/user-management/users/{id}/accounts
  1. Store – A store is a client-managed resource repository. A store resource lets an API client put resources in, get them back out, and decide when to delete them. A store never generates new URIs.
    Use “plural” name to denote store resource archetype.
http://api.example.com/cart-management/users/{id}/carts
http://api.example.com/song-management/users/{id}/playlists
  1. Controller –  A controller resource models a procedural concept. Controller resources are like executable functions, with parameters and return values; inputs and outputs.
    Use “verb” to denote controller archetype.
http://api.example.com/cart-management/users/{id}/cart/checkout
http://api.example.com/song-management/users/{id}/playlist/play

Best practices & rules

There are some best practices and rules which will be very useful. 

  • Use (/) to indicate hierarchical relationships.
http://api.example.com/device-management/managed-devices/{id}/scripts
http://api.example.com/device-management/managed-devices/{id}/scripts/{id}
  • Do not use trailing forward slash (/).
http://api.example.com/device-management/managed-devices/
http://api.example.com/device-management/managed-devices 	/*This is much better version*/
  • Use hyphens (-) instead of underscores (_).
http://api.example.com/inventory-management/managed-entities/{id}/install-script-location  //More readable
http://api.example.com/inventory-management/managedEntities/{id}/installScriptLocation  //Less readable

Not (_)

http://api.example.com/inventory-management/managed-entities/{id}/install-script-location  //More readable
http://api.example.com/inventory_management/managed_entities/{id}/install_script_location  //More error prone
  • Use lower case letters in URI.
http://api.example.org/my-folder/my-doc  //1
HTTP://API.EXAMPLE.ORG/my-folder/my-doc  //2
http://api.example.org/My-Folder/my-doc  //3
  • Don’t use file extensions.
http://api.example.com/device-management/managed-devices.xml  /*Do not use it*/
http://api.example.com/device-management/managed-devices 	/*This is correct URI*/

HATEOAS driven REST API

REST architectural style lets you use hypermedia links in the response contents so that the client can dynamically navigate to the appropriate resource by traversing the hypermedia links. Above is conceptually the same as a web user browsing through web pages by clicking the relevant hyperlinks to achieve a final goal.
For example HTTP GET http://api.domain.com/management/departments/10

{
    "departmentId": 10,
    "departmentName": "Administration",
    "locationId": 1700,
    "managerId": 200,
    "links": [
        {
            "href": "10/employees",
            "rel": "employees",
            "type" : "GET"
        }
    ]
}

In the preceding example, the response returned by the server contains hypermedia links to employee resources 10/employees, which can be traversed by the client to read employees belonging to the department.

Idempotent REST API

When making multiple identical requests has the same effect as making a single request – then that REST API is called idempotent. Except POST everything is idempotent, because POST will create a new resource N times if you invoke it N times.

Best practices to Secure REST API

  • Always use HTTPS.
  • Use password hash.
  • Never expose information on URL.
    Usernames, passwords, session tokens, and API keys should not appear in the URL, as this can be captured in web server logs, which makes them easily exploitable.
https://api.domain.com/user-management/users/{id}/someAction?apiKey=abcd123456789  //Very BAD !!
  • Use OAuth instead of basic auth.
  • Add timestamp in request it will prevent the basic reply attack from people who are trying brute force to your system.

Application state vs Resource state

  • Application state : Application state is server-side data which servers store to identify incoming client requests, their previous interaction details, and current context information.
  • Resource state : Resource state is the current state of a resource on a server at any point of time – and it has nothing to do with the interaction between client and server. It is what you get as a response from the server as API response. You refer to it as resource representation.

Advantages of statelessness

  • Statelessness helps in scaling the APIs to millions of concurrent users by deploying it to multiple servers. Any server can handle any request because there is no session related dependency.
  • No server side synchronization.
  • Easy to cache.
  • The server never loses track of “where” each client is in the application because the client sends all necessary information with each request.

Giving my first talk in the Flatcar Community call

Hey there everyone! From the past couple of month I have been learning more about Operating Systems and Linux. And after that I started contributing to Flatcar linux. And On 10 August I gave my life’s first talk in the flatcar community call.

What is Flatcar Linux?

Flatcar Container Linux is a container optimized OS that ships a minimal OS image, which includes only the tools needed to run containers. The OS is shipped through an immutable filesystem and includes automatic atomic updates.

What is community call?

So every month the community organise a zoom meeting where they discuss about news, status updates of their various feature and also upcoming release plannings. Here you can see all the meetings that happened previously with their slides and also information of the upcoming calls. Also there is a spotlight section where I shared my experience of my first flatcar contribution and my journey that how I started.

What I did in my first contribution?

I actually worked on a project called locksmith. locksmith is a reboot manager for the Flatcar update engine which is able to use etcd to ensure that only a subset of a cluster of machines are rebooting at any given time. Also locksmithd runs as a daemon on Flatcar hosts and is responsible for controlling the reboot behaviour after updates.

I worked on a issue that was based on semaphore. The issue was mainly If no semaphore was acquired before then the locksmithd will reboot outside of the reboot window after an update. The expected behaviour was –

  • Try to get the semaphore in infinite loop.
  • If it fails but still in the reboot window then sleep for a certain interval.
  • If it fail but not in reboot window then sleep until the next window.
  • If it succeed then reboot.

How was my experience?

As it was my first talk I was a bit nervous about it. But I finally gathered the courage and gave the talk. After the talk I watched it later and found that the introduction part could have been improved a little bit.

Resources

Here is the YouTube recording.
Here is the slides that I presented.