Send emails through ansible

Recently I was watching this video and got to know about `mail` attribute in ansible and we can send emails through it and I gave it a try and finally after some hacks I sent an email from my one account to another.

Here below code I have written to send the email –

---
- hosts: webservers
  tasks:
  - name: Sending an e-mail using Gmail SMTP servers
    mail:
      host: smtp.gmail.com
      port: 587
      username: your@email.com
      password: yourPassword
      to: Aniruddha Basak <some@email.com>
      subject: Ansible check email
      body: I am using {{ ansible_facts['os_family'] }}.
    delegate_to: localhost

Above we are using gmail SMTP server. In case you are using 2FA of google then you have to add a device password for that and write the password in the password field.

Understanding various git commands

If you are working on a project then you have to maintain it properly and that’s why we use git as our VCS(version control system) and github or gitlab as our cloud service where we store all our code remotely. Below there will be some basic git command and their description.

Photo by Yancy Min on Unsplash

Initializing git repository

When you first start a project using git you have to initialize the git in side your project directory.

$ cd my_folder
$ git init
Initialized empty git repository in my_folder/.git/

Cloning repository

Suppose if you want to work on a project that is already on github so in order to contribute to that project first you have to clone that repo to your local machine. Here git clone comes to help you in that.

$ git clone ssh://git@github.com/[username]/[repository-name].git

Status of a repository

If you want to check the status of a repository like which files have added and deleted and modified then you have to execute the below commands. It will also show the tracked and untracked files of the repository.

$ git status
On branch master
Changes not staged for commit:
(use "git add <file>..." to update what will be committed)
(use "git checkout –– <file>..." to discard changes in working directory)
modified: tasks.txt
no changes added to commit (use "git add" and/or "git commit –a")

Adding files to staging area

If you add a new file to your repository or you have modified some files then before committing you have to move those files to staging area. Here you have certain cases like you want to add files to staging area by one by on or you want to add all files to staging or you want to add all tracked files to staging area except untracked files. Below commands for all cases.

$ #Add specific file to staging area
$ git add [file_name]
$
$ #Add all new and changed file to staging area
$ git add -A
$
$ #Add all tracked files to staging area
$ git add -u

Committing changes

After adding files to staging area you have to commit those changes. Basically you are saving those changes as commits. Every commits has it’s commit message and unique commit ID.

$ git commit -m [commit message]

Removing files or folders

Suppose you have to delete some files from index which is not needed then you have to tell git to remove those and here git rm comes to help you.

$ git rm -r [file-name]

Branching

It is recommended that you should not change directly in the master branch. When you are experimenting something or adding any feature then always you should create a new branch and make changes there. So bellow command is showing you how to create a new branch.

$ git checkout -b <your_branch>
Switched to a new branch "your_branch"

Now type git branch to see all local branches. And if you want to see all the branches include local and remote then you have to type git branch -a. Now if you again want to come back to your master branch then you have to type below command.

$ git checkout master
Switched to branch 'master'

If you want to switch to the last checkout branch then type git checkout -. Now sometimes you have to delete a specific branch so in this case you have two branches one is in your local machine and one is in your server.

$ #delete your local branch
$ git branch -d <your_local_branch>
Deleted branch <your_local_branch> (was 4335a0a).
$
$ #delete your remote branch
$ git push origin --delete <your_remote_branch>

Merging

When you have finished your all features and changes in your branch it is now ok to merge with master branch. In order to merge any branch with master branch there we execute the below command.

$ git checkout master
Switched to branch 'master'
$ git merge <your_branch>

`Basically in merging there is two branches one is source branch and another is target branch and in the above scenario the master branch is target branch and other is source branch. If you want to do the above operation in one line then you can do –

git merge [target branch]

Stashing

Use git stash when you want to record the current state of the working directory and the index, but want to go back to a clean working directory. The command saves your local modifications away and reverts the working directory to match the HEAD commit.
And use git stash clear to remove all stashed entries.

Sharing & updating project

After all committing stuff it is the time to push the changed to the remote server or otherwise if your local computer get disturbed you will loose all changes. Use the below command –

$ git push origin <your_branch>

Sometimes you are just working on a specific branch for a long time and it is possible that git will remember the branch name for you. Use git push -u origin <your_branch> to make git remember your branch name. After this you just have to execute the command
git push and it will automatically push the commits in the remembered branch.

Now suppose you have two laptops and from both you are maintaining the project so it is very necessary for you that you keep updated the project in the both laptops. So git pull will update the local repository to the latest commits.

So git has local repository and remote repository so when ever we push something it will go to the remote repository. So it is very important to tell git that what is our remote repository. In order to add the remote repository we execute the bellow commands –

$ git remote add origin ssh://git@github.com/[username]/[repository-name].git

Sometimes we clone the repository in the https format url or we have set the remote url in https format but we are planning to switch to ssh. Then we will execute the below command and everything is changed . Below we are switching to ssh remote url.

$ git remote set-url origin ssh://git@github.com/[username]/[repository-name].git

Inspection & Comparison

If you wand to see commit history then I already published a blog about git log you should checkout. Now if you want to see what you have changed after the last commit then do
git diff. It is good to see the changes you have made in the feature branch before merging to the target branch. Execute the below command for that –

$ git diff [source branch] [target branch]

Filter your commits through git log

The purpose of any version control system is to record changes to your code. This gives you the power to go back into your project history and see who contributed what and when and figure out where bugs were introduced. But, having all of this history available is useless if you don’t know how to navigate it. That’s where the git log command comes in. Below various git log options that will be very helpful while maintaining a big project:

Photo by Brina Blum on Unsplash

Oneline

The --oneline option will give you a output of commits to a single line. By default it will show you the commit ID and first line of the commit message.

aniruddha ~/Desktop/foobar master 
$ git log --oneline 
505a2c6 (HEAD -> master) Add the file hellomars.txt and update helloworld.txt
fb8c32d First commit

By amount

The most basic filter that git gives us the limit the number of commits. You can pass -<n> after the git log.

$ git log -3

By author

When you are looking for commits created by a specific person then --author="name" will be the option you have to use. This will return all commits matches with the author name you have given.

$ git log --author="Aniruddha"

You can use regular expressions to search for commits. In the below example the following commands searches for commits by either Aniruddha or Sayan.

$ git log --author="Aniruddha\|Sayan"

By commit message

To filter the commits by their commit message, use the --grep flag. This works like the --author flag discussed in the above but it matches against the commit message instead of the author.

$ git log --grep="First"

By file

Some times you are only interested in changes that happened to a particular file. To see the history related to a file you just have to to pass the file path. Below we want to see the commits related to foo.py.

$ git log -- foo.py

See changed file names

If you want to see the files you have changed according to your git commit then execute the below git command.

$ git log --name-only

Thank you 🙂

Authenticate github using ssh

SSH is a tool by which we can create a secure connection between the client and server. Basically it is an asymmetric type encryption that creates two key one is public and one is private. Public key can be distributed publicly and it is used for encrypt the data and the private key have to be kept secret for decrypting the data. Below commands will create the ssh keys:

$ ssh-keygen -t rsa -C "your_email@example.com"
Generating public/private rsa key pair.
Enter file in which to save the key (/home/ani/.ssh/id_rsa): 
Enter passphrase (empty for no passphrase): 
Enter same passphrase again: 
Your identification has been saved in /home/ani/.ssh/id_rsa.
Your public key has been saved in /home/ani/.ssh/id_rsa.pub.
The key fingerprint is:
SHA256:Up6KjbnEV4Hgfo75YM393QdQsK3Z0aTNBz0DoirrW+c ylo@klar
The key's randomart image is:
+---[RSA 2048]----+
|    .      ..oo..|
|   . . .  . .o.X.|
|    . . o.  ..+ B|
|   .   o.o  .+ ..|
|    ..o.S   o..  |
|   . %o=      .  |
|    @.B...     . |
|   o.=. o. . .  .|
|    .oo  E. . .. |
+----[SHA256]-----+

Now you have created a pair of keys and you have to add the public key in the github. Follow below instructions after doing the above the task –

  1. Go to your github page and open settings.
  2. Now click the SSH and GPG keys.
  3. Now click New SSH key.
  4. In the title field add a relevant title fr your key. Like if you are using Linux on your PC the give a title like “Linux PC”.
  5. Then paste your public key in the box.
  6. Then click Add SSH key.
  7. If it ask for your password then confirm your password.

Thank you 🙂

Approve all your blog comment using javascript magic

I usually check and reply all my blog comments on friday night and approve the spam comments. Last friday I was checking my spam comments and saw that there was almost more than 70 comments and I thought approving all comments will be a hectic process and I was thinking of a process that I can do it with one click. Then I came up with the solution of writing a JavaScript script to approve all comments.

Here below the script –

var x = document.getElementsByClassName('comment__action comment__action-approve');
for (var i = 0; i < x.length; i++) {
    x[i].click();
}

The above code actually select the approve button and gives me an array then x[i].click(); this will click each approve button one by one.

Thank you 🙂

Take user input by ansible prompts

In ansible we generally execute instruction written in playbook one after another. In some cases we have to collect some data from the user that can be a sensitive data or not. In this scenario we use prompts to collect the data from the user. We write all our prompts inside vars_prompt section. By default the data we write as input is private and we can change it by private: no . Below there is a basic example of ansible prompt.

---
- hosts: webservers
  vars_prompt:
    - name: username
      prompt: "What is your username?"
      private: no
    
    - name: password_1
      prompt: "What is your password?"

Here is the result –

$ ansible-playbook prompt.yml 
What is your username?: ani
What is your password?: 

PLAY [webservers] ***************************************************************************************************************************

TASK [Gathering Facts] **********************************************************************************************************************
ok: [localhost]

PLAY RECAP **********************************************************************************************************************************
localhost                  : ok=1    changed=0    unreachable=0    failed=0    skipped=0    rescued=0    ignored=0 

We can even set a default value for any prompt and by default: 'value' and if there is a need of verifying any value like password then we can do this by confirm: yes . Below there is a example –

---
- hosts: webservers
  vars_prompt:
    - name: release_version
      prompt: "What is product release version?"
      private: no
      default: "1.0"
    
    - name: password_2
      prompt: "Enter password_2"
      confirm: yes

Here is the result of the above playbook –

$ ansible-playbook prompt.yml 
What is product release version? [1.0]: 2.0
Enter password_2: 
confirm Enter password_2: 
***** VALUES ENTERED DO NOT MATCH ****
Enter password_2: 
confirm Enter password_2: 

PLAY [webservers] ***************************************************************************************************************************

TASK [Gathering Facts] **********************************************************************************************************************
ok: [localhost]

PLAY RECAP **********************************************************************************************************************************
localhost                  : ok=1    changed=0    unreachable=0    failed=0    skipped=0    rescued=0    ignored=0

In the above you can see that the default value in the square bracket what we have mention in the playbook and if we skip that part then ansible will take the default value. You can see that we have entered the confirmed password wrong so it is showing a warning like this **** VALUES ENTERED DO NOT MATCH **** and second time we have entered the right value that’s why everything is ok.

Thank you 🙂

Unit testing in rust

What is test?

Testing is a process by which we assure that the code we have written for our software is working in expected manner. Basically there are two types of test. One is unit test that we going to cover in this blog and another is integration test.

What is unit testing?

It is a testing process where we test the smallest possible part of a software. It usually has one or a few inputs and usually a single output. In procedural programming a unit refers to a individual program, function, procedure etc. For a object oriented programming a unit refers to method belong to super class, abstract class or child class.

How to do it?

Rust looks for all unit tests in the src/ directory. I would suggest maintain this file hierarchy for unit tests. Inside your src directory create a tests directory and then write all tests inside it and don’t forget to import all test in the mod.rs file.

.
├── Cargo.lock
├── Cargo.toml
├── src
│   ├── main.rs
│   └── tests
│       ├── mod.rs
│       └── unit_test.rs

There are some macros we use while testing:

  • assert!(expression) – panic if false.
  • assert_eq!(left, right) and assert_ne!(left, right) – testing left and right expressions for equality and inequality respectively.

Here below an example of unit test in the rust. Inside your src/main.rs

mod tests;

fn main() {
    println!("Hello, world!");
}
pub fn add(a: i32, b: i32) -> i32 {
    a + b
}
pub fn bad_add(a: i32, b: i32) -> i32 {
    a - b
}

Unit test go into the test module with the #[cfg(test)]. Inside your src/tests/unit_test.rs

#[cfg(test)]
mod tests {
    use crate::*;

    #[test]
    fn test_add() {
        assert_eq!(add(1, 2), 3);
    }

    #[test]
    fn test_bad_add() {
        assert_eq!(bad_add(1, 2), 3);
    }
}

Tests can be run with cargo test.

$ cargo test
   Compiling unit_test v0.1.0 (/home/aniruddha/Desktop/Rust lang/unit_test)
    Finished dev [unoptimized + debuginfo] target(s) in 0.30s
     Running target/debug/deps/unit_test-4578ae065cfb1e10

running 2 tests
test tests::unit_test::tests::test_add ... ok
test tests::unit_test::tests::test_bad_add ... FAILED

failures:

---- tests::unit_test::tests::test_bad_add stdout ----
thread 'tests::unit_test::tests::test_bad_add' panicked at 'assertion failed: `(left == right)`
  left: `-1`,
 right: `3`', src/tests/unit_test.rs:12:9
note: Run with `RUST_BACKTRACE=1` environment variable to display a backtrace.


failures:
    tests::unit_test::tests::test_bad_add

test result: FAILED. 1 passed; 1 failed; 0 ignored; 0 measured; 0 filtered out

You can ignore test also. Here below the example –

#[cfg(test)]
mod tests {
    use crate::*;

    #[test]
    #[ignore]
    fn ignored_test() {
        assert_eq!(add(0, 0), 0);
    }
}

Here is the result –

$ cargo test
   Compiling unit_test v0.1.0 (/home/aniruddha/Desktop/Rust lang/unit_test)
    Finished dev [unoptimized + debuginfo] target(s) in 0.35s
     Running target/debug/deps/unit_test-4578ae065cfb1e10

running 1 test
test tests::unit_test::tests::ignored_test ... ignored

test result: ok. 0 passed; 0 failed; 1 ignored; 0 measured; 0 filtered out

Thank you 🙂

Encrypt your files by Ansible Vault

So, in previous blog I have given you a brief description about ansible roles. In this blog I am going to discuss about another very important topic that is ansible vault.

Why we need encryption?

We write our ansible playbook, jinja templates and other files in normal unencrypted format but we can’t keep our crucial information such as password in unencrypted format so that’s why we need encryption.

How we can encrypt?

Just like we use ansible-playbook to run our ansible playbooks we have a option called ansible-vault to do our encryption and decryption work. Here below some important and useful commands –

  • we use ansible-vault create a_file to create a encrypted file
$ ansible-vault create a_file
New Vault password: 
Confirm New Vault password:
  • we use ansible-vault encrypt a_existing_file to encrypt a existing file
$ ansible-vault encrypt a_existing_file 
New Vault password: 
Confirm New Vault password: 
Encryption successful
  • we use ansible-vault view a_file to view the encrypted file
$ ansible-vault view a_file 
Vault password: 
You are reading Aniruddha's Blog ;)
  • we use ansible-vault decrypt a_file to decrypt any encrypted file
$ ansible-vault decrypt a_file 
Vault password: 
Decryption successful

What is vault ID?

A vault ID is an identifier for one or more vault secrets. Vault IDs, you must provide an ID of your choosing and a source to obtain it’s password (either prompt or a file path). We use --vault-id to provide the ID. Here below examples how we do –

  • We encrypt a file with ID
$ ansible-vault encrypt --vault-id id1@prompt a_file 
New vault password (id1): 
Confirm vew vault password (id1): 
Encryption successful
  • we view a file with ID
$ ansible-vault view --vault-id id1@prompt a_file 
Vault password (id1): 
You are reading Aniruddha's Blog ;)
  • we decrypt a file with ID
$ ansible-vault decrypt --vault-id id1@prompt a_file 
Vault password (id1): 
Decryption successful

Thank you 🙂

Customize your vim like any other IDE

So, almost 2 years ago I heard about C programming language from my close friend and I thought I should give it a try and I asked “how can I start with it?” and he gave me some notes and said just install turboC++ and start writing code. So, I installed it and started with C language and eventually I got demotivated and quit programming after some time. Actually it was not fun and engaging with that environment. After one year in my college their was a seminar of network security and the guest speaker said that everyone should try python once and it is really amazing language. And I went home and searched for the python in the YouTube and found lots of tutorial on it and picked up one and that tutorial is using VSCode after that I loved that text editor and still I use it in my daily work.

In my recent days I got to know about a new text editor called vim and I recently installed and start using it. The most funny thing is when I first open a file with it I was not able to exit from it :p. Now I use it in most of the time while writing code. When you will first install it then it will look something below like this

The vim allow us to customize it however we want and we customize it by writing some code in the ~/.vimrc and if the file is not there you can add it by doing touch ~/.vimrc command. Now add below line to your file first.

syntax enable
set tabstop=4
set shiftwidth=4
set expandtab
set number
filetype indent on
set autoindent
set ruler
set showcmd
set incsearch
set hlsearch

Now we will add some plugins in this file and then we are going to install it.There are couple of ways to install plugins and the way i like is by Vundle And the section below shows how to add plugins .

set nocompatible   
filetype off                  

set rtp+=~/.vim/bundle/Vundle.vim
call vundle#begin()

Plugin 'VundleVim/Vundle.vim'
Plugin 'wting/rust.vim'
Plugin 'racer-rust/vim-racer'
Plugin 'scrooloose/nerdTree' "Documents tree
Plugin 'davidhalter/jedi-vim' "python autocompletion
Plugin 'tpope/vim-fugitive' "git integration
Plugin 'scrooloose/syntastic' "code syntaxis
Plugin 'Raimondi/delimitMate' "automatic closing of quotes, parenthesis...
Plugin 'yggdroot/indentline' "Show indent lines (useful for loops)
Plugin 'tmhedberg/SimpylFold' "Python folding
Plugin 'vim-airline/vim-airline'
Plugin 'vim-airline/vim-airline-themes'
Plugin 'ervandew/supertab' "<Tab> for code completion
Plugin 'ctrlpvim/ctrlp.vim'    "Finder for vim
Plugin 'jistr/vim-nerdtree-tabs'

call vundle#end()   

filetype plugin indent on

That’s how we add Plugins to the vim by writing Plugin and after that what plugin we want to install. Now to install plugins first type Esc then : and after that type PluginInstall then press enter and all the plugins will install one by one.

Now vim is looks something like blow image.

Thank you 🙂

Reusable playbooks, Ansible roles

So, in this blog I have told you about “what is Ansible, playbooks, hosts?” now it is the time to go further with roles. In the world of programming we have a principle called DRY which is “Don’t repeat yourself” means don’t do any work again and again. For that we try to make a process that we have to write the code one time and we can use it again and again without rewriting the whole code again and again.

In the era of automation Ansible also gives us a certain technique that if we need a part of playbook multiple files then we can write it inside a role and use it again and again. It also helps us to reduce the complexity of the code like it is always better to break down a big playbook in a small parts and use it.

Basically for the role we define it inside a roles directory. inside this directory we define or every role and inside the role their will be multiple directories below you can see it –

$ tree .
.
├── roles
│   └── webservers
│       ├── defaults
│       ├── files
│       ├── handlers
│       ├── meta
│       ├── tasks
│       ├── templates
│       └── vars
└── web.yml

Roles expect files to be in certain directory names. Roles must include at least one of these directories, however it is perfectly ok to exclude any which are not being used. When in use, each directory must contain a main.yml file, which contains the relevant content:

  • tasks – contains the main list of tasks to be executed by the role.
  • handlers contains handlers which may be used by the roles or elsewhere in the code.
  • defaults contains variables for the roles.
  • vars contains other variables for the roles.
  • files contains files which will deploy by the roles.
  • templates contains templates which can be deployed via roles.
  • meta contains meta data.

So, now see by example how roles helps us in the various aspects of automation. Below the code is without roles –


---
- hosts: webservers
  gather_facts: yes
  remote_user: root
  become: true
  tasks:
    - name: Ensure group {{ item }} exist
    group:
      name: "{{ item }}"
      state: present
    loop:
      - mozilla
      - dgplug
  - name: add several users to several group
    user:
      name: "{{ item.name }}"
      state: present
      groups: "{{ item.groups }}"
    loop:
      - { name: 'testuser1', groups: 'mozilla' }
      - { name: 'testuser2', groups: 'dgplug' }
  - name: remove several users to several group
    user:
      name: "{{ item.name }}"
      state: absent
      groups: "{{ item.groups }}"
    loop:
      - { name: 'testuser1', groups: 'mozilla' }
      - { name: 'testuser2', groups: 'dgplug' }

Now see how we can implement this via roles –

# web.yml
---
- hosts: webservers
  become: true
  roles:
    - role: '/home/aniruddha/Desktop/ansible-playbook/roles/webservers'

# roles/webservers/tasks/add_group/main.yml
---
tasks:
  - name: Ensure group {{ item }} exist
  group:
    name: "{{ item }}"
    state: present
  loop:
    - mozilla
    - dgplug

# roles/webservers/tasks/add_user_to_group/main.yml
---
tasks:
  - name: add several users to several group
  user:
    name: "{{ item.name }}"
    state: present
    groups: "{{ item.groups }}"
  loop:
    - { name: 'testuser1', groups: 'mozilla' }
    - { name: 'testuser2', groups: 'dgplug' }

# roles/webservers/tasks/remove_user_from_group/main.yml
---
tasks:
  - name: Remove several users from different group
  user:
    name: "{{ item.name }}"
    state: absent
    group: "{{ item.groups }}"
  loop:
    - { name: 'testuser1', groups: 'mozilla' }
    - { name: 'testuser2', groups: 'dgplug' }

Thank you 🙂