Containerization - What & Why ??

Containerization, always a word which describe to hold something, literally taken from the world of freight transportation which allows to put lot of different product/item into one box and move around the world without worries of damages. Quite a definition :) Isn't it ?
                 In simple word, or I say in IT term, Containerization is a process or a way which allows user to have a sandbox environment with required software specific to versions which you can flush whenever you are done with it and re-instantiate it when needed.

Now questions comes then what is the difference between a Virtual Machine and Container which we are going to discuss next.

In last decade, Virtual Machines (VM) allow IT giants/users to have one physical machine and host different application and its variants in VMs which shares the resources from host machine. But this comes with a small price, the bottleneck of resources shared. Your physical machine limit to host VMs is totally depends on its resources such as storage, processing power or memory cause VM requires these as contains the guest os and application with its dependencies. Guest OS itself eats lots of host storage & memory and required to be patched on timely manner to support your application.

VM stack is somewhat look like below -

Containerization has removed guest OS dependency and uses Host machine and OS which substantially reduce the size of container as well as resources consumption which brings lot more pros over virtual machine. Containerization stack is as below -

In next post, we will discuss about pros and cons of VM and Containerization.

Like the below page to get the update  
Facebook Page      Facebook Group      Twitter Feed      Google+ Feed      Telegram Group     

Read More

MongoDB Index in Python - Simple Index

Like RDBMS Systems MongoDB also provide Indexes to improve it's performance to process the query quicker and return the resultset. Mongo supports different type of indexes such as SingleKey, Compound, MultiKey, PartialKey and Text Indexes. We will look into these ones one by one.

Starting with Simple Index or One Key Index which use only one key from the collection/document [quivalent as  Table/Row in RDBMS systems], Let's see how -

Mongo Shell Command:  db.<collectionName>.createIndex({<field>:<direction>})
pyMongo Command:      db.<collectionName>.create_index([(<field>, <direction>)

Let's analyze the impact of Index creation on Query Performance, first via mongo shell, second in python - 

In MongoShell:

In our example, we are taking the collection 'people' as an example which has the field 'last_name'

db.people.find({last_name:'Tucker'}).explain('executionStats')

The above command will generate the executions stats for a query where last_name == Tuker .

as the execution plan shows, mongoDB scanned the whole collection (total 50747 documents for fetching 65 records) to fetch the result which is costly when your collection is big.

Now, Creating a Simple Index or Single Key Index

db.people.createIndex({"last_name":1}) 

Now, querying again the same - 

db.people.find({last_name:'Tucker'}).explain('executionStats')

This time MongoDB finds that there is an Index available on last_name columns which has been used to fetch the result. It scanned only 65 index keys to fetch 65 records. 

Single Key Index can be used in below scenarios - 
   - Querying on the range of Indexed Key values
   - Querying on selected values of Indexed Key

Advantage:
  - Returned result will be sorted by Index Key, no need to put a sort operation if sorting on the index key
  - Index key can be used in any sort order - Ascending or Descending

Consideration while Designing Single Key Index:
  - Do not create Single Key Index on each field available on collections, it will slow down the performance of select and write query both.

Like the below page to get the update  
Facebook Page      Facebook Group      Twitter Feed      Google+ Feed      Telegram Group     

Read More

Let's Learn - Git - Pull Specific Folder - sparsecheckout

What if your git repository has lots of folders but you have to work on a specific file in a particular folder. This git feature is called Sparse checkout. Previous Versions of git doesn't support this feature which forces you to download the whole repository. Sometime repository is too big to download and time-consuming process.

Current git versions support Sparse checkout which allows you to clone or fetch only a particular folder from a very big repository. Let's see how we can achieve it.

Task - Need to sync a folder named 'other' from 'DataGenX' repository 

Step #1: Initialize the Repository
Create a folder where you want to sync your git repo folder and Initialize git

Step #2: Add the Remote Repository
Add the remote Git repository with this local git repo as below -

Step #3: Create and Checkout a branch [Optional Step]
Creating of the branch is a totally optional step but it is advisable to create.

Step #4: Enable the Sparse Checkout Properties
Now, we have to enable the Sparse checkout properties and adding the folder name (in our case - 'other') in property file which we want to sync.

Step #5: Pull the Specific Folder
This is the last step where we pull the specific folder as below -

git pull <remote> <pull_branch_name>  #not locally created

while running this command, need to give proper branch name from where you want to pull the data, In our case, it is master.

Step #6: List and work with synced directory

Commands as below - 
==

Like the below page to get the update  
Facebook Page      Facebook Group      Twitter Feed      Google+ Feed      Telegram Group     

Read More

How to Query on MongoDB by _id - INTERMEDIATE II

What if we have to query on MongoDB collections based on the "_id" field, Can we really query on "_id" field ? If so, what is the syntax ? Let's try this out -

Let's first fetch a document's id -

$ db.user.find_one()

{'_id': ObjectId('5c16e863817810ed3fc5e5f9'),
 'Fname': 'atul',
 'Lname': 'Singh',
 'Grade': 12.0,
 'College': 'SGM',
 'Job': 'Student',
 'Address': 'Young St.'}

Now, we will pick the object id and use this to fetch the same document from collection

$ db.user.find_one({'_id':'5c16e863817810ed3fc5e5f9'}) #this will return nothing

$ db.user.find_one({'_id':ObjectId('5c16e863817810ed3fc5e5f9')})
---------------------------------------------------------------------------
NameError                                 Traceback (most recent call last)
 in 
----> 1 db.user.find_one({'_id':ObjectId('5c16e863817810ed3fc5e5f9')})

NameError: name 'ObjectId' is not defined

We have received this error because ObjectId is not the same as its string representation, it must be converted to ObjectId from a string before it is passed to find command.

$ from bson.objectid import ObjectId
$ db.user.find_one({'_id':ObjectId('5c16e863817810ed3fc5e5f9')})

{'_id': ObjectId('5c16e863817810ed3fc5e5f9'),
 'Fname': 'atul',
 'Lname': 'Singh',
 'Grade': 12.0,
 'College': 'SGM',
 'Job': 'Student',
 'Address': 'Young St.'}

IPYTHON Notebook can be found HERE

Like the below page to get the update  
Facebook Page      Facebook Group      Twitter Feed      Google+ Feed      Telegram Group     

Read More

MongoDB with Python - Basics IV - Update & Delete Operation

Welcome to one more quick session on MongoDB CRUD Basics with Update and Delete Operations. MDB Provides below methods under these operations -

** Update
      * update_one
      * update_many
      * replace_one

** Delete
      * delete_one
      * delete_many

For on Mongo DB -> Link

As it is quite clear from the name itself (_one and _many) that these methods perform the operation on single or multiple records based on the passed condition.
There are so many operator supported by update statement, few are as below -

$set - Add new or update field value
$unset  - Remove field
$inc - increment the current value
$push - push element into array field
$push with $each - push multiple elements into array field
$pop - pull out last value from array field

There are many more Update operators support by MongoDB, Full list can be found HERE


CRUD Operation - (Update & Delete) : Link

Like the below page to get the update  
Facebook Page      Facebook Group      Twitter Feed      Google+ Feed      Telegram Group     

Read More

How to iterate MongoDB Cursor in Python - Intermediate I

Whenever querying on mongodb, always store the output into a variable, called cursor, before performing any operation on data. It will keep your data into variable without messing your output ground. PyMongo Cursor variable supports few functions which helps with few information without actual seeing your data such as retrieved data count or distinct values in particular key etc.

In this session, we will learn about the mongo db cursor variables, for this exercise also we are going to use 'USER' database hosted on free tier MongoDB Atlas (M0) Server.

Jupyter Notebook can be accessed HERE also

= =


Next Post on this Series and more on MongoDB can be find here -> LINK

Like the below page to get the update  
Facebook Page      Facebook Group      Twitter Feed      Google+ Feed      Telegram Group     

Read More

MongoDB with Python - Basics III - Find/Select Operation

Hoping you guys are enjoying the NoSql journey so far (previous posts links), till now we have seen basic CRUD operation. From this post onward, I am diving in details of these operations and starting with FIND or SELECT operation in MongoDB. We will learn what are the ways and options provided by MongoDB to select or project the data.

When you start working with complex queries you might {as I have said "Might"} face difficulties with tracking of braces {([ as I've experienced the same with me/my team/students and colleagues. But no worries, Jupyter Notebook provides a  couple highlighter for braces when selected or you can use notepad++ also (which I think is not so useful as you are not gonna copy/paste the syntax so frequently).

I advise everyone to avoid the writing queries directly on mongo shell prompt as it doesn't provide any intelligence and not so good in fixing queries if made mistake.

I am sure you will love this post as well and if have any question feel free to ask in comment section below.

For on Mongo DB -> Link

There are many more Find/Read Operators supported by MongoDB, Full list can be found HERE

CRUD Operation - (Read) : Link




Next Post on this Series and more on MongoDB can be find here -> LINK

Like the below page to get the update  
Facebook Page      Facebook Group      Twitter Feed      Google+ Feed      Telegram Group     

Read More