Showing posts with label Learning. Show all posts
Showing posts with label Learning. Show all posts

Friday, 1 February 2019

Let's Learn - Git - Pull Specific Folder - sparsecheckout


What if your git repository has lots of folders but you have to work on a specific file in a particular folder. This git feature is called Sparse checkout. Previous Versions of git doesn't support this feature which forces you to download the whole repository. Sometime repository is too big to download and time-consuming process.

Current git versions support Sparse checkout which allows you to clone or fetch only a particular folder from a very big repository. Let's see how we can achieve it.

Task - Need to sync a folder named 'other' from 'DataGenX' repository 

Step #1: Initialize the Repository
Create a folder where you want to sync your git repo folder and Initialize git



Step #2: Add the Remote Repository
Add the remote Git repository with this local git repo as below -



Step #3: Create and Checkout a branch [Optional Step]
Creating of the branch is a totally optional step but it is advisable to create.



Step #4: Enable the Sparse Checkout Properties
Now, we have to enable the Sparse checkout properties and adding the folder name (in our case - 'other') in property file which we want to sync.



Step #5: Pull the Specific Folder
This is the last step where we pull the specific folder as below -

git pull <remote> <pull_branch_name>  #not locally created


while running this command, need to give proper branch name from where you want to pull the data, In our case, it is master.

Step #6: List and work with synced directory



Commands as below - 
==




Like the below page to get the update  
Facebook Page      Facebook Group      Twitter Feed      Google+ Feed      Telegram Group     


Sunday, 25 November 2018

Let's Learn - Git Branching #2


Continuing from the last post Let's Learn - Git Branching #1 where we have learned about git branches, how to create and do the parallel development with them.

a. Merge the changes to master:  we have completed development with branch "atul" and ready to merge those changes/module with the main branch "master". If there are no changes have been done in master after creating "atul" branch, merging process is quite easy. We will look into this first -

i. Git log of "atul" branch: 
==

From the screenshot, we can see that "atul" branch has initiated from "master" branch from below commit point -
master -  de70bc2450802c80dc5520afa20b53de30488e5a
atul -  781d0bb325d67ba251e9db3af4d8ef90c246e406

ii. Git log of "master" branch: Let''s quickly check the master log as well


In master branch, last commit where the HEAD is pointing to "de70bc2450802c80dc5520afa20b53de30488e5a" same commit which we can see inside "atul" branch which tells us that there is no changes have been done in master after branching "atul" out.

This kind of merge is simple, as this is just a repoing of HEAD and done. Let's see how we can merge -
==

This command will merge the "atul" branch changes to "master", we can confirm the same from master git log.


Let's quickly make a new change in the master branch and check the git log







after committing the changes (third.txt) to master, git log shows that HEAD point to the latest change






Like the below page to get the update  
Facebook Page      Facebook Group      Twitter Feed      Google+ Feed      Telegram Group     


Thursday, 22 November 2018

Let's Learn - Git Branching #1


Git branching feature is the most notable feature provided by Git for parallel development without impacting other developers. Using git branches, Developers can work on different module parallel with same base code and merge them later into one to build the complete picture.

We will start with the simple branching scenarios, where 2 branch has been initiated with same base code and merge them into one after few modifications.

a. List Git Branch:
==

b. Create Git Branch:
==

By above command, you can create a branch which has the same base code as master and when you list the branch, now you can see 2 names - atul (branch we created) and master (original). Asterisk * sign will tell us which repo is currently active.


c. Switch to Git Branch (checkout branch): 
===
To checkout branch is very easy and can be done by above command quickly. Just remember that whatever changes you have done in this branch will be local (remain) in this branch only.

d. Let's make some changes in the branch:
To make some changes, what I have done is created a new file name "atul.txt" and committed to this branch named "atul". You can found all the commands as below -
= =

now, we have made the changes, let's check the git log for this branch -


Here we have to understand one important branching concept, as you can see there are 2 branches (master and atul) pointer.

origin/master, master - this pointer will tell this branch "atul" has been branched from "master" from this commit de70bc2450802c80dc5520afa20b53de30488e5a

HEAD -> atul, origin/atul - this pointer will tell you that HEAD is pointing to "atul" branch means you are inside "atul" branch and commit 0ba89ab2d62a74c48951ba20b2c0f369c9a675a3 has been done

Let's quickly move to master branch and check whether these changes are there or not ??

e. Moving back to master:
Moving back process is the same as moving into the branch and check the git log in "master" branch


Here you can see that changes which we have committed inside "atul" branch hasn't been reflected in "master" branch.

This is how Git support Parallel Development with branching where the developer can create their own branch and develop the different features parallel.

f. What's the differences: 
We have created 2 branches named: "master" & "atul" and made some additional changes inside "atul" branch and commit to it. We can check the difference between these 2 branches by executing below command :
==

We will discuss about how to interpret this diff report in the next post, till then.....Happy Learning........





Like the below page to get the update  
Facebook Page      Facebook Group      Twitter Feed      Google+ Feed      Telegram Group     


Wednesday, 14 November 2018

Let's Learn - Password-less setup for Git


Every time when committing changes to github repository, we need to provide github credential which is annoying. Github provides solution for this repeatable activity by setting up password-less communication between github repo and local repository via SSH keys.


1. Generate SSH Keys -
ssh-keygen -t rsa -C your_registered_email@example.com



2, View and Copy the public key



3. Add the public keys to github repository.
a. Open your repo --> Setting



b. Click on Deploy --> Add Keys


c. Paste copied public key and save


SSH Setup has been completed, Now test it by pushing some changes to repository.

4. Making changes to README.md file


5. Stage and Commit


6. Last, push changes to remote.


Changes have been pushed to Remote repository without asking user credentials. You can use the same public key to all your git repository.

Hope you have enjoyed the post, will meet again with new stuff.....till then.......Happy Learning !!!..................



Like the below page to get update  
Facebook Page      Facebook Group      Twitter Feed      Google+ Feed      Telegram Group     


Tuesday, 13 November 2018

Let's Learn - Git Basics #2


Continuing from last post Let's Learn - Git Basics #1 where we have created the github account and a new repository. It is always a good practice to put all your code/work in one place so create a folder named something easily recognizable such as "gitCode" (in my case I have used "Git"). This will ensure that all your git code will be there for easy access.

Configuration:
Before starting to work, we will add configuration settings on our "Git" folder by executing below commands.


Check Configuration:
After setting up, you can view what your configurations look like by below commands -
=

=
=

Command to check Status:
To check the status of our work, GIT provides a command as below, It's very helpful while working with git - 
=
=
Working with new Repository:
a. We have already created a new repository on github named "git-learn", let's create a local repo as well.
==


b. After creating a local repository, we need to make it git repository by command "git init", this will create a hidden folder inside "git-learn" which holds all the changes done in local repository
=
==

c. Out git repository is ready, now let's make the first change inside it by creating a README.md file which holds the information about this repository. This is a markdown file which is used for repository documentation
==

d. After making this change, let's check the local git repo status.
==

"git status" command will display the current status of repo, Here, we can see README.md file under untracked files category (we will discuss about different type of status type later).
Untracked Files: Files about which git is not aware, as this is a new file. To push this file to remote git, we need to stage this file.

e. For staging this file, run the command as below -
==

After staging the file, we check the repo status which tells that there is a new file which has been staged to commit. But this changes hasn't been committed for push to remote repo.

f. Committing the changes is very easy and it's advisable to provide a proper comment while commit which help you in future while reviewing all the commits done on remote repo
==

g. Changes are committed, now we are good to push these changes to remote repository. But before that, as this is a new local repository, we need to tell this to which remote git repository it should commit to, Hence run the below add command, you have to run this command once per repository, so from next commit, no need to run this command unless you want to change the remote repo url.

When you push the local changes to remote, it will ask your github credentials as below (push will ask github credential every time which can be annoying and can be avoidable by setting up ssh keys with github account)
==

h. After entering github credentials, it will start pushing the data to remote git repo. 


i. When push is complete, we can check the current repo status as above.
j. You can view all the commits have been done on repository by "git log" command
==


In next post, we will see the type of changes done in git repository and how to setup password-less git repository.
Till then, Keep Loving this Blog .......... Happy Learning...........



Like the below page to get update  
Facebook Page      Facebook Group      Twitter Feed      Google+ Feed      Telegram Group     


Monday, 12 November 2018

Let's Learn - Git Basics #1


Github, Another awesome creation by Linux Father - Linus Torvalds. Github is an open source Code repository management service/tool which provides you ease of parallel development without any hassle and care to maintain the code versions and merge issue. If you want to know, read and learn here - https://lab.github.com

Git Basics #1:

1. If you are new to Github, Create your account first on https://github.com
 

2. After creating your account, create a new repository by clicking on  "New" button

3. Fill the Repository Name and descriptions as below -

You can initialize the README file and add the other files but I haven't done it as will add them from command line.

4. Once your repository has been created, Github shows you these commands to replicate the repo and use them in your local system.


If Repository is new -

If this is existing repository (which is not as we have created it just now)


Installing Git Client

1. Go to git-scm link - https://git-scm.com/downloads and choose Git client for your OS.
2. For windows users, It's pretty straight forward to download the client and install it as regular software.
3. For Linux users, It's little tricky as different linux distro have different software management or repository and command line to install it. But Git also take care of all the commands for most popular linux distro on this link - https://git-scm.com/download/linux

Once, You have the Git Client and Git Account Setup, We can start with the basics with #2 post.






Like the below page to get update  
Facebook Page      Facebook Group      Twitter Feed      Google+ Feed      Telegram Group     


Tuesday, 16 January 2018

Python Pickle - to save your efforts


This tutorial is for Python beginners who just started getting dirty in Python :-) Lots of time when we are working on some data set and completed lots of cleaning and pre-processing steps, It's advisable from GURUs to save that intermediary dataset to avoid the re-do all the steps if something unexpected happen with Python, Jupyter notebook or your system for rebooted without your permission, weird though :-/



So, let's learn how to save any dataset or variable with Pickle library -

To Import:


To Save: 


To Read:





Like the below page to get update  
https://www.facebook.com/datastage4you
https://twitter.com/datagenx
https://plus.google.com/+AtulSingh0/posts
https://datagenx.slack.com/messages/datascience/