Sunday, 14 January 2018

Mongo DB - Installation and Configuration


MongoDB  is an open-source document database, and the leading NoSQL database. Written in C++.
  
MongoDB features:
    Document-Oriented Storage
    Full Index Support
    Replication & High Availability
    Auto-Sharding
    Querying
    Fast In-Place Updates
    Map/Reduce
    GridFS


Reduce cost, accelerate time to market, and mitigate risk with proactive support and enterprise-grade capabilities.


Today, We will see how to install and run the MongoDB.

MongoDB Installation on Linux


1. DOWNLOAD the stable version of MongoDB. It will a tar file
2. Extract the tar file to some directory.
 
$ tar -xvf mongodb.tar -C /learn/mongodb


3.  change the permisson of folder to user who run the db here-  In my case User - hduser and Group - hadoop
$ chown -R hduser:hadoop /learn/mongodb

4. Add the env var in .bashrc
export MONGO_HOME=/learn/mongodb
export PATH=$PATH:$MONGO_HOME/bin







5. Create the default DB directory for Mongo
$ mkdir -R /data/db
$ chown -R hduser:hadoop /data/db

This is by default, you can specify ur db path when starting the mongo db






$ mongod --dbpath /app/mongodata
this command will start the mongodb. in other terminal you can start work on db. "--dbpath /app/mongodata" is totally optional

If you just use just $ mongod , it will start n use the default db which we have defined in step 5.


Please don't close the current terminal, It can be kill the mongodb process.







6. Start working on MongoDB
$ mongo










Like the below page to get update  
https://www.facebook.com/datastage4you
https://twitter.com/datagenx
https://plus.google.com/+AtulSingh0/posts
https://datagenx.slack.com/messages/datascience/

Tuesday, 2 January 2018

Conda Commands


Sharing some CONDA commands which fasten the administration of different python environment. All these commands are available on Anaconda Site 








Like the below page to get update  
https://www.facebook.com/datastage4you
https://twitter.com/datagenx
https://plus.google.com/+AtulSingh0/posts
https://datagenx.slack.com/messages/datascience/

Wednesday, 20 December 2017

How to export the datastage job in individual dsx


This script will export the datastage jobs into individual dsx files as sometimes, we need individual dsx to maintain the code version or syncing it with SVN.

Manually export of each datastage job is quite hectic and boring hence this script :-)

This script requires little bit information from you and set you free for your other chores

Prerequisites for this script -

1. Job List to Export
2. DataStage Client Installation Location
3. DataStage Credential






Like the below page to get update  
https://www.facebook.com/datastage4you
https://twitter.com/datagenx
https://plus.google.com/+AtulSingh0/posts
https://datagenx.slack.com/messages/datascience/

Sunday, 19 November 2017

Get Job Parameters for all DataStage Job


Feels awesome while writing this post after 7 week, In this post I am going to share a script which will fetch the job parameters for all the datastage job available in given project.

This week, one of my colleague ask to compare all the jobs parameter from one project (older) to another project (migrated),  So, wrote down this script which worked like charm for me and hoping will work for you as well :-)





=


Like the below page to get update  
https://www.facebook.com/datastage4you
https://twitter.com/datagenx
https://plus.google.com/+AtulSingh0/posts
https://datagenx.slack.com/messages/datascience/

Friday, 29 September 2017

Letters and Symbols in MarkdDown

Thursday, 28 September 2017

Mathematics in Markdown


From Wiki - 
Markdown is a lightweight markup language with plain text formatting syntax designed so that it can be converted to HTML and many other formats using a tool by the same name.[8] Markdown is often used to format readme files, for writing messages in online discussion forums, and to create rich text using a plain text editor(extensions - *.markdown , *.md ). website
Best thing of markdown files is you can convert the same into html without any issue. 



I've introduced with Markdown files when I have started to put my code on GitHub (https://github.com/atulsingh0). Started with little up n downs but after I've fallen for it, Its easy to write ReadMe or Math Equations files in markdown with little help.

In this tutorial, I have focused on Mathematics part only, for writing math formulas, Markdown is using LaTeX symbols for Greek letters, Brackets, Sign operator and lots of other symbols.

I have consolidated few of them and will add more,
Hoping, you will find it useful -  Direct Link

==








Like the below page to get update  
https://www.facebook.com/datastage4you
https://twitter.com/datagenx
https://plus.google.com/+AtulSingh0/posts
https://datagenx.slack.com/messages/datascience/

Thursday, 14 September 2017

Evaluation Sequence in Transformer Stage - A Quick DataStage Recipe



Recipe:

What is evaluation sequence in Transformer Stage Or Order of Stage & Loop Variable and Derivations

Ingredients:

1. Transformer Stage
     a. Stage Variables
     b. Loop Variables
     c. Derivations


How To:

Evaluate each stage variable initial value
For each input row to process:
Evaluate each stage variable derivation value, unless the derivation is empty
For each output link:
Evaluate each column derivation value
Write the output record
Next output link
Next input row


** The stage variables and the columns within a link are evaluated in the order in which they are displayed in the Transformer editor. Similarly, the output links are also evaluated in the order in which they are displayed




Like the below page to get update  
https://www.facebook.com/datastage4you
https://twitter.com/datagenx
https://plus.google.com/+AtulSingh0/posts
https://datagenx.slack.com/messages/datascience/