Showing posts with label Database. Show all posts
Showing posts with label Database. Show all posts

Saturday, 23 June 2018

yet another In Memory DataBase - MemSQL

Writing this post after so many days, yet another IN-MEMORY database is in market which tag line promised with"The DataBase for Real-Time Applications". 

As per MemSQL site - MemSQL is a scalable SQL database that ingests data continuously to perform operational analytics for the front lines of your business. Ingest millions of events per day with ACID transactions while simultaneously analyzing billions of rows of data in relational SQL, JSON, or Geospatial formats.

In my current assignment, I've been asked to look into the capabilities of this db, so starting with very first step "Installation" - 

Installation in linux is quite simple, if you are OK with linux commands, You can follow the installation from HERE

1. Download the software (with sudo or root user) - 

sudo su - root

2. Extract the tar ball

tar zxvf memsql-ops-6.0.11.tar.gz

This command will extract lots n lots of files :-)

3. Run the installer script

cd memsql-ops-6.0.11

sudo ./ --simple-cluster

By default, MemSQL supports the machine with 4 cpu core and 8 GB of RAM (which is little unfair;-)) so remove this constraints by below argument -

cd memsql-ops-6.0.11

sudo ./ --simple-cluster --ignore-min-requirements

After being successful installation of MemSQL, it will start setting up MemSQL WebUI.

You can access the MemSQL WebUI on the sever's 9000 port by default. 


4. To connect to MemSQL command line, execute - 


In next post, I will explain how this db is different than other in-memory db. Till then, Keep Learning , Keep Loving.

Like the below page to get update  
Facebook Page      Facebook Group      Twitter Feed      Google+ Feed      Telegram Group     

Sunday, 14 January 2018

Mongo DB - Installation and Configuration

MongoDB  is an open-source document database, and the leading NoSQL database. Written in C++.
MongoDB features:
    Document-Oriented Storage
    Full Index Support
    Replication & High Availability
    Fast In-Place Updates

Reduce cost, accelerate time to market, and mitigate risk with proactive support and enterprise-grade capabilities.

Today, We will see how to install and run the MongoDB.

MongoDB Installation on Linux

1. DOWNLOAD the stable version of MongoDB. It will a tar file
2. Extract the tar file to some directory.
$ tar -xvf mongodb.tar -C /learn/mongodb

3.  change the permisson of folder to user who run the db here-  In my case User - hduser and Group - hadoop
$ chown -R hduser:hadoop /learn/mongodb

4. Add the env var in .bashrc
export MONGO_HOME=/learn/mongodb

5. Create the default DB directory for Mongo
$ mkdir -R /data/db
$ chown -R hduser:hadoop /data/db

This is by default, you can specify ur db path when starting the mongo db

$ mongod --dbpath /app/mongodata
this command will start the mongodb. in other terminal you can start work on db. "--dbpath /app/mongodata" is totally optional

If you just use just $ mongod , it will start n use the default db which we have defined in step 5.

Please don't close the current terminal, It can be kill the mongodb process.

6. Start working on MongoDB
$ mongo

Like the below page to get update

Friday, 7 July 2017

ICONV mystery - the UV function

Iconv (Internal CONVersion) is a function supported by UniVerse DB (UV db) to convert the DATA, not only DATE, into internal format. DataStage Server Jobs are using lots of UV functions to manipulate the data.

Today, I will try to unwrap the mystery behind the Iconv function and try to put the details in simpler words. We will not go into data conversion details but date conversion which is used by DataStage :-)

Like most of other date functions (Parallel one), Iconv also accept the date(string) and its format.

Suppose, Date =   June 17, 2017

To Convert this date into internal format, we have to use -

Iconv("2017-06-17", D-YMD)  = 18066
Iconv("2017/06/17", D/YMD)   = 18066
Iconv("2017:17:06", D:YDM)  = 18066
Iconv("17-06-17", D-Y2MD)    = 18066

D-  --> D for Delimiter followed by delimiter char
Y --> year in YYYY
M --> month in MM
D --> date in DD

As we can see, if we provide the date format with date string, Iconv convert the date to an integer no and it is very important to do because now datastage can understand the given date and we can use Oconv function to re-format the date as required.

I will cover Oconv in next post, till then Keep Learning !!

Like the below page to get update

Thursday, 21 July 2016

DBMS Books

If you dont have the dropbox account, create from here -

Database System Concepts 6E - Korth               Book Link and PPTs Link  
Database System 6E_Navathe - Navathe            Book Link and PPTs Link    

** The books and its content belongs only to its author and publication, Shared here for personal learning purpose. 

Like the below page to get update

Thursday, 19 May 2016

5 Tips For Better DataStage Design #13

1. The query used in the database should be in such a way that required number of rows are fetched. Do not extract the columns which are not required.

2. For parallel jobs, sequential File should not be read using same partitioning.

3. For huge amount of data, use of sequential file stage is not a good practice. This stage also should not be used for intermediate storage between jobs. It degrades the performance of the job.

4. The number of lookups in a job design should be minimum. Join stage is a good alternative to lookup stage.

5. For parallel jobs, proper portioning method is to be used for better job performance and accurate flow of data.

Like the below page to get update

Tuesday, 15 December 2015

How to use Universe Shell (uvsh) in DataStage?

In DataStage Administration, we have to use datastage command line (universe shell) to get the information directly from the datastage universe database.

While accessing it from command line what novice admin do is -

$ uvsh
This directory is not set up for DataStage.
Would you like to set it up(Y/N)?   
Confused ? What to do ?

Always answer that question "no", it means you're in the wrong place.
Always launch "uvsh" or "dssh" from one of two places - $DSHOME or inside a project directory. For the latter you're good to go, for the former you'll need to LOGTO your project name before you issue any sql.

How to use UVSH?

## Entered into the $DSHOME
$ cd $DSHOME

## Sourced the dsenv file
$ . dsenv

## Change directory to the project directory.
$ LOGTO <project_name>

## Run uvsh command 
$ $DSHOME/bin/uvsh

Many Datastage admin support to execute command from Datastage Administrator or use dssh instead of uvsh.

How to use DSSH?
## Sourced the dsenv file
$ . $DSHOME/dsenv

## Change directory to the project directory.
$ LOGTO <project_name>

## Run dssh command 
$ $DSHOME/bin/dssh

Like the below page to get update!forum/datagenx