Reading DSParam - datastage parameter file

I am sharing a utility which can help you to read DSParam file which holds all the environmental datastage parameters.

Utility to view contents of DSParams file. Useful when trying to see what all the customer has set at the project level.

Usage:
$ cat DSParams | ./DSParamReader.pl | more
or
$ cat DSParams | ./DSParamReader.pl > outputfile

Instructions:
1. copy script text below to a file (DSParamReader.pl) on a UNIX system
2. Set execute permissions on this file. chmod 777 envvar.pl
3. Usually perl is in /usr/bin/perl but you might have to adjust this path if neccessary. (hint "which perl" should tell you which one to use)
4. cat the DSParams file from the project you are concerned with and redirect the output to this script. You may have to put the Fully Qualified Path for this file.
5. capture the output to screen or file. File may be useful to have the customer send the info to you in email.








Like the below page to get update  
https://www.facebook.com/datastage4you
https://twitter.com/datagenx
https://plus.google.com/+AtulSingh0/posts
https://datagenx.slack.com/messages/datascience/

Read More

Python Points #14 - Code a childhood game

Level : Intermediate

Try to code this famous childhood game played in india, known as "Raja, Mantri, Chor, Sipahi", in python by seeing the game output shared below -

Little bit about Game:
Chits are made for Raja/King(100 points), Mantri/Minister(80 points),Chor/Thief(0 points) and Sipahi/Insprector(50 points). These chits are then thrown in the middle and 4 players pick one each. Raja/King then exclaims ‘Mera Manrti kaun?’ (Who is my minister?){In my game/code, King is so smart and asked directly to Mantri/Minister} Mantri/Minister responds and s/he is then asked to identify the Chor/Thief (Who stole my Queen's neckless ). If he guesses correctly then the points are retained if s/he is incorrect that he has to surrender the points to the Chor/Thief. The player with highest point wins in the end.

Like the below page to get update  
https://www.facebook.com/datastage4you
https://twitter.com/datagenx
https://plus.google.com/+AtulSingh0/posts
https://datagenx.slack.com/messages/datascience/

Read More

How to use Universe Shell (uvsh) in DataStage?

In DataStage Administration, we have to use datastage command line (universe shell) to get the information directly from the datastage universe database.

While accessing it from command line what novice admin do is -

$ uvsh
This directory is not set up for DataStage.
Would you like to set it up(Y/N)?   

Confused ? What to do ?

Always answer that question "no", it means you're in the wrong place.
Always launch "uvsh" or "dssh" from one of two places - $DSHOME or inside a project directory. For the latter you're good to go, for the former you'll need to LOGTO your project name before you issue any sql.

How to use UVSH?

## Entered into the $DSHOME
$ cd $DSHOME

## Sourced the dsenv file
$ . dsenv

## Change directory to the project directory.
$ LOGTO <project_name>

## Run uvsh command 
$ $DSHOME/bin/uvsh

Many Datastage admin support to execute command from Datastage Administrator or use dssh instead of uvsh.

How to use DSSH?

## Sourced the dsenv file
$ . $DSHOME/dsenv

## Change directory to the project directory.
$ LOGTO <project_name>


## Run dssh command 
$ $DSHOME/bin/dssh

Like the below page to get update  
https://www.facebook.com/datastage4you
https://twitter.com/datagenx
https://plus.google.com/+AtulSingh0/posts
https://groups.google.com/forum/#!forum/datagenx

Read More

5 Tips For Better DataStage Design #3

• Stage Variables allow you to hold data from a previous record when the next record, allowing you to compare between previous and current records. Stage variables also allow you return multiple errors for a record of information. By being able to evaluate all data in a record and not just error on the first exception that is found, the cleanup of data is more efficient and requires less iteration.

• Use Annotations for describing steps done at stages. Use Description Annotation as job title; as Description Annotation also appears in Job properties>Short Job Description and also in the Job Report when generated.

• When mapping a decimal field to a char field or vice versa , it is always better to convert the value in the field using the "Type Conversion" functions  "DecimalToString" or "StringToDecimal" as applicable while mapping.

• Always use Hash Partition in Join and Aggregator stages. The hash key should be the same as the key used to join/aggregate. If Join/Aggregator stages do not produce desirable results, try running in sequential mode (verify results; if still incorrect problem is with data/logic) and then run in parallel using Hash partition.

• In Job sequences; always use "Reset if required, then run" option in Job Activity stages. (Note: This is not a default option)

Like the below page to get update  
https://www.facebook.com/datastage4you
https://twitter.com/datagenx
https://plus.google.com/+AtulSingh0/posts
https://groups.google.com/forum/#!forum/datagenx

Read More

authfile in dsjob command

While working on Datastage CLI commands, in some cases when you are working on server which is having multiple DataStage server install Or your user id is don't have sufficient permission to login on DataStage, we have to give DataStage server details like DataStage server host, username, password etc in the command line. But, this creates a small issue, anyone can see the username and password details which you have used in the command line by checking the server command history or other ways.

                           IBM Datastage provides a way to secure your username and password with authfile parameter. Today, we will see what it is and how we can use in our command line statement.

authfile:



How to use in commands:

dsjob -authfile c:\auth_file.txt -lprojects

dsjob -authfile c:\auth_file.txt -ljobs dstage1

Like the below page to get update  
https://www.facebook.com/datastage4you
https://twitter.com/datagenx
https://plus.google.com/+AtulSingh0/posts
https://groups.google.com/forum/#!forum/datagenx

Read More

5 Tips for better DataStage Design & Administration #2

1. In a DataStage server environment with more than one version of DataStage installed in the dsjob command you must specify the server name and port number of the DataStage version that you want the ksh script to run on.
dsjob -domain NONE -user username -password password -server servername:portno -run script1

2.  While doing column mapping in designer, always checks whether source/target column have same Null properties, if not, it should be handled explicitly.

3.  Nulls are a curse when it comes to using functions/routines or normal equality type expressions.
E.g. NULL = NULL doesn’t work; neither does concatenation when one of the fields is null. Changing the nulls to 0 or “” before performing operations is recommended to avoid erroneous outcomes.

4. When using String functions on decimal always use Trim function to avoid as String functions interpret an extra Space used for sign in decimal.

5. When you need to get a substring (e.g. first 2 characters from the left) of a character field:
Use <Field Name>[1,2]
Similarly for a decimal field then:
Use Trim(<Field Name>)[1,2]

Like the below page to get update  
https://www.facebook.com/datastage4you
https://twitter.com/datagenx
https://plus.google.com/+AtulSingh0/posts
https://groups.google.com/forum/#!forum/datagenx

Read More