Showing posts with label line. Show all posts
Showing posts with label line. Show all posts

Friday, 23 October 2015

authfile in dsjob command


While working on Datastage CLI commands, in some cases when you are working on server which is having multiple DataStage server install Or your user id is don't have sufficient permission to login on DataStage, we have to give DataStage server details like DataStage server host, username, password etc in the command line. But, this creates a small issue, anyone can see the username and password details which you have used in the command line by checking the server command history or other ways.

                           IBM Datastage provides a way to secure your username and password with authfile parameter. Today, we will see what it is and how we can use in our command line statement.



authfile:



How to use in commands:

dsjob -authfile c:\auth_file.txt -lprojects
dsjob -authfile c:\auth_file.txt -ljobs dstage1





Like the below page to get update  
https://www.facebook.com/datastage4you
https://twitter.com/datagenx
https://plus.google.com/+AtulSingh0/posts
https://groups.google.com/forum/#!forum/datagenx

Thursday, 8 October 2015

dsjob #3 - Managing Datastage Jobs from Command Line




0. Sourcing DSHOME in Windows or Linux
In Windows - Add the DSENGINE/bin path to Windows PATH

dsjob-path

In Linux - Add the DSENGINE/bin path to Linux PATH or source the dsenv file



1. Listing Project:
dsjob –lprojects         ---List of all projects
dsjob -authfile filename -lprojects  ---If needed username and password for dsjob

dsjob-listproject

2. Listing Jobs:
dsjob –ljobs project    ---List of all Jobs under a project
dstage1 - project name

dsjob-listjobs

3. Listing Stages:
dsjob –lstages [useid] project job|job_id --- List of all stages in a job
xfm7 -- jobname

dsjob-lstages

4. Listing Links:
dsjob –links [useid] project job|job_id stage  --- List of the links within a job design
xfm7 - jobname
Transformer_1 - stagename

dsjob-llinks

5. Listing Parameters:
dsjob –lparams [useid] project job|job_id  ---List of all parameters used within a Job
This job is not having any parameter so <none> in output.

dsjob-lparams

6. Displaying Job Information:
dsjob –jobinfo [useid] project job|job_id

dsjob-jobinfo

7. Displaying stage information
dsjob –stageinfo [useid] project job|job_id

dsjob-stageinfo

8. Displaying Link Information
dsjob –linkinfo [useid] project job|job_id stage link

dsjob-linkinfo

9. Adding a Log Entry
dsjob –log [–info | –warn] [useid] project job|job_id

dsjob-log

10. Displaying a Short Log Entry
dsjob –logsum [–type type] [ –max n ] [useid] project job|job_id

dsjob-logsum

11. Generating a Report:
dsjob –report [useid] project job|job_id [report_type]

dsjob-report


You can find many more post like this HERE



Like the below page to get update  
https://www.facebook.com/datastage4you
https://twitter.com/datagenx
https://plus.google.com/+AtulSingh0/posts
https://groups.google.com/forum/#!forum/datagenx

Tuesday, 6 October 2015

dsjob #2 - Managing Datastage Jobs from Command Line


Continued from dsjob - Managing Datastage Jobs from Command Line #1






Listing Projects, Jobs, Stages, Links, and Parameters :



We can list projects, jobs, stages, links, and job parameters using the

dsjob command. The different versions of the syntax are described in
the following sections.

The following syntax works for listing information about a Datastage job as below:

  •     dsjob –lprojects         ---List of all projects
  •     dsjob –ljobs project    ---List of all Jobs under a project
project is the name of the project containing the jobs to list.


  •     dsjob –lstages [useid] project job|job_id --- List of all stages in a job
1.    useid specify this if we intend to use a job alias (jobid) rather than a
2.    job name (job) to identify the job.
3.    project is the name of the project containing job.
4.    job is the name of the job containing the stages to list. To identify a
5.    job invocation, use the format job.invocation_id.
6.    job_id is an alias for the job that has been set using the dsjob -jobid
7.    command


  •     dsjob –links [useid] project job|job_id stage  --- List of the links within a job design
1.    useid specify this if we intend to use a job alias (jobid) rather than a
2.    job name (job) to identify the job.
3.    project is the name of the project containing job.
4.    job is the name of the job containing stage. To identify a job
5.    invocation, use the format job.invocation_id.
6.    job_id is an alias for the job that has been set using the dsjob -jobid
7.    stage is the name of the stage containing the links to list.


  •     dsjob –lparams [useid] project job|job_id  ---List of all parameters used within a Job



Setting an Alias for a Job:


The dsjob command can be used to specify own ID for a

DataStage job.Other commands can then use that alias to refer to the
job.

Syntax: dsjob –jobid [my_ID] project job


Explanations of the available options

  1. my_ID is the alias we want to set for the job. If we omit my_ID, the command will return the current alias for the specified job. An alias must be unique within the project, if the alias already exists an error message is displayed project is the name of the project containing job.
  2. job is the name of the job. To identify a job invocation, use the format job.invocation_id

    
Retrieving Information:

The dsjob command can be used to retrieve and display the available information about specific projects, jobs, stages, or links. The different

versions of the syntax are described in the following sections.

Displaying Job Information


Syntax: dsjob –jobinfo [useid] project job|job_id


Explanations of the available options:

  1. useid specify this if we intend to use a job alias (jobid) rather than a job name (job) to identify the job.
  2. project is the name of the project containing job.
  3. job is the name of the job. To identify a job invocation, use the format job.invocation_id.

The following information is displayed:
  1. The current status of the job
  2. The name of any controlling job for the job
  3. The date and time when the job started
  4. The wave number of the last or current run (internal DataStage reference number)
  5. User status

Displaying Stage Information:

Syntax: dsjob –stageinfo [useid] project job|job_id stage


Explanations of the available options:

  1. useid specify this if we intend to use a job alias (jobid) rather than a
  2. job name (job) to identify the job.
  3. project is the name of the project containing job.
  4. job is the name of the job containing stage. To identify a job
  5. invocation, use the format job.invocation_id.
  6. job_id is an alias for the job that has been set using the dsjob -jobid command.
  7. stage is the name of the stage.

The following information is displayed:
  1. The last error message reported from any link to or from the stage
  2. The stage type name, for example, Transformer or Aggregator
  3. The primary links input row number

Displaying Link Information:

Syntax: dsjob –linkinfo [useid] project job|job_id stage link


Explanations of the available options:

  1. useid specify this if we intend to use a job alias (jobid) rather than a
  2. job name (job) to identify the job.
  3. project is the name of the project containing job.
  4. job is the name of the job containing stage. To identify a job
  5. invocation, use the format job.invocation_id.
  6. job_id is an alias for the job that has been set using the dsjob -jobid command. 
  7. stage is the name of the stage containing link.
  8. link is the name of the stage.

The following information is displayed:
  1. The last error message reported by the link
  2. The number of rows that have passed down a link


Accessing Log Files:



Adding a Log Entry

Syntax: dsjob –log [–info | –warn] [useid] project job|job_id



Explanations of the available options:
  1. info specifies an information message. This is the default if no log entry type is specified.
  2. -warn specifies a warning message.
  3. useid specify this if we intend to use a job alias (jobid) rather than a
  4. job name (job) to identify the job.
  5. project is the name of the project containing job.
  6. job is the name of the job that the log entry refers to. To identify a job invocation, use the format job.invocation_id.
  7. job_id is an alias for the job that has been set using the dsjob -jobid command.

Displaying a Short Log Entry

Syntax: dsjob –logsum [–type type] [ –max n ] [useid] project job|job_id


type specifies the type of log entry to retrieve.

If –type is not specified, all the entries are retrieved.


Importing Job Executables:

The dsjob command can be used to import job executables from a

DSX file into a specified project.

Syntax: dsjob –import project DSXfilename [-OVERWRITE] [-JOB[S] jobname …] |[-LIST]



  1.  DSXfilename is the DSX file containing the job executables.
  2. -OVERWRITE specifies that any existing jobs in the project with the same name will be overwritten.
  3. -JOB[S] jobname specifies that one or more named job executables should be imported (otherwise all the executable in the DSX file are imported).
  4. -LIST causes DataStage to list the executables in a DSX file rather than import them.

Generating a Report:

The dsjob command can be used to generate an XML format report

containing job, stage, and link information.

Syntax: dsjob –report [useid] project job|job_id [report_type]


Report_type is one of the following:


 BASIC – Text string containing start/end time, time elapsed and status of job.

 DETAIL – As basic report, but also contains information about individual stages and links within the job.
 LIST – Text string containing



Examples are available HERE





Like the below page to get update  
https://www.facebook.com/datastage4you
https://twitter.com/datagenx
https://plus.google.com/+AtulSingh0/posts
https://groups.google.com/forum/#!forum/datagenx

Sunday, 4 October 2015

dsjob #1- Managing Datastage Jobs from Command Line



Datastage Command Line Interface:

The DataStage CLI comprises three groups of commands, those concerned with running DataStage Jobs, those concerned with administering DataStage projects, and those concerned with searching jobs.



dsjob command:
 
The dsjob command can be used from the UNIX command line to execute of the following tasks, All output from the dsjob command is in plain text without column headings on lists, or any other sort of description. This enables the command to be used in shell or batch scripts without extra processing.


These options are described in the following topics:


  •      The logon clause
  •      Starting a job
  •      Stopping a job
  •      Listing projects, jobs, stages, links, and parameters
  •      Setting an alias for a job
  •      Retrieving information
  •      Accessing log files
  •      Importing job executables
  •      Generating a report

Logon clause:

By default, the DataStage CLI connects to the DataStage server engine on the local system using the user name and password of the user invoking the command. We can specify a different server, user name, or password using the logon clause.

Syntax :

[–server servername][–user username][–password password]

servername specifies a different server to log on to.
username specifies a different user name to use when logging on.
password specifies a different password to use when logging on.

We can use the logon clause with any dsjob command.

Starting a Job:

We can start, stop, validate, and reset Datastage jobs using the –run option from the UNIX command line

Syntax :

dsjob –run
[ –mode [ NORMAL | RESET | VALIDATE ] ]
[ –param name=value ]
[ –warn n ]
[ –rows n ]
[ –wait ]
[ –stop ]
[ –jobstatus]
[–userstatus]
[–local]
[–opmetadata [TRUE | FALSE]]
[-disableprjhandler]
 [-disablejobhandler]
[useid] project job| job_id




Explanations of the available options

mode specifies the type of job run. NORMAL starts a job run, RESET resets the job and VALIDATE validates the job. If –mode is not specified, a normal job run is started.

param specifies a parameter value to pass to the job. The value is in the format name=value, where name is the parameter name, and value is the value to be set. If we use this to pass a value of an environment variable for a job (as we may do for parallel jobs), we need to quote the environment variable and its value, for example -param '$APT_CONFIG_FILE=sayan.apt' otherwise the current value of the environment variable will be used.

warn n sets warning limits to the value specified by n

rows n sets row limits to the value specified by n.

wait waits for the job to complete.

stop terminates a running job.

jobstatus waits for the job to complete, then returns an exit code
derived from the job status.

userstatus waits for the job to complete, then returns an exit code derived from the user status if that status is defined.
-local use this when running a DataStage job from within a shellscript on a UNIX server.

-opmetadata use this to have the job generate operational meta data as it runs.  If we specify FALSE, the job will not generate operational meta data.

-disableprjhandler use this to disable any error message handler that has been set on a project wide basis.

-disablejobhandler use this to disable any error message handler that has been set for this job.

-useid specify this if we intend to use a job alias (jobid) rather than a job name (job) to identify the job.

-project is the name of the project containing the job. job is the name of the job. To run a job invocation, use the format job.invocation_id.

job_id is an alias for the job that has been set using the dsjob -jobid command.


Stopping a Job:

We can stop a Datastage job using the –stop option from UNIX command prompt

Syntax:
dsjob –stop [useid] project job| job_id


 

Next part is available HERE




Like the below page to get update  
https://www.facebook.com/datastage4you
https://twitter.com/datagenx
https://plus.google.com/+AtulSingh0/posts
https://groups.google.com/forum/#!forum/datagenx

 

Thursday, 27 August 2015

VI Editor in Linux - 1





Introduction to vi Editor 


  • Full-screen editor
  • Two modes of operation: command and text
  • Utilizes one letter commands
  • Does not format text
  • Flexible search and replace facility with pattern matching
  • Allows for user-defined editing functions using macros

Modes of Operation



The vi editor has two modes of operation. Command mode allows the user to perform
operations on the text, such as cut, paste, cursor movement, and replacement. These
operations are all invoked by one character “commands.” Text mode (also known as
insert mode) allows the user to enter text; all characters typed are inserted into the file.

There is no way to tell which mode the editor is in, other than pressing a key on the
keyboard. If the key you press is inserted into the file, the current mode is text mode. If
the key performs an action, the current mode is command mode.

Switching from command mode to text mode is done with one of the insert text
commands, which will be covered later. Returning to command mode is done by
pressing the <Esc> key.


Starting vi 


$ vi filename
  • If the file "filename" does not exist, it will be created
  • Otherwise, vi will open the existing file
When a session is initiated, one of two things happens:
  • If the file to be edited exists, a copy of the file is put into a buffer in temporary by default.
  • If the file does not exist, an empty buffer is opened for this session.
The tilde characters represent empty lines in the editor.  The editor starts in command mode.





Tuesday, 25 August 2015

Putty - Command Line Magic Continues......





A) Delete All PuTTY Sessions Together
When you are swapping an old computer with a new computer, you may end-up transferring all PuTTY sessions to new computer. Once you’ve transferred all PuTTY sessions, execute “putty -cleanup” from the command line as shown below.

Go to the path where Putty.exe in installed and execute below command
C:>putty –cleanup


B) Magic Command to BackUp Putty Sessions
a) Open Run in Windows ( Window + R )
b) Put below command in Run
regedit /e "%userprofile%\desktop\putty-registry.reg" HKEY_CURRENT_USER\Software\Simontatham
c) This will create a reg file of Putty Session on your desktop.
d) Putty Sessions are backed up.


C) If you are fed up with putting Username again n again in multiple putty sessions.
Try this
a) Clicked on Any Saved Session and Load it
b) Click Connection > Data in the Category navigation tree.



Tuesday, 18 August 2015

Putty - Command Line Magic


We all are using ssh client PUTTY in day to day task and it is very irritating to login in different server again and again. Today, I come up with Putty command line by which we can make this so easy. Wanna to login, Just a click and Voila !!!



So let's start --

a.  First of all make a shortcut of your putty.exe file by right click --> Send to --> Desktop
b. ,This will display like below, just rename it with your server name or address to know which server is going to connect when we click on it.

c. Here, I have used my linux server  192,168,37.129
d. Now, Right click on Putty Shortcut  ---> Properties. This will display like below -


e. We have to edit the Target command  -- 

For me  :-
Server Address - 192.168.37.129
User Name - atul
Password - atul

edited command ---    -ssh user@server -pw password
For my case  ---    -ssh atul@192.168.37.129 -pw atul

Add this command to Target value , after whatever is existed there.  So Target's new value is ( in my case ) -   C:\_mine\putty.exe -ssh atul@192.168.37.129 -pw atul 



f. Click on Apply and OK.
g. For accessing 192.168.37.129 without entering username and password, simply click on edited shortcut.


** Caution ** :  Do not edit putty in public computer as your username and password is normal text which can be misused by anyone.