Showing posts with label Project. Show all posts
Showing posts with label Project. Show all posts

Sunday, 5 March 2017

How to get DataStage Project Settings


DSListProjectSettings.bat is used to report all available settings of existing projects.

USAGE: DSListProjectSettings <SERVER> <USER> <PASSWORD> <PROJECT>

The DSListProjectSettings batch file reports all available project settings and environment variable names and values to standard output. The first section shows available DataStage project properties and their settings. The second section shows environment variable names and values. When run successfully, each section will report a status code of zero.



..


Like the below page to get update  
https://www.facebook.com/datastage4you
https://twitter.com/datagenx
https://plus.google.com/+AtulSingh0/posts
https://datagenx.slack.com/messages/datascience/

Friday, 23 October 2015

authfile in dsjob command


While working on Datastage CLI commands, in some cases when you are working on server which is having multiple DataStage server install Or your user id is don't have sufficient permission to login on DataStage, we have to give DataStage server details like DataStage server host, username, password etc in the command line. But, this creates a small issue, anyone can see the username and password details which you have used in the command line by checking the server command history or other ways.

                           IBM Datastage provides a way to secure your username and password with authfile parameter. Today, we will see what it is and how we can use in our command line statement.



authfile:



How to use in commands:

dsjob -authfile c:\auth_file.txt -lprojects
dsjob -authfile c:\auth_file.txt -ljobs dstage1





Like the below page to get update  
https://www.facebook.com/datastage4you
https://twitter.com/datagenx
https://plus.google.com/+AtulSingh0/posts
https://groups.google.com/forum/#!forum/datagenx

Thursday, 8 October 2015

dsjob #3 - Managing Datastage Jobs from Command Line




0. Sourcing DSHOME in Windows or Linux
In Windows - Add the DSENGINE/bin path to Windows PATH

dsjob-path

In Linux - Add the DSENGINE/bin path to Linux PATH or source the dsenv file



1. Listing Project:
dsjob –lprojects         ---List of all projects
dsjob -authfile filename -lprojects  ---If needed username and password for dsjob

dsjob-listproject

2. Listing Jobs:
dsjob –ljobs project    ---List of all Jobs under a project
dstage1 - project name

dsjob-listjobs

3. Listing Stages:
dsjob –lstages [useid] project job|job_id --- List of all stages in a job
xfm7 -- jobname

dsjob-lstages

4. Listing Links:
dsjob –links [useid] project job|job_id stage  --- List of the links within a job design
xfm7 - jobname
Transformer_1 - stagename

dsjob-llinks

5. Listing Parameters:
dsjob –lparams [useid] project job|job_id  ---List of all parameters used within a Job
This job is not having any parameter so <none> in output.

dsjob-lparams

6. Displaying Job Information:
dsjob –jobinfo [useid] project job|job_id

dsjob-jobinfo

7. Displaying stage information
dsjob –stageinfo [useid] project job|job_id

dsjob-stageinfo

8. Displaying Link Information
dsjob –linkinfo [useid] project job|job_id stage link

dsjob-linkinfo

9. Adding a Log Entry
dsjob –log [–info | –warn] [useid] project job|job_id

dsjob-log

10. Displaying a Short Log Entry
dsjob –logsum [–type type] [ –max n ] [useid] project job|job_id

dsjob-logsum

11. Generating a Report:
dsjob –report [useid] project job|job_id [report_type]

dsjob-report


You can find many more post like this HERE



Like the below page to get update  
https://www.facebook.com/datastage4you
https://twitter.com/datagenx
https://plus.google.com/+AtulSingh0/posts
https://groups.google.com/forum/#!forum/datagenx

Tuesday, 6 October 2015

dsjob #2 - Managing Datastage Jobs from Command Line


Continued from dsjob - Managing Datastage Jobs from Command Line #1






Listing Projects, Jobs, Stages, Links, and Parameters :



We can list projects, jobs, stages, links, and job parameters using the

dsjob command. The different versions of the syntax are described in
the following sections.

The following syntax works for listing information about a Datastage job as below:

  •     dsjob –lprojects         ---List of all projects
  •     dsjob –ljobs project    ---List of all Jobs under a project
project is the name of the project containing the jobs to list.


  •     dsjob –lstages [useid] project job|job_id --- List of all stages in a job
1.    useid specify this if we intend to use a job alias (jobid) rather than a
2.    job name (job) to identify the job.
3.    project is the name of the project containing job.
4.    job is the name of the job containing the stages to list. To identify a
5.    job invocation, use the format job.invocation_id.
6.    job_id is an alias for the job that has been set using the dsjob -jobid
7.    command


  •     dsjob –links [useid] project job|job_id stage  --- List of the links within a job design
1.    useid specify this if we intend to use a job alias (jobid) rather than a
2.    job name (job) to identify the job.
3.    project is the name of the project containing job.
4.    job is the name of the job containing stage. To identify a job
5.    invocation, use the format job.invocation_id.
6.    job_id is an alias for the job that has been set using the dsjob -jobid
7.    stage is the name of the stage containing the links to list.


  •     dsjob –lparams [useid] project job|job_id  ---List of all parameters used within a Job



Setting an Alias for a Job:


The dsjob command can be used to specify own ID for a

DataStage job.Other commands can then use that alias to refer to the
job.

Syntax: dsjob –jobid [my_ID] project job


Explanations of the available options

  1. my_ID is the alias we want to set for the job. If we omit my_ID, the command will return the current alias for the specified job. An alias must be unique within the project, if the alias already exists an error message is displayed project is the name of the project containing job.
  2. job is the name of the job. To identify a job invocation, use the format job.invocation_id

    
Retrieving Information:

The dsjob command can be used to retrieve and display the available information about specific projects, jobs, stages, or links. The different

versions of the syntax are described in the following sections.

Displaying Job Information


Syntax: dsjob –jobinfo [useid] project job|job_id


Explanations of the available options:

  1. useid specify this if we intend to use a job alias (jobid) rather than a job name (job) to identify the job.
  2. project is the name of the project containing job.
  3. job is the name of the job. To identify a job invocation, use the format job.invocation_id.

The following information is displayed:
  1. The current status of the job
  2. The name of any controlling job for the job
  3. The date and time when the job started
  4. The wave number of the last or current run (internal DataStage reference number)
  5. User status

Displaying Stage Information:

Syntax: dsjob –stageinfo [useid] project job|job_id stage


Explanations of the available options:

  1. useid specify this if we intend to use a job alias (jobid) rather than a
  2. job name (job) to identify the job.
  3. project is the name of the project containing job.
  4. job is the name of the job containing stage. To identify a job
  5. invocation, use the format job.invocation_id.
  6. job_id is an alias for the job that has been set using the dsjob -jobid command.
  7. stage is the name of the stage.

The following information is displayed:
  1. The last error message reported from any link to or from the stage
  2. The stage type name, for example, Transformer or Aggregator
  3. The primary links input row number

Displaying Link Information:

Syntax: dsjob –linkinfo [useid] project job|job_id stage link


Explanations of the available options:

  1. useid specify this if we intend to use a job alias (jobid) rather than a
  2. job name (job) to identify the job.
  3. project is the name of the project containing job.
  4. job is the name of the job containing stage. To identify a job
  5. invocation, use the format job.invocation_id.
  6. job_id is an alias for the job that has been set using the dsjob -jobid command. 
  7. stage is the name of the stage containing link.
  8. link is the name of the stage.

The following information is displayed:
  1. The last error message reported by the link
  2. The number of rows that have passed down a link


Accessing Log Files:



Adding a Log Entry

Syntax: dsjob –log [–info | –warn] [useid] project job|job_id



Explanations of the available options:
  1. info specifies an information message. This is the default if no log entry type is specified.
  2. -warn specifies a warning message.
  3. useid specify this if we intend to use a job alias (jobid) rather than a
  4. job name (job) to identify the job.
  5. project is the name of the project containing job.
  6. job is the name of the job that the log entry refers to. To identify a job invocation, use the format job.invocation_id.
  7. job_id is an alias for the job that has been set using the dsjob -jobid command.

Displaying a Short Log Entry

Syntax: dsjob –logsum [–type type] [ –max n ] [useid] project job|job_id


type specifies the type of log entry to retrieve.

If –type is not specified, all the entries are retrieved.


Importing Job Executables:

The dsjob command can be used to import job executables from a

DSX file into a specified project.

Syntax: dsjob –import project DSXfilename [-OVERWRITE] [-JOB[S] jobname …] |[-LIST]



  1.  DSXfilename is the DSX file containing the job executables.
  2. -OVERWRITE specifies that any existing jobs in the project with the same name will be overwritten.
  3. -JOB[S] jobname specifies that one or more named job executables should be imported (otherwise all the executable in the DSX file are imported).
  4. -LIST causes DataStage to list the executables in a DSX file rather than import them.

Generating a Report:

The dsjob command can be used to generate an XML format report

containing job, stage, and link information.

Syntax: dsjob –report [useid] project job|job_id [report_type]


Report_type is one of the following:


 BASIC – Text string containing start/end time, time elapsed and status of job.

 DETAIL – As basic report, but also contains information about individual stages and links within the job.
 LIST – Text string containing



Examples are available HERE





Like the below page to get update  
https://www.facebook.com/datastage4you
https://twitter.com/datagenx
https://plus.google.com/+AtulSingh0/posts
https://groups.google.com/forum/#!forum/datagenx

Sunday, 4 October 2015

dsjob #1- Managing Datastage Jobs from Command Line



Datastage Command Line Interface:

The DataStage CLI comprises three groups of commands, those concerned with running DataStage Jobs, those concerned with administering DataStage projects, and those concerned with searching jobs.



dsjob command:
 
The dsjob command can be used from the UNIX command line to execute of the following tasks, All output from the dsjob command is in plain text without column headings on lists, or any other sort of description. This enables the command to be used in shell or batch scripts without extra processing.


These options are described in the following topics:


  •      The logon clause
  •      Starting a job
  •      Stopping a job
  •      Listing projects, jobs, stages, links, and parameters
  •      Setting an alias for a job
  •      Retrieving information
  •      Accessing log files
  •      Importing job executables
  •      Generating a report

Logon clause:

By default, the DataStage CLI connects to the DataStage server engine on the local system using the user name and password of the user invoking the command. We can specify a different server, user name, or password using the logon clause.

Syntax :

[–server servername][–user username][–password password]

servername specifies a different server to log on to.
username specifies a different user name to use when logging on.
password specifies a different password to use when logging on.

We can use the logon clause with any dsjob command.

Starting a Job:

We can start, stop, validate, and reset Datastage jobs using the –run option from the UNIX command line

Syntax :

dsjob –run
[ –mode [ NORMAL | RESET | VALIDATE ] ]
[ –param name=value ]
[ –warn n ]
[ –rows n ]
[ –wait ]
[ –stop ]
[ –jobstatus]
[–userstatus]
[–local]
[–opmetadata [TRUE | FALSE]]
[-disableprjhandler]
 [-disablejobhandler]
[useid] project job| job_id




Explanations of the available options

mode specifies the type of job run. NORMAL starts a job run, RESET resets the job and VALIDATE validates the job. If –mode is not specified, a normal job run is started.

param specifies a parameter value to pass to the job. The value is in the format name=value, where name is the parameter name, and value is the value to be set. If we use this to pass a value of an environment variable for a job (as we may do for parallel jobs), we need to quote the environment variable and its value, for example -param '$APT_CONFIG_FILE=sayan.apt' otherwise the current value of the environment variable will be used.

warn n sets warning limits to the value specified by n

rows n sets row limits to the value specified by n.

wait waits for the job to complete.

stop terminates a running job.

jobstatus waits for the job to complete, then returns an exit code
derived from the job status.

userstatus waits for the job to complete, then returns an exit code derived from the user status if that status is defined.
-local use this when running a DataStage job from within a shellscript on a UNIX server.

-opmetadata use this to have the job generate operational meta data as it runs.  If we specify FALSE, the job will not generate operational meta data.

-disableprjhandler use this to disable any error message handler that has been set on a project wide basis.

-disablejobhandler use this to disable any error message handler that has been set for this job.

-useid specify this if we intend to use a job alias (jobid) rather than a job name (job) to identify the job.

-project is the name of the project containing the job. job is the name of the job. To run a job invocation, use the format job.invocation_id.

job_id is an alias for the job that has been set using the dsjob -jobid command.


Stopping a Job:

We can stop a Datastage job using the –stop option from UNIX command prompt

Syntax:
dsjob –stop [useid] project job| job_id


 

Next part is available HERE




Like the below page to get update  
https://www.facebook.com/datastage4you
https://twitter.com/datagenx
https://plus.google.com/+AtulSingh0/posts
https://groups.google.com/forum/#!forum/datagenx