DataGenX - Atul's Blog: Script

Showing posts with label Script. Show all posts

Friday, 21 September 2018

Get HTTPS and SSL details of App Deployed - IIB/WMB SCRIPTS

by Atul Singh / in Application, Command Line, commands, EG, Execution Group, Flow, HTTPS, IIB, iLoveScripting, Integration, Script, Shell Script, SSL, TLS, WMB /

One more to get the HTTPS and SSL details from an Application Deployed in IIB -

- -

Like the below page to get update
Facebook Page Facebook Group Twitter Feed Google+ Feed Telegram Group

Get Rest Flow from WMB/IIB Application - IIB/WMB SCRIPTS

by Atul Singh / in Application, Deployed, EG, Execution Group, Flow, IIB, iLoveScripting, Integration, Rest, Script, Server, Shell Script, Status, WMB /

Continuing the same journey with the WMN/IIB Scripts... Sharing another script which will return the REST Flow name from the application deployed on Execution Group ...

Like the below page to get update
Facebook Page Facebook Group Twitter Feed Google+ Feed Telegram Group

Get App deployed on Execution Group - IIB/WMB Scripts

by Atul Singh / in Application, Deployed, EG, Execution Group, IIB, iLoveScripting, Integration, Script, Server, Shell Script, Status, WMB /

Now a days, I am dwelling between Development and DevOps team (which is actually a prod support team....shhhh.....don't tell anyone ;-) ) and got to learn a lot about the team, people (employee vs manager), kind of work they are doing. I believe, In this IT era, if you want your application to survive, Teach your Prod Support Team, Let them Learn and have some techy kind of people in it. Okk...now... llots of blabbering about it.... Let's jump on the real stuff ......

So, Being a Dev/DevOps member, I have created some of IIB/WMB Scripts which I want to share with you one by one.... Sharing a script which will give you the Application Details Deployed on particular EG.....

==
= ==

Like the below page to get update
Facebook Page Facebook Group Twitter Feed Google+ Feed Telegram Group

Create Alias Command with Parameter in Linux

by Atul Singh / in Alias, Argument, Command Line, commands, HowTo, iLoveScripting, Linux, Parameter, Script, Shell, Shell Script /

"Alias" - A shortcut for a command or set of commands

Linux OS provides the facility to have alias command to keep the long and frequent command to very few keystrokes, May be I am being a more cryptic with the lots of jargon words, let's try to understand this way -

When you execute a command "ll" (double Ele) in Linux prompt, it will return the result which is the same as "ls -l" because most of the modern linux os have this alias command (ll) for "ls -l" which is quick and easy. You can add your customize command in ".bashrc" (dot bashrc) in case of BASH shell or ".kshrc" in case of KORN shell.

How To:
alias pcs = 'ps -ef' # this will create an alias named pcs which is equivalent to "ps -ef"

As we can see, we can create lots of alias of frequent commands needed in our day to day work, but avoid the alias which modifies/delete system files.

Now, Coming back to the Original Topic, Can we create an alias which accepts the arguments, Answer is YES, lets check it our How To --

How To create ALIAS which accept command line parameter:

Create a script which will accept the argument and do the task, such as -

= =

Create an alias for this script as -

How to use this alias -

==

and voilà !! we are done, Now you can add so many scripts as alias which accept the parameters, Please share here if you have created something useful for you.

Like the below page to get update
Facebook Page Facebook Group Twitter Feed Google+ Feed Telegram Group

Jupyter Lab Env - Quick Start Script

by Atul Singh / in Command Line, Getting Started, iLoveScripting, Jupyter, JupyterLab, Python, Quiz, Script, tips /

Every Data Analyst, who is working in Python, is very well aware of Jupyter or Jupyter Lab.

Sometime it seems little annoying, to lazy person like me, to start the Command Prompt, go to your working code directory and type jupyter command to start the notebook.

To avoid this burden I have wrote a small batch script which will do the task for me in one click. Sharing here the same, Lazy Programmers, please share this post if you like it :-)

For Better View, Click on "View Raw"

Like the below page to get update
https://www.facebook.com/datastage4you
https://twitter.com/datagenx
https://plus.google.com/+AtulSingh0/posts
Join Telegram DataStage4You Group

How to export the datastage job in individual dsx

by Atul Singh / in Batch, Datastage, dsjob, dsx, export, iLoveScripting, InfoSphere, Job, Script, Windows /

This script will export the datastage jobs into individual dsx files as sometimes, we need individual dsx to maintain the code version or syncing it with SVN.

Manually export of each datastage job is quite hectic and boring hence this script :-)

This script requires little bit information from you and set you free for your other chores

Prerequisites for this script -

1. Job List to Export
2. DataStage Client Installation Location
3. DataStage Credential

Like the below page to get update
https://www.facebook.com/datastage4you
https://twitter.com/datagenx
https://plus.google.com/+AtulSingh0/posts
https://datagenx.slack.com/messages/datascience/

Get Job Parameters for all DataStage Job

by Atul Singh / in Command Line, commands, Datastage, dsjob, DSParam, iLoveScripting, Jobs, Parameter, Script, Shell Script /

Feels awesome while writing this post after 7 week, In this post I am going to share a script which will fetch the job parameters for all the datastage job available in given project.

This week, one of my colleague ask to compare all the jobs parameter from one project (older) to another project (migrated), So, wrote down this script which worked like charm for me and hoping will work for you as well :-)

Datastage Calling script in Remote Server

by Atul Singh / in Datastage, DIY, Job, Remote, Script, Sequence, Server, Shell Script, SSH /

Step 1: Setting up UNIX server to automatically login without prompt-ing a password.

1. SSH must be installed in both servers. (primary and remote)
2. User ID for both servers

You can find the Step by Step detail on this Link - Configuring_SSH_on_Linux

Step 2: Creating Datastage job to run script in a remote server

1. Create a new sequencer job
2. Add an Execute Command stage
3. In the Command text value in the ExecCommand tab,

type -

ssh UserB@ServerB ksh /home/b/test.ksh

This command will execute a script in the remote server.

Utility to Get Last Run Log of DataStage Job

by Atul Singh / in commands, Datastage, dsjob, Job, Log, Run, Running, Script, tips, tricks /

Love the new feature of IBM DataStage to fetch the last run log of any jobs with the help of "dsjob" command.
In older version of DataStage, it is very tedious to get the last run log but from/after v9.1 IBM added an additional feature in dsjob command. Lets see hows this works -

To Fetch Last Run Log:
To Fetch Second Last Run Log
To Fetch Third Last Run Log
To Fetch Nth Last Run Log

Get Over Running DataStage Job Details #iLoveScripting

by Atul Singh / in commands, Datastage, dsjob, iLoveScripting, List, Monitor, Script, Shell Script /

With the script below, we will get a list of jobs which are taking more time to complete than last run time. By some tweaks, we can use this script to monitor any kind of process.

Perl Script to get content difference of a file #iLoveScripting

by Atul Singh / in iLoveScripting, perl, Script, Shell Script /

This Perl script will help you to get the content difference of a file available in two directories.

Usage: diffmany.pl filename dir1 dir2

Working: Script will pick the filename from argument and start comparing its content and print the
difference. Here assumption is the same file is available in two directories.
But we can alter this script to take two file name and one directory or based on our need.

View or Download : diffmany.pl

How to get DataStage Project Settings

by Atul Singh / in Administration, Batch, Datastage, Environment, Password, Project, Properties, Script, User, Variable, Windows /

DSListProjectSettings.bat is used to report all available settings of existing projects.

USAGE: DSListProjectSettings <SERVER> <USER> <PASSWORD> <PROJECT>

The DSListProjectSettings batch file reports all available project settings and environment variable names and values to standard output. The first section shows available DataStage project properties and their settings. The second section shows environment variable names and values. When run successfully, each section will report a status code of zero.

PDF Split & Merge with Perl

by Atul Singh / in Administration, perl, Script, Shell Script /

Sharing two utility scripts (Perl), which can be used for extracting only few pages from a PDF file and also for combining different pdf files into a single pdf files. There might be lot of situation where in, we might require only few pages from a big PDF files. So instead of storing / carrying such a big PDF files, we could extract the page or pages which we require from the original big PDF files. Also we could combine different pdf files into a single pdf file using the second script. As an example, if we have extracted pages 10-15 and pages 100-120 from a big pdf file using the pdf_extract.pl and
we can combine these two pdf files (i.e. pdf which contains pages 10-15 and pdf which contains
pages 100-120) into a single pdf file using pdf_merge.pl

NOTE : These two perl scripts uses a perl module called PDF::API2. If this is not present on your system as part of the perl installation, you can download these modules from www.cpan.org and install. Please see the installation section for more details.

These two scripts can be used on windows, unix or linux. Currently tested on Windows with active perl 5.8.8, but it should work on unix and linux as well. For the pdf_extract.pl script to work on unix and linux, please change the variable called "path_separator" to "/" instead of "\\". This variable can be seen at the starting of the script. pdf_merge.pl can be used both on windows and unix/linux without any modification

Usage:

1) pdf_extract.pl

perl pdf_extract.pl -i <input pdf file> -p <page or page range which needs to be extracted>

where
-i : Please give the full path to the input PDF file
-p : Page Number or Page range which needs to be extracted from the input PDF

example : To extract pages 3 to 5, execute

perl pdf_extract.pl -i /tmp/abc.pdf -p 3-5

example : To extract only page 3, execute

perl pdf_extract.pl -i /tmp/abc.pdf -p 3

Executing with -h option will display the usage onto the screen

Example : perl ./pdf_extract.pl -h

2) pdf_merge.pl

perl pdf_merge.pl <output pdf file with full path> <input pdf file 1> <input pdf file 2> etc

Execute the script with all the pdf file which needs to be merged.
Script will merge in the same order which is given in the input

i.e. If you execute like pdf_merge.pl /tmp/out.pdf /tmp/abc.pdf /tmp/xyz.pdf

then pages from xyz.pdf will be after pages from abc.pdf

Executing with -h option will display the usage onto the screen

Example : ./pdf_merge.pl -h

CodeBase:

README File:
pdf_extract.pl
pdf_merge.pl

get Queue Depths #iLoveScripting

by Atul Singh / in depth, iLoveScripting, Message, MQ, Queue, Script, Shell, Shell Script /

Import the jobs from DS windows client #iLoveScripting

by Atul Singh / in Administration, Batch, Client, Datastage, dsjob, export, iLoveScripting, Import, Job, Jobs, Machine, Script, Windows /

As we have discussed a script which can export the datastage jobs from your client system (http://bit.ly/2frNPKj) likewise we can write another one to import the jobs. Let's see how -

DsImportJobsClient.bat :

This Script read all the *.dsx job name from the specified Directory and Sub-Directory and import to the Specified project. It can also build (Only BUILD) the existing package created on Information Server Manager and send it to the specified location on client machine.

To use the build feature you need to make sure the package has been created with all the needed jobs, saved and closed. Only update to the selected job will be taken care automatically. To add/delete a job, you need to do manually.

Modify the Import.properties and ImportJobList.txt file and Go the .bat dir and then execute the importAndBuild.bat.

Import.properties :

ImportJobList.txt :

DsImportJobsClient.bat :

Reading DSParam - datastage parameter file

by Atul Singh / in best practices, Datastage, DSParam, File, Parameter, perl, read, Script, Shell Script, tips, tricks /

I am sharing a utility which can help you to read DSParam file which holds all the environmental datastage parameters.

Utility to view contents of DSParams file. Useful when trying to see what all the customer has set at the project level.

Usage:
$ cat DSParams | ./DSParamReader.pl | more
or
$ cat DSParams | ./DSParamReader.pl > outputfile

Instructions:
1. copy script text below to a file (DSParamReader.pl) on a UNIX system
2. Set execute permissions on this file. chmod 777 envvar.pl
3. Usually perl is in /usr/bin/perl but you might have to adjust this path if neccessary. (hint "which perl" should tell you which one to use)
4. cat the DSParams file from the project you are concerned with and redirect the output to this script. You may have to put the Fully Qualified Path for this file.
5. capture the output to screen or file. File may be useful to have the customer send the info to you in email.

Like the below page to get update
https://www.facebook.com/datastage4you
https://twitter.com/datagenx
https://plus.google.com/+AtulSingh0/posts
https://datagenx.slack.com/messages/datascience/

Export the jobs from DS windows client

by Atul Singh / in Batch, Client, Datastage, dsjob, export, Job, Jobs, Machine, Script, Windows /

Datastage jobs Export/Import are occasional activity (Deployment time :-)) for a developer But it becomes very tedious if the job list are long or it's daily routine to export or import jobs.

So I have written a batch script (windows script) which we can execute from the Client Machine (where datastage clients are installed) and automate this process.

DSjobExportClient.bat :
Export Script read the job name from the file (ExportJobList.txt) and exports the jobName.dsx from the project to the export base location and maintain the folder structure specified in the ExportJobList.txt file. File "ExportJobList.txt" and “Export.properties” should be updated before running the export script.

Copy the above file on any location of your DS windows client machine and update the "ExportJobList.txt" and “Export.properties” files.
-    Export.properties
-    ExportJobList.txt
-    ExportJobs.bat

Export.properties :

ExportJobList.txt :

DsExportJobsClient.bat :

Friday, 21 September 2018

Friday, 14 September 2018

Thursday, 13 September 2018

Wednesday, 5 September 2018

Saturday, 3 February 2018

Wednesday, 20 December 2017

Sunday, 19 November 2017

Monday, 11 September 2017

Friday, 14 July 2017

Saturday, 1 April 2017

Thursday, 9 March 2017

Sunday, 5 March 2017

Tuesday, 21 February 2017

Thursday, 9 February 2017

Wednesday, 7 December 2016

DsImportJobsClient.bat :

Monday, 21 November 2016

Sunday, 6 November 2016

Monday, 12 September 2016

Tuesday, 31 May 2016

Sunday, 7 February 2016