My e-Notes about DataScience, Machine Learning, Python, Data Analytics, DataStage, DWH and ETL Concepts

Breaking

Tuesday, 15 March 2016

yet another Shell script to access DataStage job logs



Introduction


We have various data warehouses hosted on AIX/Linux/Unix operating systems. DataStage Director is one of the ETL monitoring & scheduling tool used in numerous data warehouses. In case of ETL job failures, we need to login to DataStage Director and check the log for error messages. The ETL job logs can tend to span into several pages with various types of informative messages. Usually we need to locate the error message under ‘Fatal’ message type. Doing this task by searching the DataStage log manually can be very time-consuming and tiring.

This shell script facilitates the user to access the error messages from the ease of the  AIX/Linux screen. The script also has a facility to email the filtered log messages to the user’s mailbox.


Tool Logic 


Accept jobname and other parameters while script execution.
Establish proper environment settings for local DataStage use
Locate event ids for fatal errors in the ETL joblog.
Extract detail log for fatal errors
Mail the filtered job log with the exact fatal error message to the user.


Usage 

getDSjob







Like the below page to get update  
https://www.facebook.com/datastage4you
https://twitter.com/datagenx
https://plus.google.com/+AtulSingh0/posts
https://datagenx.slack.com/messages/datascience/

No comments:

Post a comment

Disclaimer

The postings on this site are my own and don't necessarily represent IBM's or other companies positions, strategies or opinions. All content provided on this blog is for informational purposes and knowledge sharing only.
The owner of this blog makes no representations as to the accuracy or completeness of any information on this site or found by following any link on this site. The owner will not be liable for any errors or omissions in this information nor for the availability of this information. The owner will not be liable for any losses, injuries, or damages from the display or use of his information.