Showing posts with label Filter. Show all posts
Showing posts with label Filter. Show all posts

Wednesday, 23 December 2015

5 Tips For Better DataStage Design #6



#1. If you are using a copy or a filter stage either immediately after or immediately before a transformer stage, you are reducing the efficiency by using more stages because a transformer does the job of both copy stage as well as a filter stage

#2. Work done by "COPY Stage"
a) Columns order can be altered.
b) And columns can be dropped.
c) We can change the column names.



#3. When you need to run the same sequence of jobs again and again, better create a sequencer with all the jobs that you need to run. Running this sequencer will run all the jobs. You can provide the sequence as per your requirement.

#4. Sort the data as much as possible in DB and reduced the use of DS-Sort for better performance of jobs. Avoid the work done by DataStage which is possible in DB. But it doesn't mean you have to put all the complexity in SQL only, for that we are using datastage.

#5. Ensure that all the character fields are trimmed before any processing. Normally extra spaces in the data may lead to some errors like lookup mismatch which are hard to detect.





Like the below page to get update  
https://www.facebook.com/datastage4you
https://twitter.com/datagenx
https://plus.google.com/+AtulSingh0/posts
https://groups.google.com/forum/#!forum/datagenx

Friday, 11 September 2015

Linux Shell Script Scenario #3




Design a Shell Script which have to work for below scenario :-


We are having two files, File1 is having a 20,000 lines of data and File2 is having few lines, Now the requirement is - Read the File2 and remove the line from File1 which is having those word.

Suppose -  

File1 ---
TCS, Tata Consultancy Services
IBM, International Business Machine
TD, Teradata
IM, Information Management
WM, WareHouse Management
WC, Work Center


File2
IBM
WM
TCS


Output :
TD, Teradata
IM, Information Management
WC, Work Center


For more scenario -  CLICK HERE