My e-Notes about Cloud, K8s, OpenShift, DataScience, Machine Learning, Python, Data Analytics, DataStage, DWH and ETL Concepts

Breaking

Friday, 4 December 2020

5 Tips For Better DataStage Design #18

1. Use Database Connector Stages wherever possible
    - Older “Enterprise” stages must not be used
    - ODBC stages must not be used when Connectors are available 
    - Transformer & Basic Transformer Stage Usage
    - Transformer usage should be minimized (they are CPU intensive)
        * If other stages such as Copy can accomplish the same task use those instead
    - Basic Transformers must be removed (they don't work in Grid)

2. Lookup vs. Join 

    - If reference data is small  and source data is large Lookup should be used
    - If reference data is large and source data is small Join should be used
    - If both reference data and source data are small Join should be used

3. Non-sequence jobs should not have more than 15 stages.

4. “BULK” Mode should be used in Teradata Loads
    - Specify TENACY and SLEEP parameters if needed to prevent failed loads
    - Specify “Yes” to Parallel synchronization
    - NOTE: Teradata info may be refined based on feedback from EDW and Teradata teams

5. Oracle Connectors must read/write in parallel
    - Execution mode on Advanced tab must be set to “parallel”
    NOTE:  Oracle info may be refined based on FPD Oracle testing in Grid

6. DB2 Connectors must read/write in parallel   
    - Enable partitioned reads must be set to yes



Like the below page to get update  
https://www.facebook.com/datastage4you
https://twitter.com/datagenx
https://plus.google.com/+AtulSingh0/posts
https://datagenx.slack.com/messages/datascience/

No comments:

Post a comment

Disclaimer

The postings on this site are my own and don't necessarily represent IBM's or other companies positions, strategies or opinions. All content provided on this blog is for informational purposes and knowledge sharing only.
The owner of this blog makes no representations as to the accuracy or completeness of any information on this site or found by following any link on this site. The owner will not be liable for any errors or omissions in this information nor for the availability of this information. The owner will not be liable for any losses, injuries, or damages from the display or use of his information.