Project Architecture in DataStage

In general, we have 4  Architecture

• Simple
• ODS  - Operations Data source
• NDS – Normal  Data source
• Integrity  

we can implement this Architecture either in Inmon Model or Kimball Model

Top to bottom                                Bottom to top

 

Universally  -   8 Architecture

Inmon – Simple, ODS,  NDS, Integrity

Kimbell - Simple, ODS,  NDS, Integrity.

Simple

SRC ---- WH

 ODS

 

• Inmon Models are preferred when Requirements are very clear.
• Kim bell Model is preferred when   Requirements are not clear.  
• For the project, we deal with

1. ODS --à80% ODS Model
2. NDS

WHY ODS?

Sol:- when  Business Requirements are defined, click/customer requests the team  what we want exactly

.    

Project Architecture depends on no. of operational and Analysis Reports.

• More priority is given to operational and Analysis Reports
• Are we have more operational Reports we use ODS.
• Less operational Reports and more Analytical Reports, we use NDS.
• In NDS, we can’t See ODS, But in ODS, we can find warehouse and Data marts (NDS).
• ODS gets refreshed every 10 seconds.
• ODS dim Table will always be type 1 (No History)

     

Architecture

 

Data Acquisition 

Pull   You get  (take) the data

Push he sends the data

DRA:-  Data Repository Area

CSA:- Common Staging Ares

DRA:- Will get only files  -    Memory which is accessed by different departments.

Data Query:-   When the  data is being such  what the data should contain

Quality Gates:-  Requirements .

Every file should have a specific file name:-

Script 

To validate the files according to Quality  Gates.

Inclined to build a profession as DataStage Developer? Then here is the blog post on, explore Datastage Training

Loading

Loading, Stage, and warehouse is a Single Data stage.

LoadingStaging  ODS Warehousing  Box size of 100 Tear loading  

10 TB  loading

30 TB Staging

60 TB Warehouse.  

Dumping the data from DRA to loading à a copy of data

   

When we load Data to Target data, We maintain all details, that is sequence no;

For an in-depth understanding of DataStage click on

• Introduction to DataStage
• Architecture of Data Stage
• Oracle Enterprise in Data Stage
• SCD(Slow changing Dimension) in Data Stage
• DataStage Tutorials