Understanding and Tuning the Data Flow Engine
What’s In This Topic ?
- Understanding the Control Flow and Data Flow
- Learning the Data Flow architecture and transformation types
- Designing and tuning the Data Flow
- Troubleshooting Data Flow performance
This Topic focuses on how the Data Flow engine works, because as you seek to expand your knowledge and skills with SSIS, you will need to understand how to best leverage and tune your packages and Data Flows. The Topic begins with a consideration of the architecture of the engine and its components, and then describes best practices for design and optimization, including the following concepts:
- Control Flow and Data Flow comparison
- Data Flow Transformation types
- Data Flow buffer architecture and execution trees
- Data Flow execution monitoring
- Data Flow design practices
- Data Flow engine tuning
- Performance monitoring
The initial part of this Topic is more abstract and theoretical, but we’ll then move into the practical and tangible. In the concluding sections, you will apply the knowledge you have developed here, considering a methodology for optimization and looking at a few real-world scenarios.
Some of you will have worked with a previous edition of SSIS; for others, this will be your first time working with the tool. In many ways, each new version of SSIS has made improvements to the Data Flow architecture, adding more scalability and performance and fine-tuning the performance of the Data Flow engine.
For those of you with some knowledge of the SSIS pipeline architecture, two important features for the Data Flow engine are backpressure management and active component logging in the SSIS Server. These features provide scalability and advanced insight into the Data Flow. But let’s begin with the basics of the SSIS engine.