Chaining and Composing Iterators

Chaining and composing iterators allows you to build powerful data pipelines in Python. By linking together simple iterator or generator functions, you can process data step by step, each stage transforming or filtering the data before passing it to the next. This approach is especially useful when you want to apply multiple operations in sequence, such as filtering items and then transforming them, or combining data from several sources. The video above illustrates how you can use generator pipelines to keep your code modular and memory efficient, avoiding the need to create intermediate lists.

To see this in action, consider a scenario where you have a sequence of numbers and you want to filter out the even numbers, then square the remaining numbers. You can achieve this by chaining two generator functions: one for filtering and one for transforming. This approach ensures that each item is processed only as needed, making your code both concise and efficient.

The first function, filter_odds, takes an iterable of numbers and yields only those that are odd. The second function, square_numbers, takes an iterable and yields the square of each number. By passing the output of filter_odds directly to square_numbers, you create a pipeline: first filtering, then transforming the data. The final result is collected into a list and printed, showing the squares of all odd numbers from 0 to 9.

This method is memory efficient because each value is processed one at a time, rather than creating intermediate lists. It also keeps your code modular, as each generator function does one thing and can be reused in different pipelines.