Use yield to create memory-friendly, lazy evaluation generators suitable for handling large files, infinite sequences, and data pipelines; 2. yield from simplifies delegating to another generator, reduces redundant code and improves readability, suitable for recursive traversals (such as tree structures) and generator combinations; 3. The generator is used in a single time and should not be mixed with return. It is recommended to combine itertools for advanced control, ultimately achieving efficient and elegant data stream processing.

Python's yield and yield from are powerful tools for writing efficient, readable, and memory-friendly code—especially when working with large or infinite sequences. They unlock the ability to create generators , a type of iterator that produces items on demand, rather than storing everything in memory upfront.

Let's break down how to use them effectively in real-world scenarios.

What Is a Generator and Why Use yield ?

A generator is a function that returns an iterator, producing a sequence of values one at a time using yield instead of return . When yield is hit, the function pauses, saves its state, and resumes from where it left off on the next iteration.

 def count_up_to(max):
    count = 1
    While count <= max:
        yield count
        count = 1

# Usage
for num in count_up_to(5):
    print(num)

This prints:

 1
2
3
4
5

Why this matters:

• Memory efficient: Only one value exists in memory at a time.
• Lazy evaluation: Values are generated only when needed.
• Clean syntax: Looks like a regular function, but behaves like an iterator.

Use yield when you're generating a sequence that's expensive to compute or potentially infinite—like reading large files, streaming data, or mathematical sequences.

Practical Use Cases for yield

1. Processing Large Files

Reading a huge log file all at once can exhaust memory. Instead, yield lines one by one:

 def read_large_file(file_path):
    with open(file_path, 'r') as file:
        for line in file:
            yield line.strip()

Now you can process millions of lines without loading them all.

2. Infinite Sequences

Generate Fibonacci numbers forever (or until you stop iterating):

 def fibonacci():
    a, b = 0, 1
    While True:
        yield a
        a, b = b, ab

# First 10 Fibonacci numbers
fib = fibonacci()
for _ in range(10):
    print(next(fib))

3. Data Pipelines

Chain generators to build efficient processing pipelines:

 def numbers():
    for i in range(100):
        yield i

def even_only(nums):
    for n in nums:
        if n % 2 == 0:
            yield n

def squared(nums):
    for n in nums:
        yield n ** 2

# Chain them
result = squared(even_only(numbers()))
for val in result:
    print(val)

Each step processes data lazily—no intermediate lists created.

Simplifying Delegation with yield from

Before Python 3.3, if you wanted to yield all values from another generator inside a generator, you had to loop manually:

 def wrapper_old_style():
    for value in count_up_to(3):
        yield value
    for value in fibonacci():
        yield value

Now, yield from does this cleanly:

 def wrapper():
    yield from count_up_to(3)
    yield from fibonacci()

It delegates to another iterable or generator, yielding each value in turn.

Key Benefits of yield from :

• Reduces boilerplate code.
• Preserves the generator protocol (eg, handles .send() , .throw() , .close() ).
• Makes nested generators more readable and maintainedable.

Real-World Example: Tree Traversal

Suppose you're traversing a binary tree and want to yield all values in order:

 class Node:
    def __init__(self, value, left=None, right=None):
        self.value = value
        self.left = left
        self.right = right

    def inorder(self):
        if self.left:
            yield from self.left.inorder() # Delegate to left subtree
        yield self.value
        if self.right:
            yield from self.right.inorder() # Delegate to right subtree

This keeps the recursive structure clean and memory-efficient.

Common Pitfalls and Tips

• Don't mix return and yield carefully : In Python versions before 3.3, return in a generator couldn't have a value. Now, return value sets the StopIteration.value , but it's rarely needed.

• Generators are single-use : Once exhausted, they don't reset. If you need to reuse, wrap the generator in a class or re-call the function.

• Use itertools with generators : Combine with tools like islice , chain , or tee for advanced control.

Example: Get first 5 even Fibonacci numbers

 from itertools import islice

fib_evens = (n for n in fibonacci() if n % 2 == 0)
for num in islice(fib_evens, 5):
    print(num)

Final Thoughts

yield turns functions into powerful, lazy data producers. yield from making composing them effortless. Together, they help you write code that's not only efficient but also elegant and easy to reason about.

Use them when:

• You're dealing with large or infinite data.
• You want to decouple data generation from consumption.
• You're building data pipelines or recursive traversals.

They might feel unusual at first, but once you get used to thinking in streams, you'll find yourself reaching for generators more often than lists.

Basically, if you're building something that “produces a bunch of things,” ask: Should this be a generator? More often than not, the answer is yes.

The above is the detailed content of Harnessing Generators: A Practical Guide to `yield` and `yield from`. For more information, please follow other related articles on the PHP Chinese website!