Skip to Content

Comprehensions & Generators

When to use

Build a new collection by transforming/filtering another; or stream one element at a time.

Analogy

Comprehension is a factory line; generator writes each item to a delivery truck one at a time, freeing the floor.

Data-flow diagram

[ f(x) for x in src if cond ]    -> list (built all at once)
( f(x) for x in src if cond )    -> generator (lazy)
  next(g) -> 2, next(g) -> 4, ...

Deep explanation

Comprehensions are syntactic sugar for a for-loop that builds list/set/dict; they run in C, typically 30-50% faster. Generators are lazy: they yield a value per next() call and pause. Use comprehensions when you need the materialised result (length, indexing); generators when data is large or unbounded.

Examples

Example 1

squares = [x*x for x in range(10)]
even_sq = {x: x*x for x in range(10) if x%2==0}

list comp returns a list; dict comp with if filters.

Example 2

def count_up(n):
    i = 0
    while i < n:
        yield i; i += 1
for x in count_up(3): print(x)   # 0,1,2

generator with yield; state preserved between next() calls.

Example 3

import sys
big = (x for x in range(10**9))
print(sys.getsizeof(big))  # 200 bytes, not 8GB

generators hold NO accumulated state - materialises only on list(gen).

Common mistake

Calling len(gen) on a generator raises TypeError. Re-using a generator after its first full iteration (generators are single-pass). Be careful with side effects inside comprehensions - they’re hard to debug.

Key takeaway

comprehensions for materialised collections; generator expressions for streaming. sum(x*x for x in range(10**6)) runs in O(1) memory.

Production Failure Playbook

Failure scenario 1: comprehension-shadowed-variable

Failure scenario 2: generator-exhausted-mid-pipeline