What Are Generator Expressions?

The syntax is almost identical — square brackets versus round ones — but the object you get is completely different.

squares_list = [n * n for n in range(5)]
squares_gen = (n * n for n in range(5))

print(type(squares_list).__name__)     # list
print(type(squares_gen).__name__)      # generator

print(squares_list)          # [0, 1, 4, 9, 16]
print(list(squares_gen))      # [0, 1, 4, 9, 16]

# The generator is now exhausted
print(list(squares_gen))      # []

The list comprehension evaluates all five values immediately and stores them. The generator expression creates a generator object that holds nothing but the range and a position, and computes each square only when it is asked for. list on the generator then drains it and produces the same five values.

The last line is the second important difference. The generator is single-use, so once it has been drained, listing it again gives an empty list. The list comprehension, by contrast, can be traversed as many times as you like.

A third difference is that the generator cannot be indexed, has no length, and cannot be sorted or reversed. Any of those operations would need to consume the whole generator, and once consumed there is nothing left.

One syntax note: a generator expression must always be parenthesised when it is the sole argument to a function, because the call parentheses would otherwise be ambiguous. That is why you see sum(n * n for n in range(5)) with the inner parentheses, and list(...) with the brackets making the outer grouping explicit.

A generator expression is an expression that creates a generator instead of a list. It has exactly the same shape as a list comprehension — an expression, a for clause and an optional if clause — but it uses round brackets instead of square ones. Written out, it is the shorthand for calling a generator function that yields each result.

The difference is when the work happens. A list comprehension runs the whole thing immediately and returns a finished list. A generator expression returns a generator object that has not computed anything yet, and computes each value when it is pulled.

Three practical consequences follow. Memory: the generator holds one value at a time, so it can represent a sequence far larger than would fit in memory. Reuse: the generator is single-use, so once drained it stays empty, whereas a list can be traversed repeatedly. And the interface: a generator has no len, no indexing and no slicing, because producing the tenth item might require producing the first nine.

So the choice comes down to what you are going to do with the result. A generator expression is right when the values are consumed once, in a single pass, as they are produced — passing it to sum, to a database bulk insert, or to a function that streams them. A list comprehension is right when you need the values all at once, need to iterate more than once, or want to index and slice the result.

The one trap is converting it with list, which throws away the entire benefit. If you end up materialising the generator, you have paid for the laziness and got nothing back.