Comprehensions shine when a single expression turns an iterable into a new collection. They turn awkward when that expression starts hiding real work, such as a nested loop, a side effect, an early exit, or a function call for every element. This lesson is about recognising the moment a plain for loop becomes the clearer and the faster tool.
The previous lesson in this module covered "Dict and Set Comprehensions in Practice", and this lesson is the last one in the List Comprehensions in Python module.
When does a comprehension stop being clearer?
A comprehension works well when three things are true. You are turning each element into exactly one new value, the transformation is a single expression, and you do not need to stop halfway. The moment one of those breaks, the syntax starts costing you more than it saves. Reading [f(x) for x in data] is instant. Reading [f(x) for x in data if g(x) and h(x) > 0] takes a beat, and reading a nested comprehension over two iterables takes a beat and a half.
The practical rule is short. A comprehension should fit in your head as a single sentence: "for each item, keep the ones that pass this test, and turn them into that value". When you need more than one clause of that sentence, then write the loop.
When a nested loop hides inside a comprehension
The clearest example of a comprehension doing too much is a nested one. Here is a nested comprehension that flattens pairs of coordinates into a list of "ax,by" strings, followed by the version that most readers find easier to maintain.
# A nested comprehension does three things on one line.
labels = [f'{a}x{b}y' for a in (1, 2) for b in (10, 20)]
print(labels)
# An explicit nested loop shows the same result, one step at a time.
labels = []
for a in (1, 2):
for b in (10, 20):
labels.append(f'{a}x{b}y')
print(labels)
Both of them print ['1x10y', '1x20y', '2x10y', '2x20y']. The loop version is longer, and it is still the better tool the moment you need to skip a pair, count the matches, or break out early. A comprehension has no break, so if you have ever wanted to stop halfway, you have already left comprehension territory.
Side effects do not belong in a comprehension
Some code ends up doing work inside a comprehension that it should have done outside. A validation function that also logs, or a transform that also writes to a cache, turns a pure transformation into something with side effects, and that side effect now runs once per element without anybody seeing it.
seen = []
def normalise_and_record(value):
cleaned = value.strip().lower()
seen.append(cleaned) # a side effect hidden inside a transform
return cleaned
names = [' Ada ', 'GRACE', ' Alan ']
cleaned = [normalise_and_record(name) for name in names]
print(cleaned)
print(seen)
It works, and it prints ['ada', 'grace', 'alan'] followed by the recorded list. But now every reader has to know what normalise_and_record secretly does before they can reason about the comprehension. When a transform has a side effect, write the loop and make that side effect explicit.
map() and filter() are not automatically faster
It is a common belief that map() and filter() are faster than comprehensions, because they look more low-level. With a built-in function they can be, because the call happens in C. With a function that is written in Python, map() adds a wrapper call for every element, and a comprehension that inlines the same work is usually quicker. The difference is small, so measure it instead of guessing.
from timeit import timeit
data = list(range(100000))
def double(value):
return value * 2
print('map with a Python function:',
timeit(lambda: list(map(double, data)), number=5))
print('comprehension: ',
timeit(lambda: [double(value) for value in data], number=5))
print('comprehension inlined: ',
timeit(lambda: [value * 2 for value in data], number=5))
Run it on your own machine and compare the three numbers. The inlined comprehension is the fastest of the three on any modern CPython, and the gap between map and the comprehension is the price of the extra function call. Neither difference is large enough to matter for a list of a hundred items, and that is exactly why the readability argument usually wins.
When you need to stop early
Search code is where comprehensions most often mislead you. Building a whole list just to look at the first match does work that you did not need. Generator expressions with any() and all() stop early, which means that they stop as soon as the answer is known.
numbers = [3, 7, 11, 4, 18, 25]
# A list comprehension works out every element and then you look at the result.
big = [n for n in numbers if n > 10]
print(big)
# A generator with any() stops at the first match.
print(any(n > 10 for n in numbers))
# A generator with all() stops at the first failure.
print(all(n % 2 == 0 for n in [2, 4, 6, 7, 8]))
The second line prints True without checking the remaining numbers, and the third one prints False as soon as it reaches 7. If you only need a yes-or-no answer, then the generator form is both clearer and cheaper.
Picking the right tool
Here is the decision that you can reuse. One transformation, one expression, no early exit, and no side effects means you should write the comprehension. Anything more than that, such as multiple statements, a nested loop, a break, a side effect, or a condition worth explaining, means you should write the loop. A loop is not the outdated option. It is the option with more room to say what you mean. Reach for the comprehension when it makes the code shorter and clearer at the same time, which is rarer than most tutorials suggest.
Quick reference
- Comprehension: one expression per element, with no branching, no side effects, and no early exit.
- Loop: multiple statements,
breakorcontinue, counters, and accumulating results. - Generator expression: when you want lazy evaluation, short-circuiting, or a stream of values.
any()andall(): when the answer is a boolean and not a list.- When in doubt, write the loop first. It is easy to convert a loop into a comprehension afterwards, and the reverse is not true.
Now try it yourself. Take the nested coordinate comprehension from the first section and rewrite it so that it skips any pair where a == b, and print how many pairs were kept. Then convert the same problem to a comprehension that does not skip anything, and compare the two outputs. Finally, time both versions with timeit over 100,000 pairs, and note the difference.