A list comprehension in Python takes a sequence, transforms each item, and collects the results into a new list, all inside one expression that says exactly what it does. The syntax looks like a loop that has been squeezed into a single line, and once you can read it from left to right, it is easier to follow than the loop that it replaces.
This is the first lesson in the List Comprehensions in Python module, and the next lesson in this module is about "Filtering and Transforming in One Comprehension".
What is the basic shape of a list comprehension?
Square brackets, an expression, the word for, a target, the word in, and an iterable. That is the whole grammar.
squares = [n * n for n in range(5)]
print(squares)
uppercase = [name.upper() for name in ['ada', 'grace', 'alan']]
print(uppercase)
lengths = [len(word) for word in ['cat', 'otter', 'elephant']]
print(lengths)
The expression on the left is what each item becomes, and the right-hand side is the source. Read the second line out loud as "the uppercase of each name in this list", and the code stops looking like a puzzle.
Here is the same thing written as a loop, which is worth seeing once so that you recognise what the comprehension replaced.
uppercase = []
for name in ['ada', 'grace', 'alan']:
uppercase.append(name.upper())
print(uppercase)
Three lines became one, and the intermediate list variable disappeared. That last part is the real win. The comprehension produces the finished list directly, so there is nothing to initialise and nothing to append to.
Two properties carry over from the loop. The expression is worked out once for each item, in the order that the iterable produces them, so the result keeps the order of the source. And the comprehension always builds a brand new list, so the original sequence is never modified.
How do you read a comprehension out loud?
Read it as "the expression, for each item, taken from the iterable". Once you can say that, the longer versions become readable as well. A comprehension with a condition is "the expression, for each item that passes the test, taken from the iterable".
evens = [n for n in range(10) if n % 2 == 0]
print(evens)
words = [' hello ', ' world ', 'python ']
cleaned = [word.strip() for word in words if word]
print(cleaned)
When a comprehension grows past one line, format it across several lines inside parentheses. The newlines are free, and the result is much easier to read than one long line.
prices = [19.99, 4.5, 120.0, 7.25]
affordable = [
price
for price in prices
if price < 50
]
print(affordable)
What is the scope of a comprehension?
In Python 3, a comprehension runs in its own scope and not in the scope of the code around it. The loop variable therefore disappears when the comprehension ends, and that is a real difference from the equivalent for loop.
def build():
values = [1, 2, 3]
return [v * 2 for v in values]
print(build())
try:
print(v)
except NameError as error:
print('NameError:', error)
Two details follow from that. The source list is still in scope, because values is found in the enclosing function, so a comprehension can read the variables around it. And a name that is used inside the body of the comprehension resolves outward through the enclosing function, then the module, and then the built-ins, so it can read them but it cannot change them without the nonlocal or global keywords.
This matters when you shadow a name. Inside a comprehension, the loop variable shadows any outer variable of the same name for the length of the comprehension, and that outer variable is untouched afterwards.
factor = 2
def outer():
factor = 10
return [factor * v for v in [1, 2, 3]]
print(outer())
print('module level factor is still:', factor)
How do you nest for clauses?
You add another for clause. The clauses run from left to right, so the rightmost one is the innermost loop, and the expression at the front sees the innermost variables.
pairs = [(a, b) for a in [1, 2] for b in ['x', 'y']]
print(pairs)
matrix = [[1, 2], [3, 4]]
flat = [number for row in matrix for number in row]
print(flat)
The flattening example is the one to recognise. A list comprehension reads across a nested structure without any indexing, and that is why it replaced a decade of nested for loops with append calls.
Be careful with the order when the clauses involve different lengths. The right-hand clause iterates fastest, so the result is grouped by the left-hand variable. Swap the two clauses and the grouping changes with them.
What can you build a comprehension from?
Any iterable. A range, a list, a tuple, the keys of a dictionary, the characters of a string, or a generator that produces items one at a time.
data = {'a': 1, 'b': 2}
print([key.upper() for key in data])
print([char for char in 'abc'])
print([n for n in range(3)])
print([value * 2 for value in data.values()])
Reading lines from a file is the most common real use, and it has a subtlety. A comprehension over open(...) leaves the file handle open until the garbage collector gets round to it, which on some systems means that the file stays locked.
lines = ['alpha\n', 'beta\n', 'gamma\n']
print([line.strip() for line in lines])
For real files, put the comprehension inside a with block so that the handle is closed as soon as the data has been read. That detail belongs to the files module, but the habit of reaching for with around any file access is worth forming now.
When should you write a comprehension over a loop?
When the loop does exactly one thing, which is to take each item, work out a value, and collect it. That is the shape that a comprehension was designed for, and replacing such a loop is usually a clear improvement.
Keep the plain loop when the body does anything else. Keep it when the body prints, when it accumulates into more than one variable, when it breaks out early, or when it needs a comment that explains three different steps. A comprehension cannot hold a comment inside its body, so any explanation that you would have written there has to live outside it or in a helper function.
There is also a performance point worth knowing, although it rarely decides the question. A comprehension is usually a little faster than the equivalent loop, because the interpreter handles the append internally instead of performing a method lookup and a call on every iteration. That difference is small in absolute terms, and a readable loop will beat an unreadable comprehension every single time.
Comprehension recipe
| You want | Write |
|---|---|
| Transform every item | [f(x) for x in items] |
| Keep only the matching items | [f(x) for x in items if test(x)] |
| Transform and filter together | [f(x) for x in items if test(x)] |
| Flatten one level | [v for row in rows for v in row] |
| Build pairs | [(a, b) for a in first for b in second] |
| Turn a loop into a list | list(generator) |
Now try it yourself. Write a list of ten prices as strings, and then build a list of floats from them with a comprehension. Filter that list down to the values under 50 and round each one to two decimal places, all in a single comprehension. Next, flatten a list of three lists into one list, and check that the length matches the sum of the original lengths. Finally, take a loop that prints each item and convert it into a comprehension, and then convert it back into a loop, and write down which of the two versions you would rather read.