Conditions inside a comprehension come in two different forms, and confusing them is the source of most comprehension bugs. An if clause after the for filters the items. A conditional expression before the for transforms them. Read carefully and the two never overlap.
The previous lesson in this module covered "List Comprehensions: The One-Line Transform", and the next lesson in this module is about "Dict and Set Comprehensions in Practice".
How do you filter with an if clause?
The if clause goes after the for clause. Items that fail the test are dropped completely, and the output expression is never worked out for them.
values = [-3, 5, -1, 8, 0]
positives = [v for v in values if v > 0]
print(positives)
squares = [v * v for v in values if v > 0]
print(squares)
That order is worth stating plainly. The filter runs first, and the expression runs only on what survives. The second line works out v * v for three values and never touches the negative ones, and that is also why it cannot raise an error on them.
The test can be any expression that gives you true or false, and that includes a function call. That is how you filter on something that you have to work out rather than read directly.
Filtering is not a separate step here, because it is part of the same expression. There is no intermediate list of survivors sitting in memory between the filter and the transformation.
def is_even(n):
return n % 2 == 0
numbers = [1, 2, 3, 4, 5, 6]
print([n for n in numbers if is_even(n)])
print([n for n in numbers if n > 2 and is_even(n)])
What is the difference between an if clause and a conditional expression?
A conditional expression has the form value_if_true if condition else value_if_false. It sits at the front, where the output expression goes, and it chooses between two values instead of removing items.
values = [-3, 5, -1, 8, 0]
absolute = [v if v > 0 else -v for v in values]
print(absolute)
filtered = [-v for v in values if v > 0]
print(filtered)
Look at the difference in the outputs. The conditional expression keeps every item and changes the negative ones to positive, so that list has five elements. The if clause keeps only the positive items and then negates them, so that list has two. The same words, in different slots, produce completely different results.
The conditional expression is not special to comprehensions. It is an ordinary expression that you can use anywhere a value is expected, which makes it useful well beyond this syntax and makes x if condition else y a good habit to recognise wherever you see it.
Two mistakes follow from all this. Putting the condition in the wrong slot changes the length of your result, and that is usually the symptom that you notice first. And writing if where else should be, with nothing after it, is a syntax error, because a conditional expression needs both of its branches.
source = 'values = [v if v > 0 else for v in values]'
try:
compile(source, '', 'exec')
except SyntaxError as error:
print('SyntaxError:', error.msg)
How do you combine multiple conditions?
Join them with and and or inside the single if clause. With and, every condition has to be true for the item to survive.
numbers = [1, 2, 3, 4, 5, 6, 7, 8]
print([n for n in numbers if n % 2 == 0 and n > 3])
print([n for n in numbers if n in (2, 4, 8)])
print([n for n in numbers if n < 3 or n > 6])
Use parentheses when you mix and and or, because and binds more tightly than or. That is the same precedence rule that you use everywhere else in Python, but inside a comprehension it is easy to miss, because the surrounding brackets belong to the list and not to the condition.
A bare if x is a filter on truthiness, and that is how you drop empty strings, zeros, and empty containers in one go. It is short, and it can hide what you meant, so use it when truthiness really is what you mean.
mixed = ['hello', '', 'python', None, 'code']
print([text for text in mixed if text])
print([text for text in mixed if text is not None])
You can also write more than one if clause, and the effect is the same as joining them with and. The single-clause form reads better in almost every case.
When should you use any() and all() instead of building a list?
When the question is a yes-or-no one. If you only need to know whether at least one item matches, then any() stops at the first match instead of building a whole list for you to inspect afterwards.
numbers = [1, 2, 3, 4]
print(any(n > 3 for n in numbers))
print(all(n > 0 for n in numbers))
print([n for n in numbers if n > 3])
The parentheses hold a generator expression, which produces values one at a time rather than building a list up front. That laziness is what allows any() and all() to stop early, and it also keeps memory flat when you are scanning a large collection.
One edge case catches people out. all() on an empty sequence returns True, because there is no item that fails the test. any() on an empty sequence returns False. If an empty collection should be treated as a failure, then check the length explicitly.
print(all([]))
print(any([]))
values = [1, 2, 3]
print(all(v > 0 for v in values))
The same generator expression works with filter() and map(), but the comprehension is usually easier to read. Reach for filter() when you are combining it with other functions in a chain, and reach for the comprehension when readability is what matters.
What can the walrus operator do inside a comprehension?
Python 3.8 added the assignment expression :=, which binds a name and hands back the value in one go. Inside a comprehension it lets you work out something once and use it in both the test and the output expression.
data = [1, -2, 3]
doubled = [y for x in data if (y := x * 2) > 0]
print(doubled)
Without the walrus you would either work out x * 2 twice or reach for a helper function. The walrus is a real shortcut here, and it comes with two restrictions that are worth knowing before you meet them in a traceback.
try:
compile('[y for x in (y := [1, 2, 3])]', '', 'exec')
except SyntaxError as error:
print('SyntaxError:', error.msg)
First, an assignment expression cannot appear in the iterable part of a comprehension, because that part is worked out before the scope of the comprehension exists. Second, it cannot be used inside a comprehension in a class body, because the name would leak into the class namespace in a confusing way. Both of them are compile-time errors, so you find out immediately instead of at runtime.
One more consequence is worth keeping in mind. The name that a walrus binds lives in the enclosing scope and not inside the comprehension, so after the comprehension finishes the name is still there.
Filtering recipes
| Goal | Code |
|---|---|
| Keep the matching items | [x for x in items if test(x)] |
| Transform and keep everything | [f(x) if cond else g(x) for x in items] |
| Transform and filter | [f(x) for x in items if test(x)] |
| Two tests | [x for x in items if a(x) and b(x)] |
| At least one match | any(test(x) for x in items) |
| Every item matches | all(test(x) for x in items) |
| Work out once and use twice | [y for x in items if (y := f(x))] |
Now try it yourself. Take a list of numbers, keep only the even ones, and square them, all in one comprehension. Then write the same thing with a conditional expression, and notice that the length changes. Next, check whether any number is negative and whether all of them are under 100, using any and all instead of building a list. Finally, use a walrus to work out each number's absolute value once and to filter on it, and then confirm by printing the bound name afterwards that it survived outside the comprehension.