Filtering Inside Comprehensions

There are two filtering-looking constructs in a comprehension, and they do different jobs.

numbers = [1, 2, 3, 4, 5, 6, 7, 8, 9, 10]

# An if clause filters: items that fail are left out entirely
evens = [n for n in numbers if n % 2 == 0]
print(evens)                   # [2, 4, 6, 8, 10]

# Filter AND transform
squares_of_evens = [n * n for n in numbers if n % 2 == 0]
print(squares_of_evens)        # [4, 16, 36, 64, 100]

# A conditional expression changes the value, not the selection
labels = [n if n < 5 else -n for n in numbers]
print(labels)                  # [1, 2, 3, 4, -5, -6, -7, -8, -9, -10]

# More than one if means both must pass
print([n for n in numbers if n % 2 == 0 if n > 5])   # [6, 8, 10]

The if clause sits after the for clause and drops items that do not satisfy it, so the result is shorter than the input. The conditional expression sits before the for clause and runs for every item, choosing one of two values, so the result is the same length as the input. That is the distinction to be able to state clearly.

You can use both together — filter first, then transform the survivors — which is what the squares_of_evens line does. The order in the comprehension is fixed by the syntax: expression, then for clause, then if clauses. The if clauses are evaluated before the expression, so the expression never sees the items that were filtered out.

Two more clauses can be chained, and both must pass. The last line keeps the even numbers that are also greater than 5, giving 6, 8 and 10. There is no or available, so a disjunction needs a single condition built with or — n % 2 == 0 or n % 3 == 0 — which is the usual workaround.

The if clause goes after the for clause and filters. Everything that fails the test is omitted from the result entirely, so the output list is shorter than the input.

The conditional expression goes before the for clause and does not filter anything. It is an ordinary expression that evaluates to one of two values for every item, so the output list is exactly as long as the input. Writing [n if n > 0 else 0 for n in values] keeps every item and replaces the negatives with zeros.

The distinction catches people out because both use the word if. The rule of thumb is position: after the for clause means filter, before it means choose a value.

There is a practical performance difference too. The if clause is evaluated first, so the expression only runs on the items that passed. If the expression is expensive and the filter is cheap, the comprehension is doing the right thing; if the filter is expensive and the expression is cheap, the filter runs on every item either way, so there is nothing to optimise.

Two limits to know. There is no or clause, so a disjunction has to be written as one condition using or. And a comprehension cannot contain statements, so if the filter needs a try block or more than one step, an explicit loop with an if statement is the right shape.