Iterating dictionaries in Python has one surprise that catches everybody the first time. for key in my_dict gives you the keys and not the pairs. Once that is clear, keys(), values(), and items() become three obvious tools instead of three things to memorise.
The previous lesson in this module covered "dict.get() vs Direct Indexing: Avoiding KeyError", and the next lesson in this module is about "Sets: Uniqueness, Membership and Set Math".
What does for key in my_dict actually iterate?
It iterates the keys. A dictionary is not a sequence of key-value pairs, so Python walks its keys, and it walks them in the order in which they were inserted.
scores = {'ada': 91, 'grace': 88, 'alan': 95}
for name in scores:
print(name)
for name in scores:
print(name, '->', scores[name])
The second loop works, but it looks the value up with a second dictionary access every time. That is a wasted lookup whenever the value is going to be used, and it is the reason that items() exists.
One related thing to know is that a dictionary is not a sequence, so there is no d[0]. Attempting it raises KeyError: 0, because 0 is a perfectly valid key that simply is not present. If you want to reach an entry by position, then convert the keys to a list first and index that list.
What do keys(), values(), and items() actually return?
Three different views over the same dictionary, and each one shows a different part of it.
scores = {'ada': 91, 'grace': 88, 'alan': 95}
print(list(scores.keys()))
print(list(scores.values()))
print(list(scores.items()))
They are called views because they are not lists. A view is a live window onto the dictionary. It holds a reference to the dictionary rather than a snapshot of its contents, and the list(...) calls above are there to make the output readable.
That distinction matters in one practical way. If you write keys = d.keys() and then add an entry to d, then keys is not stale, because it now includes the new key. Code that assumes a snapshot behaves differently from code that assumes a window, so decide which one you mean.
How do you iterate key-value pairs together?
Use items() and unpack each pair. The key and the value arrive as two separate names, which is exactly what you want for readable code.
scores = {'ada': 91, 'grace': 88, 'alan': 95}
for name, score in scores.items():
print(name, '->', score)
print({name: score + 5 for name, score in scores.items()})
Without unpacking, items() gives you tuples, and you would have to index into each one. Writing for pair in scores.items(): and then pair[0] and pair[1] works, but it throws away the one thing that makes Python readable, which is meaningful names.
When you only need the values, values() avoids the key lookup entirely, and it is the cheapest of the three. When you only need the keys, keys() is the same as iterating the dictionary directly.
scores = {'ada': 91, 'grace': 88, 'alan': 95}
for score in scores.values():
print(score)
print(sum(scores.values()))
print(max(scores.values()))
Choosing between them is a question of what your loop body needs. If the body mentions the key, then use items() and unpack it. If the body only cares about the value, then use values(). The version to avoid is for k in d: followed by a lookup inside the loop, because it does the lookup twice the work.
What are dict views and why do they matter?
Two things follow from a view being a live window instead of a copy. First, a view reflects any change that you make to the dictionary after you created it.
scores = {'ada': 91}
view = scores.keys()
scores['grace'] = 88
print(list(view))
Second, the key view behaves like a set, and that makes set operations available directly on your dictionary. This is a genuinely useful trick for comparing two mappings.
current = {'ada', 'grace', 'alan'}
previous = {'grace', 'alan', 'linus'}
print(current - previous)
print(current & previous)
print(current ^ previous)
print(current | previous)
A real dictionary supports the same operations through keys(), so you can find the keys that were added, removed, or kept between two versions of a configuration without converting either one to a set first.
old = {'host': 'localhost', 'port': 5432, 'debug': True}
new = {'host': 'db.internal', 'port': 5432, 'timeout': 30}
print(old.keys() - new.keys())
print(new.keys() - old.keys())
print(old.keys() & new.keys())
Those three lines tell you that debug was dropped, that timeout was added, and that host and port survived. Diffing two dictionaries with set operations on their keys is a trick that is worth remembering.
Views answer membership questions as well. 'port' in d.keys() is the same test as 'port' in d, and 5432 in d.values() asks whether a value appears anywhere. The values view is the more expensive of the two, because checking a key can be done by hashing it, while checking a value means looking through all of them.
How do you build one dictionary from another?
The same unpacking that powers the loop also powers a comprehension, and that is the compact way to filter, transform, or reshape a mapping.
scores = {'ada': 91, 'grace': 88, 'alan': 95, 'linus': 40}
passed = {name: score for name, score in scores.items() if score >= 50}
print(passed)
doubled = {name: score * 2 for name, score in scores.items()}
print(doubled)
inverted = {score: name for name, score in scores.items()}
print(inverted)
The inverted version deserves a warning. If two names share a score, then the later one overwrites the earlier one, because a dictionary can only hold one value per key. Swapping keys and values is only safe when the values are unique, and that is exactly the case when you are reversing a mapping like a name-to-id table.
Merging two dictionaries is a third common transformation, and the double-star unpacking from the functions module does it cleanly.
defaults = {'host': 'localhost', 'port': 5432, 'debug': True}
overrides = {'port': 5433, 'debug': False}
settings = {**defaults, **overrides}
print(settings)
And when the keys and the values arrive in two separate lists, then zip() pairs them up and dict() turns the pairs into a mapping. Zip is for the moment of assembly. Once you have a dictionary, items() is the better way to walk it again.
names = ['ada', 'grace', 'alan']
marks = [91, 88, 95]
report = dict(zip(names, marks))
print(report)
What happens if you change a dict while iterating it?
Python detects it and raises RuntimeError: dictionary changed size during iteration. The check exists because the iterator holds a position in the dictionary, and adding or removing an entry would make that position meaningless.
scores = {'ada': 91}
try:
for name in scores:
scores['extra'] = 0
except RuntimeError as error:
print('RuntimeError:', error)
Overwriting the value of a key that already exists is fine, because the size does not change. Adding or removing keys is not fine. The safe pattern is to build a new dictionary and use it to replace the old one, or to iterate over a snapshot of the keys.
scores = {'ada': 91, 'grace': 88}
updated = {name: score + 5 for name, score in scores.items()}
print(updated)
print('original:', scores)
One related detail is that since Python 3.8, reversed() works on a dictionary and on its items, and it gives you the entries in reverse insertion order. That is handy for building a stack-like history from an ordered mapping.
scores = {'ada': 91, 'grace': 88, 'alan': 95}
print(list(reversed(scores)))
Iteration patterns
| You want | Write |
|---|---|
| Just the keys | for k in d: |
| Just the values | for v in d.values(): |
| Keys and values | for k, v in d.items(): |
| Keys as a set | set(d.keys()) |
| Compare the keys of two dicts | d1.keys() - d2.keys() |
| Build a new dict | {k: v for k, v in d.items()} |
| Reverse insertion order | reversed(d) |
Now try it yourself. Build a dictionary of five students and their marks. Loop over it once printing only the names, once printing only the marks, and once printing both with the mark turned into a letter grade. Then use keys() set arithmetic to work out which students are new compared with a second dictionary, and finish by building an upgraded copy with a comprehension while confirming that the original is untouched.