Choosing between dict.get() and direct indexing in Python is a decision about what a missing key means. Sometimes it means "use this fallback and carry on", and sometimes it means "this program is broken and it should stop". The two access styles exist because those are two different intentions.
The previous lesson in this module covered "Dictionaries: A Practical Crash Course", and the next lesson in this module is about "Iterating Dictionaries: keys(), values(), items()".
What happens when you index a missing key?
You get a KeyError, and the program stops unless something catches it. That is the correct behaviour when the key really is required, because carrying on with a made-up value would let the wrong thing happen further downstream.
settings = {'host': 'localhost'}
try:
print(settings['port'])
except KeyError as error:
print('KeyError:', error)
The error message quotes the missing key, and that makes it easy to trace back to where the lookup happened. The problem is not the exception itself. The problem is that an exception is an expensive way to say "this option was not set", especially when it happens on every single request.
How does get() differ from direct indexing?
get() never raises for a missing key. Called with one argument it returns None, and called with two it returns the second one.
settings = {'host': 'localhost', 'port': 5432}
print(settings.get('host'))
print(settings.get('timeout'))
print(settings.get('timeout', 30))
print(settings['port'])
That difference is the whole point, but it hides a subtlety. get() collapses two different situations into one value. A key that is absent and a key that is present with the value None both come back as None.
There is a performance angle as well. A successful get() is a plain lookup with no extra work, whereas a lookup that misses and is caught by a try block has to build an exception and then unwind it. The gap is small, but it is the reason why code that misses on every call is measurably slower when it uses try instead of get.
profile = {'name': 'Ada', 'nickname': None}
print(profile.get('nickname', 'no nickname'))
print(profile.get('missing', 'no nickname'))
Both lines print the same fallback, even though in the first case the key exists and its value really is None. If that distinction matters to you, then check membership first.
How do you tell "missing" from "present but None"?
Use the in operator, or catch the KeyError when the difference genuinely matters. The membership test is the cheap option and it reads clearly.
profile = {'name': 'Ada', 'nickname': None}
if 'nickname' in profile:
print('key present, value is:', profile['nickname'])
else:
print('key absent')
def read_value(data):
try:
return data['value']
except KeyError:
return 'fallback'
print(read_value({}))
print(read_value({'value': 5}))
The try block is worth using when the key is usually present and you only need to handle the rare miss, because it costs nothing on the happy path. It is the wrong tool when the key is usually missing, because then you pay the exception cost on every call.
When should you use setdefault?
setdefault is a two-in-one operation. It returns the value for the key if the key exists, and otherwise it inserts the default and returns that. It is the natural fit for "read this, and if it is not set yet, then set it".
options = {'retries': 3}
print(options.setdefault('retries', 10))
print(options.setdefault('timeout', 30))
print(options)
The first call found the key and returned the existing value of 3, and it ignored the default of 10. The second call inserted 30. Notice that the order of the keys is unchanged for the key that already existed, and that the new key is added at the end, which is how dictionaries behave.
There is one gotcha that catches people in loops. The default argument is worked out every time the call runs, whether or not the key exists. If the default is an expensive call, then you are paying for it on every iteration. Build the value first and pass it in, or use an explicit if key not in d check when the default is expensive.
How does defaultdict change the workflow?
collections.defaultdict removes the "is it there?" check entirely. You give it a factory, and asking for a missing key creates that key with the output of the factory instead of raising an error.
from collections import defaultdict
groups = defaultdict(list)
groups['python'].append('lesson 1')
groups['python'].append('lesson 2')
groups['sql'].append('lesson 1')
print(dict(groups))
That turns the classic "group the items" loop into three lines with no membership checks at all. defaultdict(int) gives you a zero for missing keys, which is ideal for counting, and defaultdict(list) is the workhorse for grouping.
The trade-off is that defaultdict creates an entry every time you ask for a missing key, including when a typo is the reason that you asked. A misspelled key leaves a stray empty entry behind instead of failing loudly. When the set of keys is under your control that is harmless, but when it comes from user input it can quietly fill a dictionary with junk.
from collections import defaultdict
counts = defaultdict(int)
for character in 'mississippi':
counts[character] += 1
print(dict(counts))
For pure counting, collections.Counter does the same job with even less code, and it gives you extras such as most_common().
from collections import Counter
counts = Counter('mississippi')
print(counts)
print(counts['z'])
print(counts.most_common(2))
Notice that Counter returns zero for a key that it has never seen, instead of raising an error. That makes it a dictionary with the missing-key problem already solved.
How do you keep a clear error when a key really is required?
Sometimes you want the program to stop, but you want it to stop with a message that says which record was wrong. A bare KeyError gives you the name of the key and nothing about the context. Checking membership first lets you raise your own error instead.
def require_field(record, field):
if field not in record:
raise ValueError('record ' + repr(record) + ' is missing required field ' + repr(field))
return record[field]
try:
require_field({'id': 7}, 'email')
except ValueError as error:
print(error)
print(require_field({'id': 7, 'email': '[email protected]'}, 'email'))
This costs one extra lookup, and it buys an error message that points at the actual problem instead of at a dictionary access buried three frames deep. It is the pattern to reach for when the data comes from outside your program and you want to fail with something that a human can act on.
One more variant is worth knowing. d.pop(key, default) both looks up and removes. If the key is present, you get the value and the entry disappears. If it is absent, you get the default and nothing is removed. It is the safe way to consume a value exactly once.
Choosing a lookup style
| Situation | Use | Missing key |
|---|---|---|
| The key must be there | d["k"] | raises KeyError |
| An optional setting | d.get("k", default) | returns the default |
Missing and None have to differ | "k" in d | returns False |
| Read once, then set it if it is absent | d.setdefault("k", v) | inserts v |
| Group or count as you go | defaultdict | creates the entry |
| Count occurrences | Counter | returns zero |
Now try it yourself. Take a dictionary of API responses where some records are missing fields. Read one field with direct indexing and catch the KeyError. Read it with get() and a fallback. Then read it with a membership test, so that you can tell a missing field from a field whose value is None. Next, build a defaultdict(list) that groups words by their first letter, and finish with a Counter that finds the two most common words in a sentence.