How Does the for Loop Iterate in Python?

A Python for loop does not work with indexes at all. Instead, it asks the object for an iterator, and then it calls next on that iterator again and again until the iterator is empty.

for number in [10, 20, 30]:
    print(number)

for letter in 'abc':
    print(letter)

for key in {'a': 1, 'b': 2}:
    print(key)

for index in range(3):
    print(index)

The output is 10, 20, and 30 for the first loop. It is a, b, and c for the second one. It is a and b for the third loop, because a dictionary gives you its keys in the order they were added. It is 0, 1, and 2 for the last loop, because range(3) starts at zero and stops before the stop value.

Two details are worth pointing out here. Iterating a dictionary directly gives you the keys, so a very common mistake is to expect the values instead. When you want both halves of each pair, you use items() instead. And range(3) gives you 0, 1, and 2 because it starts at zero and stops before the number you gave it.

Underneath, the loop does the same thing as asking for an iterator with the two-argument form of iter and then calling next over and over. That is why any object with __iter__ and __next__ can be used in a for loop, and it is also why a for loop over a generator uses the values one at a time instead of building a list first.

The trap that catches people out is changing a list while you are looping over it. A list iterator keeps a position rather than a copy of the list, so removing an item shifts the later items down one place and the loop skips one of them. Appending to the list keeps the loop going for longer than you expected, because the iterator moves towards the current end of the list every time it runs.

Iterating a dictionary directly gives you the keys. The values and the pairs are available through dedicated methods, and those methods return live views rather than copies.

scores = {'ada': 99, 'grace': 87, 'alan': 91}

for name in scores:
    print(name)

for name in scores.keys():
    print(name, scores[name])

for name, score in scores.items():
    print(name, score)

for score in scores.values():
    print(score)

The first loop prints ada, grace, and alan, which are the keys only, in the order they were added. The second loop prints the same three names together with their scores. The third loop pulls each key and value apart into two names, and that is the form to use when you need both halves. The last loop prints only the three scores.

The detail worth making here is that keys(), values(), and items() return view objects and not lists. A view shows any change that you make to the dictionary after you created it, so it is a live window rather than a snapshot. When you want a frozen copy, you wrap the result in list().

The unpacking in the items() loop is the same tuple unpacking that you would write by hand, and it is the most readable option whenever you need both halves. It also fails loudly with a ValueError if the pairs are not the length you expected, which is much better than quietly zipping a structure that has the wrong shape.

One last comparison is worth mentioning. For a dictionary, iterating items() is usually faster than iterating keys() and then looking every key up, because it saves a second lookup for each key.

Because a list iterator in Python keeps a position index instead of a snapshot of the list. Each time you ask for the next item, it reads whatever sits at the current index and then moves the index along. Nothing is copied, so any change that you make to the list is visible to the loop on its next step.

The result for appending is that the loop keeps going until it reaches the current end of the list. If you append inside the loop, the list grows, and the iterator will eventually reach the items that you added. That is exactly how you end up with a loop that never stops, because the append never stops either.

Removing an item is worse. Deleting the current element shifts every later element one place to the left, so the index that the iterator holds now points one past the element that it just read, and that element is skipped completely. Removing an element that comes before the current position shifts the current element down, so the iterator reads it a second time.

The rules that follow from all of this are the practical answer. Never change a list while you are looping over it. To change the items in place, loop over a copy with for item in list(items):. To add items safely, build a new list with a comprehension, or collect the items in a separate list and add them afterwards. To filter, use a comprehension such as [x for x in items if keep(x)], which produces a new list and leaves the original alone.

The same reasoning applies to dictionaries. Setting an existing key while you iterate is allowed, because it does not change the size of the dictionary. Adding a brand new key raises a RuntimeError that tells you the dictionary changed size during the iteration.