What Are Sets Used For in Python?

A set is an unordered collection of unique, hashable items. The two things it is best at are membership tests and removing duplicates.

tags = ["python", "web", "python", "data", "web"]

unique = set(tags)
print(type(unique).__name__)    # set
print(len(unique))               # 3 - duplicates are gone
print("python" in unique)        # True
print("ruby" in unique)          # False

# Building a set directly
numbers = {1, 2, 2, 3}
print(numbers)                  # {1, 2, 3}
print(len(numbers))              # 3

The list of five tags becomes a set of three, because a set stores each item only once. Membership testing is where the set earns its keep: the in operator on a set is a hash lookup, so it is constant time, while the same test on a list has to scan element by element. On a list of a million items that is the difference between instant and visibly slow, and it is why converting to a set before a loop of membership tests is a standard optimisation.

Two behaviours to note. Sets are unordered, so there is no first element and the iteration order depends on the hash table — never rely on it. And equality is by content, so {1, 2} == {2, 1} is True, which is a different rule from lists, where order matters.

A set holds any hashable value, and it can mix types in a single set because equality is what matters, not the type.

mixed = {1, "two", 3.0, (4, 5), frozenset({6})}
print(len(mixed))            # 5

# Lists and dicts are not hashable
try:
    {1, 2, [3]}
except TypeError as err:
    print("TypeError:", err)

# A frozenset is hashable, so it can live inside a set
print(hash(frozenset({1, 2})) == hash(frozenset({2, 1})))

The TypeError says that a list is unhashable, and it is worth being able to explain why. A hash must stay constant for the lifetime of an object, because a set uses it to place the object in a bucket. A list can change its contents at any time, so its hash would change and the set's internal structure would be corrupted. The same argument rules out dicts.

Tuples are hashable, which is why the set above can hold (4, 5). The rule is that a tuple is hashable only if all of its elements are hashable, so a tuple containing a list is also unhashable.

There is one subtlety in the mixed set. Because 1 == 1.0 == True, a set containing both 1 and True has only one element, since they are equal. That is why you should be careful putting booleans and integers in the same set — the boolean disappears.

The last line shows that a frozenset hashes consistently regardless of the order its elements were added in, which is exactly what makes frozensets usable as dictionary keys and as elements of another set.

Sets exist for three jobs: membership testing, removing duplicates and expressing relationships between groups of items.

The membership test is the big performance win. The in operator on a set is a hash lookup and takes constant time, while on a list it is a linear scan. Any code that tests membership inside a loop should convert the container to a set first — that one change is often the difference between a function that runs in milliseconds and one that runs in minutes.

Removing duplicates is the second job. list(set(items)) is the quick way, though it loses the order, and list(dict.fromkeys(items)) keeps the order and is just as fast.

The third job is the set algebra: union, intersection, difference and symmetric difference, with operators and with named methods. That is how you find which items two groups share, which is a question that comes up constantly in permissions, tags, deduplication and comparing test coverage.

How they differ from the other two containers is worth stating clearly. A list is ordered, allows duplicates and is indexable. A dictionary maps keys to values and is also ordered. A set is unordered, holds only unique items, and is not indexable — you cannot ask for the first element, because there is no first element.

One practical note: because a set is unordered, if you need deterministic output from iterating one, sort it first with sorted().