How Do encode() and decode() Work?

A str holds Unicode text that a person can read. A bytes object holds a sequence of byte values between 0 and 255, with no encoding attached to it at all. The encode() method turns text into bytes, and the decode() method turns bytes back into text. Both of them need the name of an encoding.

text = "héllo"
data = text.encode("utf-8")

print(type(text).__name__)          # str
print(type(data).__name__)          # bytes
print(data)                         # b'h\xc3\xa9llo'
print(data.decode("utf-8") == text)  # True
print(len(text), len(data))         # 5 6

# bytes behave like a sequence of integers
print(list(data[:3]))               # [104, 195, 169]

# UTF-8 is the default
print(text.encode() == data)        # True

The two lengths differ because of the accented character. The letter e with an acute accent is one character in the text, but it takes two bytes in UTF-8, because that code point sits outside the ASCII range and is stored as a multi-byte sequence. The bytes output shows the hex escape for the two bytes that make up the accented letter.

Three things are worth emphasising here. First, encoding is not symmetric in the way people often expect. Many different encodings can produce the same bytes, but only the right one will decode them back into the original text, and decoding with the wrong encoding usually raises a UnicodeDecodeError instead of returning nonsense. Second, UTF-8 is the default for both encode() and decode(), and it is the right default for almost all new work, because it is the encoding used across the web and it can represent every Unicode code point.

Third, bytes are not text. Printing bytes shows you the b prefix and the numeric escapes, and a bytes object has none of the string methods. There is no bytes.upper() that returns text, and bytes.split() does not accept a string argument. If a library gives you bytes, decode them early and work with text from that point on.

A str is a sequence of Unicode code points, which is human-readable text. A bytes object is a sequence of byte values with no interpretation attached, so it could be UTF-8 text, a PNG image, a compressed blob, or anything else at all. The two are different types, and Python will not quietly turn one into the other, because guessing the encoding is exactly the mistake that produces garbled text.

You need encode() when you are sending text to something that works in bytes. That includes a socket, a file that you opened in binary mode, and any library that expects bytes. You need decode() when you receive bytes and want to work with them as text. That includes reading a file that you opened in binary mode, handling a response body, and parsing data that arrived over the network.

The everyday rule is to keep text as text for as long as possible, and to convert only at the boundary. Open text files in text mode and let Python take care of the encoding. Work with str inside your own functions. And encode or decode only where the data crosses into or out of a system that thinks in bytes.

There are two traps to watch for. The first one is decoding with the wrong encoding, which either raises a UnicodeDecodeError or quietly produces nonsense. Always decode with UTF-8 unless you have solid evidence for something else, and when you read a file, pass its encoding explicitly. The second trap is calling .decode() on something that is already text, or .encode() on something that is already bytes. That raises an AttributeError, and the type errors are your friend here, because they stop the guesswork before it becomes a bug.