Multithreading vs Multiprocessing in Python

The decision comes down to what your work is waiting on.

Use threads when the work is I/O-bound: network calls, database queries, file reads, waiting on a lock or a subprocess. Threads share memory, so passing results between them is free, and the interpreter switches between them whenever one blocks. A thousand threads waiting on HTTP responses is a perfectly reasonable design.

Use processes when the work is CPU-bound: number crunching, image processing, compression, heavy parsing. Processes have separate memory and therefore separate interpreters, so they run on separate cores with no GIL contention. The cost is that they cannot share ordinary Python objects — state has to be sent through a multiprocessing.Queue or a Pipe, and everything crossing that boundary has to be picklable.

There is a third option worth knowing. The concurrent.futures module gives you both under one interface: ThreadPoolExecutor for I/O and ProcessPoolExecutor for CPU work. The executor API is the same for both, so switching is a one-line change, and futures give you a clean way to submit work, wait for a result and handle a failure.

Two practical points for an interview. The overhead of a process is much higher than a thread, because starting a process means starting a whole interpreter and copying or re-creating the state. And in Python 3.8 and later, the default start method on Linux changed to fork, which is cheap but can be unsafe in a program that has threads; the forkserver and spawn methods are the safer choices when you care.

Threads in one process share a single address space, so a list written by one thread is immediately visible to another.

import threading


def worker(results, index):
    results[index] = f"result {index}"


results = [None] * 4

threads = [threading.Thread(target=worker, args=(results, i)) for i in range(4)]
for thread in threads:
    thread.start()
for thread in threads:
    thread.join()

print(results)

The output is the list of four finished strings, in the order the indexes were given. Each thread wrote into the same list object that the main thread created and passed in, with no copying and no serialisation.

Processes cannot work that way. Each process has its own memory, so a list created in the parent is a completely different object in the child. Getting a value back requires an explicit channel: multiprocessing.Queue for a producer-consumer pattern, Pipe for a two-way connection, or a Pool that maps work over a set of inputs and collects the results.

That channel is where the cost comes from. Everything sent through it has to be pickled into bytes and unpickled on the other side, which costs time and memory proportional to the size of the data. It also means only picklable objects can cross — ordinary classes, open file handles, database connections and locks cannot. That is why passing large arrays between processes is expensive, and why the right design is usually to keep the data local to one worker and pass only identifiers between them.