Threading vs asyncio: Choosing Concurrency

Python gives you two ways to do several things at once, and they suit opposite problems. Threads overlap waiting; asyncio overlaps waiting in a single thread with less overhead. Picking the wrong one leaves you with a program that is no faster than the version you started with.

What does the GIL actually change?

CPython holds a global interpreter lock, so only one thread executes Python bytecode at a time. Switching between threads is cheap, but the work itself is serialised. That means two threads doing arithmetic do not run in parallel - they take turns, and the total time is roughly the same as running the work twice in a row.

from concurrent.futures import ThreadPoolExecutor
from time import perf_counter


def spin(n: int) -> int:
    """Do a small amount of pure-Python arithmetic."""
    return sum(i * i for i in range(n))


work = 2_000_000

start = perf_counter()
spin(work)
spin(work)
sequential = perf_counter() - start

start = perf_counter()
with ThreadPoolExecutor(max_workers=2) as pool:
    list(pool.map(spin, [work, work]))
threaded = perf_counter() - start

print(f"sequential: {sequential:.3f}s")
print(f"threaded:    {threaded:.3f}s")
print(f"half of sequential: {sequential / 2:.3f}s")

The absolute numbers depend on your machine, but the relationship does not. If the two threads were really running in parallel, the threaded run would take about half the sequential time - the half of sequential line prints that target. On the machine this was written on, the sequential run took 1.944 seconds and the threaded run 1.949 seconds, while the target was 0.972 seconds. The threads are taking turns, not running side by side. The exact figures move a little from run to run; the ratio is the point.

That is the whole GIL story in one experiment: threads overlap waiting, and they do not overlap computing. If your workload is CPU-bound and you need real parallelism, the answer is multiprocessing or a native library that releases the GIL - not more threads.

When do threads actually help?

Threads are excellent at the other kind of work: waiting for something. A network call, a database query or a file read spends almost all of its time blocked, and the interpreter releases the GIL while it waits. That is where threads earn their keep.

from concurrent.futures import ThreadPoolExecutor
from time import perf_counter, sleep


def fetch(name: str, delay: float) -> str:
    """Pretend to wait for a slow network response."""
    sleep(delay)
    return f"{name} done"


start = perf_counter()
with ThreadPoolExecutor(max_workers=3) as pool:
    results = list(pool.map(fetch, ["a", "b", "c"], [1.0, 0.8, 0.6]))
elapsed = perf_counter() - start

print(results)
print(f"elapsed: {elapsed:.2f}s")

Three waits of 1.0, 0.8 and 0.6 seconds take 2.4 seconds in sequence and about 1.0 second here, because the waits overlap. The ThreadPoolExecutor is the tidy way to do this: it manages a pool of worker threads, hands out the work, and shuts the pool down when the with block ends.

How is asyncio different?

asyncio takes the opposite approach. There is one thread, and instead of blocking it, the program awaits - it tells the event loop "I am waiting for this, wake me when it is ready" and then does something else. That means no thread switching, no locks and no risk of two threads corrupting shared state.

import asyncio
from time import perf_counter


async def fetch(name: str, delay: float) -> str:
    """Pretend to wait for a slow network response."""
    await asyncio.sleep(delay)
    return f"{name} done"


async def main() -> None:
    start = perf_counter()
    results = await asyncio.gather(
        fetch("a", 1.0),
        fetch("b", 0.8),
        fetch("c", 0.6),
    )
    print(results)
    print(f"elapsed: {perf_counter() - start:.2f}s")


asyncio.run(main())

asyncio.gather starts all three coroutines and waits for all of them, so the wall-clock time is the longest wait - about 1.0 second rather than the 2.4 seconds the sequential version needs. asyncio.run is the entry point: it creates an event loop, runs the coroutine, and closes the loop again.

What happens when you block inside a coroutine?

Here is the mistake that catches everyone. Calling a blocking function inside a coroutine stalls the entire event loop - every other task waits, because there is only one thread and it is busy.

import asyncio
from time import perf_counter, sleep


async def blocking_task(name: str) -> str:
    sleep(1.0)          # a blocking call inside a coroutine
    return f"{name} done"


async def main() -> None:
    start = perf_counter()
    results = await asyncio.gather(blocking_task("a"), blocking_task("b"))
    print(results)
    print(f"elapsed: {perf_counter() - start:.2f}s")


asyncio.run(main())

The elapsed time is about 2.0 seconds, not 1.0 - gather ran the two tasks one after the other because the first one froze the loop. The fix is to hand blocking work to a thread with asyncio.to_thread.

import asyncio
from time import perf_counter, sleep


def blocking_task(name: str) -> str:
    sleep(1.0)
    return f"{name} done"


async def main() -> None:
    start = perf_counter()
    results = await asyncio.gather(
        asyncio.to_thread(blocking_task, "a"),
        asyncio.to_thread(blocking_task, "b"),
    )
    print(results)
    print(f"elapsed: {perf_counter() - start:.2f}s")


asyncio.run(main())

Now the elapsed time is about 1.0 second. asyncio.to_thread runs the blocking function in a worker thread and hands the result back to the loop, which is the bridge between the two models.

Which should you choose?

The decision rule is about what your program is waiting for. If the answer is "mostly the operating system" - network, disk, database - concurrency helps, and asyncio is the more efficient tool. If the answer is "mostly arithmetic", only separate processes help.

What are the classic asyncio mistakes?

• Blocking calls in a coroutine. time.sleep, requests.get and file reads freeze the whole loop. Use asyncio.sleep, an async client, or asyncio.to_thread.
• Creating a task and forgetting it. asyncio.create_task returns a task that must be awaited or scheduled on the loop; otherwise the work can be cancelled without warning.
• No timeouts. A hung request hangs the program. Wrap every network call in asyncio.wait_for with a deadline.
• Unlimited concurrency. A thousand simultaneous requests will exhaust the server's connection limit. Use an asyncio.Semaphore to cap how many run at once.

What to remember

• Threads and asyncio both overlap waiting; neither one speeds up pure-Python CPU work.
• Threads are simple and fit blocking libraries; asyncio scales to many concurrent connections.
• asyncio.gather runs coroutines concurrently; asyncio.run owns the event loop.
• Never call blocking code inside a coroutine - use asyncio.to_thread instead.
• Measure before you add concurrency, and always give network calls a timeout.

Try it yourself: Write a function that pretends to fetch a page with a one-second wait, then call it three times in three different ways: sequentially, with a ThreadPoolExecutor, and with asyncio.gather. Print the elapsed time for each. The sequential run takes about three seconds and the other two about one - that gap is the whole argument for concurrency. Then replace the wait with a CPU-heavy loop and repeat the experiment to see the gap disappear.