Python gives you two ways to do several things at the same time, and they suit opposite problems. Threads overlap the waiting. asyncio overlaps the waiting as well, but inside a single thread and with less overhead. If you pick the wrong one, you end up with a program that is no faster than the version that you started with.
The previous lesson in this module covered "Writing Your First Tests with pytest", and the next lesson in this module is about "Capstone: A JSON-Powered Command-Line Data Tool".
What does the GIL actually change?
CPython holds a global interpreter lock, so only one thread executes Python bytecode at any moment. Switching between threads is cheap, but the work itself is done one after the other. That means two threads that do arithmetic do not run in parallel. They take turns, and the total time is roughly the same as running the work twice in a row.
from concurrent.futures import ThreadPoolExecutor
from time import perf_counter
def spin(n):
"""Do a small amount of pure-Python arithmetic."""
total = 0
for number in range(n):
total = total + number * number
return total
work = 2000000
start = perf_counter()
spin(work)
spin(work)
sequential = perf_counter() - start
start = perf_counter()
with ThreadPoolExecutor(max_workers=2) as pool:
list(pool.map(spin, [work, work]))
threaded = perf_counter() - start
print('sequential:', round(sequential, 3), 'seconds')
print('threaded: ', round(threaded, 3), 'seconds')
print('half of sequential:', round(sequential / 2, 3), 'seconds')
The absolute numbers depend on your own machine, but the relationship between them does not. If the two threads were really running in parallel, the threaded run would take about half the sequential time, and that is exactly what the "half of sequential" line prints. On the machine where this was written, the sequential run took 1.944 seconds and the threaded run took 1.949 seconds, while the target was 0.972 seconds. The threads are taking turns and not running side by side. The exact figures move a little from run to run, and the ratio is the point.
That is the whole GIL story in one experiment. Threads overlap waiting, and they do not overlap computing. If your workload is CPU-bound and you need real parallelism, the answer is multiprocessing or a native library that releases the GIL, and not more threads.
When do threads actually help?
Threads are excellent at the other kind of work, which is waiting for something. A network call, a database query, or a file read spends almost all of its time blocked, and the interpreter releases the GIL while it waits. That is where threads earn their keep.
from concurrent.futures import ThreadPoolExecutor
from time import perf_counter, sleep
def fetch(name, delay):
"""Pretend to wait for a slow network response."""
sleep(delay)
return name + ' done'
start = perf_counter()
with ThreadPoolExecutor(max_workers=3) as pool:
results = list(pool.map(fetch, ["a", "b", "c"], [1.0, 0.8, 0.6]))
elapsed = perf_counter() - start
print(results)
print('elapsed:', round(elapsed, 2), 'seconds')
Three waits of 1.0, 0.8, and 0.6 seconds take 2.4 seconds one after another, and about 1.0 second here, because the waits overlap. The ThreadPoolExecutor is the tidy way to do this. It manages a pool of worker threads, hands out the work, and shuts the pool down when the with block ends.
How is asyncio different?
asyncio takes the opposite approach. There is one thread, and instead of blocking it, the program awaits. It tells the event loop "I am waiting for this, wake me when it is ready", and then it goes and does something else. That means there is no thread switching, there are no locks, and there is no risk of two threads corrupting shared state.
import asyncio
from time import perf_counter
async def fetch(name, delay):
"""Pretend to wait for a slow network response."""
await asyncio.sleep(delay)
return name + ' done'
async def main():
start = perf_counter()
results = await asyncio.gather(
fetch("a", 1.0),
fetch("b", 0.8),
fetch("c", 0.6),
)
print(results)
print('elapsed:', round(perf_counter() - start, 2), 'seconds')
asyncio.run(main())
asyncio.gather starts all three coroutines and waits for all of them, so the total time is the longest single wait. That is about 1.0 second instead of the 2.4 seconds that the sequential version needs. asyncio.run is the entry point. It creates an event loop, runs the coroutine inside it, and closes the loop again afterwards.
What happens when you block inside a coroutine?
Here is the mistake that catches everybody. Calling a blocking function inside a coroutine stalls the whole event loop, so every other task has to wait, because there is only one thread and it is busy.
import asyncio
from time import perf_counter, sleep
async def blocking_task(name):
sleep(1.0) # a blocking call inside a coroutine
return name + ' done'
async def main():
start = perf_counter()
results = await asyncio.gather(blocking_task("a"), blocking_task("b"))
print(results)
print('elapsed:', round(perf_counter() - start, 2), 'seconds')
asyncio.run(main())
The elapsed time is about 2.0 seconds and not 1.0 second, because gather ran the two tasks one after the other. The first one froze the loop, so the second one had to wait for it. The fix is to hand the blocking work to a thread with asyncio.to_thread.
import asyncio
from time import perf_counter, sleep
def blocking_task(name):
sleep(1.0)
return name + ' done'
async def main():
start = perf_counter()
results = await asyncio.gather(
asyncio.to_thread(blocking_task, "a"),
asyncio.to_thread(blocking_task, "b"),
)
print(results)
print('elapsed:', round(perf_counter() - start, 2), 'seconds')
asyncio.run(main())
Now the elapsed time is about 1.0 second. asyncio.to_thread runs the blocking function in a worker thread and hands the result back to the loop, and that is the bridge between the two models.
Which should you choose?
| Situation | Use | Because |
|---|---|---|
| Waiting on many HTTP requests or database queries | asyncio | Thousands of connections in one thread, with predictable scheduling |
| Waiting on a few blocking libraries that you cannot change | Threads | Simpler, and the libraries already block the thread properly |
| Heavy CPU work, such as maths, images, or parsing | multiprocessing | Neither threads nor asyncio can use more than one core for Python bytecode |
| A mixed program | Both | Use asyncio for your own network code and asyncio.to_thread for the blocking library calls |
| A simple script with three slow calls | Neither | Sequential code is easier to read and fast enough, so add concurrency when you have measured a problem |
The decision rule is about what your program is waiting for. If the answer is "mostly the operating system", which means the network, the disk, or a database, then concurrency helps and asyncio is the more efficient tool. If the answer is "mostly arithmetic", then only separate processes help.
What are the classic asyncio mistakes?
- Blocking calls inside a coroutine.
time.sleep,requests.get, and file reads freeze the whole loop. Useasyncio.sleep, an async client, orasyncio.to_threadinstead. - Creating a task and forgetting it.
asyncio.create_taskreturns a task that has to be awaited or scheduled on the loop, and otherwise the work can be cancelled without any warning. - No timeouts. A request that hangs hangs the whole program. Put a deadline around every network call with
asyncio.wait_for. - Unlimited concurrency. A thousand simultaneous requests will exhaust the connection limit of the server. Use an
asyncio.Semaphoreto cap how many run at once.
What to remember
- Threads and asyncio both overlap waiting, and neither one speeds up pure-Python CPU work.
- Threads are simple and fit blocking libraries, while asyncio scales up to many concurrent connections.
asyncio.gatherruns coroutines concurrently, andasyncio.runowns the event loop.- Never call blocking code inside a coroutine. Use
asyncio.to_threadinstead. - Measure before you add concurrency, and always give network calls a timeout.
Try it yourself: Write a function that pretends to fetch a page with a one-second wait, and then call it three times in three different ways. Call it sequentially, call it with a ThreadPoolExecutor, and call it with asyncio.gather. Print the elapsed time for each one. The sequential run takes about three seconds and the other two take about one, and that gap is the whole argument for concurrency. Then replace the wait with a CPU-heavy loop and repeat the experiment to watch the gap disappear.