Async vs Multithreading

Multithreading and async I/O both solve the same problem, "don't waste time waiting," in opposite ways. Multithreading adds more workers that each block; async uses a single worker that never blocks. Picking the right one depends on whether your program spends its time waiting or computing.

The problem#

Most server time is spent waiting, not computing. Take one HTTP request:

The total is 54 ms, but only 4 ms is real CPU work, so ~93% of the time is spent waiting.

Multithreading: more workers#

Give each request its own thread. While Thread 1 waits on the database, Thread 2 handles another request.

Each thread blocks on I/O, and the OS switches between them.

It works, but each thread costs a stack (often ~1 MB), context switches add overhead, and 10,000 concurrent requests means 10,000 threads. A thread pool bounds the count but doesn't change the model.

Async: one worker that never waits#

One thread handles everything. When it hits I/O, it starts the operation, registers what to do when it completes, and moves straight on.

1. Request 1 arrives: start DB query 1, don't wait.
2. Request 2 arrives: start DB query 2, don't wait.
3. Request 3 arrives: start DB query 3, don't wait.
4. Query 1 is done: process 1, respond 1.
5. Query 2 is done: process 2, respond 2.
6. Query 3 is done: process 3, respond 3.

The event loop#

How does the thread know which I/O finished? An event loop repeatedly asks the OS (via epoll on Linux, kqueue on macOS/BSD, IOCP on Windows) which operations are ready:

while True:
    events = os_poll()        # which sockets/files are ready?
    for event in events:
        event.callback()      # continue the work that was waiting on it

Node.js, Python's asyncio, and frameworks like Netty work this way: one loop, thousands of concurrent connections.

import asyncio

async def handle(request_id):
    # await yields control to the event loop while the "query" is in flight
    await asyncio.sleep(0.05)          # stands in for a 50 ms DB call
    return f"response {request_id}"

async def main():
    # 1,000 requests overlap on ONE thread: total ≈ 50 ms, not 50 s
    results = await asyncio.gather(*(handle(i) for i in range(1000)))
    print(len(results))

asyncio.run(main())

Side by side, the difference is how much of each thread's time is useful:

When to use which#

The deciding question is whether your workload mostly waits or mostly computes.

Node.js is a good example of "both": your JavaScript runs on one event loop, while libuv uses a small background thread pool for work like file system calls, DNS lookups and crypto.

Trade-offs#

Each model pays a different price for its strengths.

The biggest async pitfall: never block the event loop. A synchronous DB driver or a CPU-heavy loop inside an async handler stalls every other connection. Hand such work to a thread or process pool.

Simple mental model#

Ask what your program spends its time doing:

Some examples:

• API server making many DB/HTTP calls: async.
• Image processing pipeline: multithreading / multiprocessing.
• LLM-backed service: both. Async while waiting on the model API, and a pool for heavy parsing or post-processing.

For CPU work you only need as many threads as cores. On 8 cores, 8 threads run truly in parallel; 16 still run only 8 at a time and add switching; 100 is pure waste.

In Python, the GIL changes the CPU-bound answer: use processes, not threads.

Key takeaways#

• Multithreading overlaps waiting by giving each task its own blocking thread; async overlaps it with one non-blocking event loop.
• Async wins for I/O-heavy, high-concurrency work because it avoids per-connection stacks and switches.
• Threads (or processes) win for CPU-bound work because they use multiple cores.
• Don't block the event loop; offload CPU or blocking calls to a pool.
• CPU-bound parallelism tops out at one thread per core.