How Does the Executor Framework Manage Threads?

The Executor framework separates two concerns that used to be tangled together: creating threads and running tasks. You hand work to an ExecutorService, and it decides when and on which thread to run it, using a pool of reusable threads. Because the threads are reused, you avoid the cost of creating one per task, and because the pool is bounded, a burst of work cannot spawn an unbounded number of threads.

The factory methods you will actually use. newFixedThreadPool(n) gives you exactly n threads and queues everything else. newCachedThreadPool() gives you threads on demand and reclaims idle ones. newSingleThreadExecutor() serialises all the work onto one thread, which is how you get a sequential pipeline. And newVirtualThreadPerTaskExecutor() creates a fresh virtual thread per task.

There are two older variants worth knowing about. newSingleThreadExecutor has an unbounded queue, so a slow consumer causes the queue to grow without limit and the program runs out of memory rather than failing fast. newFixedThreadPool is also unbounded. The ThreadPoolExecutor constructor lets you supply your own queue and rejection policy, and that is what you want when backpressure matters.

import java.util.concurrent.ExecutorService;
import java.util.concurrent.Executors;
import java.util.concurrent.TimeUnit;

public class Main {
    public static void main(String[] args) throws InterruptedException {
        ExecutorService fixed = Executors.newFixedThreadPool(3);
        ExecutorService single = Executors.newSingleThreadExecutor();

        for (int i = 1; i <= 5; i++) {
            int task = i;
            fixed.submit(() -> {
                System.out.println("task " + task + " on " + Thread.currentThread().getName());
                sleepQuietly(20);
            });
        }
        single.submit(() -> System.out.println("single task on " + Thread.currentThread().getName()));

        fixed.shutdown();
        single.shutdown();
        fixed.awaitTermination(2, TimeUnit.SECONDS);
        single.awaitTermination(2, TimeUnit.SECONDS);
        System.out.println("pools shut down");
    }

    static void sleepQuietly(long ms) {
        try {
            Thread.sleep(ms);
        } catch (InterruptedException e) {
            Thread.currentThread().interrupt();
        }
    }
}

The output shows the pools at work. The five tasks share the three threads of the fixed pool, so they run in batches, while the single-threaded executor runs its one task on its own thread.

The other half of the API is invokeAll and invokeAny, which submit a collection of Callables and wait for them. They are worth knowing because they return a list of Futures and handle the exception wrapping for you.

There are four methods, and the differences matter.

shutdown() stops the pool accepting new tasks, lets the tasks already running finish, and then terminates the worker threads. This is the normal shutdown and the one you want in almost every case.

shutdownNow() tries to stop immediately. It attempts to interrupt the running tasks, and the queued tasks that never started are returned to you as a list. The interruption is a request — a task that swallows the interrupt and keeps going will not actually stop.

awaitTermination(timeout, unit) waits for the pool to finish after a shutdown has been requested, returning true if it terminated in time and false if it did not. It is the piece people forget, and without it a shutdown is a request rather than a guarantee.

isShutdown() tells you whether shutdown has been requested, and isTerminated() whether the pool has actually finished. They are different states, and a program that checks only the first may exit while tasks are still running.

import java.util.concurrent.ExecutorService;
import java.util.concurrent.Executors;
import java.util.concurrent.TimeUnit;

public class Main {
    public static void main(String[] args) throws InterruptedException {
        ExecutorService pool = Executors.newFixedThreadPool(2);

        pool.submit(() -> System.out.println("task 1 ran"));
        pool.submit(() -> {
            sleepQuietly(500);
            System.out.println("task 2 ran slowly");
        });

        // shutdown: no new tasks, existing ones finish.
        pool.shutdown();
        System.out.println("is shutdown    : " + pool.isShutdown());
        System.out.println("is terminated  : " + pool.isTerminated());

        boolean finished = pool.awaitTermination(2, TimeUnit.SECONDS);
        System.out.println("terminated     : " + finished);

        // shutdownNow: interrupt the running tasks.
        ExecutorService other = Executors.newSingleThreadExecutor();
        other.submit(() -> {
            sleepQuietly(5000);
            System.out.println("should not print");
        });
        other.shutdownNow();
        System.out.println("interrupted the running task");
    }

    static void sleepQuietly(long ms) {
        try {
            Thread.sleep(ms);
        } catch (InterruptedException e) {
            Thread.currentThread().interrupt();
        }
    }
}

The output shows the states in order. Immediately after shutdown the pool reports that shutdown was requested but that it has not terminated, because the slow task is still running. After the await it reports that it terminated. The second pool is stopped with shutdownNow, and the interrupted task never reaches its print statement.

Two practical rules. In modern Java, ExecutorService implements AutoCloseable, so the cleanest shutdown is a try-with-resources block — the close method calls shutdown and waits. And never let a pool outlive the object that owns it: an executor with non-daemon threads keeps the JVM alive, which is why an application that creates a pool and forgets to shut it down appears to hang on exit.