Once you are past the raw threads, the standard library gives you a set of tools that handle the awkward parts. You get the management of a pool of the workers, the collecting of the results, the waiting for a group of the tasks, and the protecting of the shared collections. Knowing which of these to reach for is the difference between the concurrency that works and the concurrency that deadlocks under the load.
The previous lesson in this module covered "What Causes Race Conditions and How Does synchronized Fix Them?", and the next lesson in this module is about "How Does JVM Memory and Garbage Collection Actually Work?".
What does an ExecutorService give you?
An executor owns a pool of the threads and it runs your tasks on them. You submit a task and you get back a handle, rather than creating a thread per unit of the work. The pool reuses its threads, and that is the point. The thread creation is expensive, and an unbounded pool will exhaust the resources of the machine long before your work is finished.
import java.util.concurrent.ExecutorService;
import java.util.concurrent.Executors;
import java.util.concurrent.TimeUnit;
public class PoolDemo {
public static void main(String[] args) throws Exception {
ExecutorService pool = Executors.newFixedThreadPool(3);
for (int i = 1; i <= 5; i++) {
int task = i;
pool.execute(() -> {
System.out.println("task " + task + " on " + Thread.currentThread().getName());
});
}
pool.shutdown();
pool.awaitTermination(5, TimeUnit.SECONDS);
System.out.println("pool terminated : " + pool.isTerminated());
}
}The output is this.
task 2 on pool-1-thread-2
task 3 on pool-1-thread-3
task 1 on pool-1-thread-1
task 4 on pool-1-thread-2
task 5 on pool-1-thread-3
pool terminated : trueThere are two details in that program that matter for any real service. The pool was given three threads, so the five tasks are processed by three workers rather than five at once. And the calls shutdown and awaitTermination are the polite way to finish. You stop accepting the new work, and then you wait for the outstanding work to complete. Without the shutdown, the threads of the pool keep the JVM alive.
How do you get a result back?
You submit a Callable instead of a Runnable and you get a Future, which is a promise of a result. The call get blocks until the task finishes and it returns the value, and it also throws if the task threw. The calls isDone and isCancelled let you check without blocking.
import java.util.ArrayList;
import java.util.List;
import java.util.concurrent.Callable;
import java.util.concurrent.ExecutorService;
import java.util.concurrent.Executors;
import java.util.concurrent.Future;
import java.util.concurrent.TimeUnit;
public class FutureAndInvokeAll {
public static void main(String[] args) throws Exception {
ExecutorService pool = Executors.newFixedThreadPool(4);
Future<Integer> first = pool.submit(() -> 21 * 2);
System.out.println("result : " + first.get());
System.out.println("done before get : " + first.isDone());
List<Callable<String>> jobs = new ArrayList<>();
for (int i = 1; i <= 3; i++) {
int id = i;
jobs.add(() -> "job-" + id);
}
List<Future<String>> results = pool.invokeAll(jobs);
for (Future<String> result : results) {
System.out.println("collected : " + result.get());
}
pool.shutdown();
}
}The output is this.
result : 42
done before get : true
collected : job-1
collected : job-2
collected : job-3The call invokeAll is the batch version. You give it a list of the tasks and it runs them all, it waits for every one, and it returns a list of the futures in the order that you submitted them. The results came back in order even though the tasks ran in parallel, which is exactly what you want for a batch job.
What does CompletableFuture add?
A Future gives you one blocking call. The class CompletableFuture gives you a pipeline of the non-blocking steps. Each stage takes the previous result, it transforms it, and it returns another future, so you can describe a chain of the asynchronous work without a single thread waiting on another.
import java.util.concurrent.CompletableFuture;
public class CompletableFutureDemo {
public static void main(String[] args) throws Exception {
CompletableFuture<String> supply = CompletableFuture.supplyAsync(() -> "loaded data");
CompletableFuture<Integer> length = supply.thenApply(String::length);
System.out.println("length : " + length.get());
CompletableFuture<Void> pipeline = CompletableFuture
.supplyAsync(() -> "ana")
.thenApply(name -> name.toUpperCase())
.thenAccept(System.out::println)
.thenRun(() -> System.out.println("pipeline done"));
pipeline.join();
System.out.println("all done");
}
}The output is this.
length : 11
ANA
pipeline done
all doneThere are two things to notice. The first program starts the work on a background thread and it only blocks at the very end, at the call get. The second one builds a four-stage pipeline and it finishes with the call join. The stages run on the different threads where the library can, and the whole thing reads as a sequence of the transformations rather than a pile of the callbacks.
What are the latches and the semaphores for?
A CountDownLatch is a gate. The threads count down as they finish a stage, and another thread waits for the count to reach the zero. It is the standard way to say "wait until these three things are ready". A Semaphore limits how many threads may hold a resource at once. With one permit, it is a mutex that you can time out on. With several, it is a pool of the connections or a rate limit.
import java.util.concurrent.CountDownLatch;
import java.util.concurrent.ExecutorService;
import java.util.concurrent.Executors;
import java.util.concurrent.Semaphore;
import java.util.concurrent.TimeUnit;
public class LatchAndSemaphore {
public static void main(String[] args) throws Exception {
CountDownLatch latch = new CountDownLatch(2);
ExecutorService pool = Executors.newFixedThreadPool(2);
pool.execute(() -> {
System.out.println("worker 1 ready");
latch.countDown();
});
pool.execute(() -> {
System.out.println("worker 2 ready");
latch.countDown();
});
latch.await(2, TimeUnit.SECONDS);
System.out.println("all workers signalled");
Semaphore semaphore = new Semaphore(2);
semaphore.acquire();
semaphore.acquire();
System.out.println("permits left : " + semaphore.availablePermits());
semaphore.release();
System.out.println("after release : " + semaphore.availablePermits());
pool.shutdown();
}
}The output is this.
worker 1 ready
worker 2 ready
all workers signalled
permits left : 0
after release : 1Notice that the semaphore starts with two permits and the program takes both of them, which leaves zero, and then it releases one. In a real service, that permit count is how you cap the number of the simultaneous calls to something expensive, like a database or a third-party API.
What about the shared collections?
The collections in java.util.concurrent are designed for the concurrent use. The class ConcurrentHashMap allows the reads and the writes from many threads without the external locking. The class CopyOnWriteArrayList is efficient when a list is read far more often than it is written, because the writes copy the whole list. The class BlockingQueue is the classic producer-consumer structure. The producers add to the tail, the consumers take from the head, and the queue handles the waiting.
What happens when a task inside an executor fails?
It depends on how you submitted it, and the difference catches the people out. The call submit returns a Future that captures the failure, so it surfaces when you call get. The call execute returns nothing, so a thrown exception goes to the uncaught exception handler of the thread, and the failure is only visible in the log of the executor.
import java.util.concurrent.ExecutionException;
import java.util.concurrent.ExecutorService;
import java.util.concurrent.Executors;
import java.util.concurrent.Future;
public class TaskFailures {
public static void main(String[] args) throws Exception {
ExecutorService pool = Executors.newFixedThreadPool(1);
Future<String> future = pool.submit(() -> {
throw new IllegalStateException("payment gateway refused");
});
try {
System.out.println(future.get());
} catch (ExecutionException e) {
System.out.println("captured : " + e.getCause().getMessage());
}
pool.shutdown();
}
}The output is this.
captured : payment gateway refusedNote that the call get throws an ExecutionException whose cause is the real exception. That wrapper is why you always unwrap it before you log it or react to it. Otherwise the log shows a wrapper class and the message is buried one level down.
There are two habits that keep the failures visible. You catch the exception inside the task when the task is supposed to handle it, and you return a result that says "failed" rather than letting it escape. And when you use the call execute, you attach an exception handler to the thread factory, so that a failure is logged with the task that it came from, instead of disappearing into the output of the default handler.
The habits to keep
To summarise the tools in one line each.
| Problem | Tool |
|---|---|
| Run many tasks on a pool of threads | ExecutorService |
| Get one result back | Future |
| Chain asynchronous steps | CompletableFuture |
| Wait for a group of tasks | CountDownLatch |
| Limit concurrent access | Semaphore |
| Share a map between threads | ConcurrentHashMap |
| Producer and consumer | BlockingQueue |
- Always shut an executor down when you are done with it.
- Use a bounded pool that is sized to the work, and not an unbounded one.
- Use Future.get with a timeout, so that a stuck task cannot hang the program.
- Prefer
invokeAllfor a batch of the similar tasks. - Use the concurrent collections rather than synchronising a plain one yourself.
- Keep the shared state in as few objects as possible.
Try it yourself: build an executor with four threads and submit twenty tasks that each sleep for a random short time and that return a value. Collect the results with invokeAll and print them in the submission order, and then again sorted by the value. Next, replace the batch call with a CompletableFuture chain and confirm that the final value matches.