What Does the JIT Compiler Do?

The JIT compiler translates hot bytecode into native machine code, and it does it at run time rather than ahead of time, which is why the JVM can specialise the code for the machine it is actually running on and for the workload it is actually seeing.

The trigger is a counter. Each method has invocation and loop counters, and when a method crosses a threshold the compiler is queued to compile it. Until then it is interpreted, so a method that is called once costs almost nothing to compile. Once compiled, the compiled version replaces the interpreter for that method.

The compiler does considerably more than translate. It profiles the method as it executes — recording which branches are taken, which types flow into the parameters, and whether a call site always receives the same implementation. It then specialises: an inlined method that is always called with the same type gets that exact type baked in, a branch that is never taken is pruned, and a loop gets unrolled. This is why the JVM often outperforms a static ahead-of-time compiler on long-running workloads, and why it can be slower on short-lived ones.

Because the specialisation is based on observed behaviour, it can become wrong. If a call site that only ever received a String suddenly receives an Integer, the compiled code is invalid and the JVM throws it away, returning to the interpreter and recompiling. That is a deoptimisation, and it is the mechanism behind the occasional surprise that a long-running application suddenly gets slower for a moment.

Here is the same thing as runnable code:

public class Main {
    public static void main(String[] args) {
        // The JIT compiler kicks in once a method is called often enough.
        long start = System.nanoTime();
        long total = 0;
        for (int i = 0; i < 1; i++) {
            total += work(i);
        }
        long interpretedMs = (System.nanoTime() - start) / 1_000_000;

        start = System.nanoTime();
        for (int i = 0; i < 100_000; i++) {
            total += work(i);
        }
        long compiledMs = (System.nanoTime() - start) / 1_000_000;

        System.out.println("total        : " + total);
        System.out.println("first call ms: " + interpretedMs);
        System.out.println("hot loop ms  : " + compiledMs);
        System.out.println("the numbers vary by machine and JIT state");
    }

    static long work(int i) {
        return i * 3L + 1;
    }
}

Tiered compilation is the strategy of compiling a method in a cheap form first and only promoting it to the expensive optimising compiler if it stays hot. Without it, the JVM would have to choose between two bad options: compile everything immediately, which makes start-up slow and wastes effort on methods that run once, or wait too long before compiling, which makes the early life of the program slow.

The tiers work like this. Level zero is the interpreter. Level one compiles a method quickly with minimal optimisation — the point is to get native code running without spending much time compiling it. Level two is the full optimising compiler, with profiling-driven specialisation, inlining and loop optimisations, and it also emits code for the branches that were not taken so that a deoptimisation can jump back in. Level three compiles for the rare case where a method is genuinely hot but has not been profiled as expected. And level four is the warm tier for methods that ran hot once and are unlikely to be hot again.

A deoptimisation is what happens when the specialised code turns out to be invalid. The compiled code baked in an assumption — that a field is a particular type, that a branch is never taken, that a null check always succeeds — and the assumption is broken. The JVM discards the compiled version, restores the interpreter at that point, and lets the method be recompiled with the new information. It is why the first call through a rarely-taken branch is often slower than the hundredth, and why a program that suddenly becomes polymorphic at one call site can get slower rather than faster.

The CompilationMXBean exposes the compiler's state directly. getTotalCompilationTime gives you the cumulative milliseconds spent compiling, getLoadedClassCount and getTotalLoadedClassCount tell you how many classes have been loaded and how many in total, and the runtime MXBean gives you the time since the VM started, so you can compute a compilation rate.

import java.lang.management.CompilationMXBean;
import java.lang.management.ManagementFactory;

public class Main {
    public static void main(String[] args) {
        CompilationMXBean compilation = ManagementFactory.getCompilationMXBean();

        System.out.println("jit compiler : " + compilation.getName());
        System.out.println("total compile time ms : " + compilation.getTotalCompilationTime());
        System.out.println("timing supported     : "
                + compilation.isCompilationTimeMonitoringSupported());

        // Warm the JIT up, then watch the counters move.
        for (int i = 0; i < 200_000; i++) {
            hot(i);
        }
        System.out.println("after warmup, compile time ms : " + compilation.getTotalCompilationTime());
    }

    static long hot(int i) {
        return i * 2L + 7;
    }
}

The output shows the compiler's identity and its counters, then the same compilation time again after the hot loop has given the JIT something to chew on. The absolute numbers are specific to the machine and the workload, but the shape is what matters: compilation time grows while a method is being promoted, and then flattens out once it is compiled.

For deeper visibility the JVM gives you flags rather than an API. Compilation logging tells you which methods were compiled and at which tier, and the diagnostic commands let you dump the compiler's internal state. What you are usually looking for is a method that stays interpreted — one that is never hot enough to be compiled, which usually means it is called rarely or is being run through a megamorphic call site that defeats inlining.