What is the JIT compiler in Java and how does it improve performance?

1. Short Answer

The Just-In-Time (JIT) compiler is a runtime component of the JVM that translates frequently executed Java bytecode into native machine code. Instead of interpreting every bytecode instruction on every execution, the JIT compiles "hot" methods once and caches the native code, allowing subsequent calls to run at near-native CPU speed.

2. Interpretation vs JIT Compilation

When you run a Java program, the JVM starts by interpreting bytecode:

• The interpreter reads each bytecode instruction one at a time and executes it.
• This is slow because the same instruction is decoded on every invocation.

The JIT compiler monitors execution and identifies hot spots — methods or loops that run many times. Once a method crosses an invocation threshold, the JIT compiles it to native machine code. All future calls to that method execute the compiled native code directly.

// Execution flow of a Java program:
//
//  Source (.java)
//       |
//   javac compiler
//       |
//  Bytecode (.class)
//       |
//  JVM starts — Interpreter runs bytecode
//       |
//  (method called many times → JIT kicks in)
//       |
//  JIT compiles bytecode → native machine code
//       |
//  CPU executes native code directly (fast!)

3. HotSpot JVM and Tiered Compilation

Oracle's HotSpot JVM (also used in OpenJDK) implements tiered compilation, which uses two JIT compilers with different optimization trade-offs:

3.1 C1 Compiler (Client Compiler)

C1 compiles quickly with minimal optimization. It is designed for fast startup. Methods first compiled by C1 execute faster than interpreted code but are not maximally optimized. C1 also instruments the code to collect runtime profiling data (call frequencies, type information).

3.2 C2 Compiler (Server Compiler)

C2 is a highly optimizing compiler that uses the profiling data collected by C1. It performs speculative optimizations based on observed runtime behavior — for example, if a virtual method call is always dispatched to the same concrete class, C2 can devirtualize it and inline it. If the assumption is later violated (a different class appears), the JVM deoptimizes and falls back to the interpreter.

4. Key JIT Optimizations

4.1 Method Inlining

The most impactful JIT optimization. The JVM replaces a method call with the body of the called method, eliminating call overhead and enabling further optimizations. The JVM typically inlines methods whose bytecode size is under ~35 bytes (tunable via -XX:MaxInlineSize).

// Before inlining:
int result = add(a, b);  // method call overhead

private int add(int x, int y) { return x + y; }

// After inlining (conceptually — done in bytecode):
int result = a + b;  // no call overhead, no frame allocation

4.2 Escape Analysis

The JIT analyzes whether an object created inside a method "escapes" — is returned, stored in a field, or passed to another thread. If not, the object can be allocated on the stack (avoiding GC) or have its fields replaced with scalar local variables (scalar replacement).

// JIT may apply scalar replacement here:
// Point object never escapes sum() — the JIT can eliminate the allocation
public int sum(int x, int y) {
    Point p = new Point(x, y);  // may be eliminated by escape analysis
    return p.x + p.y;           // replaced with: return x + y;
}

4.3 On-Stack Replacement (OSR)

OSR allows the JVM to switch from interpreted to JIT-compiled code in the middle of a long-running loop, without waiting for the method to return. This is critical for methods with very long loops that would otherwise run interpreted for a long time before the JIT kicks in.

4.4 Dead Code Elimination

The JIT removes code whose result is never used, or branches that can never be taken based on profiling.

4.5 Loop Unrolling and Vectorization

The JIT can unroll loops (execute multiple iterations per loop step) and, with SIMD CPU instructions, vectorize array operations for significant throughput gains.

4.6 Devirtualization

If profiling shows that a virtual method call always dispatches to one concrete type, the JIT can replace the virtual dispatch with a direct call (and inline it). This is called speculative devirtualization.

5. Observing JIT in Action

You can observe JIT compilation with JVM flags:

# Print every method as it is JIT-compiled
java -XX:+PrintCompilation MyApp

# Sample output:
#  timestamp  compile_id  flags  tier  method_name     size deopt
     164    1       3       java.lang.String::hashCode (55 bytes)
     201    2       4       com.example.MyClass::hotMethod (120 bytes)
     215    2       4  %    com.example.MyClass::hotLoop (86 bytes)  ← OSR

# Flags meaning:
#   %  = On-Stack Replacement (OSR)
#   !  = method has exception handlers
#   s  = synchronized method
#   b  = blocking compiler (rare)
#   n  = native wrapper compiled

# Show inlining decisions
java -XX:+UnlockDiagnosticVMOptions -XX:+PrintInlining MyApp

# Disable JIT entirely (for benchmarking baseline)
java -Xint MyApp

# Force all code through C1 only (no C2)
java -XX:TieredStopAtLevel=1 MyApp

# Force immediate C2 compilation (skip C1 profiling — usually slower overall)
java -XX:-TieredCompilation MyApp

6. AOT vs JIT

7. Interview Context

Strong answers mention:

• The two-compiler approach: C1 for fast startup, C2 for peak performance.
• Profiling data collected at lower tiers drives C2 optimizations at tier 4.
• Deoptimization: when a speculative assumption (e.g., monomorphic call site) is violated, the JVM can deoptimize back to interpreter and recompile.
• JMH for correct benchmarking — warmup lets JIT reach steady state before measuring.
• GraalVM as a modern alternative JIT/AOT platform.

8. Common Pitfalls

• Benchmarking without warmup — measuring performance before JIT kicks in gives misleadingly slow results. Always warm up your code before measuring.
• Thinking JIT is always faster than AOT — JIT has better peak throughput for long-running apps due to runtime profiling; AOT wins on startup time and memory.
• Disabling JIT "to reduce CPU usage" — running with -Xint drastically reduces throughput. This is almost never the right approach.
• Not understanding deoptimization — if a class hierarchy changes (e.g., a new subclass is loaded at runtime), the JVM must deoptimize previously inlined virtual calls.

9. Conclusion

The JIT compiler is the engine behind Java's competitive performance. By monitoring runtime behavior, identifying hot methods, and applying aggressive optimizations like inlining, escape analysis, devirtualization, and loop vectorization, the JIT makes Java programs run at speeds comparable to statically compiled languages — often after just a few seconds of warmup. Understanding tiered compilation (C1 → C2) and key optimizations will help you write JIT-friendly code and reason confidently about Java performance in interviews.