JMH is a framework for writing precise Java microbenchmarks that can avoid measurement deviations caused by JVM optimization. 1. Use Maven or Gradle to add jmh-core and jmh-generator-annprocess dependencies and enable annotation processing. 2. Write benchmark test methods and annotate configuration parameters with @Benchmark, @BenchmarkMode, @Warmup, @Measurement, @Fork, etc. 3. Prevent the return value of the time-consuming operation from being eliminated by JIT optimization through return or Blackhole.consume(). 4. Use @State(Scope.Thread) to define the state class and initialize the test data in the @Setup method to avoid contaminating the benchmark method. 5. Run the test through mvn clean install and java -jar and analyze the results of average time, error and units. 6. Using microbenchmarks only when comparing algorithms, data structures, or JVM configurations is not a substitute for system-wide load testing. Correct use of JMH can accurately answer the question of "what is faster to implement" and draw reliable conclusions.

When measuring the performance of Java code, simply using System.currentTimeMillis() or System.nanoTime() can lead to misleading results due to JVM optimizations like JIT compilation, dead code elimination, and warm-up effects. That's where the Java Microbenchmark Harness (JMH) comes in — a robust framework designed specifically for writing and running accurate microbenchmarks.

What Is JMH?

JMH is an open-source project developed as part of the OpenJDK. It helps developers write reliable benchmarks by handling common pitfalls automatically. It's widely used in performance-critical applications, libraries, and frameworks (like JDK itself) to compare small units of code under controlled conditions.

You don't just use JMH — you design benchmarks with it. It ensures proper warm-up, prevents code elimination, manages threading, and provide statistical analysis of results.

Setting Up JMH in Your Project

The easiest way to get started is by using Maven or Gradle.

With Maven:

Add this to your pom.xml :

 <dependency>
    <groupId>org.openjdk.jmh</groupId>
    <artifactId>jmh-core</artifactId>
    <version>1.37</version>
</dependency>
<dependency>
    <groupId>org.openjdk.jmh</groupId>
    <artifactId>jmh-generator-annprocess</artifactId>
    <version>1.37</version>
    <scope>provided</scope>
</dependency>

Then enable annotation processing in your IDE or build tool so that @Benchmark methods are processed correctly.

Alternatively, use the JMH Maven plugin to generate a self-contained benchmark JAR:

 mvn archetype:generate \
    -DinteractiveMode=false \
    -DarchetypeGroupId=org.openjdk.jmh \
    -DarchetypeArtifactId=jmh-java-benchmark-archetype\
    -DgroupId=com.example \
    -DartifactId=jmh-demo

This creates a project with everything preconfigured.

Writing Your First Benchmark

Here's a simple example comparing two ways of concatenating strings: StringBuilder vs String .

 @Benchmark
@BenchmarkMode(Mode.AverageTime)
@OutputTimeUnit(TimeUnit.NANOSECONDS)
@Fork(1)
@Warmup(iterations = 3, time = 1)
@Measurement(iterations = 5, time = 1)
public String testStringBuilder() {
    StringBuilder sb = new StringBuilder();
    sb.append("hello");
    sb.append("world");
    return sb.toString();
}

@Benchmark
@BenchmarkMode(Mode.AverageTime)
@OutputTimeUnit(TimeUnit.NANOSECONDS)
@Fork(1)
@Warmup(iterations = 3, time = 1)
@Measurement(iterations = 5, time = 1)
public String testStringConcat() {
    String s = "hello";
    s = "world";
    return s;
}

Key annotations explained:

• @Benchmark : Marks the method to be measured.
• @BenchmarkMode : Defines what metric to collect. Mode.AverageTime means nanoseconds per operation.
• @OutputTimeUnit : Sets the time unit for results.
• @Fork : Number of separate JVM processes to run. Helps isolate results.
• @Warmup : How many iterations to run before actual measurement (lets JIT optimize).
• @Measurement : How many iterations to use for collecting data.

Run the benchmark via:

 mvn clean install
java -jar target/benchmarks.jar

Common Pitfalls & Best Practices

Even with JMH, incorrect usage can skew results. Here are key things to watch:

• Avoid dead code elimination : If the result isn't used or returned, JVM may optimize the whole thing away. Always return a value from your benchmark method when appropriate.

   @Benchmark
    public String myBenchmark() {
        // do work
        return result; // important!
    }

• Use Blackhole to consume values when you don't want to return them:

   @Benchmark
    public void useBlackhole(Blackhole bh) {
        String result = expensiveOperation();
        bh.consume(result);
    }

• Don't put setup logic inside the benchmark method . Use @Setup :

   @State(Scope.Thread)
    public static class MyState {
        public List<String> data;
  
        @Setup
        public void setup() {
            data = Arrays.asList("a", "b", "c");
        }
    }
  
    @Benchmark
    public void processList(MyState state) {
        // use state.data
    }

• Understand Scope in @State :

  • Scope.Thread : Each thread gets its own instance.
  • Scope.Benchmark : Shared across all threads (use carefully).
  • Scope.Group : For threads within a thread group.

• Control parallelism with @Threads and @Fork to simulate real-world concurrency.

Interpreting Results

Sample output:

 Benchmark Mode Cnt Score Error Units
MyBenchmark.testStringBuilder avgt 5 8.234 ± 0.198 ns/op
MyBenchmark.testStringConcat avgt 5 12.451 ± 0.302 ns/op

This tells you that on average, StringBuilder was faster than string concatenation in this test.

JMH also reports throughput ( Mode.Throughput ), sample times, and even generates flame graphs if configured.

When Should You Use JMH?

• Comparing two algorithms (eg, sorting, hashing).
• Evaluating data structure performance (eg, ArrayList vs LinkedList ).
• Testing impact of JVM flags or GC settings.
• Validating performance assumptions before merge optimizations.

But remember: microbenchmarks aren't a substitute for real-world load testing . They show isolated behavior, not system-wide performance.

Basically, if you're trying to answer “Is this loop faster than that one?” or “Does this cache strategy reduce allocation?”, JMH is the right tool — as long as you use it right.

The above is the detailed content of Benchmarking Java Code Performance with JMH. For more information, please follow other related articles on the PHP Chinese website!