Skip to main content

Monitoring Spring Boot Applications With Prometheus and Grafana

Gunnala SreekanthReddy11 min read
Monitoring Spring Boot Applications With Prometheus and Grafana

Logs tell you what happened after something went wrong. Metrics tell you whether anything is going wrong right now. Monitoring a Spring Boot application in production means having both, and the second one is the gap most teams leave open until an incident forces the issue.

This walkthrough builds a complete Spring Boot monitoring pipeline: Actuator exposes the numbers, Micrometer formats them, Prometheus scrapes and stores them, and Grafana draws them. Everything runs locally in Docker, and every configuration file is inline so there's nothing to download from a third party.

What this Spring Boot monitoring setup gives you

  • A Spring Boot app publishing JVM, HTTP, and custom business metrics on /actuator/prometheus
  • Prometheus scraping that endpoint every 15 seconds and retaining the history
  • A Grafana dashboard showing request rate, error rate, and p95 latency
  • An alert that fires when the 5xx rate crosses five percent
  • The two production mistakes that make this setup expensive, and how to avoid both

How Actuator, Micrometer, Prometheus, and Grafana fit together

It helps to be precise about what each tool does, because their responsibilities overlap in conversation but not in code.

Micrometer is a metrics facade, roughly what SLF4J is for logging. Your code records measurements against a MeterRegistry without knowing where they end up. Swap the registry implementation and the same instrumentation feeds Datadog, New Relic, or Prometheus instead.

Spring Boot Actuator exposes operational endpoints over HTTP, including the one that renders Micrometer's registry in Prometheus text format.

Prometheus is a time-series database that pulls. It calls your endpoint on a schedule and stores what it finds. This pull model is why you don't configure your app to push anywhere: it just publishes, and Prometheus comes to it.

Grafana queries Prometheus and renders the result. It stores no metrics of its own.

Step 1: Expose metrics from Spring Boot

Two dependencies. The starter brings in Actuator, and the registry teaches Micrometer to speak Prometheus.

<dependency>
  <groupId>org.springframework.boot</groupId>
  <artifactId>spring-boot-starter-actuator</artifactId>
</dependency>
<dependency>
  <groupId>io.micrometer</groupId>
  <artifactId>micrometer-registry-prometheus</artifactId>
  <scope>runtime</scope>
</dependency>

Or with Gradle:

implementation 'org.springframework.boot:spring-boot-starter-actuator'
runtimeOnly 'io.micrometer:micrometer-registry-prometheus'

Actuator ships with almost everything switched off, so the endpoint needs explicit opt-in. In application.yml:

spring:
  application:
    name: orders-service

management:
  endpoints:
    web:
      exposure:
        include: health,info,prometheus
  endpoint:
    health:
      show-details: when-authorized
  metrics:
    tags:
      application: ${spring.application.name}

That last block matters more than it looks. management.metrics.tags stamps a common tag onto every metric the app emits. Once you're running more than one service against one Prometheus, application is how you tell their numbers apart, and retrofitting it later means rewriting every dashboard query.

Start the app and confirm the endpoint responds:

curl -s localhost:8080/actuator/prometheus | head -20

You should see plain text with # HELP and # TYPE comments between the samples. If you get a 404, the endpoint isn't in the exposure.include list. If you get a 200 with almost nothing in it, the registry dependency is missing and Actuator is serving an empty page.

Step 2: Run Prometheus and Grafana

Use Compose rather than two docker run commands. It keeps the configuration in version control and survives a restart.

Save this as prometheus.yml:

global:
  scrape_interval: 15s
  evaluation_interval: 15s

scrape_configs:
  - job_name: 'spring-boot'
    metrics_path: '/actuator/prometheus'
    static_configs:
      - targets: ['host.docker.internal:8080']
        labels:
          application: 'orders-service'

And this as docker-compose.yml alongside it:

services:
  prometheus:
    image: prom/prometheus:latest
    container_name: prometheus
    ports:
      - "9090:9090"
    volumes:
      - ./prometheus.yml:/etc/prometheus/prometheus.yml
      - prometheus-data:/prometheus
    extra_hosts:
      - "host.docker.internal:host-gateway"

  grafana:
    image: grafana/grafana:latest
    container_name: grafana
    ports:
      - "3000:3000"
    environment:
      - GF_SECURITY_ADMIN_PASSWORD=changeme
    volumes:
      - grafana-data:/var/lib/grafana
    depends_on:
      - prometheus

volumes:
  prometheus-data:
  grafana-data:

Bring both up:

docker compose up -d

The networking gotcha that catches everyone

Prometheus runs inside a container. Your Spring Boot app, during development, runs on the host. Inside that container, localhost means the container itself, so a target of localhost:8080 will never resolve to your app.

host.docker.internal is the hostname that points back at the host machine. On Docker Desktop it resolves automatically. On Linux it doesn't, which is why the Compose file above declares extra_hosts with host-gateway. Leave that line out on a Linux box and the target sits permanently in DOWN state with a DNS error.

If neither option is available, the fallback is your machine's LAN address, such as 192.168.1.20:8080. It works, but it breaks whenever DHCP hands you a different address, so treat it as a last resort rather than the default.

Check it at http://localhost:9090/targets. You want UP next to the spring-boot job. If it's down, the error text on that page names the cause directly, and it's almost always this.

Step 3: Build the Grafana dashboard

Open http://localhost:3000 and sign in with admin and the password from the Compose file. Add Prometheus as a data source with the URL http://prometheus:9090. Use the service name, not localhost: Grafana is also in a container, and Compose puts both on the same network where prometheus resolves.

Rather than building panels by hand, import dashboard ID 4701, the JVM (Micrometer) dashboard. It gives you heap, garbage collection, thread counts, and CPU immediately, and it's a reasonable base to fork.

For request-level panels, these four queries cover most of what you'll want to see. Micrometer renames http.server.requests to http_server_requests_seconds on the way out, which is why the metric names look different from the Java side.

Request rate per second, by endpoint:

sum(rate(http_server_requests_seconds_count[5m])) by (uri)

Error rate as a proportion of all traffic:

sum(rate(http_server_requests_seconds_count{status=~"5.."}[5m]))
  / sum(rate(http_server_requests_seconds_count[5m]))

95th percentile latency, by endpoint:

histogram_quantile(0.95,
  sum(rate(http_server_requests_seconds_bucket[5m])) by (le, uri))

Heap usage against the maximum:

jvm_memory_used_bytes{area="heap"} / jvm_memory_max_bytes{area="heap"}

Step 4: Add custom Spring Boot metrics with Micrometer

JVM metrics tell you the service is alive. They don't tell you it's useful. A Spring Boot service can hold a flat heap and a clean garbage collection profile while quietly failing every payment it receives, which is why custom business metrics belong beside the default ones.

Inject MeterRegistry and build meters in the constructor, not per request. Registering a meter is a lookup on every call, and building it once keeps that off the hot path.

@Service
public class OrderService {

    private final Counter ordersPlaced;
    private final Counter ordersRejected;
    private final Timer checkoutTimer;

    public OrderService(MeterRegistry registry) {
        this.ordersPlaced = Counter.builder("orders.placed")
                .description("Orders that completed checkout")
                .register(registry);

        this.ordersRejected = Counter.builder("orders.rejected")
                .description("Orders refused at checkout")
                .tag("reason", "payment_declined")
                .register(registry);

        this.checkoutTimer = Timer.builder("checkout.duration")
                .description("End-to-end checkout time")
                .publishPercentileHistogram()
                .register(registry);
    }

    public Order placeOrder(OrderRequest request) {
        return checkoutTimer.record(() -> {
            Order order = process(request);
            ordersPlaced.increment();
            return order;
        });
    }
}

For a value that goes up and down rather than only up, such as a queue depth, register a gauge against the live object. Micrometer holds a weak reference and reads it at scrape time.

Gauge.builder("orders.queue.depth", orderQueue, Queue::size)
     .description("Orders waiting to be processed")
     .register(registry);

If you only want timing on a method and don't need the reference, @Timed is shorter, and works once you've declared a TimedAspect bean.

Step 5: Latency percentiles that survive aggregation

This is the step most Spring Boot monitoring tutorials skip, and it's the one that decides whether your latency numbers mean anything once you scale past a single instance.

Micrometer can publish precomputed percentiles, and it can publish histogram buckets. They look interchangeable on one instance. They aren't. Precomputed percentiles cannot be averaged across instances. The mean of three p95 values is not the p95 of the combined traffic, and any dashboard that adds them up is showing you a number with no meaning.

Histogram buckets aggregate correctly, because Prometheus sums the buckets first and computes the quantile afterwards. Configure buckets, not percentiles:

management:
  metrics:
    distribution:
      percentiles-histogram:
        http.server.requests: true
      slo:
        http.server.requests: 50ms,100ms,200ms,500ms,1s

The slo list adds explicit bucket boundaries at the thresholds you actually care about, which sharpens the quantile estimate around them. Set them near your service level objective rather than spreading them evenly.

Step 6: Alert on symptoms, not causes

A dashboard nobody is looking at catches nothing at three in the morning. Save this as alerts.yml:

groups:
  - name: spring-boot
    rules:
      - alert: HighErrorRate
        expr: |
          sum(rate(http_server_requests_seconds_count{status=~"5.."}[5m]))
            / sum(rate(http_server_requests_seconds_count[5m])) > 0.05
        for: 5m
        labels:
          severity: critical
        annotations:
          summary: "5xx rate above 5% for five minutes"

      - alert: SlowEndpoint
        expr: |
          histogram_quantile(0.95,
            sum(rate(http_server_requests_seconds_bucket[5m])) by (le, uri)) > 1
        for: 10m
        labels:
          severity: warning
        annotations:
          summary: "p95 latency above one second on {{ $labels.uri }}"

Reference it from prometheus.yml and mount it in the container:

rule_files:
  - 'alerts.yml'

The for clause is the part worth understanding. It requires the condition to hold continuously for that duration before the alert fires, which is what stops a single slow garbage collection pause from paging someone. Alert on user-visible symptoms such as error rate and latency. Leave heap and thread counts on the dashboard, where they belong for diagnosis after an alert has already fired.

Before you monitor Spring Boot in production

Don't expose the Actuator Prometheus endpoint publicly

/actuator/prometheus leaks a detailed map of your service: every endpoint path, dependency, and traffic pattern. The cleanest fix is to move Actuator to a separate port and leave that port off the public load balancer.

management:
  server:
    port: 9091
  endpoints:
    web:
      exposure:
        include: health,info,prometheus

Prometheus scrapes port 9091 inside your network. The internet only ever reaches 8080. If a separate port isn't practical, put Spring Security in front of the Actuator paths and give Prometheus a dedicated credential.

Watch your Prometheus tag cardinality

Prometheus creates a distinct time series for every unique combination of metric name and tag values. Tag a metric with something unbounded and you create an unbounded number of series, which is the single most reliable way to run a Prometheus server out of memory.

Never tag with user IDs, order IDs, session IDs, email addresses, or raw request paths. A tag of /orders/8837 creates one series per order. This is exactly why Micrometer's built-in HTTP metrics tag with the templated uri of /orders/{id} rather than the path that was actually requested.

The rule of thumb: a tag is safe when you can name every value it will ever take. Status codes, HTTP methods, endpoint templates, and region names are all fine. Anything generated per user or per request is not.

Common questions about Spring Boot monitoring

Do I need Micrometer if I already have Spring Boot Actuator?

You need both, and Actuator brings Micrometer with it. Actuator serves the HTTP endpoint; Micrometer collects and formats the measurements behind it. What you add manually is the registry, micrometer-registry-prometheus, which is what renders the data in the text format Prometheus expects.

Why is my Prometheus target DOWN when scraping Spring Boot?

In order of how often it's the cause: the scrape target is localhost from inside a container, which resolves to the container rather than your app; prometheus isn't in the Actuator exposure.include list, giving a 404; or you're on Linux without the host-gateway entry, so host.docker.internal won't resolve. The error text on the /targets page distinguishes all three.

Is it safe to expose /actuator/prometheus in production?

Not on a public interface. The endpoint publishes every route, dependency, and traffic pattern in your service. Move Actuator to a separate management port that your load balancer doesn't route to, or put Spring Security in front of it with a dedicated credential for Prometheus.

How often should Prometheus scrape a Spring Boot application?

15 seconds suits most services and is the default used above. Going below five seconds multiplies your storage and cardinality for resolution you rarely act on. The interval should be shorter than the shortest for duration in your alert rules, otherwise an alert can't gather enough samples to fire.

Which Grafana dashboard works with Spring Boot metrics?

Dashboard ID 4701, JVM (Micrometer), covers heap, garbage collection, threads, and CPU out of the box. It reads the standard Micrometer metric names, so it works as soon as your scrape target is up. Treat it as a starting point and add your own request-rate, error-rate, and latency panels beside it.

Where to take your Spring Boot monitoring next

You now have a Spring Boot monitoring pipeline that answers "is it healthy right now?" in one glance and pages someone when it isn't. Three natural extensions from here:

  • Alertmanager routes those alerts to Slack, PagerDuty, or email, with grouping and silencing so one incident doesn't produce forty notifications
  • Grafana Loki puts logs beside metrics, so a spike on a graph links straight to the log lines underneath it
  • Micrometer Tracing adds distributed traces, which is where you go once "which service is slow?" stops being obvious

The setup above is the foundation each of those builds on. Get the metrics right first.

Ready to build something that matters?

We solve problems that don't have Stack Overflow answers. Let's talk.

Book a Discovery Call