← Middleware: understand and operate the chain
08 / 12 · 20 MIN

Measure the service and locate degradation

Interpret percentiles, sampling, and signals by route.

Understand the concept

Average latency may stay low while a minority of requests take a long time. A percentile describes a distribution over a defined population and window; do not calculate a global p95 simply by averaging per-instance p95 values. Compare equivalent measurements and observe volume and errors together. A low-CPU server may be blocked waiting for I/O or a dependency. Metrics guide investigation but do not always identify a cause alone.

Apply and decide

In a fictional case, /lookup represents 99% of traffic and /close just 1%. If /close fails completely, overall success remains 99%, even though closing is unavailable. Separate critical operations and interpret indicators by impact. Use correlation identifiers in logs or traces with appropriate access. Do not use every unique request identifier as a label on every metric series: this multiplies combinations and cost.

Workplace application

To investigate degradation, keep population and window comparable. A critical route can fail completely while remaining almost invisible in an aggregate. Do not reconstruct global p95 from the average, even weighted, of individual percentiles. Use compatible histograms or suitable observations. Retain unique identifiers in controlled logs or traces; metric labels should have planned cardinality.

IN PRACTICE

The load balancer dashboard shows 99% success. Support confirms every batch close fails. Filter the affected route and population, follow a sample request, and compare behavior before and after the change.

Common pitfalls

Averaging percentiles; creating one metric series per request.

Related topics: Map the request path · Investigate the JVM, heap, and pauses

Take this idea with you

A healthy aggregate can hide an unavailable critical operation.

Create account

Reference: Histograms and summaries · DR Middleware 2026.4; HotSpot JDK 25; JDBC 25; PostgreSQL 18; RabbitMQ 4.3; OpenSSL 3.5; explicitly scoped runtime references