Extending APM Capabilities with End-User Data: RUM Integration

Extending APM Capabilities with End-User Data: RUM Integration

Application Performance Monitoring (APM) agents instrument backend servers, databases, and microservices. However, server-side spans terminate at the API gateway or reverse proxy boundary, leaving engineers blind to client-side network stalls, JavaScript execution bottlenecks, Core Web Vitals regressions, and CDN edge cache delays.

Site Reliability Engineers bridge this observability divide by integrating Real User Monitoring (RUM) directly with APM distributed tracing pipelines. By propagating W3C traceparent headers from client browser navigation contexts into backend microservices, teams establish end-to-end trace graphs that correlate frontend user impact with server-side SQL queries. This guide explores RUM data pipelines, OpenTelemetry browser instrumentation, and diagnostic workflows.

1. Latency Decomposition and Ingestion Capacity Models

RUM measures the total user-perceived duration ((T_{\text{RUM}})) across distinct physical phases:

[T_{\text{RUM}} = T_{\text{network}} + T_{\text{server}} + T_{\text{response}} + T_{\text{browser}}]

While APM accurately measures internal server execution ((T_{\text{APM}} \approx T_{\text{server}})), the client and transit overhead ((T_{\text{network+client}})) is derived by computing:

[T_{\text{network+client}} \approx T_{\text{RUM}} - T_{\text{APM}}]

To budget telemetry pipeline costs and prevent collector saturation, model monthly event volume ((\text{MonthlyEvents})):

[\text{MonthlyEvents} = \text{DailyUsers} \times \text{SessionsPerUser} \times \text{EventsPerSession} \times \text{SamplingRate} \times 30]

Tail-sampling strategies allow teams to ingest (100%) of failed transactions while sampling healthy sessions at (5%) to control storage expenditures.

2. RUM vs APM Signal Coverage Matrix

Combining client-side RUM with server-side APM eliminates blind spots across modern web architectures:

3. SRE Core Web Vitals and Frontend Threshold Matrix

Establish operational thresholds for client-side user experience and beacon reliability:

4. OpenTelemetry Browser SDK and Beacon Implementation

Capture Navigation Timing metrics and dispatch structured payloads using navigator.sendBeacon():

// Extract browser navigation timing metrics
const nav = performance.getEntriesByType("navigation")[0];

const rumPayload = {
  service: "pingzo-frontend",
  version: "2026.09.01",
  url: window.location.pathname,
  timings: {
    dns_duration_ms: nav.domainLookupEnd - nav.domainLookupStart,
    tcp_duration_ms: nav.connectEnd - nav.connectStart,
    tls_duration_ms: nav.secureConnectionStart ? nav.connectEnd - nav.secureConnectionStart : 0,
    ttfb_ms: nav.responseStart - nav.requestStart,
    download_ms: nav.responseEnd - nav.responseStart,
    dom_interactive_ms: nav.domInteractive
  }
};

// Dispatch non-blocking beacon to RUM collector
navigator.sendBeacon(
  "/api/rum/v1/events",
  new Blob([JSON.stringify(rumPayload)], { type: "application/json" })
);

Configure OpenTelemetry in the browser to inject W3C traceparent headers into outgoing fetch calls:

import { WebTracerProvider } from "@opentelemetry/sdk-trace-web";
import { BatchSpanProcessor } from "@opentelemetry/sdk-trace-base";
import { OTLPTraceExporter } from "@opentelemetry/exporter-trace-otlp-http";
import { registerInstrumentations } from "@opentelemetry/instrumentation";
import { FetchInstrumentation } from "@opentelemetry/instrumentation-fetch";

const provider = new WebTracerProvider();
const exporter = new OTLPTraceExporter({
  url: "https://telemetry.pingzoapp.com/v1/traces"
});

provider.addSpanProcessor(
  new BatchSpanProcessor(exporter, {
    maxQueueSize: 512,
    scheduledDelayMillis: 5000,
    maxExportBatchSize: 128
  })
);
provider.register();

registerInstrumentations({
  instrumentations: [
    new FetchInstrumentation({
      propagateTraceHeaderCorsUrls: [/.+/]
    })
  ]
});

Validate collector endpoints and TLS negotiation states using CLI tools:

# Test RUM telemetry ingestion endpoint using HTTP/2
curl --http2 -sS -o /dev/null -w 'Status: %{http_code} | Total: %{time_total}s\n' \
  -X POST "https://telemetry.pingzoapp.com/v1/traces" \
  -H "Content-Type: application/json" \
  -d '{"resourceSpans":[]}'

# Inspect collector TLS 1.3 certificate validity
openssl s_client -connect telemetry.pingzoapp.com:443 -servername telemetry.pingzoapp.com -tls1_3 </dev/null

[!NOTE] SRE Error Budget Alert: Translate frontend availability targets into allowable downtime allowances with our SLA Calculator. If client beacons report elevated connection latencies across regions, verify nameserver resolution using the DNS Lookup tool.

5. Troubleshooting RUM-to-APM Trace Correlation

Follow this structured runbook when client RUM sessions fail to link to backend APM traces:

1. Verify browser beacon dispatch: Inspect browser DevTools Network tabs to ensure /api/rum or /v1/traces endpoints return HTTP 200 OK or 202 Accepted.
2. Audit CORS and CSP headers: Ensure that Content-Security-Policy directives allow connect-src to the telemetry domain, and verify Access-Control-Allow-Origin headers.
3. Inspect W3C traceparent headers: Confirm that outgoing API requests include traceparent: 00-{traceId}-{spanId}-01 and that reverse proxies do not strip trace headers.
4. Validate backend span adoption: Check that backend API gateways extract incoming trace context to create child spans rather than generating new root trace IDs.
5. Audit trace sampling rules: Verify that backend APM collectors do not drop traces sampled by the frontend RUM SDK.
6. Evaluate collector queue depth: Monitor collector metrics (otelcol_exporter_enqueue_failed_spans) to identify queue overflows and ingestion throttling.
7. Check server-timing propagation: Ensure backend services emit Server-Timing: app;dur=42.1 headers so browser agents can record precise backend processing durations.
8. Correlate regressions with release tags: Filter RUM telemetry by git_sha and release.version to isolate frontend bundle regressions from backend infrastructure faults.
9. Validate recovery: Confirm that end-to-end trace graphs show connected client-to-database execution spans before closing the investigation.