How to Read a Traceroute for DNS and Network Analysis
When a client application reports connection failures or latency spikes, SRE teams must determine where the network path is degrading. Is the failure caused by a bad routing path at a local ISP, an authoritative DNS outage, packet loss on a transit provider, or a firewall dropping traffic?
To isolate the root cause, network engineers run Traceroute and My Traceroute (MTR) tests. This guide details how to read traceroute data line by line, trace DNS delegation paths, and identify packet loss patterns using terminal commands.
1. Comparing Probe Protocols
Traceroute works by sending packets with incrementing Time-to-Live (TTL) values. Because intermediate routers or firewalls filter UDP and ICMP packets differently, matching the probe protocol to your application's actual traffic provides more accurate diagnostics:
| Probe Type | Command Syntax | Destination Response | Primary Diagnostic Value |
|---|---|---|---|
| UDP | traceroute pingzoapp.com | ICMP Port Unreachable | Traditional default path tracing |
| ICMP Echo | traceroute -I pingzoapp.com | ICMP Echo Reply | Diagnostic bypass of UDP filters |
| TCP SYN | traceroute -T -p 443 pingzoapp.com | TCP SYN-ACK or RST | Inspecting web service paths and firewalls |
| IPv6 Path | traceroute6 pingzoapp.com | ICMPv6 Time Exceeded | Isolating IPv6-specific transit degradation |
2. The Golden Rule of Packet Loss Triage
A common mistake is assuming that an asterisk (*) or high latency at an intermediate hop indicates a network fault. Routers prioritize forwarding transit traffic over generating ICMP responses. SRE teams use this logical metric to confirm a path fault:
[\text{Path Fault Evidence} \approx \text{Loss at Hop } N \text{ AND } \text{Persistent Loss at Hops } > N]
If Hop 4 shows (50%) packet loss but Hops 5, 6, and the final destination show (0%) loss, the packet loss is caused by router rate-limiting on CPU-generated ICMP packets. It is not a transit network bottleneck.
3. SRE Network Performance Threshold Matrix
Set alerts and benchmarks based on persistent changes downstream:
| Metric Indicator | Low Concern | Investigate | High Action Target |
|---|---|---|---|
| Persistent Packet Loss | (0%) | (> 1.0%) | (> 5.0%) |
| RTT Latency Increase | (< 10\text{ ms}) | (10\text{ ms} - 50\text{ ms}) | (> 50\text{ ms}) |
| Jitter Variance | (< 5\text{ ms}) | (5\text{ ms} - 20\text{ ms}) | (> 20\text{ ms}) |
| DNS Resolution Latency | (< 50\text{ ms}) | (50\text{ ms} - 200\text{ ms}) | (> 200\text{ ms}) |
| TCP Connect Latency | (< 100\text{ ms}) | (100\text{ ms} - 300\text{ ms}) | (> 300\text{ ms}) |
4. Diagnostic Network Commands and Packet Filters
Isolate network layers using CLI tools to verify path metrics, DNS delegation structures, and MTU sizes.
A. Trace DNS Delegation
Separate DNS zone delegation issues from network path issues:
# Trace the complete DNS zone delegation path
dig pingzoapp.com NS +trace
B. TCP Traceroute Check
Test web connection paths directly using TCP SYN packets on port 443:
sudo traceroute -T -p 443 -n pingzoapp.com
C. MTU Size Discovery
Identify Path MTU (PMTU) black holes where large packets are dropped by checking fragmentation thresholds:
# Test with standard MTU payload (1472 bytes + 28 bytes header = 1500 bytes)
ping -M do -s 1472 pingzoapp.com
# Test with smaller payload if the large check fails
ping -M do -s 1400 pingzoapp.com
D. tcpdump Packet Filter
Capture incoming ICMP Time Exceeded packets to verify traceroute responses:
sudo tcpdump -ni any 'icmp or (tcp port 443 and tcp[tcpflags] & tcp-syn != 0)'
[!TIP] Tip (DNS Verification): Use the DNS Lookup Tool to compare your target IP addresses across multiple recursive resolvers. Verify if a local resolver is directing users to an expired CDN IP before running complex network traceroutes.
5. Troubleshooting Network Path Outages
If your monitoring tools report regional connection failures, use this step-by-step diagnostic playbook:
- Isolate the failing protocol layer: Run
curlto determine if connection timeouts occur during DNS resolution, TCP connect, or TLS handshakes. - Compare IPv4 and IPv6 paths: Run separate traces to identify if routing degradations are isolated to your IPv6 (
AAAA) network path. - Audit the delegation chain: Verify that authoritative nameservers are responding to recursive resolvers without timeouts.
- Trace with MTR statistics: Run continuous checks to accumulate standard deviation and packet loss trends over 100 cycles:
mtr -rwzc 100 -i 0.2 pingzoapp.com - Examine intermediate router hostnames: Inspect reverse DNS names of hops to identify which ISP or transit provider is introducing packet loss.
- Verify WAF and security group rules: Check if firewalls are dropping TCP SYN packets, resulting in connection timeouts at the final hop.
- Identify MTU mismatches: Confirm if your TCP handshake finishes but subsequent large data transfers stall, indicating a Path MTU black-hole failure.
- Evaluate regional DNS steering: Verify if the recursive resolver is forwarding client subnet details (ECS) to allow optimal GeoDNS route steering.
- Confirm Anycast ingress points: Compare traceroute endpoints from multiple global locations to ensure users are routing to the nearest Anycast edge node.