What tcpdump filter would you use to isolate TLS handshake failures on port 443, and how do you read the result?
Quick Answer
Filter on TCP SYN and RST flags on port 443 to see connection-level failures, and separately capture the ClientHello/ServerHello exchange (without decrypting) to see whether the handshake even starts and where it stops.
Detailed Answer
For connection-level failures (server never responds, or resets immediately), filter on the TCP flags rather than trying to parse TLS: SYN packets with no matching SYN-ACK indicate the server is unreachable or a firewall is dropping the connection silently; a RST shortly after SYN-ACK usually means the listener rejected the connection or an intermediate device is tearing it down. For handshake-level failures (TCP connects fine but TLS negotiation fails), you do not need to decrypt anything — tcpdump can show you how far the handshake gets by packet count and size: a ClientHello followed immediately by a RST or a TLS alert record (visible as a small packet right after the ClientHello) tells you the failure is in TLS negotiation, not the network path. For deeper protocol detail (SNI, cipher suites, alert codes) hand the same capture to tshark -Y tls or Wireshark rather than trying to decode it by hand in tcpdump.
Always correlate against certificate lifecycle: a cluster of resets right after a deploy or at a predictable time each day/week often lines up with a certificate rotation, an expired intermediate cert, or a config reload that briefly serves the wrong cert chain.
Code Example
# Connection-level: SYNs and RSTs only, on port 443 sudo tcpdump -i eth0 -nn "tcp port 443 and (tcp[tcpflags] & (tcp-syn|tcp-rst) != 0)" # Capture full handshake for deeper analysis (still encrypted payload, # but frame sizes/order show where negotiation stops) sudo tcpdump -i eth0 -nn -s 0 -w tls_handshake.pcap "tcp port 443" # Hand off for TLS-aware decoding (SNI, alert codes, cipher suite) tshark -r tls_handshake.pcap -Y "tls.handshake or tls.alert_message"
Interview Tip
Show you know tcpdump does not need to decrypt TLS to be useful — flags, packet sizes, and timing already narrow down whether the failure is network, listener, or handshake.