Networking

Networking is a stack of encapsulation, addressing, forwarding, control messages, name resolution, encryption, proxy behavior, and state. The key is to know what each layer is responsible for and where to observe it when something breaks.

The practical command patterns are embedded in the topic pages where the failures happen. Start with Cross-Layer Incident Runbooks when the symptom crosses application, Linux, Kubernetes, DNS, proxy, and network boundaries.

Mental Model

A request from a browser to a service crosses several boundaries:

1. Application builds an HTTP request.
2. TLS may encrypt it.
3. TCP establishes a connection and segments bytes.
4. IP chooses a destination address and route.
5. Ethernet or Wi-Fi frames the packet for the local link.
6. The NIC transmits bits.
7. Switches, routers, firewalls, NAT, load balancers, and proxies may transform or forward traffic.

Each boundary can fail independently.

Data Plane and Control Plane

Many systems have a data plane and a control plane:

This separation matters during incidents. A control plane outage may prevent new changes while existing flows keep working. A data-plane failure may drop traffic even though APIs and dashboards look healthy.

Critical Subtopics

OSI Layers for Troubleshooting

The OSI model is not a perfect map of modern systems, but it is a useful checklist.

NIC and Link Layer

A NIC sends and receives frames. Important operational details:

• Link state can be up while the path is still broken upstream.
• MTU mismatches cause fragmentation or black-hole behavior.
• Offloads can hide what packets look like before the NIC modifies them.
• VLAN tagging changes which L2 domain a host participates in.
• Interface counters reveal drops, errors, overruns, and carrier problems.

Switches learn source MAC addresses and forward frames inside a VLAN. If the wrong VLAN is assigned, the host can have a perfectly healthy NIC and still be isolated from the expected gateway or peers. Access ports usually send untagged traffic for one VLAN; trunk ports carry multiple VLANs with 802.1Q tags.

ARP

ARP maps IPv4 addresses to MAC addresses on a local broadcast domain. If a host wants to send to an IP on the same subnet, it broadcasts “Who has this IP?” and caches the reply.

Quirks:

• ARP is local-link only; routers do not forward ARP broadcasts.
• Stale ARP entries can send traffic to the wrong MAC after failover.
• Duplicate IPs create unstable ARP behavior.
• Gratuitous ARP announces ownership and helps update neighbor caches.
• IPv6 uses Neighbor Discovery instead of ARP.

IP Routing

Routing answers: “Where should this packet go next?”

The host checks:

1. Is the destination local?
2. Is there a more specific route?
3. Use the default gateway if no specific route matches.

CIDR longest-prefix match is fundamental. 10.0.1.0/24 beats 10.0.0.0/8, and both beat 0.0.0.0/0.

TCP and UDP

TCP provides reliable byte streams with connection state, retransmission, ordering, flow control, and congestion control. Connection setup uses a three-way handshake: SYN, SYN-ACK, ACK.

UDP is message-oriented and connectionless. It is common for DNS, QUIC, telemetry, and latency-sensitive protocols. Reliability, ordering, and retransmission must be handled by the application or higher protocol when needed.

DHCP also uses UDP and is often the first protocol that proves whether a VLAN, switch port, relay agent, router interface, and address-management policy agree. For a focused runbook, see DHCP, Routers, and Switches.

NAT and Load Balancing

NAT rewrites addresses and/or ports. It is useful, but it creates state and can hide the original source. Load balancers may operate at:

• L4: route by IP/port and connection metadata.
• L7: route by HTTP host, path, headers, or protocol semantics.

Common pitfalls:

• Connection tracking tables fill up.
• Idle timeouts close connections.
• Source IP preservation changes routing requirements.
• Health checks do not match real user traffic.

MTU and Fragmentation

MTU is the largest frame payload for a link. Tunnels and overlays add headers, reducing effective MTU. If Path MTU Discovery fails because ICMP is blocked, large packets can disappear while small packets work.

Symptoms:

• SSH works but file transfer hangs.
• TLS handshake stalls after ClientHello or certificate exchange.
• HTTP small responses work but large responses fail.

Debugging Commands

ip addr
ip link
ip route
ip neigh
ss -tuna
traceroute example.com
ping -M do -s 1472 example.com
tcpdump -nn -i any host example.com

Debugging Order

1. Confirm local interface and IP.
2. Confirm switch port, VLAN, and trunk/access expectations.
3. Confirm route selection.
4. Confirm ARP/neighbor resolution for local next hop.
5. Confirm local firewall and listening sockets.
6. Test /etc/hosts, NSS, and DNS separately from connectivity.
7. Test TCP handshake or UDP request.
8. Capture packets on both sides when possible.
9. Check intermediate NAT/load balancer/firewall state.
10. Check ICMP, MTU, timeout, and proxy behavior when only large or long-lived requests fail.

Study Cards

Practice Deck

References

• RFC 826: Address Resolution Protocol
• RFC 4632: Classless Inter-domain Routing
• RFC 792: Internet Control Message Protocol
• RFC 768: User Datagram Protocol
• RFC 9293: Transmission Control Protocol
• RFC 9000: QUIC
• RFC 8446: The Transport Layer Security Protocol Version 1.3
• RFC 5280: PKIX Certificate and CRL Profile
• hosts(5) Linux manual page
• IEEE 802.1Q VLANs
• Linux kernel networking documentation
• VPNs and IPsec tunnels
• Firewalls, iptables, and Netfilter
• Forward and reverse proxies
• TechTarget: OSI model overview