Kubernetes Services and EndpointSlices

Services decouple clients from changing Pods. The Service object defines a stable frontend; EndpointSlices describe the current backend endpoints. The datapath is then implemented by kube-proxy or a replacement such as an eBPF-based CNI datapath.

Command Examples

kubectl get svc <service> -o wide
kubectl describe svc <service>
kubectl get endpointslice -l kubernetes.io/service-name=<service>
kubectl get pods -l <selector> -o wide
kubectl describe pod <pod>
kubectl get events --sort-by=.lastTimestamp

Example output and meaning:

Service vs EndpointSlice

Service Object

The Service defines:

• selector labels,
• port and protocol,
• targetPort,
• type such as ClusterIP, NodePort, LoadBalancer, or ExternalName,
• optional session affinity,
• traffic policy fields such as externalTrafficPolicy and internalTrafficPolicy.

Selector drift is a common outage. A Service can exist, have a ClusterIP, and still have no usable backends if labels do not match ready Pods.

Selector mismatch example:

apiVersion: v1
kind: Service
metadata:
  name: web
spec:
  selector:
    app: frontend
  ports:
    - port: 80
      targetPort: http
---
apiVersion: apps/v1
kind: Deployment
metadata:
  name: web
spec:
  selector:
    matchLabels:
      app: web
  template:
    metadata:
      labels:
        app: web
    spec:
      containers:
        - name: web
          image: nginx
          ports:
            - name: http
              containerPort: 80

This Service selects app=frontend, but the Pods are labeled app=web, so EndpointSlices remain empty. Fix either the Service selector or the Pod labels, then verify:

kubectl get svc web -o jsonpath='{.spec.selector}{"\n"}'
kubectl get pods -l app=frontend
kubectl get pods -l app=web
kubectl get endpointslice -l kubernetes.io/service-name=web -o wide

EndpointSlices

EndpointSlices are the scalable backend source of truth for Services. They group endpoints by address family, protocol, port, and Service. Endpoint conditions include readiness and terminating state, which affects whether Service traffic should be sent to a Pod.

flowchart LR
  SVC[Service selector app=web] --> EPS[EndpointSlice controller]
  POD1[Pod web-1 Ready] --> EPS
  POD2[Pod web-2 NotReady] --> EPS
  EPS --> Ready[ready endpoint used for traffic]
  EPS --> NotReady[not-ready / terminating condition excluded or deprioritized]
  Ready --> DataPath[kube-proxy, IPVS, nftables, or eBPF datapath]

Operational details:

• EndpointSlices are normally created for selector-based Services.
• Ready endpoints usually map to Pods passing readiness.
• Headless Services publish endpoint records directly.
• Stateful workloads such as NATS often use a normal Service for clients and a headless Service for stable peer DNS names.
• Dual-stack Services may have separate EndpointSlices by address family.
• Older Endpoints objects are not the main scalability path.

kube-proxy and Service Virtual IPs

For non-ExternalName Services, kube-proxy implements virtual IP behavior by watching Services and EndpointSlices. Depending on cluster mode, that implementation may use iptables, IPVS, nftables, eBPF, or a CNI-integrated datapath.

Debugging implication: the API object can be correct while the node datapath is stale, missing rules, or blocked by host firewall policy.

Practical node checks depend on mode:

kubectl -n kube-system get configmap kube-proxy -o yaml
kubectl -n kube-system logs -l k8s-app=kube-proxy --since=10m
iptables-save | grep KUBE-SVC | head
ipvsadm -Ln 2>/dev/null | head
nft list ruleset | grep -i kube | head
conntrack -S

For eBPF service replacement, use the CNI’s own status and map-inspection commands. Do not assume iptables or IPVS will show the active Service path.

LoadBalancer and NodePort Details

NodePort opens a port on nodes. LoadBalancer asks infrastructure integration to publish an external address and point it at the Service. On bare metal this requires something like MetalLB or another controller.

externalTrafficPolicy: Local can preserve client source IP, but it only sends traffic to nodes with local ready endpoints. That improves source visibility but can create uneven traffic and black holes if health checks do not account for local endpoints.

Debugging Flow

1. Confirm the Service selector matches the intended Pods.
2. Confirm Pods are Ready and expose the expected container port.
3. Inspect EndpointSlices and endpoint conditions.
4. Test the Service name and ClusterIP from a debug Pod.
5. Test direct Pod IP only to isolate Service datapath from workload behavior.
6. Check kube-proxy or CNI datapath logs on affected nodes.
7. For LoadBalancer, check external address assignment, health checks, node ports, and source IP policy.
8. Test from two different nodes to catch node-local stale rules, BPF map drift, or conntrack pressure.

Study Cards

References

• Kubernetes Services
• EndpointSlices
• Virtual IPs and Service Proxies
• Debug Services