The Kubernetes networking model explained

Kubernetes networking builds on foundational networking concepts. At its core, every device on a network needs an IP address to send and receive data. An IP address is a unique identifier assigned to a device so that packets can be routed to the right destination. In Kubernetes, every pod gets its own IP address and every node has an IP address on the cluster network. Understanding how traffic moves between these addresses is what Kubernetes networking is about.

The four networking problems

Kubernetes defines four distinct communication scenarios, each solved differently:

1. Container-to-container communication within a pod
2. Pod-to-pod communication across nodes
3. Pod-to-service communication
4. External-to-service communication

Kubernetes does not implement networking itself. It defines a set of requirements that network plugins must satisfy, and delegates the actual implementation to CNI plugins. The core requirement is a flat network where every pod can communicate directly with every other pod without NAT, regardless of which node they are on.

Container-to-container communication

To understand how containers in a pod communicate, you need to understand network namespaces. A network namespace is a Linux feature that partitions the network layer into isolated stacks per process. Each namespace has its own network interfaces, routing tables, and firewall rules. Processes in different namespaces cannot see each other's network activity.

When a pod is created, Kubernetes creates a dedicated network namespace for it, stored at a path like /var/run/netns/mypodns. All containers in that pod are placed inside the same network namespace. This means they share the same IP address and the same port space.

Because they share a network namespace, containers in the same pod communicate via localhost. Container A listening on port 8080 is reachable from container B at localhost:8080. No CNI plugin is involved at this layer. Kubernetes handles it natively at pod creation time.

The pod network namespace is attached to the root network namespace of the node, which is how pods can communicate with the outside world beyond localhost.

This is also why two containers in the same pod cannot bind to the same port. They share the port space, so a port conflict will cause the second container to fail to start.

Pod-to-pod communication

Each pod has a virtual Ethernet pair (veth pair): one end lives inside the pod network namespace and the other connects to a bridge on the node. When pod A sends a packet to pod B on the same node, it travels across the veth pair, over the node bridge, and into pod B's network namespace.

When there is no matching MAC address for the destination IP on the bridge, the packet follows the default route through the node's ethernet device (eth0) and out onto the cluster network. This is how traffic crosses nodes.

On the destination node, the packet arrives through its ethernet device, travels across the bridge, through the veth pair, and into the target pod's network namespace. The journey is the same in reverse.

Cross-node routing is the job of the CNI plugin. Kubernetes requires that every pod IP is reachable from every other pod. How this is implemented depends on the CNI:

Flannel creates an overlay network using VXLAN encapsulation. Packets are wrapped in UDP and sent across the underlying network to the destination node, where they are unwrapped and delivered to the target pod.

Calico and Cilium use BGP route advertisement or eBPF programs to route traffic at the kernel level without encapsulation, which gives better performance.

Without a CNI plugin, pod-to-pod communication across nodes does not work at all.

For a full comparison of CNI options, see the CNI plugins guide.

Node-to-node networking

Nodes communicate over the underlying infrastructure network, whether that is a cloud VPC, a physical LAN, or any other network connecting the machines.

The CNI plugin configures routing tables on each node so that when a pod sends a packet to an IP on a different node, the node kernel knows how to forward it. With Flannel this is handled through a TUN device and subnet routing entries. With Calico and Cilium, BGP or eBPF programs intercept packets at the kernel level and route them without going through user space.

From an application developer perspective, pod IPs are routable across the entire cluster. You do not need to configure any routing manually. The CNI plugin handles all of it.

Pod-to-service communication

Pods are ephemeral. Their IP addresses change every time they are rescheduled. Hardcoding pod IPs in application configuration is not viable.

Services provide a stable virtual IP address called a ClusterIP. The ClusterIP does not change and is the correct address for pods to use when communicating with a workload. kube-proxy runs on every node and maintains iptables or IPVS rules that translate Service ClusterIP addresses to the current pod IPs behind the Service.

When pod A sends traffic to a Service ClusterIP, kube-proxy intercepts the packet and forwards it to one of the healthy pods backing the Service. If a pod is rescheduled and its IP changes, kube-proxy updates the rules automatically.

External-to-service communication

Traffic from outside the cluster reaches services through two mechanisms.

LoadBalancer Services request a cloud load balancer from the provider. The cloud provider provisions an external IP and routes traffic to the Service. This works on cloud providers that support the LoadBalancer service type, including Civo.

Ingress controllers handle HTTP and HTTPS routing at Layer 7. An Ingress resource defines routing rules based on hostname and URL path. The Ingress controller reads these rules and configures a reverse proxy, typically nginx or Traefik, to route external traffic to the correct Service. Traefik is installed by default in k3s clusters on Civo.