What Is Kubernetes Networking?
SHARE:
Content
Within a Kubernetes cluster, you will have Kubernetes networking which allows all the components in the cluster to communicate with each other. What does networking – a tool that so many engineers across the globe use, but so few understand – actually look like under the hood? Kubernetes was created to run distributed systems over a cluster of machines. Distributed systems make networking a central and necessary component of implementation. Understanding the Kubernetes networking model allows you to run, monitor, and troubleshoot applications running on Kubernetes in an optimal way.
How Does Kubernetes Networking Work?
Understanding networking is crucial to understanding how Kubernetes works. At any given time, there are several different ways that networking is happening in your cluster. The following are the 4 main modes of communication within a Kubernetes cluster:
- Pod-to-Pod
- Pod-to-Service
- Internet-to-Service
- Container-to-Container
Static Port allocation
Kubernetes optimizes compute resources by sharing machines between applications. This, however, requires that no two applications are using the same networking port. Setting ports manually and hoping that the two ports never cross is not the solution to this problem – you’ll end up playing whack-a-mole all day, and you still won’t be able to get your cluster fully up and running. Another option would be to use dynamic port allocation, although this would introduce its own set of complications.
Dynamic Port Allocation
Dynamic port allocation would allow every application to take ports as flags, but the API servers would have to know how to insert dynamic port numbers into configuration blocks. Rather than having to deal with ensuring that services know which ports to look for to talk to each other, Kubernetes takes a different approach to networking.
Which Networking Model Does Kubernetes Use?
The IP-Per-Pod Model
Kubernetes uses a networking model where every pod has its own assigned IP address. All containers in a single pod share the same network namespace and IP address. It’s important to note that the pods and IP addresses are ephemeral, meaning that if a pod goes down or has to restart for any reason, it will be assigned a new IP address. That’s not something that you will have to manage, however, and you will never need to create communication between pods or deal with mapping container ports to host ports.
In this networking model, pods can be treated like virtual machines (VMs) when it comes to port allocation, naming, load balancing, and migration. Kubernetes dictates the requirements for any networking implementation as follows:
- All pods can communicate with all other pods without using Network Address Translation (NAT).
- All Kubernetes nodes can communicate with all pods without NAT.
What Are the Key Principles of the Kubernetes Networking Model?
Let’s take a deep dive into the different networking components inside a Kubernetes cluster.
Pod-To-Pod Networking
In Kubernetes, each pod is assigned its own IP address and has its own network namespace. Pods can communicate with each other via the IP address. This model is very similar to VMs, where each pod can expose network ports and address other VMs on the network by IP address and port number.
Pod-To-Pod Networking – Pods on the Same Node
You can’t fully understand communication between pods without considering the nodes that are running to ensure that the pods are up. This communication is done through a virtual ethernet device in the node on which the pod runs. When a pod makes a request to the IP address of another node, it tunnels through the node’s virtual ethernet device. This connection has two sides: the pod’s side is named eth0, and the node’s side, which is the virtual component, is named vethx (for virtual ethernet) followed by a numerical value starting at index 0.
The connection is made possible by using a network bridge to connect two networks. When the request from the pod reaches the bridge, the bridge determines if all of the pods that are accessible through the same node have the correct IP address to handle the request; if so, the network connection is established between the two pods that are using the same node.
Pod-To-Pod Networking – Pods on Different Nodes
In the other kind of pod-to-pod communication, the pods are on different nodes. If you tried to use the network bridge method for this type of communication, it would fail. Note, however, that this can vary depending on which cloud provider and network plugins you’re using for your cluster.
The operation for communication between pods on different nodes will need to be done at the cluster level. Each cluster has a table that maps IP address ranges to the nodes. The pods on those nodes will have IP addresses assigned from those ranges. Kubernetes will assign addresses to pods on node 1 and node 2, as shown in the diagram below:
Now each node has its own way of communicating with the other nodes on the cluster. From this point, the communication between pods is similar to the way in which pods on the same node communicate with one another.
Pod-To-Service Networking
Because pods are ephemeral, a pod’s IP address can change. You could configure a pod to communicate directly with another pod, but it would need to keep track of the backend IP address, which could be tricky when the IP address changes every time the pod is restarted. To resolve this issue, you can link the two pods together using a Kubernetes resource called a Service.
A Kubernetes Service enables you to assign a group of pods a single, unique IP address called a clusterIP. You can make requests to the single clusterIP endpoint, and the service will proxy requests to a pod that belongs to the Service.
This communication happens via kube-proxy, which is a small process that Kubernetes runs inside every node. This process maps the Service’s virtual IP address to a group of pod IP addresses. Once the kube-proxy process has mapped the Service’s virtual IP to a pod’s IP, the request process proceeds as outlined above.
Internet-To-Service Networking
In order to manage a Kubernetes cluster that is able to run an application that can interact with the outside world, internet connectivity is crucial. You’ll need to ensure that you can get traffic to go 1) from your Kubernetes Service out to the internet and 2) from the internet to your Kubernetes Service.
Your cloud provider will supply the NAT gateway that is necessary for initiating this type of communication. The NAT gateway gives cloud resources that do not have public-facing IP addresses access to the internet without exposing them to incoming internet connections.
Let’s use the AWS Cloud provider as an example. In AWS, a Kubernetes cluster runs within a VPC. Every node is assigned a private IP address that is accessible from within the Kubernetes cluster. For traffic to be accessible outside of the cluster, you must attach an internet gateway to your VPC.
Both egress and ingress assist in setting up external access:
- Egress routes traffic from the node to an external connection.
- Ingress enables external clients to communicate with Kubernetes Services.
In order for egress to successfully route traffic from the node to an external application, an internet gateway is critical. The gateway is attached to a virtual private cloud (VPC), which is a secure and isolated private cloud computing environment hosted within a public cloud.
Ingress can allow some connections and block others from communicating with the Kubernetes Services based upon a set of pre-defined rules. The rules are established as IP tables that are either allowed or denied access to the cluster and Services.
Container-To-Container Networking
A container can communicate with another container since they share the same pod and, therefore, the same IP address. They communicate with each other by addressing the localhost. If a container wants to communicate with another container in the same pod, it can do so on port 8080 using the address localhost:8080.
Do Kubernetes Use NAT?
Kubernetes does not use Network Address Translation (NAT). All pods within a cluster can communicate with the other pods, and all nodes can communicate with all pods in the cluster without using NAT.
What Is Calico Networking?
Calico is an open source networking and network security tool that was established in 2016. Due to its increasing popularity, it has become a leader in the Kubernetes networking space.
Calico provides network connectivity between workloads on machines and enforces a network security policy between workloads. Calico has a flexible architecture that supports a wide range of deployment options using plugins.
Calico CNI (Container Network Interface)
Calico is available with its own Container Network Interface (CNI) plugins, cloud provider integrations, and a number of third-party CNI plugins. The Calico CNI connects pods to the host network namespace’s L3 routing using a virtual ethernet device pair (veth pair). Using the L3 architecture means that you can bypass the additional overhead of a network bridge, which would be necessary if you were operating from the L2 layer of the OSI model.
To learn more, you can explore this GitHub repo.
Conclusion
Networking in Kubernetes can be hard to wrap your head around sometimes. Now that you have a clearer understanding of the various means of communication inside Kubernetes clusters, you can take on these workplace challenges with confidence.