Well, if you are reading this post then I assume you are already aware of docker and containers. My main intention is to provide details of NSX-T and Kubernetes integration. But before that I will give a very high level overview of these container/K8s.
So, what are container?
Containers offer a logical packaging mechanism in which applications can be abstracted from the environment in which they actually run. This decoupling allows container-based applications to be deployed easily and consistently, regardless of whether the target environment is a private data center, the public cloud, or even the developer’s personal laptop.
For those coming from virtualized environments, containers are often compared with virtual machines (VMs). You might already be familiar with VMs: a guest operating system such as Linux or Windows runs on top of a host operating system with virtualized access to the underlying hardware. Like virtual machines, containers allow you to package your application together with libraries and other dependencies, providing isolated environments for running your software services. Nonetheless, the similarities end here as containers offer a far more lightweight unit for developers and IT Ops teams to work with, carrying a myriad of benefits.
Now what is Kubernetes?
A single application is usually a group of several containers. Simplifying, and to understand a common application stereotype, think of a simple 3-tier application with a front-end, back-end and a database. This application could easily translate to three separate containers. For any given container run time engine, to quickly deploy such lightweight application there would be multiple steps involved. And what about day two operations? Like upgrading or scaling the application. For these reasons, and many others, container orchestration is almost, if not effectively, mandatory.
Kubernetes (commonly stylized as K8s) is an open-source container-orchestration system for automating deployment, scaling and management of containerized applications. It was originally designed by Google and is now maintained by the Cloud Native Computing Foundation.
Kubernetes is cluster-based solution, which involves a Kubernetes Master and Kubernetes Nodes (also known as Workers or Minions).
Kubernetes Master Components
- The Kubernetes API server validates and configures data for the API objects, which include Pods, Services, Replication Controllers, and others. The API server services REST operations and provides the front-end to the clusters shared state through which all other components interact. The API Server is the target for all external API clients like the K8s CLI client. These external and internal components interact with the API server performing different actions to resources.
- The Kubernetes Scheduler is a policy-rich, topology-aware, workload-specific function that significantly impacts availability, performance and capacity. The scheduler needs to take into account individual and collective resource requirements, quality of service requirements, policy constraints, affinity and anti-affinity specifications, data locality, inter-workload interference, deadlines, and so on.
- The Controller Manager is a daemon that embeds the core control loops shipped with Kubernetes. A controller is a control loop that watches the shared state of the cluster through the API server and makes changes attempting to move the current state towards the desired state.
- Etcd is the consistent and highly-available key value store used as Kubernetes backing store for all cluster data.
Kubernetes Node Components
- The Kubernetes API server validates and configures data for the API objects, which include Pods, Services, Replication Controllers, and others. The API server services REST operations and provides the front-end to the clusters shared state through which all other components interact. The API Server is the target for all external API clients like the K8s CLI client. These external and internal components interact with the API server performing different actions to resources.
- The Kubernetes Scheduler is a policy-rich, topology-aware, workload-specific function that significantly impacts availability, performance and capacity. The scheduler needs to take into account individual and collective resource requirements, quality of service requirements, policy constraints, affinity and anti-affinity specifications, data locality, inter-workload interference, deadlines, and so on.
- The Controller Manager is a daemon that embeds the core control loops shipped with Kubernetes. A controller is a control loop that watches the shared state of the cluster through the API server and makes changes attempting to move the current state towards the desired state.
- Etcd is the consistent and highly-available key value store used as Kubernetes backing store for all cluster data.
Kubernetes Namespace
Kubernetes Namespaces provide a scope for names. Names of resources need to be unique within a namespace, but not across namespaces. Namespaces are a way to divide cluster resources between multiple users (via resource quota). For networking folks, you can assume namespace is like a VRF on router.
In future versions of Kubernetes, objects in the same namespace will have the same access control policies by default.
It is not necessary to use multiple namespaces just to separate slightly different resources, such as different versions of the same software: use labels to distinguish resources within the same namespace.
Kubernetes PODs
A Kubernetes Pod (as in a pod of whales or pea pod) is a group of one or more containers. Containers within a pod share an IP address and port space, and can find each other via localhost. They can also communicate with each other using standard inter-process communications like SystemV semaphores or POSIX shared memory. Containers in a Pod also share the same data volumes. Pods are a model of the pattern of multiple cooperating processes which form a cohesive unit of service and serve as unit of deployment, horizontal scaling, and replication. Colocation (co-scheduling), shared fate (e.g. termination), coordinated replication, resource sharing, and dependency management are handled automatically for containers in a Pod.
Kubernetes Controllers
A Replication Controller enforces the ‘desired’ state of a collection of Pods. E.g. it makes sure that 4 Pods are always running in the cluster. If there are too many Pods, it will kill some. If there are too few, the Replication Controller will start more. Unlike manually created pods, the pods maintained by a Replication Controller are automatically replaced if they fail, get deleted, or are terminated.
A Replica Set is the next-generation Replication Controller. The only difference between a ReplicaSet and a Replication Controller right now is the selector support. ReplicaSet supports the new set-based selector requirements whereas a Replication Controller only supports equality-based selector requirements.
A Deployment Controller provides declarative updates for Pods and ReplicaSets. You describe a desired state in a Deployment object, and the Deployment controller changes the actual state to the desired state at a controlled rate. You can define Deployments to create new ReplicaSets, or to remove existing Deployments and adopt all their resources with new Deployments.
Kubernetes Service
A Kubernetes Service is an abstraction which defines a logical set of Pods and a policy by which to access them (sometimes called a micro-service). The set of Pods targeted by a Service is (usually) determined by a Label Selector.
The kube-proxy watches the Kubernetes master for the addition and removal of Service and Endpoints objects. For each Service, it installs iptables rules which capture traffic to the Service’s clusterIP (which is virtual) and Port and redirects that traffic to one of the Service’s backend sets. For each Endpoints object, it installs iptables rules which select a backend Pod. By default, the choice of backend is random.
With NSX-T and the NSX Container Plugin (NCP), we leverage the NSX Kube-Proxy, which is a daemon running on the Kubernetes Nodes. It replaces the native distributed east-west load balancer in Kubernetes (the Kube-Proxy using IPTables) with Open vSwitch (OVS) load-balancing features.
Kubernetes Ingress
The Kubernetes Ingress is an API object that manages external access to the services in a cluster, typically HTTP. Typically, services and pods have IPs only routable by the cluster network. All traffic that ends up at an edge router is either dropped or forwarded elsewhere. An Ingress is a collection of rules that allow inbound connections to reach the cluster services.
It can be configured to give services externally-reachable URLs, load balance traffic, terminate SSL, offer name based virtual hosting, and more. Users request ingress by POSTing the Ingress resource to the API server. An ingress controller is responsible for fulfilling the Ingress, usually with a load balancer, though it may also configure your edge router or additional frontends to help handle the traffic in an HA manner.
The most common open-source projects which allow us to do this are Nginx and HAProxy which looks like the image above.
Network Policies
A Kubernetes Network Policy is a specification of how groups of pods are allowed to communicate with each other and other network endpoints. Network Policy resources use labels to select pods and define rules which specify what traffic is allowed to the selected pods.
Kubernetes Network policies are implemented by the network plugin, so you must be using a networking solution which supports NetworkPolicy – simply creating the resource without a controller to implement it will have no effect.
By default, pods are non-isolated; they accept traffic from any source. Pods become isolated by having a Kubernetes Network Policy which selects them. Once there is a Kubernetes Network Policy in a namespace selecting a particular pod, that pod will reject any connections that are not allowed by a Kubernetes Network Policy. Other pods in the namespace that are not selected by a Kubernetes Network Policy will continue to accept all traffic.
WHAT NEXT?
Now that we have an idea of Kubernetes, lets dive into Kubernetes with NSX-T.
If you need an overview of NSX-T then click here https://why2cloud.home.blog/2019/03/28/nsx-t-in-detail/
what-the-cloud 