Kubernetes is a Platform-as-a-Service (PAAS) build on top of (docker) containers, but with an additional unit of abstraction called a pod, which a) is its smallest unit of execution b) has a single external IP address, c) is a group of one or more containers where d) the group of containers are connected over a network namespace and e) each pod is isolated from others by network namespaces. Within a pod, different containers can see each other over different ports over a loopback interface. Within an instance, while different pods can see each other as different IP addresses. It has a control plane built on top of etcd, a consistent, distributed, highly available key-value store, which is an independent opensource CNCF project.
It is conceptually similar to Cloud Foundry, Mesos, OpenStack, Mirantis and similar abstraction layers.
A threat matrix for Kubernetes by MS – https://www.microsoft.com/security/blog/2020/04/02/attack-matrix-kubernetes/
From RSA’20, here’s a talk on ‘The future of Kubernetes attacks’ – https://youtu.be/CH7S5rE3j8w
From Coinbase, a blog on ‘Why Kubernetes is not part of our stack’ – https://blog.coinbase.com/container-technologies-at-coinbase-d4ae118dcb6c makes these points
- it needs a full-time compute team to maintain
- securing it is neither trivial nor well understood. SPIFFE, SPIRE, Envoy, Istio, OPA, service mesh are a few of the technologies.
This blog links to – https://k8s.af/
Another viewpoint – https://pythonspeed.com/articles/dont-need-kubernetes/
A counterpoint to the Coinbase blog – https://blog.kumina.nl/2020/07/in-response-to-container-technologies-at-coinbase/
K8S is based on a Controller pattern:
- Resources capture the desired state
- Current state is kept centralized in etcd, a distributed key-value store, similar to Consul
- Controllers reconcile current state with desired state
Pod is a top level resource, is the smallest deployment unit, and is a group of one or more containers described by a yaml file, similar to docker-compose.yml .
K8S Operator is a kind of resource manager, for Custom resources.
Spinnaker is a Continuous Delivery platform that itself runs on k8s as a set of pods which can be scaled up
A kubectl cheat sheet:
An article on cloud security https://medium.com/xm-cyber/having-fun-with-cloud-services-e281f8a7fe60 , which I think makes the point of why things are relatively complex to begin with.
A repo to list failing pods – https://github.com/edrevo/suspicious-pods
Exploring networking in k8s – https://dustinspecker.com/posts/how-do-kubernetes-and-docker-create-ip-addresses/
Plugin for Pod networking on EKS using ENIs – https://github.com/aws/amazon-vpc-cni-k8s