Introduction to KWOK (Kubernetes without Kubelet)

There has always been a high learning curve with Kubernetes and its offerings. Features such as

Daemonsets
Nodes, Pods
Taints, Tolerations, Node Affinity
Pod Affinity, Anti affinity

are native to Kubernetes and learning these after learning how to deploy services was a challenge.

This challenge was not just in terms of the complexity of the tasks but also in terms of the playground where you can execute these and play around. To learn these concepts earlier you had to

Use something like minikube to have a makeshift cluster running on your laptop [there go your laptop resources 🥵]
Make your own cluster 🥶

So what if there was a way to simulate a Kubernetes cluster and its components?

The answer came in the form of KWOK. Which means Kubernetes without Kubelet.

Kubelet is the primary “node agent” that runs on each node. It can register the node with the Kubernetes apiserver. In laymen terms Kubelet is the daemon which converts a normal VM/Machine to a Kuberenete node and connects it to other Kubernetes components like scheduler through APIserver.

When we say Kubernetes without Kubelet we tend to create nodes without the kubelet agent and thus simulate a cluster of nodes.

Use Cases of KWOK -

Learning: You can use KWOK to learn about Kubernetes concepts and features without worrying about resource waste or other consequences.
Development: You can use KWOK to develop new features or tools for Kubernetes without accessing to a real cluster or requiring other components.
Testing:
- You can generate high loads on your cluster by creating many pods or services with different resource requests or limits.
- You can simulate node failures or network partitions by changing node conditions or randomly deleting nodes.
- You can test how your controller interacts with other components or features of Kubernetes by enabling different feature gates or API versions.

Now that was enough for the theory let’s jump to a practical exercise on how you can use KWOK.

Setting up a Kubernetes Cluster through Kwok

Pre-requisites

You should have these CLI installed in your system and have a working knowledge of the same.

docker
kubectl

KWOK tools

KWOK project is offered to developers in 2 separate binaries

kwok - It is a tool for simulating the lifecycle of fake nodes, pods, and other Kubernetes API resources.
kwokctl - It is a tool to streamline the creation and management of clusters, with nodes simulated by kwok .

One functionality of this is that you can add kwok generated fake nodes to your original Kubernetes cluster with actual nodes. This can be used to stress test your cluster with a number of fake nodes.

But as we do not need to do that for a local fake cluster running we will use its All-in-one image provided in the documentation here.

Step 1

Set up the cluster in your system through the docker image.

1docker run --rm -it -p 8080:8080 registry.k8s.io/kwok/cluster:v1.26.0

This will set up a fake cluster and start it using both kwok and kwokctl on its own. Giving you access to the cluster through the port 8080 as specified in the docker command.

The docker container logs will have the following info

1You can now use your cluster with:
2
3    kubectl config use-context kwok-kwok
4
5Thanks for using kwok!

So in your kubectl CLI config changes the context to connect to the cluster

1kubectl config use-context kwok-kwok

As noted above we have exposed the cluster at port 8080 which is not the default port for kubectl so we have to specify the port in every kubectl command now

To test the connection to your cluster let’s get the namespaces

1kubectl -s :8080 get ns

If you get the namespaces we are good to go

Step 2

Create a pod

As now we have a fake kube cluster running let’s start a fake pod here

1kubectl run test-pod --image=busybox -s :8080

Now let’s check for the created pod in our cluster

1kubectl get pods -s :8080

Wait… Why is our pod in a pending state? Let’s describe it to see.

1kubectl describe pod test-pod -s :8080

Yikes. We don’t have any node to schedule our pod on.

In a real Kubernetes cluster, you would never programmatically create a new node through kubectl or yaml files as a node[VM] is what the kubelet and other Kubernetes components run on. But as this is a simulated Kubernetes cluster we can and have to simulate nodes on our own.

Step 2

Creating a node in your cluster.

To set up a node create a YAML file in your system and copy the contents below to it.

1apiVersion: v1
2kind: Node
3metadata:
4  annotations:
5    node.alpha.kubernetes.io/ttl: "0"
6    kwok.x-k8s.io/node: fake
7  labels:
8    beta.kubernetes.io/arch: arm64
9    beta.kubernetes.io/os: linux
10    kubernetes.io/arch: arm64
11    kubernetes.io/hostname: kwok-node-0
12    kubernetes.io/os: linux
13    minikube.k8s.io/name: kwok-node-0
14    minikube.k8s.io/primary: "true"
15    node-role.kubernetes.io/control-plane: ""
16    node-role.kubernetes.io/master: ""
17    type: kwok
18  name: kwok-node-0
19status:
20  allocatable:
21    cpu: "4"
22    memory: 512Gi
23    pods: "110"
24  capacity:
25    cpu: "4"
26    memory: 512Gi
27    pods: "110"
28  nodeInfo:
29    architecture: arm64
30    kubeProxyVersion: fake
31    kubeletVersion: fake
32    operatingSystem: linux

After saving this content in node.yaml run the following command to apply this to your cluster

1kubectl apply -f ./node.yaml -s :8080 --validate=false

Now let’s check nodes and pods in our cluster

Now we see that we have a single node running in our cluster and the pod status has also changed from Pending to Running

We can see the pod events also by describing the pod again to see what happened.

So this ends the introduction to KWOK. And how you can set it up in your local system. Also how it honors the Kubernetes scheduler. You can try taints, tolerations and many other kube concepts on your fake cluster with fake pods now. Have Fun!

Introduction to KWOK (Kubernetes without Kubelet)