RUNNING KUBERNETES CLUSTERS ON DOCKER CONTAINERS
Ops Engineer, there are occasions when running Kubernetes clusters on local machines becomes a preferable choice, especially when aiming to avoid the accumulation of charges from various cloud providers.
This article explores two optimal and straightforward methods for deploying your Kubernetes clusters within Docker containers, offering a cost-effective and efficient approach.
For this guide, I am going to be using a Ubuntu-based Linux system.
to check your Ubuntu flavor, run any of the below command from the terminal
$ cat /etc/os-release
OR
$ lsb_release -a
Install Docker Container RuntimeEngine
Install Docker CE on the system.
$ curl -fsSL get.docker.com -o get-docker.sh
$ sudo sh get-docker.sh
Docker service started automatically after installation
Verify the status of the Docker with
$ systemctl status docker
Output
● docker.service - Docker Application Container Engine
Loaded: loaded (/lib/systemd/system/docker.service; enabled; vendor preset: enabled)
Active: active (running) since Wed 2023-03-08 10:04:28 UTC; 24s ago
TriggeredBy: ● docker.socket
Docs: https://docs.docker.com
Main PID: 2005 (dockerd)
Tasks: 8
Memory: 24.6M
CPU: 451ms
CGroup: /system.slice/docker.service
└─2005 /usr/bin/dockerd -H fd:// --containerd=/run/containerd/containerd.sock
Verify the Docker service installation by checking the version;
$ docker --version
Docker version 23.0.1, build a5ee5b1
With all prerequisites met, the next step involves installing the tool necessary for crafting Kubernetes clusters within Docker containers.
✨ Let's start with KIND TOOL,
Run the following commands on the terminal
To Install the kind tool on Linux / macOS
Linux Os:
$ curl -s https://api.github.com/repos/kubernetes-sigs/kind/releases/latest| grep browser_download_url | grep kind-linux-amd64 | cut -d '"' -f 4 | wget -qi
$ chmod a+x kind-linux-amd64
$ sudo mv kind-linux-amd64 /usr/local/bin/kind-
Mac Os:
# Using brew
$ brew install kind
# Using Curl
$ curl -s https://api.github.com/repos/kubernetes-sigs/kind/releases/latest| grep browser_download_url | grep darwin-arm64 | cut -d '"' -f 4 | wget -qi -
Check installed version
$ kind version
kind v0.16.0 go1.19.1 linux/amd64
Run Kubernetes cluster in Docker containers with Kind tool
$ sudo kind create cluster
Output
Creating cluster "kind" ...
✓ Ensuring node image (kindest/node:v1.25.2) 🖼
✓ Preparing nodes 📦
✓ Writing configuration 📜
✓ Starting control-plane 🕹️
✓ Installing CNI 🔌
✓ Installing StorageClass 💾
Set kubectl context to "kind-kind"
You can now use your cluster with:
kubectl cluster-info --context kind-kind
Thanks for using kind! 😊
To view the container created by Kind, run
$ sudo docker ps
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
f0c2eff57d05 kindest/node:v1.25.2 "/usr/local/bin/entr…" 3 minutes ago Up 3 minutes 127.0.0.1:34989->6443/tcp kind-control-plane
kind commands can be used to manage the clusters, nodes, and kubeconfig.
$ kind help
kind creates and manages local Kubernetes clusters using Docker container 'nodes
Usage:
kind [command]
Available Commands:
build Build one of [node-image]
completion Output shell completion code for the specified shell (bash, zsh or fish)
create Creates one of [cluster]
delete Deletes one of [cluster]
export Exports one of [kubeconfig, logs]
get Gets one of [clusters, nodes, kubeconfig]
help Help about any command
load Loads images into nodes
version Prints the kind CLI version
Flags:
-h, --help help for kind
--loglevel string DEPRECATED: see -v instead
-q, --quiet silence all stderr output
-v, --verbosity int32 info log verbosity, higher value produces more output
Use "kind [command] --help" for more information about a command.
To delete created Kubernetes cluster
kind delete cluster
Manage kind Cluster with kubectl
Install Kubectl on Linux
For command-line access to the Kubernetes cluster, the essential tool is kubectl. Installing this tool is a straightforward process accomplished by executing the following command:
$ curl -LO https://storage.googleapis.com/kubernetes-release/release/`curl -s https://storage.googleapis.com/kubernetes-release/release/stable.txt`/bin/linux/amd64/kubectl
Make the kubectl binary executable.
$ chmod +x ./kubectl
Move the binary into your PATH.
Recommended by LinkedIn
sudo mv ./kubectl /usr/local/bin/kubectl
To check the version installed
$ kubectl version --client
Client Version: version.Info{Major:"1", Minor:"25", GitVersion:"v1.25.3", GitCommit:"434bfd82814af038ad94d62ebe59b133fcb50506", GitTreeState:"clean", BuildDate:"2022-10-12T10:57:26Z", GoVersion:"go1.19.2", Compiler:"gc", Platform:"linux/amd64"}
Kustomize Version: v4.5.7
For zsh users, run:
source <(kubectl completion zsh
echo "if [ $commands[kubectl] ]; then source <(kubectl completion zsh); fi" >> ~/.zshrc)
Adjust the `.kube/config` file to incorporate the configuration of the freshly established Kubernetes cluster.
$ mkdir ~/.kube
$ sudo cp /root/.kube/config ~/.kube
$ sudo chown $USER .kube/config
Run the following commands to check the kubectl configuration
$ kubectl cluster-info
Kubernetes control plane is running at https://127.0.0.1:34989
CoreDNS is running at https://127.0.0.1:34989/api/v1/namespaces/kube-system/services/kube-dns:dns/proxy
To further debug and diagnose cluster problems, use 'kubectl cluster-info dump'.
$ kubectl config get-clusters
NAME
kind-kind
$ kubectl config get-contexts
CURRENT NAME CLUSTER AUTHINFO NAMESPACE
* kind-kind kind-kind kind-kind
$ kubectl get nodes
NAME STATUS ROLES AGE VERSION
kind-control-plane Ready control-plane 11m v1.25.2
$ kubectl get namespaces
NAME STATUS AGE
default Active 11m
kube-node-lease Active 11m
kube-public Active 11m
kube-system Active 11m
local-path-storage Active 11m
$ kubectl get pods -n kube-system
NAME READY STATUS RESTARTS AGE
coredns-565d847f94-4f859 1/1 Running 0 11m
coredns-565d847f94-65vfd 1/1 Running 0 11m
etcd-kind-control-plane 1/1 Running 0 11m
kindnet-wqm6j 1/1 Running 0 11m
kube-apiserver-kind-control-plane 1/1 Running 0 11m
kube-controller-manager-kind-control-plane 1/1 Running 0 11m
kube-proxy-vx7tv 1/1 Running 0 11m
kube-scheduler-kind-control-plane 1/1 Running 0 11m
✨ For k3d Tool
Install k3d on your Linux system
There are a number of options you can use to install k3s.
See below the installation steps for your particular Linux operating system.
Using the installation script to get the latest version:
# With wget
$ wget -q -O - https://raw.githubusercontent.com/rancher/k3d/main/install.sh | bash
# Or with curl
$ curl -s https://raw.githubusercontent.com/rancher/k3d/main/install.sh | bash
Install on macOS with Homebrew:
$ brew install k3d
Confirm installed version
$ k3d version
k3d version v5.4.8
k3s version v1.25.6-k3s1 (default)
Once you have installed it, you can run k3d help to see what you can do with k3d
$ k3d help
https://k3d.io/
k3d is a wrapper CLI that helps you to easily create k3s clusters inside docker.
Nodes of a k3d cluster are docker containers running a k3s image.
All Nodes of a k3d cluster are part of the same docker network.
Usage:
k3d [flags]
k3d [command]
Available Commands:
cluster Manage cluster(s)
completion Generate completion scripts for [bash, zsh, fish, powershell | psh]
help Help about any command
image Handle container images.
kubeconfig Manage kubeconfig(s)
node Manage node(s)
version Show k3d and default k3s version
Flags:
-h, --help help for k3d
--trace Enable super verbose output (trace logging)
--verbose Enable verbose output (debug logging)
--version Show k3d and default k3s version
Use "k3d [command] --help" for more information about a command
Create a Kubernetes cluster with k3d
To create a cluster named ‘k3d-cluster’ with a single node, run the command below:
$ sudo k3d cluster create k3d-cluster
INFO[0000] Prep: Network
INFO[0000] Created network 'k3d-k3d-cluster'
INFO[0000] Created image volume k3d-k3d-cluster-images
INFO[0000] Starting new tools node...
INFO[0001] Creating node 'k3d-k3d-cluster-server-0'
INFO[0001] Pulling image 'ghcr.io/k3d-io/k3d-tools:5.4.8'
INFO[0002] Pulling image 'docker.io/rancher/k3s:v1.25.6-k3s1'
INFO[0002] Starting Node 'k3d-k3d-cluster-tools'
INFO[0005] Creating LoadBalancer 'k3d-k3d-cluster-serverlb'
INFO[0007] Pulling image 'ghcr.io/k3d-io/k3d-proxy:5.4.8'
INFO[0009] Using the k3d-tools node to gather environment information
INFO[0009] HostIP: using network gateway 172.18.0.1 address
INFO[0009] Starting cluster 'k3d-cluster'
INFO[0009] Starting servers...
INFO[0009] Starting Node 'k3d-k3d-cluster-server-0'
INFO[0015] All agents already running.
INFO[0015] Starting helpers...
INFO[0015] Starting Node 'k3d-k3d-cluster-serverlb'
INFO[0022] Injecting records for hostAliases (incl. host.k3d.internal) and for 2 network members into CoreDNS configmap...
INFO[0024] Cluster 'k3d-cluster' created successfully!
INFO[0024] You can now use it like this:
kubectl cluster-infor
Managing k3s Cluster with k3d
The created cluster can be manage either using k3d or kubectl,
Let's see how to use k3d to manage the cluster first.
To see all the command options that can be used with k3d, run
$ k3d help
https://k3d.io/
k3d is a wrapper CLI that helps you to easily create k3s clusters inside docker.
Nodes of a k3d cluster are docker containers running a k3s image.
All Nodes of a k3d cluster are part of the same docker network.
Usage:
k3d [flags]
k3d [command]
Available Commands:
cluster Manage cluster(s)
completion Generate completion scripts for [bash, zsh, fish, powershell | psh]
config Work with config file(s)
help Help about any command
image Handle container images.
kubeconfig Manage kubeconfig(s)
node Manage node(s)
registry Manage registry/registries
version Show k3d and default k3s version
Flags:
-h, --help help for k3d
--timestamps Enable Log timestamps
--trace Enable super verbose output (trace logging)
--verbose Enable verbose output (debug logging)
--version Show k3d and default k3s version
Use "k3d [command] --help" for more information about a command.
List clusters with k3d
$ sudo k3d cluster lis
NAME SERVERS AGENTS LOADBALANCER
k3d-cluster 1/1 0/0 truet
List nodes with k3d
$ sudo k3d node lis
NAME ROLE CLUSTER STATUS
k3d-k3d-cluster-server-0 server newcluster running
k3d-k3d-cluster-serverlb loadbalancer newcluster runningt
Delete a cluster with k3d
$ k3d cluster delete k3d-cluster
INFO[0001] Deleting cluster 'k3d-cluster'
INFO[0005] Deleted k3d-k3d-cluster-serverlb
INFO[0008] Deleted k3d-k3d-cluster-server-0
INFO[0008] Deleting cluster network '40d1445aaac90b4aa8c2db5c2547d3bbfa7662941808df0721373050ffea9c18'
INFO[0009] Deleting image volume 'k3d-k3d-cluster-images'
INFO[0009] Removing cluster details from default kubeconfig...
INFO[0009] Removing standalone kubeconfig file (if there is one)...
INFO[0009] Successfully deleted cluster k3d-cluster! r
Manage k3s Cluster with kubectl
To be able to use Kubectl in the K3s cluster, we need to install Kubectl first and configure the .kube/configure file to the cluster configuration settings.
source <(kubectl completion bash
echo "source <(kubectl completion bash)" >> ~/.bashrc )
mkdir ~/.kub
sudo cp /root/.kube/config ~/.kube
sudo chown $USER .kube/confige
You can now run k3s cluster commands with kubectl. For example, get all pods
$ sudo kubectl get pods --all-namespaces
NAMESPACE NAME READY STATUS RESTARTS AGE
kube-system metrics-server-7b4f8b595-26mhr 1/1 Running 0 19m
kube-system coredns-66c464876b-pljsx 1/1 Running 0 19m
kube-system local-path-provisioner-7ff9579c6-jj6mn 1/1 Running 0 19m
kube-system helm-install-traefik-zp9t4 0/1 Completed 0 19m
kube-system svclb-traefik-sdjzp 2/2 Running 0 18m
kube-system traefik-5dd496474-htq9n 1/1 Running 0 18m
To create a k3s cluster with more than one worker node, run the command below:
sudo k3d cluster create newcluster -a 2
Where -a flag specifies the number of agents you need. In this case, I have specified 2. Remember to export config.
After this, you can start to apply all your Kubernetes manifests to run your applications.
The above can as well be setup easily on your cloud instances!
