Setup Of Multi-Node Kubernetes Cluster

Here we are going to create our own multi-node Kubernetes cluster. For creating a cluster we need to run multiple OS/Nodes, we have four ways to launch/run the OS. On top of Virtual Machine, AWS EC2 Server, and Server.

Here, we are going to use Oracle VM and on top of it, we will run RHEL8 Linux OS, one master node, and two slave nodes (nodes can vary on the use cases and requirements).

Requirements for each node

1. Docker setup
2. kubectl program
3. Networking

Step 1. Create a new virtual machine (RHEL8 setup).

Give name, provide RAM at least 3 GB, and storage and approx 40 GB and create.

Goto Settings -> System and hive the priority to the Hars Disk in the Boot Order

In Processor give at least 2 CPU

Go to Network and select Bridge Adaptor for connectivity from outside the cluster as well as between nodes in the cluster.

Go to Storage and attach the iso file of RHEL8 from the location.

Now start the OS and do the following for the installation.

For Networking

Disable KDUMP

For CLI select Minimal Install

Click on Begin Installation and set the root password and confirm it.

It will take some time in installation.

Step 2. yum configure

Make a folder to mount the dvd of RHEL8

mkdir /dvd
mount /dev/cdrom /dvd

Now to configure yum

cd /etc/yum.repos.d
vi yum.repo

we can give any name but the extension should be .repo

In yum.repo file write (press i to insert)

[dvd1]
baseurl=file:///dvd/AppStream
gpgcheck=0

[dvd2]
baseurl=file:///dvd/BaseOS
gpgcheck=0

Press the Esc button and do :wq (to save and to exit)

To see the status of the software

yum repolist

Now install net-tools software to see IP and to open the file install vim

yum install net-tools vim -y

To check the IP

ifconfig enp0s3

Now login through the PUTTY tool. (Because in CLI it is quite difficult to copy and paste).

Step 3. Docker setup

For installing the docker in RHEL8 first we need to configure yum.

[docker]
baseurl=https://download.docker.com/linux/centos/7/x86_64/stable/
gpgcheck=0

Now clean the yum cache

yum clean all
yum repolist

Now install the docker

yum install docker-ce --nobest

Step 4. Disable firewall

systemctl stop firewalld
systemctl disable firewalld

Mount dvd permanent

Step 5. Setup and installation of Kubernetes programs.

Search on google <kubernetes repo for RHEL8>

cat <<EOF > /etc/yum.repos.d/kubernetes.repo
[kubernetes]
name=Kubernetes
baseurl=https://packages.cloud.google.com/yum/repos/kubernetes-el7-$basearch
enabled=1
gpgcheck=1
repo_gpgcheck=1
gpgkey=https://packages.cloud.google.com/yum/doc/yum-key.gpg https://packages.cloud.google.com/yum/doc/rpm-package-key.gpg
exclude=kubelet kubeadm kubectl
EOF

copy and paste (this will setup the yum configure for the k8s )

To check go to /etc/yum.repos.d folder

Now upgrade the k8s repo

yum upgrade -y --nobest

Clean the yum cache and do yum repolist to check the status

yum clean all 
yum repolist

Now install the k8s programs.

yum install -y kubelet kubeadm kubectl --disableexcludes=kubernetes

Step 6. Make SELinux permissive

Step 7. Start the docker services and see docker info.

systemctl start docker
systemctl enable docker
docker info

Now change the Cgroup Driver from cgroupfs to systemd. for this create a daemon.json file.

vim /etc/docker/daemon.json

Write in the file.

Now restart the docker services and see its info

systemctl restart docker
docker info

Step 7. Disable the swap. For this go to /etc/fstab file and comment or delete the swap line.

Step 8. To control the traffic of Linux install iproute software.

yum install iproute-tc

Now check the status of iptables it should be 1.

Step 9. Start the services of the kubelet. And check its status if it is active means it is working fine.

systemctl start kubelet
systemctl enable kubelet
systemctl status kubelet

Step 10. Clone for master and slave nodes (in this case we have one master and two slaves. You can clone nodes according to your requirements).

Reinitialize for unique IP

Similarly clone for slave nodes,

Step 11. To ping with name set the hostname and configure the /etc/hosts file with the IPs and name, on all the nodes.

hostnamectl set-hostname master
hostname
exec bash

vim /etc/hosts

We can scp /etc/hosts file to all the nodes.

Now ping each other by name.

Step 12. Make the IP static

For this, we need IP, netmask, gateway, dns server name.

ifconfig enp0s3   //for IP and netmask
route -n          //for gateway
cat /etc/resolv.conf

Now

cd /etc/sysconfig/network-scripts/
ls
vim ifcfg-enp0s3

In this file write IP, netmask, gateway, DNS to make it static (In this case IP, gateway, netmask, DNS as follow)

IPADDR=192.168.43.161
NETMASK=255.255.255.0
GATEWAY=192.168.43.1
DNS1=192.168.43.1

Step 13. Now AT MASTER check the docker images and running container. Till now we don't have any image or container running. Now initialize the kubeadm program with the IP range and netmask, behind the scene it will pull some docker images and run the containers.

docker images
docker ps a

kubeadm init --pod-network-cidr=10.10.1.0/16

It will take some time to initialize.

It will have some commands and token to connect with slave nodes.

copy and run these commands on the MASTER node

Now copy the token commands to run on slave nodes but before it, we have make our master ready. (so for a while we can copy and paste on the notepad).

Check the nodes on the master.

kubectl get nodes

We can see that our master is not ready yet. To make it ready we need some plugins for CNI. The Container Network Interface (CNI) is a library definition and a set of tools under the umbrella of the Cloud Native Computing Foundation project, Kubernetes uses CNI as an interface between network providers and Kubernetes networking.

Search on google <github-kube-flannel>

Copy these command and run on master

kubectl apply -f https://raw.githubusercontent.com/coreos/flannel/master/Documentation/kube-flannel.yml

Now check the master

kubectl get pods -n kube-system
kubectl get nodes

Now see docker images and containers running.

Step 14. At all the SLAVE NODES, run on the token command to connect with master.

At MASTER check the nodes

It will take some to come in the ready state. Also, check the namespace and pods at the master.

kubectl get ns
kubectl get pods -n kube-system
kubectl get nodes

Step 15. Our cluster is ready, now we have configured the client part. Here, we will use the command prompt as a client in windows. For this, we have config file at the master and we have to copy that to windows.

copy config file to the /root

cd .kube/
ls
cp config /root

Noe we will use winSCP tool to copy from RHEL to WINDOWS

Login and drag and drop

Now go to command prompt and go to the location where you have copied the config file and check.

Our Multi-Node Kubernetes Cluster is ready to use.

We can run any Kubernetes command and in that command, we have to use <--kubeconfig config> for our cluster.

Now launch the pod and use < -o wide > in the command to see in which slave node our pods are launched.

kubectl rum myweb --image=httpd --kubeconfig config //to launch the pod
kubectl get pods -o wide --kubeconfig config        //to see the pod

We can also see thisat the master node

Now let's run a replica set program to see how the pods are randomly distributed at the slave nodes.

apiVersion: apps/v1
kind: ReplicaSet
metadata:
  name: myweb-rs
spec:
  replicas: 7
  selector:
    matchExpressions:
      - { key: env, operator: In, values: [ production ]}
      - { key: region, operator: In, values: [ IN ]}
  template:
    metadata:
      name: myweb-pod
      labels:
        env: production
        region: IN
    spec:
      containers:
      - name: myweb-con
        
        image: httpd

Now see the pods

Similarly, we can run any commands and code of k8s.

THANK YOU.....!!!