DevOps Task-6: Integration of Prometheus and Grafana using Kubernetes.
Task :
Integrate Prometheus and Grafana and perform in following way:
- Deploy them as pods on top of Kubernetes by creating resources Deployment, ReplicaSet, Pods or Services.
- make their data to be remain persistent .
- both of them should be exposed to outside world.
ConfigMap :
ConfigMap is used to store the configurations of the application running in Pod. For example, the httpd.conf for httpd. You could modify the httpd.conf file in Pod but everything will be gone once Pod is replace. So we can write the desired configurations in ConfigMap and tell Pod to use such configuration whenever Pod is replaced. Therefore Pod is always running with how we wanted it to be.
PVC :
when implementing Prometheus and Grafana on K8S. Everything will work fine but as we know that Pod is stateless and could be recycle anytime, therefore any settings configured on Prometheus and Grafana would be gone when Pod is replaced.
Therefore, to solve this problem, we need to use Persistent Volumes(PV), PersistentVolumeClaims(PVC) and ConfigMap.
PVs as named the provisioned storage space in cluster to store data. Data is stored in PV rather than Pod itself, so data is retain when Pod is replaced. PVC as named is used to claim the provisioned storage space for use.
Service :
This is used to expose the pods services with a port number so that it can be accessible from the outside world.
The default port range for kubernetes services is in between 30000-32767, you can select any between them or kubernetes will provide automatically any random port in that range.
Manifest file for Prometheus :
ConfigMap :
apiVersion: v1 kind: ConfigMap metadata: name: prometheus-config data: prometheus.yml: |- global: scrape_interval: 15s evaluation_interval: 15s scrape_configs: - job_name: 'prometheus' static_configs: - targets: ['localhost:9090']
PVC :
apiVersion: v1 kind: PersistentVolumeClaim metadata: name: prometheus-pvc labels: app: prometheus spec: accessModes: - ReadWriteOnce resources: requests: storage: 20Gi
Service :
To access the Prometheus WebUI use http://<minikube ip>:30001 .
apiVersion: v1 kind: Service metadata: name: prometheus labels: app: prometheus spec: ports: - port: 9090 nodePort: 30001 selector: app: prometheus tier: backend type: NodePort
Deployment :
Now we have to create the deployment for prometheus. It will pull the prometheus image from docker hub, configure it, then create and mount the persistent volume at the destination folder.
apiVersion: apps/v1 kind: Deployment metadata: name: prometheus spec: replicas: 1 selector: matchLabels: app: prometheus tier: backend strategy: type: Recreate template: metadata: labels: app: prometheus tier: backend spec: containers: - name: prometheus image: prom/prometheus volumeMounts: - name: config-volume mountPath: /etc/prometheus - name: data mountPath: /prometheus ports: - containerPort: 80 securityContext: runAsUser: 0 volumes: - name: config-volume configMap: name: prometheus-config - name: data persistentVolumeClaim: claimName: prometheus-pvc
Manifest file for Grafana :
PVC:
apiVersion: v1 kind: PersistentVolumeClaim metadata: name: grafana-pvc labels: app: grafana spec: accessModes: - ReadWriteOnce resources: requests: storage: 20Gi
Service:
we will be able to access the Grafana WebUI at http://<minikube ip>:30002
apiVersion: v1 kind: Service metadata: name: grafana labels: app: grafana spec: ports: - port: 3000 nodePort: 30002 selector: app: grafana tier: frontend type: NodePort
Deployment :
apiVersion: apps/v1 kind: Deployment metadata: name: grafana labels: app: grafana tier: frontend spec: selector: matchLabels: app: grafana tier: frontend strategy: type: Recreate template: metadata: labels: app: grafana tier: frontend spec: containers: - image: grafana/grafana:latest name: grafana ports: - containerPort: 3000 name: grafana volumeMounts: - name: grafana-storage mountPath: /var/lib/grafana volumes: - name: grafana-storage persistentVolumeClaim: claimName: grafana-pvc
Kustomization.yml :
This file will bind the two resources (prometheus-deployment.yml and grafana-deployment.yml).
apiVersion: kustomize.config.k8s.io/v1beta1 kind: Kustomization resources: - prometheus-deployment.yaml - grafana-deployment.yaml
Now to deploy the complete setup the command to be used is...
kubectl create -k .
To view the deployments
To view the ConfigMaps...
To view the pods...
To view the PVCs...
kubectl get all
to acces the WebUI you need to go to the minikube ip with the provided port as mentioned earlier.
http://<minikube ip>:30001
and this will be the first page you will see
enter a query in the Expression text field and click the execute button to get the value or click on graph to get a graph for visualization.
Grafana WebUI:
To acces the WebUI you need to go to the minikube ip with the provided port as mentioned earlier.
http://<minikube ip>:30001
and this will be the first page you will see
enter a query in the Expression text field and click the execute button to get the value or click on graph to get a graph for visualization.
Dashboard:
Click on the Data Sources to add your data sources.
Set Prometheus as the data source.
you can add as many panel as you want with different queries.
As the pods have a persistent storage, even the pods crashes as your data is saved in a PVC your data will not be lost and replica set will launch a new pod and connect it with the PVC instantly.
GitHub Link:
https://github.com/ashishvardhan306/DevOps-Task-6/tree/master
Thank You very much...
So informative 😮