Architecture:

What is Prometheus?

Prometheus is an open-source automated monitoring and alerting system. It has become a widely accepted tool for monitoring highly dynamic container environments such as Kubernetes and Docker Swarm.

Kubernetes

Docker Swarm

It can collect metrics from various sources, including containers, servers, and applications, and store them in a time-series database. Prometheus provides a flexible query language, called PromQL, that allows you to retrieve and analyze data. It also includes a web interface and an API for data interaction.

What is Grafana?

Grafana is a multi-platform that gets data from a data source such as Prometheus and transforms it into visualizations charts. We can create our dashboards or use the existing ones provided by Grafana. We can personalize the dashboards as per our requirements.

What is Helm?

Helm is the package manager for Kubernetes. Helm Charts help you define, install, and upgrade even the most complex Kubernetes application. Charts are easy to create, version, share, and publish — so start using Helm and stop the copy-and-paste.

This article will teach you how to integrate Prometheus and Grafana on Kubernetes using Helm.

Prerequisites:

AWS account with the access key and secret key.
EC2 Instance with AWS CLI, EKSCTL, Kubectl, and Helm chart.
Kubernetes Cluster, I am using EKS Cluster for the Demo Purpose.
Dockerized Nodejs Application in Docker hub.
Prometheus and Grafana Setup.

Setup an AWS EC2 Instance:

Move to the EC2 console. Click Launch Instance.

For name use My-Eks-Cli.

Select AMIs as Ubuntu and select Instance Type as t2.micro. Create new Key Pair and Create a new Security Group with traffic allowed from SSH, HTTP, and HTTPS.

Click on Launch Instance and once EC2 Instance started, connect to it with EC2 Instance Connect.

Install AWS CLI and Configure:

Now we need to set up the AWS CLI on the EC2 machine so that we can use eksctl in the later stages

Let us get the installation done for AWS CLI 2.

Linux x86(64-bit) If you are using Linux x86(64-bit) operating system:

$ curl "https://awscli.amazonaws.com/awscli-exe-linux-x86_64.zip" -o "awscliv2.zip" 
$ sudo apt install unzip
$ unzip awscliv2.zip 
$ sudo ./aws/install

Okay now after installing the AWS CLI, let's configure the AWS CLI so that it can authenticate and communicate with the AWS environment.

Install and Setup Kubectl:

Moving forward now we need to set up the kubectl also onto the EC2 instance.

$ curl -O https://s3.us-west-2.amazonaws.com/amazon-eks/1.27.1/2023-04-19/bin/linux/amd64/kubectl
$ chmod +x ./kubectl
$ mkdir -p $HOME/bin && cp ./kubectl $HOME/bin/kubectl && export PATH=$HOME/bin:$PATH
$ echo 'export PATH=$HOME/bin:$PATH' >> ~/.bashrc
$ kubectl version --short --client

Install and Setup eksctl:

Download and extract the latest release of eksctl following command.

$ ARCH=amd64
$ PLATFORM=$(uname -s)_$ARCH
$ curl -sLO "https://github.com/weaveworks/eksctl/releases/latest/download/eksctl_$PLATFORM.tar.gz"
$ tar -xzf eksctl_$PLATFORM.tar.gz -C /tmp && rm eksctl_$PLATFORM.tar.gz
$ sudo mv /tmp/eksctl /usr/local/bin

# Check the eksctl version.
$ eksctl version

Install Helm chart:

The next tool we need is Helm Chart. Helm is a package manager for Kubernetes, an open-source container orchestration platform. Helm helps you manage Kubernetes applications by making it easy to install, update, and delete them.

Install Helm Chart - Use the following script to install the helm chart -

$ curl -fsSL -o get_helm.sh https://raw.githubusercontent.com/helm/helm/main/scripts/get-helm-3
$ chmod 700 get_helm.sh
$ ./get_helm.sh

This way we install all AWS CLI, kubectl, eksctl, and Helm. Now we need to set up the Kubernetes Cluster setup. I am using the EKS Cluster service in AWS.

Creating an Amazon EKS cluster using eksctl:

Now in this step, we are going to Create Amazon EKS clusters using eksctl

You need the following to run the eksctl command:

Name of the cluster : my-eks-monitoring
Version of Kubernetes : --version 1.24
Region : --region us-east-1
Nodegroup name/worker nodes : --nodegroup-name worker-nodes
Node Type : --nodegroup-type t3.medium
Number of nodes: --nodes 2
Minimum Number of nodes: --nodes-min 2
Maximum Number of nodes: --nodes-max 3

Here is the eksctl command -

eksctl create cluster --name my-eks-monitoring --version 1.24 --region us-east-1 --nodegroup-name worker-nodes --node-type t3.medium --nodes 2 --nodes-min 2 --nodes-max 3

It took me 20 minutes to complete this EKS cluster. If you get any error for not having sufficient data for mentioned availability zone then try it again.

aws eks update-kubeconfig --name my-eks-monitoring

Verify the EKS Kubernetes cluster on AWS Console.

You can go back to your AWS dashboard and look for Elastic Kubernetes Service -> Clusters

Now we had to check in the same server in which we have installed all the utilities which are required for this project.

In the same server run these commands to check whether the cluster is running fine or not.

$ kubectl get all --all-namespaces

Installing the Kubernetes Metrics Server:

Alright, the next step would be to install the Kubernetes Metrics server onto the Kubernetes cluster so that Prometheus can collect the performance metrics of Kubernetes.

Deploy the Metrics Server with the following command:

kubectl apply -f https://github.com/kubernetes-sigs/metrics-server/releases/latest/download/components.yaml

 $ kubectl get deployment metrics-server -n kube-system

Verify that the metrics-server deployment is running the desired number of pods with the following command.

Dockerizing Nodejs Application:

I am pasting the link below for the application code. I am using this Nodejs Application for this project.

GitHub Nodejs Application

Then we need to create a Dokcerfile for the above application which we want to use in the Prometheus and Grafana for metrics scraping.

# Set the base image to use for subsequent instructions
FROM node:alpine
# Set the working directory for any subsequent ADD, COPY, CMD, ENTRYPOINT,
# or RUN instructions that follow it in the Dockerfile
WORKDIR /usr/src/app
# Copy files or folders from source to the dest path in the image's filesystem.
COPY package*.json /usr/src/app/
COPY . /usr/src/app/
# Execute any commands on top of the current image as a new layer and commit the results.
RUN npm install --production
# Define the network ports that this container will listen to at runtime.
EXPOSE 3000
# Configure the container to be run as an executable.
ENTRYPOINT ["npm", "start"]

Now I will run this application in my docker environment for checking whether this application is running or not.

docker build .

Now we will check with the below command.

docker images

You can verify that the image is created. Now we have to create a tag for this image and Repository name by using the below command.

docker tag <existing-image-id> <new-repository-name>:<new-tag-name>

Now we have to run this image as a container and we need to check whether the application is running fine or not on the web.

This was the command I used to run this image as a container as shown in the below command.

docker run -it -p 3000:3000 eks-nodejapp:latest

Now we will check on the web with the localhost means here is the server IP Address we will use in the web as shown in the below image.

Now we have dockerized our application in the Docker. Now we will push this image into the Dockerhub by using the below commands shown below.

First of all, create a Dockerhub account with username and password and then create a Dockerhub repository as shown in the below image.

With the help of the above repository, we will push our docker image into the Docker hub account and then we will use that image in the Kubernetes Manifest file for running our application in the K8s cluster.

$ docker login 
###username and password will ask
$ docker tag <image-name> <dockerhub-username>/<repository-name>:<tag-name>
$ docker push <dockerhub-username>/<repository-name>:<tag-name>

By using these commands we will push our image into the Dockerhub. Then we need to setup the Prometheus and Grafana in EKS Cluster to set up the Observability for the NodeJS Application.

We will see further setup of Prometheus and Grafana in the EKS Cluster using Helm charts. Scraping the metrics using Prometheus and visualizing the dashboards in Grafana.