Secure Swiggy Clone App Deployment on AWS: Building a DevSecOps Pipeline with Terraform, Jenkins, SonarQube, Trivy, Argocd, and EKS.
Introduction:
In today's fast-paced digital landscape, building and deploying applications not only requires speed but also airtight security. That's where DevSecOps comes into play, blending development, security, and operations into a single, unified process.
In this guide, we will embark on a journey where we'll leverage Terraform, Jenkins CI/CD, SonarQube, Trivy, Argocd, and Amazon Elastic Kubernetes Service (EKS) to create a robust and secure pipeline for deploying applications on Amazon Web Services (AWS).
Whether you're an experienced developer looking to enhance your DevSecOps skills or a newcomer eager to explore this exciting intersection of software development and security, this guide has something valuable to offer.
Let's dive in and explore the steps to safeguard your Amazon app while ensuring smooth and efficient deployment.
Architecture:
Overview:
Infrastructure as Code: Use Terraform to define and manage AWS infrastructure for the application.
Container Orchestration: Utilize Amazon EKS for managing and scaling containerized applications.
CI/CD with Jenkins: Set up Jenkins to automate building, testing, and deploying the application.
Static Code Analysis: Incorporate SonarQube to analyze code quality and identify vulnerabilities.
Container Image Scanning: Integrate Trivy to scan container images for security issues.
Application Deployment with Argocd: Use Argocd for declarative, GitOps-based application deployment on EKS.
Project Resources:
GitHub Link:
https://github.com/sridhar-modalavalasa/Swiggy-Clone-App - Application code.
https://github.com/sridhar-modalavalasa/Swiggy-App-ArgoCD - Manifest files.
Step1: Create IAM User
Navigate to the AWS console
Create an IAM user with administration access.
Log in to the AWS Console with the above user.
Create one free-tier EC2 instance with Ubuntu.
Login to the EC2 instance and follow the below steps.
Step2: Aws Configuration
Install the AWS Cli in your EC2 Ubuntu.
Install AWS CLI:
curl "https://awscli.amazonaws.com/awscli-exe-linux-x86_64.zip" -o "awscliv2.zip"
unzip awscliv2.zip
sudo ./aws/install
aws --version
aws configure
Provide your Aws Access key and Secret Access key
Step3: Terraform files and Provision Jenkins, sonarqube
Terraform Installation in an EC2 Instance:
wget https://releases.hashicorp.com/terraform/1.3.7/terraform_1.3.7_linux_amd64.zip
unzip terraform_1.3.7_linux_amd64.zip
mv terraform /usr/local/bin
sudo mv terraform /usr/local/bin
terraform -v
resource "aws_instance" "web" {
ami = "ami-0fc5d935ebf8bc3bc" #change ami id for different region
instance_type = "t2.large"
key_name = "key"
vpc_security_group_ids = [aws_security_group.Jenkins-sg.id]
user_data = templatefile("./install.sh", {})
tags = {
Name = "Jenkins-sonarqube-trivy-vm"
}
root_block_device {
volume_size = 30
}
}
resource "aws_security_group" "Jenkins-sg" {
name = "Jenkins-sg"
description = "Allow TLS inbound traffic"
ingress = [
for port in [22, 80, 443, 8080, 9000, 3000] : {
description = "inbound rules"
from_port = port
to_port = port
protocol = "tcp"
cidr_blocks = ["0.0.0.0/0"]
ipv6_cidr_blocks = []
prefix_list_ids = []
security_groups = []
self = false
}
]
egress {
from_port = 0
to_port = 0
protocol = "-1"
cidr_blocks = ["0.0.0.0/0"]
}
tags = {
Name = "jenkins-sg"
}
}
#provider.tf
terraform {
required_providers {
aws = {
source = "hashicorp/aws"
version = "~> 5.0"
}
}
}
# Configure the AWS Provider
provider "aws" {
region = "us-east-1" #change your region
}
This will install Jenkins, Docker, Sonarqube, and Trivy by Terraform with an EC2 instance.
#!/bin/bash
sudo apt update -y
wget -O - https://packages.adoptium.net/artifactory/api/gpg/key/public | tee /etc/apt/keyrings/adoptium.asc
echo "deb [signed-by=/etc/apt/keyrings/adoptium.asc] https://packages.adoptium.net/artifactory/deb $(awk -F= '/^VERSION_CODENAME/{print$2}' /etc/os-release) main" | tee /etc/apt/sources.list.d/adoptium.list
sudo apt update -y
sudo apt install temurin-17-jdk -y
/usr/bin/java --version
curl -fsSL https://pkg.jenkins.io/debian-stable/jenkins.io-2023.key | sudo tee /usr/share/keyrings/jenkins-keyring.asc > /dev/null
echo deb [signed-by=/usr/share/keyrings/jenkins-keyring.asc] https://pkg.jenkins.io/debian-stable binary/ | sudo tee /etc/apt/sources.list.d/jenkins.list > /dev/null
sudo apt-get update -y
sudo apt-get install jenkins -y
sudo systemctl start jenkins
sudo systemctl status jenkins
#install docker
sudo apt-get update
sudo apt-get install docker.io -y
sudo usermod -aG docker ubuntu
newgrp docker
sudo chmod 777 /var/run/docker.sock
docker run -d --name sonar -p 9000:9000 sonarqube:lts-community
#install trivy
sudo apt-get install wget apt-transport-https gnupg lsb-release -y
wget -qO - https://aquasecurity.github.io/trivy-repo/deb/public.key | gpg --dearmor | sudo tee /usr/share/keyrings/trivy.gpg > /dev/null
echo "deb [signed-by=/usr/share/keyrings/trivy.gpg] https://aquasecurity.github.io/trivy-repo/deb $(lsb_release -sc) main" | sudo tee -a /etc/apt/sources.list.d/trivy.list
sudo apt-get update
sudo apt-get install trivy -y
Terraform commands to provision:
terraform init
terraform validate
terraform plan
terraform apply
The EC2 instance Jenkins-sonarqube-trivy-vm is created by Terraform with Jenkins, Sonarqube, and Trivy as userdata for the EC2 instance, which is installed during the creation of the EC2 instance.
Output:
Take the public IP address of the EC2 instance, as shown in the below image.
<Public IPV4 address>:8080. #For accessing Jenkins
sudo cat /var/lib/jenkins/secrets/initialAdminPassword
Unlock Jenkins using an administrative password and install the suggested plugins.
Jenkins will now get installed and install all the libraries.
Create a user, click save, and continue.
Jenkins Getting Started Screen.
Copy your public key again and paste it into a new tab.
<instance-public-ip>:9000
Now our Sonarqube is up and running as a Docker container.
Enter your username and password, click on login, and change your password.
username admin
password admin
Update the new password. This is the Sonar Dashboard, as shown below.
Check Trivy version
Check the Trivy version in an Ec2 instance.
trivy --version
Step 4: Install Plugins like JDK, Sonarqube Scanner, NodeJs, and OWASP Dependency Check
4A — Install Plugin
Goto Manage Jenkins →Plugins → Available Plugins
Install below plugins
1: Eclipse Temurin Installer (Install without restart)
2: SonarQube Scanner (Install without restart)
3: Sonar Quality Gates (Install Without restart)
4B: Configure Java and Nodejs in Global Tool Configuration
Goto Manage Jenkins → Tools → Install JDK (17) and NodeJs (16). Click on Apply and Save
Choose the option install from adoptium.net
Step 5: Configure Sonar Server in Manage Jenkins
Grab the public IP address of your EC2 instance.
Sonarqube works on Port 9000, so <Public IP>:9000.
Go to your Sonarqube server.
Click on Administration → Security → Users → Click on Tokens and Update Token, → Give it a name, and click on Generate Token
click on update Token
Create a token with a name and generate
copy Token
Goto Jenkins Dashboard → Manage Jenkins → Credentials → Add secret text. It should look like this
You will see this page once you click on create
Now, go to Dashboard → Manage Jenkins → System and add like the below image.
Click on Apply and Save.
The Configure System option is used in Jenkins to configure different server
Global Tool Configuration is used to configure different tools that we install using Plugins
We will install a sonar scanner in the tools.
In the Sonarqube Dashboard, add a quality gate as well.
In the sonar interface, create the quality gate as shown below:
Click on the quality gate, then create.
Click on the save option.
In the Sonarqube Dashboard, Create Webhook option as shown in below:
Administration--> Configuration-->Webhooks
Click on Create
Add details:
<http://jenkins-private-ip:8080>/sonarqube-webhook/
Let's go to our pipeline and add the script to our pipeline script.
pipeline{
agent any
tools{
jdk 'jdk17'
nodejs 'node16'
}
environment {
SCANNER_HOME=tool 'sonar-scanner'
}
stages {
stage('clean workspace'){
steps{
cleanWs()
}
}
stage('Checkout from Git'){
steps{
git branch: 'main', url: 'https://github.com/sridhar-modalavalasa/Swiggy-Clone-App.git'
}
}
stage("Sonarqube Analysis "){
steps{
withSonarQubeEnv('sonar-server') {
sh ''' $SCANNER_HOME/bin/sonar-scanner -Dsonar.projectName=Swiggy-CICD \
-Dsonar.projectKey=Swiggy-CICD '''
}
}
}
stage("quality gate"){
steps {
script {
waitForQualityGate abortPipeline: false, credentialsId: 'Sonar-token'
}
}
}
stage('Install Dependencies') {
steps {
sh "npm install"
}
}
}
}
Click on Build now, and you will see the stage view like this:
To see the report, you can go to Sonarqube Server and go to Projects.
You can see the report has been generated, and the status shows as passed. You can see that there are 801 lines it has scanned. To see a detailed report, you can go to issues.
Step 6: Install OWASP Dependency Check Plugins
Go to Dashboard → Manage Jenkins → Plugins → OWASP Dependency-Check. Click on it and install it without restarting.
First, we configured the plugin, and next, we had to configure the Tool
Goto Dashboard → Manage Jenkins → Tools →
Click on Apply and save here.
Now go to Configure → Pipeline and add this stage to your pipeline and build.
stage('OWASP FS SCAN') {
steps {
dependencyCheck additionalArguments: '--scan ./ --disableYarnAudit --disableNodeAudit', odcInstallation: 'DP-Check'
dependencyCheckPublisher pattern: '**/dependency-check-report.xml'
}
}
stage('TRIVY FS SCAN') {
steps {
sh "trivy fs . > trivyfs.txt"
}
}
You will see that in status, a graph will also be generated for vulnerabilities.
Step 7: Docker Image Build and Push
We need to install the Docker tool on our system.
Go to Dashboard → Manage Plugins → Available plugins → Search for Docker and install these plugins.
Now, goto Dashboard → Manage Jenkins → Tools →
Now go to the Dockerhub repository to generate a token and integrate with Jenkins to push the image to the specific repository.
If you observe, there is no repository related to Swiggy.
There is an icon with the first letter of your name.
Click on that My Account, --> Settings --> Create a new token and copy the token.
Goto Jenkins Dashboard → Manage Jenkins → Credentials → Add secret text. It should look like this:
Add this stage to Pipeline Script.
stage("Docker Build & Push"){
steps{
script{
withDockerRegistry(credentialsId: 'dockerhub', toolName: 'docker'){
app.push("${env.BUILD_NUMBER}")
sh "docker build -t swiggy-app ."
sh "docker tag swiggy-app shreedhar4037/swiggy-app:latest "
sh "docker push shreedhar4037/swiggy-app:latest "
}
}
}
}
stage("TRIVY"){
steps{
sh "trivy image shreedhar4037/swiggy-app:latest > trivyimage.txt"
}
}
You will be able to view the output in the Jenkins pipeline and output upon successful execution.
When you log in to Dockerhub, you will see a new image is created.
Step 8: Creation of EKS Cluster with ArgoCD
EKSCTL Installation:
Now let's install EKSCTL in Ubuntu EC2 which was created earlier.
ARCH=amd64
PLATFORM=$(uname -s)_$ARCH
curl -sLO "https://github.com/weaveworks/eksctl/releases/latest/download/eksctl_$PLATFORM.tar.gz"
tar -xzf eksctl_$PLATFORM.tar.gz -C /tmp && rm eksctl_$PLATFORM.tar.gz
sudo mv /tmp/eksctl /usr/local/bin
# Check the eksctl version.
eksctl version
Command to Create EKS Cluster using eksctl command:
eksctl create cluster --name <name-of-cluster> --nodegroup-name <nodegrpname> --node-type <instance-type> --nodes <no-of-nodes>
eksctl create cluster --name my-eks-cluster --nodegroup-name ng-test --node-type t3.medium --nodes 2
It will take 5–10 minutes to create a cluster.
As you will see in the EC2 instances running list one instance is running in the name of EKS Cluster as shown below.
EKS Cluster is up and ready and check with the below command.
Now let's install ArgoCD in the EKS Cluster.
kubectl create ns Argocd
# This will create a new namespace, argocd, where Argo CD services and application resources will live.
kubectl create namespace argocd
kubectl apply -n argocd -f https://raw.githubusercontent.com/argoproj/argo-cd/stable/manifests/install.yaml
Download Argo CD CLI:
curl -sSL -o argocd-linux-amd64 https://github.com/argoproj/argo-cd/releases/latest/download/argocd-linux-amd64
sudo install -m 555 argocd-linux-amd64 /usr/local/bin/argocd
Access The Argo CD API Server:
# By default, the Argo CD API server is not exposed with an external IP. To access the API server,
choose one of the following techniques to expose the Argo CD API server:
* Service Type Load Balancer
* Port Forwarding
Let's go with Service Type Load Balancer.
# Change the argocd-server service type to LoadBalancer.
kubectl patch svc argocd-server -n argocd -p '{"spec": {"type": "LoadBalancer"}}'
List the resources in the namespace:
kubectl get all -n argocd
Get the load balancer URL:
kubectl get svc -n argocd
Pickup the URL and paste it into the web to get the UI as shown below image:
Login Using The CLI:
argocd admin initial-password -n argocd
Login with the admin and Password in the above you will get an interface as shown below:
Click on New App:
Enter the Repository URL, set path to ./, Cluster URL to kubernetes.default.svc, the namespace to default and click save.
The GitHub URL is the Kubernetes Manifest files which I have stored and the pushed image is used in the Kubernetes deployment files.
Repo Link: https://github.com/sridhar-modalavalasa/Swiggy-App-ArgoCD
You should see the below, once you're done with the details.
Click on it.
You can see the pods running in the EKS Cluster.
We can see the out-of-pods using the load balancer URL:
kubectl get svc
With the above load balancer, you will be able to see the output as shown in the below image:
As you observe in the above image, I just want to change the address of the Swiggy Application.
Then the real magic will happen, the changes updated in the code we will try to push the code to GitHub and run a pipeline to push the image to the repository with the updated details.
I will click on the sync option which is in the ArgoCD and then the updates will be in our Swiggy Website.
I have changed the website details of the address in the code and then you can see the results in the below image.
Conclusion:
In this project, we have covered the essential steps to deploy a Swiggy app with a strong focus on security through a DevSecOps approach. By leveraging tools like Terraform, Jenkins CI/CD, SonarQube, Trivy, Argocd, and EKS, we can create a robust and secure pipeline for deploying applications on AWS.
Remember that security is an ongoing process, and it is crucial to stay updated with the latest security practices and continuously monitor and improve the security of your applications. With the knowledge gained from this guide, you can enhance your DevSecOps skills and ensure the smooth and efficient deployment of secure applications on Amazon Web Services.
Thank you for reading this post! I hope you found it helpful. If you have any feedback or questions, Please connect with me on LinkedIn at
https://www.linkedin.com/in/sridhar-modalavalasa-12b492173/
https://github.com/sridhar-modalavalasa
Your feedback is valuable to me. Thank you!