## File: README.md
# **Real-World DevOps/Cloud Projects For Learning from Beginner to Advanced** β
[![Forks][forks-shield]][forks-url]
[![Stars][stars-shield]][stars-url]
[![Issues][issues-shield]][issues-url]
[![Last Commit][commit-shield]][commit-url]
[![Code of Conduct][coc-shield]][coc-url]
[![Contributing][contrib-shield]][contrib-url]
[![Security][security-shield]][security-url]
[forks-shield]: https://img.shields.io/github/forks/NotHarshhaa/DevOps-Projects?style=for-the-badge&logo=github&logoColor=white&color=orange
[forks-url]: https://github.com/NotHarshhaa/DevOps-Projects/network/members
[stars-shield]: https://img.shields.io/github/stars/NotHarshhaa/DevOps-Projects.svg?style=for-the-badge&logo=github&logoColor=white&color=brightgreen
[stars-url]: https://github.com/NotHarshhaa/DevOps-Projects/stargazers
[issues-shield]: https://img.shields.io/github/issues/NotHarshhaa/DevOps-Projects?style=for-the-badge&logo=github&logoColor=white&color=blue
[issues-url]: https://github.com/NotHarshhaa/DevOps-Projects/issues
[commit-shield]: https://img.shields.io/github/last-commit/NotHarshhaa/DevOps-Projects?style=for-the-badge&logo=git&logoColor=white&color=ff69b4
[commit-url]: https://github.com/NotHarshhaa/DevOps-Projects/commits/master
[coc-shield]: https://img.shields.io/badge/Code%20of%20Conduct-Enforced-blueviolet?style=for-the-badge&logo=handshake&logoColor=white
[coc-url]: https://github.com/NotHarshhaa/DevOps-Projects/blob/master/CODE_OF_CONDUCT.md
[contrib-shield]: https://img.shields.io/badge/Contributions-Welcome-ff69b4?style=for-the-badge&logo=gitbook&logoColor=white
[contrib-url]: https://github.com/NotHarshhaa/DevOps-Projects/blob/master/CONTRIBUTING.md
[security-shield]: https://img.shields.io/badge/Security-Policy-10b981?style=for-the-badge&logo=shield&logoColor=white
[security-url]: https://github.com/NotHarshhaa/DevOps-Projects/blob/master/SECURITY.md
## π₯ **Project Ownership**
---
## π **Star History**
[](https://www.star-history.com/#NotHarshhaa/DevOps-Projects&Date)
---
_*Welcome to the ultimate resource for **learning DevOps through hands-on projects!** This repository is designed to cater to aspiring **DevOps engineers** of all skill levels, from beginners taking their first steps in the field to advanced users looking to deepen their knowledge and expertise.*_
---
## π§ **Purpose of the Repository**
This repository serves as a comprehensive resource for aspiring DevOps engineers to learn and implement real-world DevOps projects. It includes guides and solutions for deploying scalable systems, such as deploying a Java application on AWS using a 3-tier architecture and setting up scalable VPC architectures in the cloud.
The README files provide detailed instructions for implementing these projects, emphasizing practical deployment steps, pre-requisites, and validation processes. For example, one project focuses on deploying a Java-based login application integrated with a MySQL database, while another covers creating modular VPC network setups leveraging AWS services.
---
## π **Analysis of Features and Technologies**
The repository demonstrates extensive use of DevOps concepts and tools, focusing on AWS cloud infrastructure and automation. It features technologies such as:
- **EC2, RDS, VPC, Auto Scaling, IAM roles**
- **Maven, SonarCloud, JFrog Artifactory**
- **Monitoring via CloudWatch**
- **Custom AMIs, automation scripts**
These elements showcase a robust implementation of scalable, secure, and automated systems aligned with real-world DevOps practices.
---
## π **Real-time DevOps/Cloud Projects Showcase**
To improve readability and accessibility for users, this repository is also available as a modern and responsive web interface.
A website showcasing a curated list of major real-time DevOps and Cloud projects, ranging from beginner to advanced levels. Built using **Next.js** and styled with **Tailwind CSS**, this project leverages a modern starter template for fast and responsive development. Perfect for learning and exploring hands-on DevOps and Cloud concepts!
π **Explore the site**: [projects.prodevopsguytech.com](https://projects.prodevopsguytech.com)
---
## π **Related AWS Projects Repository**
For comprehensive AWS-specific projects and hands-on learning experiences, visit our dedicated AWS Projects repository:
**AWS Projects Repository Highlights:**
* **Real-world AWS Projects** from beginner to advanced levels
* **AWS DevOps Focus** with practical implementation guides
* **Hands-on Learning** with AWS services and best practices
* **Industry-Relevant** projects covering EC2, VPC, RDS, Lambda, and more
* **Community Driven** with active contributions and AWS expertise
π **Visit the AWS repository**: [AWS-Projects](https://github.com/NotHarshhaa/AWS-Projects)
---
## **Repository Contents for DevOps Projects from Beginner to Advanced Levels**
> [!IMPORTANT]
>
> _This repository contains a comprehensive collection of DevOps projects, each meticulously crafted to provide a hands-on learning experience. The projects are categorized into different skill levels to ensure that everyone, regardless of their current expertise, can find a suitable starting point and progressively enhance their skills._
>
> - **Beginner Projects:** Simple, foundational projects that introduce basic DevOps concepts and tools.
> - **Intermediate Projects:** More complex projects that require a good understanding of DevOps fundamentals.
> - **Advanced Projects:** Challenging projects designed to push your limits and deepen your understanding of sophisticated DevOps practices.
---
## π **Complete Projects Overview Table**
| # | Project Title | Tools & Technologies | Description |
|---|---------------|---------------------|-------------|
| **01** | Deploy Java Application on AWS 3-Tier Architecture | AWS (EC2, RDS, VPC, ALB, Auto Scaling), Maven, SonarCloud, JFrog, Nginx, Tomcat, MySQL | Production-grade Java web application deployment using AWS 3-tier architecture with high availability, auto-scaling, and comprehensive monitoring |
| **02** | Deploy Scalable VPC Architecture on AWS Cloud | AWS (VPC, EC2, NAT Gateway, Transit Gateway, CloudWatch), Apache, Git, SSM | Modular and scalable virtual network architecture with bastion host, auto-scaling groups, and VPC flow logs |
| **03** | Fun with Linux for Cloud & DevOps Engineers | Linux, AWS EC2, EBS, File Systems | Hands-on Linux fundamentals covering user management, permissions, directory structure, and file systems |
| **04** | Production-Ready Django Deployment on AWS ECS & ECR | AWS (ECS, ECR, RDS, ElastiCache, CloudFront), Django, Docker, Python, Terraform | Complete DevOps pipeline for Django applications with containerization, security scanning, and monitoring |
| **05** | Deploy Code on Docker Container using Jenkins | Jenkins, Docker, Maven, Git, AWS EC2, Tomcat | Automated build and deployment process using Jenkins pipeline with Docker containerization |
| **06** | Advanced CI/CD Pipeline with Major DevOps Tools | Terraform, Ansible, Jenkins, SonarQube, JFrog, Docker, EKS, Prometheus, Grafana | End-to-end CI/CD pipeline with infrastructure provisioning, code quality checks, and Kubernetes deployment |
| **07** | DevOps Journey Using Azure DevOps | Azure DevOps, Terraform, AKS, ACR, Application Insights, Key Vault, Inspec | Complete Azure DevOps setup with CI/CD pipelines, infrastructure testing, and monitoring |
| **08** | Kubernetes End to End Project on EKS | AWS EKS, kubectl, eksctl, Kubernetes, Docker, 2048 Game App | Deploy and manage containerized applications on Amazon EKS with load balancing and scaling |
| **09** | DevSecOps: Netflix Clone CI-CD with Monitoring | Jenkins, Docker, Kubernetes, Trivy, SonarQube, Prometheus, Grafana, OWASP, TMDB API | Secure Netflix clone deployment with comprehensive security scanning and monitoring |
| **10** | CI/CD Pipeline for .NET with DevOps Starter | Azure DevOps, .NET Core, ASP.NET, Azure App Service, Azure SQL, Application Insights | Automated CI/CD setup for .NET applications using Azure DevOps Starter |
| **11** | Two-Tier AWS Infrastructure with Terraform | Terraform, AWS (VPC, EC2, RDS, ALB, Route 53, CloudFront, WAF), S3 | Infrastructure as Code deployment of scalable two-tier architecture with security best practices |
| **12** | Deployment of Super Mario on Kubernetes using Terraform | Terraform, Kubernetes, AWS EKS, Docker, EC2 | Automated deployment of Super Mario game on Kubernetes cluster using Terraform |
| **13** | Zomato Clone: Secure Deployment with DevSecOps CI/CD | Jenkins, Docker, Kubernetes, Terraform, SonarQube, Trivy, Prometheus, Grafana | Food delivery application deployment with integrated security and monitoring |
| **14** | End-to-End CI/CD Pipeline for Android Apps | GitHub Actions, Gradle, Android, SonarQube, Firebase, Google Play Store | Automated build, test, and deployment pipeline for Android applications |
| **15** | Deploy E-Commerce Three Tier Application on AWS EKS | AWS EKS, Helm, kubectl, eksctl, ALB Controller, EBS CSI, RobotShop | Three-tier e-commerce application deployment with Helm charts and AWS EKS |
| **16** | Real Time DevOps: Deploy to Kubernetes Using Jenkins | Jenkins, Kubernetes, Docker, Maven, Git, AWS EC2 | End-to-end CI/CD project deploying applications to Kubernetes cluster via Jenkins |
| **17** | Terraform Modules for AWS Infrastructure | Terraform, AWS (VPC, EC2, S3, IAM, RDS), Modules | Reusable Terraform modules for building AWS infrastructure components |
| **18** | Jenkins Pipeline with Maven, SonarQube, Argo CD, Helm | Jenkins, Maven, SonarQube, Argo CD, Helm, Kubernetes, Docker, GitOps | Complete CI/CD pipeline for Java applications with GitOps deployment |
| **19** | Multi-Cloud Kubernetes Deployment | Kubernetes, AWS EKS, Azure AKS, GCP GKE, Terraform, Helm | Deploy applications across multiple cloud providers using Kubernetes |
| **20** | Azure DevOps Pipeline + Terraform Deployment | Azure DevOps, Terraform, Azure Service Bus, Blob Storage, Service Principal | Automated Terraform deployments via Azure DevOps with state management |
| **21** | Microservices Deployment with Service Mesh | Kubernetes, Istio, Docker, Prometheus, Grafana, Jaeger | Deploy microservices architecture with service mesh for traffic management |
| **22** | Serverless Application Deployment on AWS | AWS Lambda, API Gateway, DynamoDB, S3, CloudFormation, SAM | Build and deploy serverless applications using AWS services |
| **23** | GitLab CI/CD with Kubernetes and Helm | GitLab CI/CD, Kubernetes, Helm, Docker, AWS EKS | Automated deployment pipeline using GitLab CI/CD and Kubernetes |
| **24** | Infrastructure Monitoring with Prometheus Stack | Prometheus, Grafana, AlertManager, Node Exporter, Blackbox Exporter | Complete monitoring solution for infrastructure and applications |
| **25** | DevSecOps: Petshop Java Application Deployment | Jenkins, Docker, Kubernetes, Terraform, SonarQube, Trivy, Ansible, Maven | Secure deployment of Java-based Petshop application with automation |
| **26** | Cloud Cost Optimization with Automation | AWS Cost Explorer, Lambda, CloudWatch, Terraform, Python | Automated cloud cost monitoring and optimization strategies |
| **27** | Disaster Recovery and Backup Automation | AWS Backup, Lambda, S3, RDS, EC2, CloudFormation | Automated backup and disaster recovery solutions for AWS resources |
| **28** | Multi-Environment Deployment Strategy | Terraform, Jenkins, Kubernetes, Helm, Git, AWS/Azure | Manage multiple environments (dev, staging, prod) with IaC |
| **29** | Log Aggregation and Analysis Pipeline | ELK Stack (Elasticsearch, Logstash, Kibana), Filebeat, AWS | Centralized logging solution for distributed applications |
| **30** | Production Blog App Deployment with Full Monitoring | Jenkins, SonarQube, Nexus, Trivy, Docker, EKS, Prometheus, Grafana, Terraform | Complete CI/CD pipeline with security scanning and comprehensive monitoring |
| **31** | Container Security Scanning Pipeline | Trivy, Clair, Anchore, Jenkins, Docker, Kubernetes | Automated container vulnerability scanning in CI/CD pipeline |
| **32** | Blue-Green Deployment on Kubernetes | Kubernetes, Helm, Istio, Jenkins, AWS EKS, ArgoCD | Zero-downtime deployment strategy using blue-green methodology |
| **33** | Automated Database Migration Pipeline | Flyway, Liquibase, Jenkins, PostgreSQL, MySQL, Docker | Automated database schema migration and version control |
| **34** | Infrastructure Compliance and Security Scanning | Terraform, Checkov, tfsec, AWS Config, Jenkins | Automated infrastructure compliance checking and security scanning |
| **35** | Multi-Region High Availability Setup | AWS (Route 53, CloudFront, RDS Multi-AZ, S3), Terraform | Deploy highly available applications across multiple AWS regions |
| **36** | Kubernetes Cluster Management with GitOps | ArgoCD, Flux, Kubernetes, Helm, Git, AWS EKS | GitOps-based cluster management and application deployment |
| **37** | API Gateway and Microservices Architecture | Kong, AWS API Gateway, Lambda, ECS, Docker, Terraform | Build scalable API gateway for microservices architecture |
| **38** | Automated Testing in CI/CD Pipeline | Selenium, JUnit, Jest, Jenkins, Docker, Kubernetes | Integrate automated testing (unit, integration, E2E) in CI/CD |
| **39** | Service Mesh Implementation with Istio | Istio, Kubernetes, Prometheus, Grafana, Kiali, Jaeger | Implement service mesh for microservices with observability |
| **40** | Cloud Migration Strategy and Execution | AWS Migration Hub, Database Migration Service, Terraform, Jenkins | Plan and execute cloud migration from on-premises to AWS |
---
## **Integration of DevOps Technology with Other Technologies**
> [!NOTE]
> _In the modern tech landscape, DevOps doesn't exist in isolation. It intersects with a variety of other technologies, enhancing and being enhanced by them. This repository includes projects that integrate DevOps with several key technologies, allowing you to see how these integrations work in real-world scenarios._
>
> - **Machine Learning:** Implement DevOps practices to manage and deploy machine learning models efficiently.
> - **Version Control with Git & GitHub:** Learn how to manage code versions and collaborate with others using Git and GitHub.
> - **CI/CD Pipelines:** Set up continuous integration and continuous deployment pipelines to automate the testing and deployment of your applications.
> - **Cloud Platforms (AWS, Azure, GCP):** Deploy applications on cloud platforms and leverage their services for scalability and reliability.
> - **Containerization (Docker, Kubernetes):** Use container technologies to ensure that your applications run consistently across different environments.
---
## **Project Scope**
> [!IMPORTANT]
> The projects span a wide array of topics within the DevOps domain, each designed to provide practical experience and insights into real-world scenarios. Hereβs a detailed look at the areas covered:
>
> - **Automated Deployment:** Learn how to automate the deployment process to ensure that your applications are deployed quickly and reliably.
> - **Continuous Integration & Continuous Deployment (CI/CD):** Understand how to set up and manage CI/CD pipelines to automate the testing and deployment of your code.
> - **Infrastructure as Code (IaC):** Use tools like Terraform and CloudFormation to manage your infrastructure through code, ensuring consistency and scalability.
> - **Monitoring & Logging:** Implement monitoring and logging solutions to keep track of your applicationsβ performance and troubleshoot issues.
> - **Security & Compliance:** Learn how to incorporate security practices into your DevOps workflows to ensure that your applications are secure and compliant with regulations.
> - **Scalability & Performance Optimization:** Understand how to scale your applications and optimize their performance to handle increasing loads.
---
## **Why Explore This Repository?**
> [!NOTE]
> This repository is a treasure trove of learning opportunities, tailored to help you grow in the DevOps field. Here's why you should dive in:
>
> - **Hands-on Experience:** Each project is designed to provide you with practical, hands-on experience. You'll work through real-world challenges and gain the skills you need to succeed in the industry.
> - **Skill Enhancement:** Whether you're just starting or looking to build on existing skills, the projects are structured to guide you through a learning path that will enhance your capabilities.
> - **Industry Relevance:** Stay up-to-date with the latest trends and technologies in DevOps. The projects reflect current industry practices, ensuring that what you learn is relevant and applicable.
> - **Community Engagement:** Join a community of like-minded learners and professionals. Share your projects, seek feedback, and collaborate on exciting DevOps initiatives.
---
## **Code of Conduct**
> [!CAUTION]
>
> We are committed to fostering a welcoming and respectful environment for all contributors. Please take a moment to review our [Code of Conduct](./CODE_OF_CONDUCT.md) before participating in this community.
[](https://github.com/NotHarshhaa/DevOps-Projects/blob/master/CODE_OF_CONDUCT.md)
---
## **Security Policy**
> [!IMPORTANT]
>
> We prioritize security in all our DevOps projects. Please review our [Security Policy](./SECURITY.md) to understand our security practices, vulnerability reporting process, and how we maintain secure educational environments.
[](https://github.com/NotHarshhaa/DevOps-Projects/blob/master/SECURITY.md)
---
## **Contribute and Collaborate**
> [!TIP]
> This repository thrives on community contributions and collaboration. Hereβs how you can get involved:
>
> - **Fork the Repository:** Create your own copy of the repository to work on.
> - **Submit Pull Requests:** Contribute your projects or improvements to existing projects by submitting pull requests.
> - **Engage with Others:** Participate in discussions, provide feedback on othersβ projects, and collaborate to create better solutions.
> - **Share Your Knowledge:** If youβve developed a new project or learned something valuable, share it with the community. Your contributions can help others in their learning journey.
>
> **We follow best practices for contribution.**
[](https://github.com/NotHarshhaa/DevOps-Projects/blob/master/CONTRIBUTING.md)
---
## π **Join the Community**
> [!IMPORTANT]
> Be a part of our active DevOps community:
[](https://t.me/prodevopsguy)
[](https://github.com/NotHarshhaa)
---
## β **Hit the Star!**
**If you find this helpful, donβt forget to give this repository a star. Your support matters!** β
---
## π οΈ **Author & Community**
This project is crafted by **[Harshhaa](https://github.com/NotHarshhaa)** π‘
Iβd love to hear your feedback! Feel free to share your thoughts.
---
## π§ **Connect with me:**
[](https://linkedin.com/in/harshhaa-vardhan-reddy) [](https://github.com/NotHarshhaa) [](https://t.me/prodevopsguy) [](https://dev.to/notharshhaa) [](https://hashnode.com/@prodevopsguy)
---
## π’ **Stay Connected**
---
## π **Copyright & License**
---
## File: DevOps-Project-01/Java-Login-App/README.md
## Java Login App ##
Testing
## Sample Java Login application uses "UserDB" database and Table schema to store the Employee Login details. ##
## How to see list of Databases ##
SHOW DATABASES;
## How to create Database ##
CREATE DATABASE UserDB;
## How to list Tables ##
USE UserDB;
SHOW TABLES;
## How to create Table ##
## Below Query to create require TABLE schema to store Employee records ##
CREATE TABLE Employee (
id int unsigned auto_increment not null,
first_name varchar(250),
last_name varchar(250),
email varchar(250),
username varchar(250),
password varchar(250),
regdate timestamp,
primary key (id)
);
## List Table data ##
SELECT * FROM Employee;
## Describe Table schema ##
DESCRIBE Employee;
---
## File: DevOps-Project-01/README.md
# Deploy Java Application on AWS 3-Tier Architecture
## Table of Contents
1. [Project Overview](#project-overview)
2. [Architecture Overview](#architecture-overview)
3. [Pre-Requisites](#pre-requisites)
4. [Infrastructure Setup](#infrastructure-setup)
- [VPC and Networking](#vpc-and-networking)
- [Security Configuration](#security-configuration)
- [Database Layer](#database-layer)
5. [Application Setup](#application-setup)
- [Build Environment](#build-environment)
- [Application Deployment](#application-deployment)
- [Load Balancing and Auto Scaling](#load-balancing-and-auto-scaling)
6. [Monitoring and Maintenance](#monitoring-and-maintenance)
7. [Security Best Practices](#security-best-practices)
8. [Troubleshooting Guide](#troubleshooting-guide)
9. [Contributing](#contributing)
---
---
# Project Overview
## Introduction
This project demonstrates the deployment of a production-grade Java web application using AWS's robust 3-tier architecture. The implementation follows cloud-native best practices, ensuring high availability, scalability, and security across all application tiers.
### Key Features
- **High Availability**: Multi-AZ deployment with automated failover
- **Auto Scaling**: Dynamic resource allocation based on demand
- **Security**: Defense-in-depth approach with multiple security layers
- **Monitoring**: Comprehensive logging and monitoring setup
- **Cost Optimization**: Efficient resource utilization and management
## Architecture Overview
### Infrastructure Components
1. **Presentation Tier (Frontend)**
- Nginx web servers in Auto Scaling Group
- Public-facing Network Load Balancer
- CloudFront Distribution for static content
2. **Application Tier (Backend)**
- Apache Tomcat servers in Auto Scaling Group
- Internal Network Load Balancer
- Session management with Amazon ElastiCache
3. **Data Tier**
- Amazon RDS MySQL in Multi-AZ configuration
- Automated backups and point-in-time recovery
- Read replicas for read-heavy workloads
### Network Architecture
- **VPC Design**
- Two separate VPCs (192.168.0.0/16 and 172.32.0.0/16)
- Public and private subnets across multiple AZs
- Transit Gateway for inter-VPC communication
# Pre-Requisites
## Required Accounts and Tools
### 1. AWS Account Setup
- Create an [AWS Free Tier Account](https://aws.amazon.com/free/)
- Install AWS CLI v2
```bash
# For Linux
curl "https://awscli.amazonaws.com/awscli-exe-linux-x86_64.zip" -o "awscliv2.zip"
unzip awscliv2.zip
sudo ./aws/install
# For macOS
brew install awscli
# Configure AWS CLI
aws configure
```
### 2. Development Tools
- **Git**: Version control system
```bash
# For Linux
sudo apt-get update
sudo apt-get install git
# For macOS
brew install git
```
### 3. CI/CD Integration
- **SonarCloud Account**
- Sign up at [SonarCloud](https://sonarcloud.io/)
- Generate authentication token
- Configure project settings:
```bash
# Add to pom.xml
your_project_key
your_organization
https://sonarcloud.io
```
- **JFrog Artifactory**
- Create account on [JFrog Cloud](https://jfrog.com/start-free/)
- Set up Maven repository
- Configure authentication:
```xml
jfrog-artifactory
${env.JFROG_USERNAME}
${env.JFROG_PASSWORD}
```
# Infrastructure Setup
## VPC and Networking
### 1. VPC Creation
```bash
# Create primary VPC
aws ec2 create-vpc \
--cidr-block 192.168.0.0/16 \
--tag-specifications 'ResourceType=vpc,Tags=[{Key=Name,Value=PrimaryVPC}]' \
--region us-east-1
# Create secondary VPC
aws ec2 create-vpc \
--cidr-block 172.32.0.0/16 \
--tag-specifications 'ResourceType=vpc,Tags=[{Key=Name,Value=SecondaryVPC}]' \
--region us-east-1
```
### 2. Subnet Configuration
```bash
# Create public subnet
aws ec2 create-subnet \
--vpc-id vpc-xxx \
--cidr-block 192.168.1.0/24 \
--availability-zone us-east-1a \
--tag-specifications 'ResourceType=subnet,Tags=[{Key=Name,Value=PublicSubnet1}]'
# Create private subnet
aws ec2 create-subnet \
--vpc-id vpc-xxx \
--cidr-block 192.168.2.0/24 \
--availability-zone us-east-1b \
--tag-specifications 'ResourceType=subnet,Tags=[{Key=Name,Value=PrivateSubnet1}]'
```
### 3. Gateway Setup
```bash
# Create and attach Internet Gateway
aws ec2 create-internet-gateway
aws ec2 attach-internet-gateway --vpc-id vpc-xxx --internet-gateway-id igw-xxx
# Create NAT Gateway
aws ec2 create-nat-gateway \
--subnet-id subnet-xxx \
--allocation-id eipalloc-xxx \
--tag-specifications 'ResourceType=natgateway,Tags=[{Key=Name,Value=PrimaryNATGateway}]'
```
## Security Configuration
### 1. Security Groups
```bash
# Create frontend security group
aws ec2 create-security-group \
--group-name FrontendSG \
--description "Security group for frontend servers" \
--vpc-id vpc-xxx
# Allow inbound HTTP/HTTPS
aws ec2 authorize-security-group-ingress \
--group-id sg-xxx \
--protocol tcp \
--port 80 \
--cidr 0.0.0.0/0
aws ec2 authorize-security-group-ingress \
--group-id sg-xxx \
--protocol tcp \
--port 443 \
--cidr 0.0.0.0/0
```
### 2. IAM Roles and Policies
```json
{
"Version": "2012-10-17",
"Statement": [
{
"Effect": "Allow",
"Action": [
"s3:GetObject",
"s3:PutObject"
],
"Resource": "arn:aws:s3:::your-bucket/*"
}
]
}
```
## Database Layer
### 1. RDS Instance Creation
```bash
aws rds create-db-instance \
--db-instance-identifier prod-mysql \
--db-instance-class db.t3.medium \
--engine mysql \
--master-username admin \
--master-user-password "YourSecurePassword" \
--allocated-storage 20 \
--multi-az \
--vpc-security-group-ids sg-xxx \
--db-subnet-group-name your-db-subnet-group
```
### 2. Database Initialization
```sql
-- Connect to database
mysql -h your-rds-endpoint -u admin -p
-- Create application database
CREATE DATABASE javaapp;
USE javaapp;
-- Create users table
CREATE TABLE users (
id INT AUTO_INCREMENT PRIMARY KEY,
username VARCHAR(50) NOT NULL UNIQUE,
password VARCHAR(255) NOT NULL,
email VARCHAR(100) NOT NULL UNIQUE,
created_at TIMESTAMP DEFAULT CURRENT_TIMESTAMP
);
-- Create necessary indexes
CREATE INDEX idx_username ON users(username);
CREATE INDEX idx_email ON users(email);
```
# Application Setup
## Build Environment
### 1. Maven Configuration
```xml
11
2.5.12
org.springframework.boot
spring-boot-maven-plugin
```
### 2. Build Process
```bash
# Clean and build project
mvn clean package -DskipTests
# Run tests
mvn test
# Deploy to JFrog
mvn deploy
```
## Application Deployment
### 1. Tomcat Configuration
```bash
# Create tomcat.service
sudo tee /etc/systemd/system/tomcat.service << EOF
[Unit]
Description=Apache Tomcat Web Application Container
After=network.target
[Service]
Type=forking
Environment=JAVA_HOME=/usr/lib/jvm/java-11-openjdk-amd64
Environment=CATALINA_PID=/opt/tomcat/temp/tomcat.pid
Environment=CATALINA_HOME=/opt/tomcat
Environment=CATALINA_BASE=/opt/tomcat
Environment='CATALINA_OPTS=-Xms512M -Xmx1024M -server -XX:+UseParallelGC'
Environment='JAVA_OPTS=-Djava.awt.headless=true -Djava.security.egd=file:/dev/./urandom'
ExecStart=/opt/tomcat/bin/startup.sh
ExecStop=/opt/tomcat/bin/shutdown.sh
User=tomcat
Group=tomcat
UMask=0007
RestartSec=10
Restart=always
[Install]
WantedBy=multi-user.target
EOF
```
### 2. Nginx Configuration
```nginx
# /etc/nginx/conf.d/app.conf
upstream backend {
server internal-nlb-xxx.elb.amazonaws.com:8080;
}
server {
listen 80;
server_name example.com;
location / {
proxy_pass http://backend;
proxy_set_header Host $host;
proxy_set_header X-Real-IP $remote_addr;
}
location /static/ {
proxy_pass https://your-cloudfront-distribution.cloudfront.net;
}
}
```
## Load Balancing and Auto Scaling
### 1. Launch Template Configuration
```bash
aws ec2 create-launch-template \
--launch-template-name WebServerTemplate \
--version-description WebServerVersion1 \
--launch-template-data '{
"ImageId": "ami-xxx",
"InstanceType": "t3.micro",
"SecurityGroupIds": ["sg-xxx"],
"UserData": "IyEvYmluL2Jhc2gKCiMgSW5zdGFsbCBOZ2lueApzdWRvIHl1bSBpbnN0YWxsIG5naW54IC15Cg=="
}'
```
### 2. Auto Scaling Group
```bash
aws autoscaling create-auto-scaling-group \
--auto-scaling-group-name WebServerASG \
--launch-template LaunchTemplateName=WebServerTemplate,Version='$Latest' \
--min-size 2 \
--max-size 6 \
--desired-capacity 2 \
--vpc-zone-identifier "subnet-xxx,subnet-yyy" \
--target-group-arns "arn:aws:elasticloadbalancing:region:account-id:targetgroup/your-target-group/xxx" \
--health-check-type ELB \
--health-check-grace-period 300
```
# Monitoring and Maintenance
## CloudWatch Setup
### 1. Metrics Configuration
```bash
# Create custom metric for memory usage
cat << EOF > /opt/aws/scripts/memory-metrics.sh
#!/bin/bash
MEMORY_USAGE=\$(free | grep Mem | awk '{print \$3/\$2 * 100.0}')
aws cloudwatch put-metric-data \
--metric-name MemoryUsage \
--namespace CustomMetrics \
--value \$MEMORY_USAGE \
--dimensions InstanceId=\$(curl -s http://169.254.169.254/latest/meta-data/instance-id)
EOF
# Add to crontab
echo "* * * * * /opt/aws/scripts/memory-metrics.sh" | crontab -
```
### 2. Log Management
```bash
# Configure CloudWatch agent
cat << EOF > /opt/aws/amazon-cloudwatch-agent/etc/amazon-cloudwatch-agent.json
{
"agent": {
"metrics_collection_interval": 60,
"run_as_user": "root"
},
"logs": {
"logs_collected": {
"files": {
"collect_list": [
{
"file_path": "/opt/tomcat/logs/catalina.out",
"log_group_name": "/aws/tomcat/application",
"log_stream_name": "{instance_id}",
"timezone": "UTC"
}
]
}
}
},
"metrics": {
"metrics_collected": {
"mem": {
"measurement": [
"mem_used_percent"
]
},
"swap": {
"measurement": [
"swap_used_percent"
]
}
}
}
}
EOF
```
# Security Best Practices
## 1. Network Security
- Implement network ACLs
- Use security groups effectively
- Enable VPC Flow Logs
- Configure AWS WAF
## 2. Application Security
- Regular security patches
- Implement AWS Shield
- Use AWS Secrets Manager
- Enable AWS GuardDuty
## 3. Data Security
- Enable encryption at rest
- Use SSL/TLS for data in transit
- Regular security audits
- Implement backup strategies
# Troubleshooting Guide
## Common Issues and Solutions
### 1. Connection Issues
```bash
# Check connectivity
telnet database-endpoint 3306
# Verify security group rules
aws ec2 describe-security-groups --group-ids sg-xxx
# Test load balancer health
aws elbv2 describe-target-health --target-group-arn arn:aws:elasticloadbalancing:region:account-id:targetgroup/your-target-group/xxx
```
### 2. Performance Issues
```bash
# Check CPU usage
top -bn1
# Monitor memory usage
free -m
# Check disk usage
df -h
# Monitor Tomcat threads
ps -eLf | grep java | wc -l
```
# Contributing
## How to Contribute
1. Fork the repository
2. Create a feature branch
3. Commit your changes
4. Push to the branch
5. Create a Pull Request
## Development Setup
```bash
# Clone repository
git clone https://github.com/yourusername/your-repo.git
# Install dependencies
mvn install
# Run tests
mvn test
```
---
## π οΈ Author & Community
This project is maintained by **[Harshhaa](https://github.com/NotHarshhaa)** π‘.
Your feedback and contributions are welcome!
π§ **Connect with me:**
- **GitHub**: [@NotHarshhaa](https://github.com/NotHarshhaa)
- **Blog**: [ProDevOpsGuy](https://blog.prodevopsguytech.com)
- **Telegram Community**: [Join Here](https://t.me/prodevopsguy)
- **LinkedIn**: [Harshhaa Vardhan Reddy](https://www.linkedin.com/in/harshhaa-vardhan-reddy/)
---
## β Support the Project
If you found this project helpful, please consider:
- **Starring** β the repository
- **Sharing** it with your network
- **Contributing** to its improvement
### π’ Stay Connected
> [!Important]
> This documentation is continuously evolving. For the latest updates, please check the repository regularly.
---
## File: DevOps-Project-02/html-web-app/README.md
**Edit a file, create a new file, and clone from Bitbucket in under 2 minutes**
When you're done, you can delete the content in this README and update the file with details for others getting started with your repository.
*We recommend that you open this README in another tab as you perform the tasks below. You can [watch our video](https://youtu.be/0ocf7u76WSo) for a full demo of all the steps in this tutorial. Open the video in a new tab to avoid leaving Bitbucket.*
---
## Edit a file
You'll start by editing this README file to learn how to edit a file in GitHub.
1. Click **Source** on the left side.
2. Click the README.md link from the list of files.
3. Click the **Edit** button.
4. Delete the following text: *Delete this line to make a change to the README from GitHub.*
5. After making your change, click **Commit** and then **Commit** again in the dialog. The commit page will open and you'll see the change you just made.
6. Go back to the **Source** page.
---
## Create a file
Next, you'll add a new file to this repository.
1. Click the **New file** button at the top of the **Source** page.
2. Give the file a filename of **contributors.txt**.
3. Enter your name in the empty file space.
4. Click **Commit** and then **Commit** again in the dialog.
5. Go back to the **Source** page.
Before you move on, go ahead and explore the repository. You've already seen the **Source** page, but check out the **Commits**, **Branches**, and **Settings** pages.
---
## Clone a repository
Use these steps to clone from SourceTree, our client for using the repository command-line free. Cloning allows you to work on your files locally. If you don't yet have SourceTree, [download and install first](https://www.sourcetreeapp.com/). If you prefer to clone from the command line, see [Clone a repository](https://confluence.atlassian.com/x/4whODQ).
1. You'll see the clone button under the **Source** heading. Click that button.
2. Now click **Check out in SourceTree**. You may need to create a SourceTree account or log in.
3. When you see the **Clone New** dialog in SourceTree, update the destination path and name if you like to and then click **Clone**.
4. Open the directory you just created to see your repository files.
Now that you're more familiar with your GitHub repository, go ahead and add a new file locally. You can [push your change back to GitHub with SourceTree](https://confluence.atlassian.com/x/iqyBMg), or you can [add, commit,](https://confluence.atlassian.com/x/8QhODQ) and [push from the command line](https://confluence.atlassian.com/x/NQ0zDQ).
---
## File: DevOps-Project-02/README.md
# Deploy Scalable VPC Architecture on AWS Cloud
### TABLE OF CONTENTS
1. [Goal](https://github.com/NotHarshhaa/DevOps-Projects/blob/master/DevOps-Project-02/README.md#goal)
2. [Pre-Requisites](https://github.com/NotHarshhaa/DevOps-Projects/blob/master/DevOps-Project-02/README.md#pre-requisites)
3. [Pre-Deployment](https://github.com/NotHarshhaa/DevOps-Projects/blob/master/DevOps-Project-02/README.md#pre-deployment)
4. [VPC Deployment](https://github.com/NotHarshhaa/DevOps-Projects/blob/master/DevOps-Project-02/README.md#vpc-deployment)
5. [Validation](https://github.com/NotHarshhaa/DevOps-Projects/blob/master/DevOps-Project-02/README.md#validation)
## Goal
Deploy a Modular and Scalable Virtual Network Architecture with Amazon VPC.
## Pre-Requisites
1. You must be having an [AWS account](https://aws.amazon.com/) to create infrastructure resources on AWS cloud.
2. [Source Code](https://github.com/NotHarshhaa/DevOps-Projects/blob/master/DevOps-Project-02/html-web-app)
## Pre-Deployment
Customize the application dependencies mentioned below on AWS EC2 instance and create the Golden AMI.
1. AWS CLI
2. Install Apache Web Server
3. Install Git
4. Cloudwatch Agent
5. Push custom memory metrics to Cloudwatch.
6. AWS SSM Agent
## VPC Deployment
1. Build VPC network ( 192.168.0.0/16 ) for Bastion Host deployment as per the architecture shown above.
2. Build VPC network ( 172.32.0.0/16 ) for deploying Highly Available and Auto Scalable application servers as per the architecture shown above.
3. Create NAT Gateway in Public Subnet and update Private Subnet associated Route Table accordingly to route the default traffic to NAT for outbound internet connection.
4. Create Transit Gateway and associate both VPCs to the Transit Gateway for private communication.
5. Create Internet Gateway for each VPC and Public Subnet associated Route Table accordingly to route the default traffic to IGW for inbound/outbound internet connection.
6. Create Cloudwatch Log Group with two Log Streams to store the VPC Flow Logs of both VPCs.
7. Enable Flow Logs for both VPCs and push the Flow Logs to Cloudwatch Log Groups and store the logs in the respective Log Stream for each VPC.
8. Create Security Group for bastion host allowing port 22 from public.
9. Deploy Bastion Host EC2 instance in the Public Subnet with EIP associated.
10. Create S3 Bucket to store application specific configuration.
11. Create Launch Template with below configuration (replacing deprecated Launch Configuration).
1. Golden AMI
2. Instance Type β t3.micro (updated from t2.micro for better performance)
3. Userdata to pull the code from Git Repository to document root folder of webserver and start the httpd service.
4. IAM Role granting access to Session Manager and to S3 bucket created in the previous step to pull the configuration. (Do not grant S3 Full Access)
5. Security Group allowing port 22 from Bastion Host and Port 80 from Public.
6. Key Pair
12. Create Auto Scaling Group with Min: 2 Max: 4 with two Private Subnets associated to multiple AZs for high availability.
13. Create Target Group and associate it with ASG.
14. Create Application Load Balancer (ALB) in Public Subnets for better layer 7 routing and add Target Group as target.
15. Update route53 hosted zone with CNAME record routing the traffic to ALB.
## Validation
1. As DevOps Engineer login to Private Instances via Bastion Host.
2. Login to AWS Session Manager and access the EC2 shell from console.
3. Browse web application from public internet browser using domain name and verify that page loaded.
## π οΈ Author & Community
This project is crafted by **[Harshhaa](https://github.com/NotHarshhaa)** π‘.
Iβd love to hear your feedback! Feel free to share your thoughts.
π§ **Connect with me:**
- **GitHub**: [@NotHarshhaa](https://github.com/NotHarshhaa)
- **Blog**: [ProDevOpsGuy](https://blog.prodevopsguytech.com)
- **Telegram Community**: [Join Here](https://t.me/prodevopsguy)
- **LinkedIn**: [Harshhaa Vardhan Reddy](https://www.linkedin.com/in/harshhaa-vardhan-reddy/)
---
## β Support the Project
If you found this helpful, consider **starring** β the repository and sharing it with your network! π
### π’ Stay Connected
---
## File: DevOps-Project-03/README.md
# Fun with Linux for Cloud & DevOps Engineers
***New to Linux? Below assignment covering all the required basics of Linux to be familiar for an DevOps engineer.***
## Skills
Below skills are required to complete the deployment steps:
Linux User Management, Permissions, Directory Structure, File Systems, File Management
## Pre-Requisites
Login to AWS cloud and create Linux based EC2 instance to complete the below assignment.
## Deployment
1. Login to the server as super user and perform below
1. Create users and set passwords β user1, user2, user3
2. Create Groups β devops, aws
3. Change primary group of user2, user3 to βdevopsβ group
4. Add βawsβ group as secondary group to the βuser1β
5. Create the file and directory structure shown in the above diagram.
6. Change group of /dir1, /dir7/dir10, /f2 to βdevopsβ group
7. Change ownership of /dir1, /dir7/dir10, /f2 to βuser1β user.
2. Login as user1 and perform below
1. Create users and set passwords β user4, user5
2. Create Groups β app, database
3. Login as βuser4β and perform below
1. Create directory β /dir6/dir4
2. Create file β /f3
3. Move the file from β/dir1/f1β to β/dir2/dir1/dir2β
4. Rename the file β/f2β² to /f4β
4. Login as βuser1β and perform below
1. Create directory β β/home/user2/dir1β
2. Change to β/dir2/dir1/dir2/dir10β directory and create file β/opt/dir14/dir10/f1β using relative path method.
3. Move the file from β/opt/dir14/dir10/f1β to user1 home directory
4. Delete the directory recursively β/dir4β
5. Delete all child files and directories under β/opt/dir14β using single command.
6. Write this text βLinux assessment for an DevOps Engineer!! Learn with Fun!!β to the /f3 file and save it.
5. Login as βuser2β and perform below
1. Create file β/dir1/f2β
2. Delete /dir6
3. Delete /dir8
4. Replace the βDevOpsβ text to βdevopsβ in the /f3 file without using editor.
5. Using Vi-Editor copy the line1 and paste 10 times in the file /f3.
6. Search for the pattern βEngineerβ and replace with βengineerβ in the file /f3 using single command.
7. Delete /f3
6. Login as βrootβ user and perform below
1. Search for the file name βf3β in the server and list all absolute paths where f3 file is found.
2. Show the count of the number of files in the directory β/β
3. Print last line of the file β/etc/passwdβ
7. Login to AWS and create 5GB EBS volume in the same AZ of the EC2 instance and attach EBS volume to the Instance.
8. Login as βrootβuser and perform below
1. Create File System on the new EBS volume attached in the previous step
2. Mount the File System on /data directory
3. Verify File System utilization using βdf -hβ command β This command must show /data file system
4. Create file βf1β in the /data file system.
9. Login as βuser5β and perform below
1. Delete /dir1
2. Delete /dir2
3. Delete /dir3
4. Delete /dir5
5. Delete /dir7
6. Delete /f1 & /f4
7. Delete /opt/dir14
10. Logins as βrootβ user and perform below
1. Delete users β βuser1, user2, user3, user4, user5β
2. Delete groups β app, aws, database, devops
3. Delete home directories of all users βuser1, user2, user3, user4, user5β if any exists still.
4. Unmount /data file system
5. Delete /data directory
11. Login to AWS and detach EBS volume to the EC2 Instance and delete the volume and then terminate EC2 instance.
All done? still not confident? repeat the steps!
**Happy Learning!**
## π οΈ Author & Community
This project is crafted by **[Harshhaa](https://github.com/NotHarshhaa)** π‘.
Iβd love to hear your feedback! Feel free to share your thoughts.
π§ **Connect with me:**
- **GitHub**: [@NotHarshhaa](https://github.com/NotHarshhaa)
- **Blog**: [ProDevOpsGuy](https://blog.prodevopsguytech.com)
- **Telegram Community**: [Join Here](https://t.me/prodevopsguy)
- **LinkedIn**: [Harshhaa Vardhan Reddy](https://www.linkedin.com/in/harshhaa-vardhan-reddy/)
---
## β Support the Project
If you found this helpful, consider **starring** β the repository and sharing it with your network! π
### π’ Stay Connected
---
## File: DevOps-Project-04/README.md
# Production-Ready Django Deployment on AWS: Complete ECS & ECR DevOps Pipeline
**This comprehensive guide demonstrates how to deploy a Django-based production application onto AWS using ECS (Elastic Container Service) and ECR (Elastic Container Registry). We'll cover the complete DevOps pipeline from containerization to deployment, including security best practices, monitoring setup, and production optimization.**
## π Table of Contents
- [Overview](#overview)
- [Prerequisites](#prerequisites)
- [Project Structure](#project-structure)
- [Django Web Framework](#django-web-framework)
- [Docker & Containerization](#docker--containerization)
- [AWS ECR Setup](#aws-ecr-setup)
- [AWS ECS Deployment](#aws-ecs-deployment)
- [Security & Best Practices](#security--best-practices)
- [Monitoring & Logging](#monitoring--logging)
- [Troubleshooting](#troubleshooting)
- [Cost Optimization](#cost-optimization)
- [Alternative Deployments](#alternative-deployments)
## π― Overview
This project provides a complete DevOps pipeline for deploying Django applications on AWS cloud infrastructure. The solution includes:
- **Containerization**: Docker-based application packaging
- **Registry Management**: ECR for secure image storage
- **Orchestration**: ECS for container management
- **Scalability**: Auto-scaling and load balancing
- **Security**: IAM roles, security groups, and secrets management
- **Monitoring**: CloudWatch integration and health checks
## π Prerequisites
### Technical Requirements
- **Python 3.9+** installed locally
- **Docker Desktop** or Docker Engine
- **AWS Account** with appropriate permissions
- **AWS CLI** configured with credentials
- **Django** framework knowledge
- **Basic understanding** of containers and cloud concepts
### AWS Permissions Required
- Amazon ECR: Full access
- Amazon ECS: Full access
- Amazon EC2: Full access
- IAM: Basic permissions for role creation
- CloudWatch: Read/Write permissions
- VPC: Network configuration permissions
### Development Environment Setup
```bash
# Install Django and create project
pip install django
django-admin startproject myproject
cd myproject
# Create requirements file
pip freeze > requirements.txt
# Test locally
python manage.py runserver
```
## ποΈ Project Structure
```
myproject/
βββ myproject/
β βββ __init__.py
β βββ settings.py
β βββ urls.py
β βββ wsgi.py
βββ apps/
β βββ __init__.py
β βββ [your-apps]
βββ static/
βββ media/
βββ templates/
βββ requirements.txt
βββ Dockerfile
βββ docker-compose.yml
βββ .dockerignore
βββ .env.example
βββ README.md
```
## π Django Web Framework
**Django is a high-level Python web framework that encourages rapid development and clean, pragmatic design. Built by experienced developers, it takes care of much of the hassle of web development, so you can focus on writing your app without needing to reinvent the wheel.**
### Key Features
- **Batteries-included**: ORM, authentication, admin panel, forms
- **Security**: Built-in protection against CSRF, XSS, SQL injection
- **Scalability**: Designed for high-traffic applications
- **Documentation**: Comprehensive docs and active community
- **MVT Architecture**: Model-View-Template pattern
### Production Considerations
- **Static files handling**: Use AWS S3 + CloudFront
- **Database**: Amazon RDS (PostgreSQL/MySQL)
- **Caching**: Redis/ElastiCache
- **Session storage**: Redis or database
- **Environment variables**: AWS Secrets Manager
## π³ Docker & Containerization
### What is Docker?
**Docker is an open platform for developing, shipping, and running applications in containers. Containerization provides a lightweight, portable way to package applications with all their dependencies, ensuring consistency across different environments.**
### Benefits of Containerization
- **Portability**: Run anywhere Docker is installed
- **Isolation**: Applications don't interfere with each other
- **Scalability**: Easy to scale horizontally
- **Version Control**: Image versioning and rollbacks
- **Resource Efficiency**: Shared OS kernel, lightweight
### Docker Workflow
1. **Write Dockerfile**: Define application environment
2. **Build Image**: Create container image with dependencies
3. **Test Locally**: Verify container works as expected
4. **Push to Registry**: Store in ECR for deployment
5. **Deploy**: Run containers in ECS
### Dockerfile Best Practices
- Use multi-stage builds for smaller images
- Leverage layer caching effectively
- Use specific base image versions
- Minimize attack surface
- Optimize for production performance
## π¦ AWS Elastic Container Registry (ECR)
**Amazon Elastic Container Registry (Amazon ECR) is a fully managed container image registry service that makes it easy to store, manage, share, and deploy your container images. ECR eliminates the need to operate your own container repositories or worry about scaling the underlying infrastructure.**
### Key Features
- **Fully Managed**: No infrastructure to maintain
- **Secure**: IAM integration and encryption
- **Scalable**: Automatic scaling based on demand
- **Integrated**: Works seamlessly with ECS and EKS
- **Cost-effective**: Pay only for storage and data transfer
- **Vulnerability Scanning**: Automated security scans
### ECR Repository Setup
#### Step 1: Create Repository
```bash
# Using AWS CLI
aws ecr create-repository \
--repository-name django-app \
--image-scanning-configuration scanOnPush=true \
--image-tag-mutability MUTABLE \
--region us-east-1
```
#### Step 2: Configure Repository Policies
```json
{
"Version": "2008-10-17",
"Statement": [
{
"Sid": "AllowPull",
"Effect": "Allow",
"Principal": {
"AWS": "arn:aws:iam::account-id:user/ecs-user"
},
"Action": [
"ecr:GetDownloadUrlForLayer",
"ecr:BatchGetImage",
"ecr:BatchCheckLayerAvailability"
]
}
]
}
```
### ECR Image Management
#### Build Docker Image
```bash
# Build the Docker image
docker build -t django-app:latest .
# Verify image creation
docker images | grep django-app
```
#### Authenticate with ECR
```bash
# Get ECR login password
aws ecr get-login-password --region us-east-1 | \
docker login --username AWS --password-stdin \
123456789012.dkr.ecr.us-east-1.amazonaws.com
```
#### Tag and Push Image
```bash
# Tag image for ECR
docker tag django-app:latest \
123456789012.dkr.ecr.us-east-1.amazonaws.com/django-app:latest
# Push to ECR
docker push 123456789012.dkr.ecr.us-east-1.amazonaws.com/django-app:latest
```
#### Lifecycle Policies
```bash
# Create lifecycle policy to clean up old images
aws ecr put-lifecycle-policy \
--repository-name django-app \
--lifecycle-policy-text 'file://lifecycle-policy.json'
```
**lifecycle-policy.json:**
```json
{
"rules": [
{
"rulePriority": 1,
"description": "Keep last 10 images",
"selection": {
"tagStatus": "tagged",
"tagPrefixList": ["v"],
"countType": "imageCountMoreThan",
"countNumber": 10
},
"action": {
"type": "expire"
}
}
]
}
```
## π AWS Elastic Container Service (ECS)
**Amazon Elastic Container Service (ECS) is a highly scalable, high-performance container orchestration service that supports Docker containers and allows you to easily run applications on a managed cluster of Amazon EC2 instances or AWS Fargate.**
### ECS Launch Types
#### EC2 Launch Type
- **Full Control**: Access to underlying EC2 instances
- **Cost Optimization**: Better for sustained workloads
- **Customization**: Can install additional software
- **Networking**: Direct control over networking configuration
#### Fargate Launch Type
- **Serverless**: No EC2 instances to manage
- **Pay-per-use**: Billing based on vCPU and memory
- **Isolation**: Each task gets its own isolated environment
- **Simplicity**: Reduced operational overhead
### Key ECS Components
#### 1. ECS Cluster
```bash
# Create ECS cluster using AWS CLI
aws ecs create-cluster \
--cluster-name django-cluster \
--service-connect default \
--region us-east-1
```
#### 2. Task Definition
```json
{
"family": "django-task",
"networkMode": "awsvpc",
"requiresCompatibilities": ["FARGATE"],
"cpu": "256",
"memory": "512",
"executionRoleArn": "arn:aws:iam::account:role/ecsTaskExecutionRole",
"containerDefinitions": [
{
"name": "django-container",
"image": "123456789012.dkr.ecr.us-east-1.amazonaws.com/django-app:latest",
"portMappings": [
{
"containerPort": 8000,
"protocol": "tcp"
}
],
"environment": [
{
"name": "DJANGO_SETTINGS_MODULE",
"value": "myproject.settings.production"
}
],
"logConfiguration": {
"logDriver": "awslogs",
"options": {
"awslogs-group": "/ecs/django-app",
"awslogs-region": "us-east-1",
"awslogs-stream-prefix": "ecs"
}
}
}
]
}
```
#### 3. Service Definition
```bash
# Create ECS service
aws ecs create-service \
--cluster django-cluster \
--service-name django-service \
--task-definition django-task \
--desired-count 2 \
--launch-type FARGATE \
--network-configuration "awsvpcConfiguration={subnets=[subnet-12345,subnet-67890],securityGroups=[sg-12345],assignPublicIp=ENABLED}" \
--deployment-configuration "maximumPercent=200,minimumHealthyPercent=100" \
--health-check-grace-period-seconds 30
```
### ECS Deployment Steps
#### Step 1: Create VPC and Networking
```bash
# Create VPC
aws ec2 create-vpc --cidr-block 10.0.0.0/16 --tag-specifications 'ResourceType=vpc,Tags=[{Key=Name,Value=django-vpc}]'
# Create subnets
aws ec2 create-subnet --vpc-id vpc-12345 --cidr-block 10.0.1.0/24 --availability-zone us-east-1a
aws ec2 create-subnet --vpc-id vpc-12345 --cidr-block 10.0.2.0/24 --availability-zone us-east-1b
# Create security groups
aws ec2 create-security-group --group-name django-sg --description "Security group for Django app" --vpc-id vpc-12345
```
#### Step 2: Create IAM Roles
```bash
# Create task execution role
aws iam create-role --role-name ecsTaskExecutionRole --assume-role-policy-document file://trust-policy.json
aws iam attach-role-policy --role-name ecsTaskExecutionRole --policy-arn arn:aws:iam::aws:policy/service-role/AmazonECSTaskExecutionRolePolicy
```
#### Step 3: Register Task Definition
```bash
aws ecs register-task-definition --cli-input-json file://task-definition.json
```
#### Step 4: Create Service with Load Balancer
```bash
# Create Application Load Balancer
aws elbv2 create-load-balancer \
--name django-alb \
--subnets subnet-12345 subnet-67890 \
--security-groups sg-12345
# Create target group
aws elbv2 create-target-group \
--name django-tg \
--protocol HTTP \
--port 8000 \
--vpc-id vpc-12345 \
--target-type ip
# Create service with load balancer
aws ecs create-service \
--cluster django-cluster \
--service-name django-service \
--task-definition django-task \
--desired-count 2 \
--launch-type FARGATE \
--load-balancers targetGroupArn=arn:aws:elasticloadbalancing:region:account:targetgroup/django-tg,containerName=django-container,containerPort=8000
```
#### Step 5: Configure Auto Scaling
```bash
# Create auto scaling target
aws application-autoscaling register-scalable-target \
--service-namespace ecs \
--resource-id service/django-cluster/django-service \
--scalable-dimension ecs:service:DesiredCount \
--min-capacity 1 \
--max-capacity 10
# Create scaling policy
aws application-autoscaling put-scaling-policy \
--service-namespace ecs \
--resource-id service/django-cluster/django-service \
--scalable-dimension ecs:service:DesiredCount \
--policy-name django-scale-out \
--policy-type TargetTrackingScaling \
--target-tracking-scaling-policy-configuration file://scaling-policy.json
```
## π Security & Best Practices
### Container Security
- **Use non-root users** in containers
- **Scan images** for vulnerabilities
- **Implement resource limits** to prevent DoS attacks
- **Use secrets management** instead of environment variables
### AWS Security
- **IAM Roles**: Principle of least privilege
- **Security Groups**: Network-level security
- **VPC**: Isolated network environment
- **Encryption**: Data at rest and in transit
### Django Security Settings
```python
# production.py
SECURE_SSL_REDIRECT = True
SECURE_HSTS_SECONDS = 31536000
SECURE_HSTS_INCLUDE_SUBDOMAINS = True
SECURE_HSTS_PRELOAD = True
SECURE_CONTENT_TYPE_NOSNIFF = True
SECURE_BROWSER_XSS_FILTER = True
X_FRAME_OPTIONS = 'DENY'
CSRF_COOKIE_SECURE = True
SESSION_COOKIE_SECURE = True
```
## π Monitoring & Logging
### CloudWatch Integration
```json
{
"logConfiguration": {
"logDriver": "awslogs",
"options": {
"awslogs-group": "/ecs/django-app",
"awslogs-region": "us-east-1",
"awslogs-stream-prefix": "ecs",
"awslogs-datetime-format": "%Y-%m-%dT%H:%M:%S.%fZ"
}
}
}
```
### Health Checks
```bash
# Configure health checks in task definition
"healthCheck": {
"command": ["CMD-SHELL", "curl -f http://localhost:8000/health/ || exit 1"],
"interval": 30,
"timeout": 5,
"retries": 3,
"startPeriod": 60
}
```
### Metrics to Monitor
- **CPU and Memory utilization**
- **Request/response times**
- **Error rates**
- **Database connections**
- **Container restart counts**
## π οΈ Troubleshooting
### Common Issues
#### Container Won't Start
```bash
# Check task logs
aws logs get-log-events \
--log-group-name /ecs/django-app \
--log-stream-name ecs/django-container/abcdef123456
# Describe task failure
aws ecs describe-tasks --cluster django-cluster --tasks abcdef123456
```
#### Network Issues
```bash
# Check security group rules
aws ec2 describe-security-groups --group-ids sg-12345
# Test connectivity
aws ecs execute-command \
--cluster django-cluster \
--task abcdef123456 \
--container django-container \
--command "curl -I http://localhost:8000" \
--interactive
```
#### Performance Issues
- **Monitor CloudWatch metrics**
- **Check database connections**
- **Review container resource limits**
- **Analyze application logs**
## π° Cost Optimization
### ECS Cost Saving Tips
- **Use Fargate Spot** for non-critical workloads
- **Implement auto-scaling** to match demand
- **Right-size containers** based on actual usage
- **Schedule scaling** for predictable patterns
### ECR Cost Management
- **Implement lifecycle policies** to clean up old images
- **Use image scanning** only when needed
- **Optimize image sizes** with multi-stage builds
## π Alternative Deployments
### AWS Elastic Beanstalk
- **Simplified deployment** process
- **Built-in load balancing** and auto-scaling
- **Managed platform** updates
- **Less configuration** required
### Kubernetes (EKS)
- **More complex** but highly flexible
- **Portability** across cloud providers
- **Rich ecosystem** and tooling
- **Better for microservices** architecture
### Serverless Options
- **AWS Lambda** for API endpoints
- **Amplify** for full-stack applications
- **App Runner** for containerized apps
## π Success! π
**Congratulations! You have successfully deployed your Django Application on AWS cloud using ECS and ECR with production-ready configurations.**
### Verification Steps
1. **Check Load Balancer DNS** in browser
2. **Verify SSL certificate** is working
3. **Test application functionality**
4. **Monitor CloudWatch metrics**
5. **Review security configurations**
### Post-Deployment Checklist
- [ ] SSL/TLS certificates configured
- [ ] Database backups enabled
- [ ] Monitoring alerts set up
- [ ] Log rotation configured
- [ ] Security groups reviewed
- [ ] Auto-scaling policies tested
- [ ] Disaster recovery plan documented
### Next Steps
- **Implement CI/CD pipeline** with AWS CodePipeline
- **Add comprehensive testing** suite
- **Set up staging environment**
- **Implement blue-green deployments**
- **Add performance monitoring** with APM tools
**Happy Learning and Happy Deploying! π**
## π οΈ Author & Community
This project is crafted by **[Harshhaa](https://github.com/NotHarshhaa)** π‘.
Iβd love to hear your feedback! Feel free to share your thoughts.
π§ **Connect with me:**
- **GitHub**: [@NotHarshhaa](https://github.com/NotHarshhaa)
- **Blog**: [ProDevOpsGuy](https://blog.prodevopsguytech.com)
- **Telegram Community**: [Join Here](https://t.me/prodevopsguy)
- **LinkedIn**: [Harshhaa Vardhan Reddy](https://www.linkedin.com/in/harshhaa-vardhan-reddy/)
---
## β Support the Project
If you found this helpful, consider **starring** β the repository and sharing it with your network! π
### π’ Stay Connected
---
## File: DevOps-Project-05/README.md
# Deploy your code on a Docker Container using Jenkins on AWS
**In this blog, we are going to deploy a Java Web app on a Docker Container built on an EC2 Instance through the use of Jenkins.**
### Agenda
* Setup Jenkins
* Setup & Configure Maven and Git
* Integrating GitHub and Maven with Jenkins
* Setup Docker Host
* Integrate Docker with Jenkins
* Automate the Build and Deploy process using Jenkins
* Test the deployment
### Prerequisites
* AWS Account
* Git/ Github Account with the Source Code
* A local machine with CLI Access
* Familiarity with Docker and Git
## Step 1: Setup Jenkins Server on AWS EC2 Instance
* Setup a Linux EC2 Instance
* Install Java
* Install Jenkins
* Start Jenkins
* Access Web UI on port 8080
_*Log in to the Amazon management console, open EC2 Dashboard, click on the Launch Instance drop-down list, and click on Launch Instance as shown below:*_
_*Once the Launch an instance window opens, provide the name of your EC2 Instance:*_
_Choose an Instance Type. Here you can select the type of machine, number of vCPUs, and memory that you want to have. Select t2.micro which is free-tier eligible._
_For this demo, we will select an already existing key pair. You can create new key pair if you donβt have:_
_Now under Network Settings, Choose the default VPC with Auto-assign public IP in enable mode. Create a new Security Group, provide a name for your security group, allow ssh traffic, and custom default TCP port of 8080 which is used by Jenkins._
_Rest of the settings we will keep them at default and go ahead and click on Launch Instance_
_On the next screen you can see a success message after the successful creation of the EC2 instance, click on Connect to instance button:_
_Now connect to instance wizard will open, go to SSH client tab and copy the provided chmod and SSH command:_
_Open any SSH Client in your local machine, take the public IP of your EC2 Instance, and add the pem key and you will be able to access your EC2 machine in my case I am using MobaXterm on Windows:_
_After logging in to our EC2 machine we will install Jenkins following the instructions from the official Jenkins website:_
### To use this repository, run the following command
```
sudo wget -O /etc/yum.repos.d/jenkins.repo https://pkg.jenkins.io/redhat-stable/jenkins.repo
```
```
sudo rpm --import https://pkg.jenkins.io/redhat-stable/jenkins.io-2023.key
```
**Output:**
_Now letβs install epel packages for Amazon Linux AMI:_
_After installing epel packages, letβs install java-openjdk11:_
_Letβs check the version of Java now:_
_Now letβs install Jenkins with the below command as shown in the output:_
_After successful installation Letβs enable and start Jenkins service in our EC2 Instance:_
_Now letβs try to access the Jenkins server through our browser. For that take the public IP of your EC2 instance and paste it into your favorite browser and should see something like this:_
_To unlock Jenkins we need to go to the path /var/lib/jenkins/secrets/initialAdminPassword and fetch the admin password to proceed further:_
_Now on the Customize Jenkins page, we can go ahead and install the suggested plugins:_
_Now we can create our first Admin user, provide all the required data and proceed to save and continue._
_Now we are ready to use our Jenkins Server._
## Step 2: Integrate GitHub with Jenkins
* Install Git on Jenkins Instance
* Install Github Plugin on Jenkins GUI
* Configure Git on Jenkins GUI
_Letβs first install Git on our EC2 instance with the below command:_
_We can check the version as shown in the below screenshot:_
_To install the GitHub plugin lets go to our Jenkins Dashboard and click on manage Jenkins as shown:_
_On the next page, click on manage plugins:_
_Now in order to install any plugin we need to select Available Plugins, search for Github Integration, select the plugin, and finally click on `Install without restart` as shown below:_
_Now letβs configure Git on Jenkins. Go to Manage Jenkins, and click on Global Tool Configuration._
_Under `Git installations`, provide the name Git, and under `Path`, we can either provide the complete path where our Git is installed on the Jenkins machine or just put any name, in my case I put Git to allow Jenkins to automatically search for Git. Then click on Save to complete the installation._
## Step 3: Integrate Maven with Jenkins
* Setup Maven on Jenkins Server
* Setup Environment Variables
JAVA_HOME,M2,M2_HOME
* Install Maven Plugin
* Configure Maven and Java
_To install Maven on our Jenkins Server we will switch to the /opt directory and download the Maven package:_
_Now we will extract the tar.gz file:_
_Now we will set up Environment Variables for our root user in bash_profile in order to access Maven from any location in our Server Go to the home directory of your Jenkins server and edit the bash_profile file as shown in the below steps:_
_In the .bash_profile file, we need to add Maven and Java paths and load these values._
_To verify follow the below steps:_
_With this setup, we can execute maven commands from anywhere on the server:_
_Now we need to update the paths where Java and Maven have been installed in the Jenkins UI. We will first install the Maven Integration Plugin as shown below:_
_After clicking on Install without restart, go again to manage Jenkins and select Global Tool configuration to set the paths for Java and Maven._
**For JAVA:**
**For MAVEN:**
_Click on save and hence we have successfully Integrated Java and Maven with Jenkins._
## Step 4: Setup a Docker Host
* Setup a Linux EC2 Instance
* Install Docker
* Start Docker Services
* Run Basic Docker Commands
_Let's first launch an EC2 Instance. We will skip the steps here as we have already shown earlier how to create an EC2 Instance._
_Below is the screenshot of our newly created EC2 Instance on which we will install Docker:_
_We will first install Docker on this EC2 Instance:_
_After the successful installation of Docker, letβs verify the version of Docker:_
_Also, letβs enable and start the Docker Service:_
### Create Tomcat Docker Container
_In my previous blog, I deployed a Java code on a Tomcat VM, here in this blog we will deploy on a Tomcat Docker container._
_We will first pull the official Tomcat docker image from the Docker Hub and then run the container out of the same image._
Letβs now create a Container from the same Image with the command:
```
docker run -d --name tomcat-container -p 8081:8080 tomcat
```
_The above command runs a docker container in detached mode with the name tomcat-container and we are exposing port 8081 of our host machine with port 8080 of our container and it's using the latest image of tomcat._
Let's verify the running container on our EC2 machine:
_Before accessing our container from the browser we need to allow port 8081 in the Security Group of our EC2 docker-host machine._
Go to the Security group of your EC2 machine and click on Edit inbound rules:
_Click on Add rule, select Custom TCP as type, and in port range put 8081β9000 in case we need it in the future, and under source select from anywhere and then click on Save rules to proceed:_
Now letβs take the public IP of our Docker-host EC2 machine and with port 8081 access it from our browser:
_From the above screenshot, you can see that although there is a 404 error, it also displays Apache Tomcat at the bottom which means the installation is successful however this is a known issue with the Tomcat docker image that we will fix in the next steps._
_The above issue occurs because whenever we try to access the Tomcat server from the browser it will look for the files in /webapps directory which is empty and the actual files are being stored in /webapps.dist._
_So in order to fix this issue we will copy all the content from webapps.dist to webapps directory and that will resolve the issue._
Let's access our tomcat container and perform the steps as shown below:
Once we are in the tomcat container go to the /webapps.dist directory and copy all the content to webapps directory:
After that we should be able to access our tomcat docker container:
_Somehow if we stop this container and start another container with the same Image on a different port we will face the same issue of 404 error.
This happens because every time we launch a new container we are using a new Image as the previous container gets deleted._
_This issue can be solved by creating our own Docker Image with the appropriate changes required to run our container. This can be achieved by creating a Dockerfile on which we can mention the steps required to build the docker image and from that Image, we can run the container._
**Create a Customized Dockerfile for Tomcat:**
_To create the Dockerfile we will use the official Image of Tomcat and with it will mention the step to copy the contents from the directory /webapps.dist to /webapps:_
```
FROM tomcat:latest
RUN cp -R /usr/local/tomcat/webapps.dist/* /usr/local/tomcat/webapps
```
Now let's build the Docker Image using this Dockerfile using the below command:
```
docker build -t tomcatserver .
```
Letβs verify with the docker images command:
Now is the time to run the docker container out of our customized docker image which we built using our dockerfile. The command is as:
```
docker run -d --name tomcat-server -p 8085:8080 tomcatserver
```
_Here you should remember that we have already allowed port range 8081β9000 in the security group of our docker EC2 Instance._
Letβs verify using the docker ps command:
Also, let's try to access the Tomcat server on port 8085 from the browser:
_Hence we should be now able to launch as many times the same container without facing any issues by utilizing our customizable Dockerfile._
### Step 5: Integrate Docker with Jenkins
* Create a dockeradmin user
* Install the βPublish Over SSHβ plugin
* Add Dockerhost to Jenkins βconfigure systemsβ
_Let's first create a dockeradmin user and create a password for it as well._
Now letβs add this user to the Docker group with the below command:
```
usermod -aG docker dockeradmin
```
_Now in order to access the server using the newly created User we need to allow password-based authentication. For that, we need to do some changes in the /etc/ssh/sshd_config file._
_As you can see above we need to uncomment PasswordAuthentication yes and comment out PasswordAuthentication no._
Once we have updated the config file we need to restart the services with the command:
```
service sshd reload
Redirecting to /bin/systemctl reload sshd.service
```
_Now we would be able to log in using the dockeradmin user credentials:_
Now next step is to integrate Docker with Jenkins, for that we need to install the βPublish Over SSHβ plugin:
Go to Manage Jenkins > Manage Plugins > Available plugins and search for publish over ssh plugin:
Click on Install without restart to install the plugin.
Now we need to configure our dockerhost in Jenkins. For that go to Manage Jenkins > Configure System:
On the next page after scrolling down you would be able to see Publish over SSH section where you need to add a new SSH server with the info as shown below:
_It should be noted that it's best practice to use ssh keys however for this demo we are using password-based authentication. In the above screenshot, we have provided details of our Docker host which we created on EC2 Instance. Also, note the use of private IP under Hostname as both our Jenkins Server and Dockerhost are on the same subnet. We can also use Public IP here._
_Click on Apply and Save to proceed. With this, our Docker integration with Jenkins is successfully accomplished._
### Step 6: Create Jenkins Job to Build and Copy Artifacts on to Docker Host
_In this section, we would create a new job in Jenkins however we would copy from the existing job_
_Click on Ok to configure the job._
_On the configure settings you will notice that our new job has inherited all the settings from our previous build job. However, you can change the description and other settings:_
Now one important change we need to do is to delete the previous job post-build step and select Send files or execute commands over SSH option.
_Under the SSH server, it will display the already created ssh server which we created in our earlier steps. Then we need to provide the path from where the WAR file will be copied and then deployed on the remote server for which we also need to provide a path under the Remote directory. Then finally Apply and Save to proceed._
_Once we save the project the build will be triggered automatically due to Poll by SCM feature which we have enabled._
Now if we check the console output of our job we should see the Job has been successfully finished.
We can also verify by checking the presence webapp.war file on our docker EC2 machine:
## Step 7: Update Dockerfile to copy Artifacts to launch New Container
_In this step, we will create a Dockerfile to include the webapp.war file to launch a new container using our Java web Application. For that, we need to copy our artifacts to the location where we have our Dockerfile._
We will create a separate directory named docker under the root user of our dockerhost inside /opt.
_As you can in the above screenshot that this new docker directory is owned by root and in Jenkins configuration settings we have mentioned that the artifacts are owned by the dockeradmin user. So letβs give the ownership of this directory to dockeradmin user._
_Now let's copy the Dockerfile which we created in earlier steps to this newly created docker directory and change its ownership to dockeradmin as well:_
```
mv Dockerfile /opt/docker/
cd /opt/docker/
chown -R dockeradmin:dockeradmin /opt/docker/
ll
total 4
-rw-r--r-- 1 dockeradmin dockeradmin 89 May 10 12:08 Dockerfile
```
Now we need to configure our Jenkins job to change the remote directory from /home/dockeradmin to //opt//docker:
Click on Apply and Save to build the Job manually.
If the build is successful we can see the webapp.war file in the /opt/docker directory of our dockerhost:
_Now in our Dockerfile, we need to mention the location of this WAR file and copy this file onto the /usr/local/tomcat/webapps location in the container._
**Dockerfile:**
```
FROM tomcat:latest
RUN cp -R /usr/local/tomcat/webapps.dist/* /usr/local/tomcat/webapps
COPY ./*.war /usr/local/tomcat/webapps
```
_Letβs now build a new image using this updated Dockerfile with the command:_
```
docker build -t tomcat:v1 .
```
**Output:**
In the next step letβs now create a container out of this image with the command:
```
docker run -d --name tomcatv1 -p 8086:8080 tomcat:v1
```
**Output:**
Now letβs access this application from our browser using URL
_So far we have successfully copied the artifacts to our dockerhost and then manually used docker commands like docker build and docker run to deploy our application on the docker container._
## Step 8: Automate Build and Deployment on Docker Container
_In this step, we will try to automate end to end Jenkins pipeline right from when we commit our code to GitHub, it should build it, create an artifact and then copy the artifacts to the docker host, create a docker image, and finally create a docker container to deploy the project._
_For this automation to happen we need to go to our Jenkins job, select configure, and under Send files or execute commands over SSH there is Exec command field where we need to put some commands as shown below:_
```
cd /opt/docker;
docker build -t regapp:v1 .;
docker run -d --name registerapp -p 8087:8080 regapp:v1
```
_Now letβs do some minor changes in our code and commit the changes which will trigger the Jenkins build process and we can see the results of automation._
_As soon as I made some changes in the Readme file in the GitHub repository the build got triggered in our Jenkins Job._
If the build is successful we should see our new docker image and docker container as shown in the below screenshot:
Also, we if access our new dockerized app from our browser on port 8087, the result should be something like this:
### Conclusion
**In this blog, we learned how to automate the build and deploy process using GitHub, Jenkins, Docker, and AWS EC2.**
**Happy Learning!**
## π οΈ Author & Community
This project is crafted by **[Harshhaa](https://github.com/NotHarshhaa)** π‘.
Iβd love to hear your feedback! Feel free to share your thoughts.
π§ **Connect with me:**
- **GitHub**: [@NotHarshhaa](https://github.com/NotHarshhaa)
- **Blog**: [ProDevOpsGuy](https://blog.prodevopsguytech.com)
- **Telegram Community**: [Join Here](https://t.me/prodevopsguy)
- **LinkedIn**: [Harshhaa Vardhan Reddy](https://www.linkedin.com/in/harshhaa-vardhan-reddy/)
---
## β Support the Project
If you found this helpful, consider **starring** β the repository and sharing it with your network! π
### π’ Stay Connected
---
## File: DevOps-Project-06/README.md
# π Implementation of the Entire Advanced CI/CD Pipeline with Major DevOps Tools
A comprehensive end-to-end CI/CD pipeline leveraging **Terraform**, **Ansible**, **Jenkins**, **SonarQube**, **JFrog**, **Docker**, **EKS**, **Prometheus**, and **Grafana**.
---
## 𧩠Pipeline Architecture Overview
> Below is a step-by-step implementation breakdown of the complete DevOps CI/CD process.
**π§ Step 1: Infrastructure Provisioning (Terraform)**
- Provisioned **VPC**, **Security Groups**, **Ansible Controller**, **Jenkins Master**, and **Agent Instances** using Terraform.
π [Step Details](https://github.com/NotHarshhaa/DevOps-Projects/blob/master/DevOps-Project-06/Steps/step1.md#L1)
**π Step 2: SSH Configuration**
- Set up password-less authentication between Ansible Controller and Agent nodes.
π [Step Details](https://github.com/NotHarshhaa/DevOps-Projects/blob/master/DevOps-Project-06/Steps/step2.md#L1)
**βοΈ Step 3: Jenkins Setup (Ansible)**
- Configured Jenkins Master and Agent nodes.
- Agent configured as **Maven Build Server**.
π [Step Details](https://github.com/NotHarshhaa/DevOps-Projects/blob/master/DevOps-Project-06/Steps/step3.md#L1)
**π Step 4: Jenkins Master-Agent Integration**
- Connected Jenkins Master to Agent with credentials.
π [Step Details](https://github.com/NotHarshhaa/DevOps-Projects/blob/master/DevOps-Project-06/Steps/step4.md#L1)
**π Step 5: GitHub Integration**
- Added GitHub credentials.
- Created **Multibranch Pipeline Job**.
π [Step Details](https://github.com/NotHarshhaa/DevOps-Projects/blob/master/DevOps-Project-06/Steps/step5.md#L1)
**π¨ Step 6: Webhook Trigger Setup**
- Configured GitHub webhook trigger using **Multibranch Scan Webhook Trigger Plugin**.
π [Step Details](https://github.com/NotHarshhaa/DevOps-Projects/blob/master/DevOps-Project-06/Steps/step6.md#L1)
**π§ͺ Step 7: SonarQube Integration**
- Generated access token in SonarCloud.
- Installed SonarQube scanner plugin.
- Added SonarQube server and scanner to Jenkins.
- Configured `sonar-project.properties`.
- Added **code quality**, **unit test**, and **build** stages in `Jenkinsfile`.
π [Step-by-step Setup](https://github.com/NotHarshhaa/DevOps-Projects/blob/master/DevOps-Project-06/Steps/step7.md#L3)
**π¦ Step 8: JFrog Artifactory Integration**
- Configured JFrog credentials.
- Installed Artifactory plugin in Jenkins.
π [Details](https://github.com/NotHarshhaa/DevOps-Projects/blob/master/DevOps-Project-06/Steps/step8.md#L1)
**π³ Step 9: Docker Image & Push to JFrog**
- Built Docker image from `.jar`.
- Pushed to JFrog Artifactory using **Docker Pipeline Plugin**.
- Added **Docker Build & Publish** stage to Jenkinsfile.
π [Details](https://github.com/NotHarshhaa/DevOps-Projects/blob/master/DevOps-Project-06/Steps/step9.md#L1)
**βΈοΈ Step 10: EKS Cluster Setup**
- Created **EKS cluster** via Terraform.
- Installed `kubectl` and AWS CLI in Jenkins slave.
- Configured Kube credentials using:
```
aws eks update-kubeconfig --region --name
```
π [Details](https://github.com/NotHarshhaa/DevOps-Projects/blob/master/DevOps-Project-06/Steps/step10.md#L3)
**π Step 11: Kubernetes Deployment**
- Pulled Docker image using Kubernetes secrets.
- Deployed it to EKS cluster using **Deployment** and **Service** resources.
- Exposed via LoadBalancer.
π [Details](https://github.com/NotHarshhaa/DevOps-Projects/blob/master/DevOps-Project-06/Steps/step11.md#L1)
**π Step 12: Monitoring (Prometheus & Grafana)**
- Added Prometheus Helm repo.
- Enabled monitoring in EKS cluster.
- Changed Prometheus and Grafana services to `LoadBalancer` for browser access.
π [Details](https://github.com/NotHarshhaa/DevOps-Projects/blob/master/DevOps-Project-06/Steps/step12.md#L1)
---
## π¨βπ» Author & Community
This project is maintained by **[Harshhaa](https://github.com/NotHarshhaa)** π
### π¬ Connect with Me:
- π **GitHub**: [@NotHarshhaa](https://github.com/NotHarshhaa)
- π **Blog**: [ProDevOpsGuy](https://blog.prodevopsguytech.com)
- π¬ **Telegram**: [Join Community](https://t.me/prodevopsguy)
- π **LinkedIn**: [Harshhaa Vardhan Reddy](https://www.linkedin.com/in/harshhaa-vardhan-reddy/)
---
## β Support the Project
If you found this helpful:
- β Star this repo
- π Share it with your peers
- β
Follow for more DevOps content
---
## π’ Stay Updated
---
## File: DevOps-Project-07/labs/7-Monitoring-and-Alerting/README.md
# Monitoring and alerting
The purpose of this lab is to review and setup sample monitoring and alerting.
In this lab, you will:
- Use and review Application Insights logs and metrics
- Setup an Application Insights Availability test
- Use and review Log Analytics along with container insights