# Technical Documentation: antonputra/tutorials > ℹ️ **Provenance:** Multi-Tier Ingestion · [CodeWiki Reference](https://codewiki.google/github.com/antonputra/tutorials) ### Overview (`README.md`) # New Video - https://youtu.be/gwtoJTevxSA [

](https://youtu.be/gwtoJTevxSA) # Consulting 👨‍💼 Mentorship/On-the-Job Support/Consulting - [**Calendar**](https://calendly.com/antonputra/youtube) or **me@antonputra.com** 👀 Check out [recommendations](https://www.linkedin.com/in/anton-putra/details/recommendations/) from my previous clients. # Playlists 📚 - [Terraform Tutorial on AWS [Full Course]](https://www.youtube.com/playlist?list=PLiMWaCMwGJXkGuVPyM2hTQumcaqnXEgHA) 📚 - [AWS EKS Kubernetes Tutorial [Full Course]](https://www.youtube.com/playlist?list=PLiMWaCMwGJXnKY6XmeifEpjIfkWRo9v2l) 📚 - [Argo CD Tutorials](https://www.youtube.com/playlist?list=PLiMWaCMwGJXkktZoHhmL6sbg7ELNjv9Xw) 📚 - [Kubernetes Tutorials](https://www.youtube.com/playlist?list=PLiMWaCMwGJXnHmccp2xlBENZ1xr4FpjXF) 📚 - [Benchmarks](https://www.youtube.com/playlist?list=PLiMWaCMwGJXmcDLvMQeORJ-j_jayKaLVn) # Contents 📚 - [Lessons](docs/contents.md) # Social 🎥 - [YouTube](https://www.youtube.com/c/AntonPutra) 💼 - [LinkedIn](https://www.linkedin.com/in/anton-putra) 🛠️ - [Twitter/X](https://x.com/antonvputra) 🙋‍♂️ - [Instagram](https://www.instagram.com/aputrabay) 📨 - me@antonputra.com --- ### Overview (`lessons/006/README.md`) # Instructions to create VPC - Create VPC ```bash aws ec2 create-vpc \ --cidr-block 10.0.0.0/16 \ --tag-specifications 'ResourceType=vpc,Tags=[{Key=Name,Value=main}]' ``` - Create public subnet ```bash aws ec2 create-subnet \ --vpc-id \ --availability-zone us-east-1a \ --cidr-block 10.0.0.0/24 \ --tag-specifications 'ResourceType=subnet,Tags=[{Key=Name,Value=public}]' ``` - Create private subnet ```bash aws ec2 create-subnet \ --vpc-id \ --availability-zone us-east-1b \ --cidr-block 10.0.1.0/24 \ --tag-specifications 'ResourceType=subnet,Tags=[{Key=Name,Value=private}]' ``` - Allocate IP for NAT Gateway ```bash aws ec2 allocate-address ``` - Create NAT Gateway ```bash aws ec2 create-nat-gateway \ --subnet-id \ --allocation-id \ --tag-specifications 'ResourceType=natgateway,Tags=[{Key=Name,Value=my-nat-gateway}]' ``` - Create internet gateway ```bash aws ec2 create-internet-gateway ``` - Cretae public route table ```bash aws ec2 create-route-table \ --vpc-id \ --tag-specifications 'ResourceType=route-table,Tags=[{Key=Name,Value=public-route}]' ``` - Attach internet gateway ```bash aws ec2 attach-internet-gateway \ --internet-gateway-id \ --vpc-id vpc-0724881ef1256d0ee ``` - Create default route to Internet ```bash aws ec2 create-route \ --route-table-id \ --destination-cidr-block 0.0.0.0/0 \ --gateway-id ``` - Cretae private route table ```bash aws ec2 create-route-table \ --vpc-id \ --tag-specifications 'ResourceType=route-table,Tags=[{Key=Name,Value=private-route}]' ``` - Create default route to NAT gateway ```bash aws ec2 create-route \ --route-table-id \ --destination-cidr-block 0.0.0.0/0 \ --nat-gateway-id ``` - Change routing table for public subnet ```bash aws ec2 associate-route-table \ --route-table-id \ --subnet-id ``` - - Change routing table for private subnet ```bash aws ec2 associate-route-table \ --route-table-id \ --subnet-id ``` - Updaye public subnet to assign public IPs ```bash aws ec2 modify-subnet-attribute \ --subnet-id \ --map-public-ip-on-launch ``` --- ### Overview (`lessons/014/README.md`) # Terraform Ansible Integration | Terraform Ansible AWS Example? [Step by Step Tutorial](https://youtu.be/QxgJlJgGA0E) --- ### Overview (`lessons/020/README.md`) # Terraform Import Existing Resources AWS ### Initialize Terraform ```bash $ terraform init ``` ### Add VPC Resource and Run Terraform Plan ```bash $ terraform plan ``` ### Import AWS VPC using vpc id ```bash $ terraform import aws_vpc.main vpc- ``` ### Add Subnets to Terraform ### Import Subnets using ids ```bash $ terraform import aws_subnet.public subnet- $ terraform import aws_subnet.private subnet- ``` ### Add IGW to Terraform ### Import IGW using id ```bash $ terraform import aws_internet_gateway.igw igw- ``` ### Add EIP to Terraform ### Import EIP using Public IP ```bash $ terraform import aws_eip.nat_eip ``` ### Add NAT Gateway to Terraform ### Import NAT Gateway using id ```bash $ terraform import aws_nat_gateway.nat nat- ``` ### Add Routes and Route Associating to Terraform ### Import to Terraform State ```bash $ terraform import aws_route_table.public rtb- $ terraform import aws_route_table.private rtb- $ terraform import aws_route_table_association.public subnet-/rtb- $ terraform import aws_route_table_association.private subnet-/rtb- ``` ### Add Security Group and EC2 to Terraform ### Import to Terraform State ```bash $ terraform import aws_security_group.nginx sg- $ terraform import aws_instance.nginx i- ``` ### Add Route53 Hosted Zone and A Record to Terraform ### Import to Terraform State ```bash $ terraform import aws_route53_zone.devops $ terraform import aws_route53_record.nginx _api.devopsbyexample.io_A ``` --- ### Overview (`lessons/021/README.md`) # Helm 3 Tutorial (Create Helm 3 Hello World Example) ### Create Kubernetes Cluster (Optional) ```bash $ eksctl create cluster -f lessons/021/eksctl-cluster.yaml $ kubectl get svc -n default ``` ### Initialize a Helm Chart Repository ```bash $ helm repo add stable https://charts.helm.sh/stable ``` ### List Available Charts ```bash $ helm search repo stable ``` ### Create Hello-world Example ```bash $ helm create hello-wolrd ``` ### Install Hello-worls Helm Chart ```bash $ kubectl create namespace dev $ helm install -f hello-wolrd/values.yaml -n dev hello-wolrd ./hello-wolrd ``` ### Learn About hello-world Release ```bash $ helm ls -n dev ``` ### Get Pods, Services ```bash $ kubectl get pods -n dev $ kubectl get services -n dev ``` ### Clean Up ```bash # Uninstall a Release $ helm uninstall -n dev hello-wolrd # remove stable repo $ helm repo remove stable ``` --- ### Overview (`lessons/022/README.md`) # Helm 3 Dependencies Condition (3 Ways to Add Dependency) ## Option one ```bash $ helm create app $ helm create database ``` ### Add Dependency to your app chart ```yaml # app/Chart.yaml dependencies: - name: database version: 0.1.0 condition: database.enabled repository: file://../database ``` ### Add YAML key/value to enable Helm chart ```yaml # app/values.yaml database: enabled: true ``` ### Build Helm dependencies and install chart ```bash $ helm dependency update $ helm install -f values.yaml web . ``` --- ## Option two ```bash $ helm create app $ helm create database ``` ### Add Dependency to your app chart ```yaml # app/Chart.yaml dependencies: - name: database version: 0.1.0 enabled: true repository: file://../database ``` ### Build Helm dependencies and install chart ```bash $ helm dependency update $ helm install -f values.yaml web . ``` --- ## Option Three ```bash $ helm create app $ helm create database ``` ### Add Dependency to your app chart ```yaml # app/Chart.yaml dependencies: - name: database version: 0.1.0 repository: file://../database tags: - database ``` ### Add YAML key/value to enable Helm chart ```yaml # app/values.yaml tags: database: true ``` ### Build Helm dependencies and install chart ```bash $ helm dependency update $ helm install -f values.yaml web . ``` --- ### Overview (`lessons/038/README.md`) # Terraform EKS Cluster Creation **YouTube** [Playlist](https://youtube.com/playlist?list=PLiMWaCMwGJXkeBzos8QuUxiYT6j8JYGE5) --- ### Overview (`lessons/041/README.md`) # Kubernetes Persistent Volumes NFS (AWS | EKS | EFS Storage Class | EFS Provisioner | ReadWriteMany) ## Create EKS Cluster ```bash $ eksctl create cluster -f cluster.yaml ``` ## Delete EKS Cluster ```bash $ eksctl delete cluster -f cluster.yaml ``` --- ### Overview (`lessons/042/README.md`) # How to Create Kubernetes Secrets? ## Create EKS Cluster ```bash $ eksctl create cluster -f cluster.yaml ``` ## Delete EKS Cluster ```bash $ eksctl delete cluster -f cluster.yaml ``` --- ### Overview (`lessons/043/README.md`) # How to Create Kubernetes Secrets? ## Create EKS Cluster ```bash $ eksctl create cluster -f cluster.yaml ``` ## Delete EKS Cluster ```bash $ eksctl delete cluster -f cluster.yaml ``` --- ### Overview (`lessons/044/README.md`) # How to Create Kubernetes Secrets? ## Create EKS Cluster ```bash $ eksctl create cluster -f eks.yaml ``` ## Delete EKS Cluster ```bash $ eksctl delete cluster -f eks.yaml ``` --- ### Overview (`lessons/045/README.md`) # How Do I Get All the Pods on a NODE? (Filter Kubernetes Pods by Node NAME | Node LABEL) ## Create EKS Cluster ```bash $ eksctl create cluster -f eks.yaml ``` ## Delete EKS Cluster ```bash $ eksctl delete cluster -f eks.yaml ``` ## Deploy All Kuberentes Objects ```bash kubectl apply -f k8s ``` --- ### Overview (`lessons/046/README.md`) # How to Deploy NodeJS App to Kubernetes? ## Create EKS Cluster ```bash $ eksctl create cluster -f eks.yaml ``` ## Delete EKS Cluster ```bash $ eksctl delete cluster -f eks.yaml ``` --- ### Overview (`lessons/047/README.md`) # How to Create Lambda Container Images? [YouTube Tutorial](https://antonputra.com/how-to-create-lambda-container-images/) --- ### Overview (`lessons/048/README.md`) # How to Test Lambda Container Images? [YouTube Tutorial](https://antonputra.com/how-to-test-lambda-container-images/) --- ### Overview (`lessons/049/README.md`) # How to Install Prometheus on Kubernetes Cluster? [YouTube Tutorial](https://antonputra.com/how-to-install-prometheus-on-kubernetes-cluster/) --- ### Overview (`lessons/050/README.md`) # How to Install MongoDB on Kubernetes? [YouTube Tutorial](https://youtu.be/Voj9kIBhzR4) --- ### Overview (`lessons/051/README.md`) # How to Deploy React App to GitHub Pages? [YouTube Tutorial](https://antonputra.com/how-to-deploy-create-react-app-to-github-pages/) --- ### Overview (`lessons/052/README.md`) # How to Configure GitHub Pages Custom Domain? [YouTube Tutorial](https://antonputra.com/how-to-configure-github-pages-custom-domain/) --- ### Overview (`lessons/053/README.md`) # How to Run Terraform from CircleCI? [YouTube Tutorial](https://antonputra.com/how-to-run-terraform-from-circleci/) --- ### Overview (`lessons/054/README.md`) # How to Generate Self-Signed Certificate? [YouTube Tutorial](https://antonputra.com/how-to-generate-self-signed-certificate/) --- ### Overview (`lessons/055/README.md`) # How to Run Nginx Docker Container? [Step by Step Tutorial](https://antonputra.com/how-to-run-nginx-docker-container-with-ssl-certificate/) --- ### Overview (`lessons/056/README.md`) # AWS Session Manager Step by Step Tutorial: SSH | Port-Forward | Audit | Logs [Step by Step Tutorial](https://antonputra.com/aws-session-manager-tutorial/) --- ### Overview (`lessons/057/README.md`) # How to Get Letsencrypt WILDCARD Certificate? [Step by Step Tutorial](https://antonputra.com/get-letsencrypt-wildcard-certificate/) --- ### Overview (`lessons/058/README.md`) # Introduction to Amazon S3 Object Lambda [Step by Step Tutorial](https://antonputra.com/introduction-to-amazon-s3-object-lambda/) --- ### Overview (`lessons/060/README.md`) # Flux v2 Kubernetes Tutorial: Building CI/CD Pipeline for Kubernetes --- ### Overview (`lessons/061/README.md`) # Kubernetes Continuous Delivery with Flux v2: AWS | EKS | ECR | CD Pipeline | Fluxcd | GitOps [YouTube Tutorial](https://youtu.be/X9R5ySkiUkc) --- ### Overview (`lessons/062/README.md`) # HashiCorp Packer AWS Tutorial: AMI Example | Builder | Provisioner | EC2 | Tags | SSH | Debug [Step by Step Tutorial](https://youtu.be/R5u5IN7RAKg) --- ### Overview (`lessons/063/README.md`) # Terraform Blue Green Deployment & Terraform Canary Deployment 🔵🐣🟢 AWS | ALB | EC2 [Step by Step Tutorial](https://youtu.be/6Pw6GabncRo) --- ### Overview (`lessons/064/README.md`) # How to Create GKE Cluster from SCRATCH? 🤫 (Shared VPC | Host and Service Projects | Best Practices) [Step by Step Tutorial](https://youtu.be/TxQRtXwRxHw) --- ### Overview (`lessons/065/README.md`) # IAM Roles for Service Accounts & Pods: (IRSA EKS | IAM OIDC Provider | AWS EKS | Kubernetes) [Step by Step Tutorial](https://youtu.be/bu0M2y2g1m8) --- ### Overview (`lessons/066/README.md`) # Kubernetes Network Policy Examples & Tutorial (Calico Isolate Namespaces, Pods & IP Blocks) [YouTube Tutorial](https://youtu.be/bVeM33B2f1A) ## Create EKS cluster ```bash eksctl create cluster -f eks.yaml ``` ## Installing Calico on Amazon EKS Apply the Calico manifests ```bash kubectl apply -f https://raw.githubusercontent.com/aws/amazon-vpc-cni-k8s/master/config/master/calico-operator.yaml kubectl apply -f https://raw.githubusercontent.com/aws/amazon-vpc-cni-k8s/master/config/master/calico-crs.yaml ``` Watch the calico-system DaemonSets ```bash kubectl get daemonset calico-node -n calico-system ``` ## Create Services ## Check Connection ```bash kubectl exec service-a- -n staging -- nc -vz service-b 8080 kubectl get pods --show-labels kubectl exec service-a- -n staging -- nc -vz service-c.production 8080 ``` --- ### Overview (`lessons/067/README.md`) # How To Install and Use PyTorch [YouTube Tutorial](https://youtu.be/zInhP5xg96o) ## Create a virtual environment (Optionally) ```bash $ python3 -m venv .venv ``` ## Activate virtual environment ```bash $ source .venv/bin/activate ``` ## Install PyTorch pip package (macOS) ```bash $ pip install torch torchvision ``` ## On Linux and Windows, use the following commands for a CPU-only build ```bash $ pip install torch==1.7.1+cpu torchvision==0.8.2+cpu -f https://download.pytorch.org/whl/torch_stable.html ``` ## Create a JSON file to convert neural network output to a human-readable class name ## Create Python script to classify an image ## Run classifier ``` # python classify.py dog.jpg ``` --- ### Overview (`lessons/068/README.md`) # Git Submodules Explained: Tutorial | Example | Guide | GitHub | Add | Update [YouTube Tutorial](https://youtu.be/wTGIDDg0tK8) ## Create Two Repos ```bash # create module-a with README # create module-b with README git clone git@github.com:antonputra/module-a.git cd module-a git submodule add git@github.com:antonputra/module-b.git git status git diff --cached --submodule git commit -am 'Add module-b' git push origin main ``` ## Cloning a Project with Submodules ```bash cd .. rm -rf module-a git clone git@github.com:antonputra/module-a.git cd module-a code . git submodule update --init --recursive cd .. rm -rf module-a Optionally: git clone --recurse-submodules git@github.com:antonputra/module-a.git cd module-a code . ``` ## Pulling in Upstream Changes from the Submodule Remote ```bash # Add "update v2" to README.md in module-b and commit "Update README.md v2" cd module-b git fetch git merge origin/main cd .. git diff git diff --submodule git config --global diff.submodule log git diff # Optionally: easier way # Make another change in module-b "Update README.md v3" git submodule update --remote git status git config status.submodulesummary 1 git status git commit -am 'Update Submodule' git push origin main ``` ## Working on a Submodule ```bash code . cd module-b git checkout main git pull # add main.py print('Hello') git add main.py git commit -m 'Add main.py' cd .. git submodule update --remote --merge # (make change in module-b "Update README.md v4") git submodule update --remote --rebase git commit -am 'Update Submodule' ``` ## Publishing Submodule Changes ```bash git diff git push --recurse-submodules=check git config push.recurseSubmodules check git push git push --recurse-submodules=on-demand ``` --- ### Overview (`lessons/069/README.md`) # How to Create GKE Cluster Using TERRAFORM from Scratch? [YouTube Tutorial](https://youtu.be/XTcos7s0iDo) ## Set Up Default Credentials ```bash gcloud auth application-default login ``` --- ### Overview (`lessons/070/README.md`) # EKS Cluster Auto Scaling (Kubernetes Autoscaler | EKS Cluster Autoscaler | EKS Autoscale Nodes) [YouTube Tutorial](https://youtu.be/gwmdboC-BtE) --- ### Overview (`lessons/071/README.md`) # Kubernetes Horizontal Pod Autoscaler [YouTube Tutorial](https://youtu.be/pI_pMsuazqw) --- ### Overview (`lessons/072/README.md`) # How to Install Prometheus on Kubernetes Cluster? [YouTube Tutorial](https://youtu.be/bwUECsVDbMA) ## Steps ### 0. Create EKS Cluster (Optional) ```bash eksctl create cluster -f eks.yaml ``` ### 1. Deploy Prometheus Stack Helm Chart - Download `prometheus-values.yaml` file for **Prometheus** from [here](https://github.com/prometheus-community/helm-charts/tree/main/charts/kube-prometheus-stack) - Update following variables ```yaml etcd: false # line 37 kubeScheduler: false # line 53 adminPassword: test123 # line 628 kubeControllerManager: # line 910 enabled: false kubeEtcd: # line 1057 enabled: false kubeScheduler: # line 1122 enabled: false serviceMonitorSelector: # line 2050 matchLabels: prometheus: devops commonLabels: # line 27 prometheus: devops ``` ```bash helm repo add prometheus-community \ https://prometheus-community.github.io/helm-charts ``` ```bash helm repo update ``` ```bash helm search repo kube-prometheus-stack --max-col-width 23 ``` ```bash helm install monitoring \ prometheus-community/kube-prometheus-stack \ --values prometheus-values.yaml \ --version 16.10.0 \ --namespace monitoring \ --create-namespace ``` ```bash kubectl get pods -n monitoring ``` ```bash kubectl get svc -n monitoring ``` ```bash kubectl port-forward \ svc/monitoring-kube-prometheus-prometheus 9090 \ -n monitoring ``` - Go to `http://localhost:9090` and select `targets` ```bash kubectl get cm kube-proxy-config -n kube-system -o yaml ``` ```bash kubectl -n kube-system get cm kube-proxy-config -o yaml |sed 's/metricsBindAddress: 127.0.0.1:10249/metricsBindAddress: 0.0.0.0:10249/' | kubectl apply -f - ``` ```bash watch -n 1 -t kubectl get pods -n kube-system ``` ```bash kubectl -n kube-system patch ds kube-proxy -p "{\"spec\":{\"template\":{\"metadata\":{\"labels\":{\"updateTime\":\"`date +'%s'`\"}}}}}" ``` - Go back to `http://localhost:9090` under `targets` and `alerts` - Wait 20-30 min to get more data in Prometheus ```bash kubectl port-forward \ svc/monitoring-grafana 3000:80 \ -n monitoring ``` - Open following dashboards - Kubernetes / Compute Resources / Cluster - Kubernetes / Kubelet - USE Method / Cluster ### 2. Deploy Postgres Helm Chart ```bash helm repo add bitnami https://charts.bitnami.com/bitnami ``` ```bash helm repo update ``` ```bash helm search repo postgresql --max-col-width 23 ``` - Download `postgres-values.yaml` file for **Postgres** from [here](https://github.com/bitnami/charts/tree/master/bitnami/postgresql) - Update following variables ```yaml postgresqlDatabase: test # line 155 metrics: # line 734 enabled: true serviceMonitor: # line 744 enabled: false additionalLabels: prometheus: devops ``` ```bash helm install postgres \ bitnami/postgresql \ --values postgres-values.yaml \ --version 10.5.0 \ --namespace db \ --create-namespace ``` ```bash kubectl get pods -n db ``` ### 3. Create Service Monitor for Postgres - Create `service-monitor.yaml` ```yaml --- # https://github.com/prometheus-operator/prometheus-operator apiVersion: monitoring.coreos.com/v1 kind: ServiceMonitor metadata: name: postgres-postgresql namespace: db labels: spec: endpoints: - port: interval: 60s scrapeTimeout: 30s namespaceSelector: matchNames: - selector: matchLabels: ``` ```bash kubectl get prometheus \ monitoring-kube-prometheus-prometheus \ -o yaml \ -n monitoring ``` ```bash kubectl get endpoints -n db ``` ```bash kubectl get services -n db ``` ```bash kubectl describe endpoints postgres-postgresql-metrics -n db ``` ```bash kubectl apply -f service-monitor.yaml ``` - Import Grafana dashboard `9628` ## Clean Up ```bash helm repo remove prometheus-community bitnami helm uninstall monitoring -n monitoring helm uninstall postgres -n db kubectl delete crd alertmanagerconfigs.monitoring.coreos.com kubectl delete crd alertmanagers.monitoring.coreos.com kubectl delete crd podmonitors.monitoring.coreos.com kubectl delete crd probes.monitoring.coreos.com kubectl delete crd prometheuses.monitoring.coreos.com kubectl delete crd prometheusrules.monitoring.coreos.com kubectl delete crd servicemonitors.monitoring.coreos.com kubectl delete crd thanosrulers.monitoring.coreos.com ``` ```bash eksctl delete cluster -f eks.yaml ``` --- ### Overview (`lessons/073/README.md`) # Horizontal Pod Autoscaler CUSTOM METRICS & PROMETHEUS [YouTube Tutorial](https://youtu.be/iodq-4srXA8) # Steps ## 1. Run Sample App Locally - Go over express app `0-express` - Open 2 tabs and run it locally ``` node 0-express/server.js ``` > Server listening at http://0.0.0.0:8081 ``` curl localhost:8081/fibonacci \ -H "Content-Type: application/json" \ -d '{"number": 10}' ``` > Fibonacci number is 89! ``` curl localhost:8081/fibonacci \ -H "Content-Type: application/json" \ -d '{"number": 20}' ``` > Fibonacci number is 10946! ``` curl localhost:8081/metrics ``` > \# HELP http_requests_total Total number of http requests > \# TYPE http_requests_total counter > \# http_requests_total{method="POST"} 2 ## 2. Create EKS Cluster Using eksctl - Open `eks.yaml` file and create EKS cluster ``` eksctl create cluster -f eks.yaml ``` > 2021-06-26 18:22:20 [ℹ] nodegroup "general" has 1 node(s) > 2021-06-26 18:22:20 [ℹ] node "ip-192-168-11-151.ec2.internal" is ready > 2021-06-26 18:24:23 [ℹ] kubectl command should work with "/Users/antonputra/.kube/config", try 'kubectl get nodes' > 2021-06-26 18:24:23 [✔] EKS cluster "my-cluster-v4" in "us-east-1" region is ready ## 3. Create Namespaces in Kubernetes - Create `demo` and `monitoring` namespaces ``` kubectl apply -f 1-namespaces ``` > namespace/demo created > namespace/monitoring created ## 4. Create Prometheus Operator CRDs - Create Prometheus CRDs and RBAC ``` kubectl apply -f 2-prometheus-operator-crd ``` > clusterrole.rbac.authorization.k8s.io/prometheus-crd-view created > clusterrole.rbac.authorization.k8s.io/prometheus-crd-edit created > customresourcedefinition.apiextensions.k8s.io/alertmanagerconfigs.monitoring.coreos.com created > customresourcedefinition.apiextensions.k8s.io/alertmanagers.monitoring.coreos.com created > customresourcedefinition.apiextensions.k8s.io/podmonitors.monitoring.coreos.com created > customresourcedefinition.apiextensions.k8s.io/probes.monitoring.coreos.com created > customresourcedefinition.apiextensions.k8s.io/prometheuses.monitoring.coreos.com created > customresourcedefinition.apiextensions.k8s.io/prometheusrules.monitoring.coreos.com created > customresourcedefinition.apiextensions.k8s.io/servicemonitors.monitoring.coreos.com created > customresourcedefinition.apiextensions.k8s.io/thanosrulers.monitoring.coreos.com created ## 5. Deploy Prometheus Operator on Kubernetes ``` kubectl apply -f 3-prometheus-operator ``` > serviceaccount/prometheus-operator created > clusterrole.rbac.authorization.k8s.io/prometheus-operator created > clusterrolebinding.rbac.authorization.k8s.io/prometheus-operator created > deployment.apps/prometheus-operator created > service/prometheus-operator created ``` kubectl get pods -n monitoring ``` > NAME READY STATUS RESTARTS AGE prometheus-operator-585f487768-745xp 1/1 Running 0 11m ``` kubectl logs -l app.kubernetes.io/name=prometheus-operator -f -n monitoring ``` > level=info ts=2021-06-27T01:44:00.696399754Z caller=operator.go:355 component=prometheusoperator msg="successfully synced all caches" > level=info ts=2021-06-27T01:44:00.702534377Z caller=operator.go:267 component=thanosoperator msg="successfully synced all caches" > level=info ts=2021-06-27T01:44:00.79632208Z caller=operator.go:287 component=alertmanageroperator msg="successfully synced all caches" ## 6. Deploy Prometheus on Kubernetes ``` kubectl apply -f 4-prometheus ``` > serviceaccount/prometheus created > clusterrole.rbac.authorization.k8s.io/prometheus created > clusterrolebinding.rbac.authorization.k8s.io/prometheus created > prometheus.monitoring.coreos.com/prometheus created ``` kubectl get pods -n monitoring ``` > NAME READY STATUS RESTARTS AGE prometheus-operator-585f487768-745xp 1/1 Running 0 11m prometheus-prometheus-0 2/2 Running 1 5m17s ``` kubectl logs -l app.kubernetes.io/instance=prometheus -f -n monitoring ``` > level=info ts=2021-06-27T01:50:04.190Z caller=main.go:995 msg="Completed loading of configuration file" filename=/etc/prometheus/config_out/prometheus.env.yaml totalDuration=507.082µs remote_storage=3.213µs web_handler=388ns query_engine=1.274µs scrape=74.372µs scrape_sd=3.853µs notify=996ns notify_sd=1.554µs rules=34.528µs ## 7. Deploy Sample Express App - Open Prometheus Target page ``` kubectl port-forward svc/prometheus-operated 9090 -n monitoring ``` > Forwarding from 127.0.0.1:9090 -> 9090 > Forwarding from [::1]:9090 -> 9090 - Go to http://localhost:9090 - Use `http` to query Prometheus (empty) - Create following files - `5-demo/0-deployment.yaml` - `5-demo/1-service.yaml` - `5-demo/2-service-monitor.yaml` - `5-demo/3-hpa-http-requests.yaml` ``` kubectl apply -f 5-demo ``` > deployment.apps/express created > service/express created > servicemonitor.monitoring.coreos.com/express created > horizontalpodautoscaler.autoscaling/http created - Go back to `http://localhost:9090` target page - Port forward express app ``` kubectl port-forward svc/express 8081 -n demo ``` - Use curl to hit fibonacci enpont twice ``` curl localhost:8081/fibonacci \ -H "Content-Type: application/json" \ -d '{"number": 10}' ``` > Fibonacci number is 89! - Use `http` to query Prometheus - Get hpa ``` kubectl get hpa -n demo ``` > /500m ``` kubectl describe hpa http -n demo ``` ``` kubectl get --raw /apis/custom.metrics.k8s.io/v1beta1 | jq ``` > Error from server (NotFound): the server could not find the requested resource ## Deploy Prometheus Adapter - Create following files - `6-prometheus-adapter/0-rbac.yaml` - `6-prometheus-adapter/1-deployment.yaml` - `6-prometheus-adapter/2-service.yaml` - `6-prometheus-adapter/3-apiservice.yaml` - `6-prometheus-adapter/4-configmap.yaml` (only 1 rule) - Run PromQL `http_requests_total{namespace!="",pod!=""}` query - Deploy Prometheus Adapter ``` kubectl apply -f 6-prometheus-adapter ``` - Open 3 tabs ``` watch -n 1 -t kubectl get hpa -n demo ``` ``` watch -n 1 -t kubectl get pods -n demo ``` ``` kubectl describe hpa http -n demo ``` > Warning FailedGetPodsMetric 26s (x13 over 3m30s) horizontal-pod-autoscaler unable to get metric http_requests_per_second: unable to fetch metrics from custom metrics API: no custom metrics API (custom.metrics.k8s.io) registered ``` kubectl get --raw /apis/custom.metrics.k8s.io/v1beta1 | jq ``` > Error from server (NotFound): the server could not find the requested resource - Deploy Prometheus adapter > configmap last one ``` kubectl apply -f 6-prometheus-adapter ``` ``` kubectl get apiservice ``` ``` kubectl get --raw /apis/custom.metrics.k8s.io/v1beta1 | jq ``` ``` helm repo add prometheus-community https://prometheus-community.github.io/helm-charts helm search repo prometheus-adapter --max-col-width 23 ``` ``` helm install custom-metrics prometheus-community/prometheus-adapter \ --namespace monitoring \ --version 2.14.2 \ --values 8-prometheus-adapter-helm/1-values.yaml ``` ## Clean Up ``` helm repo remove prometheus-community eksctl delete cluster -f eks.yaml ``` --- ### Overview (`lessons/074/README.md`) # Vertical Pod Autoscaling [YouTube Tutorial](https://youtu.be/3h-vDDTZrm8) ## 1. Create EKS Cluster Using eksctl - Create EKS cluster ```bash eksctl create cluster -f eks.yaml ``` - Verify that you can connect to the cluster ``` kubectl get svc ``` ## 2. Deploy Metrics server (YAML) - List api services ```bash kubectl get apiservice ``` - Use grep and filter by `metrics` ```bash kubectl get apiservice | grep metrics ``` - Use kubectl to get metrics ```bash kubectl top pods -n kube-system ``` - Access metrics API ```bash kubectl get --raw /apis/metrics.k8s.io/v1beta1 | jq ``` - Create deployemnt files under `0-metrics-server` directory - `0-service-account.yaml` - `1-cluster-roles.yaml` - `2-role-binding.yaml` - `3-cluster-role-bindings.yaml` - `4-service.yaml` - `5-deployment.yaml` - `6-api-service.yaml` - Apply created filies ```bash kubectl apply -f 0-metrics-server ``` - Verify deployment ```bash kubectl get pods -n kube-system ``` - List api services ```bash kubectl get apiservice ``` - List services in `kube-system` namespace ```bash kubectl get svc -n kube-system ``` - Access metrics API ```bash kubectl get --raw /apis/metrics.k8s.io/v1beta1 | jq ``` - Get metrics for pods using raw command\ ```bash kubectl get --raw /apis/metrics.k8s.io/v1beta1/pods | jq ``` - Use kubectl to get metrics ```bash kubectl top pods -n kube-system ``` ## 3. Deploy Metrics server (HELM) - Find default values for metrics-server [chart](https://github.com/bitnami/charts/tree/master/bitnami/metrics-server) - Create `values.yaml` file - Add `bitnami` helm repo ```bash helm repo add bitnami https://charts.bitnami.com/bitnami ``` - Search for `metrics-server` ```bash helm search repo metrics-server --max-col-width 23 ``` - Install `metrics-server` Helm Chart ```bash helm install metrics bitnami/metrics-server \ --namespace kube-system \ --version 5.8.13 \ --values values.yaml ``` ## 3. Install Vertical Pod Autoscaler - Open Autoscaler GitHub [page](https://github.com/kubernetes/autoscaler/tree/master/vertical-pod-autoscaler) - Clone VPA repo ```bash git clone https://github.com/kubernetes/autoscaler.git ``` - Change directory ```bash cd autoscaler/vertical-pod-autoscaler ``` - Preview installation ```bash ./hack/vpa-process-yamls.sh print ``` - Install VPA ```bash ./hack/vpa-up.sh ``` - Tear down VPA ```bash ./hack/vpa-down.sh ``` ## 4. Upgrade LibreSSL on Mac/OS X - Get OpenSSL version ```bash openssl version ``` - Upgrade LibreSSL with Homebew ```bash brew install libressl ``` - Get OpenSSL version ```bash openssl version ``` - Check instalation path of OpenSSL ```bash which openssl ``` - Check version of the LibreSSL installed with Homebew ```bash /opt/homebrew/opt/libressl/bin/openssl version ``` - Try to create a soft link ```bash sudo ln -s /opt/homebrew/opt/libressl/bin/openssl /usr/bin/openssl ``` - Try to rename openssl ```bash sudo mv /usr/bin/openssl /usr/bin/openssl-old ``` - Create soft link to /usr/local/bin/ which should take precedence on your path over /usr/bin. ```bash sudo ln -s /opt/homebrew/opt/libressl/bin/openssl /usr/local/bin/openssl ``` - Open new tab and run version command ```bash openssl version ``` ## 5. Install Vertical Pod Autoscaler (Continue) - Open new tab and change directory ```bash cd Developer/autoscaler/vertical-pod-autoscaler ``` - Install VPA ```bash ./hack/vpa-up.sh ``` ## 6. Demo - Create deployment files under `1-demo` directory - `0-deployment.yaml` - `1-vpa.yaml` - Open two tabs ```bash watch -n 1 -t kubectl top pods ``` - Deploy sample app ```bash kubectl apply -f 1-demo ``` - Let's run 5-10 min and in a new tab get VPA ```bash kubectl get vpa ``` - Describe VPA ```bash kubectl describe vpa hamster-vpa ``` - Update deployment and reapply ```bash kubectl apply -f 1-demo/0-deployment.yaml ``` ## Clean Up - Delete EKS cluster ``` eksctl delete cluster -f eks.yaml sudo rm /usr/local/bin/openssl brew remove libressl helm repo remove bitnami rm -rf Developer/autoscaler ``` --- ### Overview (`lessons/075/README.md`) # AWS SAM [YouTube Tutorial](https://youtu.be/sK9-aKUOmYE) ## 1. Install AWS SAM CLI - [Instructions](https://docs.aws.amazon.com/serverless-application-model/latest/developerguide/serverless-sam-cli-install.html) ### macOS ``` brew tap aws/tap ``` ``` brew install aws-sam-cli ``` ``` sam --version ``` ## 2. Setting up AWS credentials - Create `admin` user for AWS SAM - Download Credentials - Configure AWS Cli ``` aws configure ``` ## 3. Create AWS Lambda API Gateway Node JS - Initialize SAM project ``` sam init ``` - Go over project structure - Remove generated project ``` rm -rf sam ``` - Created `sam` directory - Create `api` directory - Change directory to `api` ``` cd sam/api ``` - Run `npm init` ``` npm init ``` - Create a `function.js` file - Create `template.yaml` file ```yaml AWSTemplateFormatVersion: '2010-09-09' Transform: AWS::Serverless-2016-10-31 Description: AWS SAM Tutorial Globals: Function: MemorySize: 128 Resources: APIFunction: Type: AWS::Serverless::Function Properties: Runtime: nodejs14.x Handler: function.lambdaHandler CodeUri: api/ Timeout: 3 Events: Hello: Type: Api Properties: Path: /hello Method: POST RestApiId: Ref: HelloAPI HelloAPI: Type: AWS::Serverless::Api Properties: StageName: staging OpenApiVersion: 3.0.3 Outputs: APIFunction: Description: "Api Lambda Function ARN" Value: !GetAtt APIFunction.Arn Tutorials: Description: "YouTube Channel" Value: https://www.youtube.com/antonputra ``` ## 4. Test Lambda Locally SAM - Change directory to `sam` - Run `sam build` - Create `api/event.json` - `sam local invoke APIFunction -e api/event.json` - Run `sam deploy --guided` - Go to S3 - Get URL from console ``` curl -d '{"name": "Anton"}' https://jh7n04hpaj.execute-api.us-east-1.amazonaws.com/staging/hello ``` ## 5. Test Lambda from Console - Create new `TestName` event ```json { "body": "{\"name\": \"Anton\"}" } ``` ## Create AWS Lambda S3 File Upload NodeJS - Create `s3` folder - Run `npm init` from `s3` folder - Create `function.js` file - Install aws-sdk `npm i aws-sdk` - Add S3Function, ExampleBucket and Outputs - Run `sam build` - Run `sam deploy` - Go to S3 - Fo to lambda - Upload README to S3 - Go to CloudWatch ## Create AWS Lambda SNS Python Example - Create `sns` folder - Create `function.py`, `requirements.txt`, and `Dockerfile` - Add `SNSFunction` and `Outputs` - Create SNS topic `sns-topic-for-lambda` - Run `sam build` - Run `sam deploy --guided` - Create ECR repo `sns` - Publish message - Check CloudWatch logs ## Clen Up - Delete SNS topic `sns-topic-for-lambda` - Delete ECR repository `sns` - Delete all cloud watch log groups - Delete IAM User `admin` - Delete AWS CLI `sam` ``` brew remove aws-sam-cli ``` - Delete all S3 buckets - Delete all CloudFormation Templates --- ### Overview (`lessons/076/README.md`) # How to Build Slack Bot? (Node JS | AWS Lambda & DynamoDB - AWS Serverless | New Slack Apps) [YouTube Tutorial](https://youtu.be/rUIptoPXu_8) ## 1. Create Slack Bot App - Go to https://api.slack.com/apps - Select create an app `From scratch` - Give it a name `WALL-E` and select your namespace - Scroll down to `App Credentials` a `Display Information` - Give it a short description `Compactor robot` - Upload app icon and change color - Go to `Event Subscriptions` ## 2. Create IAM User with Full Access - Create `admin` user and place it in `Admin` group - Download credentials - Configure AWS profile with `aws configure` - Test profile with `aws sts get-caller-identity` ## 3. Create IAM Role for AWS Lambda - Create IAM Policy `AWSLambdaSlackAccess` ```json { "Version": "2012-10-17", "Statement": [ { "Effect": "Allow", "Action": "logs:CreateLogGroup", "Resource": "arn:aws:logs:us-east-1:424432388155:*" }, { "Effect": "Allow", "Action": [ "logs:CreateLogStream", "logs:PutLogEvents" ], "Resource": "arn:aws:logs:us-east-1:424432388155:log-group:/aws/lambda/wall-e:*" }, { "Effect": "Allow", "Action": "dynamodb:*", "Resource": "arn:aws:dynamodb:us-east-1:424432388155:table/todos" } ] } ``` - Create `wall-e-role` for Lambda and attach `AWSLambdaSlackAccess` IAM Policy - Optionally: `aws iam get-role --role-name wall-e-role` ## 4. Create AWS Lambda Function - Create `wall-e` directory - Run `npm init` from `wall-e` directory - Create `app.js` ```javascript exports.handler = (event, context, callback) => { const response = { 'message': `Hello World!` }; callback(null, response); }; ``` - Create `Dockerfile` ``` FROM public.ecr.aws/lambda/nodejs:14 COPY *.js package*.json /var/task/ RUN npm install CMD [ "app.handler" ] ``` - Create ECR Repository `wall-e` ```bash aws ecr get-login-password --region us-east-1 \ | docker login \ --username AWS \ --password-stdin 424432388155.dkr.ecr.us-east-1.amazonaws.com ``` ``` docker build \ -t 424432388155.dkr.ecr.us-east-1.amazonaws.com/wall-e:v0.1.0 . ``` ``` docker push \ 424432388155.dkr.ecr.us-east-1.amazonaws.com/wall-e:v0.1.0 ``` ## 5. Deploy Lambda Using Container Image - Select `Containerimage` from the console - Call function `wall-e` - Use an existing role `wall-e-role` - Select `wall-e` ECR image ## 6. Create API Gateway for Lambda Function - Create HTTP API `slack` - Test Lambda with curl ``` curl -X POST https://uss0o5l3h3.execute-api.us-east-1.amazonaws.com/production/wall-e ``` - Check logs - Use this URL for Slack ## 7. Use AWS Lambda for Slack Event Subscriptions - Create `tests/event.json` file ```json { "version": "2.0", "routeKey": "ANY /bot", "rawPath": "/default/bot", "rawQueryString": "", "headers": { "accept": "*/*", "accept-encoding": "gzip,deflate", "content-length": "129", "content-type": "application/json", "host": "4o68t2fwke.execute-api.us-east-1.amazonaws.com", "user-agent": "Slackbot 1.0 (+https://api.slack.com/robots)", "x-amzn-trace-id": "Root=1-60f9f121-0e6b301236f5d57d46fbd0e1", "x-forwarded-for": "3.94.92.68", "x-forwarded-port": "443", "x-forwarded-proto": "https", "x-slack-request-timestamp": "1626992929", "x-slack-signature": "v0=d12f7cb55add77074248241c2ec2d3c9fe4611e7879a965c92315edd8f0ec0cf" }, "requestContext": { "accountId": "424432388155", "apiId": "4o68t2fwke", "domainName": "4o68t2fwke.execute-api.us-east-1.amazonaws.com", "domainPrefix": "4o68t2fwke", "http": { "method": "POST", "path": "/default/bot", "protocol": "HTTP/1.1", "sourceIp": "3.94.92.68", "userAgent": "Slackbot 1.0 (+https://api.slack.com/robots)" }, "requestId": "C5KdVjAlIAMEPzg=", "routeKey": "ANY /bot", "stage": "default", "time": "22/Jul/2021:22:28:49 +0000", "timeEpoch": 1626992929961 }, "body": "{\"token\":\"UdG3UFNsPGoobvRzK5F2oIqe\",\"challenge\":\"6KaNtlamllYYaLZ7qhHxZbzyYut62TlDKu2wAZXp4rZlInRbcDTH\",\"type\":\"url_verification\"}", "isBase64Encoded": false } ``` - Update body of the function ```javascript exports.handler = (event, context, callback) => { const body = JSON.parse(event.body); switch (body.type) { case "url_verification": callback(null, body.challenge); break; default: callback(null); } }; ``` - Package and upload docker image ``` docker build \ -t 424432388155.dkr.ecr.us-east-1.amazonaws.com/wall-e:v0.1.1 . ``` ``` docker push \ 424432388155.dkr.ecr.us-east-1.amazonaws.com/wall-e:v0.1.1 ``` - Redeploy AWS Lambda using v0.1.1 image tag - Go back to Slack and click `Retry` ## 8. Verify Slack Requests Using Signing Secret - Optionally: Create verify.js - Run with `node verify.js` - Compare string with event.json - Create `security.js` ```javascript const crypto = require("crypto"); exports.validateSlackRequest = (event, signingSecret) => { const requestBody = event["body"]; const headers = makeLower(event.headers); const timestamp = headers["x-slack-request-timestamp"]; const slackSignature = headers["x-slack-signature"]; const baseString = 'v0:' + timestamp + ':' + requestBody; const hmac = crypto.createHmac("sha256", signingSecret) .update(baseString) .digest("hex"); const computedSlackSignature = "v0=" + hmac; const isValid = computedSlackSignature === slackSignature; return isValid; }; const makeLower = (headers) => { let lowerCaseHeaders = {} for (const key in headers) { if (headers.hasOwnProperty(key)) { lowerCaseHeaders[key.toLowerCase()] = headers[key].toLowerCase() } } return lowerCaseHeaders } ``` - Update `app.js` ```javascript const security = require('./security'); const signingSecret = process.env.SLACK_SIGNING_SECRET; exports.handler = (event, context, callback) => { if (security.validateSlackRequest(event, signingSecret)) { const body = JSON.parse(event.body); switch (body.type) { case "url_verification": callback(null, body.challenge); break; default: callback(null); } } else callback("verification failed"); }; ``` - Add Environment variable to Lambda `SLACK_SIGNING_SECRET` (next video aws lambda secrets manager integration) - Create unit test `security.test.js` - Install Jest `npm i --save-dev jest` - Update test command to `jest` - Run tests `npm test` ## 9. Process Slack Messages - Create private Slack channel `earth` - Subscribe to following events - message.groups - message.channels - Install app to workspace - Optionally: set `Always Show My Bot as Online` from `App Home` - Go to `OAuth & Permissions` to check automatically added permissions - Add `event_callback` event ```javascript case "event_callback": processRequest(body, callback); break; ``` - Create `processRequest` method ```javascript const processRequest = (body, callback) => { switch (body.event.type) { case "message": processMessages(body, callback); break; default: callback(null); } }; ``` - Create `processMessages` method in `app.js` ```javascript const processMessages = (body, callback) => { console.debug("message:", body.event.text); callback(null); }; ``` - Update `Dockerfile` to use `npm ci --production` - Package and upload docker image ``` docker build \ -t 424432388155.dkr.ecr.us-east-1.amazonaws.com/wall-e:v0.1.2 . ``` ``` docker push \ 424432388155.dkr.ecr.us-east-1.amazonaws.com/wall-e:v0.1.2 ``` - Update AWS Lambda to use `v0.1.2` tag - Optionally: Delete all log streams - Inviite `WALL-E` to `earth` channel - Post `hello` message - Open CloudWatch logs ## 10. Save Messages Using AWS Lambda to DynamoDB - Create `todos` DynamoDB table with `uuid` primary key - Replace `message.groups` and `message.channels` with `app_mention` event - Add `chat:write` instead of `message.*` - Reinstall app - Install `aws-sdk`, `uuid`, and `axios` ```bash npm i aws-sdk uuid axios ``` - Create `db.js` ```javascript const AWS = require('aws-sdk'); const { v4: uuidv4 } = require('uuid'); AWS.config.update({ region: 'us-east-1' }); const ddb = new AWS.DynamoDB({ apiVersion: '2012-08-10' }); exports.saveItem = (item, callback) => { const params = { TableName: 'todos', Item: { 'uuid': { S: uuidv4() }, 'item': { S: item } } }; ddb.putItem(params, (error, data) => { if (error) { callback(new Error(error)); } else { callback(null); } }); }; ``` - Add `app_mention` case ```javascript case "app_mention": processAppMention(body, callback); break; ``` - Import `axios` and `db` ```javascript const axios = require('axios'); const db = require('./db'); ``` - Create `token` variable and add it to Lambda ```javascript const token = process.env.SLACK_BOT_TOKEN; ``` - Create `processAppMention` method ```javascript const processAppMention = (body, callback) => { const item = body.event.text.split(":").pop().trim(); db.saveItem(item, (error, result) => { if (error !== null) { callback(error) } else { const message = { channel: body.event.channel, text: `Item: \`${item}\` is saved to *Amazon DynamoDB*!` }; axios({ method: 'post', url: 'https://slack.com/api/chat.postMessage', headers: { 'Content-Type': 'application/json; charset=utf-8', 'Authorization': `Bearer ${token}` }, data: message }) .then((response) => { callback(null); }) .catch((error) => { callback("failed to process app_mention"); }); } }); }; ``` - Package and upload docker image ``` docker build \ -t 424432388155.dkr.ecr.us-east-1.amazonaws.com/wall-e:v0.1.3 . ``` ``` docker push \ 424432388155.dkr.ecr.us-east-1.amazonaws.com/wall-e:v0.1.3 ``` - Redeploy Lambda using `v0.1.3` tag - Post `@WALL-E todo: Save the Planet!` message - Open DynamoDB `todos` table ## Clean Up - Delete ECR `wall-e` - Delete IAM User `admin` - Delete IAM Role `wall-e-role` - Delete IAM Policy `AWSLambdaSlackAccess` - Delete DynamoDB table `todos` - Delete Lambda `wall-e` - Delete CloudWatch log groups `/aws/lambda/wall-e` - Delete API Gateway `slack` - Delete docker images `docker rmi -f $(docker images -a -q)` - Delete slack bot `wall-e` - Delete `earth` Slack channel ## Links - [Basic app setup](https://api.slack.com/authentication/basics) - [Signing secrets](https://api.slack.com/authentication/verifying-requests-from-slack) - [Features - Choosing between HTTP APIs and REST APIs](https://docs.aws.amazon.com/apigateway/latest/developerguide/http-api-vs-rest.html) - [HTTP API Price](https://aws.amazon.com/about-aws/whats-new/2019/12/amazon-api-gateway-offers-faster-cheaper-simpler-apis-using-http-apis-preview/) - [Validating Library for Node JS](https://nodejs.org/api/crypto.html#crypto_crypto_createhmac_algorithm_key_options) - [Verify Requst from Slack](https://api.slack.com/authentication/verifying-requests-from-slack) - [chat.postMessage](https://api.slack.com/methods/chat.postMessage) --- ### Overview (`lessons/077/README.md`) # AWS Lambda Secrets Manager Example: 2 Ways to Grant Access | Resource Permissions [YouTube Tutorial](https://youtu.be/_VI2JkSo3DY) ## 1. Create Secret in AWS Secrets Manager - Create `SLACK_BOT_TOKEN` secret with random value - Give it a name `prod/slack-bot/token-v3` ## 2. Create IAM User with Full Access - Create `admin` user and place it in `Admin` IAM group - Configure aws cli `aws configure` ## 3. Create IAM Role for AWS Lambda - Create IAM Policy `AWSLambdaSecretsAccess` ```json { "Version": "2012-10-17", "Statement": [ { "Effect": "Allow", "Action": "logs:CreateLogGroup", "Resource": "arn:aws:logs:us-east-1:424432388155:*" }, { "Effect": "Allow", "Action": [ "logs:CreateLogStream", "logs:PutLogEvents" ], "Resource": "arn:aws:logs:us-east-1:424432388155:log-group:/aws/lambda/secret-access:*" } ] } ``` - Create `secret-access-role` IAM Role ## 4. Create AWS Lambda Function - Create `secret-access` folder - Run `npm init` - Install aws-sdk `npm i aws-sdk` - Create `app.js` - Create `Dockerfile` ## 5. Deploy Lambda Using Container Image - Create ECR `secret-access` - Build and push image ``` aws ecr get-login-password --region us-east-1 | \ docker login --username AWS \ --password-stdin 424432388155.dkr.ecr.us-east-1.amazonaws.com ``` ```bash docker build -t 424432388155.dkr.ecr.us-east-1.amazonaws.com/secret-access:v0.1.0 . ``` ```bash docker push 424432388155.dkr.ecr.us-east-1.amazonaws.com/secret-access:v0.1.0 ``` - Deploy lambda - Test with curl (fail) ## 6. Grant Access for IAM Role ```json { "Effect": "Allow", "Action": "secretsmanager:GetSecretValue", "Resource": "arn:aws:secretsmanager:us-east-1:424432388155:secret:prod/slack-bot/token-v3-" } ``` - Remove access ## 7. Create Resource-based Policy for Secret ```json { "Version": "2012-10-17", "Statement": [ { "Effect": "Allow", "Principal": { "AWS": "arn:aws:sts::424432388155:assumed-role/secret-access-role/secret-access" }, "Action": "secretsmanager:GetSecretValue", "Resource": "arn:aws:secretsmanager:us-east-1:424432388155:secret:prod/slack-bot/token-v3-" } ] } ``` ## Clean UP - Delete `admin` IAM user - Delete ECR `secret-access` - Delete `secret-access-role` IAM Role - Delete `AWSLambdaSecretsAccess` IAM Policy - Delete `secret-access` lambda - Delete `secret-access-API` API gateway - Delete CloudWatch logs - Delete secrets ## Links [Resource-based policies](https://docs.aws.amazon.com/secretsmanager/latest/userguide/auth-and-access_resource-policies.html) [AWS JSON policy elements: Principal](https://docs.aws.amazon.com/IAM/latest/UserGuide/reference_policies_elements_principal.html) --- ### Overview (`lessons/078/README.md`) # How to Secure Nginx with Lets Encrypt on Ubuntu 20.04 with Certbot? [YouTube Tutorial](https://youtu.be/R5d-hN9UtpU) ## 1. Create EC2 Instance in AWS Ubuntu 20.04 LTS - Create EC2 instance - `Ubuntu 20.04` - `t3.micro` (**cpu cores > 1**) - `public subnet` - `enable public ip` - Create Security Group `nginx` - open port `80`, and `443` - Create `devops` kep pair - Update permissions on `devops` key pair - Keys need to be only readable by you `chmod 400 devops.pem` ## 2. Install Nginx Ubuntu 20.04 LTS - SSH to the Ubuntu server ```bash ssh -i devops.pem ubuntu@34.229.113.7 ``` - Update Ubuntu packages ```bash sudo apt update ``` - Check version of `nginx` to be installed ```bash apt policy nginx ``` - Check current versions of `nginx` [here](http://nginx.org/en/download.html) - Add `nginx` deb repository ```bash sudo vi /etc/apt/sources.list.d/nginx.list ``` ``` deb https://nginx.org/packages/ubuntu/ focal nginx deb-src https://nginx.org/packages/ubuntu/ focal nginx ``` > deb lines are relative to binary packages, that you can install with apt. deb-src lines are relative to source packages (as downloaded by apt-get source $package) and next compiled. Source packages are needed only if you want to compile some package yourself, or inspect the source code for a bug. Ordinary users don't need to include such repositories. - Update Ubuntu packages ```bash sudo apt update ``` - Add GPG key ```bash sudo apt-key adv --keyserver keyserver.ubuntu.com --recv-keys ABF5BD827BD9BF62 ``` - Update Ubuntu packages ```bash sudo apt update ``` - Check version of `nginx` to be installed ```bash apt policy nginx ``` - Install `nginx` ```bash sudo apt install nginx=1.20.1-1~focal ``` - Start `nginx` ```bash sudo systemctl start nginx ``` - Enable `nginx` ```bash sudo systemctl enable nginx ``` - Check `nginx` status ```bash sudo systemctl status nginx ``` > (Can't open PID file /run/nginx.pid (yet?) after start: Operation not permitted) - Go to browser ## 3. Nginx Setup Server Block - Check the main `nginx` config ```bash cat /etc/nginx/nginx.conf ``` - Check default `nginx` config ```bash cat /etc/nginx/conf.d/default.conf ``` - Create folder for our website ```bash sudo mkdir -p /var/www/devopsbyexample.io/html ``` - Update ownership ```bash sudo chown -R $USER:$USER /var/www/devopsbyexample.io/html ``` - Update permissions ```bash sudo chmod -R 755 /var/www/devopsbyexample.io ``` - Create a web page - `vi /var/www/devopsbyexample.io/html/index.html` ```html Welcome to devopsbyexample.io!

Success! The devopsbyexample.io server block is working!

``` - Create `sites-available` directory ```bash sudo mkdir /etc/nginx/sites-available/ ``` - Create `sites-enabled` directory ```bash sudo mkdir /etc/nginx/sites-enabled ``` - Create `nginx` server block ```bash sudo vi /etc/nginx/sites-available/devopsbyexample.io ``` ```conf server { listen 80; root /var/www/devopsbyexample.io/html; index index.html; server_name devopsbyexample.io www.devopsbyexample.io; location / { try_files $uri $uri/ =404; } } ``` - Add include statement ```bash sudo vi /etc/nginx/nginx.conf ``` ``` include /etc/nginx/sites-enabled/*; ``` - Create a symlink ```bash sudo ln -s /etc/nginx/sites-available/devopsbyexample.io /etc/nginx/sites-enabled/ ``` - Test `nginx` config ```bash sudo nginx -t ``` - Reload `nginx` config ```bash sudo nginx -s reload ``` - Create A records - Check DNS (if you are using cloudflare enable full strict by ssl/tsl>overview>full_strict) ``` dig devopsbyexample.io dig www.devopsbyexample.io ``` ## 4. Install Certbot on Ubuntu 20.04 LTS - Go to official certbot [page](https://certbot.eff.org/lets-encrypt/ubuntufocal-nginx.html) - Go to install snap [page](https://snapcraft.io/docs/installing-snap-on-ubuntu) - Check snap version ```bash snap version ``` - If you don't have it `apt policy snapd` and `apt install snapd` - Ensure that your version of snapd is up to date ```bash sudo snap install core; sudo snap refresh core ``` - Remove certbot-auto and any Certbot OS packages ```bash sudo apt-get remove certbot ``` - Install Certbot ```bash sudo snap install --classic certbot ``` - Prepare the Certbot command ```bash sudo ln -s /snap/bin/certbot /usr/bin/certbot ``` - Check certbot version ``` sudo certbot --version ``` ## 5. Secure Nginx with Lets Encrypt on Ubuntu 20.04 LTS - Test certbot ```bash sudo certbot --nginx --test-cert ``` - Open nginx block ```bash cat /etc/nginx/sites-available/devopsbyexample.io ``` - Go to browser https://devopsbyexample.io - Issue real certificate ``` sudo certbot --nginx ``` - Go to browser https://devopsbyexample.io - Go to browser https://www.devopsbyexample.io - Test renewal ```bash sudo certbot renew --dry-run ``` - Check systemctl times ```bash systemctl list-timers ``` ## Clean Up - Delete EC2 instance - Delete security group `nginx` - Delete key pair `devops` - Remove A records ## Links - [Ubuntu Releases](https://wiki.ubuntu.com/Releases) - [Nginx Download](http://nginx.org/en/download.html) - [Nginx Installation](https://www.nginx.com/resources/wiki/start/topics/tutorials/install/) - [Failed to read PID from file /run/nginx.pid](https://bugs.launchpad.net/ubuntu/+source/nginx/+bug/1581864) - [Sites-Available/Sites-Enabled Not Here?](https://www.digitalocean.com/community/questions/sites-available-sites-enabled-not-here) - [Certbot](https://certbot.eff.org/lets-encrypt/ubuntufocal-nginx.html) - [Installing snap on Ubuntu](https://snapcraft.io/docs/installing-snap-on-ubuntu) --- ### Overview (`lessons/079/README.md`) # AWS EKS & Secrets Manager (File & Env | Kubernetes | Secrets Store CSI Driver | K8s) [YouTube Tutorial](https://youtu.be/Rmgo6vCytsg) ## 1. Create IAM User with Full Access - Create `admin` user and place it in `Admin` IAM group - Configure aws cli `aws configure` ## 2. Create Secret in AWS Secrets Manager - Select `Other type of secrets` - Create key: `MY_API_TOKEN` and random value: `7623fd72g3d` - Give it a name `prod/service/token` - Open created secret to check ARN ## 3. Create EKS Cluster Using eksctl - Create `eks.yaml` config file - Create EKS cluster ```bash eksctl create cluster -f eks.yaml ``` - Check connection to EKS cluster ```bash kubectl get svc ``` ## 4. Create IAM OIDC Provider for EKS - Copy `OpenID Connect provider URL` - Create Identety Provider - select `OpenID Connect` - Enter `sts.amazonaws.com` for Audience ## 5. Create IAM Policy to Read Secrets - Create `APITokenReadAccess` IAM policy ```json { "Version": "2012-10-17", "Statement": [ { "Effect": "Allow", "Action": "secretsmanager:GetSecretValue", "Resource": "" } ] } ``` ## 6. Create IAM Role for a Kubernetes Service Account - Click `Web identity` and select Identity provider that we created - Select `APITokenReadAccess` IAM Policy - Give it a name `api-token-access` - Update trust relationships on the role - Update `aud` -> `sub` - Update `sts.amazonaws.com` -> `system:serviceaccount:production:nginx` ## 7. Associate an IAM Role with Kubernetes Service Account - Create `nginx/namespace.yaml` - Create `nginx/service-account.yaml` - Apply kubernetes objects ```bash kubectl apply -f nginx ``` - Get Kubernetes namespaces ```bash kubectl get ns ``` - Describe service account ```bash kubectl get sa -n production ``` ## 8. Install the Kubernetes Secrets Store CSI Driver - Create `secrets-store-csi-driver/0-secretproviderclasses-crd.yaml` - Create `secrets-store-csi-driver/1-secretproviderclasspodstatuses-crd.yaml` - Apply CRDs ```bash kubectl apply -f secrets-store-csi-driver ``` - Create `secrets-store-csi-driver/2-service-account.yaml` - Create `secrets-store-csi-driver/3-cluster-role.yaml` - Create `secrets-store-csi-driver/4-cluster-role-binding.yaml` - Create `secrets-store-csi-driver/5-daemonset.yaml` - Create `secrets-store-csi-driver/6-csi-driver.yaml` - Apply Kubernetes objects ```bash kubectl apply -f secrets-store-csi-driver ``` - Check the logs ```bash kubectl logs -n kube-system -f -l app=secrets-store-csi-driver ``` - (Optionally) use helm chart ```bash helm repo add secrets-store-csi-driver https://raw.githubusercontent.com/kubernetes-sigs/secrets-store-csi-driver/master/charts ``` - (Optionally) install helm chart ```bash helm -n kube-system install csi-secrets-store secrets-store-csi-driver/secrets-store-csi-driver ``` ## 9. Install AWS Secrets & Configuration Provider (ASCP) - Create `aws-provider-installer/0-service-account.yaml` - Create `aws-provider-installer/1-cluster-role.yaml` - Create `aws-provider-installer/2-cluster-role-binding.yaml` - Create `aws-provider-installer/3-daemonset.yaml` - Apply aws-provider-installer ```bash kubectl apply -f aws-provider-installer ``` - Check logs ```bash kubectl logs -n kube-system -f -l app=csi-secrets-store-provider-aws ``` ## 10. Create Secret Provider Class - Create `nginx/2-secret-provider-class.yaml` ```bash kubectl apply -f nginx ``` ## 11. Demo - Create nginx `3-deployment.yaml` - Open 2 tabs ```bash kubectl logs -n kube-system -f -l app=secrets-store-csi-driver ``` ```bash kubectl apply -f nginx ``` ```bash kubectl -n production exec -it nginx- -- bash ``` - Print mounted file ```bash cat /mnt/api-token/secret-token ``` - Print environment variables with a secret ```bash echo $API_TOKEN ``` ## Bonus [kubectx](https://github.com/ahmetb/kubectx) ## Clean Up - Delete EKS Cluster ```bash eksctl delete cluster -f eks.yaml ``` - Delete IAM Policy `APITokenReadAccess` - Delete IAM Role `api-token-access` - Delete IAM User `admin` ## Links - [secrets-store-csi-driver](https://github.com/kubernetes-sigs/secrets-store-csi-driver) - [AWS Secrets & Configuration Provider (ASCP)](https://github.com/aws/secrets-store-csi-driver-provider-aws) - [Using Secrets Manager secrets in Amazon Elastic Kubernetes Service](https://docs.aws.amazon.com/secretsmanager/latest/userguide/integrating_csi_driver.html) - [How to use AWS Secrets & Configuration Provider with your Kubernetes Secrets Store CSI driver](https://aws.amazon.com/blogs/security/how-to-use-aws-secrets-configuration-provider-with-kubernetes-secrets-store-csi-driver/) - [IAM role configuration](https://docs.aws.amazon.com/eks/latest/userguide/iam-roles-for-service-accounts-technical-overview.html) - [kubectx](https://github.com/ahmetb/kubectx) --- ### Overview (`lessons/080/README.md`) # How to Get Letsencrypt Wildcard Certificate (Using Letsencrypt Nginx DNS Challenge) [YouTube Tutorial](https://youtu.be/VJPfdXN-dSc) ## Prerequisites - Ubuntu 20.04 - Nginx - Certbot - Watch this before: How to Secure Nginx with Lets Encrypt on Ubuntu 20.04 with Certbot? - https://youtu.be/R5d-hN9UtpU ## 1. Get Letsencrypt Wildcard Certificate - Request wildcard certificate ```bash sudo certbot certonly --manual --preferred-challenges dns --test-cert ``` - Enter `*.devopsbyexample.io` - Create `TXT` record with following value: `_acme-challenge.devopsbyexample.io.` - `` - [Anycast](https://en.wikipedia.org/wiki/Anycast) - Verify with dig -t txt ```bash dig -t txt +short _acme-challenge.devopsbyexample.io ``` - Press enter > Certificate is saved at: /etc/letsencrypt/live/devopsbyexample.io/fullchain.pem > Key is saved at: /etc/letsencrypt/live/devopsbyexample.io/privkey.pem - Decode certificate ```bash sudo openssl x509 -in /etc/letsencrypt/live/devopsbyexample.io/fullchain.pem -text -noout ``` ## 2. Set Up Nginx SSL Wildcard Server Block - Create folder for website ```bash sudo mkdir -p /usr/share/devopsbyexample.io/html ``` - Update ownership ```bash sudo chown -R $USER:$USER /usr/share/devopsbyexample.io/html ``` - Update permissions ```bash sudo chmod -R 755 /usr/share/devopsbyexample.io ``` - Create `index.html` page ```bash vi /usr/share/devopsbyexample.io/html/index.html ``` ```html Welcome!

Wildcard server block is working!

``` - Create nginx server block ```bash sudo vi /etc/nginx/conf.d/devopsbyexample.io.conf ``` ```conf server { listen 80; root /usr/share/devopsbyexample.io/html; index index.html; server_name *.devopsbyexample.io; location / { try_files $uri $uri/ =404; } } ``` - Test nginx config ```bash sudo nginx -t ``` - Reload nginx config ```bash sudo nginx -s reload ``` - Create `api.devopsbyexample.io` and `hello.devopsbyexample.io` A records - Try `https://api.devopsbyexample.io/` - Verify with dig ```bash dig +short api.devopsbyexample.io ``` ```bash dig +short hello.devopsbyexample.io ``` - Check in the browser http://api.devopsbyexample.io ## 3. Secure Nginx with Lets Encrypt Certificate - Update nginx config ```bash sudo vi /etc/nginx/conf.d/devopsbyexample.io.conf ``` ```conf server { listen 80; server_name *devopsbyexample.io; return 301 https://$host$request_uri; } server { listen 443 ssl; ssl_certificate /etc/letsencrypt/live/devopsbyexample.io/fullchain.pem; ssl_certificate_key /etc/letsencrypt/live/devopsbyexample.io/privkey.pem; ssl_protocols TLSv1 TLSv1.1 TLSv1.2; ssl_ciphers HIGH:!aNULL:!MD5; ... } ``` - Test nginx config ```bash sudo nginx -t ``` - Fix `server_name` - Reload nginx config ```bash sudo nginx -s reload ``` - Go to `https://api.devopsbyexample.io/` and `https://hello.devopsbyexample.io/` ## Links - [Challenge Types](https://letsencrypt.org/docs/challenge-types/) --- ### Overview (`lessons/081/README.md`) # How to Setup Auto-Renew for Letsencrypt WILDCARD Certificate with DNS challenge? [YouTube Tutorial](https://youtu.be/7jEzioFsyNo) ## Allocate Elastic IP to EC2 Instance - Add tag `Name` = `server` ## Create Ubuntu EC2 Instance - Launch `Ubuntu 20.04` EC2 instance - Create `server` SG - Create `devops` key pair - Associate Elastic IP address with EC2 - Update `devops` key pair permissions ```bash chmod 400 devops.pem ``` - SSH to the EC2 instance ```bash ssh -i devops.pem ubuntu@ ``` ## Install acme-dns Server - Create folder for `acme-dns` and change directory ```bash sudo mkdir /opt/acme-dns cd !$ ``` - Download and extract tar with `acme-dns` from GitHub ```bash sudo curl -L -o acme-dns.tar.gz \ https://github.com/joohoi/acme-dns/releases/download/v0.8/acme-dns_0.8_linux_amd64.tar.gz sudo tar -zxf acme-dns.tar.gz ``` - List files ```bash ls ``` - Clean Up ```bash sudo rm acme-dns.tar.gz ``` - Create soft link ```bash sudo ln -s \ /opt/acme-dns/acme-dns /usr/local/bin/acme-dns ``` - Create a minimal `acme-dns` user ```bash sudo adduser \ --system \ --gecos "acme-dns Service" \ --disabled-password \ --group \ --home /var/lib/acme-dns \ acme-dns ``` - Update default acme-dns config compare with IP from the AWS console. CAn't bind to the public address need to use private one. ```bash ip addr ``` ```bash sudo mkdir -p /etc/acme-dns sudo mv /opt/acme-dns/config.cfg /etc/acme-dns/ sudo vim /etc/acme-dns/config.cfg ``` - Move the systemd service and reload ```bash cat acme-dns.service sudo mv \ acme-dns.service /etc/systemd/system/acme-dns.service sudo systemctl daemon-reload ``` - Start and enable acme-dns server ```bash sudo systemctl enable acme-dns.service sudo systemctl start acme-dns.service ``` - Check acme-dns for posible errors ```bash sudo systemctl status acme-dns.service ``` - Use journalctl to debug in case of errors ```bash journalctl --unit acme-dns --no-pager --follow ``` - Create A record for your domain ```bash auth.devopsbyexample.io IN A ``` - Create NS record for auth.devopsbyexample.io pointing to auth.devopsbyexample.io. This means, that auth.devopsbyexample.io is responsible for any *.auth.devopsbyexample.io records ```bash auth.devopsbyexample.io IN NS auth.devopsbyexample.io ``` - Test acme-dns server (Split the screen) ```bash journalctl -u acme-dns --no-pager --follow ``` - From local host try to resolve random DNS record ``` dig api.devopsbyexample.io dig api.auth.devopsbyexample.io dig 7gvhsbvf.auth.devopsbyexample.io ``` ## Install acme-dns-client ```bash sudo mkdir /opt/acme-dns-client cd !$ sudo curl -L \ -o acme-dns-client.tar.gz \ https://github.com/acme-dns/acme-dns-client/releases/download/v0.2/acme-dns-client_0.2_linux_amd64.tar.gz sudo tar -zxf acme-dns-client.tar.gz ls sudo rm acme-dns-client.tar.gz sudo ln -s \ /opt/acme-dns-client/acme-dns-client /usr/local/bin/acme-dns-client ``` ## Install Certbot ```bash cd sudo snap install core; sudo snap refresh core sudo snap install --classic certbot sudo ln -s /snap/bin/certbot /usr/bin/certbot ``` ## Get Letsencrypt Wildcard Certificate - Create a new acme-dns account for your domain and set it up ```bash sudo acme-dns-client register \ -d devopsbyexample.io -s http://localhost:8080 ``` ``` dig _acme-challenge.devopsbyexample.io ``` - Get wildcard certificate ```bash sudo certbot certonly \ --manual \ --test-cert \ --preferred-challenges dns \ --manual-auth-hook 'acme-dns-client' \ -d *.devopsbyexample.io ``` ``` sudo openssl x509 -text -noout \ -in /etc/letsencrypt/live/devopsbyexample.io/fullchain.pem ``` - Renew certificate (test) ```bash sudo certbot renew \ --manual \ --test-cert \ --dry-run \ --preferred-challenges dns \ --manual-auth-hook 'acme-dns-client' ``` ``` dig -t txt _acme-challenge.devopsbyexample.io ``` ## Setup Auto-Renew for Letsencrypt WILDCARD Certificate - Setup cronjob ```bash sudo crontab -e 0 */12 * * * certbot renew --manual --test-cert --preferred-challenges dns --manual-auth-hook 'acme-dns-client' ``` ## Clen Up - Terminate EC2 Instance - Delete `devops` key pair - Delete `server` SG - Release `server` Elastic IP - Delete DNS records from google domains ## Links - [acme-dns](https://github.com/joohoi/acme-dns) - [acme-dns-client](https://github.com/acme-dns/acme-dns-client) --- ### Overview (`lessons/082/README.md`) # NGINX Ingress Controller for Kubernetes Tutorial You can find tutorial [here](https://antonputra.com/kubernetes/nginx-ingress-controller/). --- ### Overview (`lessons/083/README.md`) # Cert Manager Kubernetes Tutorial (Let's Encrypt & Nginx Ingress & ACME | 5 Examples | YAML & HELM) [YouTube Tutorial](https://youtu.be/7m4_kZOObzw) ## Prerequisites - [Kubernetes](https://kubernetes.io/) - [Helm 3](https://helm.sh/) ## Deploy Prometheus on Kubernetes ```bash kubectl apply -f prometheus/0-crd kubectl apply -f prometheus/1-prometheus-operator kubectl apply -f prometheus/2-prometheus kubectl get pods -n monitoring ``` ## Deploy Cert Manager Helm & YAML - Review default helm [values](https://github.com/jetstack/cert-manager/blob/master/deploy/charts/cert-manager/values.yaml) - Add the Jetstack Helm repository ```bash helm repo add jetstack \ https://charts.jetstack.io ``` - Update your local Helm chart repository cache ```bash helm repo update ``` - Create `cert-manager-values.yaml` file to override default variables - Search for cert-manager ```bash helm search repo cert-manager ``` - Generate YAML files ```bash helm template cert-083 jetstack/cert-manager \ --namespace cert-manager \ --version v1.5.3 \ --values cert-manager-values.yaml \ --output-dir helm-generated-yaml ``` - Create `cert-manager` namespace ```bash kubectl apply -f cert-manager-ns.yaml ``` - Install cert-manager with Helm ```bash helm install cert-083 jetstack/cert-manager \ --namespace cert-manager \ --version v1.5.3 \ --values cert-manager-values.yaml ``` - Get Helm releases ```bash helm list -n cert-manager ``` - Get pods ```bash kubectl get pods -n cert-manager ``` - [cert-manager](https://cert-manager.io/docs/concepts/) - [cainjector](https://cert-manager.io/docs/concepts/ca-injector/) - [webhook](https://cert-manager.io/docs/concepts/webhook/) - Port forward Prometheus ```bash kubectl port-forward svc/prometheus-operated 9090 -n monitoring ``` - Go to Prometheus targets `http://localhost:9090` ## Generate Self Signed Certificate (Example 1) - Create SelfSigned ClusterIssuer `example-1/0-self-signed-issuer.yaml` - Generate Self Signed Certificate `example-1/1-ca-certificate.yaml` ```bash kubectl apply -f example-1 ``` - Get certificate ```bash kubectl get certificate -n cert-manager ``` - Get secrets ```bash kubectl get secrets -n cert-manager ``` - Get x509 certificate ```bash kubectl get secrets devopsbyexample-io-key-pair \ -o yaml \ -n cert-manager ``` - Decode base64 certificte ```bash echo "base64" | base64 -d -o ca.crt ``` - Decode CA certificate ```bash openssl x509 -in ca.crt -text -noout ``` ## Generate TLS Certificate using CA (Example 2) - Create CA ClusterIssuer `example-2/0-ca-issuer.yaml` - Create `staging` namespace `example-2/1-staging-ns.yaml` - Create certificate for blog.devopsbyexample.io `example-2/example-2/2-blog-certificate.yaml` ```bash kubectl apply -f example-2 ``` - Get certificate ```bash kubectl get certificate -n staging ``` - Get secrets ```bash kubectl get secrets -n staging ``` - Get x509 certificate ```bash kubectl get secrets blog-devopsbyexample-io-key-pair \ -o yaml \ -n staging ``` - Decode base64 certificte ```bash echo "base64" | base64 -d -o blog-devopsbyexample-io.crt ``` - Decode CA certificate ```bash openssl x509 -in blog-devopsbyexample-io.crt -text -noout ``` ## Deploy Nginx Ingress Controller - Add ingress-nginx Helm repo ```bash helm repo add ingress-nginx \ https://kubernetes.github.io/ingress-nginx ``` - Update Helm repos ```bash helm repo update ``` - Search for Nginx Helm Chart ```bash helm search repo ingress-nginx ``` - Create `nginx-ingress-values.yaml` without ACME section - Install Nginx Ingress ```bash helm install ing-083 ingress-nginx/ingress-nginx \ --namespace ingress \ --version 4.0.1 \ --values nginx-ingress-values.yaml \ --create-namespace ``` - Get pods ```bash kubectl get pods -n ingress ``` - Get Ingress Class ```bash kubectl get ingressclass ``` ## Deploy Grafana on Kubernetes - Deploy Grafana using YAML - `grafana/0-secret.yaml` - `grafana/1-datasources.yaml` - `grafana/2-dashboard-providers.yaml` - `grafana/3-cert-manager.yaml` - `grafana/4-deployment.yaml` - `grafana/5-service.yaml` ```bash kubectl apply -f grafana ``` - Get services in monitoring namespaces ```bash kubectl get svc -n monitoring ``` - Create `HTTP` ingress without TLS section - `example-3/grafana.yaml` ```bash kubectl apply -f example-3 ``` - Get ingresses ```bash kubectl get ing -n monitoring ``` - Create `CNAME` record for `grafana.devopsbyexample.io` - Go to data sources and dashboards `http://grafana.devopsbyexample.io` - Logout - Install wireshark ```bash brew install wireshark ``` - Get public ip address of Grafana ```bash dig +short grafana.devopsbyexample.io ``` - Get network interfaces ```bash ifconfig ``` - Get POST TCP packets ```bash sudo tshark -i en1 -x -f "host grafana.devopsbyexample.io and port 80 and tcp[((tcp[12:1] & 0xf0) >> 2):4] = 0x504f5354" > post.pcap ``` ```bash cat post.pcap ``` - Add TLS section to grafana ingress - `example-3/grafana.yaml` ```bash kubectl apply -f example-3 ``` - Get certificates in monitoring namespace ```bash kubectl get certificates -n monitoring ``` - Get ingresses in monitoring ns ```bash kubectl get ing -n monitoring ``` - Go to `https://grafana.devopsbyexample.io` to check TLS certs - Add CA to the keychain - Login to Grafana - Listen on port 443 ```bash sudo tshark -i en1 -x -f "port 443" ``` ## Secure NGINX ingress with Let's Encrypt & ACME (Example 4) - Create `letsencrypt-staging` Issuer in `monitoring` namespace - `example-4/letsencrypt-staging-issuer.yaml` - Create `letsencrypt-staging` Issuer in `monitoring` namespace - `example-4/letsencrypt-prod-issuer.yaml` ```bash kubectl apply -f example-4 ``` - Get issuers in `monitoring` namespace ```bash kubectl get issuers -n monitoring ``` - If it's not ready describe to get error message ```bash kubectl describe issuer letsencrypt-http01-prod -n monitoring ``` - Create Prometheus ingress with `letsencrypt-prod` issuer - `example-4/2-prometheus-ing.yaml` - Apply Prometheus ingress ```bash kubectl apply -f example-4/2-prometheus-ing.yaml ``` - Get CRDs ```bash kubectl get Certificates -n monitoring kubectl describe Certificates \ prometheus-v4-devopsbyexample-io-key-pair -n monitoring kubectl get CertificateRequests -n monitoring kubectl describe CertificateRequests \ prometheus-v4-devopsbyexample-io-key-pair-ss2h8 -n monitoring kubectl get Orders -n monitoring kubectl describe Orders \ prometheus-v4-devopsbyexample-io-key-pair-ss2h8-98152280 -n monitoring kubectl get Challenges -n monitoring kubectl describe Challenges \ prometheus-v4-devopsbyexample-io-key-pair-2zjpw-9815-4236669664 -n monitoring ``` - Make sure that all pods are up in monitoring namespace including `cm-acme-http-solver-` ```bash kubectl get pods -n monitoring ``` - To complete we need to create CNAME record for prometheus ```bash kubectl get ing -n monitoring ``` - Print out acme ing ```bash kubectl get ing cm-acme-http-solver-lxtws -o yaml -n monitoring ``` - Watch Certificates and Challenges ```bash watch -n 1 -t kubectl get certificates -n monitoring watch -n 1 -t kubectl get Challenges -n monitoring ``` - Create CNAME for `prometheus-v4.devopsbyexample.io` - Go to `https://prometheus-v4.devopsbyexample.io` ## Delegate a Subdomain to Route53 > [Creating a subdomain that uses Amazon Route 53 as the DNS service without migrating the parent domain](https://docs.aws.amazon.com/Route53/latest/DeveloperGuide/CreatingNewSubdomain.html) - Create `monitoring.devopsbyexample.io` Route53 hosted zone - Get Name Servers from `Hosted zone details` - Add NS records for the subdomain `monitoring` - Create A record to test route53 zone `test.monitoring.devopsbyexample.io` -> 10.10.10.10 - Wait up to 1 minute and test with dig ```bash dig +short test.monitoring.devopsbyexample.io ``` ## Create IAM Role for Kubernetes Service Account - Create OpenID Connect provider, use `sts.amazonaws.com` for Audience - Create IAM policy with Route53 access - `CertManagerRoute53Access` ```json { "Version": "2012-10-17", "Statement": [ { "Effect": "Allow", "Action": "route53:GetChange", "Resource": "arn:aws:route53:::change/*" }, { "Effect": "Allow", "Action": [ "route53:ChangeResourceRecordSets", "route53:ListResourceRecordSets" ], "Resource": "arn:aws:route53:::hostedzone/" } ] } ``` - Create IAM role for `cert-manager-acme` cert-manager - Get service account name for cert-manager ```bash kubectl get sa -n cert-manager ``` - Update trust for IAM role to allow only `cert-083-cert-manager` service account to assume it ``` aud -> sub sts.amazonaws.com -> system:serviceaccount:cert-manager:cert-083-cert-manager ``` - Update service account manually, add following line to annotations ```bash kubectl edit sa -n cert-manager ``` ```yaml eks.amazonaws.com/role-arn: arn:aws:iam::424432388155:role/cert-manager-acme ``` - Modify the cert-manager Deployment with the correct file system permissions, so the ServiceAccount token can be read. ```bash kubectl get deployment -n cert-manager kubectl edit deployment cert-083-cert-manager -n cert-manager ``` ```yaml - --issuer-ambient-credentials ``` - Make sure pod restarted ```bash kubectl get pods -n cert-manager ``` - This change can be inclused to Helm Chart (include to `cert-manager-values.yaml` before installing or upgrade Helm release) ```yaml serviceAccount: annotations: eks.amazonaws.com/role-arn: arn:aws:iam::424432388155:role/cert-manager-acme extraArgs: - --issuer-ambient-credentials ``` ## Resolve DNS-01 challenge with cert-manager (Example 5) - Create Issuer to use dns-01 challenge - `example-5/0-letsencrypt-staging-dns01-issuer.yaml` - `example-5/1-letsencrypt-prod-dns01-issuer.yaml` - `example-5/2-grafana-ing.yaml` - Watch Certificates and Challenges ```bash watch -n 1 -t kubectl get certificates -n monitoring watch -n 1 -t kubectl get challenges -n monitoring ``` ```bash kubectl apply -f example-5 ``` - Go to Route53 to get TXT record - Create CNAME for grafana-v4.monitoring.devopsbyexample.io - Go to `https://grafana-v4.monitoring.devopsbyexample.io` - Check logs ```bash kubectl get pods -n cert-manager kubectl logs -f -n cert-manager ``` https://cert-manager.io/docs/configuration/acme/dns01/route53/ ## Monitor Cert Manager with Prometheus and Grafana ## Clean Up - Remove Helm repo ```bash helm repo remove jetstack helm repo remove ingress-nginx ``` - Remove wireshark ```bash brew remove wireshark ``` - Remove `devopsbyexample.io` CA - Delete Route53 hosted zone `monitoring.devopsbyexample.io` - Delete all DNS records in google domains - Delete IAM Role `cert-manager-acme` - Delete IAM Policy `CertManagerRoute53Access` --- ### Overview (`lessons/084/README.md`) # AWS Point to Site VPN Setup (Free OpenVPN AWS Setup | EC2 | Route53 Private DNS | Ubuntu 20.04) [YouTube Tutorial](https://youtu.be/yaXiAqH-4LE) ## Create AWS VPC - Create AWS VPC (Virtual Private Cloud) - give it a name `main` - define IPv4 CIDR block `10.0.0.0/16` ## Create AWS Internet Gateway - Create AWS Internet Gateway - call it `igw` - Attach Internet Gateway to AWS VPC ## Create AWS Public Subnet - Create public subnet - call it `public` - define IPv4 CIDR block `10.0.0.0/22`, it will give you `1,024` IP addresses, witht the last IP - `10.0.3.255` - Create `public` route table with default route to the internet gateway - Attach `public` route table to the public subnet ## Create AWS NAT Gateway - Allocate Elastic IP address for nat - tag `Name: nat` - Create NAT gateway - call it `nat` - Place it to public subnet ## Create AWS Private Subnets - Create `private-large` subnet `10.0.16.0/20` - Create `private-small` subnet `10.0.32.0/24` - Create `private` route table with default route to nat gateway - Update route tables for private subnets ## Create Ubuntu EC2 Instance - Allocate static public IP address `openvpn` - Create Ubuntu 20.04 - tag `Name: openvpn` - Instance type: `t3.small` - SG: `OpenVPN`, add `1194` custom udp from `Anywhere` - Associate Elastic IP with EC2 ## Install OpenVPN on Ubuntu 20.04 - Update permissions on the key ```bash chmod 400 devops.pem ``` - SSH to the Ubuntu server ```bash ssh -i devops.pem ubuntu@ ``` - Update Ubuntu repositories ```bash sudo apt update ``` - Check OpenVPN candidate ```bash apt policy openvpn ``` - Compate verion with the latest release of OpenVPN on [GitHub](https://github.com/OpenVPN/openvpn) - We would need to run commands as a root, let's temporary use `sudo -s` ```bash sudo -s ``` - Then import the public GPG key that is used to sign the packages: ```bash wget -O - https://swupdate.openvpn.net/repos/repo-public.gpg|apt-key add - ``` - Add OpenVPN repo ```bash echo "deb http://build.openvpn.net/debian/openvpn/stable focal main" > /etc/apt/sources.list.d/openvpn-aptrepo.list ``` - Update repositories again with the new openvpn source list ```bash apt update ``` - Exit root ```bash exit ``` - Check version of candidate again ```bash apt policy openvpn ``` - Install the latest one ```bash sudo apt install openvpn=2.5.3-focal0 ``` ## Install easy-rsa on Ubuntu 20.04 - Check the candidate verion ```bash apt policy easy-rsa ``` - Check available verions on [GitHub](https://github.com/OpenVPN/easy-rsa) - Download `easy-esa` tarball ```bash wget https://github.com/OpenVPN/easy-rsa/releases/download/v3.0.8/EasyRSA-3.0.8.tgz ``` - Untar it ```bash tar -zxf EasyRSA-3.0.8.tgz ``` - Clean UP ```bash ls rm EasyRSA-3.0.8.tgz ``` - Move `easy-rsa` to OpenVPN ```bash sudo mv EasyRSA-3.0.8/ /etc/openvpn/easy-rsa ``` - (Optionally) create soft link ```bash sudo ln -s /etc/openvpn/easy-rsa/easyrsa /usr/local/bin/ ``` - Change directory to home and test cli ```bash easyrsa --version ``` ## Creating PKI for OpenVPN with easy-rsa - Change directory to openvpn ```bash cd /etc/openvpn/easy-rsa ``` - Initialize a PKI CA ```bash easyrsa init-pki ``` - List directories ```bash ls ls pki ``` - Create vars file ```bash vim vars ``` - Create CA (security or convenience) ```bash easyrsa build-ca nopass ``` - List files ```bash ls pki ls pki/private ``` ## Generate Certificate for OpenVPN Server - Generate signing request ```bash easyrsa gen-req openvpn-server nopass ``` - Sign cert ``` easyrsa sign-req server openvpn-server ``` ## Configure OpenVPN Cryptographic Material - Generate the tls-crypt pre-shared key ```bash openvpn --genkey secret ta.key cat ta.key ``` ## Configure OpenVPN server - Enable IP forwarding ```bash sudo vim /etc/sysctl.conf ``` - Read the file and load the new values for the current session ```bash sudo sysctl -p ``` - Configure IP Tables ```bash sudo iptables -t nat -S ``` - Find out network public network interface ```bash ip route list default ``` - Configure nat routing ```bash sudo iptables \ -t nat -I POSTROUTING -s 10.8.0.0/24 -o ens5 -j MASQUERADE ``` - Save iptables ```bash sudo apt-get install iptables-persistent ``` - Create config file, leave routes out for now ```bash sudo vim /etc/openvpn/server/server.conf ``` - Check if you have `nobody` user ```bash cat /etc/passwd | grep nobody ``` - Check if you have `nogroup` ```bash cat /etc/group | grep nogroup ``` - Check subnet masks for CIDR [here](https://docs.netgate.com/pfsense/en/latest/network/cidr.html) - Start OpenVPN ```bash sudo systemctl start openvpn-server@server ``` - Check status OpenVPN ```bash sudo systemctl status openvpn-server@server ``` - Enable openvpn-server ```bash sudo systemctl enable openvpn-server@server ``` - Check logs ```bash journalctl \ --no-pager --full -u openvpn-server@server -f ``` ## Create Client Profile .ovpn Manually - Generate key pair ```bash easyrsa gen-req example-1 nopass ``` - Sign certificate request ```bash easyrsa sign-req client example-1 ``` - Create `example-1.ovpn` profile ```bash cat /etc/openvpn/easy-rsa/pki/ca.crt cat \ /etc/openvpn/easy-rsa/pki/issued/example-1.crt cat \ /etc/openvpn/easy-rsa/pki/private/example-1.key cat /etc/openvpn/easy-rsa/ta.key ``` - Install tunnelblick ```bash brew install --cask tunnelblick ``` - Install `example-1.ovpn` profile - Run journalctl on OpenVPN server ```bash journalctl \ --no-pager --full -u openvpn-server@server -f ``` - Connect to the VPN - Check routes on mac ```bash netstat -r ``` - Create Ubuntu server in `private-small` subnet - Use openvpn security group as a source - SSH to Ubuntu using private IP address ```bash ssh -i devops.pem ubuntu@ ``` ## Create Route53 Private Hosted Zone - Create `devops.pvt` private hosted zone - Create `test.devops.pvt` A record with random IP address - Try to resolve it from development host ```bash dig test.devops.pvt ``` - Enable VPC DNS resolution - To use private hosted zones, you must set the following Amazon VPC settings to true: - enableDnsHostnames - enableDnsSupport ## Revoke OpenVPN Client Certificate - Revoke `example-1` certificate ```bash cd /etc/openvpn/easy-rsa/ easyrsa revoke example-1 ``` - Generate CRL (Certificate Revocation List) ```bash easyrsa gen-crl ``` - Add CRL to OpenVPN server config ``` sudo vim /etc/openvpn/server/server.conf crl-verify /etc/openvpn/easy-rsa/pki/crl.pem sudo systemctl restart openvpn-server@server journalctl \ --no-pager --full -u openvpn-server@server -f ``` ## Generate profiles ```bash cd /etc/openvpn/ sudo mkdir client-configs cd client-configs sudo vim base.ovpn cd /etc/openvpn/easy-rsa/ easyrsa gen-req example-2 nopass easyrsa sign-req client example-2 vim gen_client_profile.sh sudo chmod +x gen_client_profile.sh sudo ./gen_client_profile.sh example-2 cat \ /etc/openvpn/client-configs/example-2.ovpn ``` - Create `example-2.ovpn` - Install `example-2.ovpn` profile - Check DNS again ```bash dig test.devops.pvt ``` ## Install docker on Ubuntu 20.04 - Set up the repository ```bash sudo apt install \ apt-transport-https \ ca-certificates \ curl \ gnupg \ lsb-release ``` - Add Docker’s official GPG key ```bash curl -fsSL https://download.docker.com/linux/ubuntu/gpg | sudo gpg --dearmor -o /usr/share/keyrings/docker-archive-keyring.gpg ``` - Set up the stable repository ```bash echo \ "deb [arch=amd64 signed-by=/usr/share/keyrings/docker-archive-keyring.gpg] https://download.docker.com/linux/ubuntu \ $(lsb_release -cs) stable" | sudo tee /etc/apt/sources.list.d/docker.list > /dev/null ``` - Install Docker Engine ```bash sudo apt update sudo apt install \ docker-ce docker-ce-cli containerd.io ``` - Install docker compose ```bash sudo apt install docker-compose ``` - Create `docker-compose.yaml` to run MySQL 5.7 ```bash vim docker-compose.yaml ``` - Run docker compose up ```bash sudo docker-compose up -d ``` - Check running containers ```bash sudo docker ps ``` ## Configure MySQL 5.7 Server for gate-sso - Install MySQL client ```bash sudo apt install mysql-client ``` - Connect to mysql ```bash mysql -u root -p -h 127.0.0.1 -P 3306 ``` - Create user for gate-sso ```sql CREATE USER 'gate' IDENTIFIED BY 'devops123'; ``` - Grant access to `gate_development` and `gate_test` databases ```sql GRANT ALL PRIVILEGES ON gate_development.* TO 'gate'; GRANT ALL PRIVILEGES ON gate_test.* TO 'gate'; FLUSH PRIVILEGES; ``` - Log out ```bash exit ``` ## Install Ruby on Rails on Ubuntu 20.04 - Check ruby version (must be >= 2.4) ```bash ruby -v ``` - Install rvm ```bash curl -L https://get.rvm.io | bash -s stable ``` - Import GPG keys ```bash curl -sSL https://rvm.io/mpapis.asc | gpg --import - curl -sSL https://rvm.io/pkuczynski.asc | gpg --import - ``` - Run script again ```bash curl -L https://get.rvm.io | bash -s stable ``` - To start using RVM, load the script environment variables using the source command: ```bash source ~/.rvm/scripts/rvm ``` - Install ruby `2.4.3` with rvm ```bash rvm install 2.4.3 ``` - Install `bundler` gem ```bash gem install bundler ``` ## Install gate-sso - Clone gate-sso GitHub repository ```bas cd /opt sudo git clone https://github.com/gate-sso/gate.git ``` - Update ownership of the repository ```bash sudo chown -R ubuntu:ubuntu gate ``` - Install gate-sso dependencies ```bash cd gate bundle install ``` - Fix deps ```bash sudo apt-get install libmysqlclient-dev ``` - Run bundle install again ```bash bundle install ``` - Install nodejs ```bash sudo apt install nodejs ``` - Run again ```bash rake app:init ``` - Create SSO in gcp - Go to APIs & Services - Create OAuth Client ID credentials - http://gate.devopsbyexample.io/users/auth/google_oauth2/callback - Update env ```bash vim .env ``` ``` GATE_SERVER_URL=http://gate.devopsbyexample.io GATE_OAUTH_CLIENT_ID=771040318735-7anf9ct0fo5a23s6gqsnmc2vo3ifbo28.apps.googleusercontent.com GATE_OAUTH_CLIENT_SECRET=8SNXY_H7RE2-nffqDGIKGZ9i GATE_HOSTED_DOMAIN=devopsbyexample.io GATE_HOSTED_DOMAINS=antonputra.com GATE_DB_HOST=127.0.0.1 GATE_DB_PASSWORD=devops123 ``` - Create A record for gate.devopsbyexample.io - Run setup ```bash rake app:setup ``` - Open port 80 on SG - Create `sudo vim /etc/openvpn/easy-rsa/gen-client-conf` (Update remote ip) - Create `sudo vim /etc/openvpn/easy-rsa/gen-client-keys` (Update remote ip) - Start rails ```bash rvmsudo rails server \ --port 80 \ --binding 0.0.0.0 \ --daemon ``` - Go to http://gate.devopsbyexample.io and download profile ## Links - [Easy-RSA 3 Quickstart README](https://github.com/OpenVPN/easy-rsa/blob/master/README.quickstart.md) - [OpenVPN - Getting started How-To](https://community.openvpn.net/openvpn/wiki/GettingStartedwithOVPN?__cf_chl_jschl_tk__=pmd_X4JgyIGE5orR7DsfIDms7ZJjTaenJB7TAkouWI00egk-1630687272-0-gqNtZGzNAfujcnBszQgR) - [Certificate Authority (CA)](https://wiki.archlinux.org/title/Easy-RSA) Starting from OpenVPN 2.4, one can also use elliptic curves for TLS connections (e.g. tls-cipher TLS-ECDHE-ECDSA-WITH-AES-256-GCM-SHA384). Elliptic curve cryptography provides more security and eliminates the need for a Diffie-Hellman parameters file. See [2] and [3]. - [TAP vs TUN](https://community.openvpn.net/openvpn/wiki/BridgingAndRouting?__cf_chl_jschl_tk__=pmd_g7hLiK8fPD6lRitFu1jxyHO2yMP9_UrR1KrxCBKE81s-1630949427-0-gqNtZGzNAfujcnBszQnl) - [TCP/IP Tutorial and Technical Overview](http://www.redbooks.ibm.com/redbooks/pdfs/gg243376.pdf) - [AWS DNS](https://d1.awsstatic.com/whitepapers/hybrid-cloud-dns-options-for-vpc.pdf) ## Clean UP - VPC `main` - Key pair `devops` - Release public IPs - Delete tunnelblick ```bash brew remove tunnelblick ``` - Delete Route53 hosted zone `devops.pvt` - Remove GCP credentials - Remove `gate.devopsbyexample.io` A record --- ### Overview (`lessons/085/README.md`) # How to Deploy React App on Firebase Hosting? (CI/CD with GitHub Actions | Preview | Custom Domain) [YouTube Tutorial](https://youtu.be/Bnd4IO3f2hU) ## Prerequisites - Firebase Account - npm 5.2+ ## Create React App from Scratch 2021 - Check `npm` version ```bash npm --version ``` - Create a new single-page application in React ```bash npx create-react-app react-demo-085 ``` - Change directory to the app ```bash cd react-demo-085 ``` - Inspect the folder content ```bash code . ``` - Run react locally ```bash npm start ``` ## Create Firebase Project - Go to [Firebase](https://firebase.google.com/) - Click `Go to console` - Click `Create a project` - Create `react-demo-085` Firebase project ## Deploy React App on Firebase Hosting - Install the Firebase CLI ```bash npm install firebase-tools ``` - Log in and test the Firebase CLI ```bash firebase login ``` - Test that the CLI is properly installed and accessing your account by listing your Firebase projects. ```bash firebase projects:list ``` - Build for React application ```bash npm run build ``` - Initialize your project ```bash firebase init hosting ``` - Deploy to your site ```bash firebase deploy --only hosting ``` - Open hosting URL - Check TLS certificate ## Firebase Hosting Add Custom Domain - Go to Firebase console - Navigate to `Hosting` - Click `Add custom domain` - Enter your domain, in my case `devopsbyexample.io` - Create A record for your domain - Optionally add `www` subdomain ## Set up automatic builds and deploys with GitHub - Create GitHub repo `react-demo-085`, let's make it private - Add existing repository ```bash git remote add origin git@github.com:antonputra/react-demo-085.git git add . git commit - 'init commit' git push origin main ``` - Reinitialize Firebase hosting ```bash firebase init hosting ``` - Create new branch for new feature ```bash git checkout -b feature-x ``` - Modify `src/App.js` - Commit, push and create PR ```bash git add . git commit -m 'Add new feature X' git push origin feature-x ``` - Go to `Actions` in GitHub repo ## Links - [Get started with Firebase Hosting](https://firebase.google.com/docs/hosting/quickstart) - [npx: an npm package runner](https://medium.com/@maybekatz/introducing-npx-an-npm-package-runner-55f7d4bd282b) - [Firebase Pricing](https://firebase.google.com/pricing?authuser=0) ## Clean Up - Remove `react-demo-085` firebase project - Remove `firebase-tools` ```bash npm uninstall -g firebase-tools ``` - Remove `react-demo-085` GitHub repo - Remove DNS records - Log out from Firebase ```bash firebase logout ``` --- ### Overview (`lessons/086/README.md`) # Build a Docker Image and Publish It to AWS ECR using Github Actions [YouTube Tutorial](https://youtu.be/Hv5UcBYseus) ## Content This guide contains instructions on how to: - Create GitHub Repository - Create Golang App - Create Dockerfile for Golang - Create GitHub Actions Workflow - Create AWS ECR Repository - Create AWS IAM User, Policy, and Group - Test GitHub Actions - Add Automatic Tagging of Releases ## Create GitHub Repository - Create GitHub repo `lesson-086` - Clone repository ```bash git clone git@github.com:antonputra/lesson-086.git ``` - Initialize the Go module ```bash go mod init github.com/antonputra/lesson-086 ``` - Create `main.go` file - Format the code and run it ```bash # go install golang.org/x/tools/cmd/goimports@latest go mod tidy goimports -w . go run main.go ``` ## Create Dockerfile for Golang - Create `Dockerfile` ## Create GitHub Actions Workflow - Create `.github/workflows/main.yml` file, use `latest` for `IMAGE_TAG` ## Create AWS ECR Repository - Go to AWS console, search for `ecr` - Create private ECR repository `lesson-086` ## Create AWS IAM User, Policy, and Group - Create IAM Policy `AllowPushPullPolicy` ```json { "Version": "2012-10-17", "Statement": [ { "Sid": "GetAuthorizationToken", "Effect": "Allow", "Action": [ "ecr:GetAuthorizationToken" ], "Resource": "*" }, { "Effect": "Allow", "Action": [ "ecr:BatchGetImage", "ecr:BatchCheckLayerAvailability", "ecr:CompleteLayerUpload", "ecr:GetDownloadUrlForLayer", "ecr:InitiateLayerUpload", "ecr:PutImage", "ecr:UploadLayerPart" ], "Resource": [ "arn:aws:ecr:us-east-1:424432388155:repository/lesson-086" ] } ] } ``` - Create `push-pull-images` IAM Group and attach `AllowPushPullPolicy` IAM Policy - Create user `github-actions` and place it to `push-pull-images` IAM Group ## Test GitHub Actions - Create initial commit ```bash git add . git commit -m 'init commit' git push origin main ``` - Check ECR repository ## Add Automatic Tagging of Releases - Create `build/git_update.sh` script - Make a change and push to remote branch from `main` - Create a new branch `aputra`, make a change, update `patch` -> `major` and push ## Links - [Workflow syntax for GitHub Actions](https://docs.github.com/en/actions/learn-github-actions/workflow-syntax-for-github-actions) - [Metadata syntax for GitHub Actions](https://docs.github.com/en/actions/creating-actions/metadata-syntax-for-github-actions) ## Clean Up - Delete `github-actions` AWS IAM User - Delete `AllowPushPullPolicy` AWS IAM Policy - Delete `push-pull-images` AWS IAM Group --- ### Overview (`lessons/087/README.md`) # Build a Docker Image and Publish It to GCP GCR & Artifact Registry using Github Actions [YouTube Tutorial](https://youtu.be/6dLHcnlPi_U) ## Content - Create GitHub Repository - Create Flask App - Create Docker Image for Flask App - Create Service Account in GCP - Create GitHub Actions Docker Build and Push Workflow - Use GitHub Actions to Push Docker to Artifact Registry - Increment Git Tag for Each Commit and Use It for Docker Image ## Create GitHub Repository - Go to [GitHub](https://github.com/) - Create empty repository `lesson-087`, keep it public - Clone repository ```bash git clone git@github.com:antonputra/lesson-087.git ``` ## Create Flask App - Create `app.py` - Create `requirements.txt` - Optionally, create `.gitignore` ## Create Docker Image for Flask App - Create `Dockerfile` - Build docker image ```bash docker build -t lesson-087:latest . ``` - Run docker locally ```bash docker run -p 5000:5000 lesson-087:latest ``` - Use curl to test ```bash curl localhost:5000/hello ``` ## Create Service Account in GCP - Create `github-actions` service account - Create and download key for service account - Add `Storage Admin` role to the service account, project wide permissions (fix later) ## Create GitHub Actions Docker Build and Push Workflow - Create `.github/workflows/gcp.yml` - Create GitHub secret `SERVICE_ACCOUNT_KEY` - Commit and push ```bash git add . git commit -m 'init commit' git push origin main ``` - Check docker image in GCR - Fix permissions, remove wide project permissions and add to GS bucket - Create empty commit to test CI ```bash git commit --allow-empty -m "Trigger CI" git push origin main ``` ## Use GitHub Actions to Push Docker to Artifact Registry - Create `images` Artifact Registry repository - Add `github-actions` service account to `images` repo with `Artifact Registry Writer` role - Add `gcloud auth configure-docker us-west2-docker.pkg.dev --quiet` to `Configure Docker Client` step - Add `Push Docker Image to Artifact Registry` step - Commit and push ```bash git add . git commit -m 'add step to push docker to artifact registry' git push origin main ``` ## Increment Git Tag for Each Commit and Use It for Docker Image - Create `scripts/git_update.sh` - Add `Automatic Tagging of Releases` step - Update `GIT_TAG` variable to `${{ steps.increment-git-tag.outputs.git-tag }}` - Commit and test ```bash git add . git commit -m 'add step to increment git tag' git push origin main ``` - Check GitHub Tag, GCR and Artifact registry ## Links - [Push to GCR GitHub Action](https://github.com/marketplace/actions/push-to-gcr-github-action) - [Uploading a Docker image to GCR using Github Actions](https://medium.com/mistergreen-engineering/uploading-a-docker-image-to-gcr-using-github-actions-92e1cdf14811) - [Using Github Actions to Build and Push Images to Google Container Registry](http://acaird.github.io/computers/2020/02/11/github-google-container-cloud-run) - [Creating a CI/CD environment for serverless containers on Cloud Run with GitHub Actions](https://cloud.google.com/community/tutorials/cicd-cloud-run-github-actions) - [Granting permissions](https://cloud.google.com/artifact-registry/docs/transition/changes-gcp#permissions) ## Clean Up - Delete `github-actions` service account - Delete `lesson-087` GCR repository - Delete `artifacts.devopsbyexample-325402.appspot.com` GS bucket - Delete `images` artifact repository --- ### Overview (`lessons/088/README.md`) # How to Create Your Own GitHub Actions? [YouTube Tutorial](https://youtu.be/jwdG6D-AB1k) ## Content - Demo - Build JavaScript GitHub Action - Test JavaScript GitHub Action ## Build JavaScript GitHub Action - Create GitHub repository `increment-git-tag` for JavaScript GitHub Action. Keep it public to allow other repositories to use this action. Also, add a README.md file, gitignore, and select MIT Licence. - Clone repository ```bash git clone git@github.com:antonputra/increment-git-tag.git ``` - Initialize the project and install dependencies ```bash cd increment-git-tag npm init -y npm i @actions/core npm i @actions/exec ``` - Open Project ```bash code . ``` - Create `action.yml` - Create `index.js` - Install vercel/ncc ```bash npm i @vercel/ncc ``` - Compile JavaScript code into single file ```bash ncc build index.js -o action ``` - Create `git_update.sh` - Make `git_update.sh` executable ```bash chmod +x chmod +x action/git_update.sh ``` - Commit and push ```bash git add . git commit -m 'init commit' git tag -a -m "increment git tag" v1 git push --follow-tags ``` ## Test JavaScript GitHub Action - Create GitHub repository `lesson-088` (private) - Clone repo ```bash git clone git@github.com:antonputra/lesson-088.git ``` - Create `.github/workflows/main.yaml` (patch) - Commit and push - Increment major version, commit and push ## Links - [Creating a JavaScript action](https://docs.github.com/en/actions/creating-actions/creating-a-javascript-action) - [@actions/core](https://github.com/actions/toolkit/tree/master/packages/core) - [@actions/exec](https://github.com/actions/toolkit/tree/master/packages/exec) --- ### Overview (`lessons/089/README.md`) # GitHub Actions Self Hosted Runner (Autoscaling with Kubernetes) [YouTube Tutorial](https://youtu.be/lD0t-UgKfEo) ## Content - Create GitHub Repository - Create EKS Cluster with Terraform - Install Cert-Manager on Kubernetes - Install actions-runner-controller - Create Single GitHub Actions Self Hosted Runner - Create Multiple GitHub Actions Self Hosted Runners - Autoscaling with Kubernetes ## Create GitHub Repository - Create private GitHub repository `lesson-089` ## Create EKS Cluster with Terraform - Go over each file under `terraform` folder - Create EKS with terraform ```bash terraform init terraform apply ``` - Configure kubectl ```bash aws eks --region us-east-1 update-kubeconfig --name demo ``` if you get an error `'NoneType' object is not iterable`, run `rm ~/.kube/config` - Check connection with Kubernetes ```bash kubectl get svc ``` ## Install Cert-Manager on Kubernetes - Add Helm repo ```bash helm repo add jetstack https://charts.jetstack.io ``` - Update Helm ```bash helm repo update ``` - Search for latest verion ```bash helm search repo cert-manager ``` - Split the screen and install Helm chart ```bash watch kubectl get pods -A ``` - Install Helm chart ```bash helm install \ cert-manager jetstack/cert-manager \ --namespace cert-manager \ --create-namespace \ --version v1.6.0 \ --set prometheus.enabled=false \ --set installCRDs=true ``` ## Install actions-runner-controller - Create GitHub App, call it `k8s-actions-89` - Create namespace ```bash kubectl create ns actions ``` - Create secret to authenticate with GitHub ```bash kubectl create secret generic controller-manager \ -n actions \ --from-literal=github_app_id=<> \ --from-literal=github_app_installation_id=<> \ --from-file=github_app_private_key=<> ``` - Add actions helm repo ```bash helm repo add actions-runner-controller https://actions-runner-controller.github.io/actions-runner-controller helm repo update helm search repo actions ``` - Install Helm chart ```bash helm install actions \ actions-runner-controller/actions-runner-controller \ --namespace actions \ --version 0.14.0 \ --set syncPeriod=1m ``` ## Create Single GitHub Actions Self Hosted Runner - Create `k8s/runner.yaml` ```bash kubectl apply -f k8s/runner.yaml kubectl get pods -n actions kubectl logs -f k8s-single-runner -c runner -n actions ``` - Go to GitHub to check self hosted runners - Clone `lesson-089` repository ```bash git clone git@github.com:antonputra/lesson-089.git ``` - Create `.github/workflows/main.yaml` - Commit & push - Check pods ```bash kubectl get pods -n actions ``` ## Create Multiple GitHub Actions Self Hosted Runners - Create `k8s/runner-deployment.yaml` ```bash kubectl apply -f k8s/runner-deployment.yaml kubectl get pods -n actions ``` - Check in GitHub repo self-hosted runners - Create 2 build jobs with docker - Create `Dockerfile` - Ceate `.github/workflows/example-2.yaml` ## Autoscaling with Kubernetes - Deploy `k8s/horizontal-runner-autoscaler.yaml` ```bash kubectl apply -f k8s/horizontal-runner-autoscaler.yaml ``` - Ceate `.github/workflows/example-3.yaml` ## Links - [Hosting your own runners](https://docs.github.com/en/actions/hosting-your-own-runners) - [GitHub Actions Runner](https://github.com/actions/runner) - [actions-runner-controller](https://github.com/actions-runner-controller/actions-runner-controller) - [actions-runner-controller values.yaml](https://github.com/actions-runner-controller/actions-runner-controller/blob/master/charts/actions-runner-controller/values.yaml) --- ### Overview (`lessons/090/README.md`) # Kubernetes Terminal: 6 Must-Have Tools to Install & Examples [YouTube Tutorial](https://youtu.be/xw3j4aNbHgQ) --- ### Overview (`lessons/091/README.md`) # Kubernetes NodePort vs LoadBalancer vs Ingress? [YouTube Tutorial](https://youtu.be/UzCgEVqIhg0) --- ### Overview (`lessons/092/README.md`) # Kubernetes Namespaces vs. Virtual Clusters [YouTube Tutorial](https://youtu.be/IP64cimwwgM) ## Install vcluster ```bash vcluster create vcluster-1 -n host-namespace-1 vcluster connect vcluster-1 --namespace host-namespace-1 kubectl get namespace kubectl create namespace demo-nginx kubectl create deployment nginx-deployment -n demo-nginx --image=nginx kubectl get pods -n demo-nginx kubectl get deployment -n host-namespace-1 kubectl get pods -n host-namespace-1 ``` ## Links - [vcluster - GitHub](https://github.com/loft-sh/vcluster) - [vcluster - getting-started](https://www.vcluster.com/docs/getting-started/setup) - [K3s](https://k3s.io/) --- ### Overview (`lessons/093/README.md`) # Kubernetes Basics: Pods, Nodes, Containers, Deployments and Clusters [YouTube Tutorial](https://youtu.be/B_X4l4HSgtc) --- ### Overview (`lessons/094/README.md`) # Kubernetes Quality of Service (QoS) Classes for Pods (Guaranteed, Burstable, BestEffort) [YouTube Tutorial](https://youtu.be/ka-HItczp2I) ## Links - [stress-ng](https://unix.stackexchange.com/questions/99334/how-to-fill-90-of-the-free-memory) --- ### Overview (`lessons/095/README.md`) # Kubernetes Pod Priority [YouTube Tutorial](https://youtu.be/sR_Zmvme3-0) ```bash kubectl get pods -A kubectl get pods -A \ -o jsonpath='{range .items[*]}{.metadata.name}{" - "}{.spec.priorityClassName}{"\n"}{end}' ``` --- ### Overview (`lessons/096/README.md`) # Kubernetes Affinity and Anti Affinity vs NodeSelector (Examples) [YouTube Tutorial](https://youtu.be/DKLMzDD3xqA) --- ### Overview (`lessons/097/README.md`) # Kubernetes Taints and Tolerations [YouTube Tutorial](https://youtu.be/6EMtEJC00FY) ```bash kubectl get nodes kubectl taint nodes node1 role=spot:NoSchedule kubectl get pods ``` --- ### Overview (`lessons/098/README.md`) # Kubernetes Pod Disruption Budget [YouTube Tutorial](https://youtu.be/e2HjRrmXMDw) --- ### Overview (`lessons/099/README.md`) # Kubernetes Local Persistent Volume: DON'T use hostPath! [YouTube Tutorial](https://youtu.be/fADm0DGgEJw) ## Postgress ```bash kubectl port-forward database-postgresql-0 5432 psql --host localhost \ --port 5432 \ --username postgres \ --password CREATE DATABASE test_hostpath; \l ``` ## Drain Nodes ```bash kubectl get nodes kubectl drain \ --delete-emptydir-data \ --ignore-daemonsets ``` ## Prepare Disks ```bash ssh -i ~/Downloads/devops.pem ec2-user@3.239.109.105 lsblk sudo mkfs -t xfs /dev/nvme1n1 sudo lsblk -o +UUID sudo mkdir \ -p /mnt/ssd-disks/ sudo mount \ /dev/nvme1n1 \ /mnt/ssd-disks/ sudo vi /etc/fstab UUID= /mnt/ssd-disks/ xfs defaults,nofail 0 2 ``` --- ### Overview (`lessons/100/README.md`) # How to Deploy Create React App to GitHub Pages? [YouTube Tutorial](https://youtu.be/K5DTIf-jWhk) ## Create React App & GitHub Repo - Create `react-pages-demo` empty GitHub repository - Create react app using `create-react-app` tool ```bash npx create-react-app react-pages-demo ``` - Test react app locally ```bash cd react-pages-demo npm start ``` ## Deploy React App to GitHub Pages - Add `homepage` attribute to the `package.json` file ```json { ... "homepage": "https://antonputra.github.io/react-pages-demo", ... } ``` - Install `gh-pages` npm module to publishing files to a gh-pages branch on GitHub ```bash npm install gh-pages ``` - Add couple of tasks to `package.json` file ```json { ... "scripts": { ... "predeploy": "npm run build", "deploy": "gh-pages -d build" } } ``` - Connect local repository to the remote ```bash git remote add origin git@github.com:antonputra/react-pages-demo.git git branch -M main git push -u origin main ``` - Deploy react app to GitHub pahes ```bash npm run deploy ``` ## Setup Custom Domain for GitHub Pages - Set your custom DNS name from the GitHub settings Pages (devopsbyexample.io) - Go to your DNS provider (in my case Google Domains) - Create `A` record for your Apex domain (devopsbyexample.io) ```bash 185.199.108.153 185.199.109.153 185.199.110.153 185.199.111.153 ``` - Create `CNAME` record for `www` subdomain - www -> antonputra.github.io - Add `CNAME` file under `public` directory ``` devopsbyexample.io ``` - Update homepage attribute ```json { ... "homepage": "https://devopsbyexample.io", ... } ``` - Commit the changes and deploy to github pages ```bash git add . git commit -m 'Add custom domain' git push origin main npm run deploy ``` ## Automate Deployment with GitHub Actions - Create `.github/workflows/github-pages.yml` GitHub actions workflow ```yaml --- name: Build and Deploy React App to GitHub Pages on: push: branches: [ main ] jobs: build-and-deploy: name: Build and Deploy runs-on: ubuntu-latest steps: - name: Checkout uses: actions/checkout@v2 - name: Build run: npm install - name: Test run: npm run test - name: Deploy run: | git config --global user.name 'Anton Putra' git config --global user.email 'me@antonputra.com' git remote set-url origin https://x-access-token:${{ secrets.GITHUB_TOKEN }}@github.com/${{ github.repository }} npm run deploy ``` - Update `src/App.js` with random text - Commit and push ```bash git add . git commit -m 'Add GitHub Actions' git push origin main ``` ## Links - [GitHub gh-pages](https://github.com/tschaub/gh-pages) --- ### Overview (`lessons/101/README.md`) # Terraform Ansible Integration: GCP Example & Multiple VMs [YouTube Tutorial](https://youtu.be/wVq5fwx1OQU) ## Prerequisites - Ansible - Terraform ## Generate SSH Key Pair - Run the following command to generate elliptic curve ssh key pair ```bash ssh-keygen -t ed25519 -f ~/.ssh/ansbile_ed25519 -C ansible ``` - **Optionally**, you can generate RSA key pair ```bash ssh-keygen -t rsa -f ~/.ssh/ansbile -C ansible -b 2048 ``` ## Add SSH Key to GCP Project - Go to compute engine section - Click Metadata - Select `SSH KEYS` tab and click `ADD SSH KEY` - Upload public `ansbile_ed25519.pub` key - `cat ~/.ssh/ansbile_ed25519.pub` ## Create Ansible Playbook for Nginx Installation - Create `roles/nginx/tasks/main.yaml` task - Create `nginx.yaml` playbook - Create Ansible config `ansible.cfg` to skip host key verification ## Create Terraform Ansible Integration Example - Create terraform file `1-example.tf` - Open default firewall rules - Verify that you have Terraform and Ansible installed ```bash command -v terraform command -v ansible-playbook ``` - Run `terraform init` to download provider and plugins - Run `terraform apply` to create Nginx instance in GCP ## Provision Multiple VMs - Destroy previous example `terraform destroy` - Copy `1-example.tf` -> `2-example.tf` - Rename `1-example.tf` -> `1-example.tf.bac` - Add local variable ``` web_servers = { nginx-000-staging = { machine_type = "e2-micro" zone = "us-central1-a" } nginx-001-staging = { machine_type = "e2-micro" zone = "us-central1-b" } } ``` ``` for_each = local.web_servers name = each.key machine_type = each.value.machine_type zone = each.value.zone ``` ``` host = self.network_interface.0.access_config.0.nat_ip ``` ``` command = "ansible-playbook -i ${self.network_interface.0.access_config.0.nat_ip}, --private-key ${local.private_key_path} nginx.yaml" ``` ``` output "nginx_ips" { value = { for k, v in google_compute_instance.nginx : k => "http://${v.network_interface.0.access_config.0.nat_ip}" } } ``` - Run `terraform apply` ## Clean Up - `terraform destroy` - rm ~/.ssh/ansbile* - Delete SSH public key from GCP project --- ### Overview (`lessons/102/README.md`) # How to Create EKS Cluster Using Terraform You can find tutorial [here](https://antonputra.com/terraform/how-to-create-eks-cluster-using-terraform/). --- ### Overview (`lessons/103/README.md`) # How to Add IAM User and IAM Role to AWS EKS Cluster? You can find tutorial [here](https://antonputra.com/kubernetes/add-iam-user-and-iam-role-to-eks/). --- ### Overview (`lessons/104/README.md`) # How to Install Prometheus and Grafana on Ubuntu? You can find tutorial [here](https://antonputra.com/monitoring/install-prometheus-and-grafana-on-ubuntu/). --- ### Overview (`lessons/105/README.md`) # How to Install MongoDB on Kubernetes? You can find tutorial [here](https://antonputra.com/kubernetes/how-to-install-mongodb-on-kubernetes/). --- ### Overview (`lessons/106/README.md`) # Google Cloud Functions Tutorial You can find tutorial [here](https://antonputra.com/google/google-cloud-functions-tutorial/). --- ### Overview (`lessons/107/README.md`) # How to Hack a Password in Python? You can find tutorial [here](https://antonputra.com/python/hack-a-password-in-python/). --- ### Overview (`lessons/108/README.md`) # How to Create GKE Cluster Using TERRAFORM? You can find tutorial [here](https://antonputra.com/google/create-gke-cluster-using-terraform). --- ### Overview (`lessons/109/README.md`) # How to Create EKS Cluster Using eksctl? You can find tutorial [here](https://antonputra.com/amazon/create-eks-cluster-using-eksctl/). --- ### Overview (`lessons/110/README.md`) # How to Deploy React App (any static website) to S3 and CloudFront? You can find tutorial [here](https://antonputra.com/amazon/deploy-react-to-s3-and-cloudfront/). --- ### Overview (`lessons/111/README.md`) # Kubernetes Kustomize Tutorial: 10 examples You can find tutorial [here](https://antonputra.com/kubernetes/kubernetes-kustomize-tutorial/). --- ### Overview (`lessons/112/README.md`) # Native EKS Ingress: AWS Load Balancer Controller & 5 Examples (Terraform, TLS, IngressGroup, IP) You can find tutorial [here](https://youtu.be/ZfjpWOC5eoE). --- ### Overview (`lessons/113/README.md`) # Create AWS EKS Fargate Using Terraform (EFS, HPA, Ingress, ALB, IRSA, Kubernetes, Helm, Tutorial) You can find tutorial [here](https://antonputra.com/amazon/create-aws-eks-fargate-using-terraform/). --- ### Overview (`lessons/114/README.md`) # Kubernetes Node Autoscaling with Karpenter (AWS EKS & Terraform) You can find tutorial [here](https://antonputra.com/amazon/kubernetes-node-autoscaling-with-karpenter/). --- ### Overview (`lessons/115/README.md`) # How to Create AWS Lambda with Terraform? (API Gateway & GET/POST & IAM S3 Access & Dependencies) You can find tutorial [here](https://antonputra.com/amazon/how-to-create-aws-lambda-with-terraform/). --- ### Overview (`lessons/116/README.md`) # How to Integrate Amazon API Gateway with Amazon EKS? (Terraform & AWS Lambda & Serverless & NLB) You can find tutorial [here](https://antonputra.com/amazon/Integrate-amazon-api-gateway-with-amazon-eks/). --- ### Overview (`lessons/117/README.md`) # AWS API Gateway - Custom Domain (Console + Terraform | Route53 + Without Route53) You can find tutorial [here](https://antonputra.com/amazon/aws-api-gateway-custom-domain/). --- ### Overview (`lessons/118/README.md`) # AWS API Gateway - EC2 Integration (Console + Terraform | Backend | Endpoint | HTTP | Node JS) You can find tutorial [here](https://antonputra.com/amazon/aws-api-gateway-ec2-integration/). --- ### Overview (`lessons/119/README.md`) # AWS API Gateway - WebSocket API + EC2 (HTTP & VPC Link & Auth & API Keys & Lambda Authorizer) You can find tutorial [here](https://antonputra.com/amazon/aws-api-gateway-websocket-api-ec2/). --- ### Overview (`lessons/120/README.md`) # How To SSH into your VM? - Google Cloud Platform (GCP | IAP tunnel | OS Login | Audit | Google IAM) You can find tutorial [here](https://antonputra.com/google/gcp-how-to-ssh-into-your-vm/). --- ### Overview (`lessons/121/README.md`) # I've LOST my private key. How to connect to EC2 instance?? (Amazon Web Services | AWS | SSH | SSM) You can find tutorial [here](https://antonputra.com/amazon/how-to-connect-to-ec2-instance/). --- ### Overview (`lessons/122/README.md`) # how to send AWS CloudWatch Alarms to Slack? (Terraform Included) You can find tutorial [here](https://antonputra.com/amazon/send-aws-cloudwatch-alarms-to-slack/). --- ### Overview (`lessons/123/README.md`) # Creating a Linux service with systemd You can find tutorial [here](https://antonputra.com/python/creating-a-linux-service-with-systemd/). --- ### Overview (`lessons/124/README.md`) # AWS Lambda Go vs. Node.js performance benchmark - Compile go code for Lambda ```bash GOOS=linux GOARCH=amd64 CGO_ENABLED=0 go build -o main main.go ``` - Run load test ```bash loadtest \ --concurrency 5 \ --maxRequests 1000 \ --rps 10 \ --keepalive \ https://.lambda-url.us-east-1.on.aws/ ``` --- ### Overview (`lessons/125/README.md`) # How to create EKS Cluster using Terraform MODULES (AWS Load Balancer Controller + Autoscaler + IRSA) # Architecture Diagram You can find tutorial [here](https://antonputra.com/amazon/create-eks-cluster-using-terraform-modules/). --- ### Overview (`lessons/126/README.md`) # AWS Lambda Python vs. Node.js performance benchmark You can find tutorial [here](https://antonputra.com/python/python-vs-nodejs-benchmark/). --- ### Overview (`lessons/127/README.md`) # How to create Application Load Balancer using Terraform? (AWS ALB | HTTPS) You can find tutorial [here](https://antonputra.com/amazon/create-alb-terraform/). --- ### Overview (`lessons/128/README.md`) # AWS Lambda Go vs. Rust performance benchmark You can find tutorial [here](https://youtu.be/wyXIA3hfP88). --- ### Overview (`lessons/129/README.md`) ## AWS Lambda vs Cloud Functions (2nd gen) performance benchmark You can find tutorial [here](https://youtu.be/kB-fM31UQBM). --- ### Overview (`lessons/130/README.md`) ## Monitoring EKS & EC2 instances with MANAGED Prometheus & Grafana You can find tutorial [here](https://youtu.be/-nUQNFAX5TI). --- ### Overview (`lessons/131/README.md`) ## Send Alerts to Slack, Email, PagerDuty - AWS Managed Prometheus (AMP) You can find tutorial [here](https://youtu.be/SvDpuVlJTDg). --- ### Overview (`lessons/132/README.md`) ## GCP Managed Prometheus Tutorial: Managed Collection + Grafana + Workload Identity + Terraform You can find tutorial [here](https://youtu.be/kdkNEJAb4XY). --- ### Overview (`lessons/133/README.md`) ## EKS Storage | Install Amazon EBS CSI driver & KMS You can find tutorial [here](https://https://youtu.be/Gj31ffZm-ag). --- ### Overview (`lessons/134/README.md`) # How to monitor Persistent Volume usage in Kubernetes using Prometheus? You can find tutorial [here](https://youtu.be/ff_XHm96PKQ). --- ### Overview (`lessons/135/README.md`) # How to monitor Containers in Kubernetes using Prometheus & cAdvisor & Grafana? CPU, Memory, Network You can find tutorial [here](https://youtu.be/dMca4jHaft8). --- ### Overview (`lessons/136/README.md`) # Monitor HTTP/REST API in Kubernetes using Prometheus & Nginx Ingress Controller You can find tutorial [here](https://youtu.be/VjFFzGFyVlY). --- ### Overview (`lessons/137/README.md`) # How to Monitor Golang with Prometheus (Counter - Gauge - Histogram - Summary) You can find tutorial [here](https://antonputra.com/monitoring/monitor-golang-with-prometheus/). --- ### Overview (`lessons/138/README.md`) # Go (Golang) vs. Rust: (HTTP/REST API in Kubernetes) Performance Benchmark You can find tutorial [here](https://youtu.be/QWLyIBkBrl0). --- ### Overview (`lessons/139/README.md`) # Use Kubernetes Gateway API instead of Ingress! (TLS - Cert Manager - Istio - Prometheus) You can find tutorial [here](https://youtu.be/nJUzGJQR3tM). --- ### Overview (`lessons/140/README.md`) # How to Monitor Nginx with Prometheus and Grafana? (Ste-by-Step - Install - Monitor - Fluentd) You can find tutorial [here](https://antonputra.com/monitoring/monitor-nginx-with-prometheus/). --- ### Overview (`lessons/141/README.md`) # How to Build Custom Prometheus Exporter? (Step-by-Step - Real-world Example - Parse Log + HTTP) You can find tutorial [here](https://youtu.be/3wT0zSsQb58). --- ### Overview (`lessons/142/README.md`) # Use Kubernetes Gateway API for Mesh (Service-to-Service Communication - Istio - Kiali - Prometheus) You can find tutorial [here](https://youtu.be/joyucBMNrmw). --- ### Overview (`lessons/143/README.md`) # Go (Golang) vs Node JS (Microservices) performance benchmark (Istio - Prometheus - Grafana) You can find tutorial [here](https://youtu.be/ntMKNlESCpM). --- ### Overview (`lessons/144/README.md`) # Nginx vs Traefik performance benchmark (HTTP - HTTPS - gRPC | Latency - CPU - Memory | Prometheus) You can find tutorial [here](https://youtu.be/bgcfEW_Yh7E). --- ### Overview (`lessons/145/README.md`) # Go (Golang) vs Java performance benchmark (Fiber vs Spring Boot | Prometheus | Minio | MongoDB) You can find tutorial [here](https://youtu.be/8CiErLxdaA8). --- ### Overview (`lessons/146/README.md`) # Nginx vs Apache HTTP Server performance benchmark (Prometheus | HTTP | HTTPS | Grafana | Ansible) You can find tutorial [here](https://youtu.be/Fx8afVZJLd4). --- ### Overview (`lessons/147/README.md`) # Go (Golang) vs Rust performance benchmark (gRPC | Tonic | Prometheus | Kubernetes | Grafana) You can find tutorial [here](https://youtu.be/IcMdPfJpbyc). --- ### Overview (`lessons/148/README.md`) # HTTP/2 vs HTTP/3 performance benchmark (Nginx - Prometheus - Grafana - GCP - Terraform - Ansible) You can find tutorial [here](https://youtu.be/-0yu_zOFilg). --- ### Overview (`lessons/149/README.md`) # gRPC vs REST performance benchmark (Protobuf - JSON - GKE - Terraform - Golang - Kubernetes - k6) You can find tutorial [here](https://youtu.be/ZwP4ly03n00). --- ### Overview (`lessons/150/README.md`) # ARM vs x86 performance benchmark (amd64 vs arm64 - AWS Graviton - AWS EKS - Kubernetes - Golang) You can find tutorial [here](https://youtu.be/ndXKHgFIIL4). --- ### Overview (`lessons/151/README.md`) # Nginx vs Envoy performance benchmark (HTTP - HTTPS - gRPC - Golang - OpenTelemetry - Prometheus) You can find tutorial [here](https://youtu.be/0Q9I-x--np4). --- ### Overview (`lessons/152/README.md`) # gRPC vs Kafka: Which one should you choose? You can find tutorial [here](https://youtu.be/Qkhfhhn8ALU). --- ### Overview (`lessons/153/README.md`) # Traefik vs Caddy 2 performance benchmark You can find tutorial [here](https://youtu.be/-r29v7UWINI). --- ### Overview (`lessons/154/README.md`) # Prometheus Operator Kubernetes Tutorial You can find tutorial [here](https://youtu.be/HOmdYtsB950). --- ### Overview (`lessons/155/README.md`) # Istio Service Mesh Tutorial You can find tutorial [here](https://youtu.be/H4YIKwAQMKk). --- ### Overview (`lessons/156/README.md`) # How to Use EKS Blueprints Terraform Modules? You can find tutorial [here](https://youtu.be/KE504NwP9vs). --- ### Overview (`lessons/157/README.md`) # DevOps Interview Questions: (Step-by-Step) #1 You can find tutorial [here](https://youtu.be/u7DbaXb8vyU). --- ### Overview (`lessons/158/README.md`) # ArgoCD Tutorial for Beginners: GitOps CD for Kubernetes You can find tutorial [here](https://youtu.be/zGndgdGa1Tc). --- ### Overview (`lessons/159/README.md`) # ArgoCD Image Updater: Decouple CI from CD! #2 You can find tutorial [here](https://youtu.be/cTQybeDzngU). --- ### Overview (`lessons/160/README.md`) # Terragrunt Tutorial: Create VPC, EKS from Scratch! (Step-by-Step) You can find tutorial [here](https://youtu.be/yduHaOj3XMg). --- ### Overview (`lessons/161/README.md`) # Terraform Tips & Tricks: loops, if-statements, and more You can find tutorial [here](https://youtu.be/7S94oUTy2z4). --- ### Overview (`lessons/162/README.md`) # How to Manage Secrets in Terraform You can find tutorial [here](https://youtu.be/3N0tGKwvBdA). --- ### Overview (`lessons/163/README.md`) # Thanos (Prometheus) Tutorial: Remote Read/Write - mTLS - Step-by-Step! You can find tutorial [here](https://youtu.be/feHSU0BMcco). --- ### Overview (`lessons/164/README.md`) # An Introduction to Terraform You can find tutorial [here](https://youtu.be/6XSroskdCF0). --- ### Overview (`lessons/165/README.md`) # How to manage Terraform State? You can find tutorial [here](https://youtu.be/GgQE85Aq2z4). --- ### Overview (`lessons/166/README.md`) # HardenEKS: Best Practices For Amazon EKS (Programmatically) You can find tutorial [here](https://youtu.be/UJQqOY3sBbY). --- ### Overview (`lessons/167/README.md`) # 9 Ways to Expose an App on Amazon EKS to the Internet You can find tutorial [here](https://youtu.be/ePqUq06WoLk). --- ### Overview (`lessons/168/README.md`) # How to build multi-arch Docker Images? (Docker Hub - ECR - EKS Demo) You can find tutorial [here](https://youtu.be/AQeGdMuJWIM). --- ### Overview (`lessons/169/README.md`) ## How do I run a PostgreSQL image? First of all, you need to run the PostgreSQL Docker container in the background. ```sh docker run --detach --name my-postgres --env POSTGRES_PASSWORD=devops123 aputra/postgres-169:15.3 ``` You don't need to install the PostgreSQL client locally on your laptop. Instead, SSH into the PostgreSQL container by running the following command. ```sh docker exec -it my-postgres psql -U postgres ``` Verify that you have all the necessary tables. ```sh \d ``` ``` List of relations Schema | Name | Type | Owner --------+----------+-------+---------- public | action | table | postgres public | customer | table | postgres public | event | table | postgres public | event_v2 | table | postgres public | student | table | postgres public | teacher | table | postgres ``` ## Inner JOIN ```sql SELECT * FROM customer JOIN event ON customer.customer_id = event.customer_id; ``` ## Left JOIN ```sql SELECT * FROM customer LEFT JOIN event ON customer.customer_id = event.customer_id; ``` ## Left Outer JOIN with Exclusion ```sql SELECT * FROM customer LEFT JOIN event ON customer.customer_id = event.customer_id WHERE event.customer_id IS NULL; ``` ## Right JOIN ```sql SELECT * FROM event_v2 RIGHT JOIN action ON event_v2.action_id = action.action_id; ``` ## Rewrite as Left JOIN ```sql SELECT * FROM action LEFT JOIN event_v2 ON action.action_id = event_v2.action_id; ``` ## Join 3 tables ```sql SELECT * FROM action LEFT JOIN event_v2 ON event_v2.action_id = action.action_id LEFT JOIN customer ON customer.customer_id = event_v2.customer_id; ``` ## Select only customer and action names ```sql SELECT action.name, customer.name FROM action LEFT JOIN event_v2 ON event_v2.action_id = action.action_id LEFT JOIN customer ON customer.customer_id = event_v2.customer_id; ``` ## Full JOIN ```sql SELECT * FROM teacher FULL OUTER JOIN student ON teacher.age = student.age; ``` ## UNION ```sql SELECT age FROM teacher UNION SELECT age FROM student; ``` ## UNION ALL ```sql SELECT age FROM teacher UNION ALL SELECT age FROM student; ``` ## CROSS JOIN ```sql SELECT * FROM teacher CROSS JOIN student; ``` ## How can I build a multi-architecture Docker image? ```sh docker build -t aputra/postgres-169-arm64:15.3 --platform linux/arm64 . docker build -t aputra/postgres-169-amd64:15.3 --platform linux/amd64 . docker push aputra/postgres-169-arm64:15.3 docker push aputra/postgres-169-amd64:15.3 docker manifest create aputra/postgres-169:15.3 \ aputra/postgres-169-arm64:15.3 \ aputra/postgres-169-amd64:15.3 docker manifest push aputra/postgres-169:15.3 ``` ## How can I manually create tables and insert data? ```sql CREATE TABLE customer ( customer_id integer PRIMARY KEY, name varchar(256), address varchar(256) ); INSERT INTO customer VALUES (1, 'Casey', '2295 Spring Avenue'), (2, 'Peter', '924 Emma Street'), (3, 'Erika', '397 Terry Lane'); CREATE TABLE event ( event_id integer PRIMARY KEY, customer_id integer REFERENCES customer(customer_id), action varchar(256) ); INSERT INTO event VALUES (101, '3', 'LOGIN'), (102, '3', 'VIEW PAGE'), (103, '1', 'LOGIN'), (104, '1', 'SEARCH'); CREATE TABLE action ( action_id integer PRIMARY KEY, name varchar(256) ); INSERT INTO action VALUES (201, 'LOGIN'), (202, 'VIEW PAGE'), (203, 'SEARCH'), (204, 'LOGOUT'); CREATE TABLE event_v2 ( event_id integer PRIMARY KEY, customer_id integer REFERENCES customer(customer_id), action_id integer REFERENCES action(action_id) ); INSERT INTO event_v2 VALUES (101, 2, 201), (102, 2, 204); CREATE TABLE teacher ( teacher_id integer PRIMARY KEY, name varchar(256), age integer ); INSERT INTO teacher VALUES (1, 'Tiffany', 28), (2, 'Mathew', 35); CREATE TABLE student ( student_id integer PRIMARY KEY, name varchar(256), age integer ); INSERT INTO student VALUES (1, 'Ben', 28), (2, 'Jenny', 21); ``` ## Select all the data ```sql SELECT * FROM customer; SELECT * FROM event; SELECT * FROM action; SELECT * FROM event_v2; SELECT * FROM teacher; SELECT * FROM student; ``` --- ### Overview (`lessons/170/README.md`) # Kubernetes Node Selector vs Node Affinity vs Pod Affinity vs Tains & Tolerations You can find tutorial [here](https://youtu.be/rX4v_L0k4Hc). ## Start minikube with 3 nodes ```bash minikube start --nodes 3 --driver=docker ``` ## Get Kubernetes node labels ```bash kubectl get node --show-labels ``` ## Set the node label ```bash # kubectl label nodes

= kubectl label nodes minikube-m03 disktype=ssd ``` ## Set the node taint ```bash # kubectl taint nodes

=: kubectl taint nodes minikube-m03 price=spot:NoSchedule ``` ## Links - [Kubernetes Scheduler](https://kubernetes.io/docs/concepts/scheduling-eviction/kube-scheduler/) - [Scheduler Performance Tuning](https://kubernetes.io/docs/concepts/scheduling-eviction/scheduler-perf-tuning/) - [Assigning Pods to Nodes](https://kubernetes.io/docs/concepts/scheduling-eviction/assign-pod-node/) --- ### Overview (`lessons/171/README.md`) # Most Common Kubernetes Deployment Strategies (Examples & Code) You can find tutorial [here](https://youtu.be/lxc4EXZOOvE). ## Start Minikube ```bash minikube start --driver=docker ``` ## Deploy Prometheus (apply 2 times) ```bash kubectl apply --server-side -R -f monitoring/ ``` ## Install Istio & Flagger ```bash cd terraform terraform apply ``` ## Test application ```bash kubectl run curl --image=alpine/curl:8.2.1 -n kube-system -i --tty --rm -- sh for i in `seq 1 1000`; do curl myapp.default:8181/version; echo ""; sleep 1; done ``` --- ### Overview (`lessons/172/README.md`) # Linkerd vs. Istio (Rust vs. C++) performance benchmark You can find tutorial [here](https://youtu.be/pzcuE0aVvP0). --- ### Overview (`lessons/173/README.md`) # Kubernetes Canary Deployment (Manual vs Automated) You can find tutorial [here](https://youtu.be/fWe6k4MmeSg). ## Start Minikube ```bash minikube start --driver=docker ``` ## Deploy All Dependencies ```bash cd terraform terraform apply ``` ## Grafana ```bash # username: admin, password: devops123 kubectl port-forward svc/grafana 3000 -n monitoring ``` ## Test application (Example 1) ```bash kubectl run curl --image=alpine/curl:8.2.1 -n default -i --tty --rm -- sh for i in `seq 1 1000`; do curl myapp:8080/version; echo ""; sleep 1; done ``` ## Test application (Example 2) ```bash kubectl run curl --image=alpine/curl:8.2.1 -n staging -i --tty --rm -- sh for i in `seq 1 1000`; do curl myapp:8080/version; echo ""; sleep 1; done ``` --- ### Overview (`lessons/174/README.md`) # Kubernetes Ingress Explained (2 Types) You can find tutorial [here](https://youtu.be/1BksUVJ1f5M). --- ### Overview (`lessons/175/README.md`) # AWS App Mesh Tutorial (EKS | Ingress | Terraform) You can find tutorial [here](https://youtu.be/m9B7RaSmiQ0). --- ### Overview (`lessons/176/README.md`) # Crossplane Tutorial (vs Terraform): Create AWS VPC - EKS - IRSA - Cluster Autoscaler - CSI Driver You can find tutorial [here](https://youtu.be/mpfqPXfX6mg). ## Install Crossplane on Kubernetes ```bash minikube start --driver=docker helm repo add crossplane-stable https://charts.crossplane.io/stable helm repo update helm search repo crossplane helm install crossplane \ --namespace crossplane-system \ --create-namespace \ --version 1.13.2 \ crossplane-stable/crossplane kubectl get pods -n crossplane-system kubectl get crds | grep crossplane.io ``` ## Create S3 Bucket using Crossplane ```bash kubectl apply -f 0-crossplane/1-provider-aws-s3.yaml kubectl get providers kubectl get pods -n crossplane-system kubectl get crds | grep aws.upbound.io kubectl create secret generic aws-secret \ -n crossplane-system \ --from-file=creds=./aws-credentials.txt kubectl apply -f 0-crossplane/0-provider-aws-config.yaml kubectl apply -f 1-s3/0-my-bucket.yaml kubectl get bucket kubectl describe bucket kubectl get bucket kubectl apply -f 1-s3/1-my-bucket-versioning.yaml kubectl get bucketversioning ``` ## Create AWS VPC using Crossplane ```bash kubectl apply -f 0-crossplane/2-provider-aws-ec2.yaml kubectl get providers kubectl apply -f 2-vpc kubectl get VPC kubectl get InternetGateway kubectl get RouteTableAssociation ``` ## Create EKS Cluster using Crossplane ```bash kubectl apply -f 0-crossplane kubectl get providers kubectl apply -f 3-eks kubectl get Cluster kubectl get NodeGroup aws configure --profile crossplane aws eks update-kubeconfig --name dev-demo --region us-east-2 --profile crossplane kubectl get nodes ``` ## Create OpenID Connect Provider (OIDC) ```bash kubectl apply -f 4-irsa kubectl get OpenIDConnectProvider kubectl get Addon ``` ## Deploy EBS CSI driver ```bash kubectl apply -f 5-storageclass ``` ## Deploy Cluster Autoscaler ```bash helm repo add autoscaler https://kubernetes.github.io/autoscaler helm search repo cluster-autoscaler helm install autoscaler \ --namespace kube-system \ --version 9.29.3 \ --values 6-cluster-autoscaler/1-helm-values.yaml \ autoscaler/cluster-autoscaler ``` --- ### Overview (`lessons/177/README.md`) # Azure Kubernetes Service (AKS) Tutorial: (Terraform - Nginx Ingress & TLS - OIDC Workload Identity) You can find tutorial [here](https://youtu.be/8HmReos6dlY). ## Create AKS cluster using Terraform ```bash az login az account list az account set --subscription terraform init terraform apply az aks get-credentials --resource-group tutorial --name dev-demo ``` ## Create Public and Private load balancers ```bash kubectl apply -f k8s/1-example kubectl get svc curl http:/// ``` ## Auto-Scaling ```bash kubectl get nodes kubectl apply -f k8s/2-example kubectl get pods kubectl describe pods nginx-v2-788b5579fd-dmbg8 kubectl get pods kubectl get nodes ``` ## Create an Ingress using Nginx Ingress ```bash kubectl get svc -n ingress kubectl apply -f k8s/3-example kubectl get pods kubectl get ing curl --resolve "echo.antonputra.pvt:80:20.96.71.30" http://echo.antonputra.pvt/ ``` ## Secure the Ingress with TLS & Cert-manager ```bash kubectl apply -f k8s/4-example kubectl get pods kubectl get ing kubectl get Certificate kubectl describe Certificate kubectl describe CertificateRequest kubectl describe Order kubectl describe Challenge kubectl get ing kubectl get Certificate dig echo.devopsbyexample.com kubectl get ing ``` ## Test Workload Identity ```bash kubectl apply -f k8s/5-example kubectl get pods -n dev kubectl exec -it azure-cli-c97fd4f7c-rp2mc -n dev -- sh az login --federated-token "$(cat $AZURE_FEDERATED_TOKEN_FILE)" --service-principal -u $AZURE_CLIENT_ID -t $AZURE_TENANT_ID az storage blob list -c test --account-name devtest2392919 kubectl delete -f k8s/5-example ``` --- ### Overview (`lessons/178/README.md`) # OpenTelemetry Golang Tutorial (Tracing in Grafana & Kubernetes & Tempo) You can find tutorial [here](https://youtu.be/ZIN7H00ulQw). ## Commands from the Tutorial ```bash git clone git@github.com:antonputra/tutorials.git cd tutorials/lessons/178 cd terraform terraform init terraform apply cd .. cd myapp go mod tidy kubectl get pods -n monitoring kubectl port-forward svc/tempo 4318 -n monitoring export OTLP_ENDPOINT=localhost:4318 curl http://0.0.0.0:8080/devices kubectl port-forward svc/grafana 3000:80 -n monitoring # Datasource URL: http://tempo.monitoring:3100 kubectl apply -f k8s kubectl get pods -n default kubectl get svc -n default kubectl port-forward svc/myapp 8080:8080 -n default ``` --- ### Overview (`lessons/179/README.md`) # ArgoCD Notifications (Successful/Failed Deployments) You can find tutorial [here](https://youtu.be/OP6IRsNiB4w). ## Commands from the Tutorial ```bash minikube start cd terraform terraform init terraform apply kubectl get pods -n argocd kubectl get svc -n argocd kubectl -n argocd get secret argocd-initial-admin-secret -o jsonpath="{.data.password}" | base64 -d kubectl port-forward svc/argocd-server 8080:80 -n argocd git add . git commit -m 'deploy nginx to staging' git push origin main kubectl apply -f k8s/ git add . git commit -m 'fix the bug' git push origin main kubectl get application nginx -o yaml -n argocd git add . git commit -m 'update spec for nginx deployment' git push origin main ``` ```yaml # nginx/deployment.yaml --- apiVersion: apps/v1 kind: Deployment metadata: name: nginx namespace: default spec: replicas: 1 selector: matchLabels: app: nginx template: metadata: labels: app: nginx spec: containers: - name: nginx image: nginx:1.25.0 ports: - containerPort: 80 ``` --- ### Overview (`lessons/180/README.md`) # Go (Golang) vs Python Performance Benchmark (Kubernetes - OpenTelemetry - Prometheus - Grafana) You can find tutorial [here](https://youtu.be/vJsqDqq1R0Y). ## Commands from the Tutorial ```bash kubectl port-forward svc/tempo 4317 -n monitoring kubectl port-forward svc/prometheus-operated 9090 -n monitoring kubectl apply -R -f lessons/180/deployment kubectl apply -f 1-test kubectl apply -f 2-test ``` --- ### Overview (`lessons/181/README.md`) # Kubernetes Autoscaling: HPA vs. VPA vs. Keda vs. CA vs. Karpenter vs. Fargate You can find tutorial [here](https://youtu.be/hsJ2qtwoWZw). --- ### Overview (`lessons/182/README.md`) # Kubernetes Workload Resources: App vs. Container vs. Pod vs. Deployment vs. StatefulSets & More You can find tutorial [here](https://youtu.be/4MEgCP7h8UU). --- ### Overview (`lessons/184/README.md`) # Kubernetes Health Checks: Liveness vs. Readiness vs. Startup Probe You can find tutorial [here](https://youtu.be/fqfieWP1jY4). --- ### Overview (`lessons/185/README.md`) # Kubernetes Architecture You can find tutorial [here](https://youtu.be/F4U1KsTnVyQ). --- ### Overview (`lessons/186/README.md`) # Apache Kafka Architecture You can find tutorial [here](https://youtu.be/IsgRatCefVc). --- ### Overview (`lessons/187/README.md`) # Docker vs Kubernetes vs Containerd You can find tutorial [here](https://youtu.be/fJ5w1YGrDdM). --- ### Overview (`lessons/188/README.md`) # Docker Networking Tutorial (Bridge - None - Host - IPvlan - Macvlan - Overlay) You can find tutorial [here](https://youtu.be/fBRgw5dyBd4). ## Commands from the Tutorial ### Bridge ```bash docker network ls docker network inspect bridge docker run -d --name myapp -p 8081:8080 aputra/myapp-188:v3 docker ps docker inspect myapp curl localhost:8081/api/info docker run -d --name myapp-v2 -p 8082:8080 aputra/myapp-188:v3 docker ps curl localhost:8081/api/info curl localhost:8082/api/info docker exec -it myapp sh curl 172.17.0.2:8080/api/info curl 172.17.0.3:8080/api/info curl myapp-v2:8080/api/info docker rm -f myapp myapp-v2 docker network create my-bridge-net --subnet 10.0.0.0/19 --gateway 10.0.0.1 docker network ls docker network inspect my-bridge-net docker run -d --name myapp -p 8081:8080 --network my-bridge-net aputra/myapp-188:v3 docker run -d --name myapp-v2 -p 8082:8080 --network my-bridge-net aputra/myapp-188:v3 curl localhost:8081/api/info curl localhost:8082/api/info docker exec -it myapp sh curl myapp-v2:8080/api/info docker rm -f myapp myapp-v2 docker network rm my-bridge-net docker compose -f 1-compose.yaml up -d ``` ### Host ```bash docker run -d --name myapp --network host aputra/myapp-188:v3 docker ps curl 192.168.50.55:8080/api/info docker rm -f myapp ``` ### None ```bash docker run -d --name myapp --network none aputra/myapp-188:v3 docker exec myapp ip addr docker run -d --name myapp -p 8081:8080 --network none aputra/myapp-188:v3 docker run -d --name myapp -p 8081:8080 aputra/myapp-188:v3 curl localhost:8081/api/info ``` ## IPvlan ```bash ip addr docker network create -d ipvlan \ --subnet=192.168.50.0/24 \ --gateway=192.168.50.1 \ -o ipvlan_mode=l2 \ -o parent=ens33 my-ipvlan-net docker network ls docker run -d --name myapp --network my-ipvlan-net aputra/myapp-188:v3 docker inspect myapp curl 192.168.50.2:8080/api/info ``` ### Macvlan ```bash docker network create -d macvlan \ --subnet=192.168.50.0/24 \ --gateway=192.168.50.1 \ -o parent=ens33 my-macvlan-net docker network ls docker run -d --name myapp --network my-macvlan-net aputra/myapp-188:v3 docker inspect myapp docker exec -it myapp sh ``` ### Overlay ```bash (host 1) ip addr (host 1) sudo ethtool -K ens33 tx-checksum-ip-generic off (host 2) ip addr (host 2) sudo ethtool -K ens33 tx-checksum-ip-generic off (host 1) docker swarm init (host 2) docker swarm join ... (host 1) docker network create -d overlay --attachable my-overlay-net (host 1) docker network ls (host 1) docker run -d --name myapp --network my-overlay-net aputra/myapp-188:v3 (host 2) docker network ls (host 2) docker run -dit --name myapp-v2 --network my-overlay-net aputra/myapp-188:v3 (host 2) docker network ls (host 2) docker exec -it myapp-v2 sh curl myapp:8080/api/info ``` --- ### Overview (`lessons/189/README.md`) # What Is a Graph Database? You can find tutorial [here](https://youtu.be/-6Xc2_IOh-0). ## Commands from the Tutorial ```bash docker run \ --publish=7474:7474 \ --publish=7687:7687 \ --env=NEO4J_PLUGINS='["graph-data-science"]' \ --env=NEO4J_AUTH=none \ neo4j:5.17.0 ``` ```bash open http://localhost:7474/ ``` ## Create a node ```bash MERGE (a:Location {name: 'A'}) RETURN a ``` ## Get a node ```bash MATCH (a:Location {name: 'A'}) RETURN a ``` ## Create another node ```bash MERGE (a:Location {name: 'B'}) ``` ## Select all nodes and relationships ```bash MATCH (n) RETURN n ``` ## Create a relationship between A and B ```bash MATCH (a:Location {name: 'A'}) MATCH (b:Location {name: 'B'}) MERGE (a)-[:ROAD {cost: 50}]->(b) ``` ## Search these nodes ```bash MATCH (a:Location {name: 'A'})-[:ROAD]-(b:Location {name: 'B'}) RETURN a,b ``` ## Create the rest of the nodes ```bash CREATE (c:Location {name: 'C'}), (d:Location {name: 'D'}), (e:Location {name: 'E'}), (f:Location {name: 'F'}); ``` ## Select all nodes and relationships ```bash MATCH (n) RETURN n ``` ## Create the remaining nodes and relationships ```bash MATCH (a:Location {name: 'A'}) MATCH (b:Location {name: 'B'}) MATCH (c:Location {name: 'C'}) MATCH (d:Location {name: 'D'}) MATCH (e:Location {name: 'E'}) MATCH (f:Location {name: 'F'}) MERGE (a)-[:ROAD {cost: 50}]->(c) MERGE (a)-[:ROAD {cost: 100}]->(d) MERGE (b)-[:ROAD {cost: 40}]->(d) MERGE (c)-[:ROAD {cost: 40}]->(d) MERGE (c)-[:ROAD {cost: 80}]->(e) MERGE (d)-[:ROAD {cost: 30}]->(e) MERGE (d)-[:ROAD {cost: 80}]->(f) MERGE (e)-[:ROAD {cost: 40}]->(f) ``` ## Select all nodes and relationships ```bash MATCH (n) RETURN n ``` ## [Optional] Create nodes and relationships in one statement ```bash CREATE (a:Location {name: 'A'}), (b:Location {name: 'B'}), (c:Location {name: 'C'}), (d:Location {name: 'D'}), (e:Location {name: 'E'}), (f:Location {name: 'F'}), (a)-[:ROAD {cost: 50}]->(b), (a)-[:ROAD {cost: 50}]->(c), (a)-[:ROAD {cost: 100}]->(d), (b)-[:ROAD {cost: 40}]->(d), (c)-[:ROAD {cost: 40}]->(d), (c)-[:ROAD {cost: 80}]->(e), (d)-[:ROAD {cost: 30}]->(e), (d)-[:ROAD {cost: 80}]->(f), (e)-[:ROAD {cost: 40}]->(f); ``` ## Create index ```bash CALL gds.graph.project( 'myGraph', 'Location', 'ROAD', { relationshipProperties: 'cost' } ) ``` ## Calculate the shortest route ```bash MATCH (source:Location {name: 'A'}), (target:Location {name: 'F'}) CALL gds.shortestPath.dijkstra.stream('myGraph', { sourceNode: source, targetNode: target, relationshipWeightProperty: 'cost' }) YIELD index, sourceNode, targetNode, totalCost, nodeIds, costs, path RETURN index, gds.util.asNode(sourceNode).name AS sourceNodeName, gds.util.asNode(targetNode).name AS targetNodeName, totalCost, [nodeId IN nodeIds | gds.util.asNode(nodeId).name] AS nodeNames, costs, nodes(path) as path ORDER BY index ``` --- ### Overview (`lessons/190/README.md`) # What is a Reverse Proxy? (vs. Forward Proxy) | Proxy servers explained You can find tutorial [here](https://youtu.be/b_O501x_F68). --- ### Overview (`lessons/191/README.md`) # Types of Load Balancing Algorithms You can find tutorial [here](https://youtu.be/gqb7LmmXuyw). --- ### Overview (`lessons/192/README.md`) # Kubernetes RBAC Explained You can find tutorial [here](https://youtu.be/iE9Qb8dHqWI). --- ### Overview (`lessons/193/README.md`) # How to debug Kubernetes? (Deployments, Services & Ingress) You can find tutorial [here](https://youtu.be/hixs2GIGrUw). --- ### Overview (`lessons/194/README.md`) # How to debug Kubernetes Ingress? (TLS - Cert-Manager - HTTP-01 & DNS-01 Challenges) You can find tutorial [here](https://youtu.be/DJ2sa49iEKo). --- ### Overview (`lessons/195/README.md`) # AWS EKS Kubernetes Tutorial You can find tutorial [here](https://youtube.com/playlist?list=PLiMWaCMwGJXnKY6XmeifEpjIfkWRo9v2l&si=wc6LIC5V2tD-Tzwl). --- ### Overview (`lessons/196/README.md`) # AWS EKS Kubernetes Tutorial You can find tutorial [here](https://youtube.com/playlist?list=PLiMWaCMwGJXnKY6XmeifEpjIfkWRo9v2l&si=wc6LIC5V2tD-Tzwl). ## Changes from EKS 1.29 to 1.30 - AWS Load Balancer controller now requires `vpcId`. - There is no longer a default storage class in EKS, so you need to explicitly set `storageClassName: gp2`. - Updated all EKS add-ons and Helm chart versions used in this tutorial. ## Clean Up - Manually delete IAM user secret keys from the AWS Console. - `terraform destroy --target helm_release.external_nginx` - `terraform destroy` --- ### Overview (`lessons/197/README.md`) # Kubernetes Tutorial for Beginners You can find tutorial [here](https://www.youtube.com/playlist?list=PLiMWaCMwGJXnHmccp2xlBENZ1xr4FpjXF). --- ### Overview (`lessons/199/README.md`) # Docker Compose Tutorial for Beginners (Networks - Volumes - Secrets - Postgres - Letsencrypt) You can find tutorial [here](https://youtu.be/YMBT1NguJJw). ## Commands ``` python3 -m venv .venv source .venv/bin/activate pip install Flask pip freeze > requirements.txt flask --app app run curl 127.0.0.1:5000/about export APP_VERSION=0.1.0 env | grep APP_VERSION pip install gunicorn docker build -t aputra/flask:0.1.0 . docker run -p 7070:8080 aputra/flask:0.1.0 curl localhost:7070/about docker run -p 7070:8080 -e APP_VERSION=0.1.0 aputra/flask:0.1.0 deactivate docker compose up docker build -t aputra/flask:latest . docker compose up --build APP_TOKEN=secret123 docker compose up curl -X POST localhost:7070/volumes curl -X GET localhost:7070/volumes docker ps docker exec -it 199-postgres-1 psql -h localhost -p 5432 -U myuser -d mydb CREATE TABLE item ( item_id serial PRIMARY KEY, priority varchar(256), task varchar(256) ); CREATE USER myapp WITH PASSWORD 'devops123'; GRANT ALL PRIVILEGES ON DATABASE mydb to myapp; GRANT ALL PRIVILEGES ON ALL TABLES IN SCHEMA public TO myapp; GRANT ALL PRIVILEGES ON ALL SEQUENCES IN SCHEMA public TO myapp; pip install "psycopg[binary,pool]" curl -H "Content-Type: application/json" -d '{"priority":"high","task":"doctor appointment"}' localhost:7070/items curl -H "Content-Type: application/json" -d '{"priority":"low","task":"buy dinner"}' localhost:7070/items SELECT * FROM item; curl localhost:7070/items ``` --- ### Overview (`lessons/200/README.md`) # How to Set Up ArgoCD for Production? (EKS - ECR - Helm - App of Apps - Image Updater - GitOps) You can find tutorial [here](https://youtu.be/_G_RY5trQao). --- ### Overview (`lessons/201/README.md`) # Java (Quarkus) vs. Go (Golang): Performance Benchmark in Kubernetes You can find tutorial [here](https://youtu.be/PL0c-SvjSVg). --- ### Overview (`lessons/202/README.md`) # C# (.NET) vs. Go (Golang): Performance Benchmark in Kubernetes You can find tutorial [here](https://youtu.be/56TUfwejKfo). --- ### Overview (`lessons/203/README.md`) # Rust (Actix) vs. Go (Golang): Performance Benchmark in Kubernetes (EKS) You can find tutorial [here](https://youtu.be/2hWfLiRGaNM). --- ### Overview (`lessons/204/README.md`) # Go (fiber) vs. Go (stdlib) vs. Go (gin): Performance Benchmark in Kubernetes You can find tutorial [here](https://youtu.be/ok5DDDNsOaQ). --- ### Overview (`lessons/205/README.md`) # Rust (Actix) vs. Go (Stdlib): Performance Benchmark in Kubernetes You can find tutorial [here](https://youtu.be/ZfvpUDGGr24). --- ### Overview (`lessons/206/README.md`) # Rust (Actix) vs Rust (Axum) vs Rust (Rocket): Performance Benchmark in Kubernetes You can find tutorial [here](https://youtu.be/KA_w_jOGils). --- ### Overview (`lessons/207/README.md`) # Zap (Zig) vs Actix (Rust) vs http.zig (Zig): Performance Benchmark in Kubernetes You can find tutorial [here](https://youtu.be/VxW0ijXAfOs). --- ### Overview (`lessons/208/README.md`) # Actix (Rust) vs Zap (Zig) vs Stdlib (Zig): Performance Benchmark in Kubernetes You can find tutorial [here](https://youtu.be/SR2LRhnL1AQ). --- ### Overview (`lessons/209/README.md`) # Django (Python) vs Stdlib (Go): Performance Benchmark in Kubernetes You can find tutorial [here](https://youtu.be/caNsPpRuBcw). --- ### Overview (`lessons/210/README.md`) # Node.js vs Golang: Performance Benchmark in Kubernetes You can find tutorial [here](https://youtu.be/ZslbMp_T90k). --- ### Overview (`lessons/211/README.md`) # Node.js vs Golang (Round 2): Performance Benchmark in Kubernetes You can find tutorial [here](https://youtu.be/h2pCxj_Fkdc). --- ### Overview (`lessons/212/README.md`) # Bun vs Node.js: Performance Benchmark in Kubernetes You can find tutorial [here](https://youtu.be/dPO4v5q9ULU). --- ### Overview (`lessons/213/README.md`) # Go (Golang) vs. Bun: Performance (Latency - Throughput - Saturation - Availability) You can find tutorial [here](https://youtu.be/ECnlX00YcPI). --- ### Overview (`lessons/214/README.md`) # MySQL vs PostgreSQL Performance Benchmark (Latency - Throughput - Saturation) You can find tutorial [here](https://youtu.be/R7jBtnrUmYI). --- ### Overview (`lessons/215/README.md`) # Rust vs Zig vs Go: Performance (Latency - Throughput - Saturation - Availability) You can find tutorial [here](https://youtu.be/3fWx5BOiUiY). --- ### Overview (`lessons/216/README.md`) # MySQL vs PostgreSQL (Round 2) Performance Benchmark (Latency - Throughput - Saturation) You can find tutorial [here](https://youtu.be/17BqoNEQKTM). ## Hardware Details | EC2 type | i3en.large | | -------------- | ----------------- | | vCPU | 2 | | Memory (GiB) | 16 | | Storage (GB) | 1 x 1250 NVMe SSD | | Network (Gbps) | Up to 25 | I'll use an i3en.large in the following test and may scale it up to an **i3en.xlarge** or **i3en.2xlarge** in future tests. ## Database Configuration Credit to @melroy89 and @nscuro for the initial database configuration [settings](https://github.com/antonputra/tutorials/issues/307). > To tune PostgreSQL, I used [Pgtune](https://pgtune.leopard.in.ua/), and for MySQL, I used [MySQLTuner-perl](https://github.com/major/MySQLTuner-perl). - PostgreSQL (17.0) [config](postgresql.conf) - MySQL (9.0) [config](50-server.cnf) ## Database Schema ``` /* Detailed source-code truncated for AI context efficiency. */ ``` ## Test Design The goal is to test CRUD operations, not to implement actual shopping cart functionality. Also, I'm not using transactions to be able to measure the performance of each query from the client side. ```sql -- Create shopping cart INSERT INTO cart(customer_id, total) VALUES (4, 0); -- Add body wash to the cart INSERT INTO cart_item(cart_id, product_id, quantity) VALUES (1, 7, 1); -- Update value of the cart UPDATE cart SET total = 10.10 WHERE customer_id = 4; -- Add 2 shampoos the cart INSERT INTO cart_item(cart_id, product_id, quantity) VALUES (1, 1, 2); -- Update value of the cart UPDATE cart SET total = 25.90 WHERE customer_id = 4; -- Create an order INSERT INTO "order"(customer_id, total) VALUES (4, 25.90); -- Add body wash to the order INSERT INTO order_item(order_id, product_id, quantity) VALUES (1, 7, 1); -- Reduce the stock quantity of the body wash UPDATE product SET stock_quantity = 99 WHERE id = 7; -- Add 2 shampoos the order INSERT INTO order_item(order_id, product_id, quantity) VALUES (1, 1, 2); -- Reduce the stock quantity of the shampoo UPDATE product SET stock_quantity = 98 WHERE id = 1; -- Delete shopping cart and items DELETE FROM cart_item WHERE cart_id = 1; DELETE FROM cart WHERE id = 1; ``` ## 2nd Test ```sql SELECT customer.username, customer.first_name, customer.last_name, customer.address, product.name, order_item.quantity, `order`.total FROM `order` LEFT JOIN customer ON customer.id = `order`.customer_id LEFT JOIN order_item ON order_item.order_id = `order`.id LEFT JOIN product ON product.id = order_item.product_id WHERE `order`.id = 431; SELECT customer.username, customer.first_name, customer.last_name, customer.address, product.name, order_item.quantity, `order`.total FROM `order` LEFT JOIN customer ON customer.id = `order`.customer_id LEFT JOIN order_item ON order_item.order_id = `order`.id LEFT JOIN product ON product.id = order_item.product_id WHERE `order`.id = $1; ``` --- ### Overview (`lessons/217/README.md`) # Deno vs. Node.js vs Bun: Performance (Latency - Throughput - Saturation - Availability) You can find tutorial [here](https://youtu.be/yJmyYosyDDM). --- ### Overview (`lessons/218/README.md`) # Apache Kafka vs RabbitMQ Performance (Latency - Throughput - Saturation) You can find tutorial [here](https://youtu.be/UPkOsXKG4ns). --- ### Overview (`lessons/219/README.md`) # Nginx vs Apache Performance (Latency - Throughput - Saturation - Availability | HTTP/2 - TLS - Gzip) You can find tutorial [here](https://youtu.be/UGp4LmocE7o). ## Apache ``` /* Detailed source-code truncated for AI context efficiency. */ ``` --- ### Overview (`lessons/220/README.md`) # Ruby on Rails vs Node.js Performance (Latency - Throughput - Saturation - Availability) You can find tutorial [here](https://youtu.be/Qp9SOOtgmS4). --- ### Overview (`lessons/221/README.md`) # Nginx vs Caddy Performance (Latency - Throughput - Saturation - Availability | HTTP/2 - TLS - Gzip) You can find tutorial [here](https://youtu.be/N5PAU-vYrN8). --- ### Overview (`lessons/222/README.md`) # Ruby vs Node.js: Performance Benchmark You can find tutorial [here](https://youtu.be/UbDRSZ3IK2Y). --- ### Overview (`lessons/223/README.md`) # SQLite vs PostgreSQL You can find tutorial [here](https://youtu.be/VzQgr-TgBzc). ## Database Schema ``` /* Detailed source-code truncated for AI context efficiency. */ ``` ## Test Design The goal is to test CRUD operations, not to implement actual shopping cart functionality. Also, I'm not using transactions to be able to measure the performance of each query from the client side. ```sql -- Create shopping cart INSERT INTO cart(customer_id, total) VALUES (4, 0); -- Add body wash to the cart INSERT INTO cart_item(cart_id, product_id, quantity) VALUES (1, 7, 1); -- Update value of the cart UPDATE cart SET total = 10.10 WHERE customer_id = 4; -- Add 2 shampoos the cart INSERT INTO cart_item(cart_id, product_id, quantity) VALUES (1, 1, 2); -- Update value of the cart UPDATE cart SET total = 25.90 WHERE customer_id = 4; -- Create an order INSERT INTO "order"(customer_id, total) VALUES (4, 25.90); -- Add body wash to the order INSERT INTO order_item(order_id, product_id, quantity) VALUES (1, 7, 1); -- Reduce the stock quantity of the body wash UPDATE product SET stock_quantity = 99 WHERE id = 7; -- Add 2 shampoos the order INSERT INTO order_item(order_id, product_id, quantity) VALUES (1, 1, 2); -- Reduce the stock quantity of the shampoo UPDATE product SET stock_quantity = 98 WHERE id = 1; -- Delete shopping cart and items DELETE FROM cart_item WHERE cart_id = 1; DELETE FROM cart WHERE id = 1; ``` ## 2nd Test ```sql SELECT customer.username, customer.first_name, customer.last_name, customer.address, product.name, order_item.quantity, `order`.total FROM `order` LEFT JOIN customer ON customer.id = `order`.customer_id LEFT JOIN order_item ON order_item.order_id = `order`.id LEFT JOIN product ON product.id = order_item.product_id WHERE `order`.id = 431; ``` --- ### Overview (`lessons/224/README.md`) # Nginx vs Traefik: What Is the BEST Reverse Proxy? You can find tutorial [here](https://youtu.be/42RNqGdpELE). --- ### Overview (`lessons/225/README.md`) # Redis vs Memcached Performance Benchmark You can find tutorial [here](https://youtu.be/hPRqxKVnpw8). --- ### Overview (`lessons/226/README.md`) # Pingora vs Nginx Performance Benchmark You can find tutorial [here](https://youtu.be/iz95STKZMqY). --- ### Overview (`lessons/227/README.md`) # MongoDB vs. PostgreSQL: Performance & Functionality You can find tutorial [here](https://youtu.be/ZZ2tx8iL3P4). ## SQL ```sql CREATE TABLE product ( id SERIAL PRIMARY KEY, jdoc jsonb ); CREATE INDEX idx__product__price ON product using BTREE(((jdoc -> 'price')::NUMERIC)); INSERT INTO product(jdoc) VALUES ('{"name": "Shampoo", "price": 7.90, "stock": 100}'); INSERT INTO product(jdoc) VALUES ('{"name": "Hairspray", "price": 11.50, "stock": 100}'); UPDATE product SET jdoc = jsonb_set(jdoc, '{stock}', '98') WHERE id = 2; SELECT id, jdoc->'price' as price, jdoc->'stock' as stock FROM product WHERE (jdoc -> 'price')::numeric < 10; DELETE FROM product WHERE id = 1; CREATE VIEW sales AS SELECT "order".product_id, "order".quantity, inventory.price FROM "order" LEFT OUTER JOIN inventory ON "order".product_id = inventory.product_id; SELECT product_id, SUM(quantity * price) AS amount_sold FROM sales GROUP BY product_id; ``` ```json db.product.insertOne({name: "Shampoo", price: 7.90, stock: 100}) db.product.insertOne({name: "Hairspray", price: 11.50, stock: 100}) db.product.updateOne({ _id: ObjectId("674705957b0ee5f68236d2b1") }, { $set: { 'stock': 98 } }) db.product.find({ price: { $lt: 10 } }) db.product.deleteOne( {"_id": ObjectId("674705957b0ee5f68236d2b1")}) db.createView( "sales", "order", [ { $lookup: { from: "inventory", localField: "product_id", foreignField: "product_id", as: "inventory_docs" } }, { $project: { _id: 0, product_id: 1, quantity: 1, price: "$inventory_docs.price" } }, { $unwind: "$price" } ] ); db.sales.aggregate( [ { $group: { _id: "$product_id", amount_sold: { $sum: { $multiply: [ "$price", "$quantity" ] } } } } ] ); ``` --- ### Overview (`lessons/228/README.md`) # Redis vs Dragonfly Performance (Latency - Throughput - Saturation) You can find tutorial [here](https://youtu.be/DgcBFb4L0dI). ## Install Dragonfly ```bash export VERSION="1.25.4" wget https://github.com/dragonflydb/dragonfly/releases/download/v${VERSION}/dragonfly_amd64.deb sudo dpkg -i dragonfly_amd64.deb sudo tee /etc/dragonfly/dragonfly.conf < 16/32 /usr/local/bin/redis-server --io-threads 9 --io-threads-do-reads yes --save --protected-mode no /usr/local/bin/valkey-server --io-threads 9 --io-threads-do-reads yes --save --protected-mode no ``` ## Redis ```bash sudo apt -y install build-essential pkg-config export REDIS_VER="8.6.0" wget https://github.com/redis/redis/archive/refs/tags/${REDIS_VER}.tar.gz tar -xzf ${REDIS_VER}.tar.gz sudo mv redis-${REDIS_VER}/ /opt/redis cd /opt/redis/ make sudo make install cat <

[circleci-image]: https://img.shields.io/circleci/build/github/nestjs/nest/master?token=abc123def456 [circleci-url]: https://circleci.com/gh/nestjs/nest

A progressive Node.js framework for building efficient and scalable server-side applications.

## Description [Nest](https://github.com/nestjs/nest) framework TypeScript starter repository. ## Installation ```bash $ npm install ``` ## Running the app ```bash # development $ npm run start # watch mode $ npm run start:dev # production mode $ npm run start:prod ``` ## Test ```bash # unit tests $ npm run test # e2e tests $ npm run test:e2e # test coverage $ npm run test:cov ``` ## Support Nest is an MIT-licensed open source project. It can grow thanks to the sponsors and support by the amazing backers. If you'd like to join them, please [read more here](https://docs.nestjs.com/support). ## Stay in touch - Author - [Kamil Myśliwiec](https://kamilmysliwiec.com) - Website - [https://nestjs.com](https://nestjs.com/) - Twitter - [@nestframework](https://twitter.com/nestframework) ## License Nest is [MIT licensed](LICENSE). --- ### Overview (`lessons/051/app123/README.md`) # Getting Started with Create React App This project was bootstrapped with [Create React App](https://github.com/facebook/create-react-app). ## Available Scripts In the project directory, you can run: ### `npm start` Runs the app in the development mode.\ Open [http://localhost:3000](http://localhost:3000) to view it in the browser. The page will reload if you make edits.\ You will also see any lint errors in the console. ### `npm test` Launches the test runner in the interactive watch mode.\ See the section about [running tests](https://facebook.github.io/create-react-app/docs/running-tests) for more information. ### `npm run build` Builds the app for production to the `build` folder.\ It correctly bundles React in production mode and optimizes the build for the best performance. The build is minified and the filenames include the hashes.\ Your app is ready to be deployed! See the section about [deployment](https://facebook.github.io/create-react-app/docs/deployment) for more information. ### `npm run eject` **Note: this is a one-way operation. Once you `eject`, you can’t go back!** If you aren’t satisfied with the build tool and configuration choices, you can `eject` at any time. This command will remove the single build dependency from your project. Instead, it will copy all the configuration files and the transitive dependencies (webpack, Babel, ESLint, etc) right into your project so you have full control over them. All of the commands except `eject` will still work, but they will point to the copied scripts so you can tweak them. At this point you’re on your own. You don’t have to ever use `eject`. The curated feature set is suitable for small and middle deployments, and you shouldn’t feel obligated to use this feature. However we understand that this tool wouldn’t be useful if you couldn’t customize it when you are ready for it. ## Learn More You can learn more in the [Create React App documentation](https://facebook.github.io/create-react-app/docs/getting-started). To learn React, check out the [React documentation](https://reactjs.org/). ### Code Splitting This section has moved here: [https://facebook.github.io/create-react-app/docs/code-splitting](https://facebook.github.io/create-react-app/docs/code-splitting) ### Analyzing the Bundle Size This section has moved here: [https://facebook.github.io/create-react-app/docs/analyzing-the-bundle-size](https://facebook.github.io/create-react-app/docs/analyzing-the-bundle-size) ### Making a Progressive Web App This section has moved here: [https://facebook.github.io/create-react-app/docs/making-a-progressive-web-app](https://facebook.github.io/create-react-app/docs/making-a-progressive-web-app) ### Advanced Configuration This section has moved here: [https://facebook.github.io/create-react-app/docs/advanced-configuration](https://facebook.github.io/create-react-app/docs/advanced-configuration) ### Deployment This section has moved here: [https://facebook.github.io/create-react-app/docs/deployment](https://facebook.github.io/create-react-app/docs/deployment) ### `npm run build` fails to minify This section has moved here: [https://facebook.github.io/create-react-app/docs/troubleshooting#npm-run-build-fails-to-minify](https://facebook.github.io/create-react-app/docs/troubleshooting#npm-run-build-fails-to-minify) --- ### Overview (`lessons/052/app123/README.md`) # Getting Started with Create React App This project was bootstrapped with [Create React App](https://github.com/facebook/create-react-app). ## Available Scripts In the project directory, you can run: ### `npm start` Runs the app in the development mode.\ Open [http://localhost:3000](http://localhost:3000) to view it in the browser. The page will reload if you make edits.\ You will also see any lint errors in the console. ### `npm test` Launches the test runner in the interactive watch mode.\ See the section about [running tests](https://facebook.github.io/create-react-app/docs/running-tests) for more information. ### `npm run build` Builds the app for production to the `build` folder.\ It correctly bundles React in production mode and optimizes the build for the best performance. The build is minified and the filenames include the hashes.\ Your app is ready to be deployed! See the section about [deployment](https://facebook.github.io/create-react-app/docs/deployment) for more information. ### `npm run eject` **Note: this is a one-way operation. Once you `eject`, you can’t go back!** If you aren’t satisfied with the build tool and configuration choices, you can `eject` at any time. This command will remove the single build dependency from your project. Instead, it will copy all the configuration files and the transitive dependencies (webpack, Babel, ESLint, etc) right into your project so you have full control over them. All of the commands except `eject` will still work, but they will point to the copied scripts so you can tweak them. At this point you’re on your own. You don’t have to ever use `eject`. The curated feature set is suitable for small and middle deployments, and you shouldn’t feel obligated to use this feature. However we understand that this tool wouldn’t be useful if you couldn’t customize it when you are ready for it. ## Learn More You can learn more in the [Create React App documentation](https://facebook.github.io/create-react-app/docs/getting-started). To learn React, check out the [React documentation](https://reactjs.org/). ### Code Splitting This section has moved here: [https://facebook.github.io/create-react-app/docs/code-splitting](https://facebook.github.io/create-react-app/docs/code-splitting) ### Analyzing the Bundle Size This section has moved here: [https://facebook.github.io/create-react-app/docs/analyzing-the-bundle-size](https://facebook.github.io/create-react-app/docs/analyzing-the-bundle-size) ### Making a Progressive Web App This section has moved here: [https://facebook.github.io/create-react-app/docs/making-a-progressive-web-app](https://facebook.github.io/create-react-app/docs/making-a-progressive-web-app) ### Advanced Configuration This section has moved here: [https://facebook.github.io/create-react-app/docs/advanced-configuration](https://facebook.github.io/create-react-app/docs/advanced-configuration) ### Deployment This section has moved here: [https://facebook.github.io/create-react-app/docs/deployment](https://facebook.github.io/create-react-app/docs/deployment) ### `npm run build` fails to minify This section has moved here: [https://facebook.github.io/create-react-app/docs/troubleshooting#npm-run-build-fails-to-minify](https://facebook.github.io/create-react-app/docs/troubleshooting#npm-run-build-fails-to-minify) --- ### Overview (`lessons/110/www.devopsbyexample.io/README.md`) # Getting Started with Create React App This project was bootstrapped with [Create React App](https://github.com/facebook/create-react-app). ## Available Scripts In the project directory, you can run: ### `npm start` Runs the app in the development mode.\ Open [http://localhost:3000](http://localhost:3000) to view it in your browser. The page will reload when you make changes.\ You may also see any lint errors in the console. ### `npm test` Launches the test runner in the interactive watch mode.\ See the section about [running tests](https://facebook.github.io/create-react-app/docs/running-tests) for more information. ### `npm run build` Builds the app for production to the `build` folder.\ It correctly bundles React in production mode and optimizes the build for the best performance. The build is minified and the filenames include the hashes.\ Your app is ready to be deployed! See the section about [deployment](https://facebook.github.io/create-react-app/docs/deployment) for more information. ### `npm run eject` **Note: this is a one-way operation. Once you `eject`, you can't go back!** If you aren't satisfied with the build tool and configuration choices, you can `eject` at any time. This command will remove the single build dependency from your project. Instead, it will copy all the configuration files and the transitive dependencies (webpack, Babel, ESLint, etc) right into your project so you have full control over them. All of the commands except `eject` will still work, but they will point to the copied scripts so you can tweak them. At this point you're on your own. You don't have to ever use `eject`. The curated feature set is suitable for small and middle deployments, and you shouldn't feel obligated to use this feature. However we understand that this tool wouldn't be useful if you couldn't customize it when you are ready for it. ## Learn More You can learn more in the [Create React App documentation](https://facebook.github.io/create-react-app/docs/getting-started). To learn React, check out the [React documentation](https://reactjs.org/). ### Code Splitting This section has moved here: [https://facebook.github.io/create-react-app/docs/code-splitting](https://facebook.github.io/create-react-app/docs/code-splitting) ### Analyzing the Bundle Size This section has moved here: [https://facebook.github.io/create-react-app/docs/analyzing-the-bundle-size](https://facebook.github.io/create-react-app/docs/analyzing-the-bundle-size) ### Making a Progressive Web App This section has moved here: [https://facebook.github.io/create-react-app/docs/making-a-progressive-web-app](https://facebook.github.io/create-react-app/docs/making-a-progressive-web-app) ### Advanced Configuration This section has moved here: [https://facebook.github.io/create-react-app/docs/advanced-configuration](https://facebook.github.io/create-react-app/docs/advanced-configuration) ### Deployment This section has moved here: [https://facebook.github.io/create-react-app/docs/deployment](https://facebook.github.io/create-react-app/docs/deployment) ### `npm run build` fails to minify This section has moved here: [https://facebook.github.io/create-react-app/docs/troubleshooting#npm-run-build-fails-to-minify](https://facebook.github.io/create-react-app/docs/troubleshooting#npm-run-build-fails-to-minify) --- ### Overview (`lessons/146/demo-certs/README.md`) ### Generate Certificate Authority ```bash cfssl gencert -initca ca-csr.json | cfssljson -bare ca ``` ### Generate Certificate for the client ```bash cfssl gencert -ca=ca.pem -ca-key=ca-key.pem -config=ca-config.json -profile=demo api-nginx-antonputra-pvt-csr.json | cfssljson -bare api-nginx-antonputra-pvt ``` --- ### Overview (`lessons/147/demo-certs/README.md`) ### Generate Certificate Authority ```bash cfssl gencert -initca ca-csr.json | cfssljson -bare ca ``` ### Generate Certificate for the client ```bash cfssl gencert -ca=ca.pem -ca-key=ca-key.pem -config=ca-config.json -profile=demo api-rust-antonputra-pvt-csr.json | cfssljson -bare api-rust-antonputra-pvt ``` --- ### Overview (`lessons/149/demo-certs/README.md`) ### Generate Certificate Authority ```bash cfssl gencert -initca ca-csr.json | cfssljson -bare ca ``` ### Generate Certificate for the client ```bash cfssl gencert -ca=ca.pem -ca-key=ca-key.pem -config=ca-config.json -profile=demo rest-antonputra-pvt-csr.json | cfssljson -bare rest-antonputra-pvt cfssl gencert -ca=ca.pem -ca-key=ca-key.pem -config=ca-config.json -profile=demo grpc-antonputra-pvt-csr.json | cfssljson -bare grpc-antonputra-pvt ``` --- ### Overview (`lessons/151/demo-certs/README.md`) ### Generate Certificate Authority ```bash cfssl gencert -initca ca-csr.json | cfssljson -bare ca ``` ### Generate Certificate for the client ```bash cfssl gencert -ca=ca.pem -ca-key=ca-key.pem -config=ca-config.json -profile=demo api-envoy-antonputra-pvt-csr.json | cfssljson -bare api-envoy-antonputra-pvt cfssl gencert -ca=ca.pem -ca-key=ca-key.pem -config=ca-config.json -profile=demo api-nginx-antonputra-pvt-csr.json | cfssljson -bare api-nginx-antonputra-pvt ``` --- ### Overview (`lessons/153/demo-certs/README.md`) ### Generate Certificate Authority ```bash cfssl gencert -initca ca-csr.json | cfssljson -bare ca ``` ### Generate Certificate for the client ```bash cfssl gencert -ca=ca.pem -ca-key=ca-key.pem -config=ca-config.json -profile=demo caddy-antonputra-pvt-csr.json | cfssljson -bare caddy-antonputra-pvt cfssl gencert -ca=ca.pem -ca-key=ca-key.pem -config=ca-config.json -profile=demo traefik-antonputra-pvt-csr.json | cfssljson -bare traefik-antonputra-pvt ``` --- ### Overview (`lessons/182/0-app/README.md`) ## How to run the application locally? ### Navigate to the '0-app' directory ```bash cd tutorials/lessons/182/0-app ``` ### Download the application's dependencies ```bash go mod tidy ``` ### Run the application ```bash go run . ``` ### Access the application from a separate terminal window ```bash curl -i localhost:8080/api/devices ``` ### Get the current status of the application. ```bash curl -i localhost:8080/status ``` --- ### Overview (`lessons/182/1-container/README.md`) ## How to build container image? ### Navigate to the '1-container' directory ```bash cd tutorials/lessons/182/1-container ``` ### Build container image ```bash docker build -t /myapp:0.1.0 . ``` ### Log in to Docker Hub to upload the image ```bash docker login --username ``` ### Push the container image to the remote repository ```bash docker push /myapp:0.1.0 ``` --- ### Overview (`lessons/182/2-pod/README.md`) ## How to create a pod in Kubernetes? ### Set up a local Kubernetes cluster using Minikube ```bash minikube start ``` ### Deploy a standalone pod in Kubernetes ```bash kubectl apply -f 2-pod/0-pod.yaml ``` ### Get the list of pods in the default namespace ```bash kubectl get pods ``` ### Tail the logs from the pod ```bash kubectl logs -f myapp ``` --- ### Overview (`lessons/191/3-dns-round-robin/README.md`) # Instructions ## Create DNS A record host: api type: A ttl: as small as posible values: - 10.0.0.1 - 10.0.0.2 - 10.0.0.3 dig api.devopsbyexample.com --- ### Overview (`lessons/193/myapp/README.md`) # myapp This project uses Quarkus, the Supersonic Subatomic Java Framework. If you want to learn more about Quarkus, please visit its website: https://quarkus.io/ . ## Running the application in dev mode You can run your application in dev mode that enables live coding using: ```shell script ./mvnw compile quarkus:dev ``` > **_NOTE:_** Quarkus now ships with a Dev UI, which is available in dev mode only at http://localhost:8080/q/dev/. ## Packaging and running the application The application can be packaged using: ```shell script ./mvnw package ``` It produces the `quarkus-run.jar` file in the `target/quarkus-app/` directory. Be aware that it’s not an _über-jar_ as the dependencies are copied into the `target/quarkus-app/lib/` directory. The application is now runnable using `java -jar target/quarkus-app/quarkus-run.jar`. If you want to build an _über-jar_, execute the following command: ```shell script ./mvnw package -Dquarkus.package.type=uber-jar ``` The application, packaged as an _über-jar_, is now runnable using `java -jar target/*-runner.jar`. ## Creating a native executable You can create a native executable using: ```shell script ./mvnw package -Dnative ``` Or, if you don't have GraalVM installed, you can run the native executable build in a container using: ```shell script ./mvnw package -Dnative -Dquarkus.native.container-build=true ``` You can then execute your native executable with: `./target/myapp-0.1.0-runner` If you want to learn more about building native executables, please consult https://quarkus.io/guides/maven-tooling. ## Provided Code ### REST Easily start your REST Web Services [Related guide section...](https://quarkus.io/guides/getting-started-reactive#reactive-jax-rs-resources) --- ### Overview (`lessons/200/java-app/README.md`) # java-app This project uses Quarkus, the Supersonic Subatomic Java Framework. If you want to learn more about Quarkus, please visit its website: . ## Running the application in dev mode You can run your application in dev mode that enables live coding using: ```shell script ./mvnw compile quarkus:dev ``` > **_NOTE:_** Quarkus now ships with a Dev UI, which is available in dev mode only at . ## Packaging and running the application The application can be packaged using: ```shell script ./mvnw package ``` It produces the `quarkus-run.jar` file in the `target/quarkus-app/` directory. Be aware that it’s not an _über-jar_ as the dependencies are copied into the `target/quarkus-app/lib/` directory. The application is now runnable using `java -jar target/quarkus-app/quarkus-run.jar`. If you want to build an _über-jar_, execute the following command: ```shell script ./mvnw package -Dquarkus.package.jar.type=uber-jar ``` The application, packaged as an _über-jar_, is now runnable using `java -jar target/*-runner.jar`. ## Creating a native executable You can create a native executable using: ```shell script ./mvnw package -Dnative ``` Or, if you don't have GraalVM installed, you can run the native executable build in a container using: ```shell script ./mvnw package -Dnative -Dquarkus.native.container-build=true ``` You can then execute your native executable with: `./target/java-app-0.1.0-runner` If you want to learn more about building native executables, please consult . ## Provided Code ### REST Easily start your REST Web Services [Related guide section...](https://quarkus.io/guides/getting-started-reactive#reactive-jax-rs-resources) --- ### Overview (`lessons/201/java-app-reactive/README.md`) # java-app This project uses Quarkus, the Supersonic Subatomic Java Framework. If you want to learn more about Quarkus, please visit its website: . ## Running the application in dev mode You can run your application in dev mode that enables live coding using: ```shell script ./mvnw compile quarkus:dev ``` > **_NOTE:_** Quarkus now ships with a Dev UI, which is available in dev mode only at . ## Packaging and running the application The application can be packaged using: ```shell script ./mvnw package ``` It produces the `quarkus-run.jar` file in the `target/quarkus-app/` directory. Be aware that it’s not an _über-jar_ as the dependencies are copied into the `target/quarkus-app/lib/` directory. The application is now runnable using `java -jar target/quarkus-app/quarkus-run.jar`. If you want to build an _über-jar_, execute the following command: ```shell script ./mvnw package -Dquarkus.package.jar.type=uber-jar ``` The application, packaged as an _über-jar_, is now runnable using `java -jar target/*-runner.jar`. ## Creating a native executable You can create a native executable using: ```shell script ./mvnw package -Dnative ``` Or, if you don't have GraalVM installed, you can run the native executable build in a container using: ```shell script ./mvnw package -Dnative -Dquarkus.native.container-build=true ``` You can then execute your native executable with: `./target/java-app-0.1.0-runner` If you want to learn more about building native executables, please consult . ## Provided Code ### REST Easily start your REST Web Services [Related guide section...](https://quarkus.io/guides/getting-started-reactive#reactive-jax-rs-resources) --- ### Overview (`lessons/201/java-app/README.md`) # java-app This project uses Quarkus, the Supersonic Subatomic Java Framework. If you want to learn more about Quarkus, please visit its website: . ## Running the application in dev mode You can run your application in dev mode that enables live coding using: ```shell script ./mvnw compile quarkus:dev ``` > **_NOTE:_** Quarkus now ships with a Dev UI, which is available in dev mode only at . ## Packaging and running the application The application can be packaged using: ```shell script ./mvnw package ``` It produces the `quarkus-run.jar` file in the `target/quarkus-app/` directory. Be aware that it’s not an _über-jar_ as the dependencies are copied into the `target/quarkus-app/lib/` directory. The application is now runnable using `java -jar target/quarkus-app/quarkus-run.jar`. If you want to build an _über-jar_, execute the following command: ```shell script ./mvnw package -Dquarkus.package.jar.type=uber-jar ``` The application, packaged as an _über-jar_, is now runnable using `java -jar target/*-runner.jar`. ## Creating a native executable You can create a native executable using: ```shell script ./mvnw package -Dnative ``` Or, if you don't have GraalVM installed, you can run the native executable build in a container using: ```shell script ./mvnw package -Dnative -Dquarkus.native.container-build=true ``` You can then execute your native executable with: `./target/java-app-0.1.0-runner` If you want to learn more about building native executables, please consult . ## Provided Code ### REST Easily start your REST Web Services [Related guide section...](https://quarkus.io/guides/getting-started-reactive#reactive-jax-rs-resources) --- ### Overview (`lessons/207/http.zig/README.md`) # Pure Zig port, using http.zig Note that there are some very app specific tweaks in the server config here, to tailor this server to the intended environment. This doesnt do much for performance, but its mainly to get the memory usage down ## Comparison with Rust On a Mac M2, im seeing these figures on local testing with - `wrk -c 100 -d 30 -t 4 http://localhost:8080/healthz` - `wrk -c 100 -d 30 -t 4 http://localhost:8080/api/devices` There is really no difference between the healthz endpoint and the api/devices endpoint that needs to JSONify a struct ... so really all the time is being spent doing IO ----- Rust in --release mode ``` 2.5 MB Binary 6 MB Memory 4 Threads Running 30s test @ http://localhost:8086/api/devices 4 threads and 100 connections Thread Stats Avg Stdev Max +/- Stdev Latency 440.79us 167.31us 4.09ms 83.65% Req/Sec 46.01k 2.53k 51.07k 79.40% 5511737 requests in 30.10s, 856.79MB read Requests/sec: 183100.15 Transfer/sec: 28.46MB ``` HTTP.zig in ReleaseSmall mode ``` 141kb Binary 2 MB Memory 4 Threads Running 30s test @ http://localhost:8086/api/devices 4 threads and 100 connections Thread Stats Avg Stdev Max +/- Stdev Latency 507.73us 184.11us 4.19ms 85.25% Req/Sec 48.57k 10.33k 57.82k 80.48% 5819106 requests in 30.10s, 688.14MB read Requests/sec: 193313.51 Transfer/sec: 22.86MB ``` HTTP.zig in ReleaseFast mode ``` Running 30s test @ http://localhost:8086/api/devices 4 threads and 100 connections Thread Stats Avg Stdev Max +/- Stdev Latency 492.28us 185.23us 6.05ms 83.81% Req/Sec 50.01k 11.59k 60.59k 79.05% 5985889 requests in 30.10s, 707.86MB read Requests/sec: 198855.02 Transfer/sec: 23.52MB ``` ---- Running both with Debug compiles, similar story ``` Actix compiled in Debug mode Thread Stats Avg Stdev Max +/- Stdev Latency 1.50ms 230.25us 5.05ms 86.76% Req/Sec 16.73k 2.05k 67.69k 83.85% 1999217 requests in 30.10s, 310.78MB read Requests/sec: 66409.49 Transfer/sec: 10.32MB ``` ``` Zig compiled in Debug mode Thread Stats Avg Stdev Max +/- Stdev Latency 0.88ms 281.05us 20.30ms 91.38% Req/Sec 28.58k 2.83k 59.14k 87.60% 3418569 requests in 30.10s, 404.26MB read Requests/sec: 113569.44 Transfer/sec: 13.43MB ``` TL;DR - much of a muchness for runtime performance ! There really isnt anything between them Actix generally has better latency, and less scatter (std deviation) Zig has a better peak throughput, and uses 1 third the memory (at least with these config tweaks) ----- The main difference (to me) is that the Actix build is pulling in a huge number of dependencies :( The zig code is only pulling in 1 dependency --- ### Overview (`lessons/208/http.zig/README.md`) # Pure Zig port, using http.zig Note that there are some very app specific tweaks in the server config here, to tailor this server to the intended environment. This doesnt do much for performance, but its mainly to get the memory usage down ## Comparison with Rust On a Mac M2, im seeing these figures on local testing with - `wrk -c 100 -d 30 -t 4 http://localhost:8080/healthz` - `wrk -c 100 -d 30 -t 4 http://localhost:8080/api/devices` There is really no difference between the healthz endpoint and the api/devices endpoint that needs to JSONify a struct ... so really all the time is being spent doing IO ----- Rust in --release mode ``` 2.5 MB Binary 6 MB Memory 4 Threads Running 30s test @ http://localhost:8086/api/devices 4 threads and 100 connections Thread Stats Avg Stdev Max +/- Stdev Latency 440.79us 167.31us 4.09ms 83.65% Req/Sec 46.01k 2.53k 51.07k 79.40% 5511737 requests in 30.10s, 856.79MB read Requests/sec: 183100.15 Transfer/sec: 28.46MB ``` HTTP.zig in ReleaseSmall mode ``` 141kb Binary 2 MB Memory 4 Threads Running 30s test @ http://localhost:8086/api/devices 4 threads and 100 connections Thread Stats Avg Stdev Max +/- Stdev Latency 507.73us 184.11us 4.19ms 85.25% Req/Sec 48.57k 10.33k 57.82k 80.48% 5819106 requests in 30.10s, 688.14MB read Requests/sec: 193313.51 Transfer/sec: 22.86MB ``` HTTP.zig in ReleaseFast mode ``` Running 30s test @ http://localhost:8086/api/devices 4 threads and 100 connections Thread Stats Avg Stdev Max +/- Stdev Latency 492.28us 185.23us 6.05ms 83.81% Req/Sec 50.01k 11.59k 60.59k 79.05% 5985889 requests in 30.10s, 707.86MB read Requests/sec: 198855.02 Transfer/sec: 23.52MB ``` ---- Running both with Debug compiles, similar story ``` Actix compiled in Debug mode Thread Stats Avg Stdev Max +/- Stdev Latency 1.50ms 230.25us 5.05ms 86.76% Req/Sec 16.73k 2.05k 67.69k 83.85% 1999217 requests in 30.10s, 310.78MB read Requests/sec: 66409.49 Transfer/sec: 10.32MB ``` ``` Zig compiled in Debug mode Thread Stats Avg Stdev Max +/- Stdev Latency 0.88ms 281.05us 20.30ms 91.38% Req/Sec 28.58k 2.83k 59.14k 87.60% 3418569 requests in 30.10s, 404.26MB read Requests/sec: 113569.44 Transfer/sec: 13.43MB ``` TL;DR - much of a muchness for runtime performance ! There really isnt anything between them Actix generally has better latency, and less scatter (std deviation) Zig has a better peak throughput, and uses 1 third the memory (at least with these config tweaks) ----- The main difference (to me) is that the Actix build is pulling in a huge number of dependencies :( The zig code is only pulling in 1 dependency --- ### Overview (`lessons/212/bun-app-static/README.md`) # bun-app-static To install dependencies: ```bash bun install ``` To run: ```bash bun run app.js ``` This project was created using `bun init` in bun v1.1.29. [Bun](https://bun.sh) is a fast all-in-one JavaScript runtime. --- ### Overview (`lessons/212/bun-app/README.md`) # bun-app To install dependencies: ```bash bun install ``` To run: ```bash bun run app.js ``` This project was created using `bun init` in bun v1.1.29. [Bun](https://bun.sh) is a fast all-in-one JavaScript runtime. --- ### Overview (`lessons/213/bun-app/README.md`) # bun-app To install dependencies: ```bash bun install ``` To run: ```bash bun run app.js ``` This project was created using `bun init` in bun v1.1.29. [Bun](https://bun.sh) is a fast all-in-one JavaScript runtime. --- ### Overview (`lessons/217/bun-app/README.md`) # bun-app To install dependencies: ```bash bun install ``` To run: ```bash bun run app.js ``` This project was created using `bun init` in bun v1.1.29. [Bun](https://bun.sh) is a fast all-in-one JavaScript runtime. --- ### Overview (`lessons/219/my-website/README.md`) This is a [Next.js](https://nextjs.org) project bootstrapped with [`create-next-app`](https://github.com/vercel/next.js/tree/canary/packages/create-next-app). ## Getting Started First, run the development server: ```bash npm run dev # or yarn dev # or pnpm dev # or bun dev ``` Open [http://localhost:3000](http://localhost:3000) with your browser to see the result. You can start editing the page by modifying `app/page.js`. The page auto-updates as you edit the file. This project uses [`next/font`](https://nextjs.org/docs/app/building-your-application/optimizing/fonts) to automatically optimize and load [Geist](https://vercel.com/font), a new font family for Vercel. ## Learn More To learn more about Next.js, take a look at the following resources: - [Next.js Documentation](https://nextjs.org/docs) - learn about Next.js features and API. - [Learn Next.js](https://nextjs.org/learn) - an interactive Next.js tutorial. You can check out [the Next.js GitHub repository](https://github.com/vercel/next.js) - your feedback and contributions are welcome! ## Deploy on Vercel The easiest way to deploy your Next.js app is to use the [Vercel Platform](https://vercel.com/new?utm_medium=default-template&filter=next.js&utm_source=create-next-app&utm_campaign=create-next-app-readme) from the creators of Next.js. Check out our [Next.js deployment documentation](https://nextjs.org/docs/app/building-your-application/deploying) for more details. --- ### Overview (`lessons/220/rails-app/README.md`) # README This README would normally document whatever steps are necessary to get the application up and running. Things you may want to cover: * Ruby version * System dependencies * Configuration * Database creation * Database initialization * How to run the test suite * Services (job queues, cache servers, search engines, etc.) * Deployment instructions * ... --- ### Overview (`lessons/220/ruby-app/README.md`) # README Basic Ruby application. 25x faster than Rails on my Mac M1. To setup database run the following commands: ```bash bin/rake db:create bin/rake db:migrate ``` To run the server: ```bash bin/iodine -p 8080 ``` Defaults ```bash WORKERS_NUM=2 ``` ### Local benchmark #### `rails-app` ```bash WORKERS_NUM=2 RAILS_ENV=production rs wrk -c 10 -t 2 -d 10 --latency http://localhost:8080/api/devices Running 10s test @ http://localhost:8080/api/devices 2 threads and 10 connections Thread Stats Avg Stdev Max +/- Stdev Latency 2.12ms 846.12us 31.20ms 94.80% Req/Sec 2.42k 239.12 2.66k 88.12% Latency Distribution 50% 1.94ms 75% 2.17ms 90% 2.63ms 99% 5.35ms 48603 requests in 10.10s, 46.07MB read Requests/sec: 4811.04 Transfer/sec: 4.56MB ``` #### `ruby-app` ```bash WORKERS_NUM=2 bin/iodine -p 8080 config.ru wrk -c 10 -t 2 -d 10 --latency http://localhost:8080/api/devices Running 10s test @ http://localhost:8080/api/devices 2 threads and 10 connections Thread Stats Avg Stdev Max +/- Stdev Latency 142.99us 660.05us 21.06ms 99.01% Req/Sec 48.48k 9.07k 60.82k 63.00% Latency Distribution 50% 83.00us 75% 112.00us 90% 160.00us 99% 798.00us 965345 requests in 10.01s, 613.14MB read Requests/sec: 96422.17 Transfer/sec: 61.24MB ``` #### `ruby-app` with docker ```bash docker build -t ruby-app . docker run --rm -p 8080:8080 --name ruby-app ruby-app wrk -c 10 -t 2 -d 10 --latency http://localhost:8080/api/devices Running 10s test @ http://localhost:8080/api/devices 2 threads and 10 connections Thread Stats Avg Stdev Max +/- Stdev Latency 785.27us 3.26ms 62.81ms 97.92% Req/Sec 12.66k 3.14k 15.44k 84.50% Latency Distribution 50% 351.00us 75% 430.00us 90% 614.00us 99% 12.92ms 251840 requests in 10.00s, 159.96MB read Requests/sec: 25176.72 Transfer/sec: 15.99MB ``` --- ### Overview (`lessons/221/my-website/README.md`) This is a [Next.js](https://nextjs.org) project bootstrapped with [`create-next-app`](https://github.com/vercel/next.js/tree/canary/packages/create-next-app). ## Getting Started First, run the development server: ```bash npm run dev # or yarn dev # or pnpm dev # or bun dev ``` Open [http://localhost:3000](http://localhost:3000) with your browser to see the result. You can start editing the page by modifying `app/page.js`. The page auto-updates as you edit the file. This project uses [`next/font`](https://nextjs.org/docs/app/building-your-application/optimizing/fonts) to automatically optimize and load [Geist](https://vercel.com/font), a new font family for Vercel. ## Learn More To learn more about Next.js, take a look at the following resources: - [Next.js Documentation](https://nextjs.org/docs) - learn about Next.js features and API. - [Learn Next.js](https://nextjs.org/learn) - an interactive Next.js tutorial. You can check out [the Next.js GitHub repository](https://github.com/vercel/next.js) - your feedback and contributions are welcome! ## Deploy on Vercel The easiest way to deploy your Next.js app is to use the [Vercel Platform](https://vercel.com/new?utm_medium=default-template&filter=next.js&utm_source=create-next-app&utm_campaign=create-next-app-readme) from the creators of Next.js. Check out our [Next.js deployment documentation](https://nextjs.org/docs/app/building-your-application/deploying) for more details. --- ### Overview (`lessons/222/elixir-app/README.md`) # App To start your Bandit server: - Run `mix setup` to install and setup dependencies - Run `iex -S mix` to start the server Now you can visit [`localhost:4000`](http://localhost:4000) from your browser. --- ### Overview (`lessons/222/ruby-app-node-equivalent/README.md`) # Ruby Device Management API A Ruby-based HTTP server application for managing IoT devices with PostgreSQL database integration and Prometheus metrics. This is a Ruby equivalent of the Node.js application in the `node-app` folder. ## Overview This application provides a RESTful API for managing device records, including: - Device listing and creation - PostgreSQL database integration - Prometheus metrics for monitoring - Health check endpoint ## Features - **HTTP Server**: Built with Rack and WEBrick - **Database**: PostgreSQL integration using the `pg` gem - **Metrics**: Prometheus histogram metrics for tracking database operation duration - **RESTful API**: Endpoints for device management - **Docker Support**: Multi-stage Dockerfile for containerized deployment ## Prerequisites - Ruby 3.3 or higher - PostgreSQL database - Bundler gem (usually comes with Ruby) ## Installation 1. Install dependencies: ```bash bundle install ``` ## Configuration The application uses a `config.yaml` file for configuration. Create or modify `config.yaml`: ```yaml --- appPort: 8080 db: user: node password: devops123 host: postgresql.antonputra.pvt database: mydb maxConnections: 75 ``` **Note**: Update the database connection details to match your PostgreSQL setup. ## Running the Application ### Local Development 1. Ensure PostgreSQL is running and accessible 2. Make sure the database and `node_device` table exist (see migration notes below) 3. Start the server: ```bash ruby app.rb ``` The server will start on `http://0.0.0.0:8080` (or the port specified in `config.yaml`). ### Using Docker 1. Build the Docker image: ```bash docker build -t ruby-app-node-equivalent . ``` 2. Run the container: ```bash docker run -p 8080:8080 ruby-app-node-equivalent ``` ## API Endpoints ### GET /api/devices Returns a list of sample devices. **Response:** ```json [ { "id": 1, "uuid": "9add349c-c35c-4d32-ab0f-53da1ba40a2a", "mac": "5F-33-CC-1F-43-82", "firmware": "2.1.6", "created_at": "2024-05-28T15:21:51.137Z", "updated_at": "2024-05-28T15:21:51.137Z" }, ... ] ``` ### POST /api/devices Creates a new device in the database. **Request Body:** ```json { "mac": "AA-BB-CC-DD-EE-FF", "firmware": "1.0.0" } ``` **Response (201 Created):** ```json { "id": 4, "uuid": "generated-uuid", "mac": "AA-BB-CC-DD-EE-FF", "firmware": "1.0.0", "created_at": "2024-11-08T12:34:56.789Z", "updated_at": "2024-11-08T12:34:56.789Z" } ``` **Error Response (400 Bad Request):** ```json { "message": "Error message" } ``` ### GET /metrics Returns Prometheus metrics in text format. This endpoint exposes the `myapp_request_duration_seconds` histogram metric that tracks database operation durations. ### GET /healthz Health check endpoint. Returns `OK` if the server is running. **Response:** ``` OK ``` ## Database Schema The application expects a PostgreSQL table named `node_device` with the following structure: ```sql CREATE TABLE node_device ( id SERIAL PRIMARY KEY, uuid VARCHAR(255) NOT NULL, mac VARCHAR(255) NOT NULL, firmware VARCHAR(255) NOT NULL, created_at TIMESTAMP NOT NULL, updated_at TIMESTAMP NOT NULL ); ``` ## Project Structure ``` ruby-app-node-equivalent/ ├── app.rb # Main HTTP server application ├── config.rb # Configuration loader ├── config.yaml # Configuration file ├── db.rb # PostgreSQL database connection ├── devices.rb # Device database operations ├── metrics.rb # Prometheus metrics setup ├── Gemfile # Ruby dependencies ├── Dockerfile # Docker build configuration ├── .dockerignore # Docker ignore patterns └── README.md # This file ``` ## Dependencies - **pg**: PostgreSQL database adapter - **prometheus-client**: Prometheus metrics client library - **rack**: Web server interface - **yaml**: YAML configuration parsing (built-in) ## Metrics The application tracks database operation duration using a Prometheus histogram metric: - **Metric Name**: `myapp_request_duration_seconds` - **Type**: Histogram - **Labels**: `op` (operation type, e.g., "db") - **Buckets**: Custom buckets optimized for low-latency operations (0.00001 to 17.5 seconds) ## Error Handling - Database errors are caught and returned as 400 Bad Request with an error message - Invalid JSON in POST requests will result in a 400 error - All errors are logged to stdout ## Notes - The server uses a 60-second keep-alive timeout - UUIDs are automatically generated using Ruby's `SecureRandom.uuid` - Timestamps are generated in ISO 8601 format with millisecond precision - The application uses a single PostgreSQL connection (connection pooling can be added if needed) --- ### Overview (`lessons/222/ruby-app/README.md`) # README Basic Ruby application. 25x faster than Rails on my Mac M1. To setup database run the following commands: ```bash bin/rake db:create bin/rake db:migrate ``` To run the server: ```bash bin/iodine -p 8080 ``` Defaults ```bash WORKERS_NUM=2 RAILS_MAX_THREADS=2 ``` ### Local benchmark #### `rails-app` ```bash WORKERS_NUM=2 RAILS_MAX_THREADS=2 RAILS_ENV=production rs wrk -c 10 -t 2 -d 10 --latency http://localhost:8080/api/devices Running 10s test @ http://localhost:8080/api/devices 2 threads and 10 connections Thread Stats Avg Stdev Max +/- Stdev Latency 2.12ms 846.12us 31.20ms 94.80% Req/Sec 2.42k 239.12 2.66k 88.12% Latency Distribution 50% 1.94ms 75% 2.17ms 90% 2.63ms 99% 5.35ms 48603 requests in 10.10s, 46.07MB read Requests/sec: 4811.04 Transfer/sec: 4.56MB ``` #### `ruby-app` ```bash WORKERS_NUM=2 RAILS_MAX_THREADS=2 bin/iodine -p 8080 config.ru wrk -c 10 -t 2 -d 10 --latency http://localhost:8080/api/devices Running 10s test @ http://localhost:8080/api/devices 2 threads and 10 connections Thread Stats Avg Stdev Max +/- Stdev Latency 142.99us 660.05us 21.06ms 99.01% Req/Sec 48.48k 9.07k 60.82k 63.00% Latency Distribution 50% 83.00us 75% 112.00us 90% 160.00us 99% 798.00us 965345 requests in 10.01s, 613.14MB read Requests/sec: 96422.17 Transfer/sec: 61.24MB ``` #### `ruby-app` with docker ```bash docker build -t ruby-app . docker run --rm -p 8080:8080 --name ruby-app ruby-app wrk -c 10 -t 2 -d 10 --latency http://localhost:8080/api/devices Running 10s test @ http://localhost:8080/api/devices 2 threads and 10 connections Thread Stats Avg Stdev Max +/- Stdev Latency 785.27us 3.26ms 62.81ms 97.92% Req/Sec 12.66k 3.14k 15.44k 84.50% Latency Distribution 50% 351.00us 75% 430.00us 90% 614.00us 99% 12.92ms 251840 requests in 10.00s, 159.96MB read Requests/sec: 25176.72 Transfer/sec: 15.99MB ``` --- ### Overview (`lessons/226/actix-app/README.md`) ## Logging ```bash RUST_LOG=debug RUST_PORT=8080 cargo run ``` --- ### Overview (`lessons/230/elixir-app/README.md`) # App To start your Bandit server: - Run `mix setup` to install and setup dependencies - Run `iex -S mix` to start the server Now you can visit [`localhost:4000`](http://localhost:4000) from your browser. --- ### Overview (`lessons/233/elixir-app/README.md`) # App To start your Bandit server: - Run `mix setup` to install and setup dependencies - Run `iex -S mix` to start the server Now you can visit [`localhost:4000`](http://localhost:4000) from your browser. --- ### Overview (`lessons/241/bun-app/README.md`) # bun-app To install dependencies: ```bash bun install ``` To run: ```bash bun run app.js ``` This project was created using `bun init` in bun v1.2.1. [Bun](https://bun.sh) is a fast all-in-one JavaScript runtime. --- ### Overview (`lessons/274/app/README.md`) ## App ```shell protoc -I proto --cpp_out=./src/market ./proto/* ``` --- ### Contents (`docs/contents.md`) # Contents - [069 - How to Create GKE Cluster Using TERRAFORM from Scratch?](../lessons/069) - [070 - EKS Cluster Auto Scaling](../lessons/070) - [071 - Kubernetes Horizontal Pod Autoscaler](../lessons/071) - [072 - How to Install Prometheus on Kubernetes Cluster?](../lessons/072) - [073 - Horizontal Pod Autoscaler CUSTOM METRICS & PROMETHEUS](../lessons/073) - [074 - Vertical Pod Autoscaling](../lessons/074) - [075 - AWS Serverless Application Model](../lessons/075) - [076 - How to Build Slack Bot?](../lessons/076) - [077 - AWS Lambda Secrets Manager Example](../lessons/077) - [078 - How to Secure Nginx with Lets Encrypt on Ubuntu 20.04 with Certbot?](../lessons/078) - [079 - AWS EKS & Secrets Manager](../lessons/079) - [080 - How to Get Letsencrypt Wildcard Certificate](../lessons/080) - [081 - How to Setup Auto-Renew for Letsencrypt WILDCARD Certificate with DNS challenge?](../lessons/081) - [082 - NGINX Ingress Controller for Kubernetes Tutorial](../lessons/082) - [083 - Cert Manager Kubernetes Tutorial](../lessons/083) - [084 - AWS Point to Site VPN Setup](../lessons/084) - [085 - How to Deploy React App on Firebase Hosting?](../lessons/085) - [086 - Build a Docker Image and Publish It to AWS ECR using Github Actions](../lessons/086) - [087 - Build a Docker Image and Publish It to GCP GCR & Artifact Registry using Github Actions](../lessons/087) - [088 - How to Create Your Own GitHub Actions?](../lessons/088) - [089 - GitHub Actions Self Hosted Runner (Autoscaling with Kubernetes)](../lessons/089) - [090 - Kubernetes Terminal: 6 Must-Have Tools to Install & Examples](../lessons/090) - [091 - Kubernetes NodePort vs LoadBalancer vs Ingress?](../lessons/091) - [092 - Kubernetes Namespaces vs. Virtual Clusters](../lessons/092) - [093 - Kubernetes Basics: Pods, Nodes, Containers, Deployments and Clusters](../lessons/093) - [094 - Kubernetes Quality of Service (QoS) Classes for Pods](../lessons/094) - [095 - Kubernetes Pod Priority](../lessons/095) - [096 - Kubernetes Affinity and Anti Affinity vs NodeSelector](../lessons/096) - [097 - Kubernetes Taints and Tolerations (Examples)](../lessons/097) - [098 - Kubernetes Pod Disruption Budget](../lessons/098) - [099 - Kubernetes Local Persistent Volume: DON'T use hostPath!](../lessons/099) - [100 - How to Deploy Create React App to GitHub Pages?](../lessons/100) - [101 - Terraform Ansible Integration: GCP Example & Multiple VMs](../lessons/101) - [102 - How to Create EKS Cluster Using Terraform?](../lessons/102) - [103 - How to Add IAM User and IAM Role to AWS EKS Cluster?](../lessons/103) - [104 - How to Install Prometheus and Grafana on Ubuntu?](../lessons/104) - [105 - How to Install MongoDB on Kubernetes?](../lessons/105) - [106 - Google Cloud Functions Tutorial](../lessons/106) - [107 - How to Hack a Password in Python?](../lessons/107) - [108 - How to Create GKE Cluster Using TERRAFORM?](../lessons/108) - [109 - How to Create EKS Cluster Using eksctl?](../lessons/109) - [110 - How to Deploy React App (any static website) to S3 and CloudFront?](../lessons/110) - [111 - Kubernetes Kustomize Tutorial: 10 examples](../lessons/111) - [112 - Native EKS Ingress: AWS Load Balancer Controller](../lessons/112) - [113 - Create AWS EKS Fargate Using Terraform](../lessons/113) - [114 - Kubernetes Node Autoscaling with Karpenter](../lessons/114) - [115 - How to Create AWS Lambda with Terraform?](../lessons/115) - [116 - AWS API Gateway - with EKS](../lessons/116) - [117 - AWS API Gateway - Custom Domain](../lessons/117) - [118 - AWS API Gateway - EC2 Integration](../lessons/118) - [119 - AWS API Gateway - WebSocket API + EC2](../lessons/119) - [120 - How To SSH into your VM? - Google Cloud Platform](../lessons/120) - [121 - How to connect to EC2 instance?](../lessons/121) - [122 - How to send AWS CloudWatch Alarms to Slack?](../lessons/122) - [123 - Creating a Linux service with systemd](../lessons/123) - [124 - AWS Lambda Go vs. Node.js performance benchmark](../lessons/124) - [125 - How to create EKS Cluster using Terraform MODULES?](../lessons/125) - [126 - AWS Lambda Python vs. Node.js performance benchmark](../lessons/126) - [127 - How to create Application Load Balancer using Terraform?](../lessons/127) - [128 - AWS Lambda Go vs. Rust performance benchmark](../lessons/128) - [129 - AWS Lambda vs Cloud Functions (2nd gen) performance benchmark](../lessons/129) - [130 - Monitoring EKS & EC2 instances with MANAGED Prometheus & Grafana](../lessons/130) - [131 - Send Alerts to Slack, Email, PagerDuty - AWS Managed Prometheus](../lessons/131) - [132 - GCP Managed Prometheus Tutorial (Managed Collection)](../lessons/132) - [133 - EKS Storage | Install Amazon EBS CSI driver & KMS](../lessons/133) - [134 - How to monitor Persistent Volume usage in Kubernetes using Prometheus?](../lessons/134) - [135 - How to monitor Containers in Kubernetes using Prometheus & cAdvisor?](../lessons/135) - [136 - Monitor HTTP/REST API in Kubernetes using Prometheus & Nginx Ingress Controller](../lessons/136) - [137 - How to Monitor Golang with Prometheus (Counter - Gauge - Histogram - Summary)](../lessons/137) - [138 - Go (Golang) vs. Rust: (HTTP/REST API in Kubernetes) Performance Benchmark](../lessons/138) - [139 - Use Kubernetes Gateway API instead of Ingress! (Cert Manager - Istio - Prometheus)](../lessons/139) - [140 - How to Monitor Nginx with Prometheus and Grafana? (Install - Monitor - Fluentd)](../lessons/140) - [141 - How to Build Custom Prometheus Exporter?](../lessons/141) - [142 - Use Kubernetes Gateway API for Mesh (Service-to-Service - Istio - Kiali - Prometheus)](../lessons/142) - [143 - Go (Golang) vs Node JS (Microservices) performance benchmark](../lessons/143) - [144 - Nginx vs Traefik performance benchmark (HTTP - HTTPS - gRPC | Prometheus)](../lessons/144) - [145 - Go (Golang) vs Java performance benchmark (Fiber vs Spring Boot)](../lessons/145) - [146 - Nginx vs Apache HTTP Server performance benchmark](../lessons/146) - [147 - Go (Golang) vs Rust performance benchmark (gRPC)](../lessons/147) - [148 - HTTP/2 vs HTTP/3 performance benchmark](../lessons/148) - [149 - gRPC vs REST performance benchmark](../lessons/149) - [150 - ARM vs x86 performance benchmark](../lessons/150) - [151 - Nginx vs Envoy performance benchmark](../lessons/151) - [152 - gRPC vs Kafka: Which one should you choose?](../lessons/152) - [153 - Traefik vs Caddy 2](../lessons/153) - [154 - Prometheus Operator Kubernetes Tutorial](../lessons/154) - [155 - Istio Service Mesh Tutorial](../lessons/155) - [156 - How to Use EKS Blueprints Terraform Modules? (EFS - Karpenter - Add User - Fargate - ALB Controller)](../lessons/156) - [157 - DevOps Interview Questions: (Step-by-Step) #1](../lessons/157) - [158 - ArgoCD Tutorial for Beginners: GitOps CD for Kubernetes #1](../lessons/158) - [159 - ArgoCD Image Updater: Decouple CI from CD! #2](../lessons/159) - [160 - Terragrunt Tutorial: Create VPC, EKS from Scratch! (Step-by-Step)](../lessons/160) - [161 - Terraform Tips & Tricks: loops, if-statements, and more](../lessons/161) - [162 - How to Manage Secrets in Terraform](../lessons/162) - [163 - Thanos (Prometheus) Tutorial: Remote Read/Write - mTLS - Step-by-Step!](../lessons/163) - [164 - An Introduction to Terraform #1](../lessons/164) - [165 - How to manage Terraform State? #2](../lessons/165) - [166 - HardenEKS: Best Practices For Amazon EKS (Programmatically)](../lessons/166) - [167 - 9 Ways to Expose an App on Amazon EKS to the Internet](../lessons/167) - [168 - How to build multi-arch Docker Images? (Docker Hub - ECR - EKS Demo)](../lessons/168) - [171 - Most Common Kubernetes Deployment Strategies](../lessons/171) - [172 - Linkerd vs. Istio (Rust vs. C++) performance benchmark](../lessons/172) - [173 - Kubernetes Canary Deployment (Manual vs Automated)](../lessons/173) - [174 - Kubernetes Ingress Explained (2 Types)](../lessons/174) - [175 - AWS App Mesh Tutorial (EKS | Ingress | Terraform)](../lessons/175) - [176 - Crossplane Tutorial (vs Terraform): Create AWS VPC - EKS - IRSA - Cluster Autoscaler](../lessons/176) - [177 - Azure Kubernetes Service (AKS) Tutorial: (Terraform - Nginx Ingress & TLS - OIDC Workload Identity)](../lessons/177) - [178 - OpenTelemetry Golang Tutorial (Tracing in Grafana & Kubernetes & Tempo)](../lessons/178) - [179 - ArgoCD Notifications (Successful/Failed Deployments)](../lessons/179) - [180 - Go (Golang) vs Python Performance Benchmark (Kubernetes - OpenTelemetry - Prometheus - Grafana)](../lessons/180) - [181 - Kubernetes Autoscaling: HPA vs. VPA vs. Keda vs. CA vs. Karpenter vs. Fargate](../lessons/181) - [182 - Kubernetes Workload Resources: App vs. Container vs. Pod vs. Deployment vs. StatefulSets & More](../lessons/182) - [184 - Kubernetes Health Checks: Liveness vs. Readiness vs. Startup Probe](../lessons/184) - [185 - Kubernetes Architecture?](../lessons/185) - [186 - Apache Kafka Architecture?](../lessons/186) - [187 - Docker vs Kubernetes vs Containerd](../lessons/187) - [188 - Docker Networking Tutorial (Bridge - None - Host - IPvlan - Macvlan - Overlay)](../lessons/188) - [189 - What Is a Graph Database?](../lessons/189) - [190 - What is a Reverse Proxy? (vs. Forward Proxy)](../lessons/190) - [191 - Types of Load Balancing Algorithms](../lessons/191) - [192 - Kubernetes RBAC Explained](../lessons/192) - [193 - How to debug Kubernetes? (Deployments, Services & Ingress)](../lessons/193) - [194 - How to debug Kubernetes Ingress? (TLS - Cert-Manager - HTTP-01 & DNS-01 Challenges)](../lessons/194) - [195 - AWS EKS Kubernetes Tutorial](../lessons/195) - [196 - AWS EKS Kubernetes Tutorial - Full Course](../lessons/196) - [197 - Kubernetes Tutorial for Beginners - Full Course](../lessons/197) - [198 - How To Structure Terraform Project (3 Levels)](../lessons/198) - [199 - Docker Compose Tutorial for Beginners](../lessons/199) - [200 - How to Set Up ArgoCD for Production?](../lessons/200) - [201 - Java (Quarkus) vs. Go (Golang): Performance Benchmark in Kubernetes](../lessons/201) - [202 - C# (.NET) vs. Go (Golang): Performance Benchmark in Kubernetes](../lessons/202) - [203 - Rust (Actix) vs. Go (Golang): Performance Benchmark in Kubernetes (EKS)](../lessons/203) - [204 - Go (fiber) vs. Go (stdlib) vs. Go (gin): Performance Benchmark in Kubernetes](../lessons/204) - [205 - Rust (Actix) vs. Go (Stdlib): Performance Benchmark in Kubernetes](../lessons/205) - [206 - Rust (Actix) vs Rust (Axum) vs Rust (Rocket): Performance Benchmark in Kubernetes](../lessons/206) - [207 - Zap (Zig) vs Actix (Rust): Performance Benchmark in Kubernetes](../lessons/207) - [208 - Actix (Rust) vs Zap (Zig) vs Stdlib (Zig): Performance Benchmark in Kubernetes](../lessons/208) - [209 - Django (Python) vs Stdlib (Go): Performance Benchmark in Kubernetes](../lessons/209) - [210 - Node.js vs Golang: Performance Benchmark in Kubernetes](../lessons/210) - [211 - Node.js vs Golang (Round 2): Performance Benchmark in Kubernetes](../lessons/211) - [212 - Bun vs Node.js: Performance Benchmark in Kubernetes #212](../lessons/212) - [213 - Go (Golang) vs. Bun: Performance (Latency - Throughput - Saturation - Availability)](../lessons/213) - [214 - MySQL vs PostgreSQL Performance Benchmark (Latency - Throughput - Saturation)](../lessons/214) - [215 - Rust vs Zig vs Go: Performance (Latency - Throughput - Saturation - Availability)](../lessons/215) - [216 - MySQL vs PostgreSQL (Round 2) Performance Benchmark (Latency - Throughput - Saturation)](../lessons/216) - [217 - Deno vs. Node.js vs Bun: Performance (Latency - Throughput - Saturation - Availability)](../lessons/217) - [218 - Apache Kafka vs RabbitMQ Performance (Latency - Throughput - Saturation)](../lessons/218) - [219 - Nginx vs Apache Performance (Latency - Throughput - Saturation - Availability | HTTP/2 - TLS - Gzip)](../lessons/219) - [220 - Ruby on Rails vs Node.js Performance (Latency - Throughput - Saturation - Availability)](../lessons/220) - [221 - Nginx vs Caddy Performance (Latency - Throughput - Saturation - Availability | HTTP/2 - TLS - Gzip)](../lessons/221) - [222 - Ruby vs Node.js: Performance Benchmark](../lessons/222) - [223 - SQLite vs PostgreSQL](../lessons/223) - [224 - Nginx vs Traefik: What Is the BEST Reverse Proxy?](../lessons/224) - [225 - Redis vs Memcached Performance Benchmark](../lessons/225) - [226 - Pingora vs Nginx Performance Benchmark](../lessons/226) - [227 - MongoDB vs. PostgreSQL: Performance & Functionality](../lessons/227) - [228 - Redis vs Dragonfly Performance (Latency - Throughput - Saturation)](../lessons/228) - [229 - FASTEST Go Web Framework: gnet vs fiber vs fasthttp vs net/http?](../lessons/229) - [230 - Elixir vs Go (Golang) Performance (Latency - Throughput - Saturation - Availability)](../lessons/230) - [231 - Python (FastAPI) vs Go (Golang) Performance Benchmark](../lessons/231) - [232 - Python (FastAPI) vs Go (Golang) Performance Benchmark (Round 2)](../lessons/232) - [233 - Elixir vs Go Performance Benchmark (Round 2)](../lessons/233) - [234 - PostgreSQL vs Amazon RDS: Performance & Price](../lessons/234) - [235 - Terraform GCP Tutorial (Getting Started with Google Cloud Platform)](../lessons/235) - [236 - FastAPI vs Node.js Performance](../lessons/236) - [237 - VM vs Kubernetes: Performance 🚀](../lessons/237) - [238 - Create GKE Cluster using Terraform](../lessons/238) - [239 - FastAPI vs Go (Golang) vs Node.js: Performance & Price](../lessons/239) - [240 - gRPC vs REST vs GraphQL: Comparison & Performance](../lessons/240) - [241 - Deno vs. Node.js vs. Bun: Performance Comparison](../lessons/241) - [242 - Rust vs. Go (Golang): Performance (Only Standard Library)](../lessons/242) - [243 - Rust vs. Go (Golang): Performance 2025](../lessons/243) - [244 - Rust vs. Go (Golang): Performance (Fastest Frameworks + PostgreSQL)](../lessons/244) - [245 - Rust vs C++ Performance](../lessons/245) - [246 - TCP vs UDP Performance (Latency & Throughput)](../lessons/246) - [249 - TCP vs UDP Performance: HUGE Improvement!](../lessons/249) - [250 - Nginx vs. HAProxy vs. Traefik: Performance](../lessons/250) - [251 - TCP vs UDP performance (Round 3)](../lessons/251) - [252 - Nginx vs HAProxy Performance (HTTP/1 - HTTP/2 - HTTPS - Compression)](../lessons/252) - [253 - Redis Streams vs Pub/Sub: Performance](../lessons/253) - [254 - Intel vs AMD vs Graviton: Performance & Price](../lessons/254) - [255 - Terraform Tutorial on AWS](../lessons/255) - [256 - Terraform AWS VPC Tutorial - Public, Private, and Isolated Subnets](../lessons/256) - [258 - Redis vs Valkey performance](../lessons/258) - [259 - Rust vs C++ Performance: Can Rust Actually Be Faster?](../lessons/259) - [260 - ZeroMQ vs Aeron: Best for Market Data? Performance (Latency & Throughput)](../lessons/264) - [265 - Rust vs C++ Performance: Can Rust Actually Be Faster? (Pt. 2)](../lessons/265) - [268 - Build a Secure AWS EKS CI/CD Pipeline: Step-by-Step Tutorial (ArgoCD + GitHub Actions)](../lessons/268) - [269 - Reduce AWS Latency by 200x: Drop from 7ms to 35μs](../lessons/269) - [270 - How to Create a Fully Private AWS EKS Cluster? (Client VPN & Resolve Private Route 53 DNS Locally)](../lessons/270) - [271 - HTTP vs WebSockets: Performance (Latency - CPU - Memory - Network)](../lessons/271) - [272 - Redis vs Valkey: Performance & Comparison](../lessons/272) - [273 - Deno vs. Node.js vs. Bun: Performance Comparison](../lessons/273) - [274 - Protobuf vs JSON: Performance Benchmark & Size](../lessons/274) - [275 - Go (Golang) vs TypeScript: Performance Benchmark](../lessons/275) - [276 - Python vs JavaScript Performance: Best Backend Language 2026 (FastAPI vs Bun)](../lessons/276) - [277 - SQLite vs PostgreSQL Performance & Comparison (2026)](../lessons/277) - [278 - Zig vs Rust Performance Benchmark 2026](../lessons/278) - [279 - 6 SQL Joins you MUST know! (Animated + Practice)](../lessons/279) --- --- METRICS --- - Files Extracted: 274 - Estimated Token Budget: ~60418 tokens - Canonical Reference: https://codewiki.google/github.com/antonputra/tutorials