# Technical Documentation: DataTalksClub/mlops-zoomcamp
> ℹ️ **Provenance:** Multi-Tier Ingestion · [CodeWiki Reference](https://codewiki.google/github.com/DataTalksClub/mlops-zoomcamp)
### Overview (`README.md`)
MLOps Zoomcamp: A Free 9-Week Course on Productionizing ML Services
Master the fundamentals of MLOps, from training and experimentation to deployment and monitoring.
Join Slack •
#course-mlops-zoomcamp Channel •
Telegram Announcements •
Course Playlist •
FAQ •
Tweet about the Course
## Quick Links
| Resource | Link |
|----------|------|
| Course materials | [GitHub repository](https://github.com/DataTalksClub/mlops-zoomcamp) |
| Video lectures | [YouTube playlist](https://www.youtube.com/playlist?list=PL3MmuxUbc_hIUISrluw_A7wDSmfOhErJK) |
| Documentation | [Zoomcamp Logistics](https://datatalks.club/docs/courses/zoomcamp-logistics/) · [MLOps Zoomcamp](https://datatalks.club/docs/courses/mlops-zoomcamp/) |
| Course platform (deadlines, homework) | [courses.datatalks.club](https://courses.datatalks.club/) |
| Slack channel | [#course-mlops-zoomcamp](https://app.slack.com/client/T01ATQK62F8/C02R98X7DS9) |
| Announcements | [Telegram](https://t.me/dtc_courses) |
| FAQ | [FAQ document](https://datatalks.club/faq/mlops-zoomcamp.html) |
## About the Course
MLOps (machine learning operations) is a must-know skill for many data professionals. This free 9-week course teaches the fundamentals of MLOps, from training and experimentation to deployment and monitoring, through structured modules, hands-on workshops, and a final project. Each module introduces core MLOps concepts and tools.
## Who Should Join
This course is for data scientists, ML engineers, and software engineers who want to learn how to put machine learning models into production and operate them reliably.
## Prerequisites
To get the most out of this course, you should have prior experience with:
- Python
- Docker
- Command line basics
- Machine learning (e.g., through [ML Zoomcamp](https://github.com/DataTalksClub/machine-learning-zoomcamp))
- 1+ year of programming experience
## How to Take the Course
There are two ways to follow the course: live and self-paced.
| | Live Cohort | Self-Paced |
|-|-|-|
| Start | Not currently scheduled | Anytime |
| Lectures | Pre-recorded | Pre-recorded |
| Homework | Graded | Available but not scored |
| Leaderboard | ✅ Yes | ❌ No |
| Peer Review | ✅ Yes | ❌ No |
| Certificate | ✅ Yes | ❌ No |
| Cost | Free | Free |
| Register | [Get updates](https://courses.datatalks.club/register/mlops-zoomcamp/) | Just start learning! |
> [!NOTE]
> We don't plan to run a live cohort in 2026. The course is fully available for self-paced study now. [Register here](https://courses.datatalks.club/register/mlops-zoomcamp/) to be notified if we run a live cohort again.
Self-paced steps:
1. Follow the materials on [GitHub](https://github.com/DataTalksClub/mlops-zoomcamp)
2. Ask questions and share progress in [Slack](https://datatalks.club/slack.html)
3. Do the homework (self-checked) and build a project for your portfolio
## Syllabus
### [Module 1: Introduction](01-intro)
- What is MLOps?
- MLOps maturity model
- NY Taxi dataset (our running example)
- Why MLOps is essential
- Course structure & environment setup
- Homework
### [Module 2: Experiment Tracking & Model Management](02-experiment-tracking)
- Introduction to experiment tracking
- MLflow basics
- Model saving and loading
- Model registry
- Hands-on MLflow exercises
- Homework
### [Module 3: Orchestration & ML Pipelines](03-orchestration)
- Workflow orchestration
- Homework
### [Module 4: Model Deployment](04-deployment)
- Deployment strategies: online (web, streaming) vs. offline (batch)
- Deploying with Flask (web service)
- Streaming deployment with AWS Kinesis & Lambda
- Batch scoring for offline processing
- Homework
### [Module 5: Model Monitoring](05-monitoring)
- Monitoring ML-based services
- Web service monitoring with Prometheus, Evidently, and Grafana
- Batch job monitoring with Prefect, MongoDB, and Evidently
- Homework
### [Module 6: Best Practices](06-best-practices)
- Unit and integration testing
- Linting, formatting, and pre-commit hooks
- CI/CD with GitHub Actions
- Infrastructure as Code (Terraform)
- Homework
## Final Project
The [final project](07-project/) integrates everything covered in the course into an end-to-end MLOps pipeline.
## Certificate
Certificates are awarded to learners who complete the final project during a live cohort. After the project is graded, the certificate appears on your course platform dashboard once the instructors issue it, and the release is announced in Slack and Telegram. See the [certificate guide](https://datatalks.club/docs/courses/zoomcamp-logistics/certification/) for how to get it and add it to LinkedIn.
## Instructors
- [Cristian Martinez](https://www.linkedin.com/in/cristian-javier-martinez-09bb7031/)
- [Alexey Grigorev](https://www.linkedin.com/in/agrigorev/)
- [Emeli Dral](https://www.linkedin.com/in/emelidral/)
## Community & Support
### Getting Help on Slack
Join the [#course-mlops-zoomcamp](https://app.slack.com/client/T01ATQK62F8/C02R98X7DS9) channel on [DataTalks.Club Slack](https://datatalks.club/slack.html) for discussions, troubleshooting, and networking.
To keep discussions organized:
- Follow [our guidelines](https://datatalks.club/docs/courses/zoomcamp-logistics/asking-questions/) when posting questions.
- Review the [community guidelines](https://datatalks.club/slack/guidelines.html).
### Learning in Public
Share your progress as you go — see the [learning in public guide](https://datatalks.club/docs/courses/zoomcamp-logistics/learning-in-public/).
## Sponsors
Interested in supporting our community? Reach out to [alexey@datatalks.club](mailto:alexey@datatalks.club).
## FAQ
A few common questions. For everything else, see the full [MLOps Zoomcamp FAQ](https://datatalks.club/faq/mlops-zoomcamp.html).
Q: Is this course really free?
A: Yes. All videos, materials, and homework are free and open-source.
Q: Do I need prior experience?
A: Yes, some. You should know Python, Docker basics, and the machine learning workflow (for example from ML Zoomcamp), plus about a year of programming experience.
Q: Is there a live cohort?
A: Not currently. The course is fully available for self-paced study. Register on the signup form to be notified if we run a live cohort again.
Q: What does "live cohort" mean? Are there live classes?
A: No mandatory live classes. All lectures are pre-recorded. "Live" means deadlines, scored homework, peer review, and certificate eligibility.
## About DataTalks.Club
DataTalks.Club is a global online community of data enthusiasts. It's a place to discuss data, learn, share knowledge, ask and answer questions, and support each other.
Website •
Join Slack Community •
Newsletter •
Upcoming Events •
YouTube •
GitHub •
LinkedIn •
X
All the activity at DataTalks.Club mainly happens on [Slack](https://datatalks.club/slack.html). We post updates there and discuss different aspects of data, career questions, and more.
At DataTalks.Club, we organize online events, community activities, and free courses. You can learn more about what we do at [DataTalks.Club docs](https://datatalks.club/docs/general/).
---
### Overview (`01-intro/README.md`)
# Module 1: Introduction
In this module, we introduce the taxi ride duration example and prepare a Linux-based development environment. We also build the baseline notebook that the rest of the course improves.
## 1.1 [Introduction](01-introduction.md)
MLOps connects design, training, deployment, and operation around a production ML problem.
## 1.2 [GitHub Codespaces](02-codespaces.md)
Set up a cloud development workspace with a repository, terminal, Docker, and Jupyter access.
## 1.3 [VM in AWS](03-aws-vm.md)
Create an optional Ubuntu VM and install the tools used by the course.
## 1.4 [Reading Parquet Data](04-read-parquet.md)
Read the current NYC taxi data format and install the Parquet engine required by pandas.
## 1.5 [Training a Ride Duration Prediction Model](05-train-duration-model.md)
Build and evaluate a first baseline for taxi ride duration.
## 1.6 [Course Overview](06-course-overview.md)
See how the modules turn a notebook into a tracked, deployed, monitored ML service.
## 1.7 [MLOps Maturity Model](07-maturity-model.md)
Choose an appropriate level of automation for a model and its operating context.
## Homework
More information is available in the [Module 1 homework](https://github.com/DataTalksClub/mlops-zoomcamp/blob/main/cohorts/2025/01-intro/homework.md).
## Optional resources
Use these links when you want to explore the setup or the baseline notebook further.
- [Duration prediction notebook](duration-prediction.ipynb)
- [MLOps maturity model](https://docs.microsoft.com/en-us/azure/architecture/example-scenario/mlops/mlops-maturity-model)
- [Docker installation instructions for Ubuntu](https://docs.docker.com/engine/install/ubuntu/#install-using-the-repository)
## Community Notes
Share community notes and resources below.
* [Local MacOS envprep by Adetbekov](https://github.com/adetbekov/mlops-zoomcamp/blob/main/01-intro/macos-local-envprep.md)
* [Environment Setup by Ayoub](https://github.com/ayoub-berdeddouch/mlops-journey/blob/main/intro-01.md)
* [Intro, Environment Setup, and MLOps Maturity Models by Bala](https://github.com/balapriyac/DTC-MLOps-Zoomcamp/tree/main/week1)
* [GCP Environment Setup by Piyush](https://github.com/piyush-an/MLOps-ZoomCamp/blob/main/01-Introduction/infrastructure.md)
* [Microsoft Azure Environment Setup by Olaide](https://github.com/josepholaide/MLOps-Practice/blob/main/Week%201/README.md)
* [Environment Preparation using GCP and pyenv by Dani](https://github.com/syahrulhamdani/dtc-mlops/blob/main/week-1-introduction/README.md)
* [Useful links by Zioalex](https://github.com/zioalex/mlops-zoomcamp/blob/main/My_notes_week1.md)
* [Notes by Alvaro Navas](https://github.com/ziritrion/mlopszoomcamp/blob/main/notes/1_intro.md)
* [Environment setup on GCP](https://gist.github.com/ziritrion/3214aa570e15ae09bf72c4587cb9d686)
* [Docker cheatsheet](https://gist.github.com/ziritrion/1842ca8a4c4851602a8733bba19ab6050)
* [Conda cheatsheet](https://gist.github.com/ziritrion/8024025672ea92b8bdeb320d6015aa0d)
* [Git cheatsheet](https://gist.github.com/ziritrion/d73ca65bf4d19c79ca842a55853cb962)
* [Python and data libraries cheatsheet](https://gist.github.com/ziritrion/9b80e47956adc0f20ecce209d494cd0a)
* [Cloud server setup using Deploifai by 98sean98](https://github.com/98sean98/mlops-zoomcamp/blob/main/01-intro/deploifai-server/readme.md)
* [Notes from first lesson by Neimv](https://gitlab.com/neimv/mlops/-/blob/main/lessons_weeks/notes_1.md)
* [Course preliminaries by particle1331](https://particle1331.github.io/ok-transformer/nb/mlops/01-intro.html)
* [Notes by Francisco Delca](https://github.com/FDelca/mlops_datatalks_notes/blob/main/Week1/Week1-LearningNotes.ipynb)
* [MLOps primer by Lorenz](https://github.com/LoHertel/Road-to-MLOps/blob/main/01-primer/README.md)
* [Notes for Week 1 by Bhagabat](https://github.com/BPrasad123/MLOps_Zoomcamp/tree/main/Week1)
* [MLOps introduction by Pathik Ghugare](https://github.com/pathikg/MLOps-Zoomcamp-DataTalks/blob/main/week-01-introduction/MLOps-Introduction.md)
* [Module 1 Notes by Stephen Barrie](https://stephen137.github.io/posts/MLOps_Zoomcamp_Week1/MLOps_Zoomcamp_Week_1.html)
* [Module 1 Notes by Hongfan](https://github.com/Muhongfan/MLops/blob/main/01-intro/README.md)
* [Comprehensive MLOps guide by Nithish Prabhu](https://ntp3105.github.io/Comprehensive-MLOps/Week-1/Introduction%20to%20MLOps.html)
* [Environment setup troubleshooting by thatwonguy](https://github.com/thatwonguy/mlops-zoomcamp/blob/personal/01-intro/README.md)
* [Chapter 1 notes by Marcus](https://github.com/mleiwe/mlops-zoomcamp/blob/Ch1_Marcus/cohorts/2024/01-intro/Ch1_Notes.md)
* [GCP setup instructions by Marcus](https://github.com/mleiwe/mlops-zoomcamp/blob/Ch1_Marcus/cohorts/2024/01-intro/GoogleCloudSetUpNotes.md)
* [Machine Learning Essentials by Niting](https://github.com/niting9881/course-mlops-zoomcamp/blob/main/01-intro/README.md)
* [GCP Environment Setup with Terraform by Artem](https://github.com/Ladn0/mlops-zoomcamp/tree/main/01-intro/notes)
* [Week 1 local setup hints by hannarud](https://github.com/hannarud/mlops-zoomcamp-2025/blob/main/week1_notes.md)
* [Week 1 detailed notes by Muhammad Shifa](https://github.com/MuhammadShifa/mlops-zoomcamp2025/tree/main/01-introduction#readme)
* Send a PR, add your notes above this line
---
### Overview (`02-experiment-tracking/README.md`)
# Module 2: Experiment Tracking and Model Management
We record experiments from the taxi duration notebook and follow a model from its first run to a shared registry. We use MLflow for local tracking and model management. We also set up a remote AWS server.
## 2.1 [Experiment Tracking Intro](01-experiment-tracking-intro.md)
Define the tracking concepts and understand why we keep them together.
## 2.2 [Getting Started with MLflow](02-mlflow-getting-started.md)
Install MLflow, start the local UI, configure SQLite, and connect the notebook to an experiment.
## 2.3 [Experiment Tracking with MLflow](03-experiment-tracking.md)
Track hyperparameter search runs, then compare their parameters and metrics.
## 2.4 [Model Management](04-model-management.md)
Keep selected model artifacts and their lineage after the experiment ends.
## 2.5 [Model Registry](05-model-registry.md)
Register model versions, review lineage, and coordinate promotion with the deployment process.
## 2.6 [MLflow in Practice](06-mlflow-in-practice.md)
Choose between local tracking and a shared server with database-backed metadata and S3 artifacts.
## 2.7 [MLflow: Benefits, Limitations and Alternatives](07-mlflow-alternatives.md)
Evaluate the trade-offs of MLflow and compare the requirements that matter when choosing a tracking platform.
## Homework
More information is available in the [Module 2 homework](https://github.com/DataTalksClub/mlops-zoomcamp/blob/main/cohorts/2025/02-experiment-tracking/homework.md).
## Optional resources
Use these links to explore the presentation, AWS setup, and current MLflow documentation.
- [Slides](https://drive.google.com/file/d/1YtkAtOQS3wvY7yts_nosVlXrLQBq5q37/view?usp=sharing)
- [MLflow on AWS](docs/mlflow_on_aws.md)
- [MLflow documentation](https://mlflow.org/docs/latest/)
Use `search_experiments` in current MLflow versions instead of `list_experiments`, and follow [the MLflow issue](https://github.com/mlflow/mlflow/issues/8941) when you update older notebooks.
## Community Notes
Share community notes and resources below.
* [Notes and general MLflow documentation by Ayoub](https://gist.github.com/Qfl3x/ccff6b0708358c040e437d52af0c2e43)
* [Minimalist MLflow code reference by Anna V](https://github.com/annnvv/mlops_zoomcamp/blob/main/notes/module2_notes_MLflow.md)
* [Notes from the second lesson by Neimv](https://gitlab.com/neimv/mlops/-/blob/main/lessons_weeks/notes_2.md)
* [Experiment tracking notes by Ayoub B](https://github.com/ayoub-berdeddouch/mlops-journey/blob/main/experiment_tracking_02.md)
* [Experiment tracking by particle1331](https://particle1331.github.io/ok-transformer/nb/mlops/03-mlflow.html)
* [Week 2 experiment tracking notes by Bengsoon Chuah](https://github.com/bengsoon/mlops-zoomcamp/blob/main/02-experiment-tracking/notes/Experiment_Tracking_notes.md)
* [Model management notes by Alvaro Pena](https://github.com/alvarofps/mlops-zoomcamp/blob/main/02-experiment-tracking/my-notes/2.4%20Model%20management.md)
* [Notes by Alvaro Navas](https://github.com/ziritrion/mlopszoomcamp/blob/main/notes/2_experiment.md)
* [Notebook by froukje](https://github.com/froukje/ml-ops-zoomcamp/blob/master/02-experiment-tracking/week02.ipynb) and [notes](https://medium.com/@falbrechtg/getting-started-with-mlflow-tracking-46a0089d6a73)
* [MLflow on GCP by Isaac Kargar](https://kargarisaac.github.io/blog/mlops/data%20engineering/2022/06/15/MLFlow-on-GCP.html)
* [Week 2 experiment tracking notes by Bhagabat](https://github.com/BPrasad123/MLOps_Zoomcamp/tree/main/Week2)
* [MLflow notes by Jaime Cabrera-Salcedo](https://github.com/jaimeh94/MLOps-Zoomcamp/tree/main/02-experiment-tracking)
* [Experiment tracking with MLflow by Hongfan](https://github.com/Muhongfan/MLops/blob/main/02-experiment-tracking/README.md)
* [Running MLflow with Docker and Minikube](https://open.substack.com/pub/asfandqazi/p/mlflow-on-minikube?r=2o17tf&utm_campaign=post&utm_medium=web)
* [Chapter 2 notes by Marcus](https://github.com/mleiwe/mlops-zoomcamp/blob/Ch2_Marcus/cohorts/2024/02-experiment-tracking/Ch2_notes.md)
* [Experiment tracking by Annaliese Tech](https://github.com/AnnalieseTech/MLOPS_ZOOMCAMP/blob/main/02_EXPERIMENT_TRACKING/EXPERIMENT_TRACKING_NOTES.md)
* [Hyperparameter tuning with MLflow by Annaliese Tech](https://github.com/AnnalieseTech/MLOPS_ZOOMCAMP/blob/main/02_EXPERIMENT_TRACKING/Hyperparameter-Tuning.md)
* [MLflow setup and tracking by Hokfu](https://github.com/Hokfu/MLOps_Zoomcamp_Study/blob/main/02-experiment-tracking/README.md)
* [2025 notes on MLflow and Hyperopt by Gabi Fonseca](https://github.com/fonsecagabriella/ml_ops/blob/main/02_experiment_tracking/__notes.md)
* [Homework 2 article by Srikanth Ganji](https://medium.com/@srikanth.unix07/mlops-zoomcamp-2025-homework-2-experiment-tracking-with-mlflow-4ea1ed783531)
* [Week 2 detailed notes by Muhammad Shifa](https://github.com/MuhammadShifa/mlops-zoomcamp2025/blob/main/02-experiment-tracking/README.md)
* Send a PR, add your notes above this line
---
### Overview (`03-orchestration/README.md`)
# Module 3: Orchestration and ML Pipelines
We turn the duration prediction notebook into a repeatable, parameterized training script and then connect it to the concepts of workflow orchestration.
## 3.1 [Introduction to ML Pipelines](01-ml-pipelines.md)
Define a training pipeline, make its dependencies explicit, and distinguish pipeline logic from orchestration.
## 3.2 [Turning the Notebook into a Python Script](02-notebook-to-script.md)
Refactor the notebook into functions, expose the training period as command-line parameters, and preserve MLflow metadata.
## 3.3 [Using an Orchestrator](03-orchestrator.md)
Schedule, parameterize, backfill, and optionally deploy the training workflow with an orchestrator.
## Homework
More information is available in the [Module 3 homework](https://github.com/DataTalksClub/mlops-zoomcamp/blob/main/cohorts/2025/03-orchestration/homework.md).
## Optional resources
MLflow
To run MLflow with Docker, create an `mlflow.dockerfile`:
```dockerfile
FROM python:3.10-slim
RUN pip install mlflow==2.12.1
EXPOSE 5000
CMD [ \
"mlflow", "server", \
"--backend-store-uri", "sqlite:///home/mlflow_data/mlflow.db", \
"--host", "0.0.0.0", \
"--port", "5000" \
]
```
Add it to `docker-compose.yaml` and persist the MLflow data directory:
```yaml
mlflow:
build:
context: .
dockerfile: mlflow.dockerfile
ports:
- "5000:5000"
volumes:
- "${PWD}/mlflow_data:/home/mlflow_data/"
```
Use the same MLflow version in the training environment. The service should then be reachable at `http://localhost:5000` from the host or at `http://mlflow:5000` from another Compose service.
## Community Notes
Share community notes and resources below.
Previous-edition notes:
- [2022 Prefect notes](https://github.com/DataTalksClub/mlops-zoomcamp/tree/main/cohorts/2022/03-orchestration)
- [2023 Prefect notes](https://github.com/DataTalksClub/mlops-zoomcamp/blob/main/cohorts/2023/03-orchestration/prefect/README.md)
- [2024 Mage notes](https://github.com/DataTalksClub/mlops-zoomcamp/blob/main/cohorts/2024/03-orchestration/README.md)
2025 notes:
* [Running Airflow + MLflow using Docker by André Calatré](https://github.com/calatre/mlops-zoomcamp/tree/main/03-orchestration)
* [Week 3 workflow orchestration and Prefect by hannarud](https://github.com/hannarud/mlops-zoomcamp-2025/blob/main/week3_notes.md)
* [Orchestration with Prefect notes and code by Muhammad Shifa](https://github.com/MuhammadShifa/mlops-zoomcamp2025/blob/main/03-orchestration/README.md)
* Send a PR, add your notes above this line
---
### Overview (`04-deployment/README.md`)
# Module 4: Model Deployment
We package the duration model as a web service and connect serving to the model registry. We also compare online, streaming, and batch deployment modes.
## 4.1 [Three Ways of Deploying a Model](01-deployment-modes.md)
Compare batch, online, and event-driven streaming deployment and choose the boundary that fits the prediction requirement.
## 4.2 [Web Services: Deploying Models with Flask and Docker](02-flask-docker.md)
Build a Flask prediction endpoint, test it, and package it with its model artifacts in Docker.
## 4.3 [Web Services: Getting Models from the Model Registry](03-model-registry-serving.md)
Load a selected MLflow model version at serving time instead of bundling a fixed artifact into the image.
## 4.4 [Streaming: Deploying Models with Kinesis and Lambda](04-streaming-kinesis-lambda.md)
Optionally consume Kinesis events with a containerized Lambda function and produce predictions.
## 4.5 [Batch: Preparing a Scoring Script](05-batch-scoring.md)
Turn a scoring notebook into a parameterized script that can run scheduled batches and backfills.
## 4.6 [Batch Scoring with Mage](06-mage-batch-scoring.md)
Use a Mage transformation block to retrieve a model from MLflow and apply it to a batch.
## Homework
More information is available in the [Module 4 homework](https://github.com/DataTalksClub/mlops-zoomcamp/blob/main/cohorts/2025/04-deployment/homework.md).
## Code and resources
The main code examples are:
- [Flask web service](web-service)
- [MLflow-backed web service](web-service-mlflow)
- [Kinesis and Lambda example](streaming)
- [Batch scoring code](batch)
The streaming exercise can create AWS charges. Treat it as optional and remove test resources when finished. Use the local tests in Module 6 when you don't want to deploy cloud resources.
## Community Notes
Share community notes and resources below.
* [Notes on model deployment and creating a modeling package by Ron M.](https://particle1331.github.io/inefficient-networks/notebooks/mlops/04-deployment/notes.html)
* [Model deployment on Google Cloud Platform by M. Ayoub C.](https://gist.github.com/Qfl3x/de2a9b98a370749a4b17a4c94ef46185)
* [Week 4 deployment notes by Bhagabat](https://github.com/BPrasad123/MLOps_Zoomcamp/tree/main/Week4)
* [Week 4 deployment notes by Ayoub B.](https://github.com/ayoub-berdeddouch/mlops-journey/blob/main/deployment-04.md)
* [Week 4 deployment notes by Waleed](https://github.com/waleedayoub/mlops-zoomcamp/blob/main/cohorts/2023/04-deployment/module4notes.waleed.md)
* [Offline, online, and streaming deployment notes by Hongfan](https://github.com/Muhongfan/MLops/blob/main/04-deployment/README.md)
* [Week 4 notes by Marcus](https://github.com/mleiwe/mlops-zoomcamp/blob/NotesBranch/cohorts/2024/04-deployment/Ch4_Notes_ML.md)
* [2025 model deployment notes by Nitin Gupta](https://github.com/niting9881/course-mlops-zoomcamp/blob/main/04-deployment/README.md)
* [Week 4 detailed notes by Muhammad Shifa](https://github.com/MuhammadShifa/mlops-zoomcamp2025/blob/main/04-deployment/README.md)
* Send a PR, add your notes above this line
---
### Overview (`05-monitoring/README.md`)
# Module 5: Model Monitoring
We define production monitoring metrics and calculate Evidently results. We store them in PostgreSQL, then expose them through Grafana dashboards and debugging reports.
## 5.1 [Intro to ML Monitoring](01-ml-monitoring.md)
Separate service health, model performance, data quality, and drift indicators.
## 5.2 [Environment Setup](02-monitoring-environment.md)
Create the Python and Docker Compose environment for metric calculation, PostgreSQL, Adminer, and Grafana.
## 5.3 [Prepare Reference and Model](03-reference-model.md)
Train a baseline model and save the reference dataset used for later comparisons.
## 5.4 [Evidently Metrics Calculation](04-evidently-metrics.md)
Compare reference and current batches, then persist selected metrics for time-series monitoring.
## 5.5 [Evidently Monitoring Dashboard](05-evidently-dashboard.md)
Create a workspace for reports and model/data behavior.
## 5.6 [Dummy Monitoring](06-dummy-monitoring.md)
Test the database and dashboard path with repeatable synthetic values before adding model metrics.
## 5.7 [Data Quality Monitoring](07-data-quality.md)
Add data-quality checks, schedule metric calculations, and choose thresholds with model owners.
## 5.8 [Save Grafana Dashboard](08-save-grafana-dashboard.md)
Provision a saved Grafana dashboard from versioned JSON and configuration files.
## 5.9 [Debugging with Test Suites and Reports](09-debugging-tests-reports.md)
Use Evidently reports and test suites to investigate a monitoring metric.
## 5.10 [Monitoring Example](10-monitoring-example.md)
Run the local end-to-end monitoring stack with Docker Compose, PostgreSQL, and Grafana.
## Homework
More information is available in the [Module 5 homework](https://github.com/DataTalksClub/mlops-zoomcamp/blob/main/cohorts/2025/05-monitoring/homework.md).
## Code and resources
The current requirements set Evidently to 0.6.7. Compare this code with the [Evidently 0.7 example](post-evidently-0.7) when updating the API.
- [Monitoring services](docker-compose.yml)
- [Metric calculation](evidently_metrics_calculation.py)
- [Dummy metric calculation](dummy_metrics_calculation.py)
- [Baseline model notebook](baseline_model_nyc_taxi_data.ipynb)
- [Debugging notebook](debugging_nyc_taxi_data.ipynb)
## Community Notes
Share community notes and resources below.
* [Week 5 notes by M. Ayoub C.](https://gist.github.com/Qfl3x/aa6b1bec35fb645ded0371c46e8aafd1)
* [Week 5 monitoring notes by Ayoub B.](https://github.com/ayoub-berdeddouch/mlops-journey/blob/main/monitoring-05.md)
* [Week 5 notes from 2023](https://github.com/dimzachar/mlops-zoomcamp/tree/master/notes/Week_5)
* [Why monitor models after deployment? by Hongfan](https://github.com/Muhongfan/MLops/blob/main/05-monitoring/README.md)
* [Week 5 detailed notes by Muhammad Shifa](https://github.com/MuhammadShifa/mlops-zoomcamp2025/blob/main/05-monitoring/README.md)
* Send a PR, add your notes above this line
---
### Overview (`06-best-practices/README.md`)
# Module 6: Best Practices
We make the deployed model easier to test, review, provision, and deliver. We move from Python tests through Terraform and GitHub Actions.
## 6.1 [Testing Python Code with pytest](01-pytest.md)
Separate model logic from cloud clients and cover the service with fast unit tests.
## 6.2 [Integration Tests with docker-compose](02-docker-compose-tests.md)
Start the service in Docker and verify its HTTP API with a deterministic test event.
## 6.3 [Testing Cloud Services with LocalStack](03-localstack.md)
Exercise Kinesis-compatible behavior locally without creating real AWS resources.
## 6.4 [Code Quality: Linting and Formatting](04-linting-formatting.md)
Use isort, Black, and Pylint to keep code consistent and catch common problems.
## 6.5 [Git Pre-commit Hooks](05-pre-commit.md)
Run project checks automatically before creating a commit.
## 6.6 [Makefiles and make](06-makefiles.md)
Give tests, quality checks, builds, and deployment steps stable command names.
## 6.7 [Terraform: Introduction](07-terraform-introduction.md)
Describe the AWS stream-based prediction infrastructure as version-controlled code.
## 6.8 [Terraform: Modules and Output Variables](08-terraform-modules.md)
Extract reusable infrastructure and pass resource identifiers between modules.
## 6.9 [Build an End-to-end Ride Prediction Workflow](09-terraform-pipeline.md)
Connect Kinesis, Lambda, S3, and ECR into a deployable prediction workflow.
## 6.10 [Test the Pipeline End to End](10-terraform-testing.md)
Send a ride event through the deployed pipeline and review the prediction output.
## 6.11 [CI/CD: Introduction](11-cicd-introduction.md)
Map pull-request checks and delivery jobs onto the ride-prediction architecture.
## 6.12 [Continuous Integration](12-continuous-integration.md)
Run tests and Terraform plan automatically for proposed changes.
## 6.13 [Continuous Delivery](13-continuous-delivery.md)
Build, publish, and deploy the service after an approved change.
## Homework
More information is available in the [Module 6 homework](https://github.com/DataTalksClub/mlops-zoomcamp/blob/main/cohorts/2025/06-best-practices/homework.md).
## Code and resources
Resources:
- [Best-practices code](code)
- [Terraform concepts and references](docs/extra-material.md)
- [AWS stream-pipeline source diagram](AWS-stream-pipeline.png)
- [CI/CD source diagram](ci_cd_zoomcamp.png)
Cloud resources can incur charges. Destroy temporary infrastructure after testing and keep credentials out of the repository.
## Community Notes
Share community notes and resources below.
* [Week 6a notes by M. Ayoub C.](https://gist.github.com/Qfl3x/267d4cff36b58de67b4e33ca3fc9983f)
* [Unit tests, integration tests, LocalStack, code quality, pre-commit, and Makefile notes by Hongfan](https://github.com/Muhongfan/MLops/blob/main/06-best-practice/README.md)
* [Week 6 notes from 2023](https://github.com/dimzachar/mlops-zoomcamp/tree/master/notes/Week_6)
* [2025 best-practices notes and FAQ by Nitin Gupta](https://github.com/niting9881/course-mlops-zoomcamp/blob/main/06-best-practices/README.md)
* [Week 6 detailed notes by Muhammad Shifa](https://github.com/MuhammadShifa/mlops-zoomcamp2025/blob/main/06-best-practices/code/README.md)
* Send a PR, add your notes above this line
---
### Overview (`07-project/README.md`)
## Course Project
[Watch the course project video](https://www.loom.com/share/8f99d25893de4fb8aaa95c0395c740b6)
### Objective
The goal of this project is to apply everything we have learned
in this course to build an end-to-end machine learning project.
## Problem statement
For the project, we will ask you to build an end-to-end ML project.
For that, you will need:
* Select a dataset that you're interested in (see [Datasets](#datasets))
* Train a model on that dataset tracking your experiments
* Create a model training pipeline
* Deploy the model in batch, web service or streaming
* Monitor the performance of your model
* Follow the best practices
## Datasets you cannot use
The NYC taxi dataset is used throughout the course modules and homework. It cannot be used for the project. Pick any other dataset.
## Technologies
You don't have to limit yourself to technologies covered in the course. You can use alternatives as well:
* **Cloud**: AWS, GCP, Azure, ...
* **Experiment tracking tools**: MLFlow, Weights & Biases, ...
* **Workflow orchestration**: Prefect, Airflow, Flyte, Kubeflow, Argo, ...
* **Monitoring**: Evidently, WhyLabs/whylogs, ...
* **CI/CD**: Github actions, Gitlab CI/CD, ...
* **Infrastructure as code (IaC)**: Terraform, Pulumi, Cloud Formation, ...
If you use a tool that wasn't covered in the course, be sure to explain what that tool does.
If you're not certain about some tools, ask in Slack.
## Peer reviewing
> [!IMPORTANT]
> To evaluate the projects, we'll use peer reviewing. This is a great opportunity for you to learn from each other.
> * To get points for your project, you need to evaluate 3 projects of your peers
> * You get 3 extra points for each evaluation
## Evaluation Criteria
* Problem description
* 0 points: The problem is not described
* 1 point: The problem is described but shortly or not clearly
* 2 points: The problem is well described and it's clear what the problem the project solves
* Cloud
* 0 points: Cloud is not used, things run only locally
* 2 points: The project is developed on the cloud OR uses localstack (or similar tool) OR the project is deployed to Kubernetes or similar container management platforms
* 4 points: The project is developed on the cloud and IaC tools are used for provisioning the infrastructure
* Experiment tracking and model registry
* 0 points: No experiment tracking or model registry
* 2 points: Experiments are tracked or models are registered in the registry
* 4 points: Both experiment tracking and model registry are used
* Workflow orchestration
* 0 points: No workflow orchestration
* 2 points: Basic workflow orchestration
* 4 points: Fully deployed workflow
* Model deployment
* 0 points: Model is not deployed
* 2 points: Model is deployed but only locally
* 4 points: The model deployment code is containerized and could be deployed to cloud or special tools for model deployment are used
* Model monitoring
* 0 points: No model monitoring
* 2 points: Basic model monitoring that calculates and reports metrics
* 4 points: Comprehensive model monitoring that sends alerts or runs a conditional workflow (e.g. retraining, generating debugging dashboard, switching to a different model) if the defined metrics threshold is violated
* Reproducibility
* 0 points: No instructions on how to run the code at all, the data is missing
* 2 points: Some instructions are there, but they are not complete OR instructions are clear and complete, the code works, but the data is missing
* 4 points: Instructions are clear, it's easy to run the code, and it works. The versions for all the dependencies are specified.
* Best practices
* [ ] There are unit tests (1 point)
* [ ] There is an integration test (1 point)
* [ ] Linter and/or code formatter are used (1 point)
* [ ] There's a Makefile (1 point)
* [ ] There are pre-commit hooks (1 point)
* [ ] There's a CI/CD pipeline (2 points)
> [!NOTE]
> It's highly recommended to create a new repository for your project (not inside an existing repo) with a meaningful title, such as
> "Car Price Prediction" or "Music Genre Classification" and include as many details as possible in the README file. ChatGPT can assist you with this. Doing so will not only make it easier to showcase your project for potential job opportunities but also have it featured on the [Projects Gallery App](#projects-gallery).
> If you leave the README file empty or with minimal details, there may be point deductions as per the [Evaluation Criteria](#evaluation-criteria).
## Resources
### Datasets
Refer to the provided [datasets](https://github.com/DataTalksClub/data-engineering-zoomcamp/blob/main/projects/datasets.md) for possible selection.
### Projects Gallery
Explore a collection of projects completed by members of our community. The projects cover a wide range of topics and utilize different tools and techniques. Feel free to delve into any project and see how others have tackled real-world problems with data, structured their code, and presented their findings. It's a great resource to learn and get ideas for your own projects.
### MLOps Zoomcamp 2023
* [2023 Projects](../cohorts/2023/07-project)
### MLOps Zoomcamp 2022
* [2022 Projects](../cohorts/2022/07-project)
---
### Overview (`cohorts/README.md`)
# Cohorts
Each `cohorts//` directory here is a frozen, self-contained copy of one
past live delivery of the course: `2022`, `2023`, `2024`, `2025`. Every one of
them has a `cohort.yaml` (`curriculum: github_archive`) recording its dates
and pointing at a notice file — added after the fact, since none of these
years originally shipped with any manifest at all. Their interior (module
directories, differing content per year) is untouched pre-migration content,
not reshaped to match any newer convention.
`cohorts/2022/` has no `README.md` — its `cohort.yaml` points its
`archive.notice_path` at `leaderboard.md` instead, the closest thing it has
to a cohort-identifying document. That's a gap worth a human decision on
whether to add a real README there; it hasn't been treated as blocking.
## The current cohort is `self-paced`, not a year
There is no live cohort scheduled for this course — see the root
[`README.md`](../README.md): *"We don't plan to run a live cohort in
2026... fully available for self-paced study now."* Root
([`01-intro/`](../01-intro/) through [`07-project/`](../07-project/)) holds
that actively-maintained self-paced material, and is what `content: root`
means in `course.yaml`.
`cohorts/self-paced/cohort.yaml` represents that: `identifier: "self-paced"`,
`delivery: self_paced`, `curriculum: current`, `start_date`/`end_date: null`
(a self-paced delivery has no cohort-wide deadline), `homework: []` (nothing
is currently graded or tracked). `course.yaml:current_cohort` names it the
same way a year would for a live delivery.
**What's still deferred**: each root module is presently a single long-form
`README.md`, not the `module.yaml` + numbered `NN-kebab.md` lesson files the
shared-root v2 contract expects. The checker (`scripts/check-zoomcamp`) knows
about this and reports it (`numbered_module_required`,
`current_module_missing`) — those are expected, not a sign something broke.
Restructuring each module's content into that shape is separate, real
editorial work, not done here.
## The conventions themselves
`DataTalksClub/zoomcamp-ops` is the authority: `STRUCTURE.md` for the
repository layout and `docs/shared-curriculum-v2.md` for the shared-root
schema this repository's `course.yaml` now follows, plus the curriculum
contract documented beside it for the YAML schemas and the unit page rules.
The website's ingestion parser is the final authority and fails loudly on
push.
---
### Overview (`04-deployment/batch/README.md`)
## Batch deployment
* Turn the notebook for training a model into a notebook for applying the model
* Turn the notebook into a script
* Clean it and parametrize
---
### Overview (`04-deployment/streaming/README.md`)
## Machine Learning for Streaming
* Scenario
* Creating the role
* Create a Lambda function, test it
* Create a Kinesis stream
* Connect the function to the stream
* Send the records
Links
* [Tutorial: Using Amazon Lambda with Amazon Kinesis](https://docs.amazonaws.cn/en_us/lambda/latest/dg/with-kinesis-example.html)
## Code snippets
### Sending data
```bash
KINESIS_STREAM_INPUT=ride_events
aws kinesis put-record \
--stream-name ${KINESIS_STREAM_INPUT} \
--partition-key 1 \
--data "Hello, this is a test."
```
Decoding base64
```python
base64.b64decode(data_encoded).decode('utf-8')
```
Record example
```json
{
"ride": {
"PULocationID": 130,
"DOLocationID": 205,
"trip_distance": 3.66
},
"ride_id": 123
}
```
Sending this record
```bash
aws kinesis put-record \
--stream-name ${KINESIS_STREAM_INPUT} \
--partition-key 1 \
--data '{
"ride": {
"PULocationID": 130,
"DOLocationID": 205,
"trip_distance": 3.66
},
"ride_id": 156
}'
```
### Test event
```json
{
"Records": [
{
"kinesis": {
"kinesisSchemaVersion": "1.0",
"partitionKey": "1",
"sequenceNumber": "49630081666084879290581185630324770398608704880802529282",
"data": "ewogICAgICAgICJyaWRlIjogewogICAgICAgICAgICAiUFVMb2NhdGlvbklEIjogMTMwLAogICAgICAgICAgICAiRE9Mb2NhdGlvbklEIjogMjA1LAogICAgICAgICAgICAidHJpcF9kaXN0YW5jZSI6IDMuNjYKICAgICAgICB9LCAKICAgICAgICAicmlkZV9pZCI6IDI1NgogICAgfQ==",
"approximateArrivalTimestamp": 1654161514.132
},
"eventSource": "aws:kinesis",
"eventVersion": "1.0",
"eventID": "shardId-000000000000:49630081666084879290581185630324770398608704880802529282",
"eventName": "aws:kinesis:record",
"invokeIdentityArn": "arn:aws:iam::XXXXXXXXX:role/lambda-kinesis-role",
"awsRegion": "eu-west-1",
"eventSourceARN": "arn:aws:kinesis:eu-west-1:XXXXXXXXX:stream/ride_events"
}
]
}
```
### Reading from the stream
```bash
KINESIS_STREAM_OUTPUT='ride_predictions'
SHARD='shardId-000000000000'
SHARD_ITERATOR=$(aws kinesis \
get-shard-iterator \
--shard-id ${SHARD} \
--shard-iterator-type TRIM_HORIZON \
--stream-name ${KINESIS_STREAM_OUTPUT} \
--query 'ShardIterator' \
)
RESULT=$(aws kinesis get-records --shard-iterator $SHARD_ITERATOR)
echo ${RESULT} | jq -r '.Records[0].Data' | base64 --decode
```
### Running the test
```bash
export PREDICTIONS_STREAM_NAME="ride_predictions"
export RUN_ID="e1efc53e9bd149078b0c12aeaa6365df"
export TEST_RUN="True"
python test.py
```
### Putting everything to Docker
```bash
docker build -t stream-model-duration:v1 .
docker run -it --rm \
-p 8080:8080 \
-e PREDICTIONS_STREAM_NAME="ride_predictions" \
-e RUN_ID="e1efc53e9bd149078b0c12aeaa6365df" \
-e TEST_RUN="True" \
-e AWS_DEFAULT_REGION="eu-west-1" \
stream-model-duration:v1
```
URL for testing:
* http://localhost:8080/2015-03-31/functions/function/invocations
### Configuring AWS CLI to run in Docker
To use AWS CLI, you may need to set the env variables:
```bash
docker run -it --rm \
-p 8080:8080 \
-e PREDICTIONS_STREAM_NAME="ride_predictions" \
-e RUN_ID="e1efc53e9bd149078b0c12aeaa6365df" \
-e TEST_RUN="True" \
-e AWS_ACCESS_KEY_ID="${AWS_ACCESS_KEY_ID}" \
-e AWS_SECRET_ACCESS_KEY="${AWS_SECRET_ACCESS_KEY}" \
-e AWS_DEFAULT_REGION="${AWS_DEFAULT_REGION}" \
stream-model-duration:v1
```
Alternatively, you can mount the `.aws` folder with your credentials to the `.aws` folder in the container:
```bash
docker run -it --rm \
-p 8080:8080 \
-e PREDICTIONS_STREAM_NAME="ride_predictions" \
-e RUN_ID="e1efc53e9bd149078b0c12aeaa6365df" \
-e TEST_RUN="True" \
-v c:/Users/alexe/.aws:/root/.aws \
stream-model-duration:v1
```
### Publishing Docker images
Creating an ECR repo
```bash
aws ecr create-repository --repository-name duration-model
```
Logging in
```bash
$(aws ecr get-login --no-include-email)
```
Pushing
```bash
REMOTE_URI="387546586013.dkr.ecr.eu-west-1.amazonaws.com/duration-model"
REMOTE_TAG="v1"
REMOTE_IMAGE=${REMOTE_URI}:${REMOTE_TAG}
LOCAL_IMAGE="stream-model-duration:v1"
docker tag ${LOCAL_IMAGE} ${REMOTE_IMAGE}
docker push ${REMOTE_IMAGE}
```
---
### Overview (`04-deployment/web-service-mlflow/README.md`)
## Getting the model for deployment from MLflow
* Take the code from the previous video
* Train another model, register with MLflow
* Put the model into a scikit-learn pipeline
* Model deployment with tracking server
* Model deployment without the tracking server
Starting the MLflow server with S3:
```bash
mlflow server \
--backend-store-uri=sqlite:///mlflow.db \
--default-artifact-root=s3://mlflow-models-alexey/
```
Downloading the artifact
```bash
export MLFLOW_TRACKING_URI="http://127.0.0.1:5000"
export MODEL_RUN_ID="6dd459b11b4e48dc862f4e1019d166f6"
mlflow artifacts download \
--run-id ${MODEL_RUN_ID} \
--artifact-path model \
--dst-path .
```
---
### Overview (`04-deployment/web-service/README.md`)
## Deploying a model as a web-service
* Creating a virtual environment with Pipenv
* Creating a script for predictiong
* Putting the script into a Flask app
* Packaging the app to Docker
```bash
docker build -t ride-duration-prediction-service:v1 .
```
```bash
docker run -it --rm -p 9696:9696 ride-duration-prediction-service:v1
```
---
### Overview (`05-monitoring/post-evidently-0.7/README.md`)
# 5. Model Monitoring
## 5.1 Intro to ML monitoring
[Watch the monitoring introduction](https://www.youtube.com/watch?v=SQ0jBwd_3kk&list=PL3MmuxUbc_hIUISrluw_A7wDSmfOhErJK)
## 5.2 Environment setup
[Watch the monitoring environment setup](https://www.youtube.com/watch?v=yixA3C1xSxc&list=PL3MmuxUbc_hIUISrluw_A7wDSmfOhErJK)
## 5.3 Prepare reference and model
[Watch the reference and model preparation video](https://www.youtube.com/watch?v=IjNrkqMYQeQ&list=PL3MmuxUbc_hIUISrluw_A7wDSmfOhErJK)
## 5.4 Evidently metrics calculation
[Watch the Evidently metrics video](https://www.youtube.com/watch?v=kP3lzh_HfWY&list=PL3MmuxUbc_hIUISrluw_A7wDSmfOhErJK)
## 5.5 Evidently Monitoring Dashboard
[Watch the Evidently dashboard video](https://www.youtube.com/watch?v=zjvYhDPzFlY&list=PL3MmuxUbc_hIUISrluw_A7wDSmfOhErJK)
## 5.6 Dummy monitoring
[Watch the dummy monitoring video](https://www.youtube.com/watch?v=s3G4PMsOMOA&list=PL3MmuxUbc_hIUISrluw_A7wDSmfOhErJK)
## 5.7 Data quality monitoring
[Watch the data quality monitoring video](https://www.youtube.com/watch?v=fytrmPbcLhI&list=PL3MmuxUbc_hIUISrluw_A7wDSmfOhErJK)
> Note: in this video we use Prefect (07:33-11:21). Feel free to skip this part. Also note that Prefect
is not officially supported in the 2024 edition of the course.
## 5.8 Save Grafana Dashboard
[Watch the dashboard saving video](https://www.youtube.com/watch?v=-c4iumyZMyw&list=PL3MmuxUbc_hIUISrluw_A7wDSmfOhErJK)
## 5.9 Debugging with test suites and reports
[Watch the debugging and reports video](https://www.youtube.com/watch?v=sNSk3ojISh8&list=PL3MmuxUbc_hIUISrluw_A7wDSmfOhErJK)
## Homework
More information [here](../cohorts/2024/05-monitoring/homework.md)
## Notes
Did you take notes? Add them here:
* [Week 5 notes by M. Ayoub C.](https://gist.github.com/Qfl3x/aa6b1bec35fb645ded0371c46e8aafd1)
* [week 5: Monitoring notes Ayoub.B](https://github.com/ayoub-berdeddouch/mlops-journey/blob/main/monitoring-05.md)
* [Week 5: 2023](https://github.com/dimzachar/mlops-zoomcamp/tree/master/notes/Week_5)
* [Week5: Why we need to monitor models after deployment? by Hongfan (Amber)](https://github.com/Muhongfan/MLops/blob/main/05-monitoring/README.md)
* Send a PR, add your notes above this line
# Monitoring example
## Prerequisites
You need following tools installed:
- `docker`
- `docker-compose` (included to Docker Desktop for Mac and Docker Desktop for Windows )
## Preparation
Note: all actions expected to be executed in repo folder.
- Create virtual environment and activate it (eg. `python -m venv venv && source ./venv/bin/activate` or `conda create -n venv python=3.11 && conda activate venv`)
- Install required packages `pip install -r requirements.txt`
- Run `baseline_model_nyc_taxi_data.ipynb` for downloading datasets, training model and creating reference dataset
## Monitoring Example
### Starting services
To start all required services, execute:
```bash
docker-compose up
```
It will start following services:
- `db` - PostgreSQL, for storing metrics data
- `adminer` - database management tool
- `grafana` - Visual dashboarding tool
### Sending data
To calculate evidently metrics with prefect and send them to database, execute:
```bash
python evidently_metrics_calculation.py
```
This script will simulate batch monitoring. Every 10 seconds it will collect data for a daily batch, calculate metrics and insert them into database. This metrics will be available in Grafana in preconfigured dashboard.
### Accsess dashboard
- In your browser go to a `localhost:3000`
The default username and password are `admin`
- Then navigate to `General/Home` menu and click on `Home`.
- In the folder `General` you will see `New Dashboard`. Click on it to access preconfigured dashboard.
### Ad-hoc debugging
Run `debugging_nyc_taxi_data.ipynb` to see how you can perform a debugging with help of Evidently `TestSuites` and `Reports`
### Stopping services
To stop all services, execute:
```bash
docker-compose down
```
---
### Overview (`06-best-practices/code/README.md`)
## Code snippets
### Building and running Docker images
```bash
docker build -t stream-model-duration:v2 .
```
```bash
docker run -it --rm \
-p 8080:8080 \
-e PREDICTIONS_STREAM_NAME="ride_predictions" \
-e RUN_ID="e1efc53e9bd149078b0c12aeaa6365df" \
-e TEST_RUN="True" \
-e AWS_DEFAULT_REGION="eu-west-1" \
stream-model-duration:v2
```
Mounting the model folder:
```
docker run -it --rm \
-p 8080:8080 \
-e PREDICTIONS_STREAM_NAME="ride_predictions" \
-e RUN_ID="Test123" \
-e MODEL_LOCATION="/app/model" \
-e TEST_RUN="True" \
-e AWS_DEFAULT_REGION="eu-west-1" \
-v $(pwd)/model:/app/model \
stream-model-duration:v2
```
### Specifying endpoint URL
```bash
aws --endpoint-url=http://localhost:4566 \
kinesis list-streams
```
```bash
aws --endpoint-url=http://localhost:4566 \
kinesis create-stream \
--stream-name ride_predictions \
--shard-count 1
```
```bash
aws --endpoint-url=http://localhost:4566 \
kinesis get-shard-iterator \
--shard-id ${SHARD} \
--shard-iterator-type TRIM_HORIZON \
--stream-name ${PREDICTIONS_STREAM_NAME} \
--query 'ShardIterator'
```
### Unable to locate credentials
If you get `'Unable to locate credentials'` error, add these
env variables to the `docker-compose.yaml` file:
```yaml
- AWS_ACCESS_KEY_ID=abc
- AWS_SECRET_ACCESS_KEY=xyz
```
### Make
Without make:
```
isort .
black .
pylint --recursive=y .
pytest tests/
```
With make:
```
make quality_checks
make test
```
To prepare the project, run
```bash
make setup
```
### IaC
w/ Terraform
#### Setup
**Installation**:
* [aws-cli](https://docs.aws.amazon.com/cli/latest/userguide/getting-started-install.html) (both versions are fine)
* [terraform client](https://www.terraform.io/downloads)
**Configuration**:
1. If you've already created an AWS account, head to the IAM section, generate your secret-key, and download it locally.
[Instructions](https://docs.aws.amazon.com/cli/latest/userguide/getting-started-prereqs.html)
2. [Configure]((https://docs.aws.amazon.com/cli/latest/userguide/getting-started-quickstart.html)) `aws-cli` with your downloaded AWS secret keys:
```shell
$ aws configure
AWS Access Key ID [None]: xxx
AWS Secret Access Key [None]: xxx
Default region name [None]: eu-west-1
Default output format [None]:
```
3. Verify aws config:
```shell
$ aws sts get-caller-identity
```
4. (Optional) Configuring with `aws profile`: [here](https://docs.aws.amazon.com/cli/latest/userguide/cli-configure-sourcing-external.html) and [here](https://registry.terraform.io/providers/hashicorp/aws/latest/docs#using-an-external-credentials-process)
#### Execution
1. To create infra (manually, in order to test on staging env)
```shell
# Initialize state file (.tfstate)
terraform init
# Check changes to new infra plan
terraform plan -var-file=vars/stg.tfvars
```
```shell
# Create new infra
terraform apply -var-file=vars/stg.tfvars
```
2. To prepare aws env (copy model artifacts, set env-vars for lambda etc.):
```
. ./scripts/deploy_manual.sh
```
3. To test the pipeline end-to-end with our new cloud infra:
```
. ./scripts/test_cloud_e2e.sh
```
4. And then check on CloudWatch logs. Or try `get-records` on the `output_kinesis_stream` (refer to `integration_test`)
5. Destroy infra after use:
```shell
# Delete infra after your work, to avoid costs on any running services
terraform destroy
```
### CI/CD
1. Create a PR (feature branch): `.github/workflows/ci-tests.yml`
* Env setup, Unit test, Integration test, Terraform plan
2. Merge PR to `develop`: `.github/workflows/cd-deploy.yml`
* Terraform plan, Terraform apply, Docker build & ECR push, Update Lambda config
### Notes
* Unfortunately, the `RUN_ID` (if set via the `ENV` or `ARG` in `Dockerfile`), disappears during lambda invocation.
We'll set it via `aws lambda update-function-configuration` CLI command (refer to `deploy_manual.sh` or `.github/workflows/cd-deploy.yml`)
---
### Overview (`cohorts/2023/README.md`)
## MLOps Zoomcamp 2023 Cohort
* [Pre-Course Live Q&A](https://www.youtube.com/watch?v=o34Q_61iA4Y&list=PL3MmuxUbc_hKqamJqQ7Ew8HxptJYnXqQM&index=1)
* [Technical FAQ](https://docs.google.com/document/d/12TlBfhIiKtyBv8RnsoJR6F72bkPDGEvPOItJIxaEzE0/edit)
* [Course Playlist: Only 2023 Live videos & homeworks](https://www.youtube.com/playlist?list=PL3MmuxUbc_hKqamJqQ7Ew8HxptJYnXqQM)
* [Leaderboard](https://docs.google.com/spreadsheets/d/e/2PACX-1vTHTc2eDorvcprX3SRd_ZejSnOjd7SUBlmr7ttYs9NsbS3G9szB9wMlMfCOLVL5XWCB0p8oaDOfffaZ/pubhtml)
* [Deadline calendar](https://docs.google.com/spreadsheets/d/e/2PACX-1vRNTwA0Of1lyprYpn2YxU-l0gvNeq-up7g7ITB42nPf2gT9Qd3PTzqTmkjAZjk1s__r7D99CsJfcZEO/pubhtml?gid=0&single=true)
[**Module 1: Introduction**](01-intro)
* [Homework](01-intro/homework.md)
* [Solution](01-intro/homework.ipynb)
[**Module 2: Experiment Tracking**](02-experiment-tracking/)
* [Homework](02-experiment-tracking/homework.md)
* [Workshop: Weights & Biases](02-experiment-tracking/wandb.md)
* [Solution MLflow](02-experiment-tracking/solution-mlflow/)
[**Module 3: Orchestration and ML pipelines**](03-orchestration/)
* [Homework](03-orchestration/homework.md)
[**Module 4: Model Deployment**](04-deployment)
* [Homework](04-deployment/homework.md)
* [Solution](04-deployment/homework_solution)
[**Module 5: Model Monitoring**](05-monitoring/)
* [Homework](05-monitoring/homework.md)
[**Module 6: Best Practices**](06-best-practices)
* [Homework](06-best-practices/homework.md)
* [Solution](06-best-practices/homework_solution)
[**Projects**](07-project/)
* [More information](07-project/README.md)
---
### Overview (`cohorts/2024/README.md`)
## MLOps Zoomcamp 2024 Cohort
* [Pre-Course Live Q&A](https://www.youtube.com/watch?v=YmllO3ld5LE)
* [Course Launch video](https://www.youtube.com/watch?v=2jM7t-NTZxs) and [Slides](https://docs.google.com/presentation/d/1Tp2VVph5_vYIazQ53VR7TYmhJjQg9wuNIKKne3wlZVU/edit?usp=sharing)
* [Technical FAQ](https://docs.google.com/document/d/12TlBfhIiKtyBv8RnsoJR6F72bkPDGEvPOItJIxaEzE0/edit)
* TODO: Course Playlist: Only 2024 Live videos & homeworks
* [Course management platform](https://courses.datatalks.club/mlops-zoomcamp-2024/)
[**Module 1: Introduction**](01-intro)
* [Homework](01-intro/homework.md)
* [Solution](01-intro/homework.ipynb)
[**Module 2: Experiment Tracking**](02-experiment-tracking/)
* [Homework](02-experiment-tracking/homework.md)
* [Solution](02-experiment-tracking/solution)
[**Module 3: Orchestration and ML pipelines**](03-orchestration/)
* [Homework](03-orchestration/homework.md)
* [Solution](https://www.loom.com/share/802c8c0b843a4d3bbd9dbea240c3593a)
[**Module 4: Model Deployment**](04-deployment)
* [Homework](04-deployment/homework.md)
[**Module 5: Model Monitoring**](05-monitoring/)
* [Homework](05-monitoring/homework.md)
[**Module 6: Best Practices**](06-best-practices)
* [Homework](06-best-practices/homework.md)
[**Project**](project.md)
* [More information](project.md)
---
### Overview (`cohorts/2025/README.md`)
## MLOps Zoomcamp 2025 Cohort
* [Pre-Course Live Q&A](https://www.youtube.com/watch?v=rv43YJQsZIw)
* [Course Launch video](https://youtube.com/live/qqZU8nBtH90) and [Slides](https://docs.google.com/presentation/d/10dP4KoVpMA1iMGBk-XWp3YcHjukoM7AxZ2v4LuZd9wE/edit?usp=sharing)
* [Technical FAQ](https://docs.google.com/document/d/12TlBfhIiKtyBv8RnsoJR6F72bkPDGEvPOItJIxaEzE0/edit)
* [Course management platform](https://courses.datatalks.club/mlops-zoomcamp-2025/)
[**Module 1: Introduction**](01-intro)
* [Homework](01-intro/homework.md)
[**Module 2: Experiment Tracking**](02-experiment-tracking/)
* [Homework](02-experiment-tracking/homework.md)
[**Module 3: Orchestration and ML pipelines**](03-orchestration/)
* [Homework](03-orchestration/homework.md)
[**Module 4: Model Deployment**](04-deployment)
* [Homework](04-deployment/homework.md)
[**Module 5: Model Monitoring**](05-monitoring/)
* [Homework](05-monitoring/homework.md)
[**Module 6: Best Practices**](06-best-practices)
* [Homework](06-best-practices/homework.md)
[**Project**](project.md)
* [More information](project.md)
---
### Overview (`cohorts/self-paced/README.md`)
# MLOps Zoomcamp — self-paced
There is no live cohort currently scheduled for this course. The curriculum
at the repository root is fully available for self-paced study: work through
the modules at your own pace, using the code and notebooks in each module
directory.
* [Register here](https://courses.datatalks.club/register/mlops-zoomcamp/) to
be notified if a live cohort runs again.
* [FAQ](https://datatalks.club/faq/mlops-zoomcamp.html)
No homework or deadlines are tracked for self-paced study.
---
### Overview (`cohorts/2022/03-orchestration/README.md`)
# 3. Orchestration and ML Pipelines
**Note:** [`orchestration.py`](orchestration.py) is a ready final version. The rest of the files were worked on together during the video tutorials.
**Note** With Prefect version [`2.2.1`](https://github.com/PrefectHQ/prefect/blob/orion/RELEASE-NOTES.md#20b8) or later `DeploymentSpec`'s are now just `Deployment`'s.
## 3.1 Negative engineering and workflow orchestration
[Watch the video](https://www.youtube.com/watch?v=eKzCjNXoCTc&list=PL3MmuxUbc_hIUISrluw_A7wDSmfOhErJK)
## 3.2 Introduction to Prefect 2.0
[Watch the video](https://www.youtube.com/watch?v=Yb6NJwI7bXw&list=PL3MmuxUbc_hIUISrluw_A7wDSmfOhErJK)
## 3.3 First Prefect flow and basics
[Watch the video](https://www.youtube.com/watch?v=MCFpURG506w&list=PL3MmuxUbc_hIUISrluw_A7wDSmfOhErJK)
## 3.4 Remote Prefect Orion deployment
[Watch the video](https://www.youtube.com/watch?v=ComkSIAB0k4&list=PL3MmuxUbc_hIUISrluw_A7wDSmfOhErJK)
## 3.5 Deployment of Prefect flow
**Note:** There are several changes to deployment in Prefect 2.3.1 since 2.0b8:
- `DeploymentSpec` in 2.0b8 now becomes `Deployment`.
- `work_queue_name` is used instead of `tags` to submit the deployment to the a specific work queue.
- You don't need to create a work queue before using the work queue. A work queue will be created if it doesn't exist.
```python
from prefect.deployments import Deployment
from prefect.orion.schemas.schedules import IntervalSchedule
from datetime import timedelta
deployment = Deployment.build_from_flow(
flow=main,
name="model_training",
schedule=IntervalSchedule(interval=timedelta(minutes=5)),
work_queue_name="ml"
)
deployment.apply()
```
[Watch the video](https://www.youtube.com/watch?v=xw9JfaWPPps&list=PL3MmuxUbc_hIUISrluw_A7wDSmfOhErJK)
Links:
* [Instructions for Hosting Prefect Orion](https://discourse.prefect.io/t/hosting-an-orion-instance-on-a-cloud-vm/967)
## 3.6 MLOps Zoomcamp 3.6 - (Optional) Work queues and agents
[Watch the video](https://www.youtube.com/watch?v=oDSf0ThKsso&list=PL3MmuxUbc_hIUISrluw_A7wDSmfOhErJK)
## 3.7 Homework
More information here: TBD
## Notes
Did you take notes? Add them here:
* [Week 3, Prefect Introduction and S3 Bucket configuration with Prefect by M. Ayoub C.](https://gist.github.com/Qfl3x/8dd69b8173f027b9468016c118f3b6a5)
* [Notes from froukje](https://github.com/froukje/ml-ops-zoomcamp/blob/master/03-orchestration/week03_orchestration.ipynb)
* [Minimalist code notes from Anna V](https://github.com/annnvv/mlops_zoomcamp/blob/main/notes/module3_notes_prefect.md)
* [Getting Started on Prefect 2.0 + Deploying worfklows for MLflow Staging by Ron Medina (Jupyter Book)](https://particle1331.github.io/inefficient-networks/notebooks/mlops/3-prefect/3-prefect.html)
* [Quickstart your homework by Zioalex](https://github.com/zioalex/mlops-zoomcamp/blob/week3/03-orchestration/homework_quickstart.md)
* [Notes from Maxime M](https://github.com/maxmarkov/mlops-zoomcamp/blob/master/lecture-notes/WEEK-3/03-orchestration.md)
* [Week3: Prefect introduction and homework notes by Bhagabat](https://github.com/BPrasad123/MLOps_Zoomcamp/tree/main/Week3)
* [Week 3: Orchestration notes by Ayoub.B](https://github.com/ayoub-berdeddouch/mlops-journey/blob/main/orchestration-03.md)
* Send a PR, add your notes above this line
---
### Overview (`cohorts/2022/05-monitoring/README.md`)
# 5. Model Monitoring
## 5.1 Monitoring for ML-based services
[Watch the monitoring introduction](https://www.youtube.com/watch?v=gMiT11Bp05A&list=PL3MmuxUbc_hIUISrluw_A7wDSmfOhErJK)
[Slides](https://drive.google.com/file/d/1wcMU75ZcNNJie4ELjsKPkITIL93wHykt/view?usp=sharing)
## 5.2 Setting up the environment
[Watch the monitoring environment setup](https://www.youtube.com/watch?v=VkkpVXW53bo&list=PL3MmuxUbc_hIUISrluw_A7wDSmfOhErJK)
## 5.3 Creating a prediction service and simulating traffic
[Watch the prediction service and traffic simulation video](https://www.youtube.com/watch?v=umQ3Mo5G1o8&list=PL3MmuxUbc_hIUISrluw_A7wDSmfOhErJK)
## 5.4 Realtime monitoring walktrough (Prometheus, Evidently, Grafana)
[Watch the realtime monitoring walkthrough](https://www.youtube.com/watch?v=r_m4VFEJ8yY&list=PL3MmuxUbc_hIUISrluw_A7wDSmfOhErJK)
## 5.5 Batch monitoring walktrough (Prefect, MongoDB, Evidently)
[Watch the batch monitoring walkthrough](https://www.youtube.com/watch?v=KefdYuue_FE&list=PL3MmuxUbc_hIUISrluw_A7wDSmfOhErJK)
## 5.6 Homework
More information here: TBD
## Notes
Did you take notes? Add them here:
* [Week 5 notes by M. Ayoub C.](https://gist.github.com/Qfl3x/aa6b1bec35fb645ded0371c46e8aafd1)
* [week 5: Monitoring notes Ayoub.B](https://github.com/ayoub-berdeddouch/mlops-journey/blob/main/monitoring-05.md)
* Send a PR, add your notes above this line
# Monitoring example
## Prerequisites
You need following tools installed:
- `docker`
- `docker-compose` (included to Docker Desktop for Mac and Docker Desktop for Windows )
## Preparation
Note: all actions expected to be executed in repo folder.
- Create virtual environment and activate it (eg. `python -m venv venv && source ./venv/bin/activate`)
- Install required packages `pip install -r requirements.txt`
- Run `python prepare.py` for downloading datasets
## Monitoring Example
### Starting services
To start all required services, execute:
```bash
docker-compose up
```
It will start following services:
- `prometheus` - TSDB for metrics
- `grafana` - Visual tool for metrics
- `mongo` - MongoDB, for storing raw data, predictions, targets and profile reports
- `evidently_service` - Evindently RT-monitoring service (draft example)
- `prediction_service` - main service, which makes predictions
### Sending data
To start sending data to service, execute:
```bash
python send_data.py
```
This script will send every second single row from dataset to prediction service along with creating file `target.csv` with actual results (so it can be loaded after)
## Batch Monitoring Example
After you stop sending data to service, you can run batch monitoring pipeline (using Prefect) by running script:
```bash
python prefect_example.py
```
This script will:
- load `target.csv` to MongoDB
- download dataset from MongoDB
- Run Evidently Model Profile and Evidently Report on this data
- Save Profile data back to MongoDB
- Save Report to `evidently_report_example.html`
You can look at Prefect steps in Prefect Orion UI
(to start it execute `prefect orion start`)
---
### Overview (`cohorts/2022/07-project/README.md`)
## Course Project
The goal of this project is to apply everything we learned
in this course and build an end-to-end machine learning project.
Remember that to pass the project, you must evaluate 3 peers. If you don't do that, your project can't be considered compelete.
### Submitting
#### Project Cohort #2
Project:
* Form: https://forms.gle/aj8LHkY7PrWG9XzW6
* Deadline: 12 September, 23:00 CEST
Peer reviewing:
* Peer review assignments: [link](https://docs.google.com/spreadsheets/d/e/2PACX-1vQYTps829bmaN-aaJPiBUc3UwtN3e_llI44DKv-rQDsmVRMS1No7XWQqOyNI4ZbFbIvN351Q-G6edCP/pubhtml) ("project 2" tab)
* Form: https://forms.gle/BeQ2HCohrM3puKf26
* Deadline: 19 September, 23:00 CEST
Project feedback: [link](https://docs.google.com/spreadsheets/d/e/2PACX-1vRB5xKkhCyAUVNSNJvxaP94RwgNbYhf3dNf_ctRHhNKvvQQB94YVBn9JRdCTdQb5NGCJdYBtjXP7tP9/pubhtml) ("feedback-02" tab)
#### Project Cohort #1
Project:
* Form: https://forms.gle/7UmQkK4BBxqdgMDp9
* Deadline: 22 August, 23:00 CEST
Peer reviewing:
* Peer review assignments: [link](https://docs.google.com/spreadsheets/d/e/2PACX-1vQYTps829bmaN-aaJPiBUc3UwtN3e_llI44DKv-rQDsmVRMS1No7XWQqOyNI4ZbFbIvN351Q-G6edCP/pubhtml) ("project 1" tab)
* Form: https://forms.gle/KaBMoYhmfeEFmiWb7
* Deadline: 29 August, 23:00 CEST
Project feedback: [link](https://docs.google.com/spreadsheets/d/e/2PACX-1vRB5xKkhCyAUVNSNJvxaP94RwgNbYhf3dNf_ctRHhNKvvQQB94YVBn9JRdCTdQb5NGCJdYBtjXP7tP9/pubhtml) ("feedback-02" tab)
### Evaluation criteria
See [here](../../../07-project/README.md)
### Misc
To get the hash for your project, use this function to hash your email:
```python
from hashlib import sha1
def compute_hash(email):
return sha1(email.lower().encode('utf-8')).hexdigest()
```
Or use [this website](http://www.sha1-online.com/).
---
### Overview (`cohorts/2023/07-project/README.md`)
## Course Project
The goal of this project is to apply everything we learned
in this course and build an end-to-end machine learning project.
Remember that to pass the project, you must evaluate 3 peers. If you don't do that, your project can't be considered compelete.
### Submitting
#### Project Cohort #1
Project:
* Form: https://forms.gle/mRRoDtqDXBytvsoD9
* Deadline: 31 July, 23:00 CEST
Peer reviewing:
* Peer review assignments: [link](https://docs.google.com/spreadsheets/d/e/2PACX-1vTAztxXsLidZV8I18gL9_qtJpxhyvyhJsEeXrP3kFyZoauGgR-S4p6b7H5yJ9kdTbUE5wAAvZgTTZ49/pubhtml?gid=0&single=true) ("project 1" tab)
* Form: https://forms.gle/MRMHDuFiP6DFShaj7
* Deadline: 10 August, 23:00 CEST
Project feedback: [link](https://docs.google.com/spreadsheets/d/e/2PACX-1vTS8Mlu6sWyu6JinFUftUl6OB5mxXlwGT2icIyQCSbhDDmW36WWyAbv2dCFJhng6Nln0o3cwvTchjcU/pubhtml?gid=0&single=true) ("feedback-01" tab)
#### Project Cohort #2
Project:
* Form: https://forms.gle/o1s3NmYE4UmFSMVD7
* Deadline: 21 August (Monday), 23:00 CEST
Peer reviewing:
* Peer review assignments: [link](https://docs.google.com/spreadsheets/d/e/2PACX-1vTAztxXsLidZV8I18gL9_qtJpxhyvyhJsEeXrP3kFyZoauGgR-S4p6b7H5yJ9kdTbUE5wAAvZgTTZ49/pubhtml?gid=1942033009&single=true) ("project 2" tab)
* Form: https://forms.gle/R4Y58WSxGDWsDBFv7
* Deadline: 29 August (Tuesday), 23:00 CEST
Project feedback: [link](https://docs.google.com/spreadsheets/d/e/2PACX-1vTS8Mlu6sWyu6JinFUftUl6OB5mxXlwGT2icIyQCSbhDDmW36WWyAbv2dCFJhng6Nln0o3cwvTchjcU/pubhtml?gid=546664034&single=true) ("feedback-02" tab)
### Evaluation criteria
See [here](../../../07-project/README.md)
### Misc
To get the hash for your project, use this function to hash your email:
```python
from hashlib import sha1
def compute_hash(email):
return sha1(email.lower().encode('utf-8')).hexdigest()
```
Or use [this website](http://www.sha1-online.com/).
---
### Overview (`cohorts/2024/03-orchestration/README.md`)
# 3. Orchestration and ML Pipelines
## [3.0 Introduction: ML pipelines and Mage](3.0/README.md)
## [3.1 Data preparation: ETL and feature engineering](3.1/README.md)
## [3.2 Training: sklearn models and XGBoost](3.2/README.md)
## [3.3 Observability: Monitoring and alerting](3.3/README.md)
## [3.4 Triggering: Inference and retraining](3.4/README.md)
## [3.5 Deploying: Running operations in production](3.5/README.md)
## [3.6 Homework](../cohorts/2024/03-orchestration/homework.md).
## Quickstart
See the [Unit 3.0](https://github.com/DataTalksClub/mlops-zoomcamp/blob/main/03-orchestration/3.0/README.md) for a Quick Start guide
## Need help?
1. [Developer documentation](https://docs.mage.ai/introduction/overview)
1. [AI chat bot](https://mageai.slack.com/archives/C05NYC4DADT)
1. Live chat with the [Mage team directly](https://mage.ai/chat)
## Notes
Did you take notes? Add them here:
* [Marcus' Notes for Ch3](https://github.com/mleiwe/mlops-zoomcamp/blob/Ch3_ML_Notes/cohorts/2024/03-orchestration/ML_Notes.md)
* Send a PR, add your notes above this line
### Notes previous editions
- [2022 Prefect notes](../cohorts/2022/03-orchestration/README.md)
- [2023 Prefect notes](../cohorts/2023/03-orchestration/prefect/README.md)
---
### Overview (`cohorts/2025/competition/README.md`)
# MLOps Zoomcamp 2025 Competition
Welcome to the MLOps Zoomcamp 2025 Competition!
We're hosting this competition to help you practice everything you learn in the course.
Unlike traditional Kaggle competitions that focus only on model accuracy, this competition emphasizes the complete MLOps lifecycle, from model development to deployment and monitoring.
## Quick Links
- **Competition Platform**: https://youare.bot/
- Register your classifier in the "Register your API" section
- View leaderboards and track your performance
- [Detailed Overview (Slides)](https://docs.google.com/presentation/d/1-DIRH8wh-Rc251oZLjFNzQF0Fm0yUYe9PmLm00b3SyM/edit?usp=sharing)
- [Competition Overview Video](https://www.youtube.com/watch?v=ZxUVBG4z5uE)
## Competition Overview
### What You'll Build
**Classifier**: Build an ML model that detects whether a chat participant is a bot or human
### Scoring
**Classifiers**: Evaluated on both:
- ML metrics: accuracy, ROC, precision, recall, F-score
- Technical metrics: latency, uptime, response rate
### Timeline
- Competition period: June 2025-August 2025
- Final submission deadline: September 1st, 2025
- Evaluation: Best consecutive 7-day performance
- Deploy anytime during the competition period
- No need to keep your service running the entire time
### Benefits
- Practical MLOps experience in a real-world scenario
- Portfolio-worthy project showcasing end-to-end ML system
- Course points for participation
- $500 of AWS vouchers divided equally among top 10 participants
## Implementation Guide
Start with these core components:
- FastAPI/Flask for serving your model
- PostgreSQL for message history (optional)
- MLflow for experiment tracking
- Prometheus/Grafana for monitoring
As you advance, consider adding:
- CI/CD pipeline for automated deployment
- Evidently for model quality monitoring
- Airflow for automated retraining
- Cloud deployment (AWS, Fly.io, etc.)
### Getting Started
1. **Example Implementations**:
- [YouAreBot Quickstart Template](https://github.com/open-cu/youarebot-quickstart): Simple Echo Bot with FastAPI endpoint
- [YouAreBot Bot Example](https://github.com/open-cu/youarebot-bot): Advanced GPT-based bot (requires OpenAI/DeepSeek API key and additional setup)
- [YouAreBot Classifier Example](https://github.com/open-cu/youarebot-classifier): FastAPI service with PostgreSQL storage and random classifier (baseline for you to improve)
2. **Deployment Options**:
- Testing: Use quickstart repo with SSH tunneling
- Production: Deploy to any cloud provider
- AWS (recommended, covered in course)
- Fly.io
- Your preferred cloud platform
## Support
For technical questions, contact Misha:
- Telegram: https://t.me/bearcolonel
- Email: mikhail.semchinov@gmail.com
---
### Overview (`cohorts/2023/03-orchestration/prefect/README.md`)
# 3. Orchestration and ML Pipelines
This section of the repo contains Python code to accompany the videos that show how to use Prefect for MLOps. We will create workflows that you can orchestrate and observe.
## 3.1 Introdution to Workflow Orchestration
[Watch the video](https://www.youtube.com/watch?v=Cqb7wyaNF08&list=PL3MmuxUbc_hIUISrluw_A7wDSmfOhErJK)
## 3.2 Introduction to Prefect
[Watch the video](https://www.youtube.com/watch?v=rTUBTvXvXvM&list=PL3MmuxUbc_hIUISrluw_A7wDSmfOhErJK)
## 3.3 Prefect Workflow
[Watch the video](https://www.youtube.com/watch?v=x3bV8yMKjtc&list=PL3MmuxUbc_hIUISrluw_A7wDSmfOhErJK)
## 3.4 Deploying Your Workflow
[Watch the video](https://www.youtube.com/watch?v=3YjagezFhOo&list=PL3MmuxUbc_hIUISrluw_A7wDSmfOhErJK)
## 3.5 Working with Deployments
[Watch the video](https://www.youtube.com/watch?v=jVmaaqs63O8&list=PL3MmuxUbc_hIUISrluw_A7wDSmfOhErJK)
## 3.6 Prefect Cloud (optional)
[Watch the video](https://www.youtube.com/watch?v=y89Ww85EUdo&list=PL3MmuxUbc_hIUISrluw_A7wDSmfOhErJK)
## 3.7 Homework
More information [here](../cohorts/2023/03-orchestration/homework.md).
## Quick setup
### Install packages
In a conda environment with Python 3.10.12 or similar, install all package dependencies with
```bash
pip install -r requirements.txt
```
### Start the Prefect server locally
Create another window and activate your conda environment. Start the Prefect API server locally with
```bash
prefect server start
```
### Alternative to self-hosted server use Prefect Cloud for added capabilties
Signup and use for free at
Authenticate through the terminal with
```bash
prefect cloud login
```
Use your [Prefect profile](https://docs.prefect.io/latest/concepts/settings/) to switch between a self-hosted server and Cloud.
## Notes
Did you take notes? Add them here:
* [Prefect local deployment using Google Cloud Storage as data storage](https://github.com/ugm2/mlops-zoomcamp-workshop/tree/main/03-workflow-orchestration/3.5)
* Notes/Blog posts by Sagar Thacker
* [Inrtoduction to Workflow Orchestratio & Prefect](https://sagarthacker.com/posts/mlops/intro_workflow_orchestration.html)
* [Prefect Blocks](https://sagarthacker.com/posts/mlops/prefect-blocks.html)
* [Prefect Deployments](https://sagarthacker.com/posts/mlops/prefect-deployment.html)
* [Prefect Cloud Deployment](https://sagarthacker.com/posts/mlops/prefect-cloud.html)
* Send a PR, add your notes above this line
### Notes 2022 Edition
To read the notes from the previous edition, see [here](../cohorts/2022/03-orchestration/README.md)
---
### Overview (`cohorts/2024/03-orchestration/3.0/README.md`)
# 3.0 Introduction: ML pipelines and Mage
## 3.0.1 ML Pipelines
## 3.0.2 Running Mage
In Codespaces
On Windows with Docker-Compose
# What is MLOps
> Operationalizing ML models involves moving them from development to production to drive business value.
## Step 1
Preparing the model for deployment involves optimizing performance, ensuring it handles real-world data, and packaging it for integration into existing systems.
## Step 2
Deploying the model involves moving it from development to production, making it accessible to users and applications.
## Step 3
Once deployed, models must be continuously monitored for accuracy and reliability, and may need retraining on new data and updates to maintain effectiveness.
## Step 4
The operationalized model must be integrated into existing workflows, applications, and decision-making processes to drive business impact.
> Effective operationalization enables organizations to move beyond experimentation and drive tangible value from ML at scale, powering intelligent applications that personalize the customer experience and creates real business value.
# Why we need to operationalize ML
## 1. Productivity
MLOps fosters collaboration between data scientists, ML engineers, and DevOps teams by providing a unified environment for experiment tracking, feature engineering, model management, and deployment. This breaks down silos and accelerates the entire machine learning lifecycle.
## 2. Reliability
MLOps ensures high-quality, reliable models in production through clean datasets, proper testing, validation, CI/CD practices, monitoring, and governance.
## 3. Reproducibility
MLOps enables reproducibility and compliance by versioning datasets, code, and models, providing transparency and auditability to ensure adherence to policies and regulations.
## 4. Time-to-value
MLOps streamlines the ML lifecycle, enabling organizations to successfully deploy more projects to production and derive tangible business value and ROI from AI/ML investments at scale.
# How Mage helps MLOps
## 1. Data preparation
Mage offers features to build, run, and manage data pipelines for data transformation and integration, including pipeline orchestration, notebook environments, data integrations, and streaming pipelines for real-time data.
## 2. Training and deployment
Mage helps prepare data, train machine learning models, and deploy them with accessible API endpoints.
## 3. Standardize complex processes
Mage simplifies MLOps by providing a unified platform for data pipelining, model development, deployment, versioning, CI/CD, and maintenance, allowing developers to focus on model creation while improving efficiency and collaboration.
---
# Example data pipeline: Quick Start
1. Clone the following respository containing the complete code for this module:
```bash
git clone https://github.com/mage-ai/mlops.git
cd mlops
```
1. Launch Mage and the database service (PostgreSQL):
```bash
./scripts/start.sh
```
If don't have bash in your enviroment, modify the following command and run it:
```bash
PROJECT_NAME=mlops \
MAGE_CODE_PATH=/home/src \
SMTP_EMAIL=$SMTP_EMAIL \
SMTP_PASSWORD=$SMTP_PASSWORD \
docker compose up
```
It is ok if you get this warning, you can ignore it
`The "PYTHONPATH" variable is not set. Defaulting to a blank string.`
1. The subproject that contains all the pipelines and code is named
[`unit_3_observability`](https://github.com/mage-ai/mlops/tree/master/mlops/unit_3_observability)
## Run example pipeline
1. Open [`http://localhost:6789`](http://localhost:6789) in your browser.
1. In the top left corner of the screen next to the Mage logo and **`mlops`** project name,
click the project selector dropdown and choose the **`unit_0_setup`** option.
1. Click on the pipeline named **`example_pipeline`**.
1. Click on the button labeled **`Run @once`**.
## Resources
1. [Code for example data pipeline](https://github.com/mage-ai/mlops/tree/master/mlops/unit_0_setup)
1. [The definitive end-to-end machine learning (ML lifecycle) guide and tutorial for data engineers](https://mageai.notion.site/The-definitive-end-to-end-machine-learning-ML-lifecycle-guide-and-tutorial-for-data-engineers-ea24db5e562044c29d7227a67e70fd56?pvs=4).
---
### Overview (`cohorts/2024/03-orchestration/3.1/README.md`)
# 3.1 Data preparation: ETL and feature engineering
# Ingestion
## 3.1.1 Creating a new project
In order to create a new project, firstly, you need to right-click on a folder in Text editor, and choose `New Mage project`. Secondly, you need to open `Settings` and click on `Register project`.
[Video](https://youtu.be/7hKrQmoARD8)
Opening a text editor:
* Go to the command center (At the top)
* Type "text editor"
## 3.1.2 Data preparation - Ingestion
The project `unit_1_data_preparation` now has an empty pipeline, and it can be developed further using blocks. The first one we'll create is an ingestion block, which uses Python code to download the parquet files from January to March of the green taxi datasets and concatenate them. Done that, generate a series of graphs and charts useful for data profiling.
* **Note**: If the time chart isn't displayed, insert the following snippet `df['lpep_pickup_datetime_cleaned'] = df['lpep_pickup_datetime'].astype(np.int64) // 10**9` just above the `dfs.append(df)` line in `ingest.py`
[Video](https://youtu.be/1lSOdTpoRug)
Code:
- [`ingest.py`](https://github.com/mage-ai/mlops/blob/master/mlops/unit_3_observability/data_loaders/ingest.py)
## 3.1.3 Utility helper functions
Utility functions are already created in the `utils` folder. They will be then imported into the transformer block.
[Video](https://youtu.be/FBh3P19lXj4)
Code
- [`cleaning.py`](https://github.com/mage-ai/mlops/blob/master/mlops/utils/data_preparation/cleaning.py)
- [`feature_selector.py`](https://github.com/mage-ai/mlops/blob/master/mlops/utils/data_preparation/feature_selector.py)
- [`splitters.py`](https://github.com/mage-ai/mlops/blob/master/mlops/utils/data_preparation/splitters.py)
## 2. Data Preparation
### Videos
1. [Data preparation block](https://youtu.be/TcTMVn3BxeY)
1. [Visualize prepared data](https://youtu.be/j0Hfaoc5wRY)
To see the correct histogram, change last two lines of the default code to:
```python
col = 'trip_distance'
x = df_1[df_1[col] <= 20][col]
```
### Code
- [`prepare.py`](https://github.com/mage-ai/mlops/blob/master/mlops/unit_3_observability/transformers/prepare.py)
---
## 3. Build training sets
### Videos
1. [Encoding functions](https://youtu.be/z8erMV-6joY)
1. [Training set block](https://youtu.be/qSzcfSHjJoY)
### Code
- [`encoders.py`](https://github.com/mage-ai/mlops/blob/master/mlops/utils/data_preparation/encoders.py)
- [`build.py`](https://github.com/mage-ai/mlops/blob/master/mlops/unit_3_observability/data_exporters/build.py)
---
## 4. Data validations using built-in testing framework
### Videos
1. [Writing data validations](https://youtu.be/tYPAl4Q8kpw)
### Code
- [`build.py`](https://github.com/mage-ai/mlops/blob/master/mlops/unit_3_observability/data_exporters/build.py)
---
## Code
1. [Complete code solution](https://github.com/mage-ai/mlops)
1. [Pipeline configuration](https://github.com/mage-ai/mlops/blob/master/mlops/unit_3_observability/pipelines/data_preparation/metadata.yaml)
---
## Resources
1. [Global Data Products](https://docs.mage.ai/orchestration/global-data-products/overview)
1. [Data validations using built-in testing framework](https://docs.mage.ai/development/data-validation)
1. [Data quality checks with Great Expectations integration](https://docs.mage.ai/development/testing/great-expectations)
1. [Unit tests](https://docs.mage.ai/development/testing/unit-tests)
1. [Feature encoding](https://www.mage.ai/blog/qualitative-data)
---
### Overview (`cohorts/2024/03-orchestration/3.2/README.md`)
# 3.2 Training: sklearn models and XGBoost
## 1. Training pipeline for sklearn models
### Videos
1. [GDP training set](https://youtu.be/KP68DuJnk4Q?si=tVHWYLCpZ2RpwuNh)
1. [Sklearn training GDP](https://youtu.be/CbHaZcq_uGo)
1. [Load models](https://youtu.be/zsMHFq2C978)
1. [Utility helper functions for loading models](https://youtu.be/fZnxDhtPxYo)
1. [Hyperparameter tuning](https://youtu.be/zfBB4KoZ7TM)
1. [Train sklearn model](https://youtu.be/P7PtegUFk3k)
### Code
- [`utils/models/sklearn.py`](https://github.com/mage-ai/mlops/blob/master/mlops/utils/models/sklearn.py)
- [`custom/load_models.py` block](https://github.com/mage-ai/mlops/blob/master/mlops/unit_3_observability/custom/load_models.py): load sklearn models dynamically
- [`transformers/hyperparameter_tuning/sklearn.py`](https://github.com/mage-ai/mlops/blob/master/mlops/unit_3_observability/transformers/hyperparameter_tuning/sklearn.py)
- [`data_exporters/sklearn.py`](https://github.com/mage-ai/mlops/blob/master/mlops/unit_3_observability/data_exporters/sklearn.py)
- [`hyperparameters/shared.py`](https://github.com/mage-ai/mlops/blob/master/mlops/utils/hyperparameters/shared.py)
---
## 2. Training pipeline for XGBoost model
### Videos
1. [Hyperparameter tuning](https://youtu.be/K_Z2Lm1Cyu4)
1. [Train XGBoost model](https://youtu.be/Y2B-ivm7Mug)
### Code
- [`utils/models/xgboost.py`](https://github.com/mage-ai/mlops/blob/master/mlops/utils/models/xgboost.py)
- [`transformers/hyperparameter_tuning/xgboost.py`](https://github.com/mage-ai/mlops/blob/master/mlops/unit_3_observability/transformers/hyperparameter_tuning/xgboost.py)
- [`data_exporters/xgboost.py`](https://github.com/mage-ai/mlops/blob/master/mlops/unit_3_observability/data_exporters/xgboost.py)
- [`hyperparameters/shared.py`](https://github.com/mage-ai/mlops/blob/master/mlops/utils/hyperparameters/shared.py)
---
## Code
1. [Complete code solution](https://github.com/mage-ai/mlops)
1. [sklearn training pipeline configuration](https://github.com/mage-ai/mlops/blob/master/mlops/unit_3_observability/pipelines/sklearn_training/metadata.yaml)
1. [XGBoost training pipeline configuration](https://github.com/mage-ai/mlops/blob/master/mlops/unit_3_observability/pipelines/xgboost_training/metadata.yaml)
---
## Resources
1. [Accuracy, precision, recall](https://www.mage.ai/blog/definitive-guide-to-accuracy-precision-recall-for-product-developers)
1. [Regression model performance metrics](https://www.mage.ai/blog/product-developers-guide-to-ml-regression-model-metrics)
---
### Overview (`cohorts/2024/03-orchestration/3.3/README.md`)
## 3.3 Observability: Monitoring and alerting
## 1. Pipeline health monitoring
1. [sklearn training pipeline health](https://youtu.be/jwte-x3VwFE)
---
## 2. Explainability
### Videos
1. [Customize layout](https://youtu.be/Skr-WnxiQ8I)
1. [XGBoost explainability dashboard](https://youtu.be/BvGZTl-UUQY)
### Code
- [`custom/dashboard_data_source.py`](https://github.com/mage-ai/mlops/blob/master/mlops/unit_3_observability/custom/dashboard_data_source.py):
this is used to produce the output needed to generate the SHAP values and chart them.
- [SHAP values](https://github.com/mage-ai/mlops/blob/master/mlops/unit_3_observability/charts/shap_values.py)
- [SHAP values bar chart](https://github.com/mage-ai/mlops/blob/master/mlops/unit_3_observability/charts/shap_values_bar.py)
- [SHAP values force plot](https://github.com/mage-ai/mlops/blob/master/mlops/unit_3_observability/charts/shap_values_force_chart.py)
---
## 3. Model performance dashboard
### Videos
1. [Overview dashboard for entire project](https://youtu.be/ScFZPSaOWK4)
1. [RMSE and MSE using time series chart](https://youtu.be/6kqHoxAL0DY)
1. [RMSE distribution using a histogram](https://youtu.be/GQMgCzI-Qrg)
1. [Training runs by model using bar chart](https://youtu.be/q4Quk6GeVRk)
1. [Training runs by model using pie chart](https://youtu.be/I5qR3OtASXs)
### Code
- [Time series RMSE](https://github.com/mage-ai/mlops/blob/master/mlops/unit_3_observability/charts/training_metrics__rmse_.py)
- [Time series MSE](https://github.com/mage-ai/mlops/blob/master/mlops/unit_3_observability/charts/time_series__mse_.py)
- [Histogram](https://github.com/mage-ai/mlops/blob/master/mlops/unit_3_observability/charts/distribution_of_performance_metrics.py)
- [Runs by model](https://github.com/mage-ai/mlops/blob/master/mlops/unit_3_observability/charts/total_runs_by_model.py)
- [`data.py`](https://github.com/mage-ai/mlops/blob/master/mlops/utils/analytics/data.py)
---
## 4. Alerting
### Videos
1. [Setup email to send alerts](https://youtu.be/DjtE3webtjE)
1. [Setup pipeline alerting](https://youtu.be/H6D7zyqSQMw)
### Code
- [Project `metadata.yaml`](https://github.com/mage-ai/mlops/blob/master/mlops/unit_3_observability/metadata.yaml)
---
## Code
1. [sklearn training pipeline health](https://github.com/mage-ai/mlops/blob/master/mlops/presenters/pipelines/xgboost_training/dashboard/block_layout.yaml)
1. [XGBoost explainability dashboard](https://github.com/mage-ai/mlops/blob/master/mlops/presenters/pipelines/sklearn_training/dashboard/block_layout.yaml)
1. [Model performance dashboard configuration](https://github.com/mage-ai/mlops/blob/master/mlops/presenters/overview/dashboard/block_layout.yaml)
---
## Resources
1. [How to interpret ML models using SHAP values](https://www.mage.ai/blog/how-to-interpret-explain-machine-learning-models-using-shap-values)
1. [Customize dashboards for your project and pipelines](https://docs.mage.ai/visualizations/dashboards)
1. [Confusion matrix](https://www.mage.ai/blog/guide-to-model-metrics-p1-matrix-performance)
1. Set up alerts for pipeline run statuses:
1. [Email](https://docs.mage.ai/integrations/observability/alerting-email)
1. [Opsgenie](https://docs.mage.ai/integrations/observability/alerting-opsgenie)
1. [Slack](https://docs.mage.ai/integrations/observability/alerting-slack)
1. [Teams](https://docs.mage.ai/integrations/observability/alerting-teams)
1. [Discord](https://docs.mage.ai/integrations/observability/alerting-discord)
1. [Telegram](https://docs.mage.ai/integrations/observability/alerting-telegram)
---
### Overview (`cohorts/2024/03-orchestration/3.4/README.md`)
# 3.4 Triggering: Inference and retraining
## 1. Retraining pipeline
### Videos
1. [Setup pipeline](https://youtu.be/ywzNac-OzFc)
1. [Trigger pipeline to run](https://youtu.be/6kcBWl3E8So)
### Code
- [`detect_new_data.py`](https://github.com/mage-ai/mlops/blob/master/mlops/unit_3_observability/sensors/detect_new_data.py)
- [`custom/retrain/sklearn.py` block](https://github.com/mage-ai/mlops/blob/master/mlops/unit_3_observability/custom/retrain/sklearn.py): trigger training pipeline for sklearn models
- [`custom/retrain/xgboost.py` block](https://github.com/mage-ai/mlops/blob/master/mlops/unit_3_observability/custom/retrain/xgboost.py): trigger training pipeline for XGBoost model
---
## 2. Inference pipeline
### Videos
1. [Make a prediction](https://youtu.be/KZaS2oG9NDc)
1. [Build pipeline](https://youtu.be/mytcFbH_ooY)
1. [Model inference playground part 1](https://youtu.be/JI0dhR7Bnhk)
1. [Model inference playground part 2](https://youtu.be/v2ls-gBBRac)
1. [Get prediction via API](https://youtu.be/J6ckSZczk8M)
### Code
- [`custom/inference.py` block](https://github.com/mage-ai/mlops/blob/master/mlops/unit_3_observability/custom/inference.py)
---
## Code
1. [Retraining pipeline `metadata.yaml`](https://github.com/mage-ai/mlops/blob/master/mlops/unit_3_observability/pipelines/automatic_retraining/metadata.yaml)
1. [Inference pipeline `metadata.yaml`](https://github.com/mage-ai/mlops/blob/master/mlops/unit_3_observability/pipelines/predict/metadata.yaml)
1. [Playground configuration settings](https://github.com/mage-ai/mlops/blob/master/mlops/unit_3_observability/interactions/playground.yaml)
---
## Resources
1. [No-code UI interactions](https://docs.mage.ai/interactions/overview)
1. [Saving triggers in code](https://docs.mage.ai/orchestration/triggers/configure-triggers-in-code)
1. [Trigger another pipeline from a block](https://docs.mage.ai/orchestration/triggers/trigger-pipeline)
1. [Trigger pipeline via API endpoint](https://docs.mage.ai/orchestration/triggers/trigger-pipeline-api)
1. [Run pipelines on a recurring schedule](https://docs.mage.ai/orchestration/triggers/schedule-pipelines)
1. [Improving model performance through retraining]()
---
### Overview (`cohorts/2024/03-orchestration/3.5/README.md`)
# 3.5 Deploying: Running operations in production
## 1. Permissions
### Videos
1. [Configure permissions on AWS](https://youtu.be/TgdFaf4mw38)
### Code
- [`custom/permissions.py` block](https://github.com/mage-ai/mlops/blob/master/mlops/unit_3_observability/custom/permissions.py)
---
## 2. Deploy
### Videos
1. [Setup and deploy using Terraform](https://youtu.be/w9zl3n2a3Wc)
### Code
- [`custom/infrastructure_setup.py` block](https://github.com/mage-ai/mlops/blob/master/mlops/unit_3_observability/custom/infrastructure_setup.py)
- [`custom/deploy.py` block](https://github.com/mage-ai/mlops/blob/master/mlops/unit_3_observability/custom/deploy.py)
- [`custom/teardown_deployed_resources.py` block](https://github.com/mage-ai/mlops/blob/master/mlops/unit_3_observability/custom/teardown_deployed_resources.py)
---
## 3. Continuous deployment and integration
### Videos
1. [CI/CD with GitHub Actions](https://youtu.be/tPkA3WjLSHE)
1. [Mage deployed](https://youtu.be/DMV2zEM50jY)
### Code
- [`custom/ci_and_cd.py` block](https://github.com/mage-ai/mlops/blob/master/mlops/unit_3_observability/custom/ci_and_cd.py)
## Code
1. [Deployment pipeline `metadata.yaml`](https://github.com/mage-ai/mlops/blob/master/mlops/unit_3_observability/pipelines/deploying_to_production/metadata.yaml)
---
## Resources
1. [Repository setup](https://docs.mage.ai/production/ci-cd/local-cloud/repository-setup)
1. AWS IAM policy permissions
1. [Terraform apply](https://docs.mage.ai/production/deploying-to-cloud/aws/terraform-apply-policy)
1. [Terraform destroy](https://docs.mage.ai/production/deploying-to-cloud/aws/terraform-destroy-policy)
1. [Terraform setup](https://docs.mage.ai/production/deploying-to-cloud/using-terraform)
1. [Configure Terraform for AWS](https://docs.mage.ai/production/deploying-to-cloud/aws/setup)
1. [CI/CD overview](https://docs.mage.ai/production/ci-cd/overview)
1. [Setup GitHub actions for CI/CD](https://docs.mage.ai/production/ci-cd/local-cloud/github-actions#github-actions-setup)
---
### 02 Experiment Tracking Docs Mlflow On Aws (`02-experiment-tracking/docs/mlflow_on_aws.md`)
# Basic AWS setup
This tutorials explains how to configure a remote tracking server on AWS. We will use an RDS database as the backend store and an s3 bucket as the artifact store.
1. First, you need to [create an AWS account](https://aws.amazon.com/free). If you open a new account, AWS allows you to use some of their products for free but take into account that **you may be charged for using the AWS services**. More information [here](https://youtu.be/rkKvzCskpLE) and [here](https://aws.amazon.com/premiumsupport/knowledge-center/free-tier-charges/).
2. Launch a new EC2 instance.
For this, you can select one of the instance types that are free tier eligible. For example, we will select an Amazon Linux OS (`Amazon Linux 2 AMI (HVM) - Kernel 5.10, SSD Volume Type`) and a `t2.micro` instance type, which are free tier eligible.
You'll also need to create a new key pair so later you can connect to the new instance using SSH. Click on "Create new key pair" and complete the details like in the image below:
Select the new key pair and then click on "Launch Instance".
Finally, you have to edit the security group so the EC2 instance accepts SSH (port 22) and HTTP connections (port 5000):
3. Create an s3 bucket to be used as the artifact store.
Go to s3 and click on "Create bucket". Fill in the bucket name as in the image below and let all the other configurations with their default values.
Note: s3 bucket names must be unique across all AWS account in all the AWS Regions within a partition, that means that once a bucket is created, the name of that bucket cannot be used by another AWS account within the same region. If you get an error saying that the bucket name was already taken you can fix it easily by just changing the name to something like `mlflow-artifacts-remote-2` or another name.
4. Create a new PostgreSQL database to be used as the backend store
Go to the RDS Console and click on "Create database". Make sure to select "PostgreSQL" engine type and the "Free tier" template.
Select a name for your DB instance, set the master username as "mlflow" and tick the option "Auto generate a password" so Amazon RDS generate a password automatically.
Finally, on the section "Additional configuration" specify a database name so RDS automatically creates an initial database for you.
After clicking on "launch database" you will be able to check the newly generated password, but take into account that the automatically generated password will be shown only once!
The password value is intentionally redacted in this documentation image. In the AWS console, copy the generated password immediately and keep it private.
You can use the default values for all the other configurations.
Take note of the following information:
* master username
* password
* initial database name
* endpoint
Once the DB instance is created, go to the RDS console, select the new db and under "Connectivity & security" select the VPC security group. Modify the security group by adding a new inbound rule that allows postgreSQL connections on the port 5432 from the security group of the EC2 instance. This way, the server will be able to connect to the postgres database.
Note that the database name might be set to "postgres" by default.
5. Connect to the EC2 instance and launch the tracking server.
Go to the EC2 Console and find the instance launched on the step 2. Click on "Connect" and then follow the steps described in the tab "SSH".
Run the following commands to install the dependencies, configure the environment and launch the server:
* `sudo yum update`
* `pip3 install mlflow boto3 psycopg2-binary`
* `aws configure` # you'll need to input your AWS credentials here
* `mlflow server -h 0.0.0.0 -p 5000 --backend-store-uri postgresql://DB_USER:DB_PASSWORD@DB_ENDPOINT:5432/DB_NAME --default-artifact-root s3://S3_BUCKET_NAME`
Note: before launching the server, check that the instance can access the s3 bucket created in the step number 3. To do that, just run this command from the EC2 instance: `aws s3 ls`. You should see the bucket listed in the result.
6. Access the remote tracking server from your local machine.
Open a new tab on your web browser and go to this address: `http://:5000` (you can find the instance's public DNS by checking the details of your instance in the EC2 Console).
---
### 06 Best Practices Docs Extra Material (`06-best-practices/docs/extra-material.md`)
## Extra Material
### Concepts of IaC and Terraform
#### Summary
**Infrastructure-as-Code (IaC)**:
* Define and automate operations around you application's infrastructure.
* Can use version control to track changes made to infrastructure
* Easy to replicate the configuration across different environments such as development, staging, and production.
#### Reference Material
We have already covered Terraform concepts at a deeper level in the [Data Engineering Zoomcamp](https://github.com/DataTalksClub/data-engineering-zoomcamp), and will not be repeating some of those basic concepts again. You can find the content here for your reference:
**Notes**:
* [Terraform Overview](https://github.com/DataTalksClub/data-engineering-zoomcamp/blob/main/01-docker-terraform/1_terraform_gcp/1_terraform_overview.md)
**Videos**:
1. For an introduction to Terraform and IaC concepts, please refer to [this video](https://www.youtube.com/watch?v=Hajwnmj0xfQ&list=PL3MmuxUbc_hJed7dXYoJw8DoCuVHhGEQb&index=11)
(from the DE Zoomcamp), especially the sections in the time-codes:
* 00:00 Introduction
* 00:35 What is Terraform?
* 01:10 What is IaC?
* 01:43 Advantages of IaC
* 14:48 Installing Terraform
* 02:28 More on Installing Terraform
2. For a quickstart tutorial, and understanding the main components of a basic Terraform script, please refer to [this video](https://www.youtube.com/watch?v=dNkEgO-CExg&list=PL3MmuxUbc_hJed7dXYoJw8DoCuVHhGEQb&index=12)
(from the DE Zoomcamp). Please note that this example uses GCP as a cloud provider, while for MLOps Zoomcamp we are using AWS.
* 00:00 Introduction
* 00:20 .terraform-version
* 01:04 main.tf
* 01:23 terraform declaration
* 03:25 provider plugins
* 04:00 resource example - google_storage_bucket
* 05:42 provider credentials
* 06:34 variables.tf
* 10:54 overview of terraform commands
* 13:35 running terraform commands
* 18:08 recap
In case you're using GCP instead of AWS, following is some setup material:
* [Local Setup for Terraform and GCP](https://github.com/DataTalksClub/data-engineering-zoomcamp/tree/main/01-docker-terraform/1_terraform_gcp)
* [GCP Overview](https://github.com/DataTalksClub/data-engineering-zoomcamp/blob/main/01-docker-terraform/1_terraform_gcp/2_gcp_overview.md)
* [main.tf](https://github.com/DataTalksClub/data-engineering-zoomcamp/blob/main/01-docker-terraform/1_terraform_gcp/terraform/terraform_basic/main.tf)
#### References
* Terraform with AWS: [Getting Started](https://learn.hashicorp.com/collections/terraform/aws-get-started) and [AWS provider library](https://registry.terraform.io/providers/hashicorp/aws/latest/docs)
* Terraform Modules: [Define](https://www.terraform.io/language/modules/develop) and [Call](https://www.terraform.io/language/modules/syntax)
### Concepts of CI/CD and GitHub Actions
#### Summary
* Using GitHub Actions to create workflows to automatically test a pull request,
build and push a Docker image, and deploy the updated lambda service to production.
* Creating specific YAML files in GitHub repo, to automatically kick off a series of automation steps.
* Motivation on automating your further tasks with GitHub Actions:
* Orchestrating a continuous training pipeline (CT) to retrain your model and generate updated model artifacts in production
* Integrating the model registry (MLflow, DVC etc.) to fetch the latest model version or experiment ID
* and many more...
#### Reference Material
* [GitHub Actions & Workflows](https://docs.github.com/en/actions/using-workflows)
* [Build-Push image to ECR](https://docs.github.com/en/actions/deployment/deploying-to-your-cloud-provider/deploying-to-amazon-elastic-container-service)
* [Python tests](https://docs.github.com/en/actions/automating-builds-and-tests/building-and-testing-python)
---
--- METRICS ---
- Files Extracted: 34
- Estimated Token Budget: ~25993 tokens
- Canonical Reference: https://codewiki.google/github.com/DataTalksClub/mlops-zoomcamp