# Technical Documentation: unit8co/darts
> ℹ️ **Provenance:** Multi-Tier Ingestion · [CodeWiki Reference](https://codewiki.google/github.com/unit8co/darts)
### Overview (README.md)
# Time Series Made Easy in Python
---
[](https://badge.fury.io/py/darts)
[](https://anaconda.org/conda-forge/u8darts-all)
[](https://hub.docker.com/r/unit8/darts)
[](https://pepy.tech/project/darts)
[](https://pepy.tech/project/u8darts)
[](https://codecov.io/gh/unit8co/darts)
[](https://github.com/psf/black) [](https://gitter.im/u8darts/darts?utm_source=badge&utm_medium=badge&utm_campaign=pr-badge&utm_content=badge)
**Darts** is a Python library for user-friendly forecasting and anomaly detection
on time series. It contains a variety of models, from classics such as ARIMA to
deep neural networks. The forecasting models can all be used in the same way,
using `fit()` and `predict()` functions, similar to scikit-learn.
The library also makes it easy to backtest models,
combine the predictions of several models, and take external data into account.
Darts supports both univariate and multivariate time series and models.
The ML-based models can be trained on potentially large datasets containing multiple time
series, and some of the models offer a rich support for probabilistic forecasting.
Darts also offers extensive anomaly detection capabilities.
For instance, it is trivial to apply PyOD models on time series to obtain anomaly scores,
or to wrap any of Darts forecasting or filtering models to obtain fully
fledged anomaly detection models.
## Documentation
* [Quickstart](https://unit8co.github.io/darts/quickstart/00-quickstart.html)
* [User Guide](https://unit8co.github.io/darts/userguide.html)
* [API Reference](https://unit8co.github.io/darts/generated_api/darts.html)
* [Examples](https://unit8co.github.io/darts/examples.html)
### High Level Introductions
* [Introductory Blog Post](https://medium.com/unit8-machine-learning-publication/darts-time-series-made-easy-in-python-5ac2947a8878)
* [Introduction video (25 minutes)](https://youtu.be/g6OXDnXEtFA)
### Articles on Selected Topics
* [Training Models on Multiple Time Series](https://medium.com/unit8-machine-learning-publication/training-forecasting-models-on-multiple-time-series-with-darts-dc4be70b1844)
* [Using Past and Future Covariates](https://medium.com/unit8-machine-learning-publication/time-series-forecasting-using-past-and-future-external-data-with-darts-1f0539585993)
* [Temporal Convolutional Networks and Forecasting](https://medium.com/unit8-machine-learning-publication/temporal-convolutional-networks-and-forecasting-5ce1b6e97ce4)
* [Probabilistic Forecasting](https://medium.com/unit8-machine-learning-publication/probabilistic-forecasting-in-darts-e88fbe83344e)
* [Transfer Learning for Time Series Forecasting](https://medium.com/unit8-machine-learning-publication/transfer-learning-for-time-series-forecasting-87f39e375278)
* [Hierarchical Forecast Reconciliation](https://medium.com/unit8-machine-learning-publication/hierarchical-forecast-reconciliation-with-darts-8b4b058bb543)
## Quick Install
We recommend to first setup a clean Python environment for your project with Python 3.11+ using your favorite tool
([conda](https://docs.conda.io/projects/conda/en/latest/user-guide/tasks/manage-environments.html "conda-env"),
[venv](https://docs.python.org/3/library/venv.html), [virtualenv](https://virtualenv.pypa.io/en/latest/) with
or without [virtualenvwrapper](https://virtualenvwrapper.readthedocs.io/en/latest/)).
Once your environment is set up you can install darts using pip:
pip install darts
For more details you can refer to our
[installation instructions](https://github.com/unit8co/darts/blob/master/INSTALL.md).
## Example Usage
### Forecasting
Create a `TimeSeries` object from a Pandas DataFrame, and split it in train/validation series:
```python
import pandas as pd
from darts import TimeSeries
# Read a pandas DataFrame
df = pd.read_csv("AirPassengers.csv", delimiter=",")
# Create a TimeSeries, specifying the time and value columns
series = TimeSeries.from_dataframe(df, "Month", "#Passengers")
# Set aside the last 36 months as a validation series
train, val = series[:-36], series[-36:]
```
Fit an exponential smoothing model, and make a (probabilistic) prediction over the validation series' duration:
```python
from darts.models import ExponentialSmoothing
model = ExponentialSmoothing()
model.fit(train)
prediction = model.predict(len(val), num_samples=1000)
```
Plot the median, 5th and 95th percentiles:
```python
import matplotlib.pyplot as plt
series.plot()
prediction.plot(label="forecast", low_quantile=0.05, high_quantile=0.95)
plt.legend()
```
### Anomaly Detection
Load a multivariate series, trim it, keep 2 components, split train and validation sets:
```python
from darts.datasets import ETTh2Dataset
series = ETTh2Dataset().load()[:10000][["MUFL", "LULL"]]
train, val = series.split_before(0.6)
```
Build a k-means anomaly scorer, train it on the train set
and use it on the validation set to get anomaly scores:
```python
from darts.ad import KMeansScorer
scorer = KMeansScorer(k=2, window=5)
scorer.fit(train)
anom_score = scorer.score(val)
```
Build a binary anomaly detector and train it over train scores,
then use it over validation scores to get binary anomaly classification:
```python
from darts.ad import QuantileDetector
detector = QuantileDetector(high_quantile=0.99)
detector.fit(scorer.score(train))
binary_anom = detector.detect(anom_score)
```
Plot (shifting and scaling some of the series
to make everything appear on the same figure):
```python
import matplotlib.pyplot as plt
series.plot()
(anom_score / 2. - 100).plot(label="computed anomaly score", c="orangered", lw=3)
(binary_anom * 45 - 150).plot(label="detected binary anomaly", lw=4)
```
## Features
* **Forecasting Models:** A large collection of forecasting models for regression as well as classification tasks; from statistical models (such as
ARIMA) to deep learning models (such as N-BEATS). See the [forecasting models](#forecasting-models) below.
* **Anomaly Detection** The `darts.ad` module contains a collection of anomaly scorers,
detectors and aggregators, which can all be combined to detect anomalies in time series.
It is easy to wrap any of Darts forecasting or filtering models to build
a fully fledged anomaly detection model that compares predictions with actuals.
The `PyODScorer` makes it trivial to use PyOD detectors on time series.
* **Multivariate Support:** `TimeSeries` can be multivariate - i.e., contain multiple time-varying
dimensions/columns instead of a single scalar value. Many models can consume and produce multivariate series.
* **Multiple Series Training (Global Models):** All machine learning based models (incl. all neural networks)
support being trained on multiple (potentially multivariate) series. This can scale to large datasets too.
* **Probabilistic Support:** `TimeSeries` objects can (optionally) represent stochastic
time series; this can for instance be used to get confidence intervals, and many models support different
flavours of probabilistic forecasting (such as estimating parametric distributions or quantiles).
Some anomaly detection scorers are also able to exploit these predictive distributions.
* **Conformal Prediction Support:** Our conformal prediction models allow to generate probabilistic forecasts with
calibrated quantile intervals for any pre-trained global forecasting model.
* **Past and Future Covariates Support:** Many models in Darts support past-observed and/or future-known
covariate (external data) time series as inputs for producing forecasts.
* **Static Covariates Support:** In addition to time-dependent data, `TimeSeries` can also contain
static data for each dimension, which can be exploited by some models.
* **Hierarchical Reconciliation:** Darts offers transformers to perform reconciliation.
These can make the forecasts add up in a way that respects the underlying hierarchy.
* **Regression Models:** It is possible to plug-in any scikit-learn compatible model
to obtain forecasts as functions of lagged values of the target series and covariates.
* **Training with Sample Weights:** All global models support being trained with sample weights. They can be
applied to each observation, forecasted time step and target column.
* **Forecast Start Shifting:** All global models support training and prediction on a shifted output window.
This is useful for example for Day-Ahead Market forecasts, or when the covariates (or target series) are reported
with a delay.
* **Explainability:** Darts has the ability to *explain* some forecasting models using SHAP values.
* **Data Processing:** Tools to easily apply (and revert) common transformations on
time series data (scaling, filling missing values, differencing, boxcox, ...)
* **Metrics:** A variety of metrics for evaluating time series' goodness of fit;
from R2-scores to Mean Absolute Scaled Error.
* **Backtesting:** Utilities for simulating historical forecasts, using moving time windows.
* **PyTorch Lightning Support:** All deep learning models are implemented using PyTorch Lightning,
supporting among other things custom callbacks, GPUs/TPUs training and custom trainers.
* **MLflow Integration:** Integration with MLflow for automated tracking, comparing, and persisting
of Darts forecasting model experiments. See the [MLflow quickstart notebook](https://unit8co.github.io/darts/examples/29-MLflow-examples.html) for examples.
* **Filtering Models:** Darts offers three filtering models: `KalmanFilter`, `GaussianProcessFilter`,
and `MovingAverageFilter`, which allow to filter time series, and in some cases obtain probabilistic
inferences of the underlying states/values.
* **Datasets** The `darts.datasets` submodule contains some popular time series datasets for rapid
and reproducible experimentation.
* **Compatibility with Multiple Backends:** `TimeSeries` objects can be created from and exported to various backends such as pandas, polars, numpy, pyarrow, xarray, and more, facilitating seamless integration with different data processing libraries.
## Forecasting Models
Here's a breakdown of the forecasting models currently implemented in Darts. Our suite includes both regression and classification models, each tailored for specific forecasting tasks. We are committed to expanding our offerings with new models and features to enhance your forecasting capabilities.
**Regression Models:** Our regression models are designed to predict continuous numerical values, making them ideal for forecasting future trends and patterns in time series data. Utilize these models to gain insights into potential future outcomes based on historical data.
| Model | Sources | Target Series Support:
Univariate/
Multivariate | Covariates Support:
Past-observed/
Future-known/
Static | Probabilistic Forecasting:
Sampled/
Distribution Parameters | Training & Forecasting on Multiple Series |
|--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|--------------------------------------------------------------|--------------------------------------------------------------------------|--------------------------------------------------------------------------|-------------------------------------------|
| **Baseline Models**
([LocalForecastingModel](https://unit8co.github.io/darts/userguide/covariates.html#local-forecasting-models-lfms)) | | | | | |
| [NaiveMean](https://unit8co.github.io/darts/generated_api/darts.models.forecasting.baselines.html#darts.models.forecasting.baselines.NaiveMean) | | ✅ ✅ | 🔴 🔴 🔴 | 🔴 🔴 | 🔴 |
| [NaiveSeasonal](https://unit8co.github.io/darts/generated_api/darts.models.forecasting.baselines.html#darts.models.forecasting.baselines.NaiveSeasonal) | | ✅ ✅ | 🔴 🔴 🔴 | 🔴 🔴 | 🔴 |
| [NaiveDrift](https://unit8co.github.io/darts/generated_api/darts.models.forecasting.baselines.html#darts.models.forecasting.baselines.NaiveDrift) | | ✅ ✅ | 🔴 🔴 🔴 | 🔴 🔴 | 🔴 |
| [NaiveMovingAverage](https://unit8co.github.io/darts/generated_api/darts.models.forecasting.baselines.html#darts.models.forecasting.baselines.NaiveMovingAverage) | | ✅ ✅ | 🔴 🔴 🔴 | 🔴 🔴 | 🔴 |
| **Statistical / Classic Models**
([LocalForecastingModel](https://unit8co.github.io/darts/userguide/covariates.html#local-forecasting-models-lfms)) | | | | | |
| [ARIMA](https://unit8co.github.io/darts/generated_api/darts.models.forecasting.arima.html#darts.models.forecasting.arima.ARIMA) | | ✅ 🔴 | 🔴 ✅ 🔴 | ✅ 🔴 | 🔴 |
| [VARIMA](https://unit8co.github.io/darts/generated_api/darts.models.forecasting.varima.html#darts.models.forecasting.varima.VARIMA) | | 🔴 ✅ | 🔴 ✅ 🔴 | ✅ 🔴 | 🔴 |
| [ExponentialSmoothing](https://unit8co.github.io/darts/generated_api/darts.models.forecasting.exponential_smoothing.html#darts.models.forecasting.exponential_smoothing.ExponentialSmoothing) | | ✅ 🔴 | 🔴 🔴 🔴 | ✅ 🔴 | 🔴 |
| [Theta](https://unit8co.github.io/darts/generated_api/darts.models.forecasting.theta.html#darts.models.forecasting.theta.Theta) and [FourTheta](https://unit8co.github.io/darts/generated_api/darts.models.forecasting.theta.html#darts.models.forecasting.theta.FourTheta) | [Theta paper](https://robjhyndman.com/papers/Theta.pdf) & [4 Theta source](https://github.com/Mcompetitions/M4-methods/blob/master/4Theta%20method.R) | ✅ 🔴 | 🔴 🔴 🔴 | 🔴 🔴 | 🔴 |
| [Prophet](https://unit8co.github.io/darts/generated_api/darts.models.forecasting.prophet_model.html#darts.models.forecasting.prophet_model.Prophet) | [Prophet repo](https://github.com/facebook/prophet) | ✅ 🔴 | 🔴 ✅ 🔴 | ✅ 🔴 | 🔴 |
| [FFT](https://unit8co.github.io/darts/generated_api/darts.models.forecasting.fft.html#darts.models.forecasting.fft.FFT) (Fast Fourier Transform) | | ✅ 🔴 | 🔴 🔴 🔴 | 🔴 🔴 | 🔴 |
| [KalmanForecaster](https://unit8co.github.io/darts/generated_api/darts.models.forecasting.kalman_forecaster.html#darts.models.forecasting.kalman_forecaster.KalmanForecaster) using the Kalman filter and N4SID for system identification | [N4SID paper](https://people.duke.edu/~hpgavin/SystemID/References/VanOverschee-Automatica-1994.pdf) | ✅ ✅ | 🔴 ✅ 🔴 | ✅ 🔴 | 🔴 |
| [TBATS](https://unit8co.github.io/darts/generated_api/darts.models.forecasting.sf_tbats.html#darts.models.forecasting.sf_tbats.TBATS) | [TBATS paper](https://robjhyndman.com/papers/ComplexSeasonality.pdf) | ✅ 🔴 | 🔴 ✅ 🔴 | ✅ ✅ | 🔴 |
| [Croston](https://unit8co.github.io/darts/generated_api/darts.models.forecasting.sf_croston.html#darts.models.forecasting.sf_croston.Croston) method | | ✅ 🔴 | 🔴 ✅ 🔴 | ✅ ✅ | 🔴 |
| [StatsForecastModel](https://unit8co.github.io/darts/generated_api/darts.models.forecasting.sf_model.html#darts.models.forecasting.sf_model.StatsForecastModel) wrapper around any [StatsForecast](https://nixtlaverse.nixtla.io/statsforecast/index.html#models) model | [Nixtla's statsforecast](https://github.com/Nixtla/statsforecast) | ✅ 🔴 | 🔴 ✅ 🔴 | ✅ ✅ | 🔴 |
| [AutoARIMA](https://unit8co.github.io/darts/generated_api/darts.models.forecasting.sf_auto_arima.html#darts.models.forecasting.sf_auto_arima.AutoARIMA) | [Nixtla's statsforecast](https://github.com/Nixtla/statsforecast) | ✅ 🔴 | 🔴 ✅ 🔴 | ✅ ✅ | 🔴 |
| [AutoETS](https://unit8co.github.io/darts/generated_api/darts.models.forecasting.sf_auto_ets.html#darts.models.forecasting.sf_auto_ets.AutoETS) | [Nixtla's statsforecast](https://github.com/Nixtla/statsforecast) | ✅ 🔴 | 🔴 ✅ 🔴 | ✅ ✅ | 🔴 |
| [AutoCES](https://unit8co.github.io/darts/generated_api/darts.models.forecasting.sf_auto_ces.html#darts.models.forecasting.sf_auto_ces.AutoCES) | [Nixtla's statsforecast](https://github.com/Nixtla/statsforecast) | ✅ 🔴 | 🔴 ✅ 🔴 | ✅ ✅ | 🔴 |
| [AutoMFLES](https://unit8co.github.io/darts/generated_api/darts.models.forecasting.sf_auto_mfles.html#darts.models.forecasting.sf_auto_mfles.AutoMFLES) | [Nixtla's statsforecast](https://github.com/Nixtla/statsforecast) | ✅ 🔴 | 🔴 ✅ 🔴 | ✅ ✅ | 🔴 |
| [AutoTBATS](https://unit8co.github.io/darts/generated_api/darts.models.forecasting.sf_auto_tbats.html#darts.models.forecasting.sf_auto_tbats.AutoTBATS) | [Nixtla's statsforecast](https://github.com/Nixtla/statsforecast) | ✅ 🔴 | 🔴 ✅ 🔴 | ✅ ✅ | 🔴 |
| [AutoTheta](https://unit8co.github.io/darts/generated_api/darts.models.forecasting.sf_auto_theta.html#darts.models.forecasting.sf_auto_theta.AutoTheta) | [Nixtla's statsforecast](https://github.com/Nixtla/statsforecast) | ✅ 🔴 | 🔴 ✅ 🔴 | ✅ ✅ | 🔴 |
| [MultivariateModel](https://unit8co.github.io/darts/generated_api/darts.models.forecasting.multivariate_model.html#darts.models.forecasting.multivariate_model.MultivariateModel) | | ✅ ✅ | 🔴 ✅ 🔴 | ✅ ✅ | 🔴 |
| **Global Baseline Models**
([GlobalForecastingModel](https://unit8co.github.io/darts/userguide/covariates.html#global-forecasting-models-gfms)) | | | | | |
| [GlobalNaiveAggregate](https://unit8co.github.io/darts/generated_api/darts.models.forecasting.global_baseline_models.html#darts.models.forecasting.global_baseline_models.GlobalNaiveAggregate) | | ✅ ✅ | 🔴 🔴 🔴 | 🔴 🔴 | ✅ |
| [GlobalNaiveDrift](https://unit8co.github.io/darts/generated_api/darts.models.forecasting.global_baseline_models.html#darts.models.forecasting.global_baseline_models.GlobalNaiveDrift) | | ✅ ✅ | 🔴 🔴 🔴 | 🔴 🔴 | ✅ |
| [GlobalNaiveSeasonal](https://unit8co.github.io/darts/generated_api/darts.models.forecasting.global_baseline_models.html#darts.models.forecasting.global_baseline_models.GlobalNaiveSeasonal) | | ✅ ✅ | 🔴 🔴 🔴 | 🔴 🔴 | ✅ |
| **Regression Models**
([GlobalForecastingModel](https://unit8co.github.io/darts/userguide/covariates.html#global-forecasting-models-gfms)) | | | | | |
| [SKLearnModel](https://unit8co.github.io/darts/generated_api/darts.models.forecasting.sklearn_model.html#darts.models.forecasting.sklearn_model.SKLearnModel): wrapper around any scikit-learn-like regression model | | ✅ ✅ | ✅ ✅ ✅ | 🔴 🔴 | ✅ |
| [LinearRegressionModel](https://unit8co.github.io/darts/generated_api/darts.models.forecasting.linear_regression_model.html#darts.models.forecasting.linear_regression_model.LinearRegressionModel) | | ✅ ✅ | ✅ ✅ ✅ | ✅ ✅ | ✅ |
| [RandomForestModel](https://unit8co.github.io/darts/generated_api/darts.models.forecasting.random_forest.html#darts.models.forecasting.random_forest.RandomForestModel) | | ✅ ✅ | ✅ ✅ ✅ | 🔴 🔴 | ✅ |
| [CatBoostModel](https://unit8co.github.io/darts/generated_api/darts.models.forecasting.catboost_model.html#darts.models.forecasting.catboost_model.CatBoostModel) | | ✅ ✅ | ✅ ✅ ✅ | ✅ ✅ | ✅ |
| [LightGBMModel](https://unit8co.github.io/darts/generated_api/darts.models.forecasting.lgbm.html#darts.models.forecasting.lgbm.LightGBMModel) | | ✅ ✅ | ✅ ✅ ✅ | ✅ ✅ | ✅ |
| [XGBModel](https://unit8co.github.io/darts/generated_api/darts.models.forecasting.xgboost.html#darts.models.forecasting.xgboost.XGBModel) | | ✅ ✅ | ✅ ✅ ✅ | ✅ ✅ | ✅ |
| **PyTorch (Lightning)-based Models**
([GlobalForecastingModel](https://unit8co.github.io/darts/userguide/covariates.html#global-forecasting-models-gfms)) | | | | | |
| [RNNModel](https://unit8co.github.io/darts/generated_api/darts.models.forecasting.rnn_model.html#darts.models.forecasting.rnn_model.RNNModel) (incl. LSTM and GRU); equivalent to DeepAR in its probabilistic version | [DeepAR paper](https://arxiv.org/abs/1704.04110) | ✅ ✅ | 🔴 ✅ 🔴 | ✅ ✅ | ✅ |
| [BlockRNNModel](https://unit8co.github.io/darts/generated_api/darts.models.forecasting.block_rnn_model.html#darts.models.forecasting.block_rnn_model.BlockRNNModel) (incl. LSTM and GRU) | | ✅ ✅ | ✅ ✅ ✅ | ✅ ✅ | ✅ |
| [NBEATSModel](https://unit8co.github.io/darts/generated_api/darts.models.forecasting.nbeats.html#darts.models.forecasting.nbeats.NBEATSModel) | [N-BEATS paper](https://arxiv.org/abs/1905.10437) | ✅ ✅ | ✅ 🔴 🔴 | ✅ ✅ | ✅ |
| [NHiTSModel](https://unit8co.github.io/darts/generated_api/darts.models.forecasting.nhits.html#darts.models.forecasting.nhits.NHiTSModel) | [N-HiTS paper](https://arxiv.org/abs/2201.12886) | ✅ ✅ | ✅ 🔴 🔴 | ✅ ✅ | ✅ |
| [TCNModel](https://unit8co.github.io/darts/generated_api/darts.models.forecasting.tcn_model.html#darts.models.forecasting.tcn_model.TCNModel) | [TCN paper](https://arxiv.org/abs/1803.01271), [DeepTCN paper](https://arxiv.org/abs/1906.04397), [blog post](https://medium.com/unit8-machine-learning-publication/temporal-convolutional-networks-and-forecasting-5ce1b6e97ce4) | ✅ ✅ | ✅ 🔴 🔴 | ✅ ✅ | ✅ |
| [TransformerModel](https://unit8co.github.io/darts/generated_api/darts.models.forecasting.transformer_model.html#darts.models.forecasting.transformer_model.TransformerModel) | | ✅ ✅ | ✅ 🔴 🔴 | ✅ ✅ | ✅ |
| [TFTModel](https://unit8co.github.io/darts/generated_api/darts.models.forecasting.tft_model.html#darts.models.forecasting.tft_model.TFTModel) (Temporal Fusion Transformer) | [TFT paper](https://arxiv.org/pdf/1912.09363.pdf), [PyTorch Forecasting](https://pytorch-forecasting.readthedocs.io/en/latest/models.html) | ✅ ✅ | ✅ ✅ ✅ | ✅ ✅ | ✅ |
| [DLinearModel](https://unit8co.github.io/darts/generated_api/darts.models.forecasting.dlinear.html#darts.models.forecasting.dlinear.DLinearModel) | [DLinear paper](https://arxiv.org/pdf/2205.13504.pdf) | ✅ ✅ | ✅ ✅ ✅ | ✅ ✅ | ✅ |
| [NLinearModel](https://unit8co.github.io/darts/generated_api/darts.models.forecasting.nlinear.html#darts.models.forecasting.nlinear.NLinearModel) | [NLinear paper](https://arxiv.org/pdf/2205.13504.pdf) | ✅ ✅ | ✅ ✅ ✅ | ✅ ✅ | ✅ |
| [TiDEModel](https://unit8co.github.io/darts/generated_api/darts.models.forecasting.tide_model.html#darts.models.forecasting.tide_model.TiDEModel) | [TiDE paper](https://arxiv.org/pdf/2304.08424.pdf) | ✅ ✅ | ✅ ✅ ✅ | ✅ ✅ | ✅ |
| [TSMixerModel](https://unit8co.github.io/darts/generated_api/darts.models.forecasting.tsmixer_model.html#darts.models.forecasting.tsmixer_model.TSMixerModel) | [TSMixer paper](https://arxiv.org/pdf/2303.06053.pdf), [PyTorch Implementation](https://github.com/ditschuk/pytorch-tsmixer) | ✅ ✅ | ✅ ✅ ✅ | ✅ ✅ | ✅ |
| [NeuralForecastModel](https://unit8co.github.io/darts/generated_api/darts.models.forecasting.nf_model.html#darts.models.forecasting.nf_model.NeuralForecastModel): wrapper around any [NeuralForecast](https://nixtlaverse.nixtla.io/neuralforecast/docs/capabilities/overview.html) base model | [NeuralForecast Documentation](https://nixtlaverse.nixtla.io/neuralforecast/docs/) | ✅ ✅ | ✅ ✅ ✅ | ✅ ✅ | ✅ |
| **Foundation Models**
([GlobalForecastingModel](https://unit8co.github.io/darts/userguide/covariates.html#global-forecasting-models-gfms)): No training required | | | | | |
| [Chronos2Model](https://unit8co.github.io/darts/generated_api/darts.models.forecasting.chronos2_model.html#darts.models.forecasting.chronos2_model.Chronos2Model) | [Chronos-2 report](https://arxiv.org/abs/2510.15821), [Amazon blog post](https://www.amazon.science/blog/introducing-chronos-2-from-univariate-to-universal-forecasting) | ✅ ✅ | ✅ ✅ 🔴 | ✅ ✅ | ✅ |
| [TimesFM2p5Model](https://unit8co.github.io/darts/generated_api/darts.models.forecasting.timesfm2p5_model.html#darts.models.forecasting.timesfm2p5_model.TimesFM2p5Model) | [TimesFM 1.0 paper](https://arxiv.org/abs/2310.10688), [Google blog post](https://research.google/blog/a-decoder-only-foundation-model-for-time-series-forecasting) | ✅ ✅ | 🔴 🔴 🔴 | ✅ ✅ | ✅ |
| [TimesFM3Model](https://unit8co.github.io/darts/generated_api/darts.models.forecasting.timesfm3_model.html#darts.models.forecasting.timesfm3_model.TimesFM3Model) | [TimesFM 1.0 paper](https://arxiv.org/abs/2310.10688), [TimesFM 3.0 model card](https://huggingface.co/google/timesfm-3.0-pytorch) | ✅ ✅ | ✅ ✅ 🔴 | ✅ ✅ | ✅ |
| [TiRexModel](https://unit8co.github.io/darts/generated_api/darts.models.forecasting.tirex_model.html#darts.models.forecasting.tirex_model.TiRexModel) | [TiRex paper](https://arxiv.org/abs/2505.23719), [TiRex GitHub](https://github.com/NX-AI/tirex) | ✅ ✅ | 🔴 🔴 🔴 | ✅ ✅ | ✅ |
| [TiRex2Model](https://unit8co.github.io/darts/generated_api/darts.models.forecasting.tirex2_model.html#darts.models.forecasting.tirex2_model.TiRex2Model) | [TiRex-2 paper](https://arxiv.org/abs/2607.01204), [TiRex-2 GitHub](https://github.com/NX-AI/tirex-2) | ✅ ✅ | ✅ ✅ 🔴 | ✅ ✅ | ✅ |
| [PatchTSTFMModel](https://unit8co.github.io/darts/generated_api/darts.models.forecasting.patchtst_fm_model.html#darts.models.forecasting.patchtst_fm_model.PatchTSTFMModel) | [PatchTST-FM paper](https://arxiv.org/abs/2602.06909), [PatchTST-FM GitHub](https://github.com/ibm-granite/granite-tsfm) | ✅ ✅ | 🔴 🔴 🔴 | ✅ ✅ | ✅ |
| [T0Model](https://unit8co.github.io/darts/generated_api/darts.models.forecasting.t0_model.html#darts.models.forecasting.t0_model.T0Model) | [T0 report](https://www.theforecastingcompany.com/papers/files/t0-technical-report.pdf), [tfc-t0 GitHub](https://github.com/theforecastingcompany/tfc-t0) | ✅ ✅ | ✅ ✅ 🔴 | ✅ ✅ | ✅ |
| **Ensemble Models**
([GlobalForecastingModel](https://unit8co.github.io/darts/userguide/covariates.html#global-forecasting-models-gfms)): Model support is dependent on ensembled forecasting models and the ensemble model itself | | | | | |
| [NaiveEnsembleModel](https://unit8co.github.io/darts/generated_api/darts.models.forecasting.naive_ensemble_model.html#darts.models.forecasting.naive_ensemble_model.NaiveEnsembleModel) | | ✅ ✅ | ✅ ✅ ✅ | ✅ ✅ | ✅ |
| [RegressionEnsembleModel](https://unit8co.github.io/darts/generated_api/darts.models.forecasting.regression_ensemble_model.html#darts.models.forecasting.regression_ensemble_model.RegressionEnsembleModel) | | ✅ ✅ | ✅ ✅ ✅ | ✅ ✅ | ✅ |
| **Conformal Models**
([GlobalForecastingModel](https://unit8co.github.io/darts/userguide/covariates.html#global-forecasting-models-gfms)): Model support is dependent on the forecasting model used | | | | | |
| [ConformalNaiveModel](https://unit8co.github.io/darts/generated_api/darts.models.forecasting.conformal_models.html#darts.models.forecasting.conformal_models.ConformalNaiveModel) | [Conformalized Prediction](https://arxiv.org/pdf/1905.03222) | ✅ ✅ | ✅ ✅ ✅ | ✅ ✅ | ✅ |
| [ConformalQRModel](https://unit8co.github.io/darts/generated_api/darts.models.forecasting.conformal_models.html#darts.models.forecasting.conformal_models.ConformalQRModel) | [Conformalized Quantile Regression](https://arxiv.org/pdf/1905.03222) | ✅ ✅ | ✅ ✅ ✅ | ✅ ✅ | ✅ |
**Classification Models:** Classification models in Darts are designed to predict categorical class labels, enabling effective time series labeling and future class prediction. These models are perfect for scenarios where identifying distinct categories or states over time is crucial.
| Model | Sources | Target Series Support:
Univariate/
Multivariate | Covariates Support:
Past-observed/
Future-known/
Static | Probabilistic Forecasting:
Sampled/
Distribution Parameters | Training & Forecasting on Multiple Series |
|----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|---------|--------------------------------------------------------------|--------------------------------------------------------------------------|--------------------------------------------------------------------------|-------------------------------------------|
| **Regression Models**
([GlobalForecastingModel](https://unit8co.github.io/darts/userguide/covariates.html#global-forecasting-models-gfms)) | | | | | |
| [SKLearnClassifierModel](https://unit8co.github.io/darts/generated_api/darts.models.forecasting.sklearn_model.html#darts.models.forecasting.sklearn_model.SKLearnClassifierModel): wrapper around any scikit-learn-like classification model | | ✅ ✅ | ✅ ✅ ✅ | ✅ ✅ | ✅ |
| [CatBoostClassifierModel](https://unit8co.github.io/darts/generated_api/darts.models.forecasting.catboost_model.html#darts.models.forecasting.catboost_model.CatBoostClassifierModel) | | ✅ ✅ | ✅ ✅ ✅ | ✅ ✅ | ✅ |
| [LightGBMClassifierModel](https://unit8co.github.io/darts/generated_api/darts.models.forecasting.lgbm.html#darts.models.forecasting.lgbm.LightGBMClassifierModel) | | ✅ ✅ | ✅ ✅ ✅ | ✅ ✅ | ✅ |
| [XGBClassifierModel](https://unit8co.github.io/darts/generated_api/darts.models.forecasting.xgboost.html#darts.models.forecasting.xgboost.XGBClassifierModel) | | ✅ ✅ | ✅ ✅ ✅ | ✅ ✅ | ✅ |
## Community & Contact
Anyone is welcome to join our [Gitter room](https://gitter.im/u8darts/darts) to ask questions, make proposals,
discuss use-cases, and more. If you spot a bug or have suggestions, GitHub issues are also welcome.
If what you want to tell us is not suitable for Gitter or Github,
feel free to send us an email at darts@unit8.co for
darts related matters or info@unit8.co for any other
inquiries.
## Contribute
The development is ongoing, and we welcome suggestions, pull requests and issues on GitHub.
All contributors will be acknowledged on the
[change log page](https://github.com/unit8co/darts/blob/master/CHANGELOG.md).
Before working on a contribution (a new feature or a fix),
[check our contribution guidelines](https://github.com/unit8co/darts/blob/master/CONTRIBUTING.md).
## Citation
If you are using Darts in your scientific work, we would appreciate citations to the following JMLR paper.
[Darts: User-Friendly Modern Machine Learning for Time Series](https://www.jmlr.org/papers/v23/21-1177.html)
Bibtex entry (or copy directly via GitHub's repository citation):
```
@article{Herzen_Darts_User-Friendly_Modern_2022,
author = {Herzen, Julien and Lässig, Francesco and Piazzetta, Samuele Giuliano and Neuer, Thomas and Tafti, Léo and Raille, Guillaume and Van Pottelbergh, Tomas and Pasieka, Marek and Skrodzki, Andrzej and Huguenin, Nicolas and Dumonal, Maxime and Kościsz, Jan and Bader, Dennis and Gusset, Frédérick and Benheddi, Mounir and Williamson, Camila and Kosinski, Michal and Petrik, Matej and Grosch, Gaël},
journal = {Journal of Machine Learning Research},
number = {124},
pages = {1--6},
title = {{Darts: User-Friendly Modern Machine Learning for Time Series}},
url = {https://jmlr.org/papers/v23/21-1177.html},
volume = {23},
year = {2022}
}
```
---
### Userguide Forecasting Overview (`docs/userguide/forecasting_overview.md`)
# Overview of Forecasting Models
Forecasting models are models that can produce predictions about future values of some time series, given the history of this series.
The forecasting models in Darts are [listed on the README](https://github.com/unit8co/darts#forecasting-models). They have different capabilities and features. For example, some models work on multidimensional series, return probabilistic forecasts, or accept other kinds of external *covariates* data in input.
Below, we give an overview of what these features mean.
## Generalities
All forecasting models work in the same way: first they are built (taking some hyper-parameters in argument), then they are fit on one or several series
by calling the `fit()` function, and finally they are used to obtain one or several forecasts by calling the `predict()` function.
**Example:**
```python
from darts.models import NaiveSeasonal
naive_model = NaiveSeasonal(K=1) # init
naive_model.fit(train) # fit
naive_forecast = naive_model.predict(n=36) # predict
```
The argument `n` of `predict()` indicates the number of time stamps to predict.
When `fit()` is provided with only one training `TimeSeries`, this series is stored, and `predict()` will return forecasts for this series.
On the other hand, some models support calling `fit()` on multiple time series (a `Sequence[TimeSeries]`). In such cases, one or several series must
be provided to `predict()`, and the model will produce forecasts for this/these time series.
**Example:**
```python
from darts.models import NBEATSModel
model = NBEATSModel(input_chunk_length=24, # init
output_chunk_length=12)
model.fit([series1, series2]) # fit on two series
forecast = model.predict(series=[series3, series4], n=36) # predict potentially different series
```
Furthermore, we define the following types of time series consumed by the models:
* **Target series:** the series that we are interested in forecasting.
* **Covariate series:** some other series that we are not interested in forecasting, but that can provide valuable inputs to the forecasting model.
## Saving and Loading Models
If you wish to save a particular model and use it elsewhere or at a later point in time, darts can achieve that. It leverages pickle and in the case of Torch models relies on saving PyTorch Lightning trainer checkpoints.
All forecasting models support saving the model on the filesystem, by calling the `save()` function, which saves that particular `ForecastingModel` object instance. When the model is to be used again, the method `load()` can be used. Please note that the methods `save_model()` and `load_model()` are deprecated.
**Security:** Loading uses safe-by-default deserialization to mitigate malicious pickle files (CWE-502). Use `trusted=True` only for files you fully trust, or register custom classes/functions via `add_safe_globals` / `with safe_globals(...)`. See the [safe model loading guide](safe_model_loading.md) for details.
**Example:**
```python
from darts.models import SKLearnModel
model = SKLearnModel(lags=4)
model.save("my_model.pkl")
model_loaded = SKLearnModel.load("my_model.pkl")
```
The parameter `path` specifies a path or file handle under which to save the model at its current state. If no `path` is specified, the model is automatically
saved under ``"{ModelClass}_{YYYY-mm-dd_HH:MM:SS}.pkl"``. E.g., ``"SKLearnModel_2020-01-01_12:00:00.pkl"``.
Optionally there is also pickle specific keyword arguments `protocol`, `fix_imports` and `buffer_callback`.
More info: [pickle.dump()](https://docs.python.org/3/library/pickle.html?highlight=dump#pickle.dump)
With torch models, the model parameters and the training state are saved. We use the trainer to save the model checkpoint.
**Example:**
```python
from darts.models import NBEATSModel
model = NBEATSModel(input_chunk_length=24,
output_chunk_length=12)
model.save("my_model.pt")
model_loaded = NBEATSModel.load("my_model.pt")
```
Private methods for torch models used under the hood:
* **save_checkpoint:** In addition, we need to use PTL save_checkpoint() to properly save the trainer and model. It is used to be able to save a snapshot of the model mid-training, and then be able to retrieve the model later.
* **load_from_checkpoint:** returns a model checkpointed during training (by default the one with lowest validation loss).
## Support for multivariate series
Some models support multivariate time series. This means that the target (and potential covariates) series provided to the model
during fit and predict stage can have multiple dimensions. The model will then in turn produce multivariate forecasts.
Here is an example, using a `KalmanForecaster` to forecast a single multivariate series made of 2 components:
```python
# use darts plotting style
from darts import set_option
set_option("plotting.use_darts_style", True)
import darts.utils.timeseries_generation as tg
from darts.models import KalmanForecaster
import matplotlib.pyplot as plt
series1 = tg.sine_timeseries(value_frequency=0.05, length=100) + 0.1 * tg.gaussian_timeseries(length=100)
series2 = tg.sine_timeseries(value_frequency=0.02, length=100) + 0.2 * tg.gaussian_timeseries(length=100)
multivariate_series = series1.stack(series2)
model = KalmanForecaster(dim_x=4)
model.fit(multivariate_series)
pred = model.predict(n=50, num_samples=100)
plt.figure(figsize=(8,6))
multivariate_series.plot(lw=3)
pred.plot(lw=3, label='forecast')
```
These models are shown with a "✅" under the `Multivariate` column on the [model list](https://github.com/unit8co/darts#forecasting-models).
## Handling multiple series
Some models support being fit on multiple time series. To do this, it is enough to simply provide a Python `Sequence` of `TimeSeries` (for instance a list of `TimeSeries`) to `fit()`. When a model is fit this way, the `predict()` function will expect the argument `series` to be set, containing
one or several `TimeSeries` (i.e., a single or a `Sequence` of `TimeSeries`) that need to be forecasted.
The advantage of training on multiple series is that a single model can be exposed to more patterns occurring across all series in the training dataset. That can often be beneficial, especially for larger models with more capacity.
In turn, the advantage of having `predict()` providing forecasts for potentially several series at once is that the computation can often be batched and vectorized across the multiple series, which is computationally faster than calling `predict()` multiple times on isolated series.
These models are shown with a "✅" under the `Multiple-series training` column on the [model list](https://github.com/unit8co/darts#forecasting-models).
You can also find out programmatically, whether a model supports multiple series.
```python
from darts.models import SKLearnModel
from darts.models.forecasting.forecasting_model import GlobalForecastingModel
# when True, multiple time series are supported
supports_multi_ts = issubclass(SKLearnModel, GlobalForecastingModel)
```
[This article on multi-series forecasting](https://medium.com/unit8-machine-learning-publication/training-forecasting-models-on-multiple-time-series-with-darts-dc4be70b1844) provides more explanations about training models on multiple series.
## Support for Covariates
Some models support *covariate series*. Covariate series are time series that the models can take as inputs, but will not forecast.
We distinguish between *past covariates* and *future covariates*:
* Past covariates are covariate time series whose values are **not** known into the future at prediction time. Those can for instance represent signals that have to be measured and are not known upfront. Models do not use the future values of `past_covariates` when making forecasts.
* Future covariates are covariate time series whose values are known into the future at prediction time (up until the forecast horizon). These can represent signals such as calendar information, holidays, weather forecasts, etc. Models that accept `future_covariates` will consume the future values (up to the forecast horizon) when making forecasts.
Past and future covariates can be used by providing respectively `past_covariates` and `future_covariates` arguments to `fit()` and `predict()`.
When a model is trained on multiple target series, one covariate has to be provided per target series. The covariate series themselves can be multivariate
and contain multiple "covariate dimensions"; see the [TimeSeries guide](https://unit8co.github.io/darts/userguide/timeseries.html) for how to build multivariate series.
It is not necessary to worry about the covariates series having the exact right time spans (e.g., so that the last timestamp of future covariates matches the forecast horizon). Darts takes care of slicing the covariates behind the scenes, based on the time axes of the target(s) and that of the covariates.
Models supporting past (resp. future) covariates are indicated with a "✅" under the `Past-observed covariates support` (resp. `Future-known covariates support`) columns on the [model list](https://github.com/unit8co/darts#forecasting-models),
To know more about covariates, please refer to the [covariates section of the user guide](https://unit8co.github.io/darts/userguide/covariates.html).
In addition, you can have a look at [this article on covariates](https://medium.com/unit8-machine-learning-publication/time-series-forecasting-using-past-and-future-external-data-with-darts-1f0539585993) for some examples of how to use past and future covariates.
## Probabilistic forecasts
Most models in Darts can produce probabilistic forecasts. For these models, the `TimeSeries` returned by `predict()` will be probabilistic, and contain a certain number of Monte Carlo samples representing the forecasted distribution over time and components. The number of samples can be directly determined by the argument `num_samples` of the `predict()` function (leaving `num_samples=1` will return a deterministic `TimeSeries`).
Models supporting probabilistic forecasts are indicated with a "✅" in the `Probabilistic` column on the [model list](https://github.com/unit8co/darts#forecasting-models).
The actual probabilistic distribution of the forecasts depends on the model.
Some models such as ARIMA, Exponential Smoothing, (T)BATS or KalmanForecaster make normality assumptions and the resulting distribution is a Gaussian with time-dependent parameters. For example:
```python
from darts.datasets import AirPassengersDataset
from darts import TimeSeries
from darts.models import ExponentialSmoothing
series = AirPassengersDataset().load()
train, val = series[:-36], series[-36:]
model = ExponentialSmoothing()
model.fit(train)
pred = model.predict(n=36, num_samples=500)
series.plot()
pred.plot(label='forecast')
```
### Probabilistic neural networks
All neural networks (torch-based models) in Darts have a rich support to estimate different kinds of probability distributions.
When creating the model, it is possible to provide one of the *likelihood models* available in [darts.utils.likelihood_models.torch](https://unit8co.github.io/darts/generated_api/darts.utils.likelihood_models.torch.html), which determine the distribution that will be estimated by the model.
In such cases, the model will output the parameters of the distribution, and it will be trained by minimising the negative log-likelihood of the training samples.
Most of the likelihood models also support prior values for the distribution's parameters, in which case the training loss is regularized by a Kullback-Leibler divergence term pushing the resulting distribution in the direction of the distribution specified by the prior parameters.
The strength of this regularization term can also be specified when creating the likelihood model object.
For example, the code below trains a TCNModel to fit a Laplace distribution. So the neural network outputs 2 parameters (location and scale) of the Laplace distribution. We also specify a prior value of 0.1 on the scale parameter.
```python
from darts.datasets import AirPassengersDataset
from darts import TimeSeries
from darts.models import TCNModel
from darts.dataprocessing.transformers import Scaler
from darts.utils.likelihood_models.torch import LaplaceLikelihood
series = AirPassengersDataset().load()
train, val = series[:-36], series[-36:]
scaler = Scaler()
train = scaler.fit_transform(train)
val = scaler.transform(val)
series = scaler.transform(series)
model = TCNModel(input_chunk_length=30,
output_chunk_length=12,
likelihood=LaplaceLikelihood(prior_b=0.1))
model.fit(train, epochs=400)
pred = model.predict(n=36, num_samples=500)
series.plot()
pred.plot(label='forecast')
```
It is also possible to perform quantile regression (using arbitrary quantiles) with neural networks, by using [darts.utils.likelihood_models.torch.QuantileRegression](https://unit8co.github.io/darts/generated_api/darts.utils.likelihood_models.torch.html#darts.utils.likelihood_models.torch.QuantileRegression), in which case the network will be trained with the pinball loss. This produces an empirical non-parametric distribution, and it can often be a good option in practice, when one is not sure of the "real" distribution, or when fitting parametric likelihoods give poor results.
For example, the code snippet below is almost exactly the same as the preceding snippet; the only difference is that it now uses a `QuantileRegression` likelihood, which means that the neural network will be trained with a pinball loss, and its number of outputs will be dynamically configured to match the number of quantiles.
```python
from darts.datasets import AirPassengersDataset
from darts import TimeSeries
from darts.models import TCNModel
from darts.dataprocessing.transformers import Scaler
from darts.utils.likelihood_models.torch import QuantileRegression
series = AirPassengersDataset().load()
train, val = series[:-36], series[-36:]
scaler = Scaler()
train = scaler.fit_transform(train)
val = scaler.transform(val)
series = scaler.transform(series)
model = TCNModel(input_chunk_length=30,
output_chunk_length=12,
likelihood=QuantileRegression(quantiles=[0.01, 0.05, 0.2, 0.5, 0.8, 0.95, 0.99]))
model.fit(train, epochs=400)
pred = model.predict(n=36, num_samples=500)
series.plot()
pred.plot(label='forecast')
```
### Capturing model uncertainty using Monte Carlo Dropout
In Darts, dropout can also be used as an additional way to capture model uncertainty, following the approach described in [1]. This is sometimes referred to as *epistemic uncertainty*, and can be seen as a way to marginalize over a family of models represented by all the different dropout activation functions.
This feature is readily available for all deep learning models integrating some dropout (except RNN models - we refer to the dropout API reference documentations for a mention of models supporting this). It only requires to specify `mc_dropout=True` at prediction time. For example, the code below trains a TCN model (using the default MSE loss) with a dropout rate of 10%, and then produces a probabilistic forecasts using Monte Carlo Dropout:
```python
from darts.datasets import AirPassengersDataset
from darts import TimeSeries
from darts.models import TCNModel
from darts.dataprocessing.transformers import Scaler
from darts.utils.likelihood_models.torch import QuantileRegression
series = AirPassengersDataset().load()
train, val = series[:-36], series[-36:]
scaler = Scaler()
train = scaler.fit_transform(train)
val = scaler.transform(val)
series = scaler.transform(series)
model = TCNModel(input_chunk_length=30,
output_chunk_length=12,
dropout=0.1)
model.fit(train, epochs=400)
pred = model.predict(n=36, mc_dropout=True, num_samples=500)
series.plot()
pred.plot(label='forecast')
```
Monte Carlo Dropout can be combined with other likelihood estimation in Darts, which can be interpreted as a way to capture both epistemic and aleatoric uncertainty.
In that case, train the model with the desired `likelihood` (e.g. `QuantileRegression(...)`) so the model learns to predict the parameters of an aleatoric distribution, and at prediction time set:
- `mc_dropout=True`,
- `num_samples >> 1` (e.g. `500`),
- `predict_likelihood_parameters=False`.
Each Monte Carlo dropout pass yields one set of distribution parameters; Darts then draws one sample from that distribution per pass, so the resulting `num_samples` predictions reflect both epistemic uncertainty (different dropout masks) and aleatoric uncertainty (sampling from the predicted likelihood). You can then compute marginal quantiles, mean, std, etc. from those samples.
> Note: `mc_dropout=True` and `predict_likelihood_parameters=True` should not be combined. `predict_likelihood_parameters=True` is designed for use with `num_samples=1` to get the model's best estimate of the distribution parameters.
### Probabilistic regression models
Some regression models can also be configured to produce probabilistic forecasts too. At the time of writing, [LinearRegressionModel](https://unit8co.github.io/darts/generated_api/darts.models.forecasting.linear_regression_model.html), [LightGBMModel](https://unit8co.github.io/darts/generated_api/darts.models.forecasting.lgbm.html) and [XGBModel](https://unit8co.github.io/darts/generated_api/darts.models.forecasting.xgboost.html) support a `likelihood` argument. When set to `"poisson"` the model will fit a Poisson distribution, and when set to `"quantile"` the model will use the pinball loss to perform quantile regression (the quantiles themselves can be specified using the `quantiles` argument).
Example:
```python
from darts.datasets import AirPassengersDataset
from darts import TimeSeries
from darts.models import LinearRegressionModel
series = AirPassengersDataset().load()
train, val = series[:-36], series[-36:]
model = LinearRegressionModel(lags=30,
likelihood="quantile",
quantiles=[0.05, 0.1, 0.25, 0.5, 0.75, 0.9, 0.95])
model.fit(train)
pred = model.predict(n=36, num_samples=500)
series.plot()
pred.plot(label='forecast')
```
### Multi-step probabilistic forecasts: Scenarios vs Marginal Distributions
When calling `predict(n, num_samples=k)` with `k > 1`, the returned `TimeSeries` has shape `(time, components, samples)`. A common question is whether the individual sample trajectories along the `samples` dimension can be interpreted as **realistic, coherent temporal scenarios** (accounting for autocorrelation over time and cross-dimensional correlations), or whether they only represent **independent pointwise marginal distributions** at each time step.
The distinction primarily depends on whether the forecast is generated **autoregressively** or via **direct multi-step chunks**:
#### 1. Purely Autoregressive Rollouts (Scenario Generation)
For models operating purely autoregressively (e.g. global forecasting models with `output_chunk_length = 1`):
- Forecasts are generated step-by-step into the future.
- At time step $t$, a sample is drawn from the predicted distribution for each sample index $i \in \{1, \dots, k\}$.
- For each scenario $i$, the sampled value at $t$ is fed back into the model as "historic" input to forecast the distribution at $t+1$.
- **Temporal correlation:** Because each trajectory's future is conditioned on its own simulated past values, **autocorrelation is preserved across time**. The individual sample paths $i$ function as realistic Monte Carlo simulated scenarios.
- **Cross-component correlation:** The random noise/innovations injected at each step are sampled independently across components. However, cross-component interactions in the conditional mean are captured and propagated over time through the lagged features.
#### 2. Direct Multi-Step / Chunk Forecasting (Marginal Sampling)
For direct multi-step models (e.g. global forecasting models with `output_chunk_length > 1`):
- The model outputs the parameters of the distributions (or quantiles) for the entire output chunk simultaneously.
- When generating `num_samples` forecasts, samples are drawn **independently across all time steps and components** within the chunk.
- **Temporal correlation:** Within the output chunk, there is no modeled joint covariance or temporal correlation in the sampled noise. Consequently, individual sample trajectories do **not** represent realistic dynamic scenarios; rather, the collection of samples at each time step $t$ provides a Monte Carlo estimate of the **marginal distribution** at that specific point in time.
- **Beyond the output chunk length (`n > output_chunk_length`):** Models apply chunk-level autoregression, feeding the previously sampled chunk back into the model for the next chunk.
> **Tip:** If your use case requires realistic simulated paths over time (e.g., for downstream simulation, risk analysis, or optimization under uncertainty), prefer autoregressive models with `output_chunk_length=1` (or `multi_models=False` in regression models). If you primarily need accurate pointwise prediction intervals or quantiles, direct multi-step models are often faster and avoid compounding autoregressive errors.
[1] Yarin Gal, Zoubin Ghahramani, ["Dropout as a Bayesian Approximation: Representing Model Uncertainty in Deep Learning"](https://arxiv.org/abs/1506.02142)
---
### Source Examples (`docs/source/examples.rst`)
========
Examples
========
Here you will find some example notebooks to get more familiar with the Darts' API.
All the notebooks are also available in ``ipynb`` format
`directly on github `__.
Multiple Time Series, Pre-trained Models and Covariates
=======================================================
Example notebook on training with multiple time series, pre-trained models and using covariates:
.. toctree::
:maxdepth: 1
examples/01-multi-time-series-and-covariates.ipynb
Data (pre) Processing
=====================
Data processing example notebook:
.. toctree::
:maxdepth: 1
examples/02-data-processing.ipynb
Static Covariates
=================
Static covariates example notebook:
.. toctree::
:maxdepth: 1
examples/15-static-covariates.ipynb
Transfer Learning Tutorial
==========================
A self-contained notebook showcasing examples of training some bigger models
on large datasets of time series, and using such models to forecast new time
series that have not been seen during training:
.. toctree::
:maxdepth: 1
examples/14-transfer-learning.ipynb
Hierarchical Reconciliation
===========================
A self-contained notebook showcasing how to reconcile forecasts
for hierarchical time series, using the Australian domestic tourism dataset.
.. toctree::
:maxdepth: 1
examples/16-hierarchical-reconciliation.ipynb
Hyperparameter Optimization
===========================
An example showcasing how to find good forecasting models
with Darts using the Optuna library for hyperparameter optimization.
.. toctree::
:maxdepth: 1
examples/17-hyperparameter-optimization.ipynb
Regression Models
=================
Regression models example notebook:
.. toctree::
:maxdepth: 1
examples/20-SKLearnModel-examples.ipynb
Classification Models
=====================
Classification models example notebook:
.. toctree::
:maxdepth: 1
examples/24-SKLearnClassifierModel-examples.ipynb
Conformal Prediction
====================
Conformal prediction example notebook:
.. toctree::
:maxdepth: 1
examples/23-Conformal-Prediction-examples.ipynb
Fast Fourier Transform
======================
FFT model example notebook:
.. toctree::
:maxdepth: 1
examples/03-FFT-examples.ipynb
Recurrent Neural Networks
=========================
RNN model example notebook:
.. toctree::
:maxdepth: 1
examples/04-RNN-examples.ipynb
Temporal Convolutional Networks
===============================
TCN model example notebook:
.. toctree::
:maxdepth: 1
examples/05-TCN-examples.ipynb
Transformer Model
=================
Transformer model example notebook:
.. toctree::
:maxdepth: 1
examples/06-Transformer-examples.ipynb
N-BEATS Model
=============
N-BEATS model example notebook:
.. toctree::
:maxdepth: 1
examples/07-NBEATS-examples.ipynb
DeepAR Model
============
DeepAR model example notebook:
.. toctree::
:maxdepth: 1
examples/08-DeepAR-examples.ipynb
DeepTCN Model
=============
DeepTCN model example notebook:
.. toctree::
:maxdepth: 1
examples/09-DeepTCN-examples.ipynb
Temporal Fusion Transformer (TFT) Model
=======================================
TFT model example notebook:
.. toctree::
:maxdepth: 1
examples/13-TFT-examples.ipynb
TimeSeries Dense Encoder (TiDE) Model
=====================================
TiDE model example notebook:
.. toctree::
:maxdepth: 1
examples/18-TiDE-examples.ipynb
TimeSeries Mixer (TSMixer) Model
================================
TSMixer model example notebook:
.. toctree::
:maxdepth: 1
examples/21-TSMixer-examples.ipynb
NeuralForecast Model Adapter
============================
NeuralForecastModel example notebook:
.. toctree::
:maxdepth: 1
examples/26-NeuralForecast-examples.ipynb
Chronos-2 Model
===============
Foundation model example notebook:
.. toctree::
:maxdepth: 1
examples/25-FoundationModel-examples.ipynb
Torch and Foundation Model Fine-Tuning
======================================
Torch and Foundation model fine-tuning example notebook:
.. toctree::
:maxdepth: 1
examples/27-Torch-and-Foundation-Model-Fine-Tuning-examples.ipynb
Ensemble Models
===============
Ensemble models example notebook:
.. toctree::
:maxdepth: 1
examples/19-EnsembleModel-examples.ipynb
Explainability
==============
Explainability example notebook showcasing the use of Darts' explainability module for both PyTorch and SKLearn models:
.. toctree::
:maxdepth: 1
examples/28-Explainability-examples.ipynb
MLflow Integration
==================
MLflow integration example notebook:
.. toctree::
:maxdepth: 1
examples/29-MLflow-examples.ipynb
Kalman Filter Model
===================
Kalman filter model example notebook:
.. toctree::
:maxdepth: 1
examples/10-Kalman-filter-examples.ipynb
Gaussian Process Filter Model
=============================
Gaussian process filter model example notebook:
.. toctree::
:maxdepth: 1
examples/11-GP-filter-examples.ipynb
Anomaly Detection
=================
Anomaly detection example notebook:
.. toctree::
:maxdepth: 1
examples/22-anomaly-detection-examples.ipynb
Dynamic Time Warping (DTW)
==========================
Dynamic time warping example notebook:
.. toctree::
:maxdepth: 1
examples/12-Dynamic-Time-Warping-example.ipynb
---
### Source Index (`docs/source/index.rst`)
:notoc:
Time Series Made Easy in Python
===============================
.. image:: static/darts-logo-light.png
:alt: darts
:align: center
:width: 100%
:class: only-light
.. image:: static/darts-logo-dark.png
:alt: darts
:align: center
:width: 100%
:class: only-dark
.. raw:: html
|
**Darts** is a Python library for user-friendly **forecasting and anomaly detection** on time series.
It contains a variety of models, from classics such as ARIMA to deep neural networks.
All models can be used in the same way, using ``fit()`` and ``predict()`` functions, similar to scikit-learn.
The library also makes it easy to backtest models, combine the predictions of several models, and take external data into account.
Darts supports both **univariate and multivariate** time series and models, and offers extensive support for **probabilistic forecasting**.
.. grid:: 1 2 3 3
:gutter: 4
:padding: 2 2 0 0
:class-container: sd-text-center
.. grid-item-card:: Quickstart
:class-card: intro-card
:shadow: md
.. image:: static/grid_card_icons/quickstart.png
:class: sd-card-img-top only-light
:alt: Quickstart
.. image:: static/grid_card_icons_dark/quickstart.png
:class: sd-card-img-top only-dark
:alt: Quickstart
Introduction to Darts' main concepts and workflow
+++
.. button-ref:: quickstart/00-quickstart
:ref-type: doc
:click-parent:
:color: secondary
:expand:
To the quickstart guide
.. grid-item-card:: Models
:class-card: intro-card
:shadow: md
.. image:: static/grid_card_icons/models.png
:class: sd-card-img-top only-light
:alt: Models
.. image:: static/grid_card_icons_dark/models.png
:class: sd-card-img-top only-dark
:alt: Models
Available models and supported features
+++
.. button-ref:: forecasting-models
:ref-type: ref
:click-parent:
:color: secondary
:expand:
To the models table
.. grid-item-card:: API Reference
:class-card: intro-card
:shadow: md
.. image:: static/grid_card_icons/api.png
:class: sd-card-img-top only-light
:alt: API Reference
.. image:: static/grid_card_icons_dark/api.png
:class: sd-card-img-top only-dark
:alt: API Reference
Detailed API documentation with methods and parameters
+++
.. button-ref:: generated_api/darts
:ref-type: doc
:click-parent:
:color: secondary
:expand:
To the API reference
.. grid-item-card:: Examples
:class-card: intro-card
:shadow: md
.. image:: static/grid_card_icons/examples.png
:class: sd-card-img-top only-light
:alt: Examples
.. image:: static/grid_card_icons_dark/examples.png
:class: sd-card-img-top only-dark
:alt: Examples
Jupyter notebooks with basic and advanced tutorials
+++
.. button-ref:: examples
:ref-type: doc
:click-parent:
:color: secondary
:expand:
To the examples
.. grid-item-card:: User Guide
:class-card: intro-card
:shadow: md
.. image:: static/grid_card_icons/user-guide.png
:class: sd-card-img-top only-light
:alt: User Guide
.. image:: static/grid_card_icons_dark/user-guide.png
:class: sd-card-img-top only-dark
:alt: User Guide
In-depth information on key concepts and features
+++
.. button-ref:: userguide
:ref-type: doc
:click-parent:
:color: secondary
:expand:
To the user guide
.. grid-item-card:: How to Contribute
:class-card: intro-card
:shadow: md
.. image:: static/grid_card_icons/contribute.png
:class: sd-card-img-top only-light
:alt: Contribute
.. image:: static/grid_card_icons_dark/contribute.png
:class: sd-card-img-top only-dark
:alt: Contribute
Guidelines for contributing to the Darts library
+++
.. button-link:: https://github.com/unit8co/darts/blob/master/CONTRIBUTING.md
:click-parent:
:color: secondary
:expand:
To the contributing guide
----
High Level Introductions
-------------------------
* `Introductory Blog Post `_
* `Introduction video (25 minutes) `_
Articles on Selected Topics
---------------------------
* `Training Models on Multiple Time Series `_
* `Using Past and Future Covariates `_
* `Temporal Convolutional Networks and Forecasting `_
* `Probabilistic Forecasting `_
* `Transfer Learning for Time Series Forecasting `_
* `Hierarchical Forecast Reconciliation `_
Quick Install
-------------
We recommend to first setup a clean Python environment for your project with Python 3.11+ using your favorite tool
(`conda `_,
`venv `_, `virtualenv `_ with
or without `virtualenvwrapper `_).
Once your environment is set up you can install darts using pip:
.. code-block:: bash
pip install darts
For more details you can refer to our `installation instructions `_.
Example Usage
-------------
Forecasting
~~~~~~~~~~~
Create a ``TimeSeries`` object from a Pandas DataFrame, and split it in train/validation series:
.. code-block:: python
import pandas as pd
from darts import TimeSeries
# Read a pandas DataFrame
df = pd.read_csv("AirPassengers.csv", delimiter=",")
# Create a TimeSeries, specifying the time and value columns
series = TimeSeries.from_dataframe(df, "Month", "#Passengers")
# Set aside the last 36 months as a validation series
train, val = series[:-36], series[-36:]
Fit an exponential smoothing model, and make a (probabilistic) prediction over the validation series' duration:
.. code-block:: python
from darts.models import ExponentialSmoothing
model = ExponentialSmoothing()
model.fit(train)
prediction = model.predict(len(val), num_samples=1000)
Plot the median, 5th and 95th percentiles:
.. code-block:: python
import matplotlib.pyplot as plt
series.plot()
prediction.plot(label="forecast", low_quantile=0.05, high_quantile=0.95)
plt.legend()
.. image:: https://github.com/unit8co/darts/raw/master/static/images/example.png
:alt: darts forecast example
:align: center
Anomaly Detection
~~~~~~~~~~~~~~~~~
Load a multivariate series, trim it, keep 2 components, split train and validation sets:
.. code-block:: python
from darts.datasets import ETTh2Dataset
series = ETTh2Dataset().load()[:10000][["MUFL", "LULL"]]
train, val = series.split_before(0.6)
Build a k-means anomaly scorer, train it on the train set
and use it on the validation set to get anomaly scores:
.. code-block:: python
from darts.ad import KMeansScorer
scorer = KMeansScorer(k=2, window=5)
scorer.fit(train)
anom_score = scorer.score(val)
Build a binary anomaly detector and train it over train scores,
then use it over validation scores to get binary anomaly classification:
.. code-block:: python
from darts.ad import QuantileDetector
detector = QuantileDetector(high_quantile=0.99)
detector.fit(scorer.score(train))
binary_anom = detector.detect(anom_score)
Plot (shifting and scaling some of the series to make everything appear on the same figure):
.. code-block:: python
import matplotlib.pyplot as plt
series.plot()
(anom_score / 2. - 100).plot(label="computed anomaly score", c="orangered", lw=3)
(binary_anom * 45 - 150).plot(label="detected binary anomaly", lw=4)
.. image:: https://github.com/unit8co/darts/raw/master/static/images/example_ad.png
:alt: darts anomaly detection example
:align: center
Features
--------
* **Forecasting Models:** A large collection of forecasting models for regression as well as classification tasks; from statistical models (such as ARIMA) to deep learning models (such as N-BEATS). See the forecasting models table below.
* **Anomaly Detection:** The ``darts.ad`` module contains a collection of anomaly scorers, detectors and aggregators, which can all be combined to detect anomalies in time series. It is easy to wrap any of Darts forecasting or filtering models to build a fully fledged anomaly detection model that compares predictions with actuals. The ``PyODScorer`` makes it trivial to use PyOD detectors on time series.
* **Multivariate Support:** ``TimeSeries`` can be multivariate - i.e., contain multiple time-varying dimensions/columns instead of a single scalar value. Many models can consume and produce multivariate series.
* **Multiple Series Training (Global Models):** All machine learning based models (incl. all neural networks) support being trained on multiple (potentially multivariate) series. This can scale to large datasets too.
* **Probabilistic Support:** ``TimeSeries`` objects can (optionally) represent stochastic time series; this can for instance be used to get confidence intervals, and many models support different flavours of probabilistic forecasting (such as estimating parametric distributions or quantiles). Some anomaly detection scorers are also able to exploit these predictive distributions.
* **Conformal Prediction Support:** Our conformal prediction models allow to generate probabilistic forecasts with calibrated quantile intervals for any pre-trained global forecasting model.
* **Past and Future Covariates Support:** Many models in Darts support past-observed and/or future-known covariate (external data) time series as inputs for producing forecasts.
* **Static Covariates Support:** In addition to time-dependent data, ``TimeSeries`` can also contain static data for each dimension, which can be exploited by some models.
* **Hierarchical Reconciliation:** Darts offers transformers to perform reconciliation. These can make the forecasts add up in a way that respects the underlying hierarchy.
* **Regression Models:** It is possible to plug-in any scikit-learn compatible model to obtain forecasts as functions of lagged values of the target series and covariates.
* **Training with Sample Weights:** All global models support being trained with sample weights. They can be applied to each observation, forecasted time step and target column.
* **Forecast Start Shifting:** All global models support training and prediction on a shifted output window. This is useful for example for Day-Ahead Market forecasts, or when the covariates (or target series) are reported with a delay.
* **Explainability:** Darts has the ability to *explain* some forecasting models using SHAP values.
* **Data Processing:** Tools to easily apply (and revert) common transformations on time series data (scaling, filling missing values, differencing, boxcox, ...)
* **Metrics:** A variety of metrics for evaluating time series' goodness of fit; from R2-scores to Mean Absolute Scaled Error.
* **Backtesting:** Utilities for simulating historical forecasts, using moving time windows.
* **PyTorch Lightning Support:** All deep learning models are implemented using PyTorch Lightning, supporting among other things custom callbacks, GPUs/TPUs training and custom trainers.
* **MLflow Integration:** Integration with MLflow for automated tracking, comparing, and persisting of Darts forecasting model experiments. See the `MLflow quickstart notebook `_ for examples.
* **Filtering Models:** Darts offers three filtering models: ``KalmanFilter``, ``GaussianProcessFilter``, and ``MovingAverageFilter``, which allow to filter time series, and in some cases obtain probabilistic inferences of the underlying states/values.
* **Datasets:** The ``darts.datasets`` submodule contains some popular time series datasets for rapid and reproducible experimentation.
* **Compatibility with Multiple Backends:** ``TimeSeries`` objects can be created from and exported to various backends such as pandas, polars, numpy, pyarrow, xarray, and more, facilitating seamless integration with different data processing libraries.
.. _forecasting-models:
Forecasting Models
------------------
Here's a breakdown of the forecasting models currently implemented in Darts. Our suite includes both regression and classification models, each tailored for specific forecasting tasks. We are committed to expanding our offerings with new models and features to enhance your forecasting capabilities.
Regression Models
~~~~~~~~~~~~~~~~~
Our regression models are designed to predict continuous numerical values, making them ideal for forecasting future trends and patterns in time series data. Utilize these models to gain insights into potential future outcomes based on historical data.
.. list-table::
:header-rows: 1
:widths: 30 10 10 10 10 20
* - Model
- Target Series Support: Univariate / Multivariate
- Covariates Support: Past-observed / Future-known / Static
- Probabilistic Forecasting: Sampled / Distribution Parameters
- Training & Forecasting on Multiple Series
- Sources
* - **Baseline Models** (`LocalForecastingModel `_)
-
-
-
-
-
* - `NaiveMean `_
- ✅ ✅
- 🔴 🔴 🔴
- 🔴 🔴
- 🔴
-
* - `NaiveSeasonal `_
- ✅ ✅
- 🔴 🔴 🔴
- 🔴 🔴
- 🔴
-
* - `NaiveDrift `_
- ✅ ✅
- 🔴 🔴 🔴
- 🔴 🔴
- 🔴
-
* - `NaiveMovingAverage `_
- ✅ ✅
- 🔴 🔴 🔴
- 🔴 🔴
- 🔴
-
* - **Statistical / Classic Models** (`LocalForecastingModel `_)
-
-
-
-
-
* - `ARIMA `_
- ✅ 🔴
- 🔴 ✅ 🔴
- ✅ 🔴
- 🔴
-
* - `VARIMA `_
- 🔴 ✅
- 🔴 ✅ 🔴
- ✅ 🔴
- 🔴
-
* - `ExponentialSmoothing `_
- ✅ 🔴
- 🔴 🔴 🔴
- ✅ 🔴
- 🔴
-
* - `Theta `_ and `FourTheta `_
- ✅ 🔴
- 🔴 🔴 🔴
- 🔴 🔴
- 🔴
- `Theta paper `_ & `4 Theta source `_
* - `Prophet `_
- ✅ 🔴
- 🔴 ✅ 🔴
- ✅ 🔴
- 🔴
- `Prophet repo `_
* - `FFT `_ (Fast Fourier Transform)
- ✅ 🔴
- 🔴 🔴 🔴
- 🔴 🔴
- 🔴
-
* - `KalmanForecaster `_ using the Kalman filter and N4SID for system identification
- ✅ ✅
- 🔴 ✅ 🔴
- ✅ 🔴
- 🔴
- `N4SID paper `_
* - `TBATS `_
- ✅ 🔴
- 🔴 ✅ 🔴
- ✅ ✅
- 🔴
- `TBATS paper `_
* - `Croston `_ method
- ✅ 🔴
- 🔴 ✅ 🔴
- ✅ ✅
- 🔴
-
* - `StatsForecastModel `_ wrapper around any `StatsForecast `_ model
- ✅ 🔴
- 🔴 ✅ 🔴
- ✅ ✅
- 🔴
- `Nixtla's statsforecast `_
* - `AutoARIMA `_
- ✅ 🔴
- 🔴 ✅ 🔴
- ✅ ✅
- 🔴
- `Nixtla's statsforecast `_
* - `AutoETS `_
- ✅ 🔴
- 🔴 ✅ 🔴
- ✅ ✅
- 🔴
- `Nixtla's statsforecast `_
* - `AutoCES `_
- ✅ 🔴
- 🔴 ✅ 🔴
- ✅ ✅
- 🔴
- `Nixtla's statsforecast `_
* - `AutoMFLES `_
- ✅ 🔴
- 🔴 ✅ 🔴
- ✅ ✅
- 🔴
- `Nixtla's statsforecast `_
* - `AutoTBATS `_
- ✅ 🔴
- 🔴 ✅ 🔴
- ✅ ✅
- 🔴
- `Nixtla's statsforecast `_
* - `AutoTheta `_
- ✅ 🔴
- 🔴 ✅ 🔴
- ✅ ✅
- 🔴
- `Nixtla's statsforecast `_
* - `MultivariateModel `_
- ✅ ✅
- 🔴 ✅ 🔴
- ✅ ✅
- 🔴
-
* - **Global Baseline Models** (`GlobalForecastingModel `_)
-
-
-
-
-
* - `GlobalNaiveAggregate `_
- ✅ ✅
- 🔴 🔴 🔴
- 🔴 🔴
- ✅
-
* - `GlobalNaiveDrift `_
- ✅ ✅
- 🔴 🔴 🔴
- 🔴 🔴
- ✅
-
* - `GlobalNaiveSeasonal `_
- ✅ ✅
- 🔴 🔴 🔴
- 🔴 🔴
- ✅
-
* - **Regression Models** (`GlobalForecastingModel `_)
-
-
-
-
-
* - `SKLearnModel `_: wrapper around any scikit-learn-like regression model
- ✅ ✅
- ✅ ✅ ✅
- 🔴 🔴
- ✅
-
* - `LinearRegressionModel `_
- ✅ ✅
- ✅ ✅ ✅
- ✅ ✅
- ✅
-
* - `RandomForestModel `_
- ✅ ✅
- ✅ ✅ ✅
- 🔴 🔴
- ✅
-
* - `CatBoostModel `_
- ✅ ✅
- ✅ ✅ ✅
- ✅ ✅
- ✅
-
* - `LightGBMModel `_
- ✅ ✅
- ✅ ✅ ✅
- ✅ ✅
- ✅
-
* - `XGBModel `_
- ✅ ✅
- ✅ ✅ ✅
- ✅ ✅
- ✅
-
* - **PyTorch (Lightning)-based Models** (`GlobalForecastingModel `_)
-
-
-
-
-
* - `RNNModel `_ (incl. LSTM and GRU); equivalent to DeepAR in its probabilistic version
- ✅ ✅
- 🔴 ✅ 🔴
- ✅ ✅
- ✅
- `DeepAR paper `_
* - `BlockRNNModel `_ (incl. LSTM and GRU)
- ✅ ✅
- ✅ ✅ ✅
- ✅ ✅
- ✅
-
* - `NBEATSModel `_
- ✅ ✅
- ✅ 🔴 🔴
- ✅ ✅
- ✅
- `N-BEATS paper `_
* - `NHiTSModel `_
- ✅ ✅
- ✅ 🔴 🔴
- ✅ ✅
- ✅
- `N-HiTS paper `_
* - `TCNModel `_
- ✅ ✅
- ✅ 🔴 🔴
- ✅ ✅
- ✅
- `TCN paper `_, `DeepTCN paper `_, `blog post `_
* - `TransformerModel `_
- ✅ ✅
- ✅ 🔴 🔴
- ✅ ✅
- ✅
-
* - `TFTModel `_ (Temporal Fusion Transformer)
- ✅ ✅
- ✅ ✅ ✅
- ✅ ✅
- ✅
- `TFT paper `_, `PyTorch Forecasting `_
* - `DLinearModel `_
- ✅ ✅
- ✅ ✅ ✅
- ✅ ✅
- ✅
- `DLinear paper `_
* - `NLinearModel `_
- ✅ ✅
- ✅ ✅ ✅
- ✅ ✅
- ✅
- `NLinear paper `_
* - `TiDEModel `_
- ✅ ✅
- ✅ ✅ ✅
- ✅ ✅
- ✅
- `TiDE paper `_
* - `TSMixerModel `_
- ✅ ✅
- ✅ ✅ ✅
- ✅ ✅
- ✅
- `TSMixer paper `_, `PyTorch Implementation `_
* - `NeuralForecastModel `_: wrapper around any `NeuralForecast `_ base model
- ✅ ✅
- ✅ ✅ ✅
- ✅ ✅
- ✅
- `NeuralForecast Documentation `_
* - **Foundation Models** (`GlobalForecastingModel `_): No training required
-
-
-
-
-
* - `Chronos2Model `_
- ✅ ✅
- ✅ ✅ 🔴
- ✅ ✅
- ✅
- `Chronos-2 paper `_, `Amazon blog post `_
* - `TimesFM2p5Model `_
- ✅ ✅
- 🔴 🔴 🔴
- ✅ ✅
- ✅
- `TimesFM 1.0 paper `_, `Google blog post `_
* - `TimesFM3Model `_
- ✅ ✅
- ✅ ✅ 🔴
- ✅ ✅
- ✅
- `TimesFM 1.0 paper `_, `TimesFM 3.0 model card `_
* - `TiRexModel `_
- ✅ ✅
- 🔴 🔴 🔴
- ✅ ✅
- ✅
- `TiRex paper `_, `TiRex GitHub `_
* - `TiRex2Model `_
- ✅ ✅
- ✅ ✅ 🔴
- ✅ ✅
- ✅
- `TiRex-2 paper `_, `TiRex-2 GitHub `_
* - `PatchTSTFMModel `_
- ✅ ✅
- 🔴 🔴 🔴
- ✅ ✅
- ✅
- `PatchTST-FM paper `_, `PatchTST-FM Github `_
* - `T0Model `_
- ✅ ✅
- ✅ ✅ 🔴
- ✅ ✅
- ✅
- `T0 report `_, `tfc-t0 GitHub `_
* - **Ensemble Models** (`GlobalForecastingModel `_): Model support is dependent on ensembled forecasting models and the ensemble model itself
-
-
-
-
-
* - `NaiveEnsembleModel `_
- ✅ ✅
- ✅ ✅ ✅
- ✅ ✅
- ✅
-
* - `RegressionEnsembleModel `_
- ✅ ✅
- ✅ ✅ ✅
- ✅ ✅
- ✅
-
* - **Conformal Models** (`GlobalForecastingModel `_): Model support is dependent on the forecasting model used
-
-
-
-
-
* - `ConformalNaiveModel `_
- ✅ ✅
- ✅ ✅ ✅
- ✅ ✅
- ✅
- `Conformalized Prediction `_
* - `ConformalQRModel `_
- ✅ ✅
- ✅ ✅ ✅
- ✅ ✅
- ✅
- `Conformalized Quantile Regression `_
Classification Models
~~~~~~~~~~~~~~~~~~~~~
Classification models in Darts are designed to predict categorical class labels, enabling effective time series labeling and future class prediction. These models are perfect for scenarios where identifying distinct categories or states over time is crucial.
.. list-table::
:header-rows: 1
:widths: 30 10 10 10 10 20
* - Model
- Target Series Support: Univariate / Multivariate
- Covariates Support: Past-observed / Future-known / Static
- Probabilistic Forecasting: Sampled / Distribution Parameters
- Training & Forecasting on Multiple Series
- Sources
* - **Regression Models** (`GlobalForecastingModel `_)
-
-
-
-
-
* - `SKLearnClassifierModel `_: wrapper around any scikit-learn-like classification model
- ✅ ✅
- ✅ ✅ ✅
- ✅ ✅
- ✅
-
* - `CatBoostClassifierModel `_
- ✅ ✅
- ✅ ✅ ✅
- ✅ ✅
- ✅
-
* - `LightGBMClassifierModel `_
- ✅ ✅
- ✅ ✅ ✅
- ✅ ✅
- ✅
-
* - `XGBClassifierModel `_
- ✅ ✅
- ✅ ✅ ✅
- ✅ ✅
- ✅
-
Community & Contact
-------------------
Anyone is welcome to join our `Gitter room `_ to ask questions, make proposals, discuss use-cases, and more. If you spot a bug or have suggestions, GitHub issues are also welcome.
If what you want to tell us is not suitable for Gitter or Github, feel free to send us an email at darts@unit8.co for darts related matters or info@unit8.co for any other inquiries.
Contribute
----------
The development is ongoing, and we welcome suggestions, pull requests and issues on GitHub. All contributors will be acknowledged on the `change log page `_.
Before working on a contribution (a new feature or a fix), `check our contribution guidelines `_.
Citation
--------
If you are using Darts in your scientific work, we would appreciate citations to the following JMLR paper.
`Darts: User-Friendly Modern Machine Learning for Time Series `_
Bibtex entry:
.. code-block:: bibtex
@article{JMLR:v23:21-1177,
author = {Julien Herzen and Francesco Lässig and Samuele Giuliano Piazzetta and Thomas Neuer and Léo Tafti and Guillaume Raille and Tomas Van Pottelbergh and Marek Pasieka and Andrzej Skrodzki and Nicolas Huguenin and Maxime Dumonal and Jan Kościsz and Dennis Bader and Frédérick Gusset and Mounir Benheddi and Camila Williamson and Michal Kosinski and Matej Petrik and Gaël Grosch},
title = {Darts: User-Friendly Modern Machine Learning for Time Series},
journal = {Journal of Machine Learning Research},
year = {2022},
volume = {23},
number = {124},
pages = {1-6},
url = {http://jmlr.org/papers/v23/21-1177.html}
}
.. toctree::
:hidden:
Quickstart
.. toctree::
:hidden:
User Guide
.. toctree::
:hidden:
API Reference
.. toctree::
:hidden:
Examples
.. toctree::
:hidden:
Release Notes
---
### Source Userguide (`docs/source/userguide.rst`)
==========
User Guide
==========
You will find here some more detailed information about Darts.
**Note:** The user guide is not yet complete and still under construction.
.. toctree::
:maxdepth: 2
userguide/timeseries.md
userguide/forecasting_overview.md
userguide/torch_forecasting_models.md
userguide/gpu_and_tpu_usage.md
userguide/covariates.md
userguide/safe_model_loading.md
userguide/hyperparameter_optimization.md
userguide/faq.md
---
### Userguide Covariates (`docs/userguide/covariates.md`)
# Covariates
Sections about past and future covariates were written for darts version 0.15.0 and later.
Sections about static covariates were written for darts version 0.20.0 and later.
## Summary - TL;DR
In Darts, **covariates** refer to external data that can be used as inputs to models to help improve forecasts.
In the context of forecasting models, the **target** is the series to be forecasted/predicted, and the
covariates themselves are not predicted. We distinguish three kinds of covariates:
* **past covariates** are (by definition) covariates known only into the past (e.g. measurements)
* **future covariates** are (by definition) covariates known into the future (e.g., weather forecasts)
* **static covariates** are (by definition) covariates constant over time (e.g., product IDs). Check out our static covariates example notebook [here](https://unit8co.github.io/darts/examples/15-static-covariates.html) for more information.
Models in Darts accept `past_covariates` and/or `future_covariates` in their `fit()` and `predict()` methods, depending on their capabilities (some models accept no covariates at all). Both target and covariates must be a `TimeSeries` object. The models will raise an error if covariates were used that are not supported.
```python
# create one of Darts' forecasting models
model = SomeForecastingModel(...)
# fitting model with past and future covariates
model.fit(target=target,
past_covariates=past_covariates_train,
future_covariates=future_covariates_train)
# predict the next n=12 steps
model.predict(n=12,
series=target, # only required for Global Forecasting Models
past_covariates=past_covariates_pred,
future_covariates=future_covariates_pred)
```
Different to past and future covariates, static covariates must be embedded in the **target** series. Because of that, working with static covariates follows a different methodology. You can check out how to use static covariates in [this example](https://unit8co.github.io/darts/examples/15-static-covariates.html).
If you have several covariate variables that you want to use as past (or future) covariates, you have to `stack()` all of them into a single `past_covariates` (or `future_covariates`) object.
```python
# stack two TimeSeries with stack()
past_covariates = past_covariates.stack(other_past_covariates)
# or with concatenate()
from darts import concatenate
past_covariates = concatenate([past_covariates, other_past_covariates], axis=1)
```
Darts' forecasting models expect one past and/or future covariate series per target series. If you use multiple target series with one of Darts' Global Forecasting Models, you must supply the same number of dedicated covariates to `fit()`.
```python
# fit using multiple (two) target series
model.fit(target=[target, target_2],
past_covariates=[past_covariates, past_covariates_2],
# optional future_covariates,
)
# you must give the specific target and covariate series that you want to predict
model.predict(n=12,
series=target_2,
past_covariates=past_covariates_2,
# optional future_covariates,
)
```
If you train a model using `past_covariates`, you'll have to provide these `past_covariates` also at prediction time to `predict()`. This applies to `future_covariates` too, with a nuance that `future_covariates` have to extend far enough into the future at prediction time (all the way to the forecast horizon `n`). This can be seen in the graph below. `past_covariates` needs to include at least the same time steps as `target`, and `future_covariates` must include at least the same time span plus additional `n` forecast horizon time steps.
**You can use the same `*_covariates` for both training and prediction, given that they contain the required time spans.**
**Figure 1: Top level summary of how forecasting models work with target and covariates for a prediction with forecast horizon n=2**
There are some extra nuances that might be good to know. For instance, deep learning models in Darts
can (in general) forecast `output_chunk_length` points at a time. However it is still possible for models
trained with past covariates to make forecasts for some horizon `n > output_chunk_length` if the `past_covariates`
are known far enough into the future. In such cases, the forecasts are obtained by consuming future values
of the past covariates, and using auto-regression on the target series. If you want to know more details, read on.
## Introduction - What are covariates (in Darts)?
Past, future and static covariates provide additional information/context that can be useful to improve the prediction of the `target` series. The `target` series is the variable we wish to predict the future for. We do not predict the covariates themselves, only use them for prediction of the `target`.
Past and future covariates hold information about the past (up to and including present time) or future. This is always relative to the prediction point (in time) after which we want to forecast the future.
In Darts, we refer to these two types as `past_covariates` and `future_covariates`.
Static covariates hold time independent (constant / static) information about the `target` series. We refer to them as `static_covariates`. They must be embedded in the `target` series. Working with static covariates follows a slightly different approach than with past or future covariates. Check out our [notebook on static covariates](https://unit8co.github.io/darts/examples/15-static-covariates.html) to learn more.
Darts' forecasting models have different support modes for `*_covariates`. Some do not support covariates at all, others support only past or future covariates and some even support all three (more on that in [this subsection](#forecasting-model-covariate-support)).
Let's have a look at some examples of past, future, and static covariates:
- `past_covariates`: typically measurements (past data) or temporal attributes
- daily average **measured** temperatures (known only in the past)
- day of week, month, year, ...
- `future_covariates`: typically forecasts (future known data) or temporal attributes
- daily average **forecasted** temperatures (known in the future)
- day of week, month, year, ...
- `static_covariates`: time independent/constant/static `target` characteristics
- categorical:
- location of `target` (country, city, .. name)
- `target` identifier: (product ID, store ID, ...)
- numerical:
- population of `target`'s country/market area (assuming it stays constant over the forecasting horizon)
- average temperature of `target`'s region (assuming it stays constant over the forecasting horizon)
Temporal attributes are powerful because they are known in advance and can help models capture trends and / or seasonal patterns of the `target` series.
Static attributes are powerful when working with multiple `targets` (either multiple `TimeSeries`, or multivariate series containing multiple dimensions each). The time independent information can help models identify the nature/environment of the underlying series and improve forecasts across different `targets`.
In this guide we'll focus on past and future covariates. Here's a simple rule-of-thumb to know if your series are **past** or **future covariates**:
*If the values are known in advance, they are future covariates (or can be used as past covariates). If they are not, they **must** be past covariates.*
You might imagine cases where you want to train a model supporting only `past_covariates` (such as `TCNModel`, see Table 1). In this case, you could use for instance say, the **forecasted** temperature as a past covariate for the model *even though you also have access to temperature forecasts in the future*. Knowing such "future values of past covariates" can allow you to make forecasts further into the future (for Darts' deep learning models with forecast horizons `n > output_chunk_length`). Similarly most models consuming future covariates can also use "historic values of future covariates".
Side note: if you don't have future values (e.g. of measured temperatures), nothing prevents you from applying one of Darts' forecasting models to forecast future temperatures, and then use this as `future_covariates`. Darts is not attempting to forecast the covariates for you, as this would introduce an extra "hidden" modeling step, which we think is best left to the users.
## Forecasting Model Covariate Support
Darts' forecasting models accept optional `past_covariates` and / or `future_covariates` in their `fit()` and `predict()` methods (and `static_covariates` embedded in the `target` series), depending on their capabilities. Table 1 shows the supported covariate types for each model. The models will raise an error if covariates were used that are not supported.
### Local Forecasting Models (LFMs):
LFMs are models that can be trained on a single target series only. In Darts most models in this category tend to be simpler statistical models (such as ETS or ARIMA). LFMs accept only a single `target` (and covariate) time series and usually train on the entire series you supplied when calling `fit()` at once. They can also predict in one go for any number of predictions `n` after the end of the training series.
### Global Forecasting Models (GFMs)
GFMs are models that can be trained on multiple target (and covariate) time series. Different to LFMs, the GFMs train and predict on fixed-length sub-samples (chunks) of the input data. In Darts, these are the global (naive) baseline models, regression models, PyTorch (Lightning)-based models (neural networks), foundation models, as well ensemble models (depending on their ensemble model and / or the forecasting models they ensemble).
----
| Model | Past Covariates | Future Covariates | Static Covariates |
|-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|-----------------|:-----------------:|:-----------------:|
| **Local Forecasting Models (LFMs)** | | | |
| Naive Baselines (a) | | | |
| [ARIMA](https://unit8co.github.io/darts/generated_api/darts.models.forecasting.arima.html#darts.models.forecasting.arima.ARIMA) | | ✅ | |
| [VARIMA](https://unit8co.github.io/darts/generated_api/darts.models.forecasting.varima.html#darts.models.forecasting.varima.VARIMA) | | ✅ | |
| [ExponentialSmoothing](https://unit8co.github.io/darts/generated_api/darts.models.forecasting.exponential_smoothing.html#darts.models.forecasting.exponential_smoothing.ExponentialSmoothing) | | | |
| [TBATS](https://unit8co.github.io/darts/generated_api/darts.models.forecasting.sf_tbats.html#darts.models.forecasting.sf_tbats.TBATS) | | ✅ | |
| [Theta](https://unit8co.github.io/darts/generated_api/darts.models.forecasting.theta.html#darts.models.forecasting.theta.Theta) and [FourTheta](https://unit8co.github.io/darts/generated_api/darts.models.forecasting.theta.html#darts.models.forecasting.theta.FourTheta) | | | |
| [Prophet](https://unit8co.github.io/darts/generated_api/darts.models.forecasting.prophet_model.html#darts.models.forecasting.prophet_model.Prophet) | | ✅ | |
| [FFT](https://unit8co.github.io/darts/generated_api/darts.models.forecasting.fft.html#darts.models.forecasting.fft.FFT) (Fast Fourier Transform) | | | |
| [KalmanForecaster](https://unit8co.github.io/darts/generated_api/darts.models.forecasting.kalman_forecaster.html#darts.models.forecasting.kalman_forecaster.KalmanForecaster) | | ✅ | |
| [Croston](https://unit8co.github.io/darts/generated_api/darts.models.forecasting.croston.html#darts.models.forecasting.croston.Croston) method | | ✅ | |
| [StatsForecastModel](https://unit8co.github.io/darts/generated_api/darts.models.forecasting.sf_model.html#darts.models.forecasting.sf_model.StatsForecastModel) | | ✅ | |
| [AutoARIMA](https://unit8co.github.io/darts/generated_api/darts.models.forecasting.sf_auto_arima.html#darts.models.forecasting.sf_auto_arima.AutoARIMA) | | ✅ | |
| [AutoETS](https://unit8co.github.io/darts/generated_api/darts.models.forecasting.sf_auto_ets.html#darts.models.forecasting.sf_auto_ets.AutoETS) | | ✅ | |
| [AutoCES](https://unit8co.github.io/darts/generated_api/darts.models.forecasting.sf_auto_ces.html#darts.models.forecasting.sf_auto_ces.AutoCES) | | ✅ | |
| [AutoMFLES](https://unit8co.github.io/darts/generated_api/darts.models.forecasting.sf_auto_mfles.html#darts.models.forecasting.sf_auto_mfles.AutoMFLES) | | ✅ | |
| [AutoTBATS](https://unit8co.github.io/darts/generated_api/darts.models.forecasting.sf_auto_tbats.html#darts.models.forecasting.sf_auto_tbats.AutoTBATS) | | ✅ | |
| [AutoTheta](https://unit8co.github.io/darts/generated_api/darts.models.forecasting.sf_auto_theta.html#darts.models.forecasting.sf_auto_theta.AutoTheta) | | ✅ | |
| [MultivariateModel](https://unit8co.github.io/darts/generated_api/darts.models.forecasting.multivariate_model.html#darts.models.forecasting.multivariate_model.MultivariateModel) | | ✅ | |
| **Global Forecasting Models (GFMs)** | | | |
| Global Naive Baselines (b) | | | |
| SKLearn-Like Regression Models (c) | ✅ | ✅ | ✅ |
| SKLearn-Like Classification Models (d) | ✅ | ✅ | ✅ |
| [RNNModel](https://unit8co.github.io/darts/generated_api/darts.models.forecasting.rnn_model.html#darts.models.forecasting.rnn_model.RNNModel) (e) | | ✅ | |
| [BlockRNNModel](https://unit8co.github.io/darts/generated_api/darts.models.forecasting.block_rnn_model.html#darts.models.forecasting.block_rnn_model.BlockRNNModel) (f) | ✅ | ✅ | ✅ |
| [NBEATSModel](https://unit8co.github.io/darts/generated_api/darts.models.forecasting.nbeats.html#darts.models.forecasting.nbeats.NBEATSModel) | ✅ | | |
| [NHiTSModel](https://unit8co.github.io/darts/generated_api/darts.models.forecasting.nhits.html#darts.models.forecasting.nhits.NHiTSModel) | ✅ | | |
| [TCNModel](https://unit8co.github.io/darts/generated_api/darts.models.forecasting.tcn_model.html#darts.models.forecasting.tcn_model.TCNModel) | ✅ | | |
| [TransformerModel](https://unit8co.github.io/darts/generated_api/darts.models.forecasting.transformer_model.html#darts.models.forecasting.transformer_model.TransformerModel) | ✅ | | |
| [TFTModel](https://unit8co.github.io/darts/generated_api/darts.models.forecasting.tft_model.html#darts.models.forecasting.tft_model.TFTModel) | ✅ | ✅ | ✅ |
| [DLinearModel](https://unit8co.github.io/darts/generated_api/darts.models.forecasting.dlinear.html#darts.models.forecasting.dlinear.DLinearModel) | ✅ | ✅ | ✅ |
| [NLinearModel](https://unit8co.github.io/darts/generated_api/darts.models.forecasting.nlinear.html#darts.models.forecasting.nlinear.NLinearModel) | ✅ | ✅ | ✅ |
| [TiDEModel](https://unit8co.github.io/darts/generated_api/darts.models.forecasting.tide_model.html#darts.models.forecasting.tide_model.TiDEModel) | ✅ | ✅ | ✅ |
| [TSMixerModel](https://unit8co.github.io/darts/generated_api/darts.models.forecasting.tsmixer_model.html#darts.models.forecasting.tsmixer_model.TSMixerModel) | ✅ | ✅ | ✅ |
| [NeuralForecastModel](https://unit8co.github.io/darts/generated_api/darts.models.forecasting.nf_model.html#darts.models.forecasting.nf_model.NeuralForecastModel) (g) | ✅ | ✅ | ✅ |
| [Chronos2Model](https://unit8co.github.io/darts/generated_api/darts.models.forecasting.chronos2_model.html#darts.models.forecasting.chronos2_model.Chronos2Model) | ✅ | ✅ | |
| [TimesFM2p5Model](https://unit8co.github.io/darts/generated_api/darts.models.forecasting.timesfm2p5_model.html#darts.models.forecasting.timesfm2p5_model.TimesFM2p5Model) | | | |
| [TimesFM3Model](https://unit8co.github.io/darts/generated_api/darts.models.forecasting.timesfm3_model.html#darts.models.forecasting.timesfm3_model.TimesFM3Model) | ✅ | ✅ | |
| [TiRexModel](https://unit8co.github.io/darts/generated_api/darts.models.forecasting.tirex_model.html#darts.models.forecasting.tirex_model.TiRexModel) | | | |
| [TiRex2Model](https://unit8co.github.io/darts/generated_api/darts.models.forecasting.tirex2_model.html#darts.models.forecasting.tirex2_model.TiRex2Model) | ✅ | ✅ | |
| Ensemble Models (h) | ✅ | ✅ | ✅ |
| Conformal Prediction Models (i) | ✅ | ✅ | ✅ |
**Table 1: Darts' forecasting models and their covariate support**
(a) Naive Baselines including [NaiveDrift](https://unit8co.github.io/darts/generated_api/darts.models.forecasting.baselines.html#darts.models.forecasting.baselines.NaiveDrift), [NaiveMean](https://unit8co.github.io/darts/generated_api/darts.models.forecasting.baselines.html#darts.models.forecasting.baselines.NaiveMean), [NaiveMovingAverage](https://unit8co.github.io/darts/generated_api/darts.models.forecasting.baselines.html#darts.models.forecasting.baselines.NaiveMovingAverage), and [NaiveSeasonal](https://unit8co.github.io/darts/generated_api/darts.models.forecasting.baselines.html#darts.models.forecasting.baselines.NaiveSeasonal).
(b) Global Naive Baselines including [GlobalNaiveAggregate](https://unit8co.github.io/darts/generated_api/darts.models.forecasting.global_baseline_models.html#darts.models.forecasting.global_baseline_models.GlobalNaiveAggregate), [GlobalNaiveDrift](https://unit8co.github.io/darts/generated_api/darts.models.forecasting.global_baseline_models.html#darts.models.forecasting.global_baseline_models.GlobalNaiveDrift), and [GlobalNaiveSeasonal](https://unit8co.github.io/darts/generated_api/darts.models.forecasting.global_baseline_models.html#darts.models.forecasting.global_baseline_models.GlobalNaiveSeasonal).
(c) SKLearn-Like Regression Models including [SKLearnModel](https://unit8co.github.io/darts/generated_api/darts.models.forecasting.sklearn_model.html#darts.models.forecasting.sklearn_model.SKLearnModel), [LinearRegressionModel](https://unit8co.github.io/darts/generated_api/darts.models.forecasting.linear_regression_model.html#darts.models.forecasting.linear_regression_model.LinearRegressionModel), [RandomForestModel](https://unit8co.github.io/darts/generated_api/darts.models.forecasting.random_forest.html#darts.models.forecasting.random_forest.RandomForestModel), [LightGBMModel](https://unit8co.github.io/darts/generated_api/darts.models.forecasting.lgbm.html#darts.models.forecasting.lgbm.LightGBMModel), [XGBModel](https://unit8co.github.io/darts/generated_api/darts.models.forecasting.xgboost.html#darts.models.forecasting.xgboost.XGBModel), and [CatBoostModel](https://unit8co.github.io/darts/generated_api/darts.models.forecasting.catboost_model.html#darts.models.forecasting.catboost_model.CatBoostModel). SKLearnModel is a special kind of GFM which can use arbitrary lags on covariates (past and/or future) and past targets to predict continuous numerical values.
(d) SKLearn-Like Classification Models including [SKLearnClassifierModel](https://unit8co.github.io/darts/generated_api/darts.models.forecasting.sklearn_model.html#darts.models.forecasting.sklearn_model.SKLearnClassifierModel), [LightGBMClassifierModel](https://unit8co.github.io/darts/generated_api/darts.models.forecasting.lgbm.html#darts.models.forecasting.lgbm.LightGBMClassifierModel), [XGBClassifierModel](https://unit8co.github.io/darts/generated_api/darts.models.forecasting.xgboost.html#darts.models.forecasting.xgboost.XGBClassifierModel), and [CatBoostClassifierModel](https://unit8co.github.io/darts/generated_api/darts.models.forecasting.catboost_model.html#darts.models.forecasting.catboost_model.CatBoostClassifierModel). SKLearnClassifierModel is a special kind of GFM which can use arbitrary lags on covariates (past and/or future) and past targets to predict categorical class labels.
(e) [RNNModel](https://unit8co.github.io/darts/generated_api/darts.models.forecasting.rnn_model.html#darts.models.forecasting.rnn_model.RNNModel) including `LSTM` and `GRU`; equivalent to DeepAR in its probabilistic version
(f) [BlockRNNModel](https://unit8co.github.io/darts/generated_api/darts.models.forecasting.block_rnn_model.html#darts.models.forecasting.block_rnn_model.BlockRNNModel) including `LSTM` and `GRU`
(g) [NeuralForecastModel](https://unit8co.github.io/darts/generated_api/darts.models.forecasting.nf_model.html#darts.models.forecasting.nf_model.NeuralForecastModel) covariate support depends on the NeuralForecast base model used. See [NeuralForecast model catalog](https://nixtlaverse.nixtla.io/neuralforecast/docs/capabilities/overview.html) for more details.
(h) Ensemble Model including [RegressionEnsembleModel](https://unit8co.github.io/darts/generated_api/darts.models.forecasting.regression_ensemble_model.html#darts.models.forecasting.regression_ensemble_model.RegressionEnsembleModel), and [NaiveEnsembleModel](https://unit8co.github.io/darts/generated_api/darts.models.forecasting.naive_ensemble_model.html#darts.models.forecasting.naive_ensemble_model.NaiveEnsembleModel). The covariate support is given by the covariate support of the ensembled forecasting models.
(i) Conformal Prediction Model including [ConformalNaiveModel](https://unit8co.github.io/darts/generated_api/darts.models.forecasting.conformal_models.html#darts.models.forecasting.conformal_models.ConformalNaiveModel), and [ConformalQRModel](https://unit8co.github.io/darts/generated_api/darts.models.forecasting.conformal_models.html#darts.models.forecasting.conformal_models.ConformalQRModel). The covariate support is given by the covariate support of the underlying forecasting model.
----
## Quick guide on how to use past and/or future covariates with Darts' forecasting models
It is very simple to use covariates with Darts' forecasting models. There are just some requirements they have to fulfill.
Just like the `target` series, each of your past and / or future covariates series must be a `TimeSeries` object. When you train your model with `fit()` using past and /or future covariates, you have to supply the same types of covariates to `predict()`. Depending on the choice of your model and how long your forecast horizon `n` is, there might be different time span requirements for your covariates. You can find these requirements in the [next subsection](#covariate-time-span-requirements-for-local-and-global-forecasting-models).
**You can even use the same `*_covariates` for fitting and prediction if they contain the required time spans. This is because Darts will "intelligently" slice them for you based on the target time axis.**
```python
# create one of Darts' forecasting model
model = SomeForecastingModel(...)
# fit the model
model.fit(target,
past_covariates=past_covariate,
future_covariates=future_covariates)
# make a prediction with the same covariate types
pred = model.predict(n=1,
series=target, # this is only required for GFMs
past_covariates=past_covariates,
future_covariates=future_covariates)
```
To use multiple past and / or future covariates with your `target`, you have to stack them all together into a single dedicated `TimeSeries`:
```python
# stack() time series
past_covariates = past_covariates.stack(past_covariates2)
# or concatenate()
from darts import concatenate
past_covariates = concatenate([past_covariates, past_covariates2, ...], axis=1)
```
GFMs can be trained on multiple `target` series. You have to supply one covariate TimeSeries per `target` TimeSeries you use with `fit()`. At prediction time you have to specify which `target` series you want to predict and supply the corresponding covariates:
```python
from darts.models import NBEATSModel
# multiple time series
all_targets = [target1, target2, ...]
all_past_covariates = [past_covariates1, past_covariates2, ...]
# create a GFM model, train and predict
model = NBEATSModel(input_chunk_length=1, output_chunk_length=1)
model.fit(all_targets,
past_covariates=all_past_covariates)
pred = model.predict(n=1,
series=all_targets[0],
past_covariates=all_past_covariates[0])
```
## Covariate time span requirements for Local and Global Forecasting Models
There are differences in how Darts' "Local" and "Global" Forecasting Models perform training and prediction. Specifically, how they extract/work with the data supplied during fit() and predict().
Depending on the model you use and how long your forecast horizon `n` is, there might be different time span requirements for your covariates.
### Local Forecasting Models (LFMs):
LFMs usually train on the entire `target` and `future_covariates` series (if supported) you supplied when calling `fit()` at once. They can also predict in one go for forecast horizon `n` after the end of the `target`.
*Time span requirements to use the same future covariates series for both `fit()` and `predict()`:*
- `future_covariates`: **at least** the same time span as `target` plus the next `n` time steps after the end of `target`
### Global Forecasting Models (GFMs):
GFMs train and predict on fixed-length chunks (sub-samples) of the `target` and `*_covariates` series (if supported). Each chunk contains an input chunk - representing the sample's past - and an output chunk - the sample's future. The length of these chunks has to be specified at model creation with parameters `input_chunk_length` and `output_chunk_length` (one notable exception is `RNNModel` which always uses an `output_chunk_length` of 1).
Depending on your forecast horizon `n`, the model can either predict in one go, or auto-regressively, by predicting on multiple chunks in the future. That is the reason why when predicting with `past_covariates` you have to supply additional "future values of your `past_covariates`".
**Time span requirements to use the same past and / or future covariates series for both `fit()` and `predict()`:**
- with `n <= output_chunk_length`:
- `past_covariates`: **at least** the same time span as `target`
- `future_covariates`: **at least** the same time span as `target` plus the next `output_chunk_length` time steps after the end of `target`
- with `n > output_chunk_length`:
- `past_covariates`: **at least** the same time span as `target` plus the next `n - output_chunk_length` time steps after the end of `target`
- `future_covariates`: **at least** the same time span as `target` plus the next `n` time steps after the end of `target`
If you want to know more details about how covariates are used behind the scenes in Global Forecasting Models, read our [guide on Torch Forecasting Models](https://unit8co.github.io/darts/userguide/torch_forecasting_models.html) (PyTorch based GFMs). It gives a step-by-step explanation of the training and prediction process using one of our Torch Forecasting Models.
## Examples
Here are a few examples showcasing how to use covariates with Darts forecasting models:
- [Static covariates tutorial including TFTModel example](https://unit8co.github.io/darts/examples/15-static-covariates.html)
- [Past covariates with GFMs](https://unit8co.github.io/darts/examples/01-multi-time-series-and-covariates.html#Covariates-Series)
- [Past and future covariates with TFTModel](https://unit8co.github.io/darts/examples/13-TFT-examples.html#Training)
- [Past and future covariates with SKLearnModels (previously named RegressionModel)](https://medium.com/unit8-machine-learning-publication/time-series-forecasting-using-past-and-future-external-data-with-darts-1f0539585993)
---
### Userguide Faq (`docs/userguide/faq.md`)
# Frequently Asked Questions
- *Is Darts just a wrapper around other libraries?*
No. When it makes sense, we reuse existing implementations (e.g. from statsforecasts), but we often write our own implementations (e.g., of neural networks). Furthermore, Darts models often have more features than their original counter-parts. For instance, unlike the original version, our implementation of N-BEATS supports multivariate time series, past covariates, and probabilistic forecasts.
- *Darts looks like an awesome project, can I contribute?*
Absolutely! We are constantly welcoming contributions from the community. If you contribute, you will be acknowledged in the [wall of fame (a.k.a the changelog)](https://github.com/unit8co/darts/blob/master/CHANGELOG.md)! Contributions don't have to be code only but can also be e.g., documentation. In addition, we're also happy to receive suggestions in the form of issues on Github. The best place to start for contributors is the [contribution guidelines](https://github.com/unit8co/darts/blob/master/CONTRIBUTING.md).
- *I have some ideas for contributing a new model into Darts, is that possible?*
In general, yes, we do welcome reference implementations of new models. However, we are loosely filtering to keep models that are either classics, or convincingly shown (e.g., in a paper or some other forms of evidence) to be state-of-the-art in some respect.
- *How can I make Darts work on Google Colab?*
Colab may have issues with recent versions of pyyaml. Installing pyyaml 5.4.1 before installing Darts solves these issues:
```
!pip install pyyaml==5.4.1
```
- *I'm getting some NaNs in my forecasts, what should I do?*
Usually this means one of two things:
- You have some NaNs in your training `TimeSeries` (targets or covariates). This is the most frequent situation, and it will lead most models to always forecast NaNs. Note that NaNs can be introduced when you convert data from a `pd.DataFrame` into a `TimeSeries` if some dates are missing and the `freq` argument is incorrectly set.
- Training the model causes some numerical divergence. If you're working with neural networks, make sure your data is properly scaled, and if the issue persists, try reducing the learning rate.
- *My forecasting models give bad results, can you help?*
Getting good forecasts is about more than just calling the `fit()`/`predict()` functions, and always involves some data science work to understand which approaches are appropriate. We cannot give general answers, however if you have important forecasting problems or need help to industrialize your forecasts, Unit8 provides technical consulting. Feel free to contact us.
- *Can I use probabilistic forecast samples as realistic simulated scenarios over time?*
See the [overview on probabilistic forecasts](https://unit8co.github.io/darts/userguide/forecasting_overview.html#multi-step-probabilistic-forecasts-scenarios-vs-marginal-distributions) for more details.
---
### Userguide Gpu And Tpu Usage (`docs/userguide/gpu_and_tpu_usage.md`)
# Using Torch Models with GPUs and TPUs
This section was written for Darts 0.17.0 and later.
We assume that you already know about Torch Forecasting Models in Darts. If you're new to the topic we recommend you to read the [guide on Torch Forecasting Models](https://unit8co.github.io/darts/userguide/torch_forecasting_models.html) first. This guide also contains a section about performance recommendations, which we recommend reading first. Finally, here is also an [Recurrent Neural Network (RNN) Model example](https://unit8co.github.io/darts/examples/04-RNN-examples.html), on which this section is going to be based on.
## Use CPU
By default all models will run on CPU. As shown in the RNN example above, we'll import the Air Passenger dataset, as well as other necessary modules.
```python
import numpy as np
import pandas as pd
import matplotlib.pyplot as plt
from darts.dataprocessing.transformers import Scaler
from darts.models import RNNModel
from darts.metrics import mape
from darts.datasets import AirPassengersDataset
```
Now we read and scale the data like this:
```python
# Read data:
series = AirPassengersDataset().load()
series = series.astype(np.float32)
# Create training and validation sets:
train, val = series.split_after(pd.Timestamp("19590101"))
# Normalize the time series (note: we avoid fitting the transformer on the validation set)
transformer = Scaler()
train_transformed = transformer.fit_transform(train)
val_transformed = transformer.transform(val)
series_transformed = transformer.transform(series)
```
Next we will create our RNN like this:
```python
my_model = RNNModel(
model="RNN",
hidden_dim=20,
dropout=0,
batch_size=16,
n_epochs=300,
optimizer_kwargs={"lr": 1e-3},
model_name="Air_RNN",
log_tensorboard=True,
random_state=42,
training_length=20,
input_chunk_length=14,
force_reset=True,
)
```
and fit it to the data:
```python
my_model.fit(train_transformed, val_series=val_transformed)
```
where in the output we can see that no other processing unit is used to train our model:
```
GPU available: False, used: False
TPU available: False, using: 0 TPU cores
IPU available: False, using: 0 IPUs
| Name | Type | Params
--------------------------------------
0 | criterion | MSELoss | 0
1 | rnn | RNN | 460
2 | V | Linear | 21
--------------------------------------
481 Trainable params
0 Non-trainable params
481 Total params
0.004 Total estimated model params size (MB)
Epoch 299: 100% 8/8 [00:00<00:00, 42.49it/s, loss=0.00285, v_num=logs]
```
Now the model is ready to start predicting, which won't be shown here since it's included in the example linked in the start of this guide.
## Use a GPU
GPUs can dramatically improve the performance of your model in terms of processing time. By using an Accelerator in the [Pytorch Lightning Trainer](https://pytorch-lightning.readthedocs.io/en/stable/common/trainer.html#accelerator), we can enjoy the benefits of a GPU. We only need to instruct our model to use our machine's GPU through PyTorch Lightning Trainer parameters, which are expressed as the `pl_trainer_kwargs` dictionary, like this:
```python
my_model = RNNModel(
model="RNN",
...
force_reset=True,
pl_trainer_kwargs={
"accelerator": "gpu",
"devices": [0]
},
)
```
which now outputs:
```
GPU available: True, used: True
TPU available: False, using: 0 TPU cores
IPU available: False, using: 0 IPUs
LOCAL_RANK: 0 - CUDA_VISIBLE_DEVICES: [0]
| Name | Type | Params
--------------------------------------
0 | criterion | MSELoss | 0
1 | rnn | RNN | 460
2 | V | Linear | 21
--------------------------------------
481 Trainable params
0 Non-trainable params
481 Total params
0.004 Total estimated model params size (MB)
Epoch 299: 100% 8/8 [00:00<00:00, 39.81it/s, loss=0.00285, v_num=logs]
```
From the output we can see that the GPU is both available and used. The rest of the code doesn't require any change, i.e. it's irrelevant if we are using a GPU or CPU.
### Multi GPU support
Darts utilizes [Lightning's multi GPU capabilities](https://pytorch-lightning.readthedocs.io/en/stable/accelerators/gpu_intermediate.html) to be able to capitalize on scalable hardware.
Multiple parallelization strategies exist for multiple GPU training, which - because of different strategies for multiprocessing and data handling - interact strongly with the execution environment.
Currently in Darts the `ddp_spawn` distribution strategy is tested.
As per the description of the [Lightning documentation](https://pytorch-lightning.readthedocs.io/en/stable/accelerators/gpu_intermediate.html#distributed-data-parallel-spawn) has some noteworthy limitations, eg. it __can not run__ in:
- Jupyter Notebook, Google COLAB, Kaggle, etc.
- In case you have a nested script without a root package
This in practice means, that execution has to happen in a separate `.py` script, that has the following general context around the code executing the training:
```python
import torch
if __name__ == '__main__':
torch.multiprocessing.freeze_support()
```
The __main__ pattern is necessary (see [this](https://pytorch.org/docs/stable/notes/windows.html#multiprocessing-error-without-if-clause-protection)) even when your execution __does not__ happen in a windows environment.
Beyond this, no other major modification to your models is necessary other than allowing multi GPU training in the `pl_trainer_args` for example like
`pl_trainer_kwargs = {"accelerator": "gpu", "devices": -1, "auto_select_gpus": True}`
This method automatically selects all available GPUs for training. Manual setting of the number of devices is also possible.
The `ddp` family of strategies creates individual subprocesses for each GPU, so contents of the memory (notably the `Dataloder`) gets copied over. Thus, as per the [description of lightning docs](https://pytorch-lightning.readthedocs.io/en/stable/accelerators/gpu_intermediate.html#distributed-data-parallel) caution is advised in setting the `Dataloader(num_workers=N)` too high, since according to it:
"Dataloader(num_workers=N), where N is large, bottlenecks training with DDP… ie: it will be VERY slow or won’t work at all. This is a PyTorch limitation."
Usage of other distribution strategies with Darts currently _might_ very well work, but are yet untested and subject to individual setup / experimentation.
## Use a TPU
Tensor Processing Unit (TPU) is an AI accelerator application-specific integrated circuit (ASIC) developed by Google specifically for neural network machine learning.
There are three main ways to get access to a TPU:
* Google Colab
* Google Cloud (GCP)
* Kaggle
If you are using a TPU in the Google Colab kind of notebook, then you should first install these:
```
!pip install cloud-tpu-client==0.10 https://storage.googleapis.com/tpu-pytorch/wheels/torch_xla-1.9-cp37-cp37m-linux_x86_64.whl
!pip install torch==1.9.0+cu111 torchvision==0.10.0+cu111 torchtext==0.10.0 -f https://download.pytorch.org/whl/cu111/torch_stable.html
!pip install pyyaml==5.4.1
```
and then instruct our model to use a TPU or more. In our example we are using four TPUs, like this:
```python
my_model = RNNModel(
model="RNN",
...
force_reset=True,
pl_trainer_kwargs={
"accelerator": "tpu",
"tpu_cores": [4]
},
)
```
which outputs:
```
WARNING:root:TPU has started up successfully with version pytorch-1.9
GPU available: False, used: False
TPU available: True, using: [4] TPU cores
IPU available: False, using: 0 IPUs
| Name | Type | Params
--------------------------------------
0 | criterion | MSELoss | 0
1 | rnn | RNN | 460
2 | V | Linear | 21
--------------------------------------
481 Trainable params
0 Non-trainable params
481 Total params
0.002 Total estimated model params size (MB)
Epoch 299: 100% 8/8 [00:00<00:00, 8.52it/s, loss=0.00285, v_num=logs]
```
From the output we can see that our model is using 4 TPUs.
---
### Userguide Hyperparameter Optimization (`docs/userguide/hyperparameter_optimization.md`)
# Hyperparameter Optimization in Darts
There is nothing special in Darts when it comes to hyperparameter optimization.
The main thing to be aware of is probably the existence of PyTorch Lightning callbacks for early stopping and pruning of experiments with Darts' deep learning based TorchForecastingModels.
Below, we show examples of hyperparameter optimization done with [Optuna](https://optuna.org/) and
[Ray Tune](https://docs.ray.io/en/latest/tune/examples/tune-pytorch-lightning.html).
## Hyperparameter optimization with Optuna
[Optuna](https://optuna.org/) is a great option for hyperparameter optimization with Darts. Below, we show a minimal example
using PyTorch Lightning callbacks for pruning experiments.
For the sake of the example, we train a `TCNModel` on a single series, and optimize (probably overfitting) its hyperparameters by minimizing the prediction error on a validation set.
You can also have a look at [this notebook](https://github.com/unit8co/darts/blob/master/examples/17-hyperparameter-optimization.ipynb)
for a more complete example.
> **NOTE** (2023-19-02): Optuna's `PyTorchLightningPruningCallback` raises an error with pytorch-lightning>=1.8. Until this fixed, a workaround is proposed [here](https://github.com/optuna/optuna-examples/issues/166#issuecomment-1403112861).
```
/* Detailed source-code truncated for AI context efficiency. */
```
## Hyperparameter optimization with Ray Tune
[Ray Tune](https://docs.ray.io/en/latest/tune/examples/tune-pytorch-lightning.html) is another option for hyperparameter optimization with automatic pruning.
Here is an example of how to use Ray Tune to with the `NBEATSModel` model using the [Asynchronous Hyperband scheduler](https://blog.ml.cmu.edu/2018/12/12/massively-parallel-hyperparameter-optimization/). The example was tested with ray version `ray==2.32.0`.
```
/* Detailed source-code truncated for AI context efficiency. */
```
## Hyperparameter optimization using `gridsearch()`
Each forecasting models in Darts offer a `gridsearch()` method for basic hyperparameter search.
This method is limited to very simple cases, with very few hyperparameters, and working with a single time series only.
---
### Userguide Safe Model Loading (`docs/userguide/safe_model_loading.md`)
# Safe model loading
Darts saves most forecasting models with Python `pickle` (`.pkl`) or, for PyTorch models, a Darts `.pt` wrapper plus a Lightning `.ckpt` checkpoint. Unpickling untrusted files can execute arbitrary code ([CWE-502](https://cwe.mitre.org/data/definitions/502.html)).
**Safe loading is the default** for all models, and related checkpoint/weight loaders. Restricted deserialization only reconstructs globals that pass Darts’ allow-list (known-safe libraries, class hierarchies, and checkpoint-driven registration for `.ckpt` files).
In most cases, safe loading should work out of the box.
## When loading fails
If a global is not allow-listed, loading raises an error that names the missing `{module}.{qualname}`. This can happen for example when using custom PyTorch Lightning callbacks, custom encoders, and others. If you trust that global (or file in general), you can still load using one of the steps below.
1. **Register missing globals as safe globals** — Same entry shapes as [PyTorch `add_safe_globals`](https://docs.pytorch.org/docs/stable/notes/serialization.html#torch.serialization.add_safe_globals) (a callable, or `(callable, qualname_str)` when the name in the file differs from the object’s module path). Registration binds each pickled `{module}.{qualname}` to the **exact** callable or class you provide, so loading works even when that name is not importable (for example functions defined in `__main__`, notebooks, or one-off scripts). Use the `(callable, qualname_str)` form when the string in the saved file does not match `{callable.__module__}.{callable.__qualname__}`:
```python
import pandas as pd
from darts.datasets import AirPassengersDataset
from darts.models import LinearRegressionModel
from darts.utils.serialization import add_safe_globals, safe_globals
def my_encoder(idx: pd.DatetimeIndex):
"""Custom encoder example adding the series' month value."""
return idx.month
# add encoder as a future covariate, fit and save the model
model = LinearRegressionModel(
lags=12,
lags_future_covariates=[0],
add_encoders={"custom": {"future": [my_encoder]}}
)
series = AirPassengersDataset().load()
model.fit(series)
model.save("model.pkl")
# by default, loading the model will fail since `my_encoder` is not an
# allow-listed global. To fix this, either add the safe globals within a context
with safe_globals([my_encoder]):
model = LinearRegressionModel.load("model.pkl")
# or add them globally via `add_safe_globals()`:
add_safe_globals([my_encoder])
model = LinearRegressionModel.load("model.pkl")
```
2. **Full trust opt-out** — Only if safe loading is not feasible and you fully trust the file source:
```python
model = NBEATSModel.load("model.pt", trusted=True)
```
## Related API
- Find more information in [the API docs](https://unit8co.github.io/darts/generated_api/darts.utils.serialization.registry.html)
---
### Userguide Timeseries (`docs/userguide/timeseries.md`)
# `TimeSeries`
`TimeSeries` is the main data class in Darts. A `TimeSeries` represents a univariate or multivariate time series, with a proper time index. The time index can either be of type `pandas.DatetimeIndex` (containing datetimes), or of type `pandas.RangeIndex` (containing integers; useful for representing sequential data without specific timestamps). In some cases, `TimeSeries` can even represent *probabilistic* series, in order for instance to obtain confidence intervals.
`TimeSeries` are used in Darts in order to offer a consistent API dedicated to time series:
* `TimeSeries` are guaranteed to have a proper time index (integer or datetime based): complete and time-sorted.
* All models in Darts consume `TimeSeries` and produce `TimeSeries`.
* `TimeSeries` allows to do all sort of time series operations (such as splitting at timestamps, concatenating, etc).
* `TimeSeries` offers user-friendly way to represent multidimensional and probabilistic series (e.g. obtain marginal quantiles).
Using a dedicated type to represent time series (instead of say, Pandas DataFrame, NumPy array, ...) removes the need to rely on conventions about the formats expected by the different models and functions.
## Multivariate time series -vs- multiple time series
We distinguish univariate from multivariate series:
* A **multivariate** series contain multiple dimensions (i.e. multiple values per time step).
* A **univariate** series contains only one dimension (i.e., single scalar value for each time step).
Sometimes the dimensions are called *components*. A single `TimeSeries` object can be either univariate (if it has a single component), or multivariate (if it has multiple components). In a multivariate series, all components share the same time axis. I.e., they all share the same time stamps.
Some models in Darts (and all machine learning models) support multivariate series. This means that they can take multivariate series in inputs (either as targets or as covariates), and the forecasts they produce will have a dimensionality matching that of the targets.
In addition, some models can work on *multiple time series*, meaning that they can be trained on multiple `TimeSeries` objects, and used to forecasts multiple `TimeSeries` objects in one go. This is sometimes referred to as panel data. In such cases, the different `TimeSeries` need not share the same time index -- for instance, some series might be in 1990 and others in 2000. In fact, the series need not even have the same frequency. The models handling multiple series expect Python `Sequence`s of `TimeSeries` in inputs (for example, a simple list of `TimeSeries`).
**Figure 1:** A single multivariate series. In Darts this is represented by a single `TimeSeries` object.
**Figure 2:** Multiple time series. In Darts this is represented by multiple `TimeSeries` objects. The series may or may not share the same time axis, and may or may not be multivariate.
* **Example of a multivariate series:** The blood pressure and heart rate of a single patient over time (one multivariate series with 2 components).
* **Example of multiple series:** The blood pressure and heart rate of multiple patients; potentially measured at different times for different patients (one multivariate series with 2 components per patient).
### Should I use a multivariate series or multiple series for my problem?
Using multivariate series will provide an opportunity for models to capture the dimensions jointly, whereas using multiple series will expose the models to multiple observations. So which representation is right is problem-dependent, and in some cases might not even be clear beforehand and require experimentation. Nevertheless, if you are in a situation where you are trying to model several "correlated" components varying together across time (e.g. the coordinates of a moving object, or multiple measurements related to a single entity), it is often natural to capture those as multiple components in single series. If on the other hand you have several instance of such measurements (e.g., obtained in different experiments or observations), this can mean that you have multiple time series. Furthermore, if your values do not share the same time spans, they should probably be represented as different series.
## Probabilistic and deterministic series
In Darts, probabilistic forecasts are represented by drawing Monte Carlo samples from the underlying probabilistic models. This representation allows `TimeSeries` to represent arbitrary joint distributions (over time and components) without relying on any pre-defined parametric form. Based on this, we define two types of time series:
* A **Probabilistic** (or **stochastic**) series contains multiple samples.
* A **deterministic** series contains only one sample.
## Creating `TimeSeries`
`TimeSeries` objects can be created using factory methods, for example:
* [TimeSeries.from_dataframe()](https://unit8co.github.io/darts/generated_api/darts.timeseries.html#darts.timeseries.TimeSeries.from_dataframe) can create `TimeSeries` from a Pandas Dataframe having one or several columns representing values (columns correspond to components, and several columns would correspond to a multivariate series).
* [TimeSeries.from_values()](https://unit8co.github.io/darts/generated_api/darts.timeseries.html#darts.timeseries.TimeSeries.from_values) can create `TimeSeries` from a 1-D, 2-D or 3-D NumPy array. It will generate an integer-based time index (of type `pandas.RangeIndex`). 1-D corresponds to univariate deterministic series, 2-D to multivariate deterministic series, and 3-D to multivariate stochastic series.
* [TimeSeries.from_times_and_values()](https://unit8co.github.io/darts/generated_api/darts.timeseries.html#darts.timeseries.TimeSeries.from_times_and_values) is similar to `TimeSeries.from_values()` but also accepts a time index.
More methods are documented in the [TimeSeries API documentation](https://unit8co.github.io/darts/generated_api/darts.timeseries.html).
Furthermore, it is possible to concatenate series along different axes using the function `concatenate()`. `axis=0` corresponds to time, `axis=1` corresponds to component and `axis=2` correspond to stochastic sample dimensions. For instance:
```python
from darts import concatenate
my_multivariate_series = concatenate([series1, series2, ...], axis=1)
```
produces a multivariate series from some series that share the same time axis.
## Implementation
Behind the scenes, `TimeSeries` is wrapping around a 3-dimensional `xarray.DataArray` object. The dimensions are *(time, component, sample)*, where the size of the *component* dimension is larger than 1 for multivariate series and the size of the *sample* dimension is larger than 1 for stochastic series. The `DataArray` is itself backed by a 3-dimensional NumPy array, and it has a time index (either `pandas.DatetimeIndex` or `pandas.RangeIndex`) on the *time* dimension and another `pandas.Index` on the *component* (or "columns") dimension. `TimeSeries` is intended to be immutable and most operations return new `TimeSeries` objects.
## Exporting data from a `TimeSeries`
`TimeSeries` objects offer a few ways to export the data, for example:
* [TimeSeries.to_dataframe()](https://unit8co.github.io/darts/generated_api/darts.timeseries.html#darts.timeseries.TimeSeries.pd_dataframe) to export a Dataframe (for deterministic series)
* [TimeSeries.data_array()](https://unit8co.github.io/darts/generated_api/darts.timeseries.html#darts.timeseries.TimeSeries.data_array) to export the `xarray` `DataArray` holding this series' data (and indexes).
* [TimeSeries.values()](https://unit8co.github.io/darts/generated_api/darts.timeseries.html#darts.timeseries.TimeSeries.values) to export a NumPy array containing the values of one sample from the series.
* [TimeSeries.all_values()](https://unit8co.github.io/darts/generated_api/darts.timeseries.html#darts.timeseries.TimeSeries.all_values) to export a NumPy array containing the values of all samples of a stochastic series.
## Static Covariates
Optionally, `TimeSeries` objects can contain static data (referred to as *static covariates*), which can be exploited by some models. Example of static covariates can be:
* Store location - for instance for each store (component) in a multivariate series
* Product ID
* Sensor type
* ...
The static covariates have to be specified by a pandas DataFrame whose rows match the `TimeSeries`'s components, and whose columns represent the dimensions of the static covariates. They can be added to a `TimeSeries` when using most factory methods, using the `static_covariates` parameter. They can also be added to an existing `TimeSeries` using the `with_static_covariates()` method.
## Hierarchical Time Series
Optionally, `TimeSeries` objects can contain a hierarchy, which specifies how its different components are grouped together. The hierarchy itself is specified as a Python dictionary, mapping the components' names to a list of names of their parent(s) in the hierarchy.
For instance, the following hierarchy means that the two components `"a"` and `"b"` add up to `"total"`:
```python
hierarchy = {"a": ["total"], "b": ["total"]}
```
Hierarchies can be used for posthoc forecast reconciliation. Darts offers several reconciliation transformers (usable with `fit()`/`transform()`) - see the [corresponding API documentation](https://unit8co.github.io/darts/generated_api/darts.dataprocessing.transformers.reconciliation.html).
## More information and documentation
The full list of attributes and methods of the `TimeSeries` class is listed in the [API Documentation](https://unit8co.github.io/darts/generated_api/darts.timeseries.html). In addition, the [Quickstart](https://unit8co.github.io/darts/quickstart/00-quickstart.html#Building-and-manipulating-TimeSeries) contains some examples of `TimeSeries` manipulation.
---
### Userguide Torch Forecasting Models (`docs/userguide/torch_forecasting_models.md`)
# Torch Forecasting Models
This document was written for darts version 0.15.0 and later.
We assume that you already know about covariates in Darts. If you're new to the topic we recommend you to read our [guide on covariates](https://unit8co.github.io/darts/userguide/covariates.html) first.
## Content of this guide
1. Introduction section covers the most important points about Torch Forecasting Models (TFMs):
- [How to use TFMs](#introduction)
- [Top-level look at chunks](#top-level-look-at-training-and-predicting-with-chunks)
- [TFM covariates support](#torch-forecasting-model-covariates-support)
- [Time span requirements for target and covariate series](#required-target-time-spans-for-training-validation-and-prediction)
2. Input data usage section gives an in-depth guide of how input data is used when training and predicting with TFMs:
- [Simple training](#training)
- [Training with validation set](#training-with-a-validation-dataset)
- [Forecast and prediction](#forecasting)
3. Advanced functionalities section provides some example of TFMs advanced features:
- [Model saving and loading](#saving-and-loading-model-states)
- [Checkpoint saving and loading](#automatic-checkpointing)
- [Manual model saving and loading](#manual-saving-and-loading)
- [GPU training and CPU loading](#training-and-saving-on-gpu-loading-on-cpu)
- [Re-training or fine-tuning a pre-trained model](#retraining-or-finetuning-a-pretrained-model)
- [ONNX export for inference](#exporting-model-to-onnx-format-for-inference)
- [Using callbacks](#callbacks)
- [Early Stopping](#example-with-early-stopping)
- [Custom Callback](#example-of-custom-callback-to-store-losses)
4. [Performance optimization section](#performance-recommendations) lists tricks to speed up the computation during training.
## Introduction
In Darts, **Torch Forecasting Models (TFMs)** are broadly speaking "machine learning based" models, which denote PyTorch-based (deep learning) models.
TFMs train and predict on fixed-length chunks (sub-samples) of your input `target` and `*_covariates` series (if supported). `Target` is the series for which we want to predict the future, `*_covariates` are the past and / or future covariates.
Each chunk contains an input chunk - representing the sample's past - and an output chunk - the sample's future. The sample's prediction point lies at the end of the input chunk. The length of these chunks has to be specified at model creation with parameters `input_chunk_length` and `output_chunk_length` (more on chunks in [the next subsection](#top-level-look-at-training-and-predicting-with-chunks)).
```python
# model that looks 7 time steps back (past) and 1 time step ahead (future)
model = SomeTorchForecastingModel(input_chunk_length=7,
output_chunk_length=1,
**model_kwargs)
```
All TFMs can be trained on single or multiple `target` series and, depending on their covariate support (covered in [this subsection on covariates support](#torch-forecasting-model-covariates-support)), `past_covariates` and / or `future_covariates`. When using covariates you have to supply one dedicated past and / or future covariates series for each target series.
Optionally, you can use a validation set with dedicated covariates during training. If the covariates have the required time spans, you can use the same for training, validation and prediction. (covered in [this subsection on time span requirements](#required-target-time-spans-for-training-validation-and-prediction))
```python
# fit the model on a single target series with optional past and / or future covariates
model.fit(target,
past_covariates=past_covariates,
future_covariates=future_covariates,
val_series=target_val, # optionally, use a validation set
val_past_covariates=past_covariates_val,
val_future_covariates=future_covariates_val)
# fit the model on multiple target series
model.fit([target, target2, ...],
past_covariates=[past_covariates, past_covariates2, ...],
...
)
```
You can produce forecasts for any input `target` TimeSeries or for several targets given as a sequence of TimeSeries. This will also work on series that have not been seen during training, as long as each series contains at least `input_chunk_length` time steps.
```python
# predict the next n=3 time steps for any input series with `series`
prediction = model.predict(n=3,
series=target,
past_covariates=past_covariates,
future_covariates=future_covariates)
```
If you want to know more about the training and prediction process of our Torch Forecasting Models and how they work with covariates, read on.
## Top level look at training and predicting with chunks
In Figure 1 you can see how your data is distributed to the input and output chunks for each sample when calling `fit()` or `predict()`. For this example we look at data with daily frequency. The input chunk extracts values from `target` and optionally from `past_covariates` and / or `future_covariates` that fall into the input chunk time span. These "past" values of `future_covariates` are called "historic future covariates".
The output chunk only takes optional `future_covariates` values that fall into the output chunk time span. The future values of our `past_covariates` - "future past covariates" - are only used to provide the input chunk of upcoming samples with new data.
All this information is used to predict the "future target" - the next `output_chunk_length` points after the end of "past target".
**Figure 1: Top level look at training / predicting on chunks with Torch Forecasting Models**
When calling `predict()` and depending on your forecast horizon `n`, the model can either predict in one go (if `n <= output_chunk_length`), or auto-regressively, by predicting on multiple chunks in the future (if `n > output_chunk_length`). That is the reason why when predicting with `past_covariates` you might have to supply additional "future values of your `past_covariates`".
## Torch Forecasting Model Covariates Support
Under the hood, Darts has 5 types of `{X}CovariatesModel` classes implemented to cover different combinations of the covariate types mentioned before:
| Class | past covariates | future past covariates | future covariates | historic future covariates |
|-------------------------|:---------------:|:----------------------:|:-----------------:|:--------------------------:|
| `PastCovariatesModel` | ✅ | ✅ | | |
| `FutureCovariatesModel` | | | ✅ | |
| `DualCovariatesModel` | | | ✅ | ✅ |
| `SplitCovariatesModel` | ✅ | ✅ | ✅ | |
| `MixedCovariatesModel` | ✅ | ✅ | ✅ | ✅ |
**Table 1: Darts' "{X}CovariatesModels" covariate support**
Each Torch Forecasting Model inherits from one `{X}CovariatesModel` (covariate class names are abbreviated by the `X`-part):
| TFM | `Past` | `Future` | `Dual` | `Split` | `Mixed` |
|--------------------|:------:|:--------:|:------:|:-------:|:-------:|
| `RNNModel` | | | ✅ | | |
| `BlockRNNModel` | ✅ | | | | |
| `NBEATSModel` | ✅ | | | | |
| `TCNModel` | ✅ | | | | |
| `TransformerModel` | ✅ | | | | |
| `TFTModel` | | | | | ✅ |
| `NLinearModel` | | | | | ✅ |
| `DLinearModel` | | | | | ✅ |
| `TiDEModel` | | | | | ✅ |
| `TSMixerModel` | | | | | ✅ |
**Table 2: Darts' Torch Forecasting Model covariate support**
## Required target time spans for training, validation and prediction
The relevant data is extracted automatically by the models, based on the time axes of the series.
You can use the same covariates series for both `fit()` and `predict()` if they meet the requirements below.
**Training** only works if at least one sample with an input and output chunk can be extracted from the data you passed to `fit()`. This applies both to training and validation data. In terms of minimum required time spans, this means:
- `target` series of minimum length `input_chunk_length + output_chunk_length`
- `*_covariates` time span requirements for `fit()` from [covariates guide section 2.3.](https://unit8co.github.io/darts/userguide/covariates.html#id6)
For **prediction** you have to supply the `target` series that you wish to forecast. For any forecast horizon `n` the minimum time span requirements are:
- `target` series of minimum length `input_chunk_length`
- `*_covariates` time span requirements for `predict()` also from from [covariates guide section 2.3.](https://unit8co.github.io/darts/userguide/covariates.html#id6)
Side note: Our `*RNNModels` accept a `training_length` parameter at model creation instead of `output_chunk_length`. Internally the `output_chunk_length` for these models is automatically set to `1`. For training, past `target` must have a minimum length of `training_length + 1` and for prediction, a length of `input_chunk_length`.
## In-depth look at how input data is used when training and predicting with TFMs
### Training
Let's have a look at how the models work under the hood.
Let's assume we run an ice-cream shop and we want to predict sales for the next day.
We have one year (365 days) past data of our end-of-day ice-cream sales and of the average measured daily ambient temperature.
We also noticed that our ice-cream sales depend on the day of the week so we want to include this in our model.
- past target: actual past ice-cream sales `ice_cream_sales`
- future target: predict the ice-cream sales for the next day
- past covariates: measured average daily temperatures in the past `temperature`
- future covariates: day of the week for past and future `weekday`
Checking Table 1, a model that would accommodate this kind of covariates would be a
`SplitCovariatesModel` (if we don't use historic values of future covariates), or
`MixedCovariatesModel` (if we do). We choose a `MixedCovariatesModel` - the `TFTModel`.
Imagine that we saw a pattern in our past ice cream sales that repeated week after week.
So we set `input_chunk_length = 7` days to let the model look back an entire week into the past.
The `output_chunk_length` can be set to `1` day to predict the next day.
Now we can create a model and train it! Figure 2 shows you how `TFTModel` will use our data.
```python
from darts.models import TFTModel
model = TFTModel(input_chunk_length=7, output_chunk_length=1)
model.fit(series=ice_cream_sales,
past_covariates=temperature,
future_covariates=weekday)
```
**Figure 2: Overview of a single sequence from our ice-cream sales example**; Mon1 - Sun1 stand for the first 7 days from our training dataset (week 1 of the year). Mon2 is the Monday of week 2.
When calling `fit()`, the models will build an appropriate `darts.utils.data.TorchTrainingDataset`, which specifies how to slice the data to obtain training samples. If you want to control this slicing yourself, you can instantiate your own `TorchTrainingDataset` and call `model.fit_from_dataset()` instead of `fit()`. By default, most models (though not all) will build *sequential* datasets, which basically means that all sub-slices of length `input_chunk_length + output_chunk_length` in the provided series will be used for training.
So during training, the torch models will go through the training data in sequences (see Figure 3). Using information from the **input chunk** and **output chunk**, the model predicts the future target on the output chunk. The training loss is evaluated between the predicted future target and the actual target value on the output chunk. The model trains itself by minimizing the loss over all sequences.
**Figure 3: Prediction and loss evaluation in a single sequence**
After having completed computations on the first sequence, the model moves to the next one and performs the same training steps. *The starting point of each sequence is selected randomly from the sequential dataset*. Figure 4 shows how this would look like if by pure chance the second sequence started one time step (day) after the first.
This sequence-to-sequence process is repeated until all 365 days were covered.
Side note: Having "long" `target` series can result in a very large number of training sequences / samples. You can set an upper bound for the number of sequences / samples per `target` that the model should be trained on with `fit()`-parameter `max_samples_per_ts`. This will take the most recent sequences for every `target` series (sequences closest to `target` end).
```python
# fit only on the 10 "most recent" sequences
model.fit(target, max_samples_per_ts=10)
```
**Figure 4: Sequence-to-sequence: Move to next sequence and repeat training steps**
### Training with a validation dataset
You can also train your models with a validation dataset:
```python
# create train and validation sets
ice_cream_sales_train, ice_cream_sales_val = ice_cream_sales.split_after(training_cutoff)
# train with validation set
model.fit(series=ice_cream_sales_train,
past_covariates=temperature,
future_covariates=weekday,
val_series=ice_cream_sales_val,
val_past_covariates=temperature,
val_future_covariates=weekday)
```
If you split your data, you have to define a `training_cutoff` (a date or fraction at which to split the dataset) so that both the train and validation datasets satisfy the minimum length requirements
from [this subsection on time span requirements](#required-target-time-spans-for-training-validation-and-prediction)
Instead of splitting by time, you can also use another subset of time series as validation set.
The model trains itself the same way as before but additionally evaluates the loss on the validation dataset. If you want to keep track of the best performing model on the validation set, you have to enable checkpoint saving as shown next.
## Forecasting
After having trained the model, we want to predict the future ice-cream sales for any number of days after our 365 days training data.
The actual prediction works very similar to how we trained the data on sequences. Depending on the number of days we want to predict - the forecast horizon `n` - we distinguish between two cases:
- If `n <= output_chunk_length`: we can predict `n` in one go (using one "internal model call")
- in our example: predict the next day's ice-cream sales (`n = 1`)
- If `n > output_chunk_length`: we must predict `n` by calling the internal model multiple times. Each call outputs `output_chunk_length` prediction points. We go through as many calls as needed until we get to the final `n` prediction points, in an auto-regressive fashion.
- in our example: predict ice-cream sales for the next 3 days at once (`n = 3`)
To do this we have to supply additional `past_covariates` for the next `n - output_chunk_length = 2` time steps (days) after the end of our 365 days training data. Unfortunately, we do not have measured `temperature` for the future. But let's assume we have access to temperature forecasts for the next 2 days. We can just append them to `temperature` and the prediction will work!
```python
temperature = temperature.concatenate(temperature_forecast, axis=0)
```
```python
prediction = model.predict(n=n,
series=ice_cream_sales_train,
past_covariates=temperature,
future_covariates=weekday)
```
**Figure 5: Forecast with a single sequence for n <= output_chunk_length**
**Figure 6: Auto-regressive forecast for n > output_chunk_length**
## Advanced Functionnalities
### Saving and Loading Model States
Darts offers several ways for saving and loading TFMs. Two artifacts are stored on every save: a Darts `.pt` wrapper and a PyTorch (Lightning) `.ckpt` checkpoint containing the internal model's state dict and states of other objects.
**Security:** Loading uses safe-by-default deserialization to mitigate malicious pickle files (CWE-502). Use `trusted=True` only for files you fully trust, or register custom classes/functions via `add_safe_globals` / `with safe_globals(...)`. See the [safe model loading guide](safe_model_loading.md) for details.
**Note:** Backward compatibility is not always guaranteed. Loading a model saved by an older Darts version might fail.
#### Automatic checkpointing
Automic checkpointing during training allows you to:
- keep track of the model state over the latest 5 epochs, and the best performing epoch based on the validation set loss
- load a model from checkpoint to resume training in case it was interrupted
- load a model from checkpoint for inference / forecasting
You can activate checkpointing at model creation:
```python
model = SomeTorchForecastingModel(..., model_name='my_model', save_checkpoints=True)
# checkpoints are saved automatically
model.fit(...)
# load the model state that performed best on validation set
best_model = model.load_from_checkpoint(model_name='my_model', best=True)
```
#### Manual saving and loading
You can also manually save the model at its current state and load it:
```python
model.save("/your/path/to/save/model.pt")
loaded_model = model.load("/your/path/to/save/model.pt")
```
#### Training and saving on GPU, loading on CPU
You can load a model to CPU that was trained and saved on GPU (see detailed [documentation](https://unit8co.github.io/darts/userguide/gpu_and_tpu_usage.html)):
```python
# define a model using gpu as accelerator
model = SomeTorchForecastingModel(
...,
model_name='my_model',
save_checkpoints=True,
pl_trainer_kwargs={"accelerator":"gpu", "devices": -1},
)
# train the model, automatic checkpoints will be created
model.fit(...)
# specify the device to which the model should be loaded
loaded_model = SomeTorchForecastingModel.load_from_checkpoint(
model_name='my_model',
best=True,
map_location="cpu",
)
loaded_model.to_cpu()
# run inference
loaded_model.predict(...)
```
Manual saves can also be loaded to CPU:
```python
model.save("/your/path/to/save/model.pt")
loaded_model = model.load("/your/path/to/save/model.pt", map_location="cpu")
loaded_model.to_cpu()
```
#### Retraining or finetuning a pretrained model
To re-train or fine-tune a model using a different optimizer and/or learning rate scheduler, you can load the weights from the automatic checkpoints into a new model:
```python
# model with identical architecture but different optimizer (default: torch.optim.Adam)
model_finetune = SomeTorchForecastingModel(
..., # use identical parameters & values as in original model
optimizer_cls=torch.optim.SGD,
optimizer_kwargs={"lr": 0.001},
)
# load the weights from a checkpoint
model_finetune.load_weights_from_checkpoint(model_name='my_model', best=True)
model_finetune.fit(...)
```
and similarly for manual saves and the learning rate scheduler:
```python
# model with identical architecture but different lr scheduler (default: None)
model_finetune = SomeTorchForecastingModel(
..., # use identical parameters & values as in original model
lr_scheduler_cls=torch.optim.lr_scheduler.ExponentialLR,
lr_scheduler_kwargs={"gamma": 0.09}
)
# load the weights from a manual save
model_finetune.load_weights("/your/path/to/save/model.pt")
```
Additionally, all `TorchForecastingModel` (including `FoundationModel`) support fine-tuning through the `enable_finetuning` parameter. This allows you to freeze or unfreeze specific model parameters for transfer learning:
```python
# full fine-tuning: all parameters are trainable
model = SomeTorchForecastingModel(..., enable_finetuning=True)
# partial fine-tuning: freeze specific layers
model = SomeTorchForecastingModel(..., enable_finetuning={"freeze": ["layer.pattern.*"]})
# partial fine-tuning: only unfreeze specific layers
model = SomeTorchForecastingModel(..., enable_finetuning={"unfreeze": ["layer.pattern.*"]})
# optionally, load pre-trained weights (not required for foundation models)
model.load_weights("/your/path/to/save/model.pt")
# fine-tune
model.fit(...)
```
For a comprehensive walkthrough of fine-tuning Torch Forecasting Models and Foundation Models, check out the [Fine-Tuning example notebook](https://unit8co.github.io/darts/examples/27-Torch-and-Foundation-Model-Fine-Tuning-examples.html).
#### Exporting model to ONNX format for inference
Exporting models to ONNX format and running inference optional dependencies `onnx>=1.0.0`, `onnxruntime>=1.24.1`, and `onnxscript>=0.7.0`. After export, inference can run **without PyTorch** using only `onnxruntime`, NumPy, and Darts' torch-free helpers in `darts.utils.onnx.inference`.
`to_onnx()` exports the model to ONNX format.
```python
model = SomeTorchForecastingModel(...)
model.fit(...)
horizon = model.output_chunk_length
# export requires torch + onnx
onnx_filename = "example.onnx"
model.to_onnx(onnx_filename, export_params=True)
```
Now forecast using Darts' `run_onnx_prediction()` which mirrors `TorchForecastingModel.predict()` for deterministic models with ``num_samples=1`` and probabilistic models with ``predict_likelihood_parameters=True``.
```python
import onnxruntime as ort
from darts.utils.onnx import run_onnx_prediction
session = ort.InferenceSession(onnx_filename)
# returns forecasts as (a sequence of) `TimeSeries`
forecast = run_onnx_prediction(
n=horizon,
session=session,
series=series,
past_covariates=ts_past,
future_covariates=ts_future,
)
```
For a single ONNX step (advanced / custom loops), extract features with `prepare_onnx_inputs` using the spec:
```python
from darts.utils.onnx.inference import OnnxModelSpec, prepare_onnx_inputs
session = ort.InferenceSession(onnx_filename)
spec = OnnxModelSpec.from_session(session)
onnx_inputs = prepare_onnx_inputs(
series=series,
spec=spec,
past_covariates=ts_past,
future_covariates=ts_future,
)
forecast = session.run(spec.output_names, onnx_inputs)
```
Raw ONNX output tensors (NumPy arrays) have shape `(batch, output_chunk_length, n_components, n_likelihood_params)`. Models trained with a `likelihood` export those raw parameters (sampling is not in the graph).
### Callbacks
Callbacks are a powerful way to monitor or control the behavior of the model during the training process. Some examples:
- Performance Monitoring: compute additional metrics (in addition of the default losses)
- Early stopping: stop the training once the model has converged
- ...
With callbacks you can add custom code to an existing process at predefined points / hooks.
The code is triggered once the process execution reaches the corresponding hooks. Some example hooks:
- beginning / end of training
- beginning / end of train / validation step
- ...
Some useful predefined PyTorch Lightning callbacks can be found [here](https://lightning.ai/docs/pytorch/stable/extensions/callbacks.html#built-in-callbacks).
#### Example with Early Stopping
Early stopping is an efficient way to avoid overfitting and reduce training time.
It will exit the training process once the validation loss has not significantly improved over some epochs.
You can use Early Stopping with any `TorchForecastingModel`, leveraging PyTorch Lightning's [EarlyStopping](https://lightning.ai/docs/pytorch/stable/api/lightning.pytorch.callbacks.EarlyStopping.html#lightning.pytorch.callbacks.EarlyStopping) callback:
```python
import pandas as pd
from pytorch_lightning.callbacks import EarlyStopping
from torchmetrics import MeanAbsolutePercentageError
from darts.dataprocessing.transformers import Scaler
from darts.datasets import AirPassengersDataset
from darts.models import NBEATSModel
# read data
series = AirPassengersDataset().load()
# create training and validation sets:
train, val = series.split_after(pd.Timestamp(year=1957, month=12, day=1))
# normalize the time series
transformer = Scaler()
train = transformer.fit_transform(train)
val = transformer.transform(val)
# any TorchMetric or val_loss can be used as the monitor
torch_metrics = MeanAbsolutePercentageError()
# early stop callback
my_stopper = EarlyStopping(
monitor="val_MeanAbsolutePercentageError", # "val_loss",
patience=5,
min_delta=0.05,
mode='min',
)
pl_trainer_kwargs = {"callbacks": [my_stopper]}
# create the model
model = NBEATSModel(
input_chunk_length=24,
output_chunk_length=12,
n_epochs=500,
torch_metrics=torch_metrics,
pl_trainer_kwargs=pl_trainer_kwargs)
# use validation set for early stopping
model.fit(
series=train,
val_series=val,
)
```
To use early-stopping and pruning in the context of hyperparameter optimization, check out [this guide](https://unit8co.github.io/darts/userguide/hyperparameter_optimization.html).
#### Example of custom callback to store losses
Training and validation loss can be automatically logged with [tensorboard](https://www.tensorflow.org/tensorboard). When activated, Darts will by default store the logs to a folder `darts_logs` in the current working directory. You can change this with model parameters `work_dir`, and `model_name`.
```python
model = SomeTorchForecastingModel(..., log_tensorboad, save_checkpoints=True)
model.fit(...)
```
After installing the tensorboard library, you can visualize the logs from the command line:
```bash
tensorboad --log_dir darts_logs
```
Let's check out how to implement a **custom callback** to make the model losses accessible in Python.
```python
from pytorch_lightning.callbacks import Callback
class LossLogger(Callback):
def __init__(self):
self.train_loss = []
self.val_loss = []
# will automatically be called at the end of each epoch
def on_train_epoch_end(self, trainer: "pl.Trainer", pl_module: "pl.LightningModule") -> None:
self.train_loss.append(float(trainer.callback_metrics["train_loss"]))
def on_validation_epoch_end(self, trainer: "pl.Trainer", pl_module: "pl.LightningModule") -> None:
self.val_loss.append(float(trainer.callback_metrics["val_loss"]))
loss_logger = LossLogger()
model = SomeTorchForecastingModel(
...,
nr_epochs_val_period=1, # perform validation after every epoch
pl_trainer_kwargs={"callbacks": [loss_logger]}
)
# fit must include validation set for "val_loss"
model.fit(...)
```
*Note* : The callback will give one more element in the `loss_logger.val_loss` as the model trainer performs a validation sanity check before the training begins.
## Performance Recommendations
This section recaps the main factors impacting the performance when
training and using torch-based models.
### Build your `TimeSeries` using 32-bits data
The models in Darts will dynamically cast themselves (to 64 or 32-bits)
to follow the dtype in the `TimeSeries`. Large performance and memory gains
can often be obtained when everything (data and model) is in float32.
To achieve this, it is enough to build your `TimeSeries` from arrays (or Dataframe-backing array) having dtype `np.float32`, or simply call `my_series32 = my_series.astype(np.float32)`. Calling `my_series.dtype` gives you the dtype of your `TimeSeries`.
### Use a GPU
In many cases using a GPU will provide a drastic speedup compared to CPU.
It can also incur some overheads (for transferring data to/from the GPU),
so some testing and tuning is often necessary.
We refer to our [GPU/TPU guide](https://unit8co.github.io/darts/userguide/gpu_and_tpu_usage.html)
for more information on how to setup a GPU (or a TPU) via PyTorch Lightning.
### Tune the batch size
A larger batch size tends to speed up the training because it reduces the number
of backward passes per epoch and has the potential to better parallelize computation. However it also changes the training dynamics (e.g. you might need more epochs, and the convergence dynamics is affected). Furthermore larger batch sizes increase memory consumption. So here too some testing is required.
### Tune `num_loader_workers`
All deep learning models in Darts have a parameter `dataloader_kwargs` in their `fit()` and `predict()` functions, which configures the PyTorch DataLoaders. The `num_workers` parameter for PyTorch DataLoaders can be set using the `num_workers` key in the `dataloader_kwargs` dictionary.
Setting `num_workers > 0` will use additional workers to load the data. This typically incurs some overhead (notably increasing memory consumption), but in some cases it can also substantially improve performance.
The ideal value depends on many factors such as the batch size, whether you are using a GPU, the number of CPU cores available, and whether
loading the data involved I/O operations (if the series are stored on disk).
### Small models first
Of course one of the main factors affecting performance is the model size
(number of parameters) and the number of operations required by forward/backward passes. Models in Darts can be tuned (e.g. number of layers, attention heads, widths etc), and these hyper-parameters tend to have a large impact on performance. When starting out, it is a good idea to build models of modest size first.
### Data in Memory and I/O bottlenecks
It's helpful to load all your `TimeSeries` in memory upfront if you can.
Darts offers the possibility to train models on any `Sequence[TimeSeries]`,
which means that for big datasets, you can write your own `Sequence` implementation, and read the time series lazily from disk. This will typically incur a high I/O cost, though. So when training on multiple series, first try to build a simple `List[TimeSeries]` upfront, and see if it holds in the computer memory.
### Do not use *all* possible sub-series for training
By default, when calling `fit()`, the models in Darts will build a `TorchTrainingDataset` instance that is
suitable for the model that you are using (e.g., `PastCovariatesTorchModel`, `FutureCovariatesTorchModel`, etc.).
By default, these training datasets will often contain *all* possible consecutive (input, output) subseries present
in each `TimeSeries`. If your `TimeSeries` are long, this can result in a large amount of training samples, which directly (linearly)
impacts the time required to train the model for one epoch. You have two options to limit this:
* Specify some `max_samples_per_ts` argument to the `fit()` function. This will use only the most recent `max_samples_per_ts` samples
per `TimeSeries` for training.
* If this option does not do what you want, you can implement your own `TorchTrainingDataset` instance, and define
how to slice your `TimeSeries` for training yourself. Each `__getitem__` must return a `TorchTrainingSample`. We suggest to have a look at [this submodule](https://github.com/unit8co/darts/tree/master/darts/utils/data/torch_datasets)
to see examples of how to do it.
-------------
### Example Benchmark
As an example, we show here the time required to train one epoch on the first 80% of the energy dataset (`darts.datasets.EnergyDataset`), which consists of one multivariate series that is 28050 timesteps long and has 28 dimensions.
We train two models; `NBEATSModel` and `TFTModel`, with default parameters and `input_chunk_length=48` and `output_chunk_length=12` (which results in 27991 training samples with default sequential training datasets). For the TFT model, we also set the parameter `add_cyclic_encoder='hour'`. The tests are made on a Intel CPU i9-10900K CPU @ 3.70GHz, with an Nvidia RTX 2080s GPU, 32 GB of RAM. All `TimeSeries` are pre-loaded in memory and given to the models as a list.
| Model | Dataset | dtype | CUDA | Batch size | num workers | time per epoch |
|---------------|---------|-------|------|------------|-------------|----------------|
| `NBEATSModel` | Energy | 64 | no | 32 | 0 | 283s |
| `NBEATSModel` | Energy | 64 | no | 32 | 2 | 285s |
| `NBEATSModel` | Energy | 64 | no | 32 | 4 | 282s |
| `NBEATSModel` | Energy | 64 | no | 1024 | 0 | 58s |
| `NBEATSModel` | Energy | 64 | no | 1024 | 2 | 57s |
| `NBEATSModel` | Energy | 64 | no | 1024 | 4 | 58s |
| `NBEATSModel` | Energy | 64 | yes | 32 | 0 | 63s |
| `NBEATSModel` | Energy | 64 | yes | 32 | 2 | 62s |
| `NBEATSModel` | Energy | 64 | yes | 1024 | 0 | 13.3s |
| `NBEATSModel` | Energy | 64 | yes | 1024 | 2 | 12.1s |
| `NBEATSModel` | Energy | 64 | yes | 1024 | 4 | 12.3s |
| | | | | | | |
| `NBEATSModel` | Energy | 32 | no | 32 | 0 | 117s |
| `NBEATSModel` | Energy | 32 | no | 32 | 2 | 115s |
| `NBEATSModel` | Energy | 32 | no | 32 | 4 | 117s |
| `NBEATSModel` | Energy | 32 | no | 1024 | 0 | 28.4s |
| `NBEATSModel` | Energy | 32 | no | 1024 | 2 | 27.4s |
| `NBEATSModel` | Energy | 32 | no | 1024 | 4 | 27.5s |
| `NBEATSModel` | Energy | 32 | yes | 32 | 0 | 41.5s |
| `NBEATSModel` | Energy | 32 | yes | 32 | 2 | 40.6s |
| `NBEATSModel` | Energy | 32 | yes | 1024 | 0 | 2.8s |
| `NBEATSModel` | Energy | 32 | yes | 1024 | 2 | 1.65 |
| `NBEATSModel` | Energy | 32 | yes | 1024 | 4 | 1.8s |
| | | | | | | |
| `TFTModel` | Energy | 64 | no | 32 | 0 | 78s |
| `TFTModel` | Energy | 64 | no | 32 | 2 | 72s |
| `TFTModel` | Energy | 64 | no | 32 | 4 | 72s |
| `TFTModel` | Energy | 64 | no | 1024 | 0 | 46s |
| `TFTModel` | Energy | 64 | no | 1024 | 2 | 38s |
| `TFTModel` | Energy | 64 | no | 1024 | 4 | 39s |
| `TFTModel` | Energy | 64 | yes | 32 | 0 | 125s |
| `TFTModel` | Energy | 64 | yes | 32 | 2 | 115s |
| `TFTModel` | Energy | 64 | yes | 1024 | 0 | 59s |
| `TFTModel` | Energy | 64 | yes | 1024 | 2 | 50s |
| `TFTModel` | Energy | 64 | yes | 1024 | 4 | 50s |
| | | | | | | |
| `TFTModel` | Energy | 32 | no | 32 | 0 | 70s |
| `TFTModel` | Energy | 32 | no | 32 | 2 | 62.6s |
| `TFTModel` | Energy | 32 | no | 32 | 4 | 63.6 |
| `TFTModel` | Energy | 32 | no | 1024 | 0 | 31.9s |
| `TFTModel` | Energy | 32 | no | 1024 | 2 | 45s |
| `TFTModel` | Energy | 32 | no | 1024 | 4 | 44s |
| `TFTModel` | Energy | 32 | yes | 32 | 0 | 73s |
| `TFTModel` | Energy | 32 | yes | 32 | 2 | 58s |
| `TFTModel` | Energy | 32 | yes | 1024 | 0 | 41s |
| `TFTModel` | Energy | 32 | yes | 1024 | 2 | 31s |
| `TFTModel` | Energy | 32 | yes | 1024 | 4 | 31s |
---
--- METRICS ---
- Files Extracted: 12
- Estimated Token Budget: ~57339 tokens
- Canonical Reference: https://codewiki.google/github.com/unit8co/darts
