Technical Documentation: unit8co/darts
โน๏ธ Provenance: Multi-Tier Ingestion ยท CodeWiki Reference
Overview (README.md)
Time Series Made Easy in Python
Darts is a Python library for user-friendly forecasting and anomaly detection
on time series. It contains a variety of models, from classics such as ARIMA to
deep neural networks. The forecasting models can all be used in the same way,
using fit() and predict() functions, similar to scikit-learn.
The library also makes it easy to backtest models,
combine the predictions of several models, and take external data into account.
Darts supports both univariate and multivariate time series and models.
The ML-based models can be trained on potentially large datasets containing multiple time
series, and some of the models offer a rich support for probabilistic forecasting.
Darts also offers extensive anomaly detection capabilities.
For instance, it is trivial to apply PyOD models on time series to obtain anomaly scores,
or to wrap any of Darts forecasting or filtering models to obtain fully
fledged anomaly detection models.
Documentation
High Level Introductions
Articles on Selected Topics
- Training Models on Multiple Time Series
- Using Past and Future Covariates
- Temporal Convolutional Networks and Forecasting
- Probabilistic Forecasting
- Transfer Learning for Time Series Forecasting
- Hierarchical Forecast Reconciliation
Quick Install
We recommend to first setup a clean Python environment for your project with Python 3.11+ using your favorite tool
(conda,
venv, virtualenv with
or without virtualenvwrapper).
Once your environment is set up you can install darts using pip:
pip install dartsFor more details you can refer to our
installation instructions.
Example Usage
Forecasting
Create a TimeSeries object from a Pandas DataFrame, and split it in train/validation series:
import pandas as pd
from darts import TimeSeries
# Read a pandas DataFrame
df = pd.read_csv("AirPassengers.csv", delimiter=",")
# Create a TimeSeries, specifying the time and value columns
series = TimeSeries.from_dataframe(df, "Month", "#Passengers")
# Set aside the last 36 months as a validation series
train, val = series[:-36], series[-36:]Fit an exponential smoothing model, and make a (probabilistic) prediction over the validation series' duration:
from darts.models import ExponentialSmoothing
model = ExponentialSmoothing()
model.fit(train)
prediction = model.predict(len(val), num_samples=1000)Plot the median, 5th and 95th percentiles:
import matplotlib.pyplot as plt
series.plot()
prediction.plot(label="forecast", low_quantile=0.05, high_quantile=0.95)
plt.legend()Anomaly Detection
Load a multivariate series, trim it, keep 2 components, split train and validation sets:
from darts.datasets import ETTh2Dataset
series = ETTh2Dataset().load()[:10000][["MUFL", "LULL"]]
train, val = series.split_before(0.6)Build a k-means anomaly scorer, train it on the train set
and use it on the validation set to get anomaly scores:
from darts.ad import KMeansScorer
scorer = KMeansScorer(k=2, window=5)
scorer.fit(train)
anom_score = scorer.score(val)Build a binary anomaly detector and train it over train scores,
then use it over validation scores to get binary anomaly classification:
from darts.ad import QuantileDetector
detector = QuantileDetector(high_quantile=0.99)
detector.fit(scorer.score(train))
binary_anom = detector.detect(anom_score)Plot (shifting and scaling some of the series
to make everything appear on the same figure):
import matplotlib.pyplot as plt
series.plot()
(anom_score / 2. - 100).plot(label="computed anomaly score", c="orangered", lw=3)
(binary_anom * 45 - 150).plot(label="detected binary anomaly", lw=4)Features
Forecasting Models: A large collection of forecasting models for regression as well as classification tasks; from statistical models (such as
ARIMA) to deep learning models (such as N-BEATS). See the forecasting models below.Anomaly Detection The
darts.admodule contains a collection of anomaly scorers,
detectors and aggregators, which can all be combined to detect anomalies in time series.
It is easy to wrap any of Darts forecasting or filtering models to build
a fully fledged anomaly detection model that compares predictions with actuals.
ThePyODScorermakes it trivial to use PyOD detectors on time series.Multivariate Support:
TimeSeriescan be multivariate - i.e., contain multiple time-varying
dimensions/columns instead of a single scalar value. Many models can consume and produce multivariate series.Multiple Series Training (Global Models): All machine learning based models (incl. all neural networks)
support being trained on multiple (potentially multivariate) series. This can scale to large datasets too.Probabilistic Support:
TimeSeriesobjects can (optionally) represent stochastic
time series; this can for instance be used to get confidence intervals, and many models support different
flavours of probabilistic forecasting (such as estimating parametric distributions or quantiles).
Some anomaly detection scorers are also able to exploit these predictive distributions.Conformal Prediction Support: Our conformal prediction models allow to generate probabilistic forecasts with
calibrated quantile intervals for any pre-trained global forecasting model.Past and Future Covariates Support: Many models in Darts support past-observed and/or future-known
covariate (external data) time series as inputs for producing forecasts.Static Covariates Support: In addition to time-dependent data,
TimeSeriescan also contain
static data for each dimension, which can be exploited by some models.Hierarchical Reconciliation: Darts offers transformers to perform reconciliation.
These can make the forecasts add up in a way that respects the underlying hierarchy.Regression Models: It is possible to plug-in any scikit-learn compatible model
to obtain forecasts as functions of lagged values of the target series and covariates.Training with Sample Weights: All global models support being trained with sample weights. They can be
applied to each observation, forecasted time step and target column.Forecast Start Shifting: All global models support training and prediction on a shifted output window.
This is useful for example for Day-Ahead Market forecasts, or when the covariates (or target series) are reported
with a delay.Explainability: Darts has the ability to explain some forecasting models using SHAP values.
Data Processing: Tools to easily apply (and revert) common transformations on
time series data (scaling, filling missing values, differencing, boxcox, ...)Metrics: A variety of metrics for evaluating time series' goodness of fit;
from R2-scores to Mean Absolute Scaled Error.Backtesting: Utilities for simulating historical forecasts, using moving time windows.
PyTorch Lightning Support: All deep learning models are implemented using PyTorch Lightning,
supporting among other things custom callbacks, GPUs/TPUs training and custom trainers.MLflow Integration: Integration with MLflow for automated tracking, comparing, and persisting
of Darts forecasting model experiments. See the MLflow quickstart notebook for examples.Filtering Models: Darts offers three filtering models:
KalmanFilter,GaussianProcessFilter,
andMovingAverageFilter, which allow to filter time series, and in some cases obtain probabilistic
inferences of the underlying states/values.Datasets The
darts.datasetssubmodule contains some popular time series datasets for rapid
and reproducible experimentation.Compatibility with Multiple Backends:
TimeSeriesobjects can be created from and exported to various backends such as pandas, polars, numpy, pyarrow, xarray, and more, facilitating seamless integration with different data processing libraries.
Forecasting Models
Here's a breakdown of the forecasting models currently implemented in Darts. Our suite includes both regression and classification models, each tailored for specific forecasting tasks. We are committed to expanding our offerings with new models and features to enhance your forecasting capabilities.
Regression Models: Our regression models are designed to predict continuous numerical values, making them ideal for forecasting future trends and patterns in time series data. Utilize these models to gain insights into potential future outcomes based on historical data.
| Model | Sources | Target Series Support:Univariate/Multivariate | Covariates Support:Past-observed/Future-known/Static | Probabilistic Forecasting:Sampled/Distribution Parameters | Training & Forecasting on Multiple Series |
|---|---|---|---|---|---|
| Baseline Models(LocalForecastingModel) | |||||
| NaiveMean | โ โ | ๐ด ๐ด ๐ด | ๐ด ๐ด | ๐ด | |
| NaiveSeasonal | โ โ | ๐ด ๐ด ๐ด | ๐ด ๐ด | ๐ด | |
| NaiveDrift | โ โ | ๐ด ๐ด ๐ด | ๐ด ๐ด | ๐ด | |
| NaiveMovingAverage | โ โ | ๐ด ๐ด ๐ด | ๐ด ๐ด | ๐ด | |
| Statistical / Classic Models(LocalForecastingModel) | |||||
| ARIMA | โ ๐ด | ๐ด โ ๐ด | โ ๐ด | ๐ด | |
| VARIMA | ๐ด โ | ๐ด โ ๐ด | โ ๐ด | ๐ด | |
| ExponentialSmoothing | โ ๐ด | ๐ด ๐ด ๐ด | โ ๐ด | ๐ด | |
| Theta and FourTheta | Theta paper & 4 Theta source | โ ๐ด | ๐ด ๐ด ๐ด | ๐ด ๐ด | ๐ด |
| Prophet | Prophet repo | โ ๐ด | ๐ด โ ๐ด | โ ๐ด | ๐ด |
| FFT (Fast Fourier Transform) | โ ๐ด | ๐ด ๐ด ๐ด | ๐ด ๐ด | ๐ด | |
| KalmanForecaster using the Kalman filter and N4SID for system identification | N4SID paper | โ โ | ๐ด โ ๐ด | โ ๐ด | ๐ด |
| TBATS | TBATS paper | โ ๐ด | ๐ด โ ๐ด | โ โ | ๐ด |
| Croston method | โ ๐ด | ๐ด โ ๐ด | โ โ | ๐ด | |
| StatsForecastModel wrapper around any StatsForecast model | Nixtla's statsforecast | โ ๐ด | ๐ด โ ๐ด | โ โ | ๐ด |
| AutoARIMA | Nixtla's statsforecast | โ ๐ด | ๐ด โ ๐ด | โ โ | ๐ด |
| AutoETS | Nixtla's statsforecast | โ ๐ด | ๐ด โ ๐ด | โ โ | ๐ด |
| AutoCES | Nixtla's statsforecast | โ ๐ด | ๐ด โ ๐ด | โ โ | ๐ด |
| AutoMFLES | Nixtla's statsforecast | โ ๐ด | ๐ด โ ๐ด | โ โ | ๐ด |
| AutoTBATS | Nixtla's statsforecast | โ ๐ด | ๐ด โ ๐ด | โ โ | ๐ด |
| AutoTheta | Nixtla's statsforecast | โ ๐ด | ๐ด โ ๐ด | โ โ | ๐ด |
| MultivariateModel | โ โ | ๐ด โ ๐ด | โ โ | ๐ด | |
| Global Baseline Models(GlobalForecastingModel) | |||||
| GlobalNaiveAggregate | โ โ | ๐ด ๐ด ๐ด | ๐ด ๐ด | โ | |
| GlobalNaiveDrift | โ โ | ๐ด ๐ด ๐ด | ๐ด ๐ด | โ | |
| GlobalNaiveSeasonal | โ โ | ๐ด ๐ด ๐ด | ๐ด ๐ด | โ | |
| Regression Models(GlobalForecastingModel) | |||||
| SKLearnModel: wrapper around any scikit-learn-like regression model | โ โ | โ โ โ | ๐ด ๐ด | โ | |
| LinearRegressionModel | โ โ | โ โ โ | โ โ | โ | |
| RandomForestModel | โ โ | โ โ โ | ๐ด ๐ด | โ | |
| CatBoostModel | โ โ | โ โ โ | โ โ | โ | |
| LightGBMModel | โ โ | โ โ โ | โ โ | โ | |
| XGBModel | โ โ | โ โ โ | โ โ | โ | |
| PyTorch (Lightning)-based Models(GlobalForecastingModel) | |||||
| RNNModel (incl. LSTM and GRU); equivalent to DeepAR in its probabilistic version | DeepAR paper | โ โ | ๐ด โ ๐ด | โ โ | โ |
| BlockRNNModel (incl. LSTM and GRU) | โ โ | โ โ โ | โ โ | โ | |
| NBEATSModel | N-BEATS paper | โ โ | โ ๐ด ๐ด | โ โ | โ |
| NHiTSModel | N-HiTS paper | โ โ | โ ๐ด ๐ด | โ โ | โ |
| TCNModel | TCN paper, DeepTCN paper, blog post | โ โ | โ ๐ด ๐ด | โ โ | โ |
| TransformerModel | โ โ | โ ๐ด ๐ด | โ โ | โ | |
| TFTModel (Temporal Fusion Transformer) | TFT paper, PyTorch Forecasting | โ โ | โ โ โ | โ โ | โ |
| DLinearModel | DLinear paper | โ โ | โ โ โ | โ โ | โ |
| NLinearModel | NLinear paper | โ โ | โ โ โ | โ โ | โ |
| TiDEModel | TiDE paper | โ โ | โ โ โ | โ โ | โ |
| TSMixerModel | TSMixer paper, PyTorch Implementation | โ โ | โ โ โ | โ โ | โ |
| NeuralForecastModel: wrapper around any NeuralForecast base model | NeuralForecast Documentation | โ โ | โ โ โ | โ โ | โ |
| Foundation Models(GlobalForecastingModel): No training required | |||||
| Chronos2Model | Chronos-2 report, Amazon blog post | โ โ | โ โ ๐ด | โ โ | โ |
| TimesFM2p5Model | TimesFM 1.0 paper, Google blog post | โ โ | ๐ด ๐ด ๐ด | โ โ | โ |
| TimesFM3Model | TimesFM 1.0 paper, TimesFM 3.0 model card | โ โ | โ โ ๐ด | โ โ | โ |
| TiRexModel | TiRex paper, TiRex GitHub | โ โ | ๐ด ๐ด ๐ด | โ โ | โ |
| TiRex2Model | TiRex-2 paper, TiRex-2 GitHub | โ โ | โ โ ๐ด | โ โ | โ |
| PatchTSTFMModel | PatchTST-FM paper, PatchTST-FM GitHub | โ โ | ๐ด ๐ด ๐ด | โ โ | โ |
| T0Model | T0 report, tfc-t0 GitHub | โ โ | โ โ ๐ด | โ โ | โ |
| Ensemble Models(GlobalForecastingModel): Model support is dependent on ensembled forecasting models and the ensemble model itself | |||||
| NaiveEnsembleModel | โ โ | โ โ โ | โ โ | โ | |
| RegressionEnsembleModel | โ โ | โ โ โ | โ โ | โ | |
| Conformal Models(GlobalForecastingModel): Model support is dependent on the forecasting model used | |||||
| ConformalNaiveModel | Conformalized Prediction | โ โ | โ โ โ | โ โ | โ |
| ConformalQRModel | Conformalized Quantile Regression | โ โ | โ โ โ | โ โ | โ |
Classification Models: Classification models in Darts are designed to predict categorical class labels, enabling effective time series labeling and future class prediction. These models are perfect for scenarios where identifying distinct categories or states over time is crucial.
| Model | Sources | Target Series Support:Univariate/Multivariate | Covariates Support:Past-observed/Future-known/Static | Probabilistic Forecasting:Sampled/Distribution Parameters | Training & Forecasting on Multiple Series |
|---|---|---|---|---|---|
| Regression Models(GlobalForecastingModel) | |||||
| SKLearnClassifierModel: wrapper around any scikit-learn-like classification model | โ โ | โ โ โ | โ โ | โ | |
| CatBoostClassifierModel | โ โ | โ โ โ | โ โ | โ | |
| LightGBMClassifierModel | โ โ | โ โ โ | โ โ | โ | |
| XGBClassifierModel | โ โ | โ โ โ | โ โ | โ |
Community & Contact
Anyone is welcome to join our Gitter room to ask questions, make proposals,
discuss use-cases, and more. If you spot a bug or have suggestions, GitHub issues are also welcome.
If what you want to tell us is not suitable for Gitter or Github,
feel free to send us an email at [email protected] for
darts related matters or [email protected] for any other
inquiries.
Contribute
The development is ongoing, and we welcome suggestions, pull requests and issues on GitHub.
All contributors will be acknowledged on the
change log page.
Before working on a contribution (a new feature or a fix),
check our contribution guidelines.
Citation
If you are using Darts in your scientific work, we would appreciate citations to the following JMLR paper.
Darts: User-Friendly Modern Machine Learning for Time Series
Bibtex entry (or copy directly via GitHub's repository citation):
@article{Herzen_Darts_User-Friendly_Modern_2022,
author = {Herzen, Julien and Lรคssig, Francesco and Piazzetta, Samuele Giuliano and Neuer, Thomas and Tafti, Lรฉo and Raille, Guillaume and Van Pottelbergh, Tomas and Pasieka, Marek and Skrodzki, Andrzej and Huguenin, Nicolas and Dumonal, Maxime and Koลcisz, Jan and Bader, Dennis and Gusset, Frรฉdรฉrick and Benheddi, Mounir and Williamson, Camila and Kosinski, Michal and Petrik, Matej and Grosch, Gaรซl},
journal = {Journal of Machine Learning Research},
number = {124},
pages = {1--6},
title = {{Darts: User-Friendly Modern Machine Learning for Time Series}},
url = {https://jmlr.org/papers/v23/21-1177.html},
volume = {23},
year = {2022}
}Userguide Forecasting Overview (`docs/userguide/forecasting_overview.md`)
Overview of Forecasting Models
Forecasting models are models that can produce predictions about future values of some time series, given the history of this series.
The forecasting models in Darts are listed on the README. They have different capabilities and features. For example, some models work on multidimensional series, return probabilistic forecasts, or accept other kinds of external covariates data in input.
Below, we give an overview of what these features mean.
Generalities
All forecasting models work in the same way: first they are built (taking some hyper-parameters in argument), then they are fit on one or several series
by calling the fit() function, and finally they are used to obtain one or several forecasts by calling the predict() function.
Example:
from darts.models import NaiveSeasonal
naive_model = NaiveSeasonal(K=1) # init
naive_model.fit(train) # fit
naive_forecast = naive_model.predict(n=36) # predictThe argument n of predict() indicates the number of time stamps to predict.
When fit() is provided with only one training TimeSeries, this series is stored, and predict() will return forecasts for this series.
On the other hand, some models support calling fit() on multiple time series (a Sequence[TimeSeries]). In such cases, one or several series must
be provided to predict(), and the model will produce forecasts for this/these time series.
Example:
from darts.models import NBEATSModel
model = NBEATSModel(input_chunk_length=24, # init
output_chunk_length=12)
model.fit([series1, series2]) # fit on two series
forecast = model.predict(series=[series3, series4], n=36) # predict potentially different seriesFurthermore, we define the following types of time series consumed by the models:
- Target series: the series that we are interested in forecasting.
- Covariate series: some other series that we are not interested in forecasting, but that can provide valuable inputs to the forecasting model.
Saving and Loading Models
If you wish to save a particular model and use it elsewhere or at a later point in time, darts can achieve that. It leverages pickle and in the case of Torch models relies on saving PyTorch Lightning trainer checkpoints.
All forecasting models support saving the model on the filesystem, by calling the save() function, which saves that particular ForecastingModel object instance. When the model is to be used again, the method load() can be used. Please note that the methods save_model() and load_model() are deprecated.
Security: Loading uses safe-by-default deserialization to mitigate malicious pickle files (CWE-502). Use trusted=True only for files you fully trust, or register custom classes/functions via add_safe_globals / with safe_globals(...). See the safe model loading guide for details.
Example:
from darts.models import SKLearnModel
model = SKLearnModel(lags=4)
model.save("my_model.pkl")
model_loaded = SKLearnModel.load("my_model.pkl")The parameter path specifies a path or file handle under which to save the model at its current state. If no path is specified, the model is automatically
saved under "{ModelClass}_{YYYY-mm-dd_HH:MM:SS}.pkl". E.g., "SKLearnModel_2020-01-01_12:00:00.pkl".
Optionally there is also pickle specific keyword arguments protocol, fix_imports and buffer_callback.
More info: pickle.dump()
With torch models, the model parameters and the training state are saved. We use the trainer to save the model checkpoint.
Example:
from darts.models import NBEATSModel
model = NBEATSModel(input_chunk_length=24,
output_chunk_length=12)
model.save("my_model.pt")
model_loaded = NBEATSModel.load("my_model.pt")Private methods for torch models used under the hood:
- save_checkpoint: In addition, we need to use PTL save_checkpoint() to properly save the trainer and model. It is used to be able to save a snapshot of the model mid-training, and then be able to retrieve the model later.
- load_from_checkpoint: returns a model checkpointed during training (by default the one with lowest validation loss).
Support for multivariate series
Some models support multivariate time series. This means that the target (and potential covariates) series provided to the model
during fit and predict stage can have multiple dimensions. The model will then in turn produce multivariate forecasts.
Here is an example, using a KalmanForecaster to forecast a single multivariate series made of 2 components:
# use darts plotting style
from darts import set_option
set_option("plotting.use_darts_style", True)
import darts.utils.timeseries_generation as tg
from darts.models import KalmanForecaster
import matplotlib.pyplot as plt
series1 = tg.sine_timeseries(value_frequency=0.05, length=100) + 0.1 * tg.gaussian_timeseries(length=100)
series2 = tg.sine_timeseries(value_frequency=0.02, length=100) + 0.2 * tg.gaussian_timeseries(length=100)
multivariate_series = series1.stack(series2)
model = KalmanForecaster(dim_x=4)
model.fit(multivariate_series)
pred = model.predict(n=50, num_samples=100)
plt.figure(figsize=(8,6))
multivariate_series.plot(lw=3)
pred.plot(lw=3, label='forecast')These models are shown with a "โ
" under the Multivariate column on the model list.
Handling multiple series
Some models support being fit on multiple time series. To do this, it is enough to simply provide a Python Sequence of TimeSeries (for instance a list of TimeSeries) to fit(). When a model is fit this way, the predict() function will expect the argument series to be set, containing
one or several TimeSeries (i.e., a single or a Sequence of TimeSeries) that need to be forecasted.
The advantage of training on multiple series is that a single model can be exposed to more patterns occurring across all series in the training dataset. That can often be beneficial, especially for larger models with more capacity.
In turn, the advantage of having predict() providing forecasts for potentially several series at once is that the computation can often be batched and vectorized across the multiple series, which is computationally faster than calling predict() multiple times on isolated series.
These models are shown with a "โ
" under the Multiple-series training column on the model list.
You can also find out programmatically, whether a model supports multiple series.
from darts.models import SKLearnModel
from darts.models.forecasting.forecasting_model import GlobalForecastingModel
# when True, multiple time series are supported
supports_multi_ts = issubclass(SKLearnModel, GlobalForecastingModel)This article on multi-series forecasting provides more explanations about training models on multiple series.
Support for Covariates
Some models support covariate series. Covariate series are time series that the models can take as inputs, but will not forecast.
We distinguish between past covariates and future covariates:
- Past covariates are covariate time series whose values are not known into the future at prediction time. Those can for instance represent signals that have to be measured and are not known upfront. Models do not use the future values of
past_covariateswhen making forecasts. - Future covariates are covariate time series whose values are known into the future at prediction time (up until the forecast horizon). These can represent signals such as calendar information, holidays, weather forecasts, etc. Models that accept
future_covariateswill consume the future values (up to the forecast horizon) when making forecasts.
Past and future covariates can be used by providing respectively past_covariates and future_covariates arguments to fit() and predict().
When a model is trained on multiple target series, one covariate has to be provided per target series. The covariate series themselves can be multivariate
and contain multiple "covariate dimensions"; see the TimeSeries guide for how to build multivariate series.
It is not necessary to worry about the covariates series having the exact right time spans (e.g., so that the last timestamp of future covariates matches the forecast horizon). Darts takes care of slicing the covariates behind the scenes, based on the time axes of the target(s) and that of the covariates.
Models supporting past (resp. future) covariates are indicated with a "โ
" under the Past-observed covariates support (resp. Future-known covariates support) columns on the model list,
To know more about covariates, please refer to the covariates section of the user guide.
In addition, you can have a look at this article on covariates for some examples of how to use past and future covariates.
Probabilistic forecasts
Most models in Darts can produce probabilistic forecasts. For these models, the TimeSeries returned by predict() will be probabilistic, and contain a certain number of Monte Carlo samples representing the forecasted distribution over time and components. The number of samples can be directly determined by the argument num_samples of the predict() function (leaving num_samples=1 will return a deterministic TimeSeries).
Models supporting probabilistic forecasts are indicated with a "โ
" in the Probabilistic column on the model list.
The actual probabilistic distribution of the forecasts depends on the model.
Some models such as ARIMA, Exponential Smoothing, (T)BATS or KalmanForecaster make normality assumptions and the resulting distribution is a Gaussian with time-dependent parameters. For example:
from darts.datasets import AirPassengersDataset
from darts import TimeSeries
from darts.models import ExponentialSmoothing
series = AirPassengersDataset().load()
train, val = series[:-36], series[-36:]
model = ExponentialSmoothing()
model.fit(train)
pred = model.predict(n=36, num_samples=500)
series.plot()
pred.plot(label='forecast')Probabilistic neural networks
All neural networks (torch-based models) in Darts have a rich support to estimate different kinds of probability distributions.
When creating the model, it is possible to provide one of the likelihood models available in darts.utils.likelihood_models.torch, which determine the distribution that will be estimated by the model.
In such cases, the model will output the parameters of the distribution, and it will be trained by minimising the negative log-likelihood of the training samples.
Most of the likelihood models also support prior values for the distribution's parameters, in which case the training loss is regularized by a Kullback-Leibler divergence term pushing the resulting distribution in the direction of the distribution specified by the prior parameters.
The strength of this regularization term can also be specified when creating the likelihood model object.
For example, the code below trains a TCNModel to fit a Laplace distribution. So the neural network outputs 2 parameters (location and scale) of the Laplace distribution. We also specify a prior value of 0.1 on the scale parameter.
from darts.datasets import AirPassengersDataset
from darts import TimeSeries
from darts.models import TCNModel
from darts.dataprocessing.transformers import Scaler
from darts.utils.likelihood_models.torch import LaplaceLikelihood
series = AirPassengersDataset().load()
train, val = series[:-36], series[-36:]
scaler = Scaler()
train = scaler.fit_transform(train)
val = scaler.transform(val)
series = scaler.transform(series)
model = TCNModel(input_chunk_length=30,
output_chunk_length=12,
likelihood=LaplaceLikelihood(prior_b=0.1))
model.fit(train, epochs=400)
pred = model.predict(n=36, num_samples=500)
series.plot()
pred.plot(label='forecast')It is also possible to perform quantile regression (using arbitrary quantiles) with neural networks, by using darts.utils.likelihood_models.torch.QuantileRegression, in which case the network will be trained with the pinball loss. This produces an empirical non-parametric distribution, and it can often be a good option in practice, when one is not sure of the "real" distribution, or when fitting parametric likelihoods give poor results.
For example, the code snippet below is almost exactly the same as the preceding snippet; the only difference is that it now uses a QuantileRegression likelihood, which means that the neural network will be trained with a pinball loss, and its number of outputs will be dynamically configured to match the number of quantiles.
from darts.datasets import AirPassengersDataset
from darts import TimeSeries
from darts.models import TCNModel
from darts.dataprocessing.transformers import Scaler
from darts.utils.likelihood_models.torch import QuantileRegression
series = AirPassengersDataset().load()
train, val = series[:-36], series[-36:]
scaler = Scaler()
train = scaler.fit_transform(train)
val = scaler.transform(val)
series = scaler.transform(series)
model = TCNModel(input_chunk_length=30,
output_chunk_length=12,
likelihood=QuantileRegression(quantiles=[0.01, 0.05, 0.2, 0.5, 0.8, 0.95, 0.99]))
model.fit(train, epochs=400)
pred = model.predict(n=36, num_samples=500)
series.plot()
pred.plot(label='forecast')Capturing model uncertainty using Monte Carlo Dropout
In Darts, dropout can also be used as an additional way to capture model uncertainty, following the approach described in [1]. This is sometimes referred to as epistemic uncertainty, and can be seen as a way to marginalize over a family of models represented by all the different dropout activation functions.
This feature is readily available for all deep learning models integrating some dropout (except RNN models - we refer to the dropout API reference documentations for a mention of models supporting this). It only requires to specify mc_dropout=True at prediction time. For example, the code below trains a TCN model (using the default MSE loss) with a dropout rate of 10%, and then produces a probabilistic forecasts using Monte Carlo Dropout:
from darts.datasets import AirPassengersDataset
from darts import TimeSeries
from darts.models import TCNModel
from darts.dataprocessing.transformers import Scaler
from darts.utils.likelihood_models.torch import QuantileRegression
series = AirPassengersDataset().load()
train, val = series[:-36], series[-36:]
scaler = Scaler()
train = scaler.fit_transform(train)
val = scaler.transform(val)
series = scaler.transform(series)
model = TCNModel(input_chunk_length=30,
output_chunk_length=12,
dropout=0.1)
model.fit(train, epochs=400)
pred = model.predict(n=36, mc_dropout=True, num_samples=500)
series.plot()
pred.plot(label='forecast')Monte Carlo Dropout can be combined with other likelihood estimation in Darts, which can be interpreted as a way to capture both epistemic and aleatoric uncertainty.
In that case, train the model with the desired likelihood (e.g. QuantileRegression(...)) so the model learns to predict the parameters of an aleatoric distribution, and at prediction time set:
mc_dropout=True,num_samples >> 1(e.g.500),predict_likelihood_parameters=False.
Each Monte Carlo dropout pass yields one set of distribution parameters; Darts then draws one sample from that distribution per pass, so the resulting num_samples predictions reflect both epistemic uncertainty (different dropout masks) and aleatoric uncertainty (sampling from the predicted likelihood). You can then compute marginal quantiles, mean, std, etc. from those samples.
Note:
mc_dropout=Trueandpredict_likelihood_parameters=Trueshould not be combined.predict_likelihood_parameters=Trueis designed for use withnum_samples=1to get the model's best estimate of the distribution parameters.
Probabilistic regression models
Some regression models can also be configured to produce probabilistic forecasts too. At the time of writing, LinearRegressionModel, LightGBMModel and XGBModel support a likelihood argument. When set to "poisson" the model will fit a Poisson distribution, and when set to "quantile" the model will use the pinball loss to perform quantile regression (the quantiles themselves can be specified using the quantiles argument).
Example:
from darts.datasets import AirPassengersDataset
from darts import TimeSeries
from darts.models import LinearRegressionModel
series = AirPassengersDataset().load()
train, val = series[:-36], series[-36:]
model = LinearRegressionModel(lags=30,
likelihood="quantile",
quantiles=[0.05, 0.1, 0.25, 0.5, 0.75, 0.9, 0.95])
model.fit(train)
pred = model.predict(n=36, num_samples=500)
series.plot()
pred.plot(label='forecast')Multi-step probabilistic forecasts: Scenarios vs Marginal Distributions
When calling predict(n, num_samples=k) with k > 1, the returned TimeSeries has shape (time, components, samples). A common question is whether the individual sample trajectories along the samples dimension can be interpreted as realistic, coherent temporal scenarios (accounting for autocorrelation over time and cross-dimensional correlations), or whether they only represent independent pointwise marginal distributions at each time step.
The distinction primarily depends on whether the forecast is generated autoregressively or via direct multi-step chunks:
1. Purely Autoregressive Rollouts (Scenario Generation)
For models operating purely autoregressively (e.g. global forecasting models with output_chunk_length = 1):
- Forecasts are generated step-by-step into the future.
- At time step $t$, a sample is drawn from the predicted distribution for each sample index $i \in {1, \dots, k}$.
- For each scenario $i$, the sampled value at $t$ is fed back into the model as "historic" input to forecast the distribution at $t+1$.
- Temporal correlation: Because each trajectory's future is conditioned on its own simulated past values, autocorrelation is preserved across time. The individual sample paths $i$ function as realistic Monte Carlo simulated scenarios.
- Cross-component correlation: The random noise/innovations injected at each step are sampled independently across components. However, cross-component interactions in the conditional mean are captured and propagated over time through the lagged features.
2. Direct Multi-Step / Chunk Forecasting (Marginal Sampling)
For direct multi-step models (e.g. global forecasting models with output_chunk_length > 1):
- The model outputs the parameters of the distributions (or quantiles) for the entire output chunk simultaneously.
- When generating
num_samplesforecasts, samples are drawn independently across all time steps and components within the chunk. - Temporal correlation: Within the output chunk, there is no modeled joint covariance or temporal correlation in the sampled noise. Consequently, individual sample trajectories do not represent realistic dynamic scenarios; rather, the collection of samples at each time step $t$ provides a Monte Carlo estimate of the marginal distribution at that specific point in time.
- Beyond the output chunk length (
n > output_chunk_length): Models apply chunk-level autoregression, feeding the previously sampled chunk back into the model for the next chunk.
Tip: If your use case requires realistic simulated paths over time (e.g., for downstream simulation, risk analysis, or optimization under uncertainty), prefer autoregressive models with
output_chunk_length=1(ormulti_models=Falsein regression models). If you primarily need accurate pointwise prediction intervals or quantiles, direct multi-step models are often faster and avoid compounding autoregressive errors.
[1] Yarin Gal, Zoubin Ghahramani, "Dropout as a Bayesian Approximation: Representing Model Uncertainty in Deep Learning"
Source Examples (`docs/source/examples.rst`)
======== Examples
Here you will find some example notebooks to get more familiar with the Darts' API.
All the notebooks are also available in ipynb formatdirectly on github <https://github.com/unit8co/darts/tree/master/examples>__.
Multiple Time Series, Pre-trained Models and Covariates
Example notebook on training with multiple time series, pre-trained models and using covariates:
.. toctree::
:maxdepth: 1
examples/01-multi-time-series-and-covariates.ipynb
Data (pre) Processing
Data processing example notebook:
.. toctree::
:maxdepth: 1
examples/02-data-processing.ipynb
Static Covariates
Static covariates example notebook:
.. toctree::
:maxdepth: 1
examples/15-static-covariates.ipynb
Transfer Learning Tutorial
A self-contained notebook showcasing examples of training some bigger models
on large datasets of time series, and using such models to forecast new time
series that have not been seen during training:
.. toctree::
:maxdepth: 1
examples/14-transfer-learning.ipynb
Hierarchical Reconciliation
A self-contained notebook showcasing how to reconcile forecasts
for hierarchical time series, using the Australian domestic tourism dataset.
.. toctree::
:maxdepth: 1
examples/16-hierarchical-reconciliation.ipynb
Hyperparameter Optimization
An example showcasing how to find good forecasting models
with Darts using the Optuna library for hyperparameter optimization.
.. toctree::
:maxdepth: 1
examples/17-hyperparameter-optimization.ipynb
Regression Models
Regression models example notebook:
.. toctree::
:maxdepth: 1
examples/20-SKLearnModel-examples.ipynb
Classification Models
Classification models example notebook:
.. toctree::
:maxdepth: 1
examples/24-SKLearnClassifierModel-examples.ipynb
Conformal Prediction
Conformal prediction example notebook:
.. toctree::
:maxdepth: 1
examples/23-Conformal-Prediction-examples.ipynb
Fast Fourier Transform
FFT model example notebook:
.. toctree::
:maxdepth: 1
examples/03-FFT-examples.ipynb
Recurrent Neural Networks
RNN model example notebook:
.. toctree::
:maxdepth: 1
examples/04-RNN-examples.ipynb
Temporal Convolutional Networks
TCN model example notebook:
.. toctree::
:maxdepth: 1
examples/05-TCN-examples.ipynb
Transformer Model
Transformer model example notebook:
.. toctree::
:maxdepth: 1
examples/06-Transformer-examples.ipynb
N-BEATS Model
N-BEATS model example notebook:
.. toctree::
:maxdepth: 1
examples/07-NBEATS-examples.ipynb
DeepAR Model
DeepAR model example notebook:
.. toctree::
:maxdepth: 1
examples/08-DeepAR-examples.ipynb
DeepTCN Model
DeepTCN model example notebook:
.. toctree::
:maxdepth: 1
examples/09-DeepTCN-examples.ipynb
Temporal Fusion Transformer (TFT) Model
TFT model example notebook:
.. toctree::
:maxdepth: 1
examples/13-TFT-examples.ipynb
TimeSeries Dense Encoder (TiDE) Model
TiDE model example notebook:
.. toctree::
:maxdepth: 1
examples/18-TiDE-examples.ipynb
TimeSeries Mixer (TSMixer) Model
TSMixer model example notebook:
.. toctree::
:maxdepth: 1
examples/21-TSMixer-examples.ipynb
NeuralForecast Model Adapter
NeuralForecastModel example notebook:
.. toctree::
:maxdepth: 1
examples/26-NeuralForecast-examples.ipynb
Chronos-2 Model
Foundation model example notebook:
.. toctree::
:maxdepth: 1
examples/25-FoundationModel-examples.ipynb
Torch and Foundation Model Fine-Tuning
Torch and Foundation model fine-tuning example notebook:
.. toctree::
:maxdepth: 1
examples/27-Torch-and-Foundation-Model-Fine-Tuning-examples.ipynb
Ensemble Models
Ensemble models example notebook:
.. toctree::
:maxdepth: 1
examples/19-EnsembleModel-examples.ipynb
Explainability
Explainability example notebook showcasing the use of Darts' explainability module for both PyTorch and SKLearn models:
.. toctree::
:maxdepth: 1
examples/28-Explainability-examples.ipynb
MLflow Integration
MLflow integration example notebook:
.. toctree::
:maxdepth: 1
examples/29-MLflow-examples.ipynb
Kalman Filter Model
Kalman filter model example notebook:
.. toctree::
:maxdepth: 1
examples/10-Kalman-filter-examples.ipynb
Gaussian Process Filter Model
Gaussian process filter model example notebook:
.. toctree::
:maxdepth: 1
examples/11-GP-filter-examples.ipynb
Anomaly Detection
Anomaly detection example notebook:
.. toctree::
:maxdepth: 1
examples/22-anomaly-detection-examples.ipynb
Dynamic Time Warping (DTW)
Dynamic time warping example notebook:
.. toctree::
:maxdepth: 1
examples/12-Dynamic-Time-Warping-example.ipynb
Source Index (`docs/source/index.rst`)
:notoc:
Time Series Made Easy in Python
.. image:: static/darts-logo-light.png
:alt: darts
:align: center
:width: 100%
:class: only-light
.. image:: static/darts-logo-dark.png
:alt: darts
:align: center
:width: 100%
:class: only-dark
.. raw:: html
|
Darts is a Python library for user-friendly forecasting and anomaly detection on time series.
It contains a variety of models, from classics such as ARIMA to deep neural networks.
All models can be used in the same way, using fit() and predict() functions, similar to scikit-learn.
The library also makes it easy to backtest models, combine the predictions of several models, and take external data into account.
Darts supports both univariate and multivariate time series and models, and offers extensive support for probabilistic forecasting.
.. grid:: 1 2 3 3
:gutter: 4
:padding: 2 2 0 0
:class-container: sd-text-center
.. grid-item-card:: Quickstart
:class-card: intro-card
:shadow: md
.. image:: static/grid_card_icons/quickstart.png
:class: sd-card-img-top only-light
:alt: Quickstart
.. image:: static/grid_card_icons_dark/quickstart.png
:class: sd-card-img-top only-dark
:alt: Quickstart
Introduction to Darts' main concepts and workflow
+++
.. button-ref:: quickstart/00-quickstart
:ref-type: doc
:click-parent:
:color: secondary
:expand:
To the quickstart guide
.. grid-item-card:: Models
:class-card: intro-card
:shadow: md
.. image:: static/grid_card_icons/models.png
:class: sd-card-img-top only-light
:alt: Models
.. image:: static/grid_card_icons_dark/models.png
:class: sd-card-img-top only-dark
:alt: Models
Available models and supported features
+++
.. button-ref:: forecasting-models
:ref-type: ref
:click-parent:
:color: secondary
:expand:
To the models table
.. grid-item-card:: API Reference
:class-card: intro-card
:shadow: md
.. image:: static/grid_card_icons/api.png
:class: sd-card-img-top only-light
:alt: API Reference
.. image:: static/grid_card_icons_dark/api.png
:class: sd-card-img-top only-dark
:alt: API Reference
Detailed API documentation with methods and parameters
+++
.. button-ref:: generated_api/darts
:ref-type: doc
:click-parent:
:color: secondary
:expand:
To the API reference
.. grid-item-card:: Examples
:class-card: intro-card
:shadow: md
.. image:: static/grid_card_icons/examples.png
:class: sd-card-img-top only-light
:alt: Examples
.. image:: static/grid_card_icons_dark/examples.png
:class: sd-card-img-top only-dark
:alt: Examples
Jupyter notebooks with basic and advanced tutorials
+++
.. button-ref:: examples
:ref-type: doc
:click-parent:
:color: secondary
:expand:
To the examples
.. grid-item-card:: User Guide
:class-card: intro-card
:shadow: md
.. image:: static/grid_card_icons/user-guide.png
:class: sd-card-img-top only-light
:alt: User Guide
.. image:: static/grid_card_icons_dark/user-guide.png
:class: sd-card-img-top only-dark
:alt: User Guide
In-depth information on key concepts and features
+++
.. button-ref:: userguide
:ref-type: doc
:click-parent:
:color: secondary
:expand:
To the user guide
.. grid-item-card:: How to Contribute
:class-card: intro-card
:shadow: md
.. image:: static/grid_card_icons/contribute.png
:class: sd-card-img-top only-light
:alt: Contribute
.. image:: static/grid_card_icons_dark/contribute.png
:class: sd-card-img-top only-dark
:alt: Contribute
Guidelines for contributing to the Darts library
+++
.. button-link:: https://github.com/unit8co/darts/blob/master/CONTRIBUTING.md
:click-parent:
:color: secondary
:expand:
To the contributing guideHigh Level Introductions
Introductory Blog Post <https://medium.com/unit8-machine-learning-publication/darts-time-series-made-easy-in-python-5ac2947a8878>_Introduction video (25 minutes) <https://youtu.be/g6OXDnXEtFA>_
Articles on Selected Topics
Training Models on Multiple Time Series <https://medium.com/unit8-machine-learning-publication/training-forecasting-models-on-multiple-time-series-with-darts-dc4be70b1844>_Using Past and Future Covariates <https://medium.com/unit8-machine-learning-publication/time-series-forecasting-using-past-and-future-external-data-with-darts-1f0539585993>_Temporal Convolutional Networks and Forecasting <https://medium.com/unit8-machine-learning-publication/temporal-convolutional-networks-and-forecasting-5ce1b6e97ce4>_Probabilistic Forecasting <https://medium.com/unit8-machine-learning-publication/probabilistic-forecasting-in-darts-e88fbe83344e>_Transfer Learning for Time Series Forecasting <https://medium.com/unit8-machine-learning-publication/transfer-learning-for-time-series-forecasting-87f39e375278>_Hierarchical Forecast Reconciliation <https://medium.com/unit8-machine-learning-publication/hierarchical-forecast-reconciliation-with-darts-8b4b058bb543>_
Quick Install
We recommend to first setup a clean Python environment for your project with Python 3.11+ using your favorite tool
(conda <https://docs.conda.io/projects/conda/en/latest/user-guide/tasks/manage-environments.html>,venv <https://docs.python.org/3/library/venv.html>, virtualenv <https://virtualenv.pypa.io/en/latest/>_ with
or without virtualenvwrapper <https://virtualenvwrapper.readthedocs.io/en/latest/>_).
Once your environment is set up you can install darts using pip:
.. code-block:: bash
pip install dartsFor more details you can refer to our installation instructions <https://github.com/unit8co/darts/blob/master/INSTALL.md>_.
Example Usage
Forecasting
Create a ``TimeSeries`` object from a Pandas DataFrame, and split it in train/validation series:
.. code-block:: python
import pandas as pd
from darts import TimeSeries
# Read a pandas DataFrame
df = pd.read_csv("AirPassengers.csv", delimiter=",")
# Create a TimeSeries, specifying the time and value columns
series = TimeSeries.from_dataframe(df, "Month", "#Passengers")
# Set aside the last 36 months as a validation series
train, val = series[:-36], series[-36:]
Fit an exponential smoothing model, and make a (probabilistic) prediction over the validation series' duration:
.. code-block:: python
from darts.models import ExponentialSmoothing
model = ExponentialSmoothing()
model.fit(train)
prediction = model.predict(len(val), num_samples=1000)
Plot the median, 5th and 95th percentiles:
.. code-block:: python
import matplotlib.pyplot as plt
series.plot()
prediction.plot(label="forecast", low_quantile=0.05, high_quantile=0.95)
plt.legend()
.. image:: https://github.com/unit8co/darts/raw/master/static/images/example.png
:alt: darts forecast example
:align: center
Anomaly DetectionLoad a multivariate series, trim it, keep 2 components, split train and validation sets:
.. code-block:: python
from darts.datasets import ETTh2Dataset
series = ETTh2Dataset().load()[:10000][["MUFL", "LULL"]]
train, val = series.split_before(0.6)Build a k-means anomaly scorer, train it on the train set
and use it on the validation set to get anomaly scores:
.. code-block:: python
from darts.ad import KMeansScorer
scorer = KMeansScorer(k=2, window=5)
scorer.fit(train)
anom_score = scorer.score(val)Build a binary anomaly detector and train it over train scores,
then use it over validation scores to get binary anomaly classification:
.. code-block:: python
from darts.ad import QuantileDetector
detector = QuantileDetector(high_quantile=0.99)
detector.fit(scorer.score(train))
binary_anom = detector.detect(anom_score)Plot (shifting and scaling some of the series to make everything appear on the same figure):
.. code-block:: python
import matplotlib.pyplot as plt
series.plot()
(anom_score / 2. - 100).plot(label="computed anomaly score", c="orangered", lw=3)
(binary_anom * 45 - 150).plot(label="detected binary anomaly", lw=4).. image:: https://github.com/unit8co/darts/raw/master/static/images/example_ad.png
:alt: darts anomaly detection example
:align: center
Features
Forecasting Models: A large collection of forecasting models for regression as well as classification tasks; from statistical models (such as ARIMA) to deep learning models (such as N-BEATS). See the forecasting models table below.
Anomaly Detection: The
darts.admodule contains a collection of anomaly scorers, detectors and aggregators, which can all be combined to detect anomalies in time series. It is easy to wrap any of Darts forecasting or filtering models to build a fully fledged anomaly detection model that compares predictions with actuals. ThePyODScorermakes it trivial to use PyOD detectors on time series.Multivariate Support:
TimeSeriescan be multivariate - i.e., contain multiple time-varying dimensions/columns instead of a single scalar value. Many models can consume and produce multivariate series.Multiple Series Training (Global Models): All machine learning based models (incl. all neural networks) support being trained on multiple (potentially multivariate) series. This can scale to large datasets too.
Probabilistic Support:
TimeSeriesobjects can (optionally) represent stochastic time series; this can for instance be used to get confidence intervals, and many models support different flavours of probabilistic forecasting (such as estimating parametric distributions or quantiles). Some anomaly detection scorers are also able to exploit these predictive distributions.Conformal Prediction Support: Our conformal prediction models allow to generate probabilistic forecasts with calibrated quantile intervals for any pre-trained global forecasting model.
Past and Future Covariates Support: Many models in Darts support past-observed and/or future-known covariate (external data) time series as inputs for producing forecasts.
Static Covariates Support: In addition to time-dependent data,
TimeSeriescan also contain static data for each dimension, which can be exploited by some models.Hierarchical Reconciliation: Darts offers transformers to perform reconciliation. These can make the forecasts add up in a way that respects the underlying hierarchy.
Regression Models: It is possible to plug-in any scikit-learn compatible model to obtain forecasts as functions of lagged values of the target series and covariates.
Training with Sample Weights: All global models support being trained with sample weights. They can be applied to each observation, forecasted time step and target column.
Forecast Start Shifting: All global models support training and prediction on a shifted output window. This is useful for example for Day-Ahead Market forecasts, or when the covariates (or target series) are reported with a delay.
Explainability: Darts has the ability to explain some forecasting models using SHAP values.
Data Processing: Tools to easily apply (and revert) common transformations on time series data (scaling, filling missing values, differencing, boxcox, ...)
Metrics: A variety of metrics for evaluating time series' goodness of fit; from R2-scores to Mean Absolute Scaled Error.
Backtesting: Utilities for simulating historical forecasts, using moving time windows.
PyTorch Lightning Support: All deep learning models are implemented using PyTorch Lightning, supporting among other things custom callbacks, GPUs/TPUs training and custom trainers.
MLflow Integration: Integration with MLflow for automated tracking, comparing, and persisting of Darts forecasting model experiments. See the
MLflow quickstart notebook <https://unit8co.github.io/darts/examples/29-MLflow-examples.html>_ for examples.Filtering Models: Darts offers three filtering models:
KalmanFilter,GaussianProcessFilter, andMovingAverageFilter, which allow to filter time series, and in some cases obtain probabilistic inferences of the underlying states/values.Datasets: The
darts.datasetssubmodule contains some popular time series datasets for rapid and reproducible experimentation.Compatibility with Multiple Backends:
TimeSeriesobjects can be created from and exported to various backends such as pandas, polars, numpy, pyarrow, xarray, and more, facilitating seamless integration with different data processing libraries.
.. _forecasting-models:
Forecasting Models
Here's a breakdown of the forecasting models currently implemented in Darts. Our suite includes both regression and classification models, each tailored for specific forecasting tasks. We are committed to expanding our offerings with new models and features to enhance your forecasting capabilities.
Regression Models
Our regression models are designed to predict continuous numerical values, making them ideal for forecasting future trends and patterns in time series data. Utilize these models to gain insights into potential future outcomes based on historical data.
.. list-table::
:header-rows: 1
:widths: 30 10 10 10 10 20
* - Model
- Target Series Support: Univariate / Multivariate
- Covariates Support: Past-observed / Future-known / Static
- Probabilistic Forecasting: Sampled / Distribution Parameters
- Training & Forecasting on Multiple Series
- Sources
* - **Baseline Models** (`LocalForecastingModel <https://unit8co.github.io/darts/userguide/covariates.html#local-forecasting-models-lfms>`_)
-
-
-
-
-
* - `NaiveMean <https://unit8co.github.io/darts/generated_api/darts.models.forecasting.baselines.html#darts.models.forecasting.baselines.NaiveMean>`_
- โ
โ
- ๐ด ๐ด ๐ด
- ๐ด ๐ด
- ๐ด
-
* - `NaiveSeasonal <https://unit8co.github.io/darts/generated_api/darts.models.forecasting.baselines.html#darts.models.forecasting.baselines.NaiveSeasonal>`_
- โ
โ
- ๐ด ๐ด ๐ด
- ๐ด ๐ด
- ๐ด
-
* - `NaiveDrift <https://unit8co.github.io/darts/generated_api/darts.models.forecasting.baselines.html#darts.models.forecasting.baselines.NaiveDrift>`_
- โ
โ
- ๐ด ๐ด ๐ด
- ๐ด ๐ด
- ๐ด
-
* - `NaiveMovingAverage <https://unit8co.github.io/darts/generated_api/darts.models.forecasting.baselines.html#darts.models.forecasting.baselines.NaiveMovingAverage>`_
- โ
โ
- ๐ด ๐ด ๐ด
- ๐ด ๐ด
- ๐ด
-
* - **Statistical / Classic Models** (`LocalForecastingModel <https://unit8co.github.io/darts/userguide/covariates.html#local-forecasting-models-lfms>`_)
-
-
-
-
-
* - `ARIMA <https://unit8co.github.io/darts/generated_api/darts.models.forecasting.arima.html#darts.models.forecasting.arima.ARIMA>`_
- โ
๐ด
- ๐ด โ
๐ด
- โ
๐ด
- ๐ด
-
* - `VARIMA <https://unit8co.github.io/darts/generated_api/darts.models.forecasting.varima.html#darts.models.forecasting.varima.VARIMA>`_
- ๐ด โ
- ๐ด โ
๐ด
- โ
๐ด
- ๐ด
-
* - `ExponentialSmoothing <https://unit8co.github.io/darts/generated_api/darts.models.forecasting.exponential_smoothing.html#darts.models.forecasting.exponential_smoothing.ExponentialSmoothing>`_
- โ
๐ด
- ๐ด ๐ด ๐ด
- โ
๐ด
- ๐ด
-
* - `Theta <https://unit8co.github.io/darts/generated_api/darts.models.forecasting.theta.html#darts.models.forecasting.theta.Theta>`_ and `FourTheta <https://unit8co.github.io/darts/generated_api/darts.models.forecasting.theta.html#darts.models.forecasting.theta.FourTheta>`_
- โ
๐ด
- ๐ด ๐ด ๐ด
- ๐ด ๐ด
- ๐ด
- `Theta paper <https://robjhyndman.com/papers/Theta.pdf>`_ & `4 Theta source <https://github.com/Mcompetitions/M4-methods/blob/master/4Theta%20method.R>`_
* - `Prophet <https://unit8co.github.io/darts/generated_api/darts.models.forecasting.prophet_model.html#darts.models.forecasting.prophet_model.Prophet>`_
- โ
๐ด
- ๐ด โ
๐ด
- โ
๐ด
- ๐ด
- `Prophet repo <https://github.com/facebook/prophet>`_
* - `FFT <https://unit8co.github.io/darts/generated_api/darts.models.forecasting.fft.html#darts.models.forecasting.fft.FFT>`_ (Fast Fourier Transform)
- โ
๐ด
- ๐ด ๐ด ๐ด
- ๐ด ๐ด
- ๐ด
-
* - `KalmanForecaster <https://unit8co.github.io/darts/generated_api/darts.models.forecasting.kalman_forecaster.html#darts.models.forecasting.kalman_forecaster.KalmanForecaster>`_ using the Kalman filter and N4SID for system identification
- โ
โ
- ๐ด โ
๐ด
- โ
๐ด
- ๐ด
- `N4SID paper <https://people.duke.edu/~hpgavin/SystemID/References/VanOverschee-Automatica-1994.pdf>`_
* - `TBATS <https://unit8co.github.io/darts/generated_api/darts.models.forecasting.sf_tbats.html#darts.models.forecasting.sf_tbats.TBATS>`_
- โ
๐ด
- ๐ด โ
๐ด
- โ
โ
- ๐ด
- `TBATS paper <https://robjhyndman.com/papers/ComplexSeasonality.pdf>`_
* - `Croston <https://unit8co.github.io/darts/generated_api/darts.models.forecasting.sf_croston.html#darts.models.forecasting.sf_croston.Croston>`_ method
- โ
๐ด
- ๐ด โ
๐ด
- โ
โ
- ๐ด
-
* - `StatsForecastModel <https://unit8co.github.io/darts/generated_api/darts.models.forecasting.sf_model.html#darts.models.forecasting.sf_model.StatsForecastModel>`_ wrapper around any `StatsForecast <https://nixtlaverse.nixtla.io/statsforecast/index.html#models>`_ model
- โ
๐ด
- ๐ด โ
๐ด
- โ
โ
- ๐ด
- `Nixtla's statsforecast <https://github.com/Nixtla/statsforecast>`_
* - `AutoARIMA <https://unit8co.github.io/darts/generated_api/darts.models.forecasting.sf_auto_arima.html#darts.models.forecasting.sf_auto_arima.AutoARIMA>`_
- โ
๐ด
- ๐ด โ
๐ด
- โ
โ
- ๐ด
- `Nixtla's statsforecast <https://github.com/Nixtla/statsforecast>`_
* - `AutoETS <https://unit8co.github.io/darts/generated_api/darts.models.forecasting.sf_auto_ets.html#darts.models.forecasting.sf_auto_ets.AutoETS>`_
- โ
๐ด
- ๐ด โ
๐ด
- โ
โ
- ๐ด
- `Nixtla's statsforecast <https://github.com/Nixtla/statsforecast>`_
* - `AutoCES <https://unit8co.github.io/darts/generated_api/darts.models.forecasting.sf_auto_ces.html#darts.models.forecasting.sf_auto_ces.AutoCES>`_
- โ
๐ด
- ๐ด โ
๐ด
- โ
โ
- ๐ด
- `Nixtla's statsforecast <https://github.com/Nixtla/statsforecast>`_
* - `AutoMFLES <https://unit8co.github.io/darts/generated_api/darts.models.forecasting.sf_auto_mfles.html#darts.models.forecasting.sf_auto_mfles.AutoMFLES>`_
- โ
๐ด
- ๐ด โ
๐ด
- โ
โ
- ๐ด
- `Nixtla's statsforecast <https://github.com/Nixtla/statsforecast>`_
* - `AutoTBATS <https://unit8co.github.io/darts/generated_api/darts.models.forecasting.sf_auto_tbats.html#darts.models.forecasting.sf_auto_tbats.AutoTBATS>`_
- โ
๐ด
- ๐ด โ
๐ด
- โ
โ
- ๐ด
- `Nixtla's statsforecast <https://github.com/Nixtla/statsforecast>`_
* - `AutoTheta <https://unit8co.github.io/darts/generated_api/darts.models.forecasting.sf_auto_theta.html#darts.models.forecasting.sf_auto_theta.AutoTheta>`_
- โ
๐ด
- ๐ด โ
๐ด
- โ
โ
- ๐ด
- `Nixtla's statsforecast <https://github.com/Nixtla/statsforecast>`_
* - `MultivariateModel <https://unit8co.github.io/darts/generated_api/darts.models.forecasting.multivariate_model.html#darts.models.forecasting.multivariate_model.MultivariateModel>`_
- โ
โ
- ๐ด โ
๐ด
- โ
โ
- ๐ด
-
* - **Global Baseline Models** (`GlobalForecastingModel <https://unit8co.github.io/darts/userguide/covariates.html#global-forecasting-models-gfms>`_)
-
-
-
-
-
* - `GlobalNaiveAggregate <https://unit8co.github.io/darts/generated_api/darts.models.forecasting.global_baseline_models.html#darts.models.forecasting.global_baseline_models.GlobalNaiveAggregate>`_
- โ
โ
- ๐ด ๐ด ๐ด
- ๐ด ๐ด
- โ
-
* - `GlobalNaiveDrift <https://unit8co.github.io/darts/generated_api/darts.models.forecasting.global_baseline_models.html#darts.models.forecasting.global_baseline_models.GlobalNaiveDrift>`_
- โ
โ
- ๐ด ๐ด ๐ด
- ๐ด ๐ด
- โ
-
* - `GlobalNaiveSeasonal <https://unit8co.github.io/darts/generated_api/darts.models.forecasting.global_baseline_models.html#darts.models.forecasting.global_baseline_models.GlobalNaiveSeasonal>`_
- โ
โ
- ๐ด ๐ด ๐ด
- ๐ด ๐ด
- โ
-
* - **Regression Models** (`GlobalForecastingModel <https://unit8co.github.io/darts/userguide/covariates.html#global-forecasting-models-gfms>`_)
-
-
-
-
-
* - `SKLearnModel <https://unit8co.github.io/darts/generated_api/darts.models.forecasting.sklearn_model.html#darts.models.forecasting.sklearn_model.SKLearnModel>`_: wrapper around any scikit-learn-like regression model
- โ
โ
- โ
โ
โ
- ๐ด ๐ด
- โ
-
* - `LinearRegressionModel <https://unit8co.github.io/darts/generated_api/darts.models.forecasting.linear_regression_model.html#darts.models.forecasting.linear_regression_model.LinearRegressionModel>`_
- โ
โ
- โ
โ
โ
- โ
โ
- โ
-
* - `RandomForestModel <https://unit8co.github.io/darts/generated_api/darts.models.forecasting.random_forest.html#darts.models.forecasting.random_forest.RandomForestModel>`_
- โ
โ
- โ
โ
โ
- ๐ด ๐ด
- โ
-
* - `CatBoostModel <https://unit8co.github.io/darts/generated_api/darts.models.forecasting.catboost_model.html#darts.models.forecasting.catboost_model.CatBoostModel>`_
- โ
โ
- โ
โ
โ
- โ
โ
- โ
-
* - `LightGBMModel <https://unit8co.github.io/darts/generated_api/darts.models.forecasting.lgbm.html#darts.models.forecasting.lgbm.LightGBMModel>`_
- โ
โ
- โ
โ
โ
- โ
โ
- โ
-
* - `XGBModel <https://unit8co.github.io/darts/generated_api/darts.models.forecasting.xgboost.html#darts.models.forecasting.xgboost.XGBModel>`_
- โ
โ
- โ
โ
โ
- โ
โ
- โ
-
* - **PyTorch (Lightning)-based Models** (`GlobalForecastingModel <https://unit8co.github.io/darts/userguide/covariates.html#global-forecasting-models-gfms>`_)
-
-
-
-
-
* - `RNNModel <https://unit8co.github.io/darts/generated_api/darts.models.forecasting.rnn_model.html#darts.models.forecasting.rnn_model.RNNModel>`_ (incl. LSTM and GRU); equivalent to DeepAR in its probabilistic version
- โ
โ
- ๐ด โ
๐ด
- โ
โ
- โ
- `DeepAR paper <https://arxiv.org/abs/1704.04110>`_
* - `BlockRNNModel <https://unit8co.github.io/darts/generated_api/darts.models.forecasting.block_rnn_model.html#darts.models.forecasting.block_rnn_model.BlockRNNModel>`_ (incl. LSTM and GRU)
- โ
โ
- โ
โ
โ
- โ
โ
- โ
-
* - `NBEATSModel <https://unit8co.github.io/darts/generated_api/darts.models.forecasting.nbeats.html#darts.models.forecasting.nbeats.NBEATSModel>`_
- โ
โ
- โ
๐ด ๐ด
- โ
โ
- โ
- `N-BEATS paper <https://arxiv.org/abs/1905.10437>`_
* - `NHiTSModel <https://unit8co.github.io/darts/generated_api/darts.models.forecasting.nhits.html#darts.models.forecasting.nhits.NHiTSModel>`_
- โ
โ
- โ
๐ด ๐ด
- โ
โ
- โ
- `N-HiTS paper <https://arxiv.org/abs/2201.12886>`_
* - `TCNModel <https://unit8co.github.io/darts/generated_api/darts.models.forecasting.tcn_model.html#darts.models.forecasting.tcn_model.TCNModel>`_
- โ
โ
- โ
๐ด ๐ด
- โ
โ
- โ
- `TCN paper <https://arxiv.org/abs/1803.01271>`_, `DeepTCN paper <https://arxiv.org/abs/1906.04397>`_, `blog post <https://medium.com/unit8-machine-learning-publication/temporal-convolutional-networks-and-forecasting-5ce1b6e97ce4>`_
* - `TransformerModel <https://unit8co.github.io/darts/generated_api/darts.models.forecasting.transformer_model.html#darts.models.forecasting.transformer_model.TransformerModel>`_
- โ
โ
- โ
๐ด ๐ด
- โ
โ
- โ
-
* - `TFTModel <https://unit8co.github.io/darts/generated_api/darts.models.forecasting.tft_model.html#darts.models.forecasting.tft_model.TFTModel>`_ (Temporal Fusion Transformer)
- โ
โ
- โ
โ
โ
- โ
โ
- โ
- `TFT paper <https://arxiv.org/pdf/1912.09363.pdf>`_, `PyTorch Forecasting <https://pytorch-forecasting.readthedocs.io/en/latest/models.html>`_
* - `DLinearModel <https://unit8co.github.io/darts/generated_api/darts.models.forecasting.dlinear.html#darts.models.forecasting.dlinear.DLinearModel>`_
- โ
โ
- โ
โ
โ
- โ
โ
- โ
- `DLinear paper <https://arxiv.org/pdf/2205.13504.pdf>`_
* - `NLinearModel <https://unit8co.github.io/darts/generated_api/darts.models.forecasting.nlinear.html#darts.models.forecasting.nlinear.NLinearModel>`_
- โ
โ
- โ
โ
โ
- โ
โ
- โ
- `NLinear paper <https://arxiv.org/pdf/2205.13504.pdf>`_
* - `TiDEModel <https://unit8co.github.io/darts/generated_api/darts.models.forecasting.tide_model.html#darts.models.forecasting.tide_model.TiDEModel>`_
- โ
โ
- โ
โ
โ
- โ
โ
- โ
- `TiDE paper <https://arxiv.org/pdf/2304.08424.pdf>`_
* - `TSMixerModel <https://unit8co.github.io/darts/generated_api/darts.models.forecasting.tsmixer_model.html#darts.models.forecasting.tsmixer_model.TSMixerModel>`_
- โ
โ
- โ
โ
โ
- โ
โ
- โ
- `TSMixer paper <https://arxiv.org/pdf/2303.06053.pdf>`_, `PyTorch Implementation <https://github.com/ditschuk/pytorch-tsmixer>`_
* - `NeuralForecastModel <https://unit8co.github.io/darts/generated_api/darts.models.forecasting.nf_model.html#darts.models.forecasting.nf_model.NeuralForecastModel>`_: wrapper around any `NeuralForecast <https://nixtlaverse.nixtla.io/neuralforecast/docs/capabilities/overview.html>`_ base model
- โ
โ
- โ
โ
โ
- โ
โ
- โ
- `NeuralForecast Documentation <https://nixtlaverse.nixtla.io/neuralforecast/docs/>`_
* - **Foundation Models** (`GlobalForecastingModel <https://unit8co.github.io/darts/userguide/covariates.html#global-forecasting-models-gfms>`_): No training required
-
-
-
-
-
* - `Chronos2Model <https://unit8co.github.io/darts/generated_api/darts.models.forecasting.chronos2_model.html#darts.models.forecasting.chronos2_model.Chronos2Model>`_
- โ
โ
- โ
โ
๐ด
- โ
โ
- โ
- `Chronos-2 paper <https://arxiv.org/abs/2510.15821>`_, `Amazon blog post <https://www.amazon.science/blog/introducing-chronos-2-from-univariate-to-universal-forecasting>`_
* - `TimesFM2p5Model <https://unit8co.github.io/darts/generated_api/darts.models.forecasting.timesfm2p5_model.html#darts.models.forecasting.timesfm2p5_model.TimesFM2p5Model>`_
- โ
โ
- ๐ด ๐ด ๐ด
- โ
โ
- โ
- `TimesFM 1.0 paper <https://arxiv.org/abs/2310.10688>`_, `Google blog post <https://research.google/blog/a-decoder-only-foundation-model-for-time-series-forecasting>`_
* - `TimesFM3Model <https://unit8co.github.io/darts/generated_api/darts.models.forecasting.timesfm3_model.html#darts.models.forecasting.timesfm3_model.TimesFM3Model>`_
- โ
โ
- โ
โ
๐ด
- โ
โ
- โ
- `TimesFM 1.0 paper <https://arxiv.org/abs/2310.10688>`_, `TimesFM 3.0 model card <https://huggingface.co/google/timesfm-3.0-pytorch>`_
* - `TiRexModel <https://unit8co.github.io/darts/generated_api/darts.models.forecasting.tirex_model.html#darts.models.forecasting.tirex_model.TiRexModel>`_
- โ
โ
- ๐ด ๐ด ๐ด
- โ
โ
- โ
- `TiRex paper <https://arxiv.org/abs/2505.23719>`_, `TiRex GitHub <https://github.com/NX-AI/tirex>`_
* - `TiRex2Model <https://unit8co.github.io/darts/generated_api/darts.models.forecasting.tirex2_model.html#darts.models.forecasting.tirex2_model.TiRex2Model>`_
- โ
โ
- โ
โ
๐ด
- โ
โ
- โ
- `TiRex-2 paper <https://arxiv.org/abs/2607.01204>`_, `TiRex-2 GitHub <https://github.com/NX-AI/tirex-2>`_
* - `PatchTSTFMModel <https://unit8co.github.io/darts/generated_api/darts.models.forecasting.patchtst_fm_model.html#darts.models.forecasting.patchtst_fm_model.PatchTSTFMModel>`_
- โ
โ
- ๐ด ๐ด ๐ด
- โ
โ
- โ
- `PatchTST-FM paper <https://arxiv.org/abs/2602.06909>`_, `PatchTST-FM Github <https://github.com/ibm-granite/granite-tsfm>`_
* - `T0Model <https://unit8co.github.io/darts/generated_api/darts.models.forecasting.t0_model.html#darts.models.forecasting.t0_model.T0Model>`_
- โ
โ
- โ
โ
๐ด
- โ
โ
- โ
- `T0 report <https://www.theforecastingcompany.com/papers/files/t0-technical-report.pdf>`_, `tfc-t0 GitHub <https://github.com/theforecastingcompany/tfc-t0>`_
* - **Ensemble Models** (`GlobalForecastingModel <https://unit8co.github.io/darts/userguide/covariates.html#global-forecasting-models-gfms>`_): Model support is dependent on ensembled forecasting models and the ensemble model itself
-
-
-
-
-
* - `NaiveEnsembleModel <https://unit8co.github.io/darts/generated_api/darts.models.forecasting.naive_ensemble_model.html#darts.models.forecasting.naive_ensemble_model.NaiveEnsembleModel>`_
- โ
โ
- โ
โ
โ
- โ
โ
- โ
-
* - `RegressionEnsembleModel <https://unit8co.github.io/darts/generated_api/darts.models.forecasting.regression_ensemble_model.html#darts.models.forecasting.regression_ensemble_model.RegressionEnsembleModel>`_
- โ
โ
- โ
โ
โ
- โ
โ
- โ
-
* - **Conformal Models** (`GlobalForecastingModel <https://unit8co.github.io/darts/userguide/covariates.html#global-forecasting-models-gfms>`_): Model support is dependent on the forecasting model used
-
-
-
-
-
* - `ConformalNaiveModel <https://unit8co.github.io/darts/generated_api/darts.models.forecasting.conformal_models.html#darts.models.forecasting.conformal_models.ConformalNaiveModel>`_
- โ
โ
- โ
โ
โ
- โ
โ
- โ
- `Conformalized Prediction <https://arxiv.org/pdf/1905.03222>`_
* - `ConformalQRModel <https://unit8co.github.io/darts/generated_api/darts.models.forecasting.conformal_models.html#darts.models.forecasting.conformal_models.ConformalQRModel>`_
- โ
โ
- โ
โ
โ
- โ
โ
- โ
- `Conformalized Quantile Regression <https://arxiv.org/pdf/1905.03222>`_
Classification ModelsClassification models in Darts are designed to predict categorical class labels, enabling effective time series labeling and future class prediction. These models are perfect for scenarios where identifying distinct categories or states over time is crucial.
.. list-table::
:header-rows: 1
:widths: 30 10 10 10 10 20
- Model
- Target Series Support: Univariate / Multivariate
- Covariates Support: Past-observed / Future-known / Static
- Probabilistic Forecasting: Sampled / Distribution Parameters
- Training & Forecasting on Multiple Series
- Sources
- Regression Models (
GlobalForecastingModel <https://unit8co.github.io/darts/userguide/covariates.html#global-forecasting-models-gfms>_)
- Regression Models (
SKLearnClassifierModel <https://unit8co.github.io/darts/generated_api/darts.models.forecasting.sklearn_model.html#darts.models.forecasting.sklearn_model.SKLearnClassifierModel>_: wrapper around any scikit-learn-like classification model- โ โ
- โ โ โ
- โ โ
- โ
CatBoostClassifierModel <https://unit8co.github.io/darts/generated_api/darts.models.forecasting.catboost_model.html#darts.models.forecasting.catboost_model.CatBoostClassifierModel>_- โ โ
- โ โ โ
- โ โ
- โ
LightGBMClassifierModel <https://unit8co.github.io/darts/generated_api/darts.models.forecasting.lgbm.html#darts.models.forecasting.lgbm.LightGBMClassifierModel>_- โ โ
- โ โ โ
- โ โ
- โ
XGBClassifierModel <https://unit8co.github.io/darts/generated_api/darts.models.forecasting.xgboost.html#darts.models.forecasting.xgboost.XGBClassifierModel>_- โ โ
- โ โ โ
- โ โ
- โ
Community & Contact
Anyone is welcome to join our Gitter room <https://gitter.im/u8darts/darts>_ to ask questions, make proposals, discuss use-cases, and more. If you spot a bug or have suggestions, GitHub issues are also welcome.
If what you want to tell us is not suitable for Gitter or Github, feel free to send us an email at [email protected] for darts related matters or [email protected] for any other inquiries.
Contribute
The development is ongoing, and we welcome suggestions, pull requests and issues on GitHub. All contributors will be acknowledged on the change log page <https://github.com/unit8co/darts/blob/master/CHANGELOG.md>_.
Before working on a contribution (a new feature or a fix), check our contribution guidelines <https://github.com/unit8co/darts/blob/master/CONTRIBUTING.md>_.
Citation
If you are using Darts in your scientific work, we would appreciate citations to the following JMLR paper.
Darts: User-Friendly Modern Machine Learning for Time Series <https://www.jmlr.org/papers/v23/21-1177.html>_
Bibtex entry:
.. code-block:: bibtex
@article{JMLR:v23:21-1177,
author = {Julien Herzen and Francesco Lรคssig and Samuele Giuliano Piazzetta and Thomas Neuer and Lรฉo Tafti and Guillaume Raille and Tomas Van Pottelbergh and Marek Pasieka and Andrzej Skrodzki and Nicolas Huguenin and Maxime Dumonal and Jan Koลcisz and Dennis Bader and Frรฉdรฉrick Gusset and Mounir Benheddi and Camila Williamson and Michal Kosinski and Matej Petrik and Gaรซl Grosch},
title = {Darts: User-Friendly Modern Machine Learning for Time Series},
journal = {Journal of Machine Learning Research},
year = {2022},
volume = {23},
number = {124},
pages = {1-6},
url = {http://jmlr.org/papers/v23/21-1177.html}
}.. toctree::
:hidden:
Quickstart<quickstart/00-quickstart.ipynb>
.. toctree::
:hidden:
User Guide
.. toctree::
:hidden:
API Reference<generated_api/darts>
.. toctree::
:hidden:
Examples
.. toctree::
:hidden:
Release Notes<release_notes/RELEASE_NOTES>
Source Userguide (`docs/source/userguide.rst`)
========== User Guide
You will find here some more detailed information about Darts.
Note: The user guide is not yet complete and still under construction.
.. toctree::
:maxdepth: 2
userguide/timeseries.md
userguide/forecasting_overview.md
userguide/torch_forecasting_models.md
userguide/gpu_and_tpu_usage.md
userguide/covariates.md
userguide/safe_model_loading.md
userguide/hyperparameter_optimization.md
userguide/faq.md
Userguide Covariates (`docs/userguide/covariates.md`)
Covariates
Sections about past and future covariates were written for darts version 0.15.0 and later.
Sections about static covariates were written for darts version 0.20.0 and later.
Summary - TL;DR
In Darts, covariates refer to external data that can be used as inputs to models to help improve forecasts.
In the context of forecasting models, the target is the series to be forecasted/predicted, and the
covariates themselves are not predicted. We distinguish three kinds of covariates:
- past covariates are (by definition) covariates known only into the past (e.g. measurements)
- future covariates are (by definition) covariates known into the future (e.g., weather forecasts)
- static covariates are (by definition) covariates constant over time (e.g., product IDs). Check out our static covariates example notebook here for more information.
Models in Darts accept past_covariates and/or future_covariates in their fit() and predict() methods, depending on their capabilities (some models accept no covariates at all). Both target and covariates must be a TimeSeries object. The models will raise an error if covariates were used that are not supported.
# create one of Darts' forecasting models
model = SomeForecastingModel(...)
# fitting model with past and future covariates
model.fit(target=target,
past_covariates=past_covariates_train,
future_covariates=future_covariates_train)
# predict the next n=12 steps
model.predict(n=12,
series=target, # only required for Global Forecasting Models
past_covariates=past_covariates_pred,
future_covariates=future_covariates_pred)Different to past and future covariates, static covariates must be embedded in the target series. Because of that, working with static covariates follows a different methodology. You can check out how to use static covariates in this example.
If you have several covariate variables that you want to use as past (or future) covariates, you have to stack() all of them into a single past_covariates (or future_covariates) object.
# stack two TimeSeries with stack()
past_covariates = past_covariates.stack(other_past_covariates)
# or with concatenate()
from darts import concatenate
past_covariates = concatenate([past_covariates, other_past_covariates], axis=1)Darts' forecasting models expect one past and/or future covariate series per target series. If you use multiple target series with one of Darts' Global Forecasting Models, you must supply the same number of dedicated covariates to fit().
# fit using multiple (two) target series
model.fit(target=[target, target_2],
past_covariates=[past_covariates, past_covariates_2],
# optional future_covariates,
)
# you must give the specific target and covariate series that you want to predict
model.predict(n=12,
series=target_2,
past_covariates=past_covariates_2,
# optional future_covariates,
)If you train a model using past_covariates, you'll have to provide these past_covariates also at prediction time to predict(). This applies to future_covariates too, with a nuance that future_covariates have to extend far enough into the future at prediction time (all the way to the forecast horizon n). This can be seen in the graph below. past_covariates needs to include at least the same time steps as target, and future_covariates must include at least the same time span plus additional n forecast horizon time steps.
You can use the same *_covariates for both training and prediction, given that they contain the required time spans.
Figure 1: Top level summary of how forecasting models work with target and covariates for a prediction with forecast horizon n=2
There are some extra nuances that might be good to know. For instance, deep learning models in Darts
can (in general) forecast output_chunk_length points at a time. However it is still possible for models
trained with past covariates to make forecasts for some horizon n > output_chunk_length if the past_covariates
are known far enough into the future. In such cases, the forecasts are obtained by consuming future values
of the past covariates, and using auto-regression on the target series. If you want to know more details, read on.
Introduction - What are covariates (in Darts)?
Past, future and static covariates provide additional information/context that can be useful to improve the prediction of the target series. The target series is the variable we wish to predict the future for. We do not predict the covariates themselves, only use them for prediction of the target.
Past and future covariates hold information about the past (up to and including present time) or future. This is always relative to the prediction point (in time) after which we want to forecast the future.
In Darts, we refer to these two types as past_covariates and future_covariates.
Static covariates hold time independent (constant / static) information about the target series. We refer to them as static_covariates. They must be embedded in the target series. Working with static covariates follows a slightly different approach than with past or future covariates. Check out our notebook on static covariates to learn more.
Darts' forecasting models have different support modes for *_covariates. Some do not support covariates at all, others support only past or future covariates and some even support all three (more on that in this subsection).
Let's have a look at some examples of past, future, and static covariates:
past_covariates: typically measurements (past data) or temporal attributes- daily average measured temperatures (known only in the past)
- day of week, month, year, ...
future_covariates: typically forecasts (future known data) or temporal attributes- daily average forecasted temperatures (known in the future)
- day of week, month, year, ...
static_covariates: time independent/constant/statictargetcharacteristics- categorical:
- location of
target(country, city, .. name) targetidentifier: (product ID, store ID, ...)
- location of
- numerical:
- population of
target's country/market area (assuming it stays constant over the forecasting horizon) - average temperature of
target's region (assuming it stays constant over the forecasting horizon)
- population of
- categorical:
Temporal attributes are powerful because they are known in advance and can help models capture trends and / or seasonal patterns of the target series.
Static attributes are powerful when working with multiple targets (either multiple TimeSeries, or multivariate series containing multiple dimensions each). The time independent information can help models identify the nature/environment of the underlying series and improve forecasts across different targets.
In this guide we'll focus on past and future covariates. Here's a simple rule-of-thumb to know if your series are past or future covariates:
If the values are known in advance, they are future covariates (or can be used as past covariates). If they are not, they must be past covariates.
You might imagine cases where you want to train a model supporting only past_covariates (such as TCNModel, see Table 1). In this case, you could use for instance say, the forecasted temperature as a past covariate for the model even though you also have access to temperature forecasts in the future. Knowing such "future values of past covariates" can allow you to make forecasts further into the future (for Darts' deep learning models with forecast horizons n > output_chunk_length). Similarly most models consuming future covariates can also use "historic values of future covariates".
Side note: if you don't have future values (e.g. of measured temperatures), nothing prevents you from applying one of Darts' forecasting models to forecast future temperatures, and then use this as future_covariates. Darts is not attempting to forecast the covariates for you, as this would introduce an extra "hidden" modeling step, which we think is best left to the users.
Forecasting Model Covariate Support
Darts' forecasting models accept optional past_covariates and / or future_covariates in their fit() and predict() methods (and static_covariates embedded in the target series), depending on their capabilities. Table 1 shows the supported covariate types for each model. The models will raise an error if covariates were used that are not supported.
Local Forecasting Models (LFMs):
LFMs are models that can be trained on a single target series only. In Darts most models in this category tend to be simpler statistical models (such as ETS or ARIMA). LFMs accept only a single target (and covariate) time series and usually train on the entire series you supplied when calling fit() at once. They can also predict in one go for any number of predictions n after the end of the training series.
Global Forecasting Models (GFMs)
GFMs are models that can be trained on multiple target (and covariate) time series. Different to LFMs, the GFMs train and predict on fixed-length sub-samples (chunks) of the input data. In Darts, these are the global (naive) baseline models, regression models, PyTorch (Lightning)-based models (neural networks), foundation models, as well ensemble models (depending on their ensemble model and / or the forecasting models they ensemble).
| Model | Past Covariates | Future Covariates | Static Covariates |
|---|---|---|---|
| Local Forecasting Models (LFMs) | |||
| Naive Baselines (a) | |||
| ARIMA | โ | ||
| VARIMA | โ | ||
| ExponentialSmoothing | |||
| TBATS | โ | ||
| Theta and FourTheta | |||
| Prophet | โ | ||
| FFT (Fast Fourier Transform) | |||
| KalmanForecaster | โ | ||
| Croston method | โ | ||
| StatsForecastModel | โ | ||
| AutoARIMA | โ | ||
| AutoETS | โ | ||
| AutoCES | โ | ||
| AutoMFLES | โ | ||
| AutoTBATS | โ | ||
| AutoTheta | โ | ||
| MultivariateModel | โ | ||
| Global Forecasting Models (GFMs) | |||
| Global Naive Baselines (b) | |||
| SKLearn-Like Regression Models (c) | โ | โ | โ |
| SKLearn-Like Classification Models (d) | โ | โ | โ |
| RNNModel (e) | โ | ||
| BlockRNNModel (f) | โ | โ | โ |
| NBEATSModel | โ | ||
| NHiTSModel | โ | ||
| TCNModel | โ | ||
| TransformerModel | โ | ||
| TFTModel | โ | โ | โ |
| DLinearModel | โ | โ | โ |
| NLinearModel | โ | โ | โ |
| TiDEModel | โ | โ | โ |
| TSMixerModel | โ | โ | โ |
| NeuralForecastModel (g) | โ | โ | โ |
| Chronos2Model | โ | โ | |
| TimesFM2p5Model | |||
| TimesFM3Model | โ | โ | |
| TiRexModel | |||
| TiRex2Model | โ | โ | |
| Ensemble Models (h) | โ | โ | โ |
| Conformal Prediction Models (i) | โ | โ | โ |
Table 1: Darts' forecasting models and their covariate support
(a) Naive Baselines including NaiveDrift, NaiveMean, NaiveMovingAverage, and NaiveSeasonal.
(b) Global Naive Baselines including GlobalNaiveAggregate, GlobalNaiveDrift, and GlobalNaiveSeasonal.
(c) SKLearn-Like Regression Models including SKLearnModel, LinearRegressionModel, RandomForestModel, LightGBMModel, XGBModel, and CatBoostModel. SKLearnModel is a special kind of GFM which can use arbitrary lags on covariates (past and/or future) and past targets to predict continuous numerical values.
(d) SKLearn-Like Classification Models including SKLearnClassifierModel, LightGBMClassifierModel, XGBClassifierModel, and CatBoostClassifierModel. SKLearnClassifierModel is a special kind of GFM which can use arbitrary lags on covariates (past and/or future) and past targets to predict categorical class labels.
(e) RNNModel including LSTM and GRU; equivalent to DeepAR in its probabilistic version
(f) BlockRNNModel including LSTM and GRU
(g) NeuralForecastModel covariate support depends on the NeuralForecast base model used. See NeuralForecast model catalog for more details.
(h) Ensemble Model including RegressionEnsembleModel, and NaiveEnsembleModel. The covariate support is given by the covariate support of the ensembled forecasting models.
(i) Conformal Prediction Model including ConformalNaiveModel, and ConformalQRModel. The covariate support is given by the covariate support of the underlying forecasting model.
Quick guide on how to use past and/or future covariates with Darts' forecasting models
It is very simple to use covariates with Darts' forecasting models. There are just some requirements they have to fulfill.
Just like the target series, each of your past and / or future covariates series must be a TimeSeries object. When you train your model with fit() using past and /or future covariates, you have to supply the same types of covariates to predict(). Depending on the choice of your model and how long your forecast horizon n is, there might be different time span requirements for your covariates. You can find these requirements in the next subsection.
You can even use the same *_covariates for fitting and prediction if they contain the required time spans. This is because Darts will "intelligently" slice them for you based on the target time axis.
# create one of Darts' forecasting model
model = SomeForecastingModel(...)
# fit the model
model.fit(target,
past_covariates=past_covariate,
future_covariates=future_covariates)
# make a prediction with the same covariate types
pred = model.predict(n=1,
series=target, # this is only required for GFMs
past_covariates=past_covariates,
future_covariates=future_covariates)To use multiple past and / or future covariates with your target, you have to stack them all together into a single dedicated TimeSeries:
# stack() time series
past_covariates = past_covariates.stack(past_covariates2)
# or concatenate()
from darts import concatenate
past_covariates = concatenate([past_covariates, past_covariates2, ...], axis=1)GFMs can be trained on multiple target series. You have to supply one covariate TimeSeries per target TimeSeries you use with fit(). At prediction time you have to specify which target series you want to predict and supply the corresponding covariates:
from darts.models import NBEATSModel
# multiple time series
all_targets = [target1, target2, ...]
all_past_covariates = [past_covariates1, past_covariates2, ...]
# create a GFM model, train and predict
model = NBEATSModel(input_chunk_length=1, output_chunk_length=1)
model.fit(all_targets,
past_covariates=all_past_covariates)
pred = model.predict(n=1,
series=all_targets[0],
past_covariates=all_past_covariates[0])Covariate time span requirements for Local and Global Forecasting Models
There are differences in how Darts' "Local" and "Global" Forecasting Models perform training and prediction. Specifically, how they extract/work with the data supplied during fit() and predict().
Depending on the model you use and how long your forecast horizon n is, there might be different time span requirements for your covariates.
Local Forecasting Models (LFMs):
LFMs usually train on the entire target and future_covariates series (if supported) you supplied when calling fit() at once. They can also predict in one go for forecast horizon n after the end of the target.
Time span requirements to use the same future covariates series for both fit() and predict():
future_covariates: at least the same time span astargetplus the nextntime steps after the end oftarget
Global Forecasting Models (GFMs):
GFMs train and predict on fixed-length chunks (sub-samples) of the target and *_covariates series (if supported). Each chunk contains an input chunk - representing the sample's past - and an output chunk - the sample's future. The length of these chunks has to be specified at model creation with parameters input_chunk_length and output_chunk_length (one notable exception is RNNModel which always uses an output_chunk_length of 1).
Depending on your forecast horizon n, the model can either predict in one go, or auto-regressively, by predicting on multiple chunks in the future. That is the reason why when predicting with past_covariates you have to supply additional "future values of your past_covariates".
Time span requirements to use the same past and / or future covariates series for both fit() and predict():
- with
n <= output_chunk_length:past_covariates: at least the same time span astargetfuture_covariates: at least the same time span astargetplus the nextoutput_chunk_lengthtime steps after the end oftarget
- with
n > output_chunk_length:past_covariates: at least the same time span astargetplus the nextn - output_chunk_lengthtime steps after the end oftargetfuture_covariates: at least the same time span astargetplus the nextntime steps after the end oftarget
If you want to know more details about how covariates are used behind the scenes in Global Forecasting Models, read our guide on Torch Forecasting Models (PyTorch based GFMs). It gives a step-by-step explanation of the training and prediction process using one of our Torch Forecasting Models.
Examples
Here are a few examples showcasing how to use covariates with Darts forecasting models:
- Static covariates tutorial including TFTModel example
- Past covariates with GFMs
- Past and future covariates with TFTModel
- Past and future covariates with SKLearnModels (previously named RegressionModel)
Userguide Faq (`docs/userguide/faq.md`)
Frequently Asked Questions
Is Darts just a wrapper around other libraries?
No. When it makes sense, we reuse existing implementations (e.g. from statsforecasts), but we often write our own implementations (e.g., of neural networks). Furthermore, Darts models often have more features than their original counter-parts. For instance, unlike the original version, our implementation of N-BEATS supports multivariate time series, past covariates, and probabilistic forecasts.
Darts looks like an awesome project, can I contribute?
Absolutely! We are constantly welcoming contributions from the community. If you contribute, you will be acknowledged in the wall of fame (a.k.a the changelog)! Contributions don't have to be code only but can also be e.g., documentation. In addition, we're also happy to receive suggestions in the form of issues on Github. The best place to start for contributors is the contribution guidelines.
I have some ideas for contributing a new model into Darts, is that possible?
In general, yes, we do welcome reference implementations of new models. However, we are loosely filtering to keep models that are either classics, or convincingly shown (e.g., in a paper or some other forms of evidence) to be state-of-the-art in some respect.
How can I make Darts work on Google Colab?
Colab may have issues with recent versions of pyyaml. Installing pyyaml 5.4.1 before installing Darts solves these issues:
text!pip install pyyaml==5.4.1I'm getting some NaNs in my forecasts, what should I do?
Usually this means one of two things:
- You have some NaNs in your training
TimeSeries(targets or covariates). This is the most frequent situation, and it will lead most models to always forecast NaNs. Note that NaNs can be introduced when you convert data from apd.DataFrameinto aTimeSeriesif some dates are missing and thefreqargument is incorrectly set. - Training the model causes some numerical divergence. If you're working with neural networks, make sure your data is properly scaled, and if the issue persists, try reducing the learning rate.
- You have some NaNs in your training
My forecasting models give bad results, can you help?
Getting good forecasts is about more than just calling the
fit()/predict()functions, and always involves some data science work to understand which approaches are appropriate. We cannot give general answers, however if you have important forecasting problems or need help to industrialize your forecasts, Unit8 provides technical consulting. Feel free to contact us.Can I use probabilistic forecast samples as realistic simulated scenarios over time?
See the overview on probabilistic forecasts for more details.
Userguide Gpu And Tpu Usage (`docs/userguide/gpu_and_tpu_usage.md`)
Using Torch Models with GPUs and TPUs
This section was written for Darts 0.17.0 and later.
We assume that you already know about Torch Forecasting Models in Darts. If you're new to the topic we recommend you to read the guide on Torch Forecasting Models first. This guide also contains a section about performance recommendations, which we recommend reading first. Finally, here is also an Recurrent Neural Network (RNN) Model example, on which this section is going to be based on.
Use CPU
By default all models will run on CPU. As shown in the RNN example above, we'll import the Air Passenger dataset, as well as other necessary modules.
import numpy as np
import pandas as pd
import matplotlib.pyplot as plt
from darts.dataprocessing.transformers import Scaler
from darts.models import RNNModel
from darts.metrics import mape
from darts.datasets import AirPassengersDatasetNow we read and scale the data like this:
# Read data:
series = AirPassengersDataset().load()
series = series.astype(np.float32)
# Create training and validation sets:
train, val = series.split_after(pd.Timestamp("19590101"))
# Normalize the time series (note: we avoid fitting the transformer on the validation set)
transformer = Scaler()
train_transformed = transformer.fit_transform(train)
val_transformed = transformer.transform(val)
series_transformed = transformer.transform(series)Next we will create our RNN like this:
my_model = RNNModel(
model="RNN",
hidden_dim=20,
dropout=0,
batch_size=16,
n_epochs=300,
optimizer_kwargs={"lr": 1e-3},
model_name="Air_RNN",
log_tensorboard=True,
random_state=42,
training_length=20,
input_chunk_length=14,
force_reset=True,
)and fit it to the data:
my_model.fit(train_transformed, val_series=val_transformed)where in the output we can see that no other processing unit is used to train our model:
GPU available: False, used: False
TPU available: False, using: 0 TPU cores
IPU available: False, using: 0 IPUs
| Name | Type | Params
--------------------------------------
0 | criterion | MSELoss | 0
1 | rnn | RNN | 460
2 | V | Linear | 21
--------------------------------------
481 Trainable params
0 Non-trainable params
481 Total params
0.004 Total estimated model params size (MB)
Epoch 299: 100% 8/8 [00:00<00:00, 42.49it/s, loss=0.00285, v_num=logs]
<darts.models.forecasting.rnn_model.RNNModel at 0x7fc75901cb10>Now the model is ready to start predicting, which won't be shown here since it's included in the example linked in the start of this guide.
Use a GPU
GPUs can dramatically improve the performance of your model in terms of processing time. By using an Accelerator in the Pytorch Lightning Trainer, we can enjoy the benefits of a GPU. We only need to instruct our model to use our machine's GPU through PyTorch Lightning Trainer parameters, which are expressed as the pl_trainer_kwargs dictionary, like this:
my_model = RNNModel(
model="RNN",
...
force_reset=True,
pl_trainer_kwargs={
"accelerator": "gpu",
"devices": [0]
},
)which now outputs:
GPU available: True, used: True
TPU available: False, using: 0 TPU cores
IPU available: False, using: 0 IPUs
LOCAL_RANK: 0 - CUDA_VISIBLE_DEVICES: [0]
| Name | Type | Params
--------------------------------------
0 | criterion | MSELoss | 0
1 | rnn | RNN | 460
2 | V | Linear | 21
--------------------------------------
481 Trainable params
0 Non-trainable params
481 Total params
0.004 Total estimated model params size (MB)
Epoch 299: 100% 8/8 [00:00<00:00, 39.81it/s, loss=0.00285, v_num=logs]
<darts.models.forecasting.rnn_model.RNNModel at 0x7ff1b5e4d4d0>From the output we can see that the GPU is both available and used. The rest of the code doesn't require any change, i.e. it's irrelevant if we are using a GPU or CPU.
Multi GPU support
Darts utilizes Lightning's multi GPU capabilities to be able to capitalize on scalable hardware.
Multiple parallelization strategies exist for multiple GPU training, which - because of different strategies for multiprocessing and data handling - interact strongly with the execution environment.
Currently in Darts the ddp_spawn distribution strategy is tested.
As per the description of the Lightning documentation has some noteworthy limitations, eg. it can not run in:
Jupyter Notebook, Google COLAB, Kaggle, etc.
In case you have a nested script without a root package
This in practice means, that execution has to happen in a separate .py script, that has the following general context around the code executing the training:
import torch
if __name__ == '__main__':
torch.multiprocessing.freeze_support()The main pattern is necessary (see this) even when your execution does not happen in a windows environment.
Beyond this, no other major modification to your models is necessary other than allowing multi GPU training in the pl_trainer_args for example like
pl_trainer_kwargs = {"accelerator": "gpu", "devices": -1, "auto_select_gpus": True}
This method automatically selects all available GPUs for training. Manual setting of the number of devices is also possible.
The ddp family of strategies creates individual subprocesses for each GPU, so contents of the memory (notably the Dataloder) gets copied over. Thus, as per the description of lightning docs caution is advised in setting the Dataloader(num_workers=N) too high, since according to it:
"Dataloader(num_workers=N), where N is large, bottlenecks training with DDPโฆ ie: it will be VERY slow or wonโt work at all. This is a PyTorch limitation."
Usage of other distribution strategies with Darts currently might very well work, but are yet untested and subject to individual setup / experimentation.
Use a TPU
Tensor Processing Unit (TPU) is an AI accelerator application-specific integrated circuit (ASIC) developed by Google specifically for neural network machine learning.
There are three main ways to get access to a TPU:
- Google Colab
- Google Cloud (GCP)
- Kaggle
If you are using a TPU in the Google Colab kind of notebook, then you should first install these:
!pip install cloud-tpu-client==0.10 https://storage.googleapis.com/tpu-pytorch/wheels/torch_xla-1.9-cp37-cp37m-linux_x86_64.whl
!pip install torch==1.9.0+cu111 torchvision==0.10.0+cu111 torchtext==0.10.0 -f https://download.pytorch.org/whl/cu111/torch_stable.html
!pip install pyyaml==5.4.1and then instruct our model to use a TPU or more. In our example we are using four TPUs, like this:
my_model = RNNModel(
model="RNN",
...
force_reset=True,
pl_trainer_kwargs={
"accelerator": "tpu",
"tpu_cores": [4]
},
)which outputs:
WARNING:root:TPU has started up successfully with version pytorch-1.9
GPU available: False, used: False
TPU available: True, using: [4] TPU cores
IPU available: False, using: 0 IPUs
| Name | Type | Params
--------------------------------------
0 | criterion | MSELoss | 0
1 | rnn | RNN | 460
2 | V | Linear | 21
--------------------------------------
481 Trainable params
0 Non-trainable params
481 Total params
0.002 Total estimated model params size (MB)
Epoch 299: 100% 8/8 [00:00<00:00, 8.52it/s, loss=0.00285, v_num=logs]
<darts.models.forecasting.rnn_model.RNNModel at 0x7ff1b5e4d4d0>From the output we can see that our model is using 4 TPUs.
Userguide Hyperparameter Optimization (`docs/userguide/hyperparameter_optimization.md`)
Hyperparameter Optimization in Darts
There is nothing special in Darts when it comes to hyperparameter optimization.
The main thing to be aware of is probably the existence of PyTorch Lightning callbacks for early stopping and pruning of experiments with Darts' deep learning based TorchForecastingModels.
Below, we show examples of hyperparameter optimization done with Optuna and
Ray Tune.
Hyperparameter optimization with Optuna
Optuna is a great option for hyperparameter optimization with Darts. Below, we show a minimal example
using PyTorch Lightning callbacks for pruning experiments.
For the sake of the example, we train a TCNModel on a single series, and optimize (probably overfitting) its hyperparameters by minimizing the prediction error on a validation set.
You can also have a look at this notebook
for a more complete example.
NOTE (2023-19-02): Optuna's
PyTorchLightningPruningCallbackraises an error with pytorch-lightning>=1.8. Until this fixed, a workaround is proposed here.
/* Detailed source-code truncated for AI context efficiency. */Hyperparameter optimization with Ray Tune
Ray Tune is another option for hyperparameter optimization with automatic pruning.
Here is an example of how to use Ray Tune to with the NBEATSModel model using the Asynchronous Hyperband scheduler. The example was tested with ray version ray==2.32.0.
/* Detailed source-code truncated for AI context efficiency. */Hyperparameter optimization using `gridsearch()`
Each forecasting models in Darts offer a gridsearch() method for basic hyperparameter search.
This method is limited to very simple cases, with very few hyperparameters, and working with a single time series only.
Userguide Safe Model Loading (`docs/userguide/safe_model_loading.md`)
Safe model loading
Darts saves most forecasting models with Python pickle (.pkl) or, for PyTorch models, a Darts .pt wrapper plus a Lightning .ckpt checkpoint. Unpickling untrusted files can execute arbitrary code (CWE-502).
Safe loading is the default for all models, and related checkpoint/weight loaders. Restricted deserialization only reconstructs globals that pass Dartsโ allow-list (known-safe libraries, class hierarchies, and checkpoint-driven registration for .ckpt files).
In most cases, safe loading should work out of the box.
When loading fails
If a global is not allow-listed, loading raises an error that names the missing {module}.{qualname}. This can happen for example when using custom PyTorch Lightning callbacks, custom encoders, and others. If you trust that global (or file in general), you can still load using one of the steps below.
Register missing globals as safe globals โ Same entry shapes as PyTorch
add_safe_globals(a callable, or(callable, qualname_str)when the name in the file differs from the objectโs module path). Registration binds each pickled{module}.{qualname}to the exact callable or class you provide, so loading works even when that name is not importable (for example functions defined in__main__, notebooks, or one-off scripts). Use the(callable, qualname_str)form when the string in the saved file does not match{callable.__module__}.{callable.__qualname__}:pythonimport pandas as pd from darts.datasets import AirPassengersDataset from darts.models import LinearRegressionModel from darts.utils.serialization import add_safe_globals, safe_globals def my_encoder(idx: pd.DatetimeIndex): """Custom encoder example adding the series' month value.""" return idx.month # add encoder as a future covariate, fit and save the model model = LinearRegressionModel( lags=12, lags_future_covariates=[0], add_encoders={"custom": {"future": [my_encoder]}} ) series = AirPassengersDataset().load() model.fit(series) model.save("model.pkl") # by default, loading the model will fail since `my_encoder` is not an # allow-listed global. To fix this, either add the safe globals within a context with safe_globals([my_encoder]): model = LinearRegressionModel.load("model.pkl") # or add them globally via `add_safe_globals()`: add_safe_globals([my_encoder]) model = LinearRegressionModel.load("model.pkl")Full trust opt-out โ Only if safe loading is not feasible and you fully trust the file source:
pythonmodel = NBEATSModel.load("model.pt", trusted=True)
Related API
- Find more information in the API docs
Userguide Timeseries (`docs/userguide/timeseries.md`)
`TimeSeries`
TimeSeries is the main data class in Darts. A TimeSeries represents a univariate or multivariate time series, with a proper time index. The time index can either be of type pandas.DatetimeIndex (containing datetimes), or of type pandas.RangeIndex (containing integers; useful for representing sequential data without specific timestamps). In some cases, TimeSeries can even represent probabilistic series, in order for instance to obtain confidence intervals.
TimeSeries are used in Darts in order to offer a consistent API dedicated to time series:
TimeSeriesare guaranteed to have a proper time index (integer or datetime based): complete and time-sorted.- All models in Darts consume
TimeSeriesand produceTimeSeries. TimeSeriesallows to do all sort of time series operations (such as splitting at timestamps, concatenating, etc).TimeSeriesoffers user-friendly way to represent multidimensional and probabilistic series (e.g. obtain marginal quantiles).
Using a dedicated type to represent time series (instead of say, Pandas DataFrame, NumPy array, ...) removes the need to rely on conventions about the formats expected by the different models and functions.
Multivariate time series -vs- multiple time series
We distinguish univariate from multivariate series:
- A multivariate series contain multiple dimensions (i.e. multiple values per time step).
- A univariate series contains only one dimension (i.e., single scalar value for each time step).
Sometimes the dimensions are called components. A single TimeSeries object can be either univariate (if it has a single component), or multivariate (if it has multiple components). In a multivariate series, all components share the same time axis. I.e., they all share the same time stamps.
Some models in Darts (and all machine learning models) support multivariate series. This means that they can take multivariate series in inputs (either as targets or as covariates), and the forecasts they produce will have a dimensionality matching that of the targets.
In addition, some models can work on multiple time series, meaning that they can be trained on multiple TimeSeries objects, and used to forecasts multiple TimeSeries objects in one go. This is sometimes referred to as panel data. In such cases, the different TimeSeries need not share the same time index -- for instance, some series might be in 1990 and others in 2000. In fact, the series need not even have the same frequency. The models handling multiple series expect Python Sequences of TimeSeries in inputs (for example, a simple list of TimeSeries).
Figure 1: A single multivariate series. In Darts this is represented by a single TimeSeries object.
Figure 2: Multiple time series. In Darts this is represented by multiple TimeSeries objects. The series may or may not share the same time axis, and may or may not be multivariate.
Example of a multivariate series: The blood pressure and heart rate of a single patient over time (one multivariate series with 2 components).
Example of multiple series: The blood pressure and heart rate of multiple patients; potentially measured at different times for different patients (one multivariate series with 2 components per patient).
Should I use a multivariate series or multiple series for my problem?
Using multivariate series will provide an opportunity for models to capture the dimensions jointly, whereas using multiple series will expose the models to multiple observations. So which representation is right is problem-dependent, and in some cases might not even be clear beforehand and require experimentation. Nevertheless, if you are in a situation where you are trying to model several "correlated" components varying together across time (e.g. the coordinates of a moving object, or multiple measurements related to a single entity), it is often natural to capture those as multiple components in single series. If on the other hand you have several instance of such measurements (e.g., obtained in different experiments or observations), this can mean that you have multiple time series. Furthermore, if your values do not share the same time spans, they should probably be represented as different series.
Probabilistic and deterministic series
In Darts, probabilistic forecasts are represented by drawing Monte Carlo samples from the underlying probabilistic models. This representation allows TimeSeries to represent arbitrary joint distributions (over time and components) without relying on any pre-defined parametric form. Based on this, we define two types of time series:
- A Probabilistic (or stochastic) series contains multiple samples.
- A deterministic series contains only one sample.
Creating `TimeSeries`
TimeSeries objects can be created using factory methods, for example:
TimeSeries.from_dataframe() can create
TimeSeriesfrom a Pandas Dataframe having one or several columns representing values (columns correspond to components, and several columns would correspond to a multivariate series).TimeSeries.from_values() can create
TimeSeriesfrom a 1-D, 2-D or 3-D NumPy array. It will generate an integer-based time index (of typepandas.RangeIndex). 1-D corresponds to univariate deterministic series, 2-D to multivariate deterministic series, and 3-D to multivariate stochastic series.TimeSeries.from_times_and_values() is similar to
TimeSeries.from_values()but also accepts a time index.
More methods are documented in the TimeSeries API documentation.
Furthermore, it is possible to concatenate series along different axes using the function concatenate(). axis=0 corresponds to time, axis=1 corresponds to component and axis=2 correspond to stochastic sample dimensions. For instance:
from darts import concatenate
my_multivariate_series = concatenate([series1, series2, ...], axis=1)produces a multivariate series from some series that share the same time axis.
Implementation
Behind the scenes, TimeSeries is wrapping around a 3-dimensional xarray.DataArray object. The dimensions are (time, component, sample), where the size of the component dimension is larger than 1 for multivariate series and the size of the sample dimension is larger than 1 for stochastic series. The DataArray is itself backed by a 3-dimensional NumPy array, and it has a time index (either pandas.DatetimeIndex or pandas.RangeIndex) on the time dimension and another pandas.Index on the component (or "columns") dimension. TimeSeries is intended to be immutable and most operations return new TimeSeries objects.
Exporting data from a `TimeSeries`
TimeSeries objects offer a few ways to export the data, for example:
TimeSeries.to_dataframe() to export a Dataframe (for deterministic series)
TimeSeries.data_array() to export the
xarrayDataArrayholding this series' data (and indexes).TimeSeries.values() to export a NumPy array containing the values of one sample from the series.
TimeSeries.all_values() to export a NumPy array containing the values of all samples of a stochastic series.
Static Covariates
Optionally, TimeSeries objects can contain static data (referred to as static covariates), which can be exploited by some models. Example of static covariates can be:
- Store location - for instance for each store (component) in a multivariate series
- Product ID
- Sensor type
- ...
The static covariates have to be specified by a pandas DataFrame whose rows match the TimeSeries's components, and whose columns represent the dimensions of the static covariates. They can be added to a TimeSeries when using most factory methods, using the static_covariates parameter. They can also be added to an existing TimeSeries using the with_static_covariates() method.
Hierarchical Time Series
Optionally, TimeSeries objects can contain a hierarchy, which specifies how its different components are grouped together. The hierarchy itself is specified as a Python dictionary, mapping the components' names to a list of names of their parent(s) in the hierarchy.
For instance, the following hierarchy means that the two components "a" and "b" add up to "total":
hierarchy = {"a": ["total"], "b": ["total"]}Hierarchies can be used for posthoc forecast reconciliation. Darts offers several reconciliation transformers (usable with fit()/transform()) - see the corresponding API documentation.
More information and documentation
The full list of attributes and methods of the TimeSeries class is listed in the API Documentation. In addition, the Quickstart contains some examples of TimeSeries manipulation.
Userguide Torch Forecasting Models (`docs/userguide/torch_forecasting_models.md`)
Torch Forecasting Models
This document was written for darts version 0.15.0 and later.
We assume that you already know about covariates in Darts. If you're new to the topic we recommend you to read our guide on covariates first.
Content of this guide
Introduction section covers the most important points about Torch Forecasting Models (TFMs):
Input data usage section gives an in-depth guide of how input data is used when training and predicting with TFMs:
Advanced functionalities section provides some example of TFMs advanced features:
Performance optimization section lists tricks to speed up the computation during training.
Introduction
In Darts, Torch Forecasting Models (TFMs) are broadly speaking "machine learning based" models, which denote PyTorch-based (deep learning) models.
TFMs train and predict on fixed-length chunks (sub-samples) of your input target and *_covariates series (if supported). Target is the series for which we want to predict the future, *_covariates are the past and / or future covariates.
Each chunk contains an input chunk - representing the sample's past - and an output chunk - the sample's future. The sample's prediction point lies at the end of the input chunk. The length of these chunks has to be specified at model creation with parameters input_chunk_length and output_chunk_length (more on chunks in the next subsection).
# model that looks 7 time steps back (past) and 1 time step ahead (future)
model = SomeTorchForecastingModel(input_chunk_length=7,
output_chunk_length=1,
**model_kwargs)All TFMs can be trained on single or multiple target series and, depending on their covariate support (covered in this subsection on covariates support), past_covariates and / or future_covariates. When using covariates you have to supply one dedicated past and / or future covariates series for each target series.
Optionally, you can use a validation set with dedicated covariates during training. If the covariates have the required time spans, you can use the same for training, validation and prediction. (covered in this subsection on time span requirements)
# fit the model on a single target series with optional past and / or future covariates
model.fit(target,
past_covariates=past_covariates,
future_covariates=future_covariates,
val_series=target_val, # optionally, use a validation set
val_past_covariates=past_covariates_val,
val_future_covariates=future_covariates_val)
# fit the model on multiple target series
model.fit([target, target2, ...],
past_covariates=[past_covariates, past_covariates2, ...],
...
)You can produce forecasts for any input target TimeSeries or for several targets given as a sequence of TimeSeries. This will also work on series that have not been seen during training, as long as each series contains at least input_chunk_length time steps.
# predict the next n=3 time steps for any input series with `series`
prediction = model.predict(n=3,
series=target,
past_covariates=past_covariates,
future_covariates=future_covariates)If you want to know more about the training and prediction process of our Torch Forecasting Models and how they work with covariates, read on.
Top level look at training and predicting with chunks
In Figure 1 you can see how your data is distributed to the input and output chunks for each sample when calling fit() or predict(). For this example we look at data with daily frequency. The input chunk extracts values from target and optionally from past_covariates and / or future_covariates that fall into the input chunk time span. These "past" values of future_covariates are called "historic future covariates".
The output chunk only takes optional future_covariates values that fall into the output chunk time span. The future values of our past_covariates - "future past covariates" - are only used to provide the input chunk of upcoming samples with new data.
All this information is used to predict the "future target" - the next output_chunk_length points after the end of "past target".
Figure 1: Top level look at training / predicting on chunks with Torch Forecasting Models
When calling predict() and depending on your forecast horizon n, the model can either predict in one go (if n <= output_chunk_length), or auto-regressively, by predicting on multiple chunks in the future (if n > output_chunk_length). That is the reason why when predicting with past_covariates you might have to supply additional "future values of your past_covariates".
Torch Forecasting Model Covariates Support
Under the hood, Darts has 5 types of {X}CovariatesModel classes implemented to cover different combinations of the covariate types mentioned before:
| Class | past covariates | future past covariates | future covariates | historic future covariates |
|---|---|---|---|---|
PastCovariatesModel |
โ | โ | ||
FutureCovariatesModel |
โ | |||
DualCovariatesModel |
โ | โ | ||
SplitCovariatesModel |
โ | โ | โ | |
MixedCovariatesModel |
โ | โ | โ | โ |
Table 1: Darts' "{X}CovariatesModels" covariate support
Each Torch Forecasting Model inherits from one {X}CovariatesModel (covariate class names are abbreviated by the X-part):
| TFM | Past |
Future |
Dual |
Split |
Mixed |
|---|---|---|---|---|---|
RNNModel |
โ | ||||
BlockRNNModel |
โ | ||||
NBEATSModel |
โ | ||||
TCNModel |
โ | ||||
TransformerModel |
โ | ||||
TFTModel |
โ | ||||
NLinearModel |
โ | ||||
DLinearModel |
โ | ||||
TiDEModel |
โ | ||||
TSMixerModel |
โ |
Table 2: Darts' Torch Forecasting Model covariate support
Required target time spans for training, validation and prediction
The relevant data is extracted automatically by the models, based on the time axes of the series.
You can use the same covariates series for both fit() and predict() if they meet the requirements below.
Training only works if at least one sample with an input and output chunk can be extracted from the data you passed to fit(). This applies both to training and validation data. In terms of minimum required time spans, this means:
targetseries of minimum lengthinput_chunk_length + output_chunk_length*_covariatestime span requirements forfit()from covariates guide section 2.3.
For prediction you have to supply the target series that you wish to forecast. For any forecast horizon n the minimum time span requirements are:
targetseries of minimum lengthinput_chunk_length*_covariatestime span requirements forpredict()also from from covariates guide section 2.3.
Side note: Our *RNNModels accept a training_length parameter at model creation instead of output_chunk_length. Internally the output_chunk_length for these models is automatically set to 1. For training, past target must have a minimum length of training_length + 1 and for prediction, a length of input_chunk_length.
In-depth look at how input data is used when training and predicting with TFMs
Training
Let's have a look at how the models work under the hood.
Let's assume we run an ice-cream shop and we want to predict sales for the next day.
We have one year (365 days) past data of our end-of-day ice-cream sales and of the average measured daily ambient temperature.
We also noticed that our ice-cream sales depend on the day of the week so we want to include this in our model.
- past target: actual past ice-cream sales
ice_cream_sales - future target: predict the ice-cream sales for the next day
- past covariates: measured average daily temperatures in the past
temperature - future covariates: day of the week for past and future
weekday
Checking Table 1, a model that would accommodate this kind of covariates would be aSplitCovariatesModel (if we don't use historic values of future covariates), orMixedCovariatesModel (if we do). We choose a MixedCovariatesModel - the TFTModel.
Imagine that we saw a pattern in our past ice cream sales that repeated week after week.
So we set input_chunk_length = 7 days to let the model look back an entire week into the past.
The output_chunk_length can be set to 1 day to predict the next day.
Now we can create a model and train it! Figure 2 shows you how TFTModel will use our data.
from darts.models import TFTModel
model = TFTModel(input_chunk_length=7, output_chunk_length=1)
model.fit(series=ice_cream_sales,
past_covariates=temperature,
future_covariates=weekday)Figure 2: Overview of a single sequence from our ice-cream sales example; Mon1 - Sun1 stand for the first 7 days from our training dataset (week 1 of the year). Mon2 is the Monday of week 2.
When calling fit(), the models will build an appropriate darts.utils.data.TorchTrainingDataset, which specifies how to slice the data to obtain training samples. If you want to control this slicing yourself, you can instantiate your own TorchTrainingDataset and call model.fit_from_dataset() instead of fit(). By default, most models (though not all) will build sequential datasets, which basically means that all sub-slices of length input_chunk_length + output_chunk_length in the provided series will be used for training.
So during training, the torch models will go through the training data in sequences (see Figure 3). Using information from the input chunk and output chunk, the model predicts the future target on the output chunk. The training loss is evaluated between the predicted future target and the actual target value on the output chunk. The model trains itself by minimizing the loss over all sequences.
Figure 3: Prediction and loss evaluation in a single sequence
After having completed computations on the first sequence, the model moves to the next one and performs the same training steps. The starting point of each sequence is selected randomly from the sequential dataset. Figure 4 shows how this would look like if by pure chance the second sequence started one time step (day) after the first.
This sequence-to-sequence process is repeated until all 365 days were covered.
Side note: Having "long" target series can result in a very large number of training sequences / samples. You can set an upper bound for the number of sequences / samples per target that the model should be trained on with fit()-parameter max_samples_per_ts. This will take the most recent sequences for every target series (sequences closest to target end).
# fit only on the 10 "most recent" sequences
model.fit(target, max_samples_per_ts=10)Figure 4: Sequence-to-sequence: Move to next sequence and repeat training steps
Training with a validation dataset
You can also train your models with a validation dataset:
# create train and validation sets
ice_cream_sales_train, ice_cream_sales_val = ice_cream_sales.split_after(training_cutoff)
# train with validation set
model.fit(series=ice_cream_sales_train,
past_covariates=temperature,
future_covariates=weekday,
val_series=ice_cream_sales_val,
val_past_covariates=temperature,
val_future_covariates=weekday)If you split your data, you have to define a training_cutoff (a date or fraction at which to split the dataset) so that both the train and validation datasets satisfy the minimum length requirements
from this subsection on time span requirements
Instead of splitting by time, you can also use another subset of time series as validation set.
The model trains itself the same way as before but additionally evaluates the loss on the validation dataset. If you want to keep track of the best performing model on the validation set, you have to enable checkpoint saving as shown next.
Forecasting
After having trained the model, we want to predict the future ice-cream sales for any number of days after our 365 days training data.
The actual prediction works very similar to how we trained the data on sequences. Depending on the number of days we want to predict - the forecast horizon n - we distinguish between two cases:
If
n <= output_chunk_length: we can predictnin one go (using one "internal model call")- in our example: predict the next day's ice-cream sales (
n = 1)
- in our example: predict the next day's ice-cream sales (
If
n > output_chunk_length: we must predictnby calling the internal model multiple times. Each call outputsoutput_chunk_lengthprediction points. We go through as many calls as needed until we get to the finalnprediction points, in an auto-regressive fashion.- in our example: predict ice-cream sales for the next 3 days at once (
n = 3)
To do this we have to supply additional
past_covariatesfor the nextn - output_chunk_length = 2time steps (days) after the end of our 365 days training data. Unfortunately, we do not have measuredtemperaturefor the future. But let's assume we have access to temperature forecasts for the next 2 days. We can just append them totemperatureand the prediction will work!pythontemperature = temperature.concatenate(temperature_forecast, axis=0)- in our example: predict ice-cream sales for the next 3 days at once (
prediction = model.predict(n=n,
series=ice_cream_sales_train,
past_covariates=temperature,
future_covariates=weekday)Figure 5: Forecast with a single sequence for n <= output_chunk_length
Figure 6: Auto-regressive forecast for n > output_chunk_length
Advanced Functionnalities
Saving and Loading Model States
Darts offers several ways for saving and loading TFMs. Two artifacts are stored on every save: a Darts .pt wrapper and a PyTorch (Lightning) .ckpt checkpoint containing the internal model's state dict and states of other objects.
Security: Loading uses safe-by-default deserialization to mitigate malicious pickle files (CWE-502). Use trusted=True only for files you fully trust, or register custom classes/functions via add_safe_globals / with safe_globals(...). See the safe model loading guide for details.
Note: Backward compatibility is not always guaranteed. Loading a model saved by an older Darts version might fail.
Automatic checkpointing
Automic checkpointing during training allows you to:
- keep track of the model state over the latest 5 epochs, and the best performing epoch based on the validation set loss
- load a model from checkpoint to resume training in case it was interrupted
- load a model from checkpoint for inference / forecasting
You can activate checkpointing at model creation:
model = SomeTorchForecastingModel(..., model_name='my_model', save_checkpoints=True)
# checkpoints are saved automatically
model.fit(...)
# load the model state that performed best on validation set
best_model = model.load_from_checkpoint(model_name='my_model', best=True)Manual saving and loading
You can also manually save the model at its current state and load it:
model.save("/your/path/to/save/model.pt")
loaded_model = model.load("/your/path/to/save/model.pt")Training and saving on GPU, loading on CPU
You can load a model to CPU that was trained and saved on GPU (see detailed documentation):
# define a model using gpu as accelerator
model = SomeTorchForecastingModel(
...,
model_name='my_model',
save_checkpoints=True,
pl_trainer_kwargs={"accelerator":"gpu", "devices": -1},
)
# train the model, automatic checkpoints will be created
model.fit(...)
# specify the device to which the model should be loaded
loaded_model = SomeTorchForecastingModel.load_from_checkpoint(
model_name='my_model',
best=True,
map_location="cpu",
)
loaded_model.to_cpu()
# run inference
loaded_model.predict(...)Manual saves can also be loaded to CPU:
model.save("/your/path/to/save/model.pt")
loaded_model = model.load("/your/path/to/save/model.pt", map_location="cpu")
loaded_model.to_cpu()Retraining or finetuning a pretrained model
To re-train or fine-tune a model using a different optimizer and/or learning rate scheduler, you can load the weights from the automatic checkpoints into a new model:
# model with identical architecture but different optimizer (default: torch.optim.Adam)
model_finetune = SomeTorchForecastingModel(
..., # use identical parameters & values as in original model
optimizer_cls=torch.optim.SGD,
optimizer_kwargs={"lr": 0.001},
)
# load the weights from a checkpoint
model_finetune.load_weights_from_checkpoint(model_name='my_model', best=True)
model_finetune.fit(...)and similarly for manual saves and the learning rate scheduler:
# model with identical architecture but different lr scheduler (default: None)
model_finetune = SomeTorchForecastingModel(
..., # use identical parameters & values as in original model
lr_scheduler_cls=torch.optim.lr_scheduler.ExponentialLR,
lr_scheduler_kwargs={"gamma": 0.09}
)
# load the weights from a manual save
model_finetune.load_weights("/your/path/to/save/model.pt")Additionally, all TorchForecastingModel (including FoundationModel) support fine-tuning through the enable_finetuning parameter. This allows you to freeze or unfreeze specific model parameters for transfer learning:
# full fine-tuning: all parameters are trainable
model = SomeTorchForecastingModel(..., enable_finetuning=True)
# partial fine-tuning: freeze specific layers
model = SomeTorchForecastingModel(..., enable_finetuning={"freeze": ["layer.pattern.*"]})
# partial fine-tuning: only unfreeze specific layers
model = SomeTorchForecastingModel(..., enable_finetuning={"unfreeze": ["layer.pattern.*"]})
# optionally, load pre-trained weights (not required for foundation models)
model.load_weights("/your/path/to/save/model.pt")
# fine-tune
model.fit(...)For a comprehensive walkthrough of fine-tuning Torch Forecasting Models and Foundation Models, check out the Fine-Tuning example notebook.
Exporting model to ONNX format for inference
Exporting models to ONNX format and running inference optional dependencies onnx>=1.0.0, onnxruntime>=1.24.1, and onnxscript>=0.7.0. After export, inference can run without PyTorch using only onnxruntime, NumPy, and Darts' torch-free helpers in darts.utils.onnx.inference.
to_onnx() exports the model to ONNX format.
model = SomeTorchForecastingModel(...)
model.fit(...)
horizon = model.output_chunk_length
# export requires torch + onnx
onnx_filename = "example.onnx"
model.to_onnx(onnx_filename, export_params=True)Now forecast using Darts' run_onnx_prediction() which mirrors TorchForecastingModel.predict() for deterministic models with num_samples=1 and probabilistic models with predict_likelihood_parameters=True.
import onnxruntime as ort
from darts.utils.onnx import run_onnx_prediction
session = ort.InferenceSession(onnx_filename)
# returns forecasts as (a sequence of) `TimeSeries`
forecast = run_onnx_prediction(
n=horizon,
session=session,
series=series,
past_covariates=ts_past,
future_covariates=ts_future,
)For a single ONNX step (advanced / custom loops), extract features with prepare_onnx_inputs using the spec:
from darts.utils.onnx.inference import OnnxModelSpec, prepare_onnx_inputs
session = ort.InferenceSession(onnx_filename)
spec = OnnxModelSpec.from_session(session)
onnx_inputs = prepare_onnx_inputs(
series=series,
spec=spec,
past_covariates=ts_past,
future_covariates=ts_future,
)
forecast = session.run(spec.output_names, onnx_inputs)Raw ONNX output tensors (NumPy arrays) have shape (batch, output_chunk_length, n_components, n_likelihood_params). Models trained with a likelihood export those raw parameters (sampling is not in the graph).
Callbacks
Callbacks are a powerful way to monitor or control the behavior of the model during the training process. Some examples:
- Performance Monitoring: compute additional metrics (in addition of the default losses)
- Early stopping: stop the training once the model has converged
- ...
With callbacks you can add custom code to an existing process at predefined points / hooks.
The code is triggered once the process execution reaches the corresponding hooks. Some example hooks:
- beginning / end of training
- beginning / end of train / validation step
- ...
Some useful predefined PyTorch Lightning callbacks can be found here.
Example with Early Stopping
Early stopping is an efficient way to avoid overfitting and reduce training time.
It will exit the training process once the validation loss has not significantly improved over some epochs.
You can use Early Stopping with any TorchForecastingModel, leveraging PyTorch Lightning's EarlyStopping callback:
import pandas as pd
from pytorch_lightning.callbacks import EarlyStopping
from torchmetrics import MeanAbsolutePercentageError
from darts.dataprocessing.transformers import Scaler
from darts.datasets import AirPassengersDataset
from darts.models import NBEATSModel
# read data
series = AirPassengersDataset().load()
# create training and validation sets:
train, val = series.split_after(pd.Timestamp(year=1957, month=12, day=1))
# normalize the time series
transformer = Scaler()
train = transformer.fit_transform(train)
val = transformer.transform(val)
# any TorchMetric or val_loss can be used as the monitor
torch_metrics = MeanAbsolutePercentageError()
# early stop callback
my_stopper = EarlyStopping(
monitor="val_MeanAbsolutePercentageError", # "val_loss",
patience=5,
min_delta=0.05,
mode='min',
)
pl_trainer_kwargs = {"callbacks": [my_stopper]}
# create the model
model = NBEATSModel(
input_chunk_length=24,
output_chunk_length=12,
n_epochs=500,
torch_metrics=torch_metrics,
pl_trainer_kwargs=pl_trainer_kwargs)
# use validation set for early stopping
model.fit(
series=train,
val_series=val,
)To use early-stopping and pruning in the context of hyperparameter optimization, check out this guide.
Example of custom callback to store losses
Training and validation loss can be automatically logged with tensorboard. When activated, Darts will by default store the logs to a folder darts_logs in the current working directory. You can change this with model parameters work_dir, and model_name.
model = SomeTorchForecastingModel(..., log_tensorboad, save_checkpoints=True)
model.fit(...)After installing the tensorboard library, you can visualize the logs from the command line:
tensorboad --log_dir darts_logsLet's check out how to implement a custom callback to make the model losses accessible in Python.
from pytorch_lightning.callbacks import Callback
class LossLogger(Callback):
def __init__(self):
self.train_loss = []
self.val_loss = []
# will automatically be called at the end of each epoch
def on_train_epoch_end(self, trainer: "pl.Trainer", pl_module: "pl.LightningModule") -> None:
self.train_loss.append(float(trainer.callback_metrics["train_loss"]))
def on_validation_epoch_end(self, trainer: "pl.Trainer", pl_module: "pl.LightningModule") -> None:
self.val_loss.append(float(trainer.callback_metrics["val_loss"]))
loss_logger = LossLogger()
model = SomeTorchForecastingModel(
...,
nr_epochs_val_period=1, # perform validation after every epoch
pl_trainer_kwargs={"callbacks": [loss_logger]}
)
# fit must include validation set for "val_loss"
model.fit(...)Note : The callback will give one more element in the loss_logger.val_loss as the model trainer performs a validation sanity check before the training begins.
Performance Recommendations
This section recaps the main factors impacting the performance when
training and using torch-based models.
Build your `TimeSeries` using 32-bits data
The models in Darts will dynamically cast themselves (to 64 or 32-bits)
to follow the dtype in the TimeSeries. Large performance and memory gains
can often be obtained when everything (data and model) is in float32.
To achieve this, it is enough to build your TimeSeries from arrays (or Dataframe-backing array) having dtype np.float32, or simply call my_series32 = my_series.astype(np.float32). Calling my_series.dtype gives you the dtype of your TimeSeries.
Use a GPU
In many cases using a GPU will provide a drastic speedup compared to CPU.
It can also incur some overheads (for transferring data to/from the GPU),
so some testing and tuning is often necessary.
We refer to our GPU/TPU guide
for more information on how to setup a GPU (or a TPU) via PyTorch Lightning.
Tune the batch size
A larger batch size tends to speed up the training because it reduces the number
of backward passes per epoch and has the potential to better parallelize computation. However it also changes the training dynamics (e.g. you might need more epochs, and the convergence dynamics is affected). Furthermore larger batch sizes increase memory consumption. So here too some testing is required.
Tune `num_loader_workers`
All deep learning models in Darts have a parameter dataloader_kwargs in their fit() and predict() functions, which configures the PyTorch DataLoaders. The num_workers parameter for PyTorch DataLoaders can be set using the num_workers key in the dataloader_kwargs dictionary.
Setting num_workers > 0 will use additional workers to load the data. This typically incurs some overhead (notably increasing memory consumption), but in some cases it can also substantially improve performance.
The ideal value depends on many factors such as the batch size, whether you are using a GPU, the number of CPU cores available, and whether
loading the data involved I/O operations (if the series are stored on disk).
Small models first
Of course one of the main factors affecting performance is the model size
(number of parameters) and the number of operations required by forward/backward passes. Models in Darts can be tuned (e.g. number of layers, attention heads, widths etc), and these hyper-parameters tend to have a large impact on performance. When starting out, it is a good idea to build models of modest size first.
Data in Memory and I/O bottlenecks
It's helpful to load all your TimeSeries in memory upfront if you can.
Darts offers the possibility to train models on any Sequence[TimeSeries],
which means that for big datasets, you can write your own Sequence implementation, and read the time series lazily from disk. This will typically incur a high I/O cost, though. So when training on multiple series, first try to build a simple List[TimeSeries] upfront, and see if it holds in the computer memory.
Do not use *all* possible sub-series for training
By default, when calling fit(), the models in Darts will build a TorchTrainingDataset instance that is
suitable for the model that you are using (e.g., PastCovariatesTorchModel, FutureCovariatesTorchModel, etc.).
By default, these training datasets will often contain all possible consecutive (input, output) subseries present
in each TimeSeries. If your TimeSeries are long, this can result in a large amount of training samples, which directly (linearly)
impacts the time required to train the model for one epoch. You have two options to limit this:
- Specify some
max_samples_per_tsargument to thefit()function. This will use only the most recentmax_samples_per_tssamples
perTimeSeriesfor training. - If this option does not do what you want, you can implement your own
TorchTrainingDatasetinstance, and define
how to slice yourTimeSeriesfor training yourself. Each__getitem__must return aTorchTrainingSample. We suggest to have a look at this submodule
to see examples of how to do it.
Example Benchmark
As an example, we show here the time required to train one epoch on the first 80% of the energy dataset (darts.datasets.EnergyDataset), which consists of one multivariate series that is 28050 timesteps long and has 28 dimensions.
We train two models; NBEATSModel and TFTModel, with default parameters and input_chunk_length=48 and output_chunk_length=12 (which results in 27991 training samples with default sequential training datasets). For the TFT model, we also set the parameter add_cyclic_encoder='hour'. The tests are made on a Intel CPU i9-10900K CPU @ 3.70GHz, with an Nvidia RTX 2080s GPU, 32 GB of RAM. All TimeSeries are pre-loaded in memory and given to the models as a list.
| Model | Dataset | dtype | CUDA | Batch size | num workers | time per epoch |
|---|---|---|---|---|---|---|
NBEATSModel |
Energy | 64 | no | 32 | 0 | 283s |
NBEATSModel |
Energy | 64 | no | 32 | 2 | 285s |
NBEATSModel |
Energy | 64 | no | 32 | 4 | 282s |
NBEATSModel |
Energy | 64 | no | 1024 | 0 | 58s |
NBEATSModel |
Energy | 64 | no | 1024 | 2 | 57s |
NBEATSModel |
Energy | 64 | no | 1024 | 4 | 58s |
NBEATSModel |
Energy | 64 | yes | 32 | 0 | 63s |
NBEATSModel |
Energy | 64 | yes | 32 | 2 | 62s |
NBEATSModel |
Energy | 64 | yes | 1024 | 0 | 13.3s |
NBEATSModel |
Energy | 64 | yes | 1024 | 2 | 12.1s |
NBEATSModel |
Energy | 64 | yes | 1024 | 4 | 12.3s |
NBEATSModel |
Energy | 32 | no | 32 | 0 | 117s |
NBEATSModel |
Energy | 32 | no | 32 | 2 | 115s |
NBEATSModel |
Energy | 32 | no | 32 | 4 | 117s |
NBEATSModel |
Energy | 32 | no | 1024 | 0 | 28.4s |
NBEATSModel |
Energy | 32 | no | 1024 | 2 | 27.4s |
NBEATSModel |
Energy | 32 | no | 1024 | 4 | 27.5s |
NBEATSModel |
Energy | 32 | yes | 32 | 0 | 41.5s |
NBEATSModel |
Energy | 32 | yes | 32 | 2 | 40.6s |
NBEATSModel |
Energy | 32 | yes | 1024 | 0 | 2.8s |
NBEATSModel |
Energy | 32 | yes | 1024 | 2 | 1.65 |
NBEATSModel |
Energy | 32 | yes | 1024 | 4 | 1.8s |
TFTModel |
Energy | 64 | no | 32 | 0 | 78s |
TFTModel |
Energy | 64 | no | 32 | 2 | 72s |
TFTModel |
Energy | 64 | no | 32 | 4 | 72s |
TFTModel |
Energy | 64 | no | 1024 | 0 | 46s |
TFTModel |
Energy | 64 | no | 1024 | 2 | 38s |
TFTModel |
Energy | 64 | no | 1024 | 4 | 39s |
TFTModel |
Energy | 64 | yes | 32 | 0 | 125s |
TFTModel |
Energy | 64 | yes | 32 | 2 | 115s |
TFTModel |
Energy | 64 | yes | 1024 | 0 | 59s |
TFTModel |
Energy | 64 | yes | 1024 | 2 | 50s |
TFTModel |
Energy | 64 | yes | 1024 | 4 | 50s |
TFTModel |
Energy | 32 | no | 32 | 0 | 70s |
TFTModel |
Energy | 32 | no | 32 | 2 | 62.6s |
TFTModel |
Energy | 32 | no | 32 | 4 | 63.6 |
TFTModel |
Energy | 32 | no | 1024 | 0 | 31.9s |
TFTModel |
Energy | 32 | no | 1024 | 2 | 45s |
TFTModel |
Energy | 32 | no | 1024 | 4 | 44s |
TFTModel |
Energy | 32 | yes | 32 | 0 | 73s |
TFTModel |
Energy | 32 | yes | 32 | 2 | 58s |
TFTModel |
Energy | 32 | yes | 1024 | 0 | 41s |
TFTModel |
Energy | 32 | yes | 1024 | 2 | 31s |
TFTModel |
Energy | 32 | yes | 1024 | 4 | 31s |
--- METRICS ---
- Files Extracted: 12
- Estimated Token Budget: ~57339 tokens
- Canonical Reference: https://codewiki.google/github.com/unit8co/darts