Technical Documentation: facebook/prophet
ℹ️ Provenance: Multi-Tier Ingestion · CodeWiki Reference
Overview (`README.md`)
Prophet: Automatic Forecasting Procedure
2026 Update: Prophet is in maintenance mode as of v1.4.0. Only bug fixes, dependency bumps, and changes to the R package to meet parity with Python will be accepted. No new features are planned.
2023 Update: We discuss our plans for the future of Prophet in this blog post: facebook/prophet in 2023 and beyond
Prophet is a procedure for forecasting time series data based on an additive model where non-linear trends are fit with yearly, weekly, and daily seasonality, plus holiday effects. It works best with time series that have strong seasonal effects and several seasons of historical data. Prophet is robust to missing data and shifts in the trend, and typically handles outliers well.
Prophet is open source software released by Facebook's Core Data Science team. It is available for download on CRAN and PyPI.
Important links
- Homepage: https://facebook.github.io/prophet/
- HTML documentation: https://facebook.github.io/prophet/docs/quick_start.html
- Issue tracker: https://github.com/facebook/prophet/issues
- Source code repository: https://github.com/facebook/prophet
- Contributing: https://facebook.github.io/prophet/docs/contributing.html
- Prophet R package: https://cran.r-project.org/package=prophet
- Prophet Python package: https://pypi.python.org/pypi/prophet/
- Release blogpost: https://research.facebook.com/blog/2017/2/prophet-forecasting-at-scale/
- Prophet paper: Sean J. Taylor, Benjamin Letham (2018) Forecasting at scale. The American Statistician 72(1):37-45 (https://peerj.com/preprints/3190.pdf).
- Helter Skelter, simplified Prophet reimplementation, written in Scala/Apache Spark, optimized for distributed computing: https://github.com/AmadeusITGroup/HelterSkelter
Installation in R - CRAN
⚠️ The CRAN version of prophet is fairly outdated. To get the latest bug fixes and updated country holiday data, we suggest installing the latest release.
Prophet is a CRAN package so you can use install.packages.
install.packages('prophet')After installation, you can get started!
Installation in R - Latest release
install.packages('remotes')
remotes::install_github('facebook/prophet@*release', subdir = 'R')Experimental backend - cmdstanr
You can also choose an experimental alternative stan backend called cmdstanr. Once you've installed prophet,
follow these instructions to use cmdstanr instead of rstan as the backend:
# R
# We recommend running this in a fresh R session or restarting your current session
install.packages(c("cmdstanr", "posterior"), repos = c("https://stan-dev.r-universe.dev", getOption("repos")))
# If you haven't installed cmdstan before, run:
cmdstanr::install_cmdstan()
# Otherwise, you can point cmdstanr to your cmdstan path:
cmdstanr::set_cmdstan_path(path = <your existing cmdstan>)
# Set the R_STAN_BACKEND environment variable
Sys.setenv(R_STAN_BACKEND = "CMDSTANR")Windows
On Windows, R requires a compiler so you'll need to follow the instructions provided by rstan. The key step is installing Rtools before attempting to install the package.
If you have custom Stan compiler settings, install from source rather than the CRAN binary.
Installation in Python - PyPI release
Prophet is on PyPI, so you can use pip to install it.
python -m pip install prophet- From v0.6 onwards, Python 2 is no longer supported.
- As of v1.0, the package name on PyPI is "prophet"; prior to v1.0 it was "fbprophet".
- As of v1.1, the minimum supported Python version is 3.7.
After installation, you can get started!
Anaconda
Prophet can also be installed through conda-forge.
conda install -c conda-forge prophetInstallation in Python - Development version
To get the latest code changes as they are merged, you can clone this repo and build from source manually. This is not guaranteed to be stable.
git clone https://github.com/facebook/prophet.git
cd prophet/python
python -m pip install -e .By default, Prophet will use a fixed version of cmdstan (downloading and installing it if necessary) to compile the model executables. If this is undesired and you would like to use your own existing cmdstan installation, you can set the environment variable PROPHET_REPACKAGE_CMDSTAN to False:
export PROPHET_REPACKAGE_CMDSTAN=False; python -m pip install -e .Linux
Make sure compilers (gcc, g++, build-essential) and Python development tools (python-dev, python3-dev) are installed. In Red Hat systems, install the packages gcc64 and gcc64-c++. If you are using a VM, be aware that you will need at least 4GB of memory to install prophet, and at least 2GB of memory to use prophet.
Windows
Using cmdstanpy with Windows requires a Unix-compatible C compiler such as mingw-gcc. If cmdstanpy is installed first, one can be installed via the cmdstanpy.install_cxx_toolchain command.
Apache Spark
Helter Skelter is a simplified reimplementation of Prophet's core forecasting approach, written natively in Scala and designed to run on Apache Spark. The algorithm is optimized for the JVM ecosystem and was built from the ground up for large-scale distributed computing.
Helter Skelter is developed and maintained by Amadeus.
GitHub repository: Helter Skelter
Maven Central artifacts:
- Scala 2.12: helter-skelter_2.12
- Scala 2.13: helter-skelter_2.13
Changelog
See Release Notes.
Version 1.5.0 (2026.10.05)
Python
- Fixed
cross_validatewith shared seasonality conditions and ensured its executor pools are closed. - Fixed
performance_metricsmutating themetricslist passed by the caller. - Fixed MAP parameter shapes when the history is constant.
- Moved
performance_metricsand the built-in metrics into a newprophet.metricsmodule to resolve a circular import. They remain importable fromprophet.diagnostics. - Added CPython 3.14 wheel builds and refreshed the macOS CI runners.
Version 1.4.0 (2026.08.01)
Python
- Supported nested Prophet models for extra regressors.
- Publish (static) API docs.
Version 1.3.0 (2026.01.27)
Python
- Support pandas>=3.0 and numpy>=2.4.
Version 1.2.2 (2026.01.25)
Python
- Version constraints on pandas (
<3) and numpy (<2.4).
R
- Update build requirements to C++17 to Comply with CRAN Policy.
- Add .tar.gz upload for R package to CI.
- Re-generated holidays.csv for R package.
Version 1.2.1 (2025.10.22)
Python
- Also copy makefile to fake cmdstan.
Version 1.2.0 (2025.05.30)
Python
- Use latest CmdStan.
- Add null check to CmdStanPyBackend cleanup() function.
Version 1.1.7 (2025.05.30)
Python
- Enable creation of custom performance metrics.
- chore: address pandas futurewarning from "M" being deprecated.
- cleanup() for cross_validate.
Version 1.1.6 (2024.09.29)
Python
- Bug fixes: include predictions for dates with missing
ythe history, zero division error in cross validation metrics. - Changed
NDArray[np.float_]toNDArray[np.float64]to be compatible with numpy 2.0
R
- Updated
holidaysdata based on holidays version 0.57.
Version 1.1.5 (2023.10.10)
Python
- Upgraded cmdstan version to 2.33.1, enabling Apple M2 support.
- Added pre-built wheels for macOS arm64 architecture (M1, M2 chips)
- Added argument
scalingto theProphet()instantiation. Allowsminmaxscaling onyinstead ofabsmaxscaling (dividing by the maximum value).scaling='absmax'by default, preserving the
behaviour of previous versions. - Added argument
holidays_modeto theProphet()instantiation. Allows holidays regressors to have
a different mode than seasonality regressors.holidays_modetakes the same value asseasonality_mode
if not specified, preserving the behaviour of previous versions. - Added two methods to the
Prophetobject:preprocess()andcalculate_initial_params(). These
do not need to be called and will not change the model fitting process. Their purpose is to provide
clarity on the pre-processing steps taken (yscaling, creating fourier series, regressor scaling,
setting changepoints, etc.) before the data is passed to the stan model. - Added argument
extra_output_columnstocross_validation(). The user can specify additional columns
frompredict()to include in the final output alongsidedsandyhat, for exampleextra_output_columns=['trend']. - prophet's custom
hdaysmodule was deprecated last version and is now removed.
R
- Updated
holidaysdata based on holidays version 0.34.
Version 1.1.4 (2023.05.30)
Python
- We now rely solely on
holidayspackage for country holidays. - Upgraded cmdstan version to 2.31.0, enabling Apple M1 support.
- Fixed bug with Windows installation caused by long paths.
R
- Updated
holidaysdata based on holidays version 0.25.
Version 1.1.2 (2023.01.20)
Python
- Sped up
.predict()by up to 10x by removing intermediate DataFrame creations. - Sped up fourier series generation, leading to at least 1.5x speed improvement for
train()andpredict()pipelines. - Fixed bug in how warm start values were being read.
- Wheels are now version-agnostic.
R
- Fixed a bug in
construct_holiday_dataframe() - Updated
holidaysdata based on holidays version 0.18.
Version 1.1.1 (2022.09.08)
- (Python) Improved runtime (3-7x) of uncertainty predictions via vectorization.
- Bugfixes relating to Python package versions and R holiday objects.
Version 1.1 (2022.06.25)
- Replaced
pystan2dependency withcmdstan+cmdstanpy. - Pre-packaged model binaries for Python package, uploaded binary distributions to PyPI.
- Improvements in the
stanmodel code, cross-validation metric calculations, holidays.
Version 1.0 (2021.03.28)
- Python package name changed from fbprophet to prophet
- Fixed R Windows build issues to get latest version back on CRAN
- Improvements in serialization, holidays, and R timezone handling
- Plotting improvements
Version 0.7 (2020.09.05)
- Built-in json serialization
- Added "flat" growth option
- Bugfixes related to
holidaysandpandas - Plotting improvements
- Improvements in cross validation, such as parallelization and directly specifying cutoffs
Version 0.6 (2020.03.03)
- Fix bugs related to upstream changes in
holidaysandpandaspackages. - Compile model during first use, not during install (to comply with CRAN policy)
cmdstanpybackend now available in Python- Python 2 no longer supported
Version 0.5 (2019.05.14)
- Conditional seasonalities
- Improved cross validation estimates
- Plotly plot in Python
- Bugfixes
Version 0.4 (2018.12.18)
- Added holidays functionality
- Bugfixes
Version 0.3 (2018.06.01)
- Multiplicative seasonality
- Cross validation error metrics and visualizations
- Parameter to set range of potential changepoints
- Unified Stan model for both trend types
- Improved future trend uncertainty for sub-daily data
- Bugfixes
Version 0.2.1 (2017.11.08)
- Bugfixes
Version 0.2 (2017.09.02)
- Forecasting with sub-daily data
- Daily seasonality, and custom seasonalities
- Extra regressors
- Access to posterior predictive samples
- Cross-validation function
- Saturating minimums
- Bugfixes
Version 0.1.1 (2017.04.17)
- Bugfixes
- New options for detecting yearly and weekly seasonality (now the default)
Version 0.1 (2017.02.23)
- Initial release
License
Prophet is licensed under the MIT license.
Overview (`docs/README.md`)
User Documentation for Prophet
This directory will contain the user and feature documentation for Prophet. The documentation will be hosted on GitHub pages.
Contributing
See CONTRIBUTING.md for details on how to add or modify content.
Jupyter Notebooks
Most of the user-facing doc pages are generated from Jupyter notebooks in the notebooks directory at the base of the source tree. Please make changes there and then rebuild the docs:
$ cd docs
$ make notebooksAPI Reference
The Python API reference is generated with pdoc directly
from the package source. It includes the public modules, classes, methods, and
their docstrings, with cross-links between documented objects.
Install the Python development dependencies, then either generate the static
reference for inclusion in the Jekyll site or serve it directly for a demo:
$ cd python
$ uv pip install --python .venv/bin/python -e ".[dev]"
$ cd ../docs
# Generate docs/api/; `bundle exec jekyll serve` will serve it at /prophet/api/
$ make api
# Serve only the API reference at http://localhost:8080/
$ make serve-apidocs/api/ is generated and committed. Run make api and include the
updated files in the same change whenever the public Python API or its
docstrings change. The existing GitHub Pages build will publish those static
files with the rest of docs/ when the change reaches main.
Run the Site Locally
The requirements for running a GitHub pages site locally is described in GitHub help. The steps below summarize these steps.
If you have run the site before, you can start with step 1 and then move on to step 5.
Ensure that you are in the same directory where this
README.mdexists (e.g., it could be in/docsonmain, in the root of agh-pagesbranch, etc). The below RubyGems commands, etc must be run from there.Make sure you have Ruby and RubyGems installed.
Ruby >= 2.2 is required for the gems. On the latest versions of Mac OS X, Ruby 2.0 is the
default. Use Homebrew and thebrew install rubycommand (or your
preferred upgrade mechanism) to install a newer version of Ruby for your Mac OS X system.Make sure you have Bundler installed.
text# may require sudo gem install bundlerInstall the project's dependencies.
text# run this in the 'docs' directory bundle installIf you get an error when installing
nokogiri, you may be running into the problem described
in this nokogiri issue. You can
eitherbrew uninstall xz(and thenbrew install xzafter the bundle is installed) orxcode-select --install(although this may not work if you have already installed command
line tools).Run Jekyll's server.
- On first run or for structural changes to the documentation (e.g., new sidebar menu item), do a full build.
textbundle exec jekyll serve- For content changes only, you can use
--incrementalfor faster builds.
textbundle exec jekyll serve --incrementalWe use
bundle execinstead of running straightjekyllbecausebundle execwill always use the version of Jekyll from ourGemfile. Just runningjekyllwill use the system version and may not necessarily be compatible.- To run using an actual IP address, you can use
--host=0.0.0.0
textbundle exec jekyll serve --host=0.0.0.0This will allow you to use the IP address associated with your machine in the URL. That way you could share it with other people.
e.g., on a Mac, you can your IP address with something like
ifconfig | grep "inet " | grep -v 127.0.0.1.Either of commands in the previous step will serve up the site on your local device at http://127.0.0.1:4000/ or http://localhost:4000.
Updating the Bundle
The site depends on Github Pages and the installed bundle is based on the github-pages gem.
Occasionally that gem might get updated with new or changed functionality. If that is the case,
you can run:
bundle updateto get the latest packages for the installation.
Overview (`python/README.md`)
Prophet: Automatic Forecasting Procedure
Prophet is a procedure for forecasting time series data based on an additive model where non-linear trends are fit with yearly, weekly, and daily seasonality, plus holiday effects. It works best with time series that have strong seasonal effects and several seasons of historical data. Prophet is robust to missing data and shifts in the trend, and typically handles outliers well.
Prophet is open source software released by Facebook's Core Data Science team .
Full documentation and examples available at the homepage: https://facebook.github.io/prophet/
Important links
- HTML documentation: https://facebook.github.io/prophet/docs/quick_start.html
- Issue tracker: https://github.com/facebook/prophet/issues
- Source code repository: https://github.com/facebook/prophet
- Implementation of Prophet in R: https://cran.r-project.org/package=prophet
Other forecasting packages
- Rob Hyndman's forecast package
- Statsmodels
Installation - PyPI release
See Installation in Python - PyPI release
Installation - Development version
See Installation in Python - Development version
Installation using Docker and docker-compose (via Makefile)
Simply type make build and if everything is fine you should be able to make shell or alternative jump directly to make py-shell.
To run the tests, inside the container cd python/prophet and then python -m pytest prophet/tests/
Example usage
>>> from prophet import Prophet
>>> m = Prophet()
>>> m.fit(df) # df is a pandas.DataFrame with 'y' and 'ds' columns
>>> future = m.make_future_dataframe(periods=365)
>>> m.predict(future)Overview (`python_shim/README.md`)
Prophet: Automatic Forecasting Procedure
As of v1.0, Prophet has moved to use the name "prophet" on PyPI and not the original name of "fbprophet". This package is now just a shim for using the prophet package. Please change references in your code to use "prophet" instead of "fbprophet".
See https://facebook.github.io/prophet/ for full documentation.
Index (`docs/index.md`)
layout: home title: Prophet id: home
Watch Introductory Video
Prophet is a procedure for forecasting time series data based on an additive model where non-linear trends are fit with yearly, weekly, and daily seasonality, plus holiday effects. It works best with time series that have strong seasonal effects and several seasons of historical data. Prophet is robust to missing data and shifts in the trend, and typically handles outliers well.
Prophet is open source software released by Facebook's Core Data Science team. It is available for download on CRAN and PyPI.
2023 Update: We discuss our plans for the future of Prophet in this blog post: facebook/prophet in 2023 and beyond
{% include content/gridblocks.html data_source=site.data.features layout="fourColumn" align="left" %}
CONTRIBUTING (`docs/CONTRIBUTING.md`)
This provides guidance on how to contribute various content to the Prophet documentation.
Getting started
You should only have to do these one time.
- Rename this file to
CONTRIBUTING.md. - Rename
EXAMPLE-README-FOR-RUNNING-DOCS.mdtoREADME.md(replacing the existingREADME.mdthat came with the template). - Rename
EXAMPLE-LICENSEtoLICENSE. - Review the template information.
- Review
./_config.yml. - Make sure you update
title,description,taglineandgacode(Google Analytics) in./_config.yml.
Basic Structure
Most content is written in markdown. You name the file something.md, then have a header that looks like this:
---
docid: getting-started
title: Getting started with ProjectName
layout: docs
permalink: /docs/getting-started.html
---Customize these values for each document, blog post, etc.
The filename of the
.mdfile doesn't actually matter; what is important is thedocidbeing unique and thepermalinkcorrect and unique too).
Landing page
Modify index.md with your new or updated content.
If you want a GridBlock as part of your content, you can do so directly with HTML:
<div class="gridBlock">
<div class="blockElement twoByGridBlock alignLeft">
<div class="blockContent">
<h3>Your Features</h3>
<ul>
<li>The <a href="http://example.org/">Example</a></li>
<li><a href="http://example.com">Another Example</a></li>
</ul>
</div>
</div>
<div class="blockElement twoByGridBlock alignLeft">
<div class="blockContent">
<h3>More information</h3>
<p>
Stuff here
</p>
</div>
</div>
</div>or with a combination of changing ./_data/features.yml and adding some Liquid to index.md, such as:
{% include content/gridblocks.html data_source=site.data.features imagealign="bottom"%}Blog
To modify a blog post, edit the appropriate markdown file in ./_posts/.
Adding a new blog post is a four-step process.
Some posts have a
permalinkandcommentsin the blog post YAML header. You will not need these for new blog posts. These are an artifact of migrating the blog from Wordpress to gh-pages.
- Create your blog post in
./_posts/in markdown (file extension.mdor.markdown). See current posts in that folder or./doc-type-examples/2016-04-07-blog-post-example.mdfor an example of the YAML format. If the./_postsdirectory does not exist, create it.
- You can add a `` tag in the middle of your post such that you show only the excerpt above that tag in the main
/blogindex on your page.
- If you have not authored a blog post before, modify the
./_data/authors.ymlfile with theauthorid you used in your blog post, along with your full name and Facebook ID to get your profile picture. - Run the site locally to test your changes. It will be at
http://127.0.0.1/blog/your-new-blog-post-title.html - Push your changes to GitHub.
Docs
To modify docs, edit the appropriate markdown file in ./_docs/.
To add docs to the site....
- Add your markdown file to the
./_docs/folder. See./doc-type-examples/docs-hello-world.mdfor an example of the YAML header format. If the./_docs/directory does not exist, create it.
- You can use folders in the
./_docs/directory to organize your content if you want.
- Update
_data/nav_docs.ymlto add your new document to the navigation bar. Use thedocidyou put in your doc markdown in as theidin the_data/nav_docs.ymlfile. - Run the site locally to test your changes. It will be at
http://127.0.0.1/docs/your-new-doc-permalink.html - Push your changes to GitHub.
Header Bar
To modify the header bar, change ./_data/nav.yml.
Top Level Page
To modify a top-level page, edit the appropriate markdown file in ./top-level/
If you want a top-level page (e.g., http://your-site.com/top-level.html) -- not in /blog/ or /docs/....
Create a markdown file in the root
./top-level/. See./doc-type-examples/top-level-example.mdfor more information.If you want a visible link to that file, update
_data/nav.ymlto add a link to your new top-level document in the header bar.This is not necessary if you just want to have a page that is linked to from another page, but not exposed as direct link to the user.
Run the site locally to test your changes. It will be at
http://127.0.0.1/your-top-level-page-permalink.htmlPush your changes to GitHub.
Code Blocks
This template uses rouge syntax highlighting out of the box. We have custom CSS for when those blocks are rendered. Rouge has plenty of supported languages.
Other Changes
- CSS:
./css/main.cssor./_sass/*.scss. - Images:
./static/images/[docs | posts]/.... - Main Blog post HTML:
./_includes/post.html - Main Docs HTML:
./_includes/doc.html
Config.Yml (`docs/_config.yml`)
Site settings
permalink: /blog/:year/:month/:day/:title.html
title: Prophet
tagline: Forecasting at scale.
Prophet is a forecasting procedure implemented in R and Python. It is fast and provides completely automated forecasts that can be tuned by hand by data scientists and analysts.
fbappid: "1615782811974223"
gacode: "UA-44373548-26"
logo: /static/logo.svg
baseurl determines the subpath of your site. For example if you're using an
organisation.github.io/reponame/ basic site URL, then baseurl would be set
as "/reponame" but either set to "" or remove it all together if you have
a top-level domain URL as it is now set to "" by default as discussed in:
http://jekyllrb.com/news/2016/10/06/jekyll-3-3-is-here/
baseurl: "/prophet"
the base hostname & protocol for your site
If baseurl is set, then the absolute url for your site would be url/baseurl
This was also be set to the right thing automatically for local development
https://github.com/blog/2277-what-s-new-in-github-pages-with-jekyll-3-3
http://jekyllrb.com/news/2016/10/06/jekyll-3-3-is-here/
url: "http://facebook.github.io"
Note: There are new filters in Jekyll 3.3 to help with absolute and relative urls
absolute_url
relative_url
So you will see these used throughout the Jekyll code in this template.
no more need for | prepend: site.url | prepend: site.baseurl
http://jekyllrb.com/news/2016/10/06/jekyll-3-3-is-here/
https://github.com/blog/2277-what-s-new-in-github-pages-with-jekyll-3-3
The GitHub repo for your project
ghrepo: "facebook/prophet"
Use these color settings to determine your colour scheme for the site.
color:
primary should be a vivid color that reflects the project's brand
primary: "#3b5998"
secondary should be a subtle light or dark color used on page backgrounds
secondary: "#f9f9f9"
Use the following to specify whether the previous two colours are 'light'
or 'dark' and therefore what colors can be overlaid on them
primary-overlay: "dark"
secondary-overlay: "light"
#Uncomment this if you want to enable Algolia doc search with your own values
#searchconfig:
apikey: ""
indexname: ""
Blog posts are built into Jekyll by default, with the `_posts` directory.
Here you can specify other types of documentation. The names here are `docs`
and `top-level`. This means their content will be in `_docs` and `_top-level`.
The permalink format is also given.
http://ben.balter.com/2015/02/20/jekyll-collections/
collections:
docs:
output: true
permalink: /docs/:name/
DO NOT ADJUST BELOW THIS LINE UNLESS YOU KNOW WHAT YOU ARE CHANGING
markdown: kramdown
kramdown:
input: GFM
syntax_highlighter: rouge
syntax_highlighter_opts:
css_class: 'rougeHighlight'
span:
line_numbers: false
block:
line_numbers: true
start_line: 1
sass:
style: :compressed
redcarpet:
extensions: [with_toc_data]
Gems
gems:
- jekyll-feed
- jekyll-seo-tag
- jekyll-sitemap
Set default open graph image for all pages
defaults:
scope:
path: ""
values:
image: /static/og_image.pngData Authors.Yml (`docs/_data/authors.yml`)
sjt:
full_name: Sean J. Taylor
fbid: 605889
bletham:
full_name: Ben Letham
fbid: 605889
Data Features.Yml (`docs/_data/features.yml`)
title: Accurate and fast.
text: |
Prophet is used in many applications across Facebook for producing reliable forecasts for planning and goal setting. We've found it to perform better than any other approach in the majority of cases. We fit models in Stan so that you get forecasts in just a few seconds.
align: lefttitle: Fully automatic.
text: |
Get a reasonable forecast on messy data with no manual effort. Prophet is robust to outliers, missing data, and dramatic changes in your time series.title: Tunable forecasts.
text: |
The Prophet procedure includes many possibilities for users to tweak and adjust forecasts. You can use human-interpretable parameters to improve your forecast by adding your domain knowledge.title: Available in R or Python.
text: |
We've implemented the Prophet procedure in R and Python, but they share the same underlying Stan code for fitting. Use whatever language you're comfortable with to get forecasts.
Data Nav.Yml (`docs/_data/nav.yml`)
title: Docs
href: docs/
category: docstitle: API Reference
href: api/
category: docstitle: GitHub
href: https://github.com/facebook/prophet
category: external
Data Nav Docs.Yml (`docs/_data/nav_docs.yml`)
- title: Documentation
items:- id: installation
- id: quick_start
- id: saturating_forecasts
- id: trend_changepoints
- id: seasonality,_holiday_effects,_and_regressors
- id: multiplicative_seasonality
- id: uncertainty_intervals
- id: outliers
- id: non-daily_data
- id: diagnostics
- id: handling_shocks
- id: additional_topics
- id: contributing
n title:, 1 items: per title:, n id: per items:
- title: A
items:
- id: B
- id: C
- title: D
- id: E
Data Promo.Yml (`docs/_data/promo.yml`)
This file determines the list of promotional elements added to the header of \
your site's homepage. Full list of plugins are shown
- type: plugin_row
children:- type: button
href: docs/installation.html
text: Install Prophet - type: button
href: docs/quick_start.html#r-api
text: Get started in R - type: button
href: docs/quick_start.html#python-api
text: Get started in Python - type: button
href: https://peerj.com/preprints/3190/
text: Read the paper
- type: button
Docs Additional Topics (`docs/_docs/additional_topics.md`)
layout: docs
docid: "additional_topics"
title: "Additional Topics"
permalink: /docs/additional_topics.html
subsections:
- title: Saving models
id: saving-models - title: Flat trend
id: flat-trend - title: Custom trends
id: custom-trends - title: Updating fitted models
id: updating-fitted-models - title: minmax scaling (new in 1.1.5)
id: minmax-scaling-(new-in-1.1.5) - title: Inspecting transformed data (new in 1.1.5)
id: inspecting-transformed-data-(new-in-1.1.5) - title: External references
id: external-references
Saving models
It is possible to save fitted Prophet models so that they can be loaded and used later.
In R, this is done with saveRDS and readRDS:
# R
saveRDS(m, file="model.RDS") # Save model
m <- readRDS(file="model.RDS") # Load modelIn Python, models should not be saved with pickle; the Stan backend attached to the model object will not pickle well, and will produce issues under certain versions of Python. Instead, you should use the built-in serialization functions to serialize the model to json:
# Python
from prophet.serialize import model_to_json, model_from_json
with open('serialized_model.json', 'w') as fout:
fout.write(model_to_json(m)) # Save model
with open('serialized_model.json', 'r') as fin:
m = model_from_json(fin.read()) # Load modelThe json file will be portable across systems, and deserialization is backwards compatible with older versions of prophet.
Flat trend
For time series that exhibit strong seasonality patterns rather than trend changes, or when we want to rely on the pattern of exogenous regressors (e.g. for causal inference with time series), it may be useful to force the trend growth rate to be flat. This can be achieved simply by passing growth='flat' when creating the model:
# R
m <- prophet(df, growth='flat')# Python
m = Prophet(growth='flat')Below is a comparison of counterfactual forecasting with exogenous regressors using linear versus flat growth.
# Python
import pandas as pd
from prophet import Prophet
import matplotlib.pyplot as plt
import warnings
warnings.simplefilter(action='ignore', category=FutureWarning)
regressor = "location_4"
target = "location_41"
cutoff = pd.to_datetime("2023-04-17 00:00:00")
df = (
pd.read_csv(
"https://raw.githubusercontent.com/facebook/prophet/main/examples/example_pedestrians_multivariate.csv",
parse_dates=["ds"]
)
.rename(columns={target: "y"})
)
train = df.loc[df["ds"] < cutoff]
test = df.loc[df["ds"] >= cutoff]# Python
def fit_model(growth):
m = Prophet(growth=growth, seasonality_mode="multiplicative", daily_seasonality=15)
m.add_regressor("location_4", mode="multiplicative")
m.fit(train)
preds = pd.merge(
test,
m.predict(test),
on="ds",
how="inner"
)
mape = ((preds["yhat"] - preds["y"]).abs() / preds_linear["y"]).mean()
return m, preds, mape# Python
m_linear, preds_linear, mape_linear = fit_model("linear")01:19:58 - cmdstanpy - INFO - Chain [1] start processing
01:19:58 - cmdstanpy - INFO - Chain [1] done processing# Python
m_flat, preds_flat, mape_flat = fit_model("flat")01:19:58 - cmdstanpy - INFO - Chain [1] start processing
01:19:58 - cmdstanpy - INFO - Chain [1] done processing# Python
m_linear.plot_components(preds_linear);# Python
m_flat.plot_components(preds_flat);# Python
fig, ax = plt.subplots(figsize=(11, 5))
ax.scatter(preds_linear["ds"], preds_linear["y"], color="black", marker=".")
ax.plot(preds_linear["ds"], preds_linear["yhat"], label=f"linear, mape={mape_linear:.1%}")
ax.plot(preds_flat["ds"], preds_flat["yhat"], label=f"flat, mape={mape_flat:.1%}")
plt.xticks(rotation=60)
ax.legend();In this example, the target sensor values can be mostly explained by the exogenous regressor (a nearby sensor). The model with linear growth assumes an increasing trend and this leads to larger and larger over-predictions in the test period, while the flat growth model mostly follows movements in the exogenous regressor and this results in a sizeable MAPE improvement.
Note that forecasting with exogenous regressors is only effective when we can be confident in the future values of the regressor. The example above is relevant to causal inference using time series, where we want to understand what y would have looked like for a past time period, and hence the exogenous regressor values are known.
In other cases -- where we don't have exogenous regressors or have to predict their future values -- if flat growth is used on a time series that doesn't have a constant trend, any trend will be fit with the noise term and so there will be high predictive uncertainty in the forecast.
Custom trends
To use a trend besides these three built-in trend functions (piecewise linear, piecewise logistic growth, and flat), you can download the source code from github, modify the trend function as desired in a local branch, and then install that local version. This PR provides a good illustration of what must be done to implement a custom trend, as does this one that implements a step function trend and this one for a new trend in R.
Updating fitted models
A common setting for forecasting is fitting models that need to be updated as additional data come in. Prophet models can only be fit once, and a new model must be re-fit when new data become available. In most settings, model fitting is fast enough that there isn't any issue with re-fitting from scratch. However, it is possible to speed things up a little by warm-starting the fit from the model parameters of the earlier model. This code example shows how this can be done in Python:
# Python
def warm_start_params(m):
"""
Retrieve parameters from a trained model in the format used to initialize a new Stan model.
Note that the new Stan model must have these same settings:
n_changepoints, seasonality features, mcmc sampling
for the retrieved parameters to be valid for the new model.
Parameters
----------
m: A trained model of the Prophet class.
Returns
-------
A Dictionary containing retrieved parameters of m.
"""
res = {}
for pname in ['k', 'm', 'sigma_obs']:
if m.mcmc_samples == 0:
res[pname] = m.params[pname][0][0]
else:
res[pname] = np.mean(m.params[pname])
for pname in ['delta', 'beta']:
if m.mcmc_samples == 0:
res[pname] = m.params[pname][0]
else:
res[pname] = np.mean(m.params[pname], axis=0)
return res
df = pd.read_csv('https://raw.githubusercontent.com/facebook/prophet/main/examples/example_wp_log_peyton_manning.csv')
df1 = df.loc[df['ds'] < '2016-01-19', :] # All data except the last day
m1 = Prophet().fit(df1) # A model fit to all data except the last day
%timeit m2 = Prophet().fit(df) # Adding the last day, fitting from scratch
%timeit m2 = Prophet().fit(df, init=warm_start_params(m1)) # Adding the last day, warm-starting from m11.33 s ± 55.9 ms per loop (mean ± std. dev. of 7 runs, 1 loop each)
185 ms ± 4.46 ms per loop (mean ± std. dev. of 7 runs, 10 loops each)As can be seen, the parameters from the previous model are passed in to the fitting for the next with the kwarg init. In this case, model fitting was about 5x faster when using warm starting. The speedup will generally depend on how much the optimal model parameters have changed with the addition of the new data.
There are few caveats that should be kept in mind when considering warm-starting. First, warm-starting may work well for small updates to the data (like the addition of one day in the example above) but can be worse than fitting from scratch if there are large changes to the data (i.e., a lot of days have been added). This is because when a large amount of history is added, the location of the changepoints will be very different between the two models, and so the parameters from the previous model may actually produce a bad trend initialization. Second, as a detail, the number of changepoints need to be consistent from one model to the next or else an error will be raised because the changepoint prior parameter delta will be the wrong size.
minmax scaling (new in 1.1.5)
Before model fitting, Prophet scales y by dividing by the maximum value in the history. For datasets with very large y values, the scaled y values may be compressed to a very small range (i.e. [0.99999... - 1.0]), which causes a bad fit. This can be fixed by setting scaling='minmax' in the Prophet constructor.
# Python
large_y = pd.read_csv(
"https://raw.githubusercontent.com/facebook/prophet/main/python/prophet/tests/data3.csv",
parse_dates=["ds"]
)# Python
m1 = Prophet(scaling="absmax")
m1 = m1.fit(large_y)08:11:23 - cmdstanpy - INFO - Chain [1] start processing
08:11:23 - cmdstanpy - INFO - Chain [1] done processing# Python
m2 = Prophet(scaling="minmax")
m2 = m2.fit(large_y)08:11:29 - cmdstanpy - INFO - Chain [1] start processing
08:11:29 - cmdstanpy - INFO - Chain [1] done processing# Python
m1.plot(m1.predict(large_y));# Python
m2.plot(m2.predict(large_y));Inspecting transformed data (new in 1.1.5)
For debugging, it's useful to understand how the raw data has been transformed before being passed to the stan fit routine. We can call the .preprocess() method to see all the inputs to stan, and .calculate_init_params() to see how the model parameters will be initialized.
# Python
df = pd.read_csv('https://raw.githubusercontent.com/facebook/prophet/main/examples/example_wp_log_peyton_manning.csv')
m = Prophet()
transformed = m.preprocess(df)# Python
transformed.y.head(n=10)0 0.746552
1 0.663171
2 0.637023
3 0.628367
4 0.614441
5 0.605884
6 0.654956
7 0.687273
8 0.652501
9 0.628148
Name: y_scaled, dtype: float64# Python
transformed.X.head(n=10)| yearly_delim_1 | yearly_delim_2 | yearly_delim_3 | yearly_delim_4 | yearly_delim_5 | yearly_delim_6 | yearly_delim_7 | yearly_delim_8 | yearly_delim_9 | yearly_delim_10 | ... | yearly_delim_17 | yearly_delim_18 | yearly_delim_19 | yearly_delim_20 | weekly_delim_1 | weekly_delim_2 | weekly_delim_3 | weekly_delim_4 | weekly_delim_5 | weekly_delim_6 | |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| 0 | -0.377462 | 0.926025 | -0.699079 | 0.715044 | -0.917267 | 0.398272 | -0.999745 | 0.022576 | -0.934311 | -0.356460 | ... | 0.335276 | -0.942120 | 0.666089 | -0.745872 | -4.338837e-01 | -0.900969 | 7.818315e-01 | 0.623490 | -9.749279e-01 | -0.222521 |
| 1 | -0.361478 | 0.932381 | -0.674069 | 0.738668 | -0.895501 | 0.445059 | -0.995827 | 0.091261 | -0.961479 | -0.274879 | ... | 0.185987 | -0.982552 | 0.528581 | -0.848883 | -9.749279e-01 | -0.222521 | 4.338837e-01 | -0.900969 | 7.818315e-01 | 0.623490 |
| 2 | -0.345386 | 0.938461 | -0.648262 | 0.761418 | -0.871351 | 0.490660 | -0.987196 | 0.159513 | -0.981538 | -0.191266 | ... | 0.032249 | -0.999480 | 0.375470 | -0.926834 | -7.818315e-01 | 0.623490 | -9.749279e-01 | -0.222521 | -4.338837e-01 | -0.900969 |
| 3 | -0.329192 | 0.944263 | -0.621687 | 0.783266 | -0.844881 | 0.534955 | -0.973892 | 0.227011 | -0.994341 | -0.106239 | ... | -0.122261 | -0.992498 | 0.211276 | -0.977426 | 5.505235e-14 | 1.000000 | 1.101047e-13 | 1.000000 | 1.984146e-12 | 1.000000 |
| 4 | -0.312900 | 0.949786 | -0.594376 | 0.804187 | -0.816160 | 0.577825 | -0.955979 | 0.293434 | -0.999791 | -0.020426 | ... | -0.273845 | -0.961774 | 0.040844 | -0.999166 | 7.818315e-01 | 0.623490 | 9.749279e-01 | -0.222521 | 4.338837e-01 | -0.900969 |
| 5 | -0.296516 | 0.955028 | -0.566362 | 0.824157 | -0.785267 | 0.619157 | -0.933542 | 0.358468 | -0.997850 | 0.065537 | ... | -0.418879 | -0.908042 | -0.130793 | -0.991410 | 9.749279e-01 | -0.222521 | -4.338837e-01 | -0.900969 | -7.818315e-01 | 0.623490 |
| 6 | -0.280044 | 0.959987 | -0.537677 | 0.843151 | -0.752283 | 0.658840 | -0.906686 | 0.421806 | -0.988531 | 0.151016 | ... | -0.553893 | -0.832588 | -0.298569 | -0.954388 | 4.338837e-01 | -0.900969 | -7.818315e-01 | 0.623490 | 9.749279e-01 | -0.222521 |
| 7 | -0.263489 | 0.964662 | -0.508356 | 0.861147 | -0.717295 | 0.696769 | -0.875539 | 0.483147 | -0.971904 | 0.235379 | ... | -0.675656 | -0.737217 | -0.457531 | -0.889193 | -4.338837e-01 | -0.900969 | 7.818315e-01 | 0.623490 | -9.749279e-01 | -0.222521 |
| 8 | -0.246857 | 0.969052 | -0.478434 | 0.878124 | -0.680398 | 0.732843 | -0.840248 | 0.542202 | -0.948090 | 0.318001 | ... | -0.781257 | -0.624210 | -0.602988 | -0.797750 | -9.749279e-01 | -0.222521 | 4.338837e-01 | -0.900969 | 7.818315e-01 | 0.623490 |
| 9 | -0.230151 | 0.973155 | -0.447945 | 0.894061 | -0.641689 | 0.766965 | -0.800980 | 0.598691 | -0.917267 | 0.398272 | ... | -0.868168 | -0.496271 | -0.730644 | -0.682758 | -7.818315e-01 | 0.623490 | -9.749279e-01 | -0.222521 | -4.338837e-01 | -0.900969 |
10 rows × 26 columns
# Python
m.calculate_initial_params(num_total_regressors=transformed.K)ModelParams(k=-0.05444079622224118, m=0.7465517318876905, delta=array([0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0.,
0., 0., 0., 0., 0., 0., 0., 0.]), beta=array([0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0.,
0., 0., 0., 0., 0., 0., 0., 0., 0.]), sigma_obs=1.0)External references
As we discuss in our 2023 blog post on the state of Prophet, we have no plans to further develop the underlying Prophet model. If you're looking for state-of-the-art forecasting accuracy, we recommend the following libraries:
statsforecast, and other packages from the Nixtla group such ashierarchicalforecastandneuralforecast.NeuralProphet, a Prophet-style model implemented in PyTorch, to be more adaptable and extensible.
These github repositories provide examples of building on top of Prophet in ways that may be of broad interest:
- forecastr: A web app that provides a UI for Prophet.
Docs Contributing (`docs/_docs/contributing.md`)
layout: docs docid: "contributing" title: "Getting Help and Contributing" permalink: /docs/contributing.html
2023 Update: We discuss our plans for the future of Prophet in this blog post: facebook/prophet in 2023 and beyond
Prophet has a non-fixed release cycle but we will be making bugfixes in response to user feedback and adding features. Please let us know if you encounter a bug by filing an issue. Github issues is also the right place to ask questions about using Prophet.
We appreciate all contributions. If you are planning to contribute back bug-fixes, please do so without any further discussion.
If you plan to contribute new features or extensions to the core, please first open an issue and discuss the feature with us. Sending a pull request is fine too, but it will likely be merged more quickly if any design decisions are settled on beforehand in an issue.
We try to keep the R and Python versions feature identical, but new features can be implemented for each method in separate commits.
The following sections will describe how you can submit a pull request for adding enhancements, documentation changes or bug fixes to the codebase.
1. Forking the Prophet Repo
You will need your own fork to work on the code. Go to the prophet project
page and hit the Fork button. You will
want to clone your fork to your machine:
$ git clone https://github.com/your-user-name/prophet.git
$ cd prophet
$ git remote add upstream https://github.com/facebook/prophet.gitThis creates the directory prophet and connects your repository to
the upstream (main project) prophet repository.
2. Creating an environment with dependencies
Before starting any development, you'll need to create an isolated prophet
development environment. This should contain the required dependencies.
Python
- Install either Anaconda anaconda or miniconda
- Make sure your conda is up to date (
conda update conda) cdto the prophet source directory that you have cloned
$ cd python
# with Anaconda
$ conda create -n prophet
$ conda activate prophet
# with venv
$ python3 -m venv prophet
$ source prophet/bin/activateR
Dependencies can be managed through Packrat or renv.
For renv, you must first initialise a new project local environment.
> setwd("path/to/prophet/R") # set R subdirectory as working directory
> install.packages('renv')
> renv::init()This should also install the dependencies listed in the DESCRIPTION automatically. Any new R packages can be installed as they are needed in the project.
You can save the state of the project:
> renv::snapshot()or load the environment:
> renv::restore()3. Building a development version of Prophet
The next step is to build and install the development version of prophet in the environment you have just created.
Python
$ python -m pip install -e ".[dev,parallel]"You should be able to import prophet from your locally built version:
$ python # start an interpreter
>>> import prophet
>>> prophet.__version__
'1.1.2' # whatever the current github version isThis will create the new environment, and not touch any of your existing environments,
nor any existing Python installation.
# to view your environments:
$ conda info -e
# to return to your root environment::
$ conda deactivateSee the full conda docs here.
R
From the terminal, cd to R subdirectory and run:
$ R CMD INSTALL .This will build and install the local version of the prophet package. Then from the R console you can load the package: library(prophet).
4. Creating a branch
You want your main branch to reflect only production-ready code, so create a
feature branch for making your changes. For example:
$ git checkout -b new-featureThis changes your working directory to the new-feature branch. Keep any
changes in this branch specific to one bug or feature so it is clear
what the branch brings to prophet. You can have many "new-features"
and switch in between them using the git checkout command.
To update this branch, you need to retrieve the changes from the main branch:
$ git fetch upstream
$ git rebase upstream/mainThis will replay your commits on top of the latest prophet git main. If this
leads to merge conflicts, you must resolve these before submitting your pull
request. If you have uncommitted changes, you will need to git stash them
prior to updating. This will effectively store your changes and they can be
reapplied after updating.
5. Testing with Continuous Integration
Adding tests is one of the most common requests after code is pushed to prophet. Therefore, it is worth getting in the habit of writing tests ahead of time so this is never an issue. Once your pull request is submitted, the Github Actions CI (continuous integration) service automatically triggers Python and R builds for Prophet and runs the tests. A pull-request will be considered for merging when you have an all ‘green’ build. If any tests are failing, then you will get a red ‘X’, where you can click through to see the individual failed tests.
Python
Prophet uses the pytest package for running tests in Python and testthat package for testing in R. All tests should go into the tests subdirectory in either the Python or R folders.
The entire test suite can be run by typing:
$ python -m pytest prophet/tests/R
The entire test suite can be run from the R console by installing devtools:
> install.packages('devtools')
> devtools::test()Alternatively the test suite can be also run from the terminal after cd to the tests directory
$ Rscript testthat.Ror for just running a single test script like test_diagnostics.R from the R console:
> library(testthat)
> source('test_diagnostics.R')6. Generating documentation
Most of the doc pages are generated from Jupyter notebooks in the notebooks directory at the base of the source tree. Please make changes there and then rebuild the docs:
$ cd docs
$ make notebooksIn R, the documentation for the source code must also generated if new parameters are added or a new function is created. This is documented with roxygen.
Run the command below before submitting a PR with any changes to the R code to update the function documentation:
> devtools::document()7. Committing your code
Keep style fixes to a separate commit to make your pull request more readable. Once you’ve made changes, you can see them by typing:
$ git statusIf you have created a new file, it is not being tracked by git. Add it by typing:
$ git add path/to/file-to-be-added.pyDoing ‘git status’ again should give something like:
# On branch new-feature
#
# modified: /relative/path/to/file-you-added.py
#Now you can commit your changes in your local repository:
$ git commit -m8. Pushing your changes
When you want your changes to appear publicly on your GitHub page, push your forked feature branch’s commits:
$ git push origin new-featureHere origin is the default name given to your remote repository on GitHub. You can see the remote repositories:
$ git remote -vIf you added the upstream repository as described above you will see something like:
origin [email protected]:yourname/prophet.git (fetch)
origin [email protected]:yourname/prophet.git (push)
upstream https://github.com/facebook/prophet.git (fetch)
upstream https://github.com/facebook/prophet.git (push)Now your code is on GitHub, but it is not yet a part of the prophet project. For that to happen, a pull request needs to be submitted on GitHub.
9. Review your code
When you’re ready to ask for a code review, file a pull request. Before you do, once again make sure that you have followed all the guidelines outlined in this document regarding code style, tests, performance tests, and documentation. You should also double check your branch changes against the branch it was based on:
- Navigate to your repository on GitHub – https://github.com/your-user-name/prophet
- Click on Branches
- Click on the Compare button for your feature branch
- Select the base and compare branches, if necessary. This will be
mainand new-feature, respectively.
10. Making a pull request
If everything looks good, you are ready to make a pull request. A pull request is how code from a local repository becomes available to the GitHub community and can be reviewed and eventually merged into the main version. This pull request and its associated changes will eventually be committed to the main branch and available in the next release. To submit a pull request:
- Navigate to your repository on GitHub
- Click on the Pull Request button
- You can then click on Commits and Files Changed to make sure everything looks okay one last time
- Write a description of your changes in the Preview Discussion tab
- Click Send Pull Request.
This request then goes to the repository maintainers, and they will review the code. If you need to make more changes, you can make them in your branch, add them to a new commit, push them to GitHub, and the pull request will be automatically updated. Pushing them to GitHub again is done by:
$ git push origin new-featureThis will automatically update your pull request with the latest code and restart the Continuous Integration tests.
11. Delete your merged branch
Once your feature branch is merged into upstream main, you can delete your remote branch via the Delete branch option in the PR and the local copy by running:
$ git branch -d new-feature12. PR checklist
- Write docstrings for any functions that are included.
- Test that the documentation builds correctly. See “Generating documentation”.
- Test your code.
- Write new tests if needed. See "Testing with Continuous Integration"
- Test the code using unittest. Running all tests takes a while, so feel free to only run the tests you think are needed based on your PR. CI will catch any failing tests.
- In R, you can also run
devtools:check()for carrying out a number of automated checks all at once, for code problems, documentation, testing, package structure, vignettes etc. This will take a few minutes to run. - Once you push your changes and make a PR, make sure you use an informative title which summarizes the changes you have made.
- If the PR addresses an issue, please reference it e.g. fixes #1234
Docs Diagnostics (`docs/_docs/diagnostics.md`)
layout: docs
docid: "diagnostics"
title: "Diagnostics"
permalink: /docs/diagnostics.html
subsections:
- title: Cross validation
id: cross-validation - title: Parallelizing cross validation
id: parallelizing-cross-validation - title: Hyperparameter tuning
id: hyperparameter-tuning
Cross validation
Prophet includes functionality for time series cross validation to measure forecast error using historical data. This is done by selecting cutoff points in the history, and for each of them fitting the model using data only up to that cutoff point. We can then compare the forecasted values to the actual values. This figure illustrates a simulated historical forecast on the Peyton Manning dataset, where the model was fit to an initial history of 5 years, and a forecast was made on a one year horizon.
The Prophet paper gives further description of simulated historical forecasts.
This cross validation procedure can be done automatically for a range of historical cutoffs using the cross_validation function. We specify the forecast horizon (horizon), and then optionally the size of the initial training period (initial) and the spacing between cutoff dates (period). By default, the initial training period is set to three times the horizon, and cutoffs are made every half a horizon.
The output of cross_validation is a dataframe with the true values y and the out-of-sample forecast values yhat, at each simulated forecast date and for each cutoff date. In particular, a forecast is made for every observed point between cutoff and cutoff + horizon. This dataframe can then be used to compute error measures of yhat vs. y.
Here we do cross-validation to assess prediction performance on a horizon of 365 days, starting with 730 days of training data in the first cutoff and then making predictions every 180 days. On this 8 year time series, this corresponds to 11 total forecasts.
# R
df.cv <- cross_validation(m, initial = 730, period = 180, horizon = 365, units = 'days')
head(df.cv)# Python
from prophet.diagnostics import cross_validation
df_cv = cross_validation(m, initial='730 days', period='180 days', horizon = '365 days')# Python
df_cv.head()| ds | yhat | yhat_lower | yhat_upper | y | cutoff | |
|---|---|---|---|---|---|---|
| 0 | 2010-02-16 | 8.954582 | 8.462876 | 9.452305 | 8.242493 | 2010-02-15 |
| 1 | 2010-02-17 | 8.720932 | 8.222682 | 9.242788 | 8.008033 | 2010-02-15 |
| 2 | 2010-02-18 | 8.604608 | 8.066920 | 9.144968 | 8.045268 | 2010-02-15 |
| 3 | 2010-02-19 | 8.526379 | 8.029189 | 9.043045 | 7.928766 | 2010-02-15 |
| 4 | 2010-02-20 | 8.268247 | 7.749520 | 8.741847 | 7.745003 | 2010-02-15 |
In R, the argument units must be a type accepted by as.difftime, which is weeks or shorter. In Python, the string for initial, period, and horizon should be in the format used by Pandas Timedelta, which accepts units of days or shorter.
Custom cutoffs can also be supplied as a list of dates to the cutoffs keyword in the cross_validation function in Python and R. For example, three cutoffs six months apart, would need to be passed to the cutoffs argument in a date format like:
# R
cutoffs <- as.Date(c('2013-02-15', '2013-08-15', '2014-02-15'))
df.cv2 <- cross_validation(m, cutoffs = cutoffs, horizon = 365, units = 'days')# Python
cutoffs = pd.to_datetime(['2013-02-15', '2013-08-15', '2014-02-15'])
df_cv2 = cross_validation(m, cutoffs=cutoffs, horizon='365 days')The performance_metrics utility can be used to compute some useful statistics of the prediction performance (yhat, yhat_lower, and yhat_upper compared to y), as a function of the distance from the cutoff (how far into the future the prediction was). The statistics computed are mean squared error (MSE), root mean squared error (RMSE), mean absolute error (MAE), mean absolute percent error (MAPE), median absolute percent error (MDAPE) and coverage of the yhat_lower and yhat_upper estimates. These are computed on a rolling window of the predictions in df_cv after sorting by horizon (ds minus cutoff). By default 10% of the predictions will be included in each window, but this can be changed with the rolling_window argument.
In Python, you can also create custom performance metric using the register_performance_metric decorator. Created metric should contain following arguments:
df: Cross-validation results dataframe.
w: Aggregation window size.
and return:
- Dataframe with columns horizon and metric.
# R
df.p <- performance_metrics(df.cv)
head(df.p)# Python
from prophet.diagnostics import performance_metrics
df_p = performance_metrics(df_cv)
df_p.head()| horizon | mse | rmse | mae | mape | mdape | smape | coverage | |
|---|---|---|---|---|---|---|---|---|
| 0 | 37 days | 0.493358 | 0.702395 | 0.503977 | 0.058376 | 0.049365 | 0.058677 | 0.676565 |
| 1 | 38 days | 0.499112 | 0.706478 | 0.508946 | 0.058951 | 0.049135 | 0.059312 | 0.675423 |
| 2 | 39 days | 0.521344 | 0.722042 | 0.515016 | 0.059547 | 0.049225 | 0.060034 | 0.672682 |
| 3 | 40 days | 0.528651 | 0.727084 | 0.517873 | 0.059852 | 0.049072 | 0.060409 | 0.676336 |
| 4 | 41 days | 0.536149 | 0.732222 | 0.518843 | 0.059927 | 0.049135 | 0.060548 | 0.681361 |
# Python
from prophet.diagnostics import register_performance_metric, rolling_mean_by_h
import numpy as np
@register_performance_metric
def mase(df, w):
"""Mean absolute scale error
Parameters
----------
df: Cross-validation results dataframe.
w: Aggregation window size.
Returns
-------
Dataframe with columns horizon and mase.
"""
e = (df['y'] - df['yhat'])
d = np.abs(np.diff(df['y'])).sum()/(df['y'].shape[0]-1)
se = np.abs(e/d)
if w < 0:
return pd.DataFrame({'horizon': df['horizon'], 'mase': se})
return rolling_mean_by_h(
x=se.values, h=df['horizon'].values, w=w, name='mase'
)
df_mase = performance_metrics(df_cv, metrics=['mase'])
df_mase.head()| horizon | mase | |
|---|---|---|
| 0 | 37 days | 0.522946 |
| 1 | 38 days | 0.528102 |
| 2 | 39 days | 0.534401 |
| 3 | 40 days | 0.537365 |
| 4 | 41 days | 0.538372 |
Cross validation performance metrics can be visualized with plot_cross_validation_metric, here shown for MAPE. Dots show the absolute percent error for each prediction in df_cv. The blue line shows the MAPE, where the mean is taken over a rolling window of the dots. We see for this forecast that errors around 5% are typical for predictions one month into the future, and that errors increase up to around 11% for predictions that are a year out.
# R
plot_cross_validation_metric(df.cv, metric = 'mape')# Python
from prophet.plot import plot_cross_validation_metric
fig = plot_cross_validation_metric(df_cv, metric='mape')The size of the rolling window in the figure can be changed with the optional argument rolling_window, which specifies the proportion of forecasts to use in each rolling window. The default is 0.1, corresponding to 10% of rows from df_cv included in each window; increasing this will lead to a smoother average curve in the figure. The initial period should be long enough to capture all of the components of the model, in particular seasonalities and extra regressors: at least a year for yearly seasonality, at least a week for weekly seasonality, etc.
Parallelizing cross validation
Cross-validation can also be run in parallel mode in Python, by setting specifying the parallel keyword. Four modes are supported
parallel=None(Default, no parallelization)parallel="processes"parallel="threads"parallel="dask"
For problems that aren't too big, we recommend using parallel="processes". It will achieve the highest performance when the parallel cross validation can be done on a single machine. For large problems, a Dask cluster can be used to do the cross validation on many machines. You will need to install Dask separately, as it will not be installed with prophet.
from dask.distributed import Client
client = Client() # connect to the cluster
df_cv = cross_validation(m, initial='730 days', period='180 days', horizon='365 days',
parallel="dask")Hyperparameter tuning
Cross-validation can be used for tuning hyperparameters of the model, such as changepoint_prior_scale and seasonality_prior_scale. A Python example is given below, with a 4x4 grid of those two parameters, with parallelization over cutoffs. Here parameters are evaluated on RMSE averaged over a 30-day horizon, but different performance metrics may be appropriate for different problems.
# Python
import itertools
import numpy as np
import pandas as pd
param_grid = {
'changepoint_prior_scale': [0.001, 0.01, 0.1, 0.5],
'seasonality_prior_scale': [0.01, 0.1, 1.0, 10.0],
}
# Generate all combinations of parameters
all_params = [dict(zip(param_grid.keys(), v)) for v in itertools.product(*param_grid.values())]
rmses = [] # Store the RMSEs for each params here
# Use cross validation to evaluate all parameters
for params in all_params:
m = Prophet(**params).fit(df) # Fit model with given params
df_cv = cross_validation(m, cutoffs=cutoffs, horizon='30 days', parallel="processes")
df_p = performance_metrics(df_cv, rolling_window=1)
rmses.append(df_p['rmse'].values[0])
# Find the best parameters
tuning_results = pd.DataFrame(all_params)
tuning_results['rmse'] = rmses
print(tuning_results)| changepoint_prior_scale | seasonality_prior_scale | rmse | |
|---|---|---|---|
| 0 | 0.001 | 0.01 | 0.757694 |
| 1 | 0.001 | 0.10 | 0.743399 |
| 2 | 0.001 | 1.00 | 0.753387 |
| 3 | 0.001 | 10.00 | 0.762890 |
| 4 | 0.010 | 0.01 | 0.542315 |
| 5 | 0.010 | 0.10 | 0.535546 |
| 6 | 0.010 | 1.00 | 0.527008 |
| 7 | 0.010 | 10.00 | 0.541544 |
| 8 | 0.100 | 0.01 | 0.524835 |
| 9 | 0.100 | 0.10 | 0.516061 |
| 10 | 0.100 | 1.00 | 0.521406 |
| 11 | 0.100 | 10.00 | 0.518580 |
| 12 | 0.500 | 0.01 | 0.532140 |
| 13 | 0.500 | 0.10 | 0.524668 |
| 14 | 0.500 | 1.00 | 0.521130 |
| 15 | 0.500 | 10.00 | 0.522980 |
# Python
best_params = all_params[np.argmin(rmses)]
print(best_params){'changepoint_prior_scale': 0.1, 'seasonality_prior_scale': 0.1}
Alternatively, parallelization could be done across parameter combinations by parallelizing the loop above.
The Prophet model has a number of input parameters that one might consider tuning. Here are some general recommendations for hyperparameter tuning that may be a good starting place.
Parameters that can be tuned
changepoint_prior_scale: This is probably the most impactful parameter. It determines the flexibility of the trend, and in particular how much the trend changes at the trend changepoints. As described in this documentation, if it is too small, the trend will be underfit and variance that should have been modeled with trend changes will instead end up being handled with the noise term. If it is too large, the trend will overfit and in the most extreme case you can end up with the trend capturing yearly seasonality. The default of 0.05 works for many time series, but this could be tuned; a range of [0.001, 0.5] would likely be about right. Parameters like this (regularization penalties; this is effectively a lasso penalty) are often tuned on a log scale.seasonality_prior_scale: This parameter controls the flexibility of the seasonality. Similarly, a large value allows the seasonality to fit large fluctuations, a small value shrinks the magnitude of the seasonality. The default is 10., which applies basically no regularization. That is because we very rarely see overfitting here (there's inherent regularization with the fact that it is being modeled with a truncated Fourier series, so it's essentially low-pass filtered). A reasonable range for tuning it would probably be [0.01, 10]; when set to 0.01 you should find that the magnitude of seasonality is forced to be very small. This likely also makes sense on a log scale, since it is effectively an L2 penalty like in ridge regression.holidays_prior_scale: This controls flexibility to fit holiday effects. Similar to seasonality_prior_scale, it defaults to 10.0 which applies basically no regularization, since we usually have multiple observations of holidays and can do a good job of estimating their effects. This could also be tuned on a range of [0.01, 10] as with seasonality_prior_scale.seasonality_mode: Options are ['additive','multiplicative']. Default is'additive', but many business time series will have multiplicative seasonality. This is best identified just from looking at the time series and seeing if the magnitude of seasonal fluctuations grows with the magnitude of the time series (see the documentation here on multiplicative seasonality), but when that isn't possible, it could be tuned.
Maybe tune?
changepoint_range: This is the proportion of the history in which the trend is allowed to change. This defaults to 0.8, 80% of the history, meaning the model will not fit any trend changes in the last 20% of the time series. This is fairly conservative, to avoid overfitting to trend changes at the very end of the time series where there isn't enough runway left to fit it well. With a human in the loop, this is something that can be identified pretty easily visually: one can pretty clearly see if the forecast is doing a bad job in the last 20%. In a fully-automated setting, it may be beneficial to be less conservative. It likely will not be possible to tune this parameter effectively with cross validation over cutoffs as described above. The ability of the model to generalize from a trend change in the last 10% of the time series will be hard to learn from looking at earlier cutoffs that may not have trend changes in the last 10%. So, this parameter is probably better not tuned, except perhaps over a large number of time series. In that setting, [0.8, 0.95] may be a reasonable range.
Parameters that would likely not be tuned
growth: Options are 'linear' and 'logistic'. This likely will not be tuned; if there is a known saturating point and growth towards that point it will be included and the logistic trend will be used, otherwise it will be linear.changepoints: This is for manually specifying the locations of changepoints. None by default, which automatically places them.n_changepoints: This is the number of automatically placed changepoints. The default of 25 should be plenty to capture the trend changes in a typical time series (at least the type that Prophet would work well on anyway). Rather than increasing or decreasing the number of changepoints, it will likely be more effective to focus on increasing or decreasing the flexibility at those trend changes, which is done withchangepoint_prior_scale.yearly_seasonality: By default ('auto') this will turn yearly seasonality on if there is a year of data, and off otherwise. Options are ['auto', True, False]. If there is more than a year of data, rather than trying to turn this off during HPO, it will likely be more effective to leave it on and turn down seasonal effects by tuningseasonality_prior_scale.weekly_seasonality: Same as foryearly_seasonality.daily_seasonality: Same as foryearly_seasonality.holidays: This is to pass in a dataframe of specified holidays. The holiday effects would be tuned withholidays_prior_scale.mcmc_samples: Whether or not MCMC is used will likely be determined by factors like the length of the time series and the importance of parameter uncertainty (these considerations are described in the documentation).interval_width: Prophetpredictreturns uncertainty intervals for each component, likeyhat_lowerandyhat_upperfor the forecastyhat. These are computed as quantiles of the posterior predictive distribution, andinterval_widthspecifies which quantiles to use. The default of 0.8 provides an 80% prediction interval. You could change that to 0.95 if you wanted a 95% interval. This will affect only the uncertainty interval, and will not change the forecastyhatat all and so does not need to be tuned.uncertainty_samples: The uncertainty intervals are computed as quantiles from the posterior predictive interval, and the posterior predictive interval is estimated with Monte Carlo sampling. This parameter is the number of samples to use (defaults to 1000). The running time for predict will be linear in this number. Making it smaller will increase the variance (Monte Carlo error) of the uncertainty interval, and making it larger will reduce that variance. So, if the uncertainty estimates seem jagged this could be increased to further smooth them out, but it likely will not need to be changed. As withinterval_width, this parameter only affects the uncertainty intervals and changing it will not affect in any way the forecastyhat; it does not need to be tuned.
Docs Handling Shocks (`docs/_docs/handling_shocks.md`)
layout: docs
docid: "handling_shocks"
title: "Handling Shocks"
permalink: /docs/handling_shocks.html
subsections:
- title: Case Study - Pedestrian Activity
id: case-study---pedestrian-activity - title: Default model without any adjustments
id: default-model-without-any-adjustments - title: Treating COVID-19 lockdowns as a one-off holidays
id: treating-covid-19-lockdowns-as-a-one-off-holidays - title: Sense checking the trend
id: sense-checking-the-trend - title: Changes in seasonality between pre- and post-COVID
id: changes-in-seasonality-between-pre--and-post-covid - title: Further reading
id: further-reading
# Python
%matplotlib inline
from prophet import Prophet
import pandas as pd
from matplotlib import pyplot as plt
import logging
logging.getLogger('prophet').setLevel(logging.ERROR)
import warnings
warnings.filterwarnings("ignore")
plt.rcParams['figure.figsize'] = 9, 6As a result of the lockdowns caused by the COVID-19 pandemic, many time series experienced "shocks" during 2020, e.g. spikes in media consumption (Netflix, YouTube), e-commerce transactions (Amazon, eBay), whilst attendance to in-person events declined dramatically.
Most of these time series would also maintain their new level for a period of time, subject to fluctuations driven by easing of lockdowns and/or vaccines.
Seasonal patterns could have also changed: for example, people may have consumed less media (in total hours) on weekdays compared to weekends before the COVID lockdowns, but during lockdowns weekday consumption could be much closer to weekend consumption.
In this page we'll explore some strategies for capturing these effects using Prophet's functionality:
Marking step changes / spikes due to COVID events as once-off.
Sustained changes in behaviour leading to trend and seasonality changes.
Case Study - Pedestrian Activity
For this case study we'll use Pedestrian Sensor data from the City of Melbourne. This data measures foot traffic from sensors in various places in the central business district, and we've chosen one sensor (Sensor_ID = 4) and aggregated the values to a daily grain.
The aggregated dataset can be found in the examples folder here.
# Python
df = pd.read_csv('https://raw.githubusercontent.com/facebook/prophet/main/examples/example_pedestrians_covid.csv')# Python
df.set_index('ds').plot();We can see two key events in the time series:
The initial drop in foot traffic around 21 March 2020, which started to recover around 6 June 2020. This corresponds to the declaration of a pandemic by WHO and subsequent lockdowns mandated by the Victorian government.
After some slow recovery, a second drop in foot traffic around July 9 2020, which began to recover around 27 October 2020. This corresponds to the "second wave" of the pandemic in metropolitan Melbourne.
There are also shorter periods of strict lockdown that lead to sudden tips in the time series: 5 days in February 2021, and 14 days in early June 2021.
Default model without any adjustments
First we'll fit a model with the default Prophet settings:
# Python
m = Prophet()
m = m.fit(df)
future = m.make_future_dataframe(periods=366)
forecast = m.predict(future)02:53:41 - cmdstanpy - INFO - Chain [1] start processing
02:53:41 - cmdstanpy - INFO - Chain [1] done processing# Python
m.plot(forecast)
plt.axhline(y=0, color='red')
plt.title('Default Prophet');# Python
m.plot_components(forecast);The model seems to fit reasonably to past data, but notice how we're capturing the dips, and the spikes after the dips, as a part of the trend component.
By default, the model assumes that these large spikes are possible in the future, even though we realistically won't see something of the same magnitude within our forecast horizon (1 year in this case). This leads to a fairly optimistic forecast of the recovery of foot traffic in 2022.
Treating COVID-19 lockdowns as a one-off holidays
To prevent large dips and spikes from being captured by the trend component, we can treat the days impacted by COVID-19 as holidays that will not repeat again in the future. Adding custom holidays is explained in more detail here. We set up a DataFrame like so to describe the periods affected by lockdowns:
# Python
lockdowns = pd.DataFrame([
{'holiday': 'lockdown_1', 'ds': '2020-03-21', 'lower_window': 0, 'ds_upper': '2020-06-06'},
{'holiday': 'lockdown_2', 'ds': '2020-07-09', 'lower_window': 0, 'ds_upper': '2020-10-27'},
{'holiday': 'lockdown_3', 'ds': '2021-02-13', 'lower_window': 0, 'ds_upper': '2021-02-17'},
{'holiday': 'lockdown_4', 'ds': '2021-05-28', 'lower_window': 0, 'ds_upper': '2021-06-10'},
])
for t_col in ['ds', 'ds_upper']:
lockdowns[t_col] = pd.to_datetime(lockdowns[t_col])
lockdowns['upper_window'] = (lockdowns['ds_upper'] - lockdowns['ds']).dt.days
lockdowns| holiday | ds | lower_window | ds_upper | upper_window | |
|---|---|---|---|---|---|
| 0 | lockdown_1 | 2020-03-21 | 0 | 2020-06-06 | 77 |
| 1 | lockdown_2 | 2020-07-09 | 0 | 2020-10-27 | 110 |
| 2 | lockdown_3 | 2021-02-13 | 0 | 2021-02-17 | 4 |
| 3 | lockdown_4 | 2021-05-28 | 0 | 2021-06-10 | 13 |
We have an entry for each lockdown period, with
dsspecifying the start of the lockdown.ds_upperis not used by Prophet, but it's a convenient way for us to calculateupper_window.upper_windowtells Prophet that the lockdown spans for x days after the start of the lockdown. Note that the holidays regression is inclusive of the upper bound.
Note that since we don't specify any future dates, Prophet will assume that these holidays will not occur again when creating the future dataframe (and hence they won't affect our projections). This is different to how we would specify a recurring holiday.
# Python
m2 = Prophet(holidays=lockdowns)
m2 = m2.fit(df)
future2 = m2.make_future_dataframe(periods=366)
forecast2 = m2.predict(future2)02:53:44 - cmdstanpy - INFO - Chain [1] start processing
02:53:45 - cmdstanpy - INFO - Chain [1] done processing# Python
m2.plot(forecast2)
plt.axhline(y=0, color='red')
plt.title('Lockdowns as one-off holidays');# Python
m2.plot_components(forecast2);Prophet is sensibly assigning a large negative effect to the days within the lockdown periods.
The forecast for the trend is not as strong / optimistic and seems fairly reasonable.
Sense checking the trend
In an environment when behaviours are constantly changing, it's important to ensure that the trend component of the model is capturing to emerging patterns without overfitting to them.
The trend changepoints documentation explains two things we could tweak with the trend component:
The changepoint locations, which by default are evenly spaced across 80% of the history. We should be mindful of where this range ends, and extend the range (either by increasing the % or adding changepoints manually) if we believe the most recent data better reflects future behaviour.
The strength of regularisation (
changepoint_prior_scale), which determines how flexible the trend is; the default value is0.05and increasing this will allow the trend to fit more closely to the observed data.
We plot the trend component and changepoints detected by our current model below.
# Python
from prophet.plot import add_changepoints_to_plot
fig = m2.plot(forecast2)
a = add_changepoints_to_plot(fig.gca(), m2, forecast2)The detected changepoints look reasonable, and the future trend tracks the latest upwards trend in activity, but not to the extent of late 2020. This seems suitable for a best guess of future activity.
We can see what the forecast would look like if we wanted to emphasize COVID patterns more in model training; we can do this by adding more potential changepoints after 2020 and making the trend more flexible.
# Python
m3_changepoints = (
# 10 potential changepoints in 2.5 years
pd.date_range('2017-06-02', '2020-01-01', periods=10).date.tolist() +
# 15 potential changepoints in 1 year 2 months
pd.date_range('2020-02-01', '2021-04-01', periods=15).date.tolist()
)# Python
# Default changepoint_prior_scale is 0.05, so 1.0 will lead to much more flexibility in comparison.
m3 = Prophet(holidays=lockdowns, changepoints=m3_changepoints, changepoint_prior_scale=1.0)
m3 = m3.fit(df)
forecast3 = m3.predict(future2)02:53:49 - cmdstanpy - INFO - Chain [1] start processing
02:53:52 - cmdstanpy - INFO - Chain [1] done processing# Python
from prophet.plot import add_changepoints_to_plot
fig = m3.plot(forecast3)
a = add_changepoints_to_plot(fig.gca(), m3, forecast3)We're seeing many changepoints detected post-COVID, matching the various fluctuations from loosening / tightening lockdowns. Overall the trend curve and forecasted trend are quite similar to our previous model, but we're seeing a lot more uncertainty because of the higher number of trend changes we picked up in the history.
We probably wouldn't pick this model over the model with default parameters as a best estimate, but it's a good demonstration of how we can incorporate our beliefs into the model about which patterns are important to capture.
Changes in seasonality between pre- and post-COVID
The seasonal component plots in the previous sections show a peak of activity on Friday compared to other days of the week. If we're not sure whether this will still hold post-lockdown, we can add conditional seasonalities to the model. Conditional seasonalities are explained in more detail here.
First we define boolean columns in the history dataframe to flag "pre covid" and "post covid" periods:
# Python
df2 = df.copy()
df2['pre_covid'] = pd.to_datetime(df2['ds']) < pd.to_datetime('2020-03-21')
df2['post_covid'] = ~df2['pre_covid']The conditional seasonality we're interested in modelling here is the day-of-week ("weekly") seasonality. To do this, we firstly turn off the default weekly_seasonality when we create the Prophet model.
# Python
m4 = Prophet(holidays=lockdowns, weekly_seasonality=False)We then add this weekly seasonality manually, as two different model components - one for pre-covid, one for post-covid. Note that fourier_order=3 is the default setting for weekly seasonality. After this we can run .fit().
# Python
m4.add_seasonality(
name='weekly_pre_covid',
period=7,
fourier_order=3,
condition_name='pre_covid',
)
m4.add_seasonality(
name='weekly_post_covid',
period=7,
fourier_order=3,
condition_name='post_covid',
);# Python
m4 = m4.fit(df2)02:53:55 - cmdstanpy - INFO - Chain [1] start processing
02:53:56 - cmdstanpy - INFO - Chain [1] done processingWe also need to create the pre_covid and post_covid flags in the future dataframe. This is so that Prophet can apply the correct weekly seasonality parameters to each future date.
# Python
future4 = m4.make_future_dataframe(periods=366)
future4['pre_covid'] = pd.to_datetime(future4['ds']) < pd.to_datetime('2020-03-21')
future4['post_covid'] = ~future4['pre_covid']# Python
forecast4 = m4.predict(future4)# Python
m4.plot(forecast4)
plt.axhline(y=0, color='red')
plt.title('Lockdowns as one-off holidays + Conditional weekly seasonality');# Python
m4.plot_components(forecast4);Interestingly, the model with conditional seasonalities suggests that, post-COVID, pedestrian activity peaks on Saturdays, instead of Fridays. This could make sense if most people are still working from home and are hence less likely to go out on Friday nights. From a prediction perspective this would only be important if we care about predicting weekdays vs. weekends accurately, but overall this kind of exploration helps us gain insight into how COVID has changed behaviours.
Further reading
A lot of the content in this page was inspired by this GitHub discussion. We've covered a few low hanging fruits for tweaking Prophet models when faced with shocks such as COVID, but there are many other possible approaches as well, such as:
Using external regressors (e.g. the lockdown stringency index). This would only be fruitful if we a) have regressor data that aligns well (in terms of location) with the series we're forecasting, and b) have control over or can predict the regressor much more accurately than the time series alone.
Detecting and removing outlier data from the training period, or throwing away older training data completely. This might be a better approach for sub-daily time series that don't have yearly seasonal patterns.
Overall though it's difficult to be confident in our forecasts in these environments when rules are constantly changing and outbreaks occur randomly. In this scenario it's more important to constantly re-train / re-evaluate our models and clearly communicate the increased uncertainty in forecasts.
# PythonDocs Holiday Effects (`docs/_docs/holiday_effects.md`)
layout: docs docid: "holiday_effects" title: "Holiday Effects" permalink: /docs/holiday_effects.html
Modeling Holidays
If you have holidays that you'd like to model, you must create a dataframe for them. It has two columns (holiday and ds) and a row for each occurrence of the holiday. It must include all occurrences of the holiday, both in the past (back as far as the historical data go) and in the future (out as far as the forecast is being made). If they won't repeat in the future, Prophet will model them and then not include them in the forecast.
You can also include columns lower_window and upper_window which extend the holiday out to [lower_window, upper_window] days around the date. For instance, if you wanted to included Christmas Eve in addition to Christmas you'd include lower_window=-1,upper_window=0. If you wanted to use Black Friday in addition to Thanksgiving, you'd include lower_window=0,upper_window=1.
Here we create a dataframe that includes the dates of all of Peyton Manning's playoff appearances:
# Python
playoffs = pd.DataFrame({
'holiday': 'playoff',
'ds': pd.to_datetime(['2008-01-13', '2009-01-03', '2010-01-16',
'2010-01-24', '2010-02-07', '2011-01-08',
'2013-01-12', '2014-01-12', '2014-01-19',
'2014-02-02', '2015-01-11', '2016-01-17',
'2016-01-24', '2016-02-07']),
'lower_window': 0,
'upper_window': 1,
})
superbowls = pd.DataFrame({
'holiday': 'superbowl',
'ds': pd.to_datetime(['2010-02-07', '2014-02-02', '2016-02-07']),
'lower_window': 0,
'upper_window': 1,
})
holidays = pd.concat((playoffs, superbowls))# R
library(dplyr)
playoffs <- data_frame(
holiday = 'playoff',
ds = as.Date(c('2008-01-13', '2009-01-03', '2010-01-16',
'2010-01-24', '2010-02-07', '2011-01-08',
'2013-01-12', '2014-01-12', '2014-01-19',
'2014-02-02', '2015-01-11', '2016-01-17',
'2016-01-24', '2016-02-07')),
lower_window = 0,
upper_window = 1
)
superbowls <- data_frame(
holiday = 'superbowl',
ds = as.Date(c('2010-02-07', '2014-02-02', '2016-02-07')),
lower_window = 0,
upper_window = 1
)
holidays <- bind_rows(playoffs, superbowls)Above we have include the superbowl days as both playoff games and superbowl games. This means that the superbowl effect will be an additional additive bonus on top of the playoff effect.
Once the table is created, holiday effects are included in the forecast by passing them in with the holidays argument. Here we do it with the Peyton Manning data from the Quickstart:
# Python
m = Prophet(holidays=holidays)
forecast = m.fit(df).predict(future)# R
m <- prophet(df, holidays = holidays)
forecast <- predict(m, future)The holiday effect can be seen in the forecast dataframe:
# R
forecast %>%
select(ds, playoff, superbowl) %>%
filter(abs(playoff + superbowl) > 0) %>%
tail(10)# Python
forecast[(forecast['playoff'] + forecast['superbowl']).abs() > 0][
['ds', 'playoff', 'superbowl']][-10:]| ds | playoff | superbowl | |
|---|---|---|---|
| 2190 | 2014-02-02 | 1.220308 | 1.204992 |
| 2191 | 2014-02-03 | 1.900465 | 1.444581 |
| 2532 | 2015-01-11 | 1.220308 | 0.000000 |
| 2533 | 2015-01-12 | 1.900465 | 0.000000 |
| 2901 | 2016-01-17 | 1.220308 | 0.000000 |
| 2902 | 2016-01-18 | 1.900465 | 0.000000 |
| 2908 | 2016-01-24 | 1.220308 | 0.000000 |
| 2909 | 2016-01-25 | 1.900465 | 0.000000 |
| 2922 | 2016-02-07 | 1.220308 | 1.204992 |
| 2923 | 2016-02-08 | 1.900465 | 1.444581 |
The holiday effects will also show up in the components plot, where we see that there is a spike on the days around playoff appearances, with an especially large spike for the superbowl:
# Python
m.plot_components(forecast);# R
prophet_plot_components(m, forecast);Prior scale for holidays and seasonality
If you find that the holidays are overfitting, you can adjust their prior scale to smooth them using the parameter holidays_prior_scale, which by default is 10:
# R
m <- prophet(df, holidays = holidays, holidays.prior.scale = 1)
forecast <- predict(m, future)
forecast %>%
select(ds, playoff, superbowl) %>%
filter(abs(playoff + superbowl) > 0) %>%
tail(10)# Python
m = Prophet(holidays=holidays, holidays_prior_scale=1).fit(df)
forecast = m.predict(future)
forecast[(forecast['playoff'] + forecast['superbowl']).abs() > 0][
['ds', 'playoff', 'superbowl']][-10:]| ds | playoff | superbowl | |
|---|---|---|---|
| 2190 | 2014-02-02 | 1.362312 | 0.693425 |
| 2191 | 2014-02-03 | 2.033471 | 0.542254 |
| 2532 | 2015-01-11 | 1.362312 | 0.000000 |
| 2533 | 2015-01-12 | 2.033471 | 0.000000 |
| 2901 | 2016-01-17 | 1.362312 | 0.000000 |
| 2902 | 2016-01-18 | 2.033471 | 0.000000 |
| 2908 | 2016-01-24 | 1.362312 | 0.000000 |
| 2909 | 2016-01-25 | 2.033471 | 0.000000 |
| 2922 | 2016-02-07 | 1.362312 | 0.693425 |
| 2923 | 2016-02-08 | 2.033471 | 0.542254 |
The magnitude of the holiday effect has been reduced compared to before, especially for superbowls, which had the fewest observations. There is a parameter seasonality_prior_scale which similarly adjusts the extent to which the seasonality model will fit the data.
Docs Installation (`docs/_docs/installation.md`)
layout: docs
docid: "installation"
title: "Installation"
permalink: /docs/installation.html
subsections:
- id: r
title: Using R - id: python
title: Using Python
Prophet has two implementations: R and Python.
Installation in R
Prophet is a CRAN package so you can use install.packages.
# R
install.packages('prophet')After installation, you can get started!
Experimental backend - cmdstanr
You can also choose an experimental alternative stan backend called cmdstanr. Once you've installed prophet,
follow these instructions to use cmdstanr instead of rstan as the backend:
# R
# We recommend running this is a fresh R session or restarting your current session
install.packages(c("cmdstanr", "posterior"), repos = c("https://stan-dev.r-universe.dev", getOption("repos")))
# If you haven't installed cmdstan before, run:
cmdstanr::install_cmdstan()
# Otherwise, you can point cmdstanr to your cmdstan path:
cmdstanr::set_cmdstan_path(path = <your existing cmdstan>)
# Set the R_STAN_BACKEND environment variable
Sys.setenv(R_STAN_BACKEND = "CMDSTANR")Windows
On Windows, R requires a compiler so you'll need to follow the instructions provided by rstan. The key step is installing Rtools before attempting to install the package.
If you have custom Stan compiler settings, install from source rather than the CRAN binary.
Installation in Python
Prophet is on PyPI, so you can use pip to install it.
python -m pip install prophet- From v0.6 onwards, Python 2 is no longer supported.
- As of v1.0, the package name on PyPI is "prophet"; prior to v1.0 it was "fbprophet".
- As of v1.1, the minimum supported Python version is 3.7.
After installation, you can get started!
Anaconda
Prophet can also be installed through conda-forge.
conda install -c conda-forge prophetDocs Multiplicative Seasonality (`docs/_docs/multiplicative_seasonality.md`)
layout: docs docid: "multiplicative_seasonality" title: "Multiplicative Seasonality" permalink: /docs/multiplicative_seasonality.html subsections:
By default Prophet fits additive seasonalities, meaning the effect of the seasonality is added to the trend to get the forecast. This time series of the number of air passengers is an example of when additive seasonality does not work:
# R
df <- read.csv('https://raw.githubusercontent.com/facebook/prophet/main/examples/example_air_passengers.csv')
m <- prophet(df)
future <- make_future_dataframe(m, 50, freq = 'm')
forecast <- predict(m, future)
plot(m, forecast)# Python
df = pd.read_csv('https://raw.githubusercontent.com/facebook/prophet/main/examples/example_air_passengers.csv')
m = Prophet()
m.fit(df)
future = m.make_future_dataframe(50, freq='MS')
forecast = m.predict(future)
fig = m.plot(forecast)This time series has a clear yearly cycle, but the seasonality in the forecast is too large at the start of the time series and too small at the end. In this time series, the seasonality is not a constant additive factor as assumed by Prophet, rather it grows with the trend. This is multiplicative seasonality.
Prophet can model multiplicative seasonality by setting seasonality_mode='multiplicative' in the input arguments:
# R
m <- prophet(df, seasonality.mode = 'multiplicative')
forecast <- predict(m, future)
plot(m, forecast)# Python
m = Prophet(seasonality_mode='multiplicative')
m.fit(df)
forecast = m.predict(future)
fig = m.plot(forecast)The components figure will now show the seasonality as a percent of the trend:
# R
prophet_plot_components(m, forecast)# Python
fig = m.plot_components(forecast)With seasonality_mode='multiplicative', holiday effects will also be modeled as multiplicative. Any added seasonalities or extra regressors will by default use whatever seasonality_mode is set to, but can be overriden by specifying mode='additive' or mode='multiplicative' as an argument when adding the seasonality or regressor.
For example, this block sets the built-in seasonalities to multiplicative, but includes an additive quarterly seasonality and an additive regressor:
# R
m <- prophet(seasonality.mode = 'multiplicative')
m <- add_seasonality(m, 'quarterly', period = 91.25, fourier.order = 8, mode = 'additive')
m <- add_regressor(m, 'regressor', mode = 'additive')# Python
m = Prophet(seasonality_mode='multiplicative')
m.add_seasonality('quarterly', period=91.25, fourier_order=8, mode='additive')
m.add_regressor('regressor', mode='additive')Additive and multiplicative extra regressors will show up in separate panels on the components plot. Note, however, that it is pretty unlikely to have a mix of additive and multiplicative seasonalities, so this will generally only be used if there is a reason to expect that to be the case.
Docs Non Daily Data (`docs/_docs/non-daily_data.md`)
layout: docs
docid: "non-daily_data"
title: "Non-Daily Data"
permalink: /docs/non-daily_data.html
subsections:
- title: Sub-daily data
id: sub-daily-data - title: Data with regular gaps
id: data-with-regular-gaps - title: Monthly data
id: monthly-data - title: Holidays with aggregated data
id: holidays-with-aggregated-data
Sub-daily data
Prophet can make forecasts for time series with sub-daily observations by passing in a dataframe with timestamps in the ds column. The format of the timestamps should be YYYY-MM-DD HH:MM:SS - see the example csv here. When sub-daily data are used, daily seasonality will automatically be fit. Here we fit Prophet to data with 5-minute resolution (daily temperatures at Yosemite):
# R
df <- read.csv('https://raw.githubusercontent.com/facebook/prophet/main/examples/example_yosemite_temps.csv')
m <- prophet(df, changepoint.prior.scale=0.01)
future <- make_future_dataframe(m, periods = 300, freq = 60 * 60)
fcst <- predict(m, future)
plot(m, fcst)# Python
df = pd.read_csv('https://raw.githubusercontent.com/facebook/prophet/main/examples/example_yosemite_temps.csv')
m = Prophet(changepoint_prior_scale=0.01).fit(df)
future = m.make_future_dataframe(periods=300, freq='H')
fcst = m.predict(future)
fig = m.plot(fcst)The daily seasonality will show up in the components plot:
# R
prophet_plot_components(m, fcst)# Python
fig = m.plot_components(fcst)Data with regular gaps
Suppose the dataset above only had observations from 12a to 6a:
# R
df2 <- df %>%
mutate(ds = as.POSIXct(ds, tz="GMT")) %>%
filter(as.numeric(format(ds, "%H")) < 6)
m <- prophet(df2)
future <- make_future_dataframe(m, periods = 300, freq = 60 * 60)
fcst <- predict(m, future)
plot(m, fcst)# Python
df2 = df.copy()
df2['ds'] = pd.to_datetime(df2['ds'])
df2 = df2[df2['ds'].dt.hour < 6]
m = Prophet().fit(df2)
future = m.make_future_dataframe(periods=300, freq='H')
fcst = m.predict(future)
fig = m.plot(fcst)The forecast seems quite poor, with much larger fluctuations in the future than were seen in the history. The issue here is that we have fit a daily cycle to a time series that only has data for part of the day (12a to 6a). The daily seasonality is thus unconstrained for the remainder of the day and is not estimated well. The solution is to only make predictions for the time windows for which there are historical data. Here, that means to limit the future dataframe to have times from 12a to 6a:
# R
future2 <- future %>%
filter(as.numeric(format(ds, "%H")) < 6)
fcst <- predict(m, future2)
plot(m, fcst)# Python
future2 = future.copy()
future2 = future2[future2['ds'].dt.hour < 6]
fcst = m.predict(future2)
fig = m.plot(fcst)The same principle applies to other datasets with regular gaps in the data. For example, if the history contains only weekdays, then predictions should only be made for weekdays since the weekly seasonality will not be well estimated for the weekends.
Monthly data
You can use Prophet to fit monthly data. However, the underlying model is continuous-time, which means that you can get strange results if you fit the model to monthly data and then ask for daily forecasts. Here we forecast US retail sales volume for the next 10 years:
# R
df <- read.csv('https://raw.githubusercontent.com/facebook/prophet/main/examples/example_retail_sales.csv')
m <- prophet(df, seasonality.mode = 'multiplicative')
future <- make_future_dataframe(m, periods = 3652)
fcst <- predict(m, future)
plot(m, fcst)# Python
df = pd.read_csv('https://raw.githubusercontent.com/facebook/prophet/main/examples/example_retail_sales.csv')
m = Prophet(seasonality_mode='multiplicative').fit(df)
future = m.make_future_dataframe(periods=3652)
fcst = m.predict(future)
fig = m.plot(fcst)This is the same issue from above where the dataset has regular gaps. When we fit the yearly seasonality, it only has data for the first of each month and the seasonality components for the remaining days are unidentifiable and overfit. This can be clearly seen by doing MCMC to see uncertainty in the seasonality:
# R
m <- prophet(df, seasonality.mode = 'multiplicative', mcmc.samples = 300)
fcst <- predict(m, future)
prophet_plot_components(m, fcst)# Python
m = Prophet(seasonality_mode='multiplicative', mcmc_samples=300).fit(df, show_progress=False)
fcst = m.predict(future)
fig = m.plot_components(fcst)WARNING:pystan:481 of 600 iterations saturated the maximum tree depth of 10 (80.2 %)
WARNING:pystan:Run again with max_treedepth larger than 10 to avoid saturationThe seasonality has low uncertainty at the start of each month where there are data points, but has very high posterior variance in between. When fitting Prophet to monthly data, only make monthly forecasts, which can be done by passing the frequency into make_future_dataframe:
# R
future <- make_future_dataframe(m, periods = 120, freq = 'month')
fcst <- predict(m, future)
plot(m, fcst)# Python
future = m.make_future_dataframe(periods=120, freq='MS')
fcst = m.predict(future)
fig = m.plot(fcst)In Python, the frequency can be anything from the pandas list of frequency strings here: https://pandas.pydata.org/pandas-docs/stable/user_guide/timeseries.html#timeseries-offset-aliases . Note that MS used here is month-start, meaning the data point is placed on the start of each month.
In monthly data, yearly seasonality can also be modeled with binary extra regressors. In particular, the model can use 12 extra regressors like is_jan, is_feb, etc. where is_jan is 1 if the date is in Jan and 0 otherwise. This approach would avoid the within-month unidentifiability seen above. Be sure to use yearly_seasonality=False if monthly extra regressors are being added.
Holidays with aggregated data
Holiday effects are applied to the particular date on which the holiday was specified. With data that has been aggregated to weekly or monthly frequency, holidays that don't fall on the particular date used in the data will be ignored: for example, a Monday holiday in a weekly time series where each data point is on a Sunday. To include holiday effects in the model, the holiday will need to be moved to the date in the history dataframe for which the effect is desired. Note that with weekly or monthly aggregated data, many holiday effects will be well-captured by the yearly seasonality, so added holidays may only be necessary for holidays that occur in different weeks throughout the time series.
Docs Outliers (`docs/_docs/outliers.md`)
layout: docs docid: "outliers" title: "Outliers" permalink: /docs/outliers.html subsections:
There are two main ways that outliers can affect Prophet forecasts. Here we make a forecast on the logged Wikipedia visits to the R page from before, but with a block of bad data:
# R
df <- read.csv('https://raw.githubusercontent.com/facebook/prophet/main/examples/example_wp_log_R_outliers1.csv')
m <- prophet(df)
future <- make_future_dataframe(m, periods = 1096)
forecast <- predict(m, future)
plot(m, forecast)# Python
df = pd.read_csv('https://raw.githubusercontent.com/facebook/prophet/main/examples/example_wp_log_R_outliers1.csv')
m = Prophet()
m.fit(df)
future = m.make_future_dataframe(periods=1096)
forecast = m.predict(future)
fig = m.plot(forecast)The trend forecast seems reasonable, but the uncertainty intervals seem way too wide. Prophet is able to handle the outliers in the history, but only by fitting them with trend changes. The uncertainty model then expects future trend changes of similar magnitude.
The best way to handle outliers is to remove them - Prophet has no problem with missing data. If you set their values to NA in the history but leave the dates in future, then Prophet will give you a prediction for their values.
# R
outliers <- (as.Date(df$ds) > as.Date('2010-01-01')
& as.Date(df$ds) < as.Date('2011-01-01'))
df$y[outliers] = NA
m <- prophet(df)
forecast <- predict(m, future)
plot(m, forecast)# Python
df.loc[(df['ds'] > '2010-01-01') & (df['ds'] < '2011-01-01'), 'y'] = None
model = Prophet().fit(df)
fig = model.plot(model.predict(future))In the above example the outliers messed up the uncertainty estimation but did not impact the main forecast yhat. This isn't always the case, as in this example with added outliers:
# R
df <- read.csv('https://raw.githubusercontent.com/facebook/prophet/main/examples/example_wp_log_R_outliers2.csv')
m <- prophet(df)
future <- make_future_dataframe(m, periods = 1096)
forecast <- predict(m, future)
plot(m, forecast)# Python
df = pd.read_csv('https://raw.githubusercontent.com/facebook/prophet/main/examples/example_wp_log_R_outliers2.csv')
m = Prophet()
m.fit(df)
future = m.make_future_dataframe(periods=1096)
forecast = m.predict(future)
fig = m.plot(forecast)Here a group of extreme outliers in June 2015 mess up the seasonality estimate, so their effect reverberates into the future forever. Again the right approach is to remove them:
# R
outliers <- (as.Date(df$ds) > as.Date('2015-06-01')
& as.Date(df$ds) < as.Date('2015-06-30'))
df$y[outliers] = NA
m <- prophet(df)
forecast <- predict(m, future)
plot(m, forecast)# Python
df.loc[(df['ds'] > '2015-06-01') & (df['ds'] < '2015-06-30'), 'y'] = None
m = Prophet().fit(df)
fig = m.plot(m.predict(future))Docs Quick Start (`docs/_docs/quick_start.md`)
layout: docs
docid: "quick_start"
title: "Quick Start"
permalink: /docs/quick_start.html
subsections:
- title: Python API
id: python-api - title: R API
id: r-api
Python API
Prophet follows the sklearn model API. We create an instance of the Prophet class and then call its fit and predict methods.
The input to Prophet is always a dataframe with two columns: ds and y. The ds (datestamp) column should be of a format expected by Pandas, ideally YYYY-MM-DD for a date or YYYY-MM-DD HH:MM:SS for a timestamp. The y column must be numeric, and represents the measurement we wish to forecast.
As an example, let's look at a time series of the log daily page views for the Wikipedia page for Peyton Manning. We scraped this data using the Wikipediatrend package in R. Peyton Manning provides a nice example because it illustrates some of Prophet's features, like multiple seasonality, changing growth rates, and the ability to model special days (such as Manning's playoff and superbowl appearances). The CSV is available here.
First we'll import the data:
# Python
import pandas as pd
from prophet import Prophet# Python
df = pd.read_csv('https://raw.githubusercontent.com/facebook/prophet/main/examples/example_wp_log_peyton_manning.csv')
df.head()| ds | y | |
|---|---|---|
| 0 | 2007-12-10 | 9.590761 |
| 1 | 2007-12-11 | 8.519590 |
| 2 | 2007-12-12 | 8.183677 |
| 3 | 2007-12-13 | 8.072467 |
| 4 | 2007-12-14 | 7.893572 |
We fit the model by instantiating a new Prophet object. Any settings to the forecasting procedure are passed into the constructor. Then you call its fit method and pass in the historical dataframe. Fitting should take 1-5 seconds.
# Python
m = Prophet()
m.fit(df)Predictions are then made on a dataframe with a column ds containing the dates for which a prediction is to be made. You can get a suitable dataframe that extends into the future a specified number of days using the helper method Prophet.make_future_dataframe. By default it will also include the dates from the history, so we will see the model fit as well.
# Python
future = m.make_future_dataframe(periods=365)
future.tail()| ds | |
|---|---|
| 3265 | 2017-01-15 |
| 3266 | 2017-01-16 |
| 3267 | 2017-01-17 |
| 3268 | 2017-01-18 |
| 3269 | 2017-01-19 |
The predict method will assign each row in future a predicted value which it names yhat. If you pass in historical dates, it will provide an in-sample fit. The forecast object here is a new dataframe that includes a column yhat with the forecast, as well as columns for components and uncertainty intervals.
# Python
forecast = m.predict(future)
forecast[['ds', 'yhat', 'yhat_lower', 'yhat_upper']].tail()| ds | yhat | yhat_lower | yhat_upper | |
|---|---|---|---|---|
| 3265 | 2017-01-15 | 8.212625 | 7.456310 | 8.959726 |
| 3266 | 2017-01-16 | 8.537635 | 7.842986 | 9.290934 |
| 3267 | 2017-01-17 | 8.325071 | 7.600879 | 9.072006 |
| 3268 | 2017-01-18 | 8.157723 | 7.512052 | 8.924022 |
| 3269 | 2017-01-19 | 8.169677 | 7.412473 | 8.946977 |
You can plot the forecast by calling the Prophet.plot method and passing in your forecast dataframe.
# Python
fig1 = m.plot(forecast)If you want to see the forecast components, you can use the Prophet.plot_components method. By default you'll see the trend, yearly seasonality, and weekly seasonality of the time series. If you include holidays, you'll see those here, too.
# Python
fig2 = m.plot_components(forecast)An interactive figure of the forecast and components can be created with plotly. You will need to install plotly 4.0 or above separately, as it will not by default be installed with prophet. You will also need to install the notebook and ipywidgets packages.
# Python
from prophet.plot import plot_plotly, plot_components_plotly
plot_plotly(m, forecast)# Python
plot_components_plotly(m, forecast)More details about the options available for each method are available in the docstrings, for example, via help(Prophet) or help(Prophet.fit).
R API
In R, we use the normal model fitting API. We provide a prophet function that performs fitting and returns a model object. You can then call predict and plot on this model object.
# R
library(prophet)R[write to console]: Loading required package: Rcpp
R[write to console]: Loading required package: rlangFirst we read in the data and create the outcome variable. As in the Python API, this is a dataframe with columns ds and y, containing the date and numeric value respectively. The ds column should be YYYY-MM-DD for a date, or YYYY-MM-DD HH:MM:SS for a timestamp. As above, we use here the log number of views to Peyton Manning's Wikipedia page, available here.
# R
df <- read.csv('https://raw.githubusercontent.com/facebook/prophet/main/examples/example_wp_log_peyton_manning.csv')We call the prophet function to fit the model. The first argument is the historical dataframe. Additional arguments control how Prophet fits the data and are described in later pages of this documentation.
# R
m <- prophet(df)Predictions are made on a dataframe with a column ds containing the dates for which predictions are to be made. The make_future_dataframe function takes the model object and a number of periods to forecast and produces a suitable dataframe. By default it will also include the historical dates so we can evaluate in-sample fit.
# R
future <- make_future_dataframe(m, periods = 365)
tail(future)ds
3265 2017-01-14
3266 2017-01-15
3267 2017-01-16
3268 2017-01-17
3269 2017-01-18
3270 2017-01-19As with most modeling procedures in R, we use the generic predict function to get our forecast. The forecast object is a dataframe with a column yhat containing the forecast. It has additional columns for uncertainty intervals and seasonal components.
# R
forecast <- predict(m, future)
tail(forecast[c('ds', 'yhat', 'yhat_lower', 'yhat_upper')])ds yhat yhat_lower yhat_upper
3265 2017-01-14 7.818359 7.071228 8.550957
3266 2017-01-15 8.200125 7.475725 8.869495
3267 2017-01-16 8.525104 7.747071 9.226915
3268 2017-01-17 8.312482 7.551904 9.046774
3269 2017-01-18 8.145098 7.390770 8.863692
3270 2017-01-19 8.156964 7.381716 8.866507You can use the generic plot function to plot the forecast, by passing in the model and the forecast dataframe.
# R
plot(m, forecast)You can use the prophet_plot_components function to see the forecast broken down into trend, weekly seasonality, and yearly seasonality.
# R
prophet_plot_components(m, forecast)An interactive plot of the forecast using Dygraphs can be made with the command dyplot.prophet(m, forecast).
More details about the options available for each method are available in the docstrings, for example, via ?prophet or ?fit.prophet. This documentation is also available in the reference manual on CRAN.
Docs Saturating Forecasts (`docs/_docs/saturating_forecasts.md`)
layout: docs
docid: "saturating_forecasts"
title: "Saturating Forecasts"
permalink: /docs/saturating_forecasts.html
subsections:
- title: Forecasting Growth
id: forecasting-growth - title: Saturating Minimum
id: saturating-minimum
Forecasting Growth
By default, Prophet uses a linear model for its forecast. When forecasting growth, there is usually some maximum achievable point: total market size, total population size, etc. This is called the carrying capacity, and the forecast should saturate at this point.
Prophet allows you to make forecasts using a logistic growth trend model, with a specified carrying capacity. We illustrate this with the log number of page visits to the R (programming language) page on Wikipedia:
# R
df <- read.csv('https://raw.githubusercontent.com/facebook/prophet/main/examples/example_wp_log_R.csv')# Python
df = pd.read_csv('https://raw.githubusercontent.com/facebook/prophet/main/examples/example_wp_log_R.csv')We must specify the carrying capacity in a column cap. Here we will assume a particular value, but this would usually be set using data or expertise about the market size.
# R
df$cap <- 8.5# Python
df['cap'] = 8.5The important things to note are that cap must be specified for every row in the dataframe, and that it does not have to be constant. If the market size is growing, then cap can be an increasing sequence.
We then fit the model as before, except pass in an additional argument to specify logistic growth:
# R
m <- prophet(df, growth = 'logistic')# Python
m = Prophet(growth='logistic')
m.fit(df)We make a dataframe for future predictions as before, except we must also specify the capacity in the future. Here we keep capacity constant at the same value as in the history, and forecast 5 years into the future:
# R
future <- make_future_dataframe(m, periods = 1826)
future$cap <- 8.5
fcst <- predict(m, future)
plot(m, fcst)# Python
future = m.make_future_dataframe(periods=1826)
future['cap'] = 8.5
fcst = m.predict(future)
fig = m.plot(fcst)The logistic function has an implicit minimum of 0, and will saturate at 0 the same way that it saturates at the capacity. It is possible to also specify a different saturating minimum.
Saturating Minimum
The logistic growth model can also handle a saturating minimum, which is specified with a column floor in the same way as the cap column specifies the maximum:
# R
df$y <- 10 - df$y
df$cap <- 6
df$floor <- 1.5
future$cap <- 6
future$floor <- 1.5
m <- prophet(df, growth = 'logistic')
fcst <- predict(m, future)
plot(m, fcst)# Python
df['y'] = 10 - df['y']
df['cap'] = 6
df['floor'] = 1.5
future['cap'] = 6
future['floor'] = 1.5
m = Prophet(growth='logistic')
m.fit(df)
fcst = m.predict(future)
fig = m.plot(fcst)To use a logistic growth trend with a saturating minimum, a maximum capacity must also be specified.
Docs Seasonality, Holiday Effects, And Regressors (`docs/_docs/seasonality,_holiday_effects,_and_regressors.md`)
layout: docs
docid: "seasonality,_holiday_effects,_and_regressors"
title: "Seasonality, Holiday Effects, And Regressors"
permalink: /docs/seasonality,_holiday_effects,_and_regressors.html
subsections:
- title: Modeling Holidays and Special Events
id: modeling-holidays-and-special-events - title: Built-in Country Holidays
id: built-in-country-holidays - title: Holidays for subdivisions (Python)
id: holidays-for-subdivisions-(python) - title: Fourier Order for Seasonalities
id: fourier-order-for-seasonalities - title: Specifying Custom Seasonalities
id: specifying-custom-seasonalities - title: Seasonalities that depend on other factors
id: seasonalities-that-depend-on-other-factors - title: Prior scale for holidays and seasonality
id: prior-scale-for-holidays-and-seasonality - title: Additional regressors
id: additional-regressors - title: Coefficients of additional regressors
id: coefficients-of-additional-regressors - title: Uncertainty in future regressor values
id: uncertainty-in-future-regressor-values
Modeling Holidays and Special Events
If you have holidays or other recurring events that you'd like to model, you must create a dataframe for them. It has two columns (holiday and ds) and a row for each occurrence of the holiday. It must include all occurrences of the holiday, both in the past (back as far as the historical data go) and in the future (out as far as the forecast is being made). If they won't repeat in the future, Prophet will model them and then not include them in the forecast.
You can also include columns lower_window and upper_window which extend the holiday out to [lower_window, upper_window] days around the date. For instance, if you wanted to include Christmas Eve in addition to Christmas you'd include lower_window=-1,upper_window=0. If you wanted to use Black Friday in addition to Thanksgiving, you'd include lower_window=0,upper_window=1. You can also include a column prior_scale to set the prior scale separately for each holiday, as described below.
Here we create a dataframe that includes the dates of all of Peyton Manning's playoff appearances:
# R
library(dplyr)
playoffs <- data_frame(
holiday = 'playoff',
ds = as.Date(c('2008-01-13', '2009-01-03', '2010-01-16',
'2010-01-24', '2010-02-07', '2011-01-08',
'2013-01-12', '2014-01-12', '2014-01-19',
'2014-02-02', '2015-01-11', '2016-01-17',
'2016-01-24', '2016-02-07')),
lower_window = 0,
upper_window = 1
)
superbowls <- data_frame(
holiday = 'superbowl',
ds = as.Date(c('2010-02-07', '2014-02-02', '2016-02-07')),
lower_window = 0,
upper_window = 1
)
holidays <- bind_rows(playoffs, superbowls)# Python
playoffs = pd.DataFrame({
'holiday': 'playoff',
'ds': pd.to_datetime(['2008-01-13', '2009-01-03', '2010-01-16',
'2010-01-24', '2010-02-07', '2011-01-08',
'2013-01-12', '2014-01-12', '2014-01-19',
'2014-02-02', '2015-01-11', '2016-01-17',
'2016-01-24', '2016-02-07']),
'lower_window': 0,
'upper_window': 1,
})
superbowls = pd.DataFrame({
'holiday': 'superbowl',
'ds': pd.to_datetime(['2010-02-07', '2014-02-02', '2016-02-07']),
'lower_window': 0,
'upper_window': 1,
})
holidays = pd.concat((playoffs, superbowls))Above we have included the superbowl days as both playoff games and superbowl games. This means that the superbowl effect will be an additional additive bonus on top of the playoff effect.
Once the table is created, holiday effects are included in the forecast by passing them in with the holidays argument. Here we do it with the Peyton Manning data from the Quickstart:
# R
m <- prophet(df, holidays = holidays)
forecast <- predict(m, future)# Python
m = Prophet(holidays=holidays)
forecast = m.fit(df).predict(future)The holiday effect can be seen in the forecast dataframe:
# R
forecast %>%
select(ds, playoff, superbowl) %>%
filter(abs(playoff + superbowl) > 0) %>%
tail(10)# Python
forecast[(forecast['playoff'] + forecast['superbowl']).abs() > 0][
['ds', 'playoff', 'superbowl']][-10:]| ds | playoff | superbowl | |
|---|---|---|---|
| 2190 | 2014-02-02 | 1.223965 | 1.201517 |
| 2191 | 2014-02-03 | 1.901742 | 1.460471 |
| 2532 | 2015-01-11 | 1.223965 | 0.000000 |
| 2533 | 2015-01-12 | 1.901742 | 0.000000 |
| 2901 | 2016-01-17 | 1.223965 | 0.000000 |
| 2902 | 2016-01-18 | 1.901742 | 0.000000 |
| 2908 | 2016-01-24 | 1.223965 | 0.000000 |
| 2909 | 2016-01-25 | 1.901742 | 0.000000 |
| 2922 | 2016-02-07 | 1.223965 | 1.201517 |
| 2923 | 2016-02-08 | 1.901742 | 1.460471 |
The holiday effects will also show up in the components plot, where we see that there is a spike on the days around playoff appearances, with an especially large spike for the superbowl:
# R
prophet_plot_components(m, forecast)# Python
fig = m.plot_components(forecast)Individual holidays can be plotted using the plot_forecast_component function (imported from prophet.plot in Python) like plot_forecast_component(m, forecast, 'superbowl') to plot just the superbowl holiday component.
Built-in Country Holidays
You can use a built-in collection of country-specific holidays using the add_country_holidays method (Python) or function (R). The name of the country is specified, and then major holidays for that country will be included in addition to any holidays that are specified via the holidays argument described above:
# R
m <- prophet(holidays = holidays)
m <- add_country_holidays(m, country_name = 'US')
m <- fit.prophet(m, df)# Python
m = Prophet(holidays=holidays)
m.add_country_holidays(country_name='US')
m.fit(df)You can see which holidays were included by looking at the train_holiday_names (Python) or train.holiday.names (R) attribute of the model:
# R
m$train.holiday.names[1] "playoff" "superbowl"
[3] "New Year's Day" "Martin Luther King Jr. Day"
[5] "Washington's Birthday" "Memorial Day"
[7] "Independence Day" "Labor Day"
[9] "Columbus Day" "Veterans Day"
[11] "Veterans Day (Observed)" "Thanksgiving"
[13] "Christmas Day" "Independence Day (Observed)"
[15] "Christmas Day (Observed)" "New Year's Day (Observed)"# Python
m.train_holiday_names0 playoff
1 superbowl
2 New Year's Day
3 Martin Luther King Jr. Day
4 Washington's Birthday
5 Memorial Day
6 Independence Day
7 Labor Day
8 Columbus Day
9 Veterans Day
10 Thanksgiving
11 Christmas Day
12 Christmas Day (Observed)
13 Veterans Day (Observed)
14 Independence Day (Observed)
15 New Year's Day (Observed)
dtype: objectThe holidays for each country are provided by the holidays package in Python. A list of available countries, and the country name to use, is available on their page: https://github.com/vacanza/python-holidays/.
In Python, most holidays are computed deterministically and so are available for any date range; a warning will be raised if dates fall outside the range supported by that country. In R, holiday dates are computed for 1995 through 2044 and stored in the package as data-raw/generated_holidays.csv. If a wider date range is needed, this script can be used to replace that file with a different date range: https://github.com/facebook/prophet/blob/main/python/scripts/generate_holidays_file.py.
As above, the country-level holidays will then show up in the components plot:
# R
forecast <- predict(m, future)
prophet_plot_components(m, forecast)# Python
forecast = m.predict(future)
fig = m.plot_components(forecast)Holidays for subdivisions (Python)
We can use the utility function make_holidays_df to easily create custom holidays DataFrames, e.g. for certain states, using data from the holidays package. This can be passed directly to the Prophet() constructor.
# Python
from prophet.make_holidays import make_holidays_df
nsw_holidays = make_holidays_df(
year_list=[2019 + i for i in range(10)], country='AU', province='NSW'
)
nsw_holidays.head(n=10)| ds | holiday | |
|---|---|---|
| 0 | 2019-01-26 | Australia Day |
| 1 | 2019-01-28 | Australia Day (Observed) |
| 2 | 2019-04-25 | Anzac Day |
| 3 | 2019-12-25 | Christmas Day |
| 4 | 2019-12-26 | Boxing Day |
| 5 | 2019-04-20 | Easter Saturday |
| 6 | 2019-04-21 | Easter Sunday |
| 7 | 2019-10-07 | Labour Day |
| 8 | 2019-06-10 | Queen's Birthday |
| 9 | 2019-08-05 | Bank Holiday |
# Python
from prophet import Prophet
m_nsw = Prophet(holidays=nsw_holidays)Fourier Order for Seasonalities
Seasonalities are estimated using a partial Fourier sum. See the paper for complete details, and this figure on Wikipedia for an illustration of how a partial Fourier sum can approximate an arbitrary periodic signal. The number of terms in the partial sum (the order) is a parameter that determines how quickly the seasonality can change. To illustrate this, consider the Peyton Manning data from the Quickstart. The default Fourier order for yearly seasonality is 10, which produces this fit:
# R
m <- prophet(df)
prophet:::plot_yearly(m)# Python
from prophet.plot import plot_yearly
m = Prophet().fit(df)
a = plot_yearly(m)The default values are often appropriate, but they can be increased when the seasonality needs to fit higher-frequency changes, and generally be less smooth. The Fourier order can be specified for each built-in seasonality when instantiating the model, here it is increased to 20:
# R
m <- prophet(df, yearly.seasonality = 20)
prophet:::plot_yearly(m)# Python
from prophet.plot import plot_yearly
m = Prophet(yearly_seasonality=20).fit(df)
a = plot_yearly(m)Increasing the number of Fourier terms allows the seasonality to fit faster changing cycles, but can also lead to overfitting: N Fourier terms corresponds to 2N variables used for modeling the cycle
Specifying Custom Seasonalities
Prophet will by default fit weekly and yearly seasonalities, if the time series is more than two cycles long. It will also fit daily seasonality for a sub-daily time series. You can add other seasonalities (monthly, quarterly, hourly) using the add_seasonality method (Python) or function (R).
The inputs to this function are a name, the period of the seasonality in days, and the Fourier order for the seasonality. For reference, by default Prophet uses a Fourier order of 3 for weekly seasonality and 10 for yearly seasonality. An optional input to add_seasonality is the prior scale for that seasonal component - this is discussed below.
As an example, here we fit the Peyton Manning data from the Quickstart, but replace the weekly seasonality with monthly seasonality. The monthly seasonality then will appear in the components plot:
# R
m <- prophet(weekly.seasonality=FALSE)
m <- add_seasonality(m, name='monthly', period=30.5, fourier.order=5)
m <- fit.prophet(m, df)
forecast <- predict(m, future)
prophet_plot_components(m, forecast)# Python
m = Prophet(weekly_seasonality=False)
m.add_seasonality(name='monthly', period=30.5, fourier_order=5)
forecast = m.fit(df).predict(future)
fig = m.plot_components(forecast)Seasonalities that depend on other factors
In some instances the seasonality may depend on other factors, such as a weekly seasonal pattern that is different during the summer than it is during the rest of the year, or a daily seasonal pattern that is different on weekends vs. on weekdays. These types of seasonalities can be modeled using conditional seasonalities.
Consider the Peyton Manning example from the Quickstart. The default weekly seasonality assumes that the pattern of weekly seasonality is the same throughout the year, but we'd expect the pattern of weekly seasonality to be different during the on-season (when there are games every Sunday) and the off-season. We can use conditional seasonalities to construct separate on-season and off-season weekly seasonalities.
First we add a boolean column to the dataframe that indicates whether each date is during the on-season or the off-season:
# R
is_nfl_season <- function(ds) {
dates <- as.Date(ds)
month <- as.numeric(format(dates, '%m'))
return(month > 8 | month < 2)
}
df$on_season <- is_nfl_season(df$ds)
df$off_season <- !is_nfl_season(df$ds)# Python
def is_nfl_season(ds):
date = pd.to_datetime(ds)
return (date.month > 8 or date.month < 2)
df['on_season'] = df['ds'].apply(is_nfl_season)
df['off_season'] = ~df['ds'].apply(is_nfl_season)Then we disable the built-in weekly seasonality, and replace it with two weekly seasonalities that have these columns specified as a condition. This means that the seasonality will only be applied to dates where the condition_name column is True. We must also add the column to the future dataframe for which we are making predictions.
# R
m <- prophet(weekly.seasonality=FALSE)
m <- add_seasonality(m, name='weekly_on_season', period=7, fourier.order=3, condition.name='on_season')
m <- add_seasonality(m, name='weekly_off_season', period=7, fourier.order=3, condition.name='off_season')
m <- fit.prophet(m, df)
future$on_season <- is_nfl_season(future$ds)
future$off_season <- !is_nfl_season(future$ds)
forecast <- predict(m, future)
prophet_plot_components(m, forecast)# Python
m = Prophet(weekly_seasonality=False)
m.add_seasonality(name='weekly_on_season', period=7, fourier_order=3, condition_name='on_season')
m.add_seasonality(name='weekly_off_season', period=7, fourier_order=3, condition_name='off_season')
future['on_season'] = future['ds'].apply(is_nfl_season)
future['off_season'] = ~future['ds'].apply(is_nfl_season)
forecast = m.fit(df).predict(future)
fig = m.plot_components(forecast)Both of the seasonalities now show up in the components plots above. We can see that during the on-season when games are played every Sunday, there are large increases on Sunday and Monday that are completely absent during the off-season.
Prior scale for holidays and seasonality
If you find that the holidays are overfitting, you can adjust their prior scale to smooth them using the parameter holidays_prior_scale. By default this parameter is 10, which provides very little regularization. Reducing this parameter dampens holiday effects:
# R
m <- prophet(df, holidays = holidays, holidays.prior.scale = 0.05)
forecast <- predict(m, future)
forecast %>%
select(ds, playoff, superbowl) %>%
filter(abs(playoff + superbowl) > 0) %>%
tail(10)# Python
m = Prophet(holidays=holidays, holidays_prior_scale=0.05).fit(df)
forecast = m.predict(future)
forecast[(forecast['playoff'] + forecast['superbowl']).abs() > 0][
['ds', 'playoff', 'superbowl']][-10:]| ds | playoff | superbowl | |
|---|---|---|---|
| 2190 | 2014-02-02 | 1.206086 | 0.964914 |
| 2191 | 2014-02-03 | 1.852077 | 0.992634 |
| 2532 | 2015-01-11 | 1.206086 | 0.000000 |
| 2533 | 2015-01-12 | 1.852077 | 0.000000 |
| 2901 | 2016-01-17 | 1.206086 | 0.000000 |
| 2902 | 2016-01-18 | 1.852077 | 0.000000 |
| 2908 | 2016-01-24 | 1.206086 | 0.000000 |
| 2909 | 2016-01-25 | 1.852077 | 0.000000 |
| 2922 | 2016-02-07 | 1.206086 | 0.964914 |
| 2923 | 2016-02-08 | 1.852077 | 0.992634 |
The magnitude of the holiday effect has been reduced compared to before, especially for superbowls, which had the fewest observations. There is a parameter seasonality_prior_scale which similarly adjusts the extent to which the seasonality model will fit the data.
Prior scales can be set separately for individual holidays by including a column prior_scale in the holidays dataframe. Prior scales for individual seasonalities can be passed as an argument to add_seasonality. For instance, the prior scale for just weekly seasonality can be set using:
# R
m <- prophet()
m <- add_seasonality(
m, name='weekly', period=7, fourier.order=3, prior.scale=0.1)# Python
m = Prophet()
m.add_seasonality(
name='weekly', period=7, fourier_order=3, prior_scale=0.1)Additional regressors
Additional regressors can be added to the linear part of the model using the add_regressor method or function. A column with the regressor value will need to be present in both the fitting and prediction dataframes. For example, we can add an additional effect on Sundays during the NFL season. On the components plot, this effect will show up in the 'extra_regressors' plot:
# R
nfl_sunday <- function(ds) {
dates <- as.Date(ds)
month <- as.numeric(format(dates, '%m'))
as.numeric((weekdays(dates) == "Sunday") & (month > 8 | month < 2))
}
df$nfl_sunday <- nfl_sunday(df$ds)
m <- prophet()
m <- add_regressor(m, 'nfl_sunday')
m <- fit.prophet(m, df)
future$nfl_sunday <- nfl_sunday(future$ds)
forecast <- predict(m, future)
prophet_plot_components(m, forecast)# Python
def nfl_sunday(ds):
date = pd.to_datetime(ds)
if date.weekday() == 6 and (date.month > 8 or date.month < 2):
return 1
else:
return 0
df['nfl_sunday'] = df['ds'].apply(nfl_sunday)
m = Prophet()
m.add_regressor('nfl_sunday')
m.fit(df)
future['nfl_sunday'] = future['ds'].apply(nfl_sunday)
forecast = m.predict(future)
fig = m.plot_components(forecast)NFL Sundays could also have been handled using the "holidays" interface described above, by creating a list of past and future NFL Sundays. The add_regressor function provides a more general interface for defining extra linear regressors, and in particular does not require that the regressor be a binary indicator. Another time series could be used as a regressor, although its future values would have to be known.
This notebook shows an example of using weather factors as extra regressors in a forecast of bicycle usage, and provides an excellent illustration of how other time series can be included as extra regressors.
The add_regressor function has optional arguments for specifying the prior scale (holiday prior scale is used by default) and whether or not the regressor is standardized - see the docstring with help(Prophet.add_regressor) in Python and ?add_regressor in R. Note that regressors must be added prior to model fitting. Prophet will also raise an error if the regressor is constant throughout the history, since there is nothing to fit from it.
The extra regressor must be known for both the history and for future dates. It thus must either be something that has known future values (such as nfl_sunday), or something that has separately been forecasted elsewhere. The weather regressors used in the notebook linked above is a good example of an extra regressor that has forecasts that can be used for future values.
One can also use as a regressor another time series that has been forecasted with a time series model, such as Prophet. For instance, if r(t) is included as a regressor for y(t), Prophet can be used to forecast r(t) and then that forecast can be plugged in as the future values when forecasting y(t). A note of caution around this approach: This will probably not be useful unless r(t) is somehow easier to forecast than y(t). This is because error in the forecast of r(t) will produce error in the forecast of y(t). One setting where this can be useful is in hierarchical time series, where there is top-level forecast that has higher signal-to-noise and is thus easier to forecast. Its forecast can be included in the forecast for each lower-level series.
Extra regressors are put in the linear component of the model, so the underlying model is that the time series depends on the extra regressor as either an additive or multiplicative factor (see the next section for multiplicativity).
Coefficients of additional regressors
To extract the beta coefficients of the extra regressors, use the utility function regressor_coefficients (from prophet.utilities import regressor_coefficients in Python, prophet::regressor_coefficients in R) on the fitted model. The estimated beta coefficient for each regressor roughly represents the increase in prediction value for a unit increase in the regressor value (note that the coefficients returned are always on the scale of the original data). If mcmc_samples is specified, a credible interval for each coefficient is also returned, which can help identify whether the regressor is meaningful to the model (a credible interval that includes the 0 value suggests the regressor is not meaningful).
Uncertainty in future regressor values
In the examples above, regressor values are linked to holidays / calendar events, so their future values are known and can be set directly in the future DataFrame and passed to .predict().
In multi-stage prediction pipelines, future regressor values are unknown and carry uncertainty. As of Prophet v1.4.0 there is support for nested Prophet models (one per extra_regressor) that integrate natively with .predict() and cross_validate().
In
.add_regressor(), setpredict_specto enable a nested Prophet model for that regressor.predictor_spec=Trueuses defaults; pass a dictionary likepredict_spec={'changepoint_range': 0.7}for a custom config. Note that we only support simple nested Prophet models, so models that require custom seasonalities with.add_seasonality()or their own regressors via.add_regressor()are not supported.When running
predict()on the main model, regressor values are filled with theyhatvalues from their respective nested Prophet models.When running
cross_validate()on the main model, each run will first predict the regressor values for the lookahead range, and those values are used to predictyfor the lookahead range. This prevents data leakage and overstated cross-validation results.
# Python
import logging
logging.basicConfig(level=logging.INFO)
import numpy as np
import pandas as pd
from prophet import Prophet
from prophet.diagnostics import cross_validation, performance_metrics
pedestrians = (
pd.read_csv(
'https://raw.githubusercontent.com/facebook/prophet/main/examples/example_pedestrians_multivariate.csv',
parse_dates=["ds"],
)
.astype({"location_4": np.float64, "location_41": np.float64})
.rename({"location_41": "y"}, axis=1)
)
target = "location_41"
test_mask = pedestrians["ds"] >= pd.to_datetime("2023-04-24 00:00:00")
pedestrians_train = pedestrians.loc[~test_mask]
pedestrians_test = pedestrians.loc[test_mask]# Python
m1 = Prophet(
changepoint_range=0.9,
seasonality_prior_scale=5.0,
)
m1.add_regressor("location_4", mode="additive", regressor_predictor={"seasonality_prior_scale": 10.0});# Python
m1.fit(pedestrians_train);INFO:prophet:Disabling yearly seasonality. Run prophet with yearly_seasonality=True to override this.
INFO:prophet:Fitting regressor model 'location_4' with 552 observations
INFO:prophet:Disabling yearly seasonality. Run prophet with yearly_seasonality=True to override this.
INFO:cmdstanpy:Chain [1] start processing
INFO:cmdstanpy:Chain [1] done processing
INFO:cmdstanpy:Chain [1] start processing
INFO:cmdstanpy:Chain [1] done processing# Python
preds = m1.predict(pedestrians_test)INFO:prophet:Running regressor model 'location_4' for 168 future dates
INFO:prophet:Collected 1000 predictive samples for regressor 'location_4'# Python
m1.plot(preds);# Python
m1.plot_components(preds);# Python
m1_cv = cross_validation(
m1,
horizon=pd.Timedelta(3, unit="D"),
period=pd.Timedelta(5, unit="D"),
initial=pd.Timedelta(14, unit="D"),
)INFO:prophet:Making 2 forecasts with cutoffs between 2023-04-15 23:00:00 and 2023-04-20 23:00:00
0%| | 0/2 [00:00<?, ?it/s]
INFO:prophet:Fitting regressor model 'location_4' with 360 observations
INFO:prophet:Disabling yearly seasonality. Run prophet with yearly_seasonality=True to override this.
INFO:cmdstanpy:Chain [1] start processing
INFO:cmdstanpy:Chain [1] done processing
INFO:cmdstanpy:Chain [1] start processing
INFO:cmdstanpy:Chain [1] done processing
INFO:prophet:Running regressor model 'location_4' for 72 future dates
INFO:prophet:Collected 1000 predictive samples for regressor 'location_4'
INFO:prophet:Fitting regressor model 'location_4' with 480 observations
INFO:prophet:Disabling yearly seasonality. Run prophet with yearly_seasonality=True to override this.
INFO:cmdstanpy:Chain [1] start processing
INFO:cmdstanpy:Chain [1] done processing
INFO:cmdstanpy:Chain [1] start processing
INFO:cmdstanpy:Chain [1] done processing
INFO:prophet:Running regressor model 'location_4' for 72 future dates
INFO:prophet:Collected 1000 predictive samples for regressor 'location_4'# Python
performance_metrics(m1_cv).tail(n=5).round(4)| horizon | mse | rmse | mae | mape | mdape | smape | coverage | |
|---|---|---|---|---|---|---|---|---|
| 61 | 2 days 20:00:00 | 121911.5202 | 349.1583 | 230.5407 | 0.0627 | 0.0506 | 0.0654 | 0.9286 |
| 62 | 2 days 21:00:00 | 120364.2551 | 346.9355 | 233.8541 | 0.0707 | 0.0547 | 0.0722 | 0.9286 |
| 63 | 2 days 22:00:00 | 117161.5735 | 342.2887 | 217.8324 | 0.0674 | 0.0539 | 0.0689 | 0.9286 |
| 64 | 2 days 23:00:00 | 127608.6985 | 357.2236 | 239.9352 | 0.0878 | 0.0601 | 0.0866 | 0.9286 |
| 65 | 3 days 00:00:00 | 192080.9740 | 438.2704 | 308.6779 | 0.2583 | 0.0734 | 0.1865 | 0.8571 |
# PythonDocs Trend Changepoints (`docs/_docs/trend_changepoints.md`)
layout: docs
docid: "trend_changepoints"
title: "Trend Changepoints"
permalink: /docs/trend_changepoints.html
subsections:
- title: Automatic changepoint detection in Prophet
id: automatic-changepoint-detection-in-prophet - title: Adjusting trend flexibility
id: adjusting-trend-flexibility - title: Specifying the locations of the changepoints
id: specifying-the-locations-of-the-changepoints
You may have noticed in the earlier examples in this documentation that real time series frequently have abrupt changes in their trajectories. By default, Prophet will automatically detect these changepoints and will allow the trend to adapt appropriately. However, if you wish to have finer control over this process (e.g., Prophet missed a rate change, or is overfitting rate changes in the history), then there are several input arguments you can use.
Automatic changepoint detection in Prophet
Prophet detects changepoints by first specifying a large number of potential changepoints at which the rate is allowed to change. It then puts a sparse prior on the magnitudes of the rate changes (equivalent to L1 regularization) - this essentially means that Prophet has a large number of possible places where the rate can change, but will use as few of them as possible. Consider the Peyton Manning forecast from the Quickstart. By default, Prophet specifies 25 potential changepoints which are uniformly placed in the first 80% of the time series. The vertical lines in this figure indicate where the potential changepoints were placed:
Even though we have a lot of places where the rate can possibly change, because of the sparse prior, most of these changepoints go unused. We can see this by plotting the magnitude of the rate change at each changepoint:
The number of potential changepoints can be set using the argument n_changepoints, but this is better tuned by adjusting the regularization. The locations of the signification changepoints can be visualized with:
# R
plot(m, forecast) + add_changepoints_to_plot(m)# Python
from prophet.plot import add_changepoints_to_plot
fig = m.plot(forecast)
a = add_changepoints_to_plot(fig.gca(), m, forecast)By default changepoints are only inferred for the first 80% of the time series in order to have plenty of runway for projecting the trend forward and to avoid overfitting fluctuations at the end of the time series. This default works in many situations but not all, and can be changed using the changepoint_range argument. For example, m = Prophet(changepoint_range=0.9) in Python or m <- prophet(changepoint.range = 0.9) in R will place potential changepoints in the first 90% of the time series.
Adjusting trend flexibility
If the trend changes are being overfit (too much flexibility) or underfit (not enough flexibility), you can adjust the strength of the sparse prior using the input argument changepoint_prior_scale. By default, this parameter is set to 0.05. Increasing it will make the trend more flexible:
# R
m <- prophet(df, changepoint.prior.scale = 0.5)
forecast <- predict(m, future)
plot(m, forecast)# Python
m = Prophet(changepoint_prior_scale=0.5)
forecast = m.fit(df).predict(future)
fig = m.plot(forecast)Decreasing it will make the trend less flexible:
# R
m <- prophet(df, changepoint.prior.scale = 0.001)
forecast <- predict(m, future)
plot(m, forecast)# Python
m = Prophet(changepoint_prior_scale=0.001)
forecast = m.fit(df).predict(future)
fig = m.plot(forecast)When visualizing the forecast, this parameter can be adjusted as needed if the trend seems to be over- or under-fit. In the fully-automated setting, see the documentation on cross validation for recommendations on how this parameter can be tuned.
Specifying the locations of the changepoints
If you wish, rather than using automatic changepoint detection you can manually specify the locations of potential changepoints with the changepoints argument. Slope changes will then be allowed only at these points, with the same sparse regularization as before. One could, for instance, create a grid of points as is done automatically, but then augment that grid with some specific dates that are known to be likely to have changes. As another example, the changepoints could be entirely limited to a small set of dates, as is done here:
# R
m <- prophet(df, changepoints = c('2014-01-01'))
forecast <- predict(m, future)
plot(m, forecast)# Python
m = Prophet(changepoints=['2014-01-01'])
forecast = m.fit(df).predict(future)
fig = m.plot(forecast)Docs Uncertainty Intervals (`docs/_docs/uncertainty_intervals.md`)
layout: docs
docid: "uncertainty_intervals"
title: "Uncertainty Intervals"
permalink: /docs/uncertainty_intervals.html
subsections:
- title: Uncertainty in the trend
id: uncertainty-in-the-trend - title: Uncertainty in seasonality
id: uncertainty-in-seasonality
By default Prophet will return uncertainty intervals for the forecast yhat. There are several important assumptions behind these uncertainty intervals.
There are three sources of uncertainty in the forecast: uncertainty in the trend, uncertainty in the seasonality estimates, and additional observation noise.
Uncertainty in the trend
The biggest source of uncertainty in the forecast is the potential for future trend changes. The time series we have seen already in this documentation show clear trend changes in the history. Prophet is able to detect and fit these, but what trend changes should we expect moving forward? It's impossible to know for sure, so we do the most reasonable thing we can, and we assume that the future will see similar trend changes as the history. In particular, we assume that the average frequency and magnitude of trend changes in the future will be the same as that which we observe in the history. We project these trend changes forward and by computing their distribution we obtain uncertainty intervals.
One property of this way of measuring uncertainty is that allowing higher flexibility in the rate, by increasing changepoint_prior_scale, will increase the forecast uncertainty. This is because if we model more rate changes in the history then we will expect more in the future, and makes the uncertainty intervals a useful indicator of overfitting.
The width of the uncertainty intervals (by default 80%) can be set using the parameter interval_width:
# R
m <- prophet(df, interval.width = 0.95)
forecast <- predict(m, future)# Python
forecast = Prophet(interval_width=0.95).fit(df).predict(future)Again, these intervals assume that the future will see the same frequency and magnitude of rate changes as the past. This assumption is probably not true, so you should not expect to get accurate coverage on these uncertainty intervals.
Uncertainty in seasonality
By default Prophet will only return uncertainty in the trend and observation noise. To get uncertainty in seasonality, you must do full Bayesian sampling. This is done using the parameter mcmc.samples (which defaults to 0). We do this here for the first six months of the Peyton Manning data from the Quickstart:
# R
m <- prophet(df, mcmc.samples = 300)
forecast <- predict(m, future)# Python
m = Prophet(mcmc_samples=300)
forecast = m.fit(df, show_progress=False).predict(future)This replaces the typical MAP estimation with MCMC sampling, and can take much longer depending on how many observations there are - expect several minutes instead of several seconds. If you do full sampling, then you will see the uncertainty in seasonal components when you plot them:
# R
prophet_plot_components(m, forecast)# Python
fig = m.plot_components(forecast)You can access the raw posterior predictive samples in Python using the method m.predictive_samples(future), or in R using the function predictive_samples(m, future).
--- METRICS ---
- Files Extracted: 26
- Estimated Token Budget: ~43708 tokens
- Canonical Reference: https://codewiki.google/github.com/facebook/prophet