# Technical Documentation: nteract/papermill
> ℹ️ **Provenance:** Multi-Tier Ingestion · [CodeWiki Reference](https://codewiki.google/github.com/nteract/papermill)
### Overview (`README.md`)
#
[](https://github.com/nteract/papermill/actions/workflows/ci.yml)
[](https://github.com/nteract/papermill/actions/workflows/ci.yml)
[](https://codecov.io/github/nteract/papermill?branch=main)
[](http://papermill.readthedocs.io/en/latest/?badge=latest)
[](https://mybinder.org/v2/gh/nteract/papermill/main?filepath=binder%2Fprocess_highlight_dates.ipynb)
[](https://mybinder.org/v2/gh/nteract/papermill/main?)
[](https://pypi.org/project/papermill/)
[](https://github.com/ambv/black)
[](https://snyk.io/advisor/python/papermill)
[](https://anaconda.org/conda-forge/papermill)
[](https://results.pre-commit.ci/latest/github/nteract/papermill/main)
**papermill** is a tool for parameterizing, executing, and analyzing
Jupyter Notebooks.
Papermill lets you:
- **parameterize** notebooks
- **execute** notebooks
This opens up new opportunities for how notebooks can be used. For
example:
- Perhaps you have a financial report that you wish to run with
different values on the first or last day of a month or at the
beginning or end of the year, **using parameters** makes this task
easier.
- Do you want to run a notebook and depending on its results, choose a
particular notebook to run next? You can now programmatically
**execute a workflow** without having to copy and paste from
notebook to notebook manually.
Papermill takes an *opinionated* approach to notebook parameterization and
execution based on our experiences using notebooks at scale in data
pipelines.
## Installation
From the command line:
```{.sourceCode .bash}
pip install papermill
```
For all optional io dependencies, you can specify individual bundles
like `s3`, or `azure` -- or use `all`. To use Black to format parameters you can add as an extra requires ['black'].
```{.sourceCode .bash}
pip install papermill[all]
```
## Python Version Support
This library currently supports Python 3.10+ versions. As minor Python
versions are officially sunset by the Python org papermill will similarly
drop support in the future.
## Usage
### Parameterizing a Notebook
To parameterize your notebook designate a cell with the tag `parameters`.
Papermill looks for the `parameters` cell and treats this cell as defaults for the parameters passed in at execution time. Papermill will add a new cell tagged with `injected-parameters` with input parameters in order to overwrite the values in `parameters`. If no cell is tagged with `parameters` the injected cell will be inserted at the top of the notebook.
Additionally, if you rerun notebooks through papermill and it will reuse the `injected-parameters` cell from the prior run. In this case Papermill will replace the old `injected-parameters` cell with the new run's inputs.
### Executing a Notebook
The two ways to execute the notebook with parameters are: (1) through
the Python API and (2) through the command line interface.
#### Execute via the Python API
```{.sourceCode .python}
import papermill as pm
pm.execute_notebook(
'path/to/input.ipynb',
'path/to/output.ipynb',
parameters = dict(alpha=0.6, ratio=0.1)
)
```
#### Execute via CLI
Here's an example of a local notebook being executed and output to an
Amazon S3 account:
```{.sourceCode .bash}
$ papermill local/input.ipynb s3://bkt/output.ipynb -p alpha 0.6 -p l1_ratio 0.1
```
**NOTE:**
If you use multiple AWS accounts, and you have [properly configured your AWS credentials](https://boto3.amazonaws.com/v1/documentation/api/latest/guide/configuration.html), then you can specify which account to use by setting the `AWS_PROFILE` environment variable at the command-line. For example:
```{.sourceCode .bash}
$ AWS_PROFILE=dev_account papermill local/input.ipynb s3://bkt/output.ipynb -p alpha 0.6 -p l1_ratio 0.1
```
In the above example, two parameters are set: `alpha` and `l1_ratio` using `-p` (`--parameters` also works). Parameter values that look like booleans or numbers will be interpreted as such. Here are the different ways users may set parameters:
```{.sourceCode .bash}
$ papermill local/input.ipynb s3://bkt/output.ipynb -r version 1.0
```
Using `-r` or `--parameters_raw`, users can set parameters one by one. However, unlike `-p`, the parameter will remain a string, even if it may be interpreted as a number or boolean.
```{.sourceCode .bash}
$ papermill local/input.ipynb s3://bkt/output.ipynb -f parameters.yaml
```
Using `-f` or `--parameters_file`, users can provide a YAML file from which parameter values should be read.
```{.sourceCode .bash}
$ papermill local/input.ipynb s3://bkt/output.ipynb -y "
alpha: 0.6
l1_ratio: 0.1"
```
Using `-y` or `--parameters_yaml`, users can directly provide a YAML string containing parameter values.
```{.sourceCode .bash}
$ papermill local/input.ipynb s3://bkt/output.ipynb -b YWxwaGE6IDAuNgpsMV9yYXRpbzogMC4xCg==
```
Using `-b` or `--parameters_base64`, users can provide a YAML string, base64-encoded, containing parameter values.
When using YAML to pass arguments, through `-y`, `-b` or `-f`, parameter values can be arrays or dictionaries:
```{.sourceCode .bash}
$ papermill local/input.ipynb s3://bkt/output.ipynb -y "
x:
- 0.0
- 1.0
- 2.0
- 3.0
linear_function:
slope: 3.0
intercept: 1.0"
```
#### Supported Name Handlers
Papermill supports the following name handlers for input and output paths during execution:
- Local file system: `local`
- HTTP, HTTPS protocol: `http://, https://`
- Amazon Web Services: [AWS S3](https://aws.amazon.com/s3/) `s3://`
- Azure: [Azure DataLake Store](https://docs.microsoft.com/en-us/azure/data-lake-store/data-lake-store-overview), [Azure Blob Store](https://docs.microsoft.com/en-us/azure/storage/blobs/storage-blobs-overview) `adl://, abs://`
- Google Cloud: [Google Cloud Storage](https://cloud.google.com/storage/) `gs://`
## Development Guide
Read [CONTRIBUTING.md](./CONTRIBUTING.md) for guidelines on how to setup a local development environment and make code changes back to Papermill.
For development guidelines look in the [DEVELOPMENT_GUIDE.md](./DEVELOPMENT_GUIDE.md) file. This should inform you on how to make particular additions to the code base.
## Documentation
We host the [Papermill documentation](http://papermill.readthedocs.io)
on ReadTheDocs.
---
### Overview (`binder/cli-simple/README.md`)
# Simple CLI Example
This example uses three notebooks:
- simple_input.ipynb
- simple_output.ipynb
- cli_example.ipynb
1. Open the cli_example.ipynb notebook.
1. Begin executing the cells.
1. One cell demonstrates running papermill from the command line.
---
### UPDATE (`docs/UPDATE.md`)
TODO: Figure out make options needed for non-api changes
```
sphinx-apidoc -f -o reference ../papermill
```
---
### Changelog (`docs/changelog.md`)
# Change Log
## 2.7.0
### Highlights
- Drop Python 3.8 and 3.9, add Python 3.13 support
- Drop `ansicolors` dependency
- Modernize packaging to use `pyproject.toml`
### Changes
- Add Python 3.13 support, drop Python 3.8/3.9 [#828](https://github.com/nteract/papermill/pull/828)
- Drop the dependency on `ansicolors` [#822](https://github.com/nteract/papermill/pull/822)
- Replace deprecated `datetime.utcnow()` [#823](https://github.com/nteract/papermill/pull/823)
- Modernize packaging to use `pyproject.toml` [#837](https://github.com/nteract/papermill/pull/837)
- Changed parameter inspection to raise the same error messages as other pathways for missing kernel name and language
### Bug Fixes
- Fix failing tests in CI, pin `azure-datalake-store` [#826](https://github.com/nteract/papermill/pull/826)
- Fix skipped HDFS tests for Python 3.12 [#839](https://github.com/nteract/papermill/pull/839)
- Unskip tests that were previously failing [#846](https://github.com/nteract/papermill/pull/846)
### Dev / CI
- Add `pyproject-fmt` and `validate-pyproject` pre-commit hooks [#858](https://github.com/nteract/papermill/pull/858)
- Add Dependabot for GitHub Actions [#857](https://github.com/nteract/papermill/pull/857)
- Update pre-commit config [#843](https://github.com/nteract/papermill/pull/843)
- Update CONTRIBUTING.md [#842](https://github.com/nteract/papermill/pull/842) [#847](https://github.com/nteract/papermill/pull/847)
- Update docs/RTD configuration [#805](https://github.com/nteract/papermill/pull/805) [#836](https://github.com/nteract/papermill/pull/836)
- Bump CI actions to latest versions [#852](https://github.com/nteract/papermill/pull/852) [#853](https://github.com/nteract/papermill/pull/853) [#854](https://github.com/nteract/papermill/pull/854) [#855](https://github.com/nteract/papermill/pull/855)
## 2.6.0
- bring back strip_color and remove ANSI color codes from exception traceback [#791](https://github.com/nteract/papermill/pull/791)
- cleaned up documentation [#790](https://github.com/nteract/papermill/pull/790)
- prevent error override, fix traceback type [#788](https://github.com/nteract/papermill/pull/788)
- Upgrade tests to moto v5 [#779](https://github.com/nteract/papermill/pull/779)
- raise PapermillExecutionError when CellExecutionError is raised without cell error output [#786](https://github.com/nteract/papermill/pull/786)
- make progress_bar param accept a dict [#778](https://github.com/nteract/papermill/pull/778)
- Fix nbformat to 5.2.0 to cell None type [#770](https://github.com/nteract/papermill/pull/770)
- Use f-strings where possible [#762](https://github.com/nteract/papermill/pull/762)
- Unmark wheel as universal [#764](https://github.com/nteract/papermill/pull/764)
---
### Extending Developing (`docs/extending-developing.rst`)
.. _developing-papermill:
Extending papermill by contributing to it
=========================================
If you find that you'd like to not only add I/O and execution handlers, but
think a fundamental aspect of the project could use some improvement, then you
may want to contribute to it.
Development of papermill happens on github, and a
`detailed guide to contributing`_ to it can be found there. There is also a
`code of conduct`_ there. Please read both documents before beginning!
.. _`detailed guide to contributing`: https://github.com/nteract/papermill/blob/main/CONTRIBUTING.md
.. _`code of conduct`: https://github.com/nteract/nteract/blob/main/CODE_OF_CONDUCT.md
---
### Extending Entry Points (`docs/extending-entry-points.rst`)
Extending papermill through entry points
========================================
What are entry points?
----------------------
The python packaging documentation describes `entry points`_ as:
Entry points are a mechanism for an installed distribution to advertise
components it provides to be discovered and used by other code. For example:
Distributions can specify console_scripts entry points, each referring to a
function. When pip (or another console_scripts aware installer) installs the
distribution, it will create a command-line wrapper for each entry point.
Applications can use entry points to load plugins; e.g. Pygments (a syntax
highlighting tool) can use additional lexers and styles from separately
installed packages. For more about this, see Creating and discovering
plugins.
When running, papermill looks for `entry points`_ that implement input / output
(I/O) handlers, and execution handlers.
Developing new I/O handlers
---------------------------
Virtually the first thing that happens when papermill is used is that the input
notebook is read in. This is managed by I/O handlers, which allow papermill to
access not just the local filesystem, but also remote services such as Amazon
S3. The same goes for writing the executed notebook to a file system: I/O
handlers allow papermill to write files to S3 or otherwise.
Creating a new handler
~~~~~~~~~~~~~~~~~~~~~~
Writing your own I/O handler requires writing a class that has four
methods. All I/O handlers should implement the following class methods:
* ``CustomIO.read(file_path)``, returning the file content
* ``CustomIO.write(file_content, file_path)``, returning nothing
* ``CustomIO.pretty_path(path)``, returning a prettified path
* ``CustomIO.listdir(path)``, returning a list of paths.
.. note::
If you don't want to support things such as ``read`` because your I/O
handler is only intended for writing (such as a publish-only platform), then
you should implement the method but raise an exception when it is used.
Ensuring your handler is found by papermill
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Once you have developed a new handler, you need to declare papermill entry
points in your ``pyproject.toml`` file.
This is done by including the ``[project.entry-points."papermill.io"]`` section
in your pyproject.toml file:
.. code-block:: toml
[project.entry-points."papermill.io"]
"sftp://" = "papermill_sftp:SFTPHandler"
This indicates to papermill that when a file path begins with ``sftp://``, it
should use the class ``papermill_sftp.SFTPHandler`` to handle reading or writing
to that path. Anything before the equal sign is the path prefix, and everything
after it is the class to be used, including where it is imported from.
Traditionally, entry points for papermill I/O handlers look like URL prefixes.
For example, the Amazon Web Services S3 handler is registered under ``s3://``,
and so is used whenever a path begins with ``s3://``.
Example: sftp I/O handler
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
As an example, let's go through how we would create an I/O handler that reads
from an sftp server and writes back to it, so we could do the following::
papermill sftp://my_ftp_server.co.uk/input.ipynb sftp://my_ftp_server.co.uk/output.ipynb
Our project structure will look like this::
papermill_sftp
|- pyproject.toml
|- src
|- papermill_sftp
|- __init__.py
We can define the I/O handler in ``src/papermill_sftp/__init__.py``. To do so,
we have to create a class that does the relevant actions.
For reading, we will download the file to a temporary path and read it in from
there. For writing, we will write to a temporary path and upload it from there.
Prettifying the path doesn't need to change the path, and we are not going to
implement a listdir option for now.
.. code-block:: python
import os
import pysftp
sftp_username = os.getenv('SFTP_USERNAME')
sftp_password = os.getenv('SFTP_PASSWORD')
class SFTPHandler:
@classmethod
def read(cls, path):
"""
Read a notebook from an SFTP server.
"""
parsed_url = urllib.parse.urlparse(path)
with tempfile.TemporaryDirectory() as tmpdir:
tmp_file = pathlib.Path(tmpdir) / pathlib.Path(parsed_url.path).name
with pysftp.Connection(
parsed_url.hostname,
username=sftp_username,
password=sftp_password,
port=(parsed_url.port or 22),
cnopts=cnopts,
) as sftp:
sftp.get(parsed_url.path, str(tmp_file))
return tmp_file.read_text()
@classmethod
def write(cls, file_content, path):
"""
Write a notebook to an SFTP server.
"""
parsed_url = urllib.parse.urlparse(path)
with tempfile.TemporaryDirectory() as tmpdir:
tmp_file = pathlib.Path(tmpdir) / "output.ipynb"
tmp_file.write_text(file_content)
with pysftp.Connection(
parsed_url.hostname,
username=sftp_username,
password=sftp_password,
port=(parsed_url.port or 22),
cnopts=cnopts,
) as sftp:
sftp.put(str(tmp_file), parsed_url.path)
@classmethod
def pretty_path(cls, path):
return path
@classmethod
def listdir(cls, path):
raise NotImplementedError
The ``pyproject.toml`` file contains the following code:
.. code-block:: toml
[build-system]
requires = ["setuptools>=61.0", "wheel"]
build-backend = "setuptools.build_meta"
[project]
name = "papermill_sftp"
version = "0.1"
description = "An SFTP I/O handler for papermill."
authors = [
{name = "My Name", email = "my.email@gmail.com"}
]
dependencies = ["pysftp"]
[project.urls]
Repository = "https://github.com/my_username/papermill_sftp.git"
[project.entry-points."papermill.io"]
"sftp://" = "papermill_sftp:SFTPHandler"
[tool.setuptools]
packages = ["papermill_sftp"]
package-dir = {"" = "src"}
When executing, papermill will check if the input or output path begin with
``sftp://``, and if so, use the SFTPHandler from the papermill_sftp project.
Developing a new engine
-----------------------
A papermill engine is a python object that can run, or execute, a notebook. The
default implementation in papermill for example takes in a notebook object, and
runs it locally on your machine.
By writing a custom engine, you could allow execution to be handled remotely, or
you could apply post-processing to the executed notebook. In the next section,
you will see a demonstration.
Creating a new engine
~~~~~~~~~~~~~~~~~~~~~
Papermill engines need to inherit from the ``papermill.engines.Engine`` class.
In order to be used, the new class needs to implement the class method
``execute_managed_notebook``. The call signature should match that of the parent
class:
.. code-block:: python
class CustomEngine(papermill.engines.Engine):
@classmethod
def execute_managed_notebook(cls, nb_man, kernel_name, **kwargs):
pass
``nb_man`` is a |nbformat.NotebookNode|_, and ``kernel_name`` is a string. Your
custom class then needs to implement the execution of the notebook. For example,
you could insert code that executes the notebook remotely on a server, or
executes the notebook many times to simulate different conditions.
As an example, the following project implements a custom engine that adds the
time it took to execute each cell as additional output after every code cell.
The project structure is::
papermill_timing
|- pyproject.toml
|- src
|- papermill_timing
|- __init__.py
The file ``src/papermill_timing/__init__.py`` will implement the engine. Since
papermill already stores information about execution timing in the metadata,
we can leverage the default engine. We will also need to use the ``nbformat``
library to create a `notebook node object`_.
.. code-block:: python
from datetime import datetime
from papermill.engines import NBClientEngine
from nbformat.v4 import new_output
class CustomEngine(NBClientEngine):
@classmethod
def execute_managed_notebook(cls, nb_man, kernel_name, **kwargs):
# call the papermill execution engine:
super().execute_managed_notebook(nb_man, kernel_name, **kwargs)
for cell in nb_man.nb.cells:
if cell.cell_type == "code" and cell.execution_count is not None:
start = datetime.fromisoformat(cell.metadata.papermill.start_time)
end = datetime.fromisoformat(cell.metadata.papermill.end_time)
output_message = f"Execution took {(end - start).total_seconds():.3f} seconds"
output_node = new_output("display_data", data={"text/plain": [output_message]})
cell.outputs = [output_node] + cell.outputs
Once this is in place, we need to add our engine as an entry point to our
``pyproject.toml`` file - for this, see the following section.
Ensuring your engine is found by papermill
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Custom engines can be specified as `entry points`_, under the
``papermill.engine`` prefix. The entry point needs to reference the class that
we have just implemented. For example, if you write an engine called
TimingEngine in a package called papermill_timing, then in the ``pyproject.toml``
file, you should specify:
.. code-block:: toml
[build-system]
requires = ["setuptools>=61.0", "wheel"]
build-backend = "setuptools.build_meta"
[project]
name = "papermill_timing"
version = "0.1"
description = "A papermill engine that logs additional timing information about code."
authors = [
{name = "My Name", email = "my.email@gmail.com"}
]
dependencies = ["papermill", "nbformat"]
[project.urls]
Repository = "https://github.com/my_username/papermill_timing.git"
[project.entry-points."papermill.engine"]
timer_engine = "papermill_timing:CustomEngine"
[tool.setuptools]
packages = ["papermill_timing"]
package-dir = {"" = "src"}
This allows users to specify the engine from ``papermill_timing`` by passing the
command line argument ``--engine timer_engine``.
In the image below, the notebook on the left was executed with the new custom
engine, while the one on the left was executed with the standard papermill
engine. As you can see, this adds our "injected" output to each code cell
.. image:: img/custom_execution_engine.png
.. _`entry points`: https://packaging.python.org/specifications/entry-points/
.. |nbformat.NotebookNode| replace:: ``nbformat.NotebookNode`` object
.. _nbformat.NotebookNode: https://nbformat.readthedocs.io/en/latest/api.html#notebooknode-objects
.. _`notebook node object`: https://nbformat.readthedocs.io/en/latest/api.html#module-nbformat.v4
---
### Extending Overview (`docs/extending-overview.rst`)
Extending papermill
===================
Papermill provides some interfaces with external services out of the box.
However, you may find that you would like papermill to do more than it currently
does. You could contribute to the papermill project yourself (see
:ref:`developing-papermill`). However, an easier method might be to extend
papermill using `entry points`_.
In general, when you run a notebook with papermill, the following happens:
1. The notebook file is read in
2. The file content is converted to a notebook python object
3. The notebook is executed
4. The notebook is written to a file
Through entry points, you can write your own tools to handle steps 1, 3, and 4.
If you find that there's more you want to contribute to papermill, consider
developing papermill itself.
.. toctree::
:maxdepth: 2
extending-entry-points
extending-developing
.. _`entry points`: https://packaging.python.org/specifications/entry-points/
---
### Index (`docs/index.rst`)
Welcome to papermill
====================
.. image:: https://github.com/nteract/papermill/actions/workflows/ci.yml/badge.svg
:target: https://github.com/nteract/papermill/actions/workflows/ci.yml
.. image:: https://codecov.io/github/nteract/papermill/coverage.svg?branch=main
:target: https://codecov.io/github/nteract/papermill?branch=main
.. image:: https://readthedocs.org/projects/papermill/badge/?version=latest
:target: http://papermill.readthedocs.io/en/latest/?badge=latest
.. image:: https://tinyurl.com/ybwovtw2
:target: https://mybinder.org/v2/gh/nteract/papermill/main?filepath=binder%2Fprocess_highlight_dates.ipynb
.. image:: https://tinyurl.com/y7uz2eh9
:target: https://mybinder.org/v2/gh/nteract/papermill/main?filepath=binder%2Fcli-simple%2Fcli_example.ipynb
.. image:: https://img.shields.io/badge/code%20style-black-000000.svg
:target: https://github.com/ambv/black
**Papermill** is a tool for parameterizing and executing Jupyter Notebooks.
Papermill lets you:
* **parameterize** notebooks
* **execute** notebooks
This opens up new opportunities for how notebooks can be used. For example:
- Perhaps you have a financial report that you wish to run with different
values on the first or last day of a month or at the beginning or end
of the year, **using parameters** makes this task easier.
- Do you want to run a notebook and depending on its results,
choose a particular notebook to run next? You can now programmatically
**execute a workflow** without having to copy and paste from notebook to
notebook manually.
Python Version Support
----------------------
This library currently supports python 3.10+ versions. As minor python
versions are officially sunset by the python org papermill will similarly
drop support in the future.
Documentation
-------------
These pages guide you through the installation and usage of papermill.
.. toctree::
:maxdepth: 2
installation
usage-workflow
usage-cli
extending-overview
troubleshooting
changelog
API Reference
-------------
If you are looking for information about a specific function, class, or method,
this documentation section will help you.
.. toctree::
:maxdepth: 3
reference/index.rst
Indices and tables
------------------
* :ref:`genindex`
* :ref:`modindex`
* :ref:`search`
---
### Installation (`docs/installation.rst`)
Installation
============
Installing papermill
--------------------
From the command line:
.. code-block:: bash
python3 -m pip install papermill
Installing In-Notebook language bindings
----------------------------------------
In-Notebook language bindings provide helpers and utilities for using Papermill
with a programming language.
Python bindings
~~~~~~~~~~~~~~~
No additional installation steps are required since python bindings are built
into **papermill**.
---
### Troubleshooting (`docs/troubleshooting.rst`)
Troubleshooting
===============
NoSuchKernel Errors (using Conda)
---------------------------------
``NoSuchKernel`` Errors can appear when running papermill on jupyter notebooks whose
kernel has been specified via conda (nb_conda). nb_conda is used to easily set
conda environment per notebook from within jupyterlab.
To illustrate, the following example demonstrates the creation of a new
environment with all the dependencies necessary for an analysis.
::
conda create -n analysis_1 python=2 ipykernel
Once nb_conda is used within the jupyter server to set the kernel for a
notebook to *analysis_1*, the notebook gets metadata similar to the following:
.. code-block:: json
{
"kernelspec": {
"display_name": "Python [conda env:analysis_1]",
"language": "python",
"name": "conda-env-analysis_1-py"
}
}
Papermill cannot use this metadata to determine that it should use
*analysis_1* to execute this notebook. Running papermill (from *analysis_1* or
another environment) will raise the following error:
::
jupyter_client.kernelspec.NoSuchKernel: No such kernel named conda-env-analysis_1-py
This can be fixed by:
* Installing jupyter
(`or at least ipykernel `_)
in *analysis_1*
::
conda install -n analysis_1 jupyter
* Expose the *analysis_1* environment as a jupyter kernel
(`this is no longer automatic `_).
::
conda activate analysis_1
jupyter kernelspec install --user --name analysis_1
* Run papermill (from any environment) specifying the correct kernel using the
``-k`` option
::
papermill my_notebook.ipynb output_notebook.ipynb -k analysis_1
---
### Usage Cli (`docs/usage-cli.rst`)
Command Line Interface
======================
papermill may be executed from the terminal. The following are the command
options:
.. code-block:: bash
Usage: papermill [OPTIONS] NOTEBOOK_PATH [OUTPUT_PATH]
This utility executes a single notebook in a subprocess.
Papermill takes a source notebook, applies parameters to the source
notebook, executes the notebook with the specified kernel, and saves the
output in the destination notebook.
The NOTEBOOK_PATH and OUTPUT_PATH can now be replaced by `-` representing
stdout and stderr, or by the presence of pipe inputs / outputs. Meaning
that
`... | papermill | ...`
with `papermill - -` being implied by the pipes will read a notebook from
stdin and write it out to stdout.
Options:
--help-notebook Display parameters information for the given
notebook path.
-p, --parameters TEXT... Parameters to pass to the parameters cell.
-r, --parameters_raw TEXT... Parameters to be read as raw string.
-f, --parameters_file TEXT Path to YAML file containing parameters.
-y, --parameters_yaml TEXT YAML string to be used as parameters.
-b, --parameters_base64 TEXT Base64 encoded YAML string as parameters.
--inject-input-path Insert the path of the input notebook as
PAPERMILL_INPUT_PATH as a notebook
parameter.
--inject-output-path Insert the path of the output notebook as
PAPERMILL_OUTPUT_PATH as a notebook
parameter.
--inject-paths Insert the paths of input/output notebooks
as
PAPERMILL_INPUT_PATH/PAPERMILL_OUTPUT_PATH
as notebook parameters.
--engine TEXT The execution engine name to use in
evaluating the notebook.
--request-save-on-cell-execute / --no-request-save-on-cell-execute
Request save notebook after each cell
execution
--autosave-cell-every INTEGER How often in seconds to autosave the
notebook during long cell executions (0 to
disable)
--prepare-only / --prepare-execute
Flag for outputting the notebook without
execution, but with parameters applied.
-k, --kernel TEXT Name of kernel to run.
--cwd TEXT Working directory to run notebook in.
--progress-bar / --no-progress-bar
Flag for turning on the progress bar.
--log-output / --no-log-output Flag for writing notebook output to the
configured logger.
--stdout-file FILENAME File to write notebook stdout output to.
--stderr-file FILENAME File to write notebook stderr output to.
--log-level [NOTSET|DEBUG|INFO|WARNING|ERROR|CRITICAL]
Set log level
--start-timeout, --start_timeout INTEGER
Time in seconds to wait for kernel to start.
--execution-timeout INTEGER Time in seconds to wait for each cell before
failing execution (default: forever)
--report-mode / --no-report-mode
Flag for hiding input.
--version Flag for displaying the version.
-h, --help Show this message and exit.
---
### Usage Execute (`docs/usage-execute.rst`)
Execute
=======
The two ways to execute the notebook with parameters are: (1) through the
Python API and (2) through the command line interface.
Execute via the Python API
~~~~~~~~~~~~~~~~~~~~~~~~~~
The `execute_notebook` function can be called to execute an input notebook
when passed a dictionary of parameters:
.. code-block:: python
execute_notebook(,
