papermill Documentation

pip install papermill

pip install papermill[all]

import papermill as pm

pm.execute_notebook(
   'path/to/input.ipynb',
   'path/to/output.ipynb',
   parameters = dict(alpha=0.6, ratio=0.1)
)

$ papermill local/input.ipynb s3://bkt/output.ipynb -p alpha 0.6 -p l1_ratio 0.1

$ AWS_PROFILE=dev_account papermill local/input.ipynb s3://bkt/output.ipynb -p alpha 0.6 -p l1_ratio 0.1

$ papermill local/input.ipynb s3://bkt/output.ipynb -r version 1.0

$ papermill local/input.ipynb s3://bkt/output.ipynb -f parameters.yaml

$ papermill local/input.ipynb s3://bkt/output.ipynb -y "
alpha: 0.6
l1_ratio: 0.1"

$ papermill local/input.ipynb s3://bkt/output.ipynb -b YWxwaGE6IDAuNgpsMV9yYXRpbzogMC4xCg==

$ papermill local/input.ipynb s3://bkt/output.ipynb -y "
x:
    - 0.0
    - 1.0
    - 2.0
    - 3.0
linear_function:
    slope: 3.0
    intercept: 1.0"

sphinx-apidoc -f -o reference ../papermill

Entry points are a mechanism for an installed distribution to advertise
components it provides to be discovered and used by other code. For example:

Distributions can specify console_scripts entry points, each referring to a
function. When pip (or another console_scripts aware installer) installs the
distribution, it will create a command-line wrapper for each entry point.

Applications can use entry points to load plugins; e.g. Pygments (a syntax
highlighting tool) can use additional lexers and styles from separately
installed packages. For more about this, see Creating and discovering
plugins.

Writing your own I/O handler requires writing a class that has four
methods. All I/O handlers should implement the following class methods:

* ``CustomIO.read(file_path)``, returning the file content
* ``CustomIO.write(file_content, file_path)``, returning nothing
* ``CustomIO.pretty_path(path)``, returning a prettified path
* ``CustomIO.listdir(path)``, returning a list of paths.

.. note::

    If you don't want to support things such as ``read`` because your I/O
    handler is only intended for writing (such as a publish-only platform), then
    you should implement the method but raise an exception when it is used.

Ensuring your handler is found by papermill

[project.entry-points."papermill.io"]
"sftp://" = "papermill_sftp:SFTPHandler"

As an example, let's go through how we would create an I/O handler that reads
from an sftp server and writes back to it, so we could do the following::

    papermill sftp://my_ftp_server.co.uk/input.ipynb sftp://my_ftp_server.co.uk/output.ipynb

Our project structure will look like this::

    papermill_sftp
        |- pyproject.toml
        |- src
            |- papermill_sftp
                |- __init__.py


We can define the I/O handler in ``src/papermill_sftp/__init__.py``. To do so,
we have to create a class that does the relevant actions.

For reading, we will download the file to a temporary path and read it in from
there. For writing, we will write to a temporary path and upload it from there.
Prettifying the path doesn't need to change the path, and we are not going to
implement a listdir option for now.

.. code-block:: python

    import os
    import pysftp

    sftp_username = os.getenv('SFTP_USERNAME')
    sftp_password = os.getenv('SFTP_PASSWORD')

    class SFTPHandler:

        @classmethod
        def read(cls, path):
            """
            Read a notebook from an SFTP server.
            """
            parsed_url = urllib.parse.urlparse(path)
            with tempfile.TemporaryDirectory() as tmpdir:
                tmp_file = pathlib.Path(tmpdir) / pathlib.Path(parsed_url.path).name
                with pysftp.Connection(
                    parsed_url.hostname,
                    username=sftp_username,
                    password=sftp_password,
                    port=(parsed_url.port or 22),
                    cnopts=cnopts,
                ) as sftp:
                    sftp.get(parsed_url.path, str(tmp_file))
                return tmp_file.read_text()

        @classmethod
        def write(cls, file_content, path):
            """
            Write a notebook to an SFTP server.
            """
            parsed_url = urllib.parse.urlparse(path)
            with tempfile.TemporaryDirectory() as tmpdir:
                tmp_file = pathlib.Path(tmpdir) / "output.ipynb"
                tmp_file.write_text(file_content)
                with pysftp.Connection(
                    parsed_url.hostname,
                    username=sftp_username,
                    password=sftp_password,
                    port=(parsed_url.port or 22),
                    cnopts=cnopts,
                ) as sftp:
                    sftp.put(str(tmp_file), parsed_url.path)

        @classmethod
        def pretty_path(cls, path):
            return path

        @classmethod
        def listdir(cls, path):
            raise NotImplementedError


The ``pyproject.toml`` file contains the following code:

.. code-block:: toml

    [build-system]
    requires = ["setuptools>=61.0", "wheel"]
    build-backend = "setuptools.build_meta"

    [project]
    name = "papermill_sftp"
    version = "0.1"
    description = "An SFTP I/O handler for papermill."
    authors = [
        {name = "My Name", email = "[email protected]"}
    ]
    dependencies = ["pysftp"]

    [project.urls]
    Repository = "https://github.com/my_username/papermill_sftp.git"

    [project.entry-points."papermill.io"]
    "sftp://" = "papermill_sftp:SFTPHandler"

    [tool.setuptools]
    packages = ["papermill_sftp"]
    package-dir = {"" = "src"}

When executing, papermill will check if the input or output path begin with
``sftp://``, and if so, use the SFTPHandler from the papermill_sftp project.

Developing a new engine
-----------------------

A papermill engine is a python object that can run, or execute, a notebook. The
default implementation in papermill for example takes in a notebook object, and
runs it locally on your machine.

By writing a custom engine, you could allow execution to be handled remotely, or
you could apply post-processing to the executed notebook. In the next section,
you will see a demonstration.

Creating a new engine
~~~~~~~~~~~~~~~~~~~~~

Papermill engines need to inherit from the ``papermill.engines.Engine`` class.

In order to be used, the new class needs to implement the class method
``execute_managed_notebook``. The call signature should match that of the parent
class:

.. code-block:: python

    class CustomEngine(papermill.engines.Engine):

        @classmethod
        def execute_managed_notebook(cls, nb_man, kernel_name, **kwargs):
            pass

``nb_man`` is a |nbformat.NotebookNode|_, and ``kernel_name`` is a string. Your
custom class then needs to implement the execution of the notebook. For example,
you could insert code that executes the notebook remotely on a server, or
executes the notebook many times to simulate different conditions.

As an example, the following project implements a custom engine that adds the
time it took to execute each cell as additional output after every code cell.

The project structure is::

    papermill_timing
        |- pyproject.toml
        |- src
            |- papermill_timing
                |- __init__.py


The file ``src/papermill_timing/__init__.py`` will implement the engine. Since
papermill already stores information about execution timing in the metadata,
we can leverage the default engine. We will also need to use the ``nbformat``
library to create a `notebook node object`_.

.. code-block:: python

    from datetime import datetime
    from papermill.engines import NBClientEngine
    from nbformat.v4 import new_output

    class CustomEngine(NBClientEngine):

        @classmethod
        def execute_managed_notebook(cls, nb_man, kernel_name, **kwargs):

            # call the papermill execution engine:
            super().execute_managed_notebook(nb_man, kernel_name, **kwargs)

            for cell in nb_man.nb.cells:

                if cell.cell_type == "code" and cell.execution_count is not None:
                    start = datetime.fromisoformat(cell.metadata.papermill.start_time)
                    end = datetime.fromisoformat(cell.metadata.papermill.end_time)
                    output_message = f"Execution took {(end - start).total_seconds():.3f} seconds"
                    output_node = new_output("display_data", data={"text/plain": [output_message]})
                    cell.outputs = [output_node] + cell.outputs

Once this is in place, we need to add our engine as an entry point to our
``pyproject.toml`` file - for this, see the following section.

Ensuring your engine is found by papermill

[build-system]
requires = ["setuptools>=61.0", "wheel"]
build-backend = "setuptools.build_meta"

[project]
name = "papermill_timing"
version = "0.1"
description = "A papermill engine that logs additional timing information about code."
authors = [
    {name = "My Name", email = "[email protected]"}
]
dependencies = ["papermill", "nbformat"]

[project.urls]
Repository = "https://github.com/my_username/papermill_timing.git"

[project.entry-points."papermill.engine"]
timer_engine = "papermill_timing:CustomEngine"

[tool.setuptools]
packages = ["papermill_timing"]
package-dir = {"" = "src"}

No additional installation steps are required since python bindings are built
into **papermill**.

---

### Troubleshooting (`docs/troubleshooting.rst`)

Troubleshooting
===============

NoSuchKernel Errors (using Conda)
---------------------------------

``NoSuchKernel`` Errors can appear when running papermill on jupyter notebooks whose
kernel has been specified via conda (nb_conda). nb_conda is used to easily set
conda environment per notebook from within jupyterlab.

To illustrate, the following example demonstrates the creation of a new
environment with all the dependencies necessary for an analysis.

::

  conda create -n analysis_1 python=2 ipykernel

Once nb_conda is used within the jupyter server to set the kernel for a
notebook to *analysis_1*, the notebook gets metadata similar to the following:

.. code-block:: json

  {
   "kernelspec": {
    "display_name": "Python [conda env:analysis_1]",
    "language": "python",
    "name": "conda-env-analysis_1-py"
   }
  }

Papermill cannot use this metadata to determine that it should use
*analysis_1* to execute this notebook. Running papermill (from *analysis_1* or
another environment) will raise the following error:

::

  jupyter_client.kernelspec.NoSuchKernel: No such kernel named conda-env-analysis_1-py

This can be fixed by:

* Installing jupyter
  (`or at least ipykernel <https://ipython.readthedocs.io/en/stable/install/kernel_install.html#kernels-for-different-environments>`_)
  in *analysis_1*

::

  conda install -n analysis_1 jupyter

* Expose the *analysis_1* environment as a jupyter kernel
  (`this is no longer automatic <https://github.com/jupyter/jupyter/issues/245>`_).

::

  conda activate analysis_1
  jupyter kernelspec install --user --name analysis_1

* Run papermill (from any environment) specifying the correct kernel using the
  ``-k`` option

::

  papermill my_notebook.ipynb output_notebook.ipynb -k analysis_1

---

### Usage Cli (`docs/usage-cli.rst`)

Command Line Interface
======================

papermill may be executed from the terminal. The following are the command
options:

.. code-block:: bash

    Usage: papermill [OPTIONS] NOTEBOOK_PATH [OUTPUT_PATH]

      This utility executes a single notebook in a subprocess.

      Papermill takes a source notebook, applies parameters to the source
      notebook, executes the notebook with the specified kernel, and saves the
      output in the destination notebook.

      The NOTEBOOK_PATH and OUTPUT_PATH can now be replaced by `-` representing
      stdout and stderr, or by the presence of pipe inputs / outputs. Meaning
      that

      `<generate input>... | papermill | ...<process output>`

      with `papermill - -` being implied by the pipes will read a notebook from
      stdin and write it out to stdout.

    Options:
      --help-notebook                 Display parameters information for the given
                                      notebook path.

      -p, --parameters TEXT...        Parameters to pass to the parameters cell.
      -r, --parameters_raw TEXT...    Parameters to be read as raw string.
      -f, --parameters_file TEXT      Path to YAML file containing parameters.
      -y, --parameters_yaml TEXT      YAML string to be used as parameters.
      -b, --parameters_base64 TEXT    Base64 encoded YAML string as parameters.
      --inject-input-path             Insert the path of the input notebook as
                                      PAPERMILL_INPUT_PATH as a notebook
                                      parameter.

      --inject-output-path            Insert the path of the output notebook as
                                      PAPERMILL_OUTPUT_PATH as a notebook
                                      parameter.

      --inject-paths                  Insert the paths of input/output notebooks
                                      as
                                      PAPERMILL_INPUT_PATH/PAPERMILL_OUTPUT_PATH
                                      as notebook parameters.

      --engine TEXT                   The execution engine name to use in
                                      evaluating the notebook.

      --request-save-on-cell-execute / --no-request-save-on-cell-execute
                                      Request save notebook after each cell
                                      execution

      --autosave-cell-every INTEGER   How often in seconds to autosave the
                                      notebook during long cell executions (0 to
                                      disable)

      --prepare-only / --prepare-execute
                                      Flag for outputting the notebook without
                                      execution, but with parameters applied.

      -k, --kernel TEXT               Name of kernel to run.
      --cwd TEXT                      Working directory to run notebook in.
      --progress-bar / --no-progress-bar
                                      Flag for turning on the progress bar.
      --log-output / --no-log-output  Flag for writing notebook output to the
                                      configured logger.

      --stdout-file FILENAME          File to write notebook stdout output to.
      --stderr-file FILENAME          File to write notebook stderr output to.
      --log-level [NOTSET|DEBUG|INFO|WARNING|ERROR|CRITICAL]
                                      Set log level
      --start-timeout, --start_timeout INTEGER
                                      Time in seconds to wait for kernel to start.
      --execution-timeout INTEGER     Time in seconds to wait for each cell before
                                      failing execution (default: forever)

      --report-mode / --no-report-mode
                                      Flag for hiding input.
      --version                       Flag for displaying the version.
      -h, --help                      Show this message and exit.

---

### Usage Execute (`docs/usage-execute.rst`)

Execute
=======

The two ways to execute the notebook with parameters are: (1) through the
Python API and (2) through the command line interface.

Execute via the Python API

To execute a notebook using the CLI, enter the ``papermill`` command in the
terminal with the input notebook, location for output notebook, and options.

.. seealso::

    :doc:`CLI reference <./usage-cli>`

Execute a notebook with parameters
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

Here's an example of a local notebook being executed and output to an
Amazon S3 account:

.. code-block:: bash

   $ papermill local/input.ipynb s3://bkt/output.ipynb -p alpha 0.6 -p l1_ratio 0.1

In the above example, two parameters are set: ``alpha`` and ``l1_ratio`` using ``-p`` (``--parameters`` also works).
Parameter values that look like booleans or numbers will be interpreted as such.

Here are the different ways users may set parameters:

Using raw strings as parameters
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

Using ``-r`` or ``--parameters_raw``, users can set parameters one by one. However, unlike ``-p``, the parameter will
remain a string, even if it may be interpreted as a number or boolean.

.. code-block:: bash

    $ papermill local/input.ipynb s3://bkt/output.ipynb -r version 1.0

Using a parameters file
^^^^^^^^^^^^^^^^^^^^^^^

Using ``-f`` or ``--parameters_file``, users can provide a YAML file from which parameter values should be read.

.. code-block:: bash

    $ papermill local/input.ipynb s3://bkt/output.ipynb -f parameters.yaml

Using a YAML string for parameters
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

Using ``-y`` or ``--parameters_yaml``, users can directly provide a YAML string containing parameter values.

.. code-block:: bash

    $ papermill local/input.ipynb s3://bkt/output.ipynb -y "
    x:
        - 0.0
        - 1.0
        - 2.0
        - 3.0
    linear_function:
        slope: 3.0
        intercept: 1.0"

Using ``-b`` or ``--parameters_base64``, users can provide a YAML string, base64-encoded, containing parameter values.

.. code-block:: bash

    $ papermill local/input.ipynb s3://bkt/output.ipynb -b YWxwaGE6IDAuNgpsMV9yYXRpbzogMC4xCg==

Note about using YAML
^^^^^^^^^^^^^^^^^^^^^

When using YAML to pass arguments, through ``-y``, ``-b`` or ``-f``, parameter values can be arrays or dictionaries:

.. code-block:: bash

    $ papermill local/input.ipynb s3://bkt/output.ipynb -y "
    x:
        - 0.0
        - 1.0
        - 2.0
        - 3.0
    linear_function:
        slope: 3.0
        intercept: 1.0"

Note about using with multiple account credentials
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

If you use multiple AWS accounts and are accessing S3 files, you can
`configure your AWS credentials <https://boto3.amazonaws.com/v1/documentation/api/latest/guide/configuration.html>`__,
to specify which account to use by setting the `AWS_PROFILE` environment
variable at the command-line. For example:

.. code-block:: bash

    $ AWS_PROFILE=dev_account papermill local/input.ipynb s3://bkt/output.ipynb -p alpha 0.6 -p l1_ratio 0.1

A similar pattern may be needed for other types of remote storage accounts.

Tracking the Execution with Cell Descriptions
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
If you want to keep track of the execution in a closer way, you can add cell descriptions to the notebook being executed that will be injected to the TQDM progress bar.

Here is an example of the TQDM output during the execution of a 4-cells notebook without cell descriptions:

.. code-block:: bash

    10:10 # papermill input_notebook.ipynb output_notebook.ipynb
    Input Notebook:  input_notebook.ipynb
    Output Notebook: output_notebook.ipynb
    Executing:   0%|                                                                    | 0/4 [00:00<?, ?cell/s]Executing
    Executing:  25%|█████████                                                           | 1/4 [00:00<?, ?cell/s]Executing
    [...]

You can inject cell descriptions by adding a comment with the following formatting at the very beginning of each cell you are interested in:

.. code-block:: bash

    #papermill_description=TQDM_DESCRIPTION

In this way, when the execution will reach the cell tagged with the description, the TQDM progress bar will show the string TQDM_DESCRIPTION. Please be careful: the comment must not contain any space, otherwise it will be ignored.

Taking back our previous example, if we add the description 'FirstCell' to cell 0 and the description 'SecondCell' to the cell 1 we will get the TQDM to display the following information:

.. code-block:: bash

    10:12 # papermill input_notebook.ipynb output_notebook.ipynb
    Input Notebook:  input_notebook.ipynb
    Output Notebook: output_notebook.ipynb
    Executing FirstCell:   0%|                                                          | 0/4 [00:00<?, ?cell/s]Executing
    Executing SecondCell:  25%|█████████                                                 | 1/4 [00:00<?, ?cell/s]Executing
    [...]

---

### Usage Inspect (`docs/usage-inspect.rst`)

Inspect
=======

The two ways to inspect the notebook to discover its parameters are: (1) through the
Python API and (2) through the command line interface.

Execute via the Python API

``inspect_notebook('path/to/input_{month}.ipynb', parameters={month='Feb'})``

To inspect a notebook using the CLI, enter the ``papermill --help-notebook`` command in the
terminal with the notebook and optionally path parameters.

.. seealso::

    :doc:`CLI reference <./usage-cli>`

Inspect a notebook
^^^^^^^^^^^^^^^^^^

Here's an example of a local notebook being inspected and an output example:

.. code-block:: bash

    papermill --help-notebook ./papermill/tests/notebooks/complex_parameters.ipynb

    Usage: papermill [OPTIONS] NOTEBOOK_PATH [OUTPUT_PATH]

    Parameters inferred for notebook './papermill/tests/notebooks/complex_parameters.ipynb':
    msg: Unknown type (default None)
    a: float (default 2.25)         Variable a
    b: List[str] (default ['Hello','World'])
                                    Nice list
    c: NoneType (default None)

---

### Usage Parameterize (`docs/usage-parameterize.rst`)

Parameterize
============

.. seealso::

    :doc:`Workflow reference <./reference/papermill-workflow>`

Generally, the first workflow step when using papermill is to parameterize the
notebook.

To do this, tag notebook cells with ``parameters``. These ``parameters`` are
later used when the notebook is executed or run.

Designate parameters for a cell
-------------------------------

To parameterize a notebook, designate a cell with the tag ``parameters``.

.. image:: img/parameters.png

Notebook
~~~~~~~~

If using the `Jupyter Notebook`_ interface

1. Activate the tagging toolbar by navigating to ``View``, ``Cell Toolbar``, and then ``Tags``
2. Enter ``parameters`` into a textbox at the top right of a cell
3. Click ``Add tag``

JupyterLab 3.0+

If using the `JupyterLab`_ interface

1. Select the cell to parameterize
2. Click the property inspector on the left side (double gear icon)
3. Type "parameters" in the "Add Tag" box and hit "Enter".

.. image:: img/lab_parameters.png

JupyterLab < 2.0
~~~~~~~~~~~~~~~~
If using JupyterLab < 2.0, consider using the
`jupyterlab-celltags`_ extension.

1. Select the cell to parameterize
2. Click the cell inspector (wrench icon)
3. Type "parameters" in the "Add Tag" box and hit "Enter".

If the extension is not installed, you can manually add the tag by
editing the Cell Metadata field in the cell inspector by adding "tags":["parameters"].

Learn more about the jupyter notebook format and metadata fields `here`_.


How parameters work
-------------------

The ``parameters`` cell is assumed to specify default values which may be
overridden by values specified at execution time.

- papermill inserts a new cell tagged ``injected-parameters`` immediately after
  the ``parameters`` cell
- ``injected-parameters`` contains only the overridden parameters
- subsequent cells are treated as normal cells, even if also tagged ``parameters``
- if no cell is tagged ``parameters``, the ``injected-parameters`` cell
  is inserted at the top of the notebook

One caveat is that a ``parameters`` cell may not behave intuitively with
inter-dependent parameters. Consider a notebook ``note.ipynb`` with two cells:

.. code-block:: python

  #parameters
  a = 1
  twice = a * 2

.. code-block:: python

  print("a =", a, "and twice =", twice)

when executed with ``papermill note.ipynb -p a 9``, the output will be
``a = 9 and twice = 2`` (not ``twice = 18``).

.. _`JupyterLab`: https://github.com/jupyterlab/jupyterlab
.. _`Jupyter Notebook`: https://github.com/jupyter/notebook
.. _`here`: https://ipython.org/ipython-doc/dev/notebook/nbformat.html#cell-metadata
.. _`jupyterlab-celltags`: https://github.com/jupyterlab/jupyterlab-celltags

---

### Usage Store (`docs/usage-store.rst`)

Store
=====

.. seealso::

    :doc:`Reference - Storage <./reference/papermill-storage>`

Papermill can store notebooks in a number of locations including
AWS S3, Azure data blobs, and Azure data lakes.

The modular architecture of papermill allows new data stores to be
added over time.

---

### Usage Workflow (`docs/usage-workflow.rst`)

Usage
=====


For an interactive example that demonstrates the usage of papermill, click the Binder link
below:

.. image:: https://mybinder.org/badge.svg
   :target: https://mybinder.org/v2/gh/nteract/papermill/main?filepath=binder%2Fprocess_highlight_dates.ipynb
   :alt: launch binder

Using papermill
---------------

The general workflow when using papermill is **parameterizing** a notebook,
**executing** it, as well as **storing** the results.
In addition to operating on a single notebook, papermill also works on
a collection of notebooks.

.. toctree::
   :maxdepth: 2

   usage-parameterize
   usage-inspect
   usage-execute
   usage-store

---

### Reference Index (`docs/reference/index.rst`)

Reference
=========

This part of the documentation lists the full API reference of all public classes and functions.

.. toctree::
   :maxdepth: 2

   papermill-cli
   papermill-workflow
   papermill-translators
   papermill-io
   papermill-storage
   papermill-utilities

---

### Reference Papermill Cli (`docs/reference/papermill-cli.rst`)

CLI
===


papermill.cli
-------------

.. automodule:: papermill.cli
    :members:
    :undoc-members:
    :show-inheritance:

Command Line options
--------------------

.. code-block:: bash

    Usage: papermill [OPTIONS] NOTEBOOK_PATH OUTPUT_PATH

      This utility executes a single notebook in a subprocess.

      Papermill takes a source notebook, applies parameters to the source
      notebook, executes the notebook with the specified kernel, and saves the
      output in the destination notebook.

      The NOTEBOOK_PATH and OUTPUT_PATH can now be replaced by `-` representing
      stdout and stderr, or by the presence of pipe inputs / outputs. Meaning
      that

      `<generate input>... | papermill | ...`

      with `papermill - -` being implied by the pipes will read a notebook from
      stdin and write it out to stdout.

    Options:
      -p, --parameters TEXT...        Parameters to pass to the parameters cell.
      -r, --parameters_raw TEXT...    Parameters to be read as raw string.
      -f, --parameters_file TEXT      Path to YAML file containing parameters.
      -y, --parameters_yaml TEXT      YAML string to be used as parameters.
      -b, --parameters_base64 TEXT    Base64 encoded YAML string as parameters.
      --inject-input-path             Insert the path of the input notebook as
                                      PAPERMILL_INPUT_PATH as a notebook
                                      parameter.
      --inject-output-path            Insert the path of the output notebook as
                                      PAPERMILL_OUTPUT_PATH as a notebook
                                      parameter.
      --inject-paths                  Insert the paths of input/output notebooks
                                      as
                                      PAPERMILL_INPUT_PATH/PAPERMILL_OUTPUT_PATH
                                      as notebook parameters.
      --engine TEXT                   The execution engine name to use in
                                      evaluating the notebook.
      --request-save-on-cell-execute / --no-request-save-on-cell-execute
                                      Request save notebook after each cell
                                      execution
      --prepare-only / --prepare-execute
                                      Flag for outputting the notebook without
                                      execution, but with parameters applied.
      -k, --kernel TEXT               Name of kernel to run.
      --cwd TEXT                      Working directory to run notebook in.
      --progress-bar / --no-progress-bar
                                      Flag for turning on the progress bar.
      --log-output / --no-log-output  Flag for writing notebook output to the
                                      configured logger.
      --stdout-file FILENAME          File to write notebook stdout output to.
      --stderr-file FILENAME          File to write notebook stderr output to.
      --log-level [NOTSET|DEBUG|INFO|WARNING|ERROR|CRITICAL]
                                      Set log level
      --start_timeout INTEGER         Time in seconds to wait for kernel to start.
      --execution_timeout INTEGER     Time in seconds to wait for each cell before
                                      failing execution (default: forever)
      --report-mode / --no-report-mode
                                      Flag for hiding input.
      --version                       Flag for displaying the version.
      -h, --help                      Show this message and exit.

---

### Reference Papermill Io (`docs/reference/papermill-io.rst`)

Input / Output
==============


papermill.iorw
--------------

.. automodule:: papermill.iorw
    :members:
    :undoc-members:
    :show-inheritance:

---

### Reference Papermill Storage (`docs/reference/papermill-storage.rst`)

Storage
=======

Azure
-----

These modules outline how to interact with Azure data stores, specifically
Azure Blob Storage and Azure Data Lakes.

papermill.abs module
~~~~~~~~~~~~~~~~~~~~

.. automodule:: papermill.abs
    :members:
    :undoc-members:
    :show-inheritance:

papermill.adl module
~~~~~~~~~~~~~~~~~~~~

.. automodule:: papermill.adl
    :members:
    :undoc-members:
    :show-inheritance:

AWS
---

This module shows how to interact with AWS S3 data stores.

papermill.s3 module
~~~~~~~~~~~~~~~~~~~

.. automodule:: papermill.s3
    :members:
    :undoc-members:
    :show-inheritance:

---

### Reference Papermill Translators (`docs/reference/papermill-translators.rst`)

Language Translators
====================

.. currentmodule:: papermill.translators


Translators
-----------

Translator
~~~~~~~~~~

.. autoclass:: papermill.translators.Translator
    :members:
    :undoc-members:

PapermillTranslators
~~~~~~~~~~~~~~~~~~~~

.. autoclass:: papermill.translators.PapermillTranslators
    :members:
    :undoc-members:

Python
------

.. autoclass:: papermill.translators.PythonTranslator
    :members:
    :undoc-members:

R
-

.. autoclass:: papermill.translators.RTranslator
    :members:
    :undoc-members:

Julia
-----

.. autoclass:: papermill.translators.JuliaTranslator
    :members:
    :undoc-members:

Scala
-----

.. autoclass:: papermill.translators.ScalaTranslator
    :members:
    :undoc-members:

Functions
---------

.. autofunction:: translate_parameters

---

### Reference Papermill Utilities (`docs/reference/papermill-utilities.rst`)

Utilities
=========


Utils
-----

.. automodule:: papermill.utils
    :members:
    :undoc-members:

Exceptions
----------

.. automodule:: papermill.exceptions
    :members:
    :undoc-members:

Log
---

.. automodule:: papermill.log
    :members:
    :undoc-members:

---

### Reference Papermill Workflow (`docs/reference/papermill-workflow.rst`)

Workflow
========


papermill.engines
-----------------

.. automodule:: papermill.engines
    :members:
    :undoc-members:
    :show-inheritance:


papermill.execute
-----------------

.. automodule:: papermill.execute
    :members:
    :undoc-members:
    :show-inheritance:

papermill.clientwrap
--------------------

.. automodule:: papermill.clientwrap
    :members:
    :undoc-members:
    :show-inheritance:

---

--- METRICS ---
- Files Extracted: 23
- Estimated Token Budget: ~12338 tokens
- Canonical Reference: https://codewiki.google/github.com/nteract/papermill