BentoML

GitHub

The easiest way to serve AI apps and models - Build Model Inference APIs, Job queues, LLM apps, Multi-model pipelines, and more!

RAW Doc

README

BentoML Documentation

A guide for docs contributors

The docs directory contains the sphinx source text for BentoML docs, visit
http://docs.bentoml.com/ to read the full documentation.

This guide is made for anyone who's interested in running BentoML documentation locally,
making changes to it and make contributions. BentoML is made by the thriving community
behind it, and you're always welcome to make contributions to the project and the
documentation.

Before starting to make a contribution to the docs, check the
issues page to make sure no one
else is working on the same thing and get feedback from our forum for larger proposals.

---

Build Docs

If you haven't already, clone the BentoML Github repo to a local directory:

bash
git clone https://github.com/bentoml/BentoML.git && cd BentoML

Note: Make sure to have PDM installed.

Install all dependencies required for building docs (mainly sphinx and its extensions):

bash
pdm install -dG docs

Build the sphinx docs:

bash
make clean html -C ./docs

The docs HTML files are now generated under docs/build/html directory, you can preview
it locally with the following command:

bash
python -m http.server 8000 -d docs/build/html

And open your browser at http://0.0.0.0:8000/ to view the generated docs.


#### Spellcheck

Install spellchecker dependencies:

bash
make install-spellchecker-deps

To run spellchecker locally:

bash
make spellcheck-doc

##### Watch Docs

We recommend using sphinx-autobuild during development, which provides a live-reloading
server, that rebuilds the documentation and refreshes any open pages automatically when
changes are saved. This enables a much shorter feedback loop which can help boost
productivity when writing documentation.

Simply run the following command from BentoML project's root directory:

bash
sphinx-autobuild docs/source docs/build/html

If you have make installed, you may also run:

bash
make watch-docs


Writing Documentation


Writing .rst (ReStructuredText) in BentoML docs

BentoML docs is built with Sphinx, which natively supports ReStructuredText.

#### Document titles and section headers

In reStructuredText, there are no heading levels assigned to certain characters as the
structure is determined from the succession of headings. However in BentoML docs, we
follow the following convention:

rst
==============
Document Title
==============

Top Level Headings
------------------

2nd level headings
~~~~~~~~~~~~~~~~~~

3rd level headings
^^^^^^^^^^^^^^^^^^

4th level heading - avoid this if possible
""""""""""""""""""""""""""""""""""""""""""

#### Adding Reference Links

When writing documentation, it is common to mention or link to other parts of the docs.

If you need to refer to a specific documentation page, use :doc: plus path to the
target documentation file under the docs/source/. e.g.:

rst
:doc:tutorial
:doc:/frameworks/pytorch

By default, this will show the title of the target document and link to it. You may also
change the title shown on current page:

rst
:doc:📖 Main Concepts <concepts/index>

It is also possible to refer to a specific section of other document pages. We use the
autosectionlabel sphinx plugin
to generate labels for every section in the documentation.

For example:

rst
:ref:frameworks/pytorch:Section Title

#### Admonitions

A note section can be created with the following syntax:

rst
.. note:: This is what the most basic admonitions look like.


.. note::
It is possible to have multiple paragraphs in the same admonition.

If you really want, you can even have lists, or code, or tables.

There are other admonition types such as caution, danger, hint, important,
seealso, and tip. Learn more about it here.

#### Code Blocks

rst
Code blocks in reStructuredText can be created in various ways::

Indenting content by 4 spaces, after a line ends with "::".
This will have default syntax highlighting (highlighting a few words and "strings").

.. code::

You can also use the code directive, or an alias: code-block, sourcecode.
This will have default syntax highlighting (highlighting a few words and "strings").

.. code:: python

print("And with the directive syntax, you can have syntax highlighting.")

.. code:: none

print("Or disable all syntax highlighting.")

There are a lot more forms of "blocks" in reStructuredText that can be used, as
seen in https://docutils.sourceforge.io/docs/ref/rst/restructuredtext.html#literal-blocks.


#### Tabs

For most scenarios in BentoML docs, use the tabs view provided by sphinx-design:
https://sphinx-design.readthedocs.io/en/furo-theme/tabs.html

rst
.. tab-set::

.. tab-item:: Label1

Content 1

.. tab-item:: Label2

Content 2

Documenting Source Code

BentoML docs relies heavily on the Python docstrings defined together with the source
code. We ask our contributors to document every public facing APIs and CLIs, including
their signatures, options, and example usage. Sphinx can then use these inline docs to
generate API References pages.

BentoML uses the sphinx.ext.autodoc
extension to include documentation from docstring. For example, a
.rst document can
create a section made from a Python Class's docstring, using the following syntax:

rst
.. autoclass:: bentoml.Bento
:members: api

Similarly, for functions:

rst
.. autofunction:: bentoml.models.list

Learn more about this syntax here.

BentoML codebase follows the Google's docstring style
for writing inline docstring. Below are some examples.

#### Define arguments in a method

Arguments should be defined with Args: prefix, followed by a line with indentation. Each argument should be followed by
its type, a new indentation for description of given field. Each argument should follow the below definition:

markdown
Args:
bento_name (:code:
str):
:class:
~bentoml.BentoService identifier with name format :obj:NAME:VERSION.
NAME can be accessed via :meth:~bentoml.BentoService.name and VERSION can
be accessed via :meth:
~bentoml.BentoService.version

For optional arguments, follow the following syntax. For example a function `func()` with following signature:

python
def func(x: str=None, a: Optional[bool]=None):
...

then documentation should look like:

markdown
Args:
x (:code:
str, optional):
Description of x ...
a (
bool, optional):
Description of a ...

#### Define a multiline code block in a method

Make sure to define something like:

markdown
Example::

# example code here
# ...

The `Example` can be replaced with any word of choice as long as there are two semicolons following. Read more about doctest

#### Define a return block in a method

If a function returns value, returns should be defined with Returns:, followed by a line with indentation. The first line
should be the type of the return, followed by a line return. An example for a return statement:

markdown
Returns:
:obj:
Dict[str,str] with keys are :class:~bentoml.BentoService nametag following with saved bundle path.

---

CONTRIBUTING

Contributing to BentoML

BentoML is an open and community-driven project. Everyone is welcome to contribute.

The decision-making process and governance structure of BentoML project can be found in the governance document: BentoML Governance Doc.

To follow development updates and discussion, join the #bentoml-contributors channel in BentoML Slack community.


Ways to contribute

There are many ways to contribute to BentoML.

* Supporting new users by answering questions on the
github issues tracker and the
#bentoml-users slack channel.

* Report issues you're facing and "Thumbs up" on issues and feature requests that are
relevant to you in BentoML's issues tracker.

* Investigate bugs and reviewing other developer's pull requests.

* Contributing code or documentation to the project by submitting a Github pull request.

* Create new example projects and contribute it to the Examples Overview page.

Submitting a bug report or a feature request

We use Github issues to track all bugs and feature requests. Feel free to open an issue
if you have found a bug or wish to see a new feature implemented.

Before submitting a github issue, ensure the bug was not already reported under
issues or currently being addressed by
other pull requests.

If you're unable to find an open issue addressing the problem,
open a new one. Be sure to
include a title and clear description, as much relevant information as
possible, and a code sample or an executable test case demonstrating
the expected behavior that is not occurring.


Contributing Code

To avoid duplicating work, it is highly recommended to search through the
issue tracker and
pull requests list. If in doubt about
duplicated work, or if you want to work on a non-trivial feature, it's recommended to
first open an issue in the issue tracker
to get some feedbacks from core developers.

One easy way to find an issue to work on is by applying the "help wanted" label in the
issues list: help wanted issues.

For detailed instructions on how to develop BentoML locally and submit a 'pull request',
follow the development guide.

If you are new to BentoML project and interested in contributing code, take a look at
the Good first issues list.
Resolving these issues allow you to start contributing to the project without much
prior knoledge and help you get familiar with its codebase.


Documentation

Improving the documentation is no less important than improving the library. If you find
a typo in the documentation, or have made improvements, do not hesitate to submit a
GitHub pull request.

Full documentation can be found under the docs/source directory. You can edit the
documentation
.rst or .md files using any text editor. Follow the instructions
here
to build documentation site locally, generate HTML output and preview your changes.

Issue Tracker Tags

Issue type tags:

| | |
| --- | --- |
| question | Any questions about the project |
| bug | Something isn't working |
| enhancement | Improving performance, usability, consistency |
| docs | Documentation, tutorials, and example projects |
| new feature | Feature requests or pull request implementing a new feature |
| test | Improving unit test coverage, e2e test, CI or build |

Tags to help new contributors:

| | |
| --- | --- |
| help wanted | An issue currently lacks a contributor |
| good first issue | Good for newcomers |

Tags for managing issues:

| | |
| --- | --- |
| duplicated | This issue or pull request already exists |
| stale | Automatically applied when an issue went quiet for more than 60 days |
| merge-hold | Requires further discussions before a pull request can be merged |


Testing and improving test coverage

High quality testing is extremely important for BentoML project. Currently BentoML has
three kind of tests: Unit tests(
tests/) and integrations (tests/integration/) are
running on Travis CI for every pull request. End-to-end tests(
e2e_tests/`) is manually
executed by the maintainer before every release and for pull requests that are
introducing major changes.

We expect pull requests that are introducing new features to have at least 90% test
coverages. Pull requests that are fixing a bug should add a test covering the issue
being fixed if possible.

---

SECURITY

Security Policy

Supported Versions

BentoML is currently under active development and releases a new version
every 2-3 weeks. We always recommend users to move to a newer version
when it became available, and we only provide security updates in the
latest version.

If you are using an older version of BentoML and would like to receive
security patches, let us know via
BentoML Slack Channel
or BentoML Discussions.


Reporting a Vulnerability

If you discover a potential security vulnerability, we kindly request that you refrain from sharing the information publicly and report it to us directly. Please send an email to [email protected] with the following details:

* Description of the potential vulnerability.
* Steps to reproduce the issue (if applicable).
* Any relevant screenshots or logs.
* Your contact information for further communication.

Alternatively, you can open a security advisory on GitHub.

Exceptions

The following reports are out of scope and will not be accepted as
security vulnerabilities:

* Reports about pickle-related vulnerabilities in the runner service or
dependency service. We consider these scenarios to be purely
theoretical and not a practical vulnerability in BentoML.

BentoML does not participate in Huntr.com's bug bounty program; we have no budget for bug bounties at this time nor do we plan to in the future.

---