# Technical Documentation: python/cpython > ℹ️ **Provenance:** Multi-Tier Ingestion · [CodeWiki Reference](https://codewiki.google/github.com/python/cpython) ### Doc README (`Doc/README.rst`) Python Documentation README ~~~~~~~~~~~~~~~~~~~~~~~~~~~ This directory contains the reStructuredText (reST) sources to the Python documentation. You don't need to build them yourself, `prebuilt versions are available `_. Documentation on authoring Python documentation, including information about both style and markup, is available in the "`Documenting Python `_" chapter of the developers guide. Building the docs ================= The documentation is built with several tools which are not included in this tree but are maintained separately and are available from `PyPI `_. * `Sphinx `_ * `blurb `_ * `python-docs-theme `_ The easiest way to install these tools is to create a virtual environment and install the tools into there. Using make ---------- To get started on Unix, you can create a virtual environment and build documentation with the commands:: make venv make html The virtual environment in the ``venv`` directory will contain all the tools necessary to build the documentation downloaded and installed from PyPI. If you'd like to create the virtual environment in a different location, you can specify it using the ``VENVDIR`` variable. You can also skip creating the virtual environment altogether, in which case the ``Makefile`` will look for instances of ``sphinx-build`` and ``blurb`` installed on your process ``PATH`` (configurable with the ``SPHINXBUILD`` and ``BLURB`` variables). On Windows, we try to emulate the ``Makefile`` as closely as possible with a ``make.bat`` file. If you need to specify the Python interpreter to use, set the ``PYTHON`` environment variable. Available make targets are: * "clean", which removes all build files and the virtual environment. * "clean-venv", which removes the virtual environment directory. * "venv", which creates a virtual environment with all necessary tools installed. * "html", which builds standalone HTML files for offline viewing. * "htmlview", which re-uses the "html" builder, but then opens the main page in your default web browser. * "htmllive", which re-uses the "html" builder, rebuilds the docs, starts a local server, and automatically reloads the page in your browser when you make changes to reST files (Unix only). * "htmlhelp", which builds HTML files and a HTML Help project file usable to convert them into a single Compiled HTML (.chm) file -- these are popular under Microsoft Windows, but very handy on every platform. To create the CHM file, you need to run the Microsoft HTML Help Workshop over the generated project (.hhp) file. The ``make.bat`` script does this for you on Windows. * "latex", which builds LaTeX source files as input to ``pdflatex`` to produce PDF documents. * "text", which builds a plain text file for each source file. * "epub", which builds an EPUB document, suitable to be viewed on e-book readers. * "linkcheck", which checks all external references to see whether they are broken, redirected or malformed, and outputs this information to stdout as well as a plain-text (.txt) file. * "changes", which builds an overview over all versionadded/versionchanged/ deprecated items in the current version. This is meant as a help for the writer of the "What's New" document. * "coverage", which builds a coverage overview for standard library modules and C API. * "pydoc-topics", which builds a Python module containing a dictionary with plain text documentation for the labels defined in ``tools/pyspecific.py`` -- pydoc needs these to show topic and keyword help. * "check", which checks for frequent markup errors. * "dist", (Unix only) which creates distributable archives of HTML, text, PDF, and EPUB builds. Without make ------------ First, install the tool dependencies from PyPI. Then, from the ``Doc`` directory, run :: sphinx-build -b . build/ where ```` is one of html, text, latex, or htmlhelp (for explanations see the make targets above). Deprecation header ================== You can define the ``outdated`` variable in ``html_context`` to show a red banner on each page redirecting to the "latest" version. The link points to the same page on ``/3/``, sadly for the moment the language is lost during the process. Contributing ============ Bugs in the content should be reported to the `Python bug tracker `_. Bugs in the toolset should be reported to the tools themselves. To help with the documentation, or report any problems, please leave a message on `discuss.python.org `_. --- ### Doc About (`Doc/about.rst`) ======================== About this documentation ======================== Python's documentation is generated from `reStructuredText`_ sources using `Sphinx`_, a documentation generator originally created for Python and now maintained as an independent project. .. _reStructuredText: https://docutils.sourceforge.io/rst.html .. _Sphinx: https://www.sphinx-doc.org/ .. In the online version of this documentation, you can submit comments and suggest changes directly on the documentation pages. Development of the documentation and its toolchain is an entirely volunteer effort, just like Python itself. If you want to contribute, please take a look at the :ref:`reporting-bugs` page for information on how to do so. New volunteers are always welcome! Many thanks go to: * Fred L. Drake, Jr., the creator of the original Python documentation toolset and author of much of the content; * the `Docutils `_ project for creating reStructuredText and the Docutils suite; * Fredrik Lundh for his Alternative Python Reference project from which Sphinx got many good ideas. Contributors to the Python documentation ---------------------------------------- Many people have contributed to the Python language, the Python standard library, and the Python documentation. See the `CPython GitHub repository `__ for a partial list of contributors. It is only with the input and contributions of the Python community that Python has such wonderful documentation -- Thank You! --- ### Doc Bugs (`Doc/bugs.rst`) .. _reporting-bugs: ***************** Dealing with Bugs ***************** Python is a mature programming language which has established a reputation for stability. In order to maintain this reputation, the developers would like to know of any deficiencies you find in Python. It can be sometimes faster to fix bugs yourself and contribute patches to Python as it streamlines the process and involves fewer people. Learn how to :ref:`contribute `. .. _reporting-documentation-bugs: Documentation bugs ================== If you find a bug in this documentation or would like to propose an improvement, please submit a bug report on the :ref:`issue tracker `. If you have a suggestion on how to fix it, include that as well. .. only:: translation If the bug or suggested improvement concerns the translation of this documentation, submit the report to the `translation’s repository `_ instead. You can also open a discussion item on our `Documentation Discourse forum `_. If you find a bug in the theme (HTML / CSS / JavaScript) of the documentation, please submit a bug report on the `python-doc-theme issue tracker `_. .. seealso:: `Documentation bugs`_ A list of documentation bugs that have been submitted to the Python issue tracker. `Issue Tracking `_ Overview of the process involved in reporting an improvement on the tracker. `Helping with Documentation `_ Comprehensive guide for individuals that are interested in contributing to Python documentation. `Documentation Translations `_ A list of GitHub pages for documentation translation and their coordination teams. .. _using-the-tracker: Using the Python issue tracker ============================== Issue reports for Python itself should be submitted via the GitHub issues tracker (https://github.com/python/cpython/issues). The GitHub issues tracker offers a web form which allows pertinent information to be entered and submitted to the developers. The first step in filing a report is to determine whether the problem has already been reported. The advantage in doing so, aside from saving the developers' time, is that you learn what has been done to fix it; it may be that the problem has already been fixed for the next release, or additional information is needed (in which case you are welcome to provide it if you can!). To do this, search the tracker using the search box at the top of the page. If the problem you're reporting is not already in the list, log in to GitHub. If you don't already have a GitHub account, create a new account using the "Sign up" link. It is not possible to submit a bug report anonymously. Being now logged in, you can submit an issue. Click on the "New issue" button in the top bar to report a new issue. The submission form has two fields, "Title" and "Comment". For the "Title" field, enter a *very* short description of the problem; fewer than ten words is good. In the "Comment" field, describe the problem in detail, including what you expected to happen and what did happen. Be sure to include whether any extension modules were involved, and what hardware and software platform you were using (including version information as appropriate). Each issue report will be reviewed by a developer who will determine what needs to be done to correct the problem. You will receive an update each time an action is taken on the issue. .. seealso:: `How to Report Bugs Effectively `_ Article which goes into some detail about how to create a useful bug report. This describes what kind of information is useful and why it is useful. `Bug Writing Guidelines `_ Information about writing a good bug report. Some of this is specific to the Mozilla project, but describes general good practices. .. _contributing-to-python: Getting started contributing to Python yourself =============================================== Beyond just reporting bugs that you find, you are also welcome to submit patches to fix them. You can find more information on how to get started patching Python in the `Python Developer's Guide`_. If you have questions, the `core-mentorship mailing list`_ is a friendly place to get answers to any and all questions pertaining to the process of fixing issues in Python. .. _Documentation bugs: https://github.com/python/cpython/issues?q=is%3Aissue+is%3Aopen+label%3Adocs .. _Python Developer's Guide: https://devguide.python.org/ .. _core-mentorship mailing list: https://mail.python.org/mailman3/lists/core-mentorship.python.org/ --- ### Doc Constraints (`Doc/constraints.txt`) # We have upper bounds on our transitive dependencies here # To avoid new releases unexpectedly breaking our build. # This file can be updated on an ad-hoc basis, # though it will probably have to be updated # whenever Doc/requirements.txt is updated. # Direct dependencies of Sphinx babel<3 colorama<0.5 imagesize<2 Jinja2<4 packaging<25 Pygments<3 requests<3 snowballstemmer<3 sphinxcontrib-applehelp<3 sphinxcontrib-devhelp<3 sphinxcontrib-htmlhelp<3 sphinxcontrib-jsmath<2 sphinxcontrib-qthelp<3 sphinxcontrib-serializinghtml<3 # Direct dependencies of Jinja2 (Jinja is a dependency of Sphinx, see above) MarkupSafe<3 --- ### Doc Contents (`Doc/contents.rst`) %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% Python Documentation contents %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% .. toctree:: whatsnew/index.rst tutorial/index.rst using/index.rst reference/index.rst builtins/index.rst library/index.rst extending/index.rst c-api/index.rst installing/index.rst howto/index.rst faq/index.rst deprecations/index.rst glossary.rst about.rst bugs.rst copyright.rst license.rst --- ### Doc Copyright (`Doc/copyright.rst`) ********* Copyright ********* Python and this documentation is: Copyright © 2001 Python Software Foundation. All rights reserved. Copyright © 2000 BeOpen.com. All rights reserved. Copyright © 1995-2000 Corporation for National Research Initiatives. All rights reserved. Copyright © 1991-1995 Stichting Mathematisch Centrum. All rights reserved. ------- See :ref:`history-and-license` for complete license and permissions information. --- ### Doc Glossary (`Doc/glossary.rst`) .. _glossary: ******** Glossary ******** .. if you add new entries, keep the alphabetical sorting! .. glossary:: ``>>>`` The default Python prompt of the :term:`interactive` shell. Often seen for code examples which can be executed interactively in the interpreter. ``...`` Can refer to: * The default Python prompt of the :term:`interactive` shell when entering the code for an indented code block, when within a pair of matching left and right delimiters (parentheses, square brackets, curly braces or triple quotes), or after specifying a decorator. .. index:: single: ...; ellipsis literal * The three dots form of the :ref:`Ellipsis ` object. abstract base class Abstract base classes complement :term:`duck-typing` by providing a way to define interfaces when other techniques like :func:`hasattr` would be clumsy or subtly wrong (for example with :ref:`magic methods `). ABCs introduce virtual subclasses, which are classes that don't inherit from a class but are still recognized by :func:`isinstance` and :func:`issubclass`; see the :mod:`abc` module documentation. Python comes with many built-in ABCs for data structures (in the :mod:`collections.abc` module), numbers (in the :mod:`numbers` module), streams (in the :mod:`io` module), import finders and loaders (in the :mod:`importlib.abc` module). You can create your own ABCs with the :mod:`abc` module. annotate function A callable that can be called to retrieve the :term:`annotations ` of an object. Annotate functions are usually :term:`functions `, automatically generated as the :attr:`~object.__annotate__` attribute of functions, classes, and modules. Annotate functions are a subset of :term:`evaluate functions `. annotation A label associated with a variable, a class attribute or a function parameter or return value, used by convention as a :term:`type hint`. Annotations of local variables cannot be accessed at runtime, but annotations of global variables, class attributes, and functions can be retrieved by calling :func:`annotationlib.get_annotations` on modules, classes, and functions, respectively. See :term:`variable annotation`, :term:`function annotation`, :pep:`484`, :pep:`526`, and :pep:`649`, which describe this functionality. Also see :ref:`annotations-howto` for best practices on working with annotations. argument A value passed to a :term:`function` (or :term:`method`) when calling the function. There are two kinds of argument: * :dfn:`keyword argument`: an argument preceded by an identifier (e.g. ``name=``) in a function call or passed as a value in a dictionary preceded by ``**``. For example, ``3`` and ``5`` are both keyword arguments in the following calls to :func:`complex`:: complex(real=3, imag=5) complex(**{'real': 3, 'imag': 5}) * :dfn:`positional argument`: an argument that is not a keyword argument. Positional arguments can appear at the beginning of an argument list and/or be passed as elements of an :term:`iterable` preceded by ``*``. For example, ``3`` and ``5`` are both positional arguments in the following calls:: complex(3, 5) complex(*(3, 5)) Arguments are assigned to the named local variables in a function body. See the :ref:`calls` section for the rules governing this assignment. Syntactically, any expression can be used to represent an argument; the evaluated value is assigned to the local variable. See also the :term:`parameter` glossary entry, the FAQ question on :ref:`the difference between arguments and parameters `, and :pep:`362`. asynchronous context manager An object which controls the environment seen in an :keyword:`async with` statement by defining :meth:`~object.__aenter__` and :meth:`~object.__aexit__` methods. Introduced by :pep:`492`. asynchronous generator Informally used to mean either an :term:`asynchronous generator function` or an :term:`asynchronous generator iterator`, depending on context. The formal terms :term:`asynchronous generator function` and :term:`asynchronous generator iterator` are uncommon in practice; "asynchronous generator" alone is almost always sufficient. asynchronous generator function A function which returns an :term:`asynchronous generator iterator`. It looks like a coroutine function defined with :keyword:`async def` except that it contains :keyword:`yield` expressions for producing a series of values usable in an :keyword:`async for` loop. See :pep:`525`. An asynchronous generator function may contain :keyword:`await` expressions as well as :keyword:`async for`, and :keyword:`async with` statements. asynchronous generator iterator An object created by an :term:`asynchronous generator function`. This is an :term:`asynchronous iterator` which when called using the :meth:`~object.__anext__` method returns an awaitable object which will execute the body of the asynchronous generator function until the next :keyword:`yield` expression. Each :keyword:`yield` temporarily suspends processing, remembering the execution state (including local variables and pending try-statements). When the *asynchronous generator iterator* effectively resumes with another awaitable returned by :meth:`~object.__anext__`, it picks up where it left off. See :pep:`492` and :pep:`525`. asynchronous iterable An object, that can be used in an :keyword:`async for` statement. Must return an :term:`asynchronous iterator` from its :meth:`~object.__aiter__` method. Introduced by :pep:`492`. asynchronous iterator An object that implements the :meth:`~object.__aiter__` and :meth:`~object.__anext__` methods. :meth:`~object.__anext__` must return an :term:`awaitable` object. :keyword:`async for` resolves the awaitables returned by an asynchronous iterator's :meth:`~object.__anext__` method until it raises a :exc:`StopAsyncIteration` exception. Introduced by :pep:`492`. atomic operation An operation that appears to execute as a single, indivisible step: no other thread can observe it half-done, and its effects become visible all at once. Python does not guarantee that high-level statements are atomic (for example, ``x += 1`` performs multiple bytecode operations and is not atomic). Atomicity is only guaranteed where explicitly documented. See also :term:`race condition` and :term:`data race`. attached thread state A :term:`thread state` that is active for the current OS thread. When a :term:`thread state` is attached, the OS thread has access to the full Python C API and can safely invoke the bytecode interpreter. Unless a function explicitly notes otherwise, attempting to call the C API without an attached thread state will result in a fatal error or undefined behavior. A thread state can be attached and detached explicitly by the user through the C API, or implicitly by the runtime, including during blocking C calls and by the bytecode interpreter in between calls. On most builds of Python, having an attached thread state implies that the caller holds the :term:`GIL` for the current interpreter, so only one OS thread can have an attached thread state at a given moment. In :term:`free-threaded builds ` of Python, threads can concurrently hold an attached thread state, allowing for true parallelism of the bytecode interpreter. attribute A value associated with an object which is usually referenced by name using dotted expressions. For example, if an object *o* has an attribute *a* it would be referenced as *o.a*. It is possible to give an object an attribute whose name is not an identifier as defined by :ref:`identifiers`, for example using :func:`setattr`, if the object allows it. Such an attribute will not be accessible using a dotted expression, and would instead need to be retrieved with :func:`getattr`. awaitable An object that can be used in an :keyword:`await` expression. Can be a :term:`coroutine` or an object with an :meth:`~object.__await__` method. See also :pep:`492`. BDFL Benevolent Dictator For Life, a.k.a. `Guido van Rossum `_, Python's creator. binary file A :term:`file object` able to read and write :term:`bytes-like objects `. Examples of binary files are files opened in binary mode (``'rb'``, ``'wb'`` or ``'rb+'``), :data:`sys.stdin.buffer `, :data:`sys.stdout.buffer `, and instances of :class:`io.BytesIO` and :class:`gzip.GzipFile`. See also :term:`text file` for a file object able to read and write :class:`str` objects. borrowed reference In Python's C API, a borrowed reference is a reference to an object, where the code using the object does not own the reference. It becomes a dangling pointer if the object is destroyed. For example, a garbage collection can remove the last :term:`strong reference` to the object and so destroy it. Calling :c:func:`Py_INCREF` on the :term:`borrowed reference` is recommended to convert it to a :term:`strong reference` in-place, except when the object cannot be destroyed before the last usage of the borrowed reference. The :c:func:`Py_NewRef` function can be used to create a new :term:`strong reference`. bytes-like object An object that supports the :ref:`bufferobjects` and can export a C-:term:`contiguous` buffer. This includes all :class:`bytes`, :class:`bytearray`, and :class:`array.array` objects, as well as many common :class:`memoryview` objects. Bytes-like objects can be used for various operations that work with binary data; these include compression, saving to a binary file, and sending over a socket. Some operations need the binary data to be mutable. The documentation often refers to these as "read-write bytes-like objects". Example mutable buffer objects include :class:`bytearray` and a :class:`memoryview` of a :class:`bytearray`. Other operations require the binary data to be stored in immutable objects ("read-only bytes-like objects"); examples of these include :class:`bytes` and a :class:`memoryview` of a :class:`bytes` object. bytecode Python source code is compiled into bytecode, the internal representation of a Python program in the CPython interpreter. The bytecode is also cached in ``.pyc`` files so that executing the same file is faster the second time (recompilation from source to bytecode can be avoided). This "intermediate language" is said to run on a :term:`virtual machine` that executes the machine code corresponding to each bytecode. Do note that bytecodes are not expected to work between different Python virtual machines, nor to be stable between Python releases. A list of bytecode instructions can be found in the documentation for :ref:`the dis module `. callable A callable is an object that can be called, possibly with a set of arguments (see :term:`argument`), with the following syntax:: callable(argument1, argument2, argumentN) A :term:`function`, and by extension a :term:`method`, is a callable. An instance of a class that implements the :meth:`~object.__call__` method is also a callable. callback A subroutine function which is passed as an argument to be executed at some point in the future. class A template for creating user-defined objects. Class definitions normally contain method definitions which operate on instances of the class. class variable A variable defined in a class and intended to be modified only at class level (i.e., not in an instance of the class). closure variable A :term:`free variable` referenced from a :term:`nested scope` that is defined in an outer scope rather than being resolved at runtime from the globals or builtin namespaces. May be explicitly defined with the :keyword:`nonlocal` keyword to allow write access, or implicitly defined if the variable is only being read. For example, in the ``inner`` function in the following code, both ``x`` and ``print`` are :term:`free variables `, but only ``x`` is a *closure variable*:: def outer(): x = 0 def inner(): nonlocal x x += 1 print(x) return inner Due to the :attr:`codeobject.co_freevars` attribute (which, despite its name, only includes the names of closure variables rather than listing all referenced free variables), the more general :term:`free variable` term is sometimes used even when the intended meaning is to refer specifically to closure variables. complex number An extension of the familiar real number system in which all numbers are expressed as a sum of a real part and an imaginary part. Imaginary numbers are real multiples of the imaginary unit (the square root of ``-1``), often written ``i`` in mathematics or ``j`` in engineering. Python has built-in support for complex numbers, which are written with this latter notation; the imaginary part is written with a ``j`` suffix, e.g., ``3+1j``. To get access to complex equivalents of the :mod:`math` module, use :mod:`cmath`. Use of complex numbers is a fairly advanced mathematical feature. If you're not aware of a need for them, it's almost certain you can safely ignore them. concurrency The ability of a computer program to perform multiple tasks at the same time. Python provides libraries for writing programs that make use of different forms of concurrency. :mod:`asyncio` is a library for dealing with asynchronous tasks and coroutines. :mod:`threading` provides access to operating system threads and :mod:`multiprocessing` to operating system processes. Multi-core processors can execute threads and processes on different CPU cores at the same time (see :term:`parallelism`). concurrent modification When multiple threads modify shared data at the same time. Concurrent modification without proper synchronization can cause :term:`race conditions `, and might also trigger a :term:`data race `, data corruption, or both. context This term has different meanings depending on where and how it is used. Some common meanings: * The temporary state or environment established by a :term:`context manager` via a :keyword:`with` statement. * The collection of key­value bindings associated with a particular :class:`contextvars.Context` object and accessed via :class:`~contextvars.ContextVar` objects. Also see :term:`context variable`. * A :class:`contextvars.Context` object. Also see :term:`current context`. context management protocol The :meth:`~object.__enter__` and :meth:`~object.__exit__` methods called by the :keyword:`with` statement. See :pep:`343`. context manager An object which implements the :term:`context management protocol` and controls the environment seen in a :keyword:`with` statement. See :pep:`343`. context variable A variable whose value depends on which context is the :term:`current context`. Values are accessed via :class:`contextvars.ContextVar` objects. Context variables are primarily used to isolate state between concurrent asynchronous tasks. contiguous .. index:: C-contiguous, Fortran contiguous A buffer is considered contiguous exactly if it is either *C-contiguous* or *Fortran contiguous*. Zero-dimensional buffers are C and Fortran contiguous. In one-dimensional arrays, the items must be laid out in memory next to each other, in order of increasing indexes starting from zero. In multidimensional C-contiguous arrays, the last index varies the fastest when visiting items in order of memory address. However, in Fortran contiguous arrays, the first index varies the fastest. coroutine Coroutines are a more generalized form of subroutines. Subroutines are entered at one point and exited at another point. Coroutines can be entered, exited, and resumed at many different points. They can be implemented with the :keyword:`async def` statement. See also :pep:`492`. coroutine function A function which returns a :term:`coroutine` object. A coroutine function may be defined with the :keyword:`async def` statement, and may contain :keyword:`await`, :keyword:`async for`, and :keyword:`async with` keywords. These were introduced by :pep:`492`. CPython The canonical implementation of the Python programming language, as distributed on `python.org `_. The term "CPython" is used when necessary to distinguish this implementation from others such as Jython or IronPython. current context The :term:`context` (:class:`contextvars.Context` object) that is currently used by :class:`~contextvars.ContextVar` objects to access (get or set) the values of :term:`context variables `. Each thread has its own current context. Frameworks for executing asynchronous tasks (see :mod:`asyncio`) associate each task with a context which becomes the current context whenever the task starts or resumes execution. cyclic isolate A subgroup of one or more objects that reference each other in a reference cycle, but are not referenced by objects outside the group. The goal of the :term:`cyclic garbage collector ` is to identify these groups and break the reference cycles so that the memory can be reclaimed. data race A situation where multiple threads access the same memory location concurrently, at least one of the accesses is a write, and the threads do not use any synchronization to control their access. Data races lead to :term:`non-deterministic` behavior and can cause data corruption. Proper use of :term:`locks ` and other :term:`synchronization primitives ` prevents data races. Note that data races can only happen in native code, but that :term:`native code` might be exposed in a Python API. See also :term:`race condition` and :term:`thread-safe`. deadlock A situation in which two or more tasks (threads, processes, or coroutines) wait indefinitely for each other to release resources or complete actions, preventing any from making progress. For example, if thread A holds lock 1 and waits for lock 2, while thread B holds lock 2 and waits for lock 1, both threads will wait indefinitely. In Python this often arises from acquiring multiple locks in conflicting orders or from circular join/await dependencies. Deadlocks can be avoided by always acquiring multiple :term:`locks ` in a consistent order. See also :term:`lock` and :term:`reentrant`. decorator A function returning another function, usually applied as a function transformation using the ``@wrapper`` syntax. Common examples for decorators are :deco:`classmethod` and :deco:`staticmethod`. The decorator syntax is merely syntactic sugar, the following two function definitions are semantically equivalent:: def f(arg): ... f = staticmethod(f) @staticmethod def f(arg): ... The same concept exists for classes, but is less commonly used there. See the documentation for :ref:`function definitions ` and :ref:`class definitions ` for more about decorators. descriptor Any object which defines the methods :meth:`~object.__get__`, :meth:`~object.__set__`, or :meth:`~object.__delete__`. When a class attribute is a descriptor, its special binding behavior is triggered upon attribute lookup. Normally, using *a.b* to get, set or delete an attribute looks up the object named *b* in the class dictionary for *a*, but if *b* is a descriptor, the respective descriptor method gets called. Understanding descriptors is a key to a deep understanding of Python because they are the basis for many features including functions, methods, properties, class methods, static methods, and reference to super classes. For more information about descriptors' methods, see :ref:`descriptors` or the :ref:`Descriptor How To Guide `. dictionary An associative array, where arbitrary keys are mapped to values. The keys can be any object with :meth:`~object.__hash__` and :meth:`~object.__eq__` methods. Called a hash in Perl. dictionary comprehension A compact way to process all or part of the elements in an iterable and return a dictionary with the results. ``results = {n: n ** 2 for n in range(10)}`` generates a dictionary containing key ``n`` mapped to value ``n ** 2``. See :ref:`comprehensions`. dictionary view The objects returned from :meth:`dict.keys`, :meth:`dict.values`, and :meth:`dict.items` are called dictionary views. They provide a dynamic view on the dictionary’s entries, which means that when the dictionary changes, the view reflects these changes. To force the dictionary view to become a full list use ``list(dictview)``. See :ref:`dict-views`. docstring A string literal which appears as the first expression in a class, function or module. While ignored when the suite is executed, it is recognized by the compiler and put into the :attr:`~definition.__doc__` attribute of the enclosing class, function or module. Since it is available via introspection, it is the canonical place for documentation of the object. duck-typing A programming style which does not look at an object's type to determine if it has the right interface; instead, the method or attribute is simply called or used ("If it looks like a duck and quacks like a duck, it must be a duck.") By emphasizing interfaces rather than specific types, well-designed code improves its flexibility by allowing polymorphic substitution. Duck-typing avoids tests using :func:`type` or :func:`isinstance`. (Note, however, that duck-typing can be complemented with :term:`abstract base classes `.) Instead, it typically employs :func:`hasattr` tests or :term:`EAFP` programming. dunder An informal short-hand for "double underscore", used when talking about a :term:`special method`. For example, ``__init__`` is often pronounced "dunder init". EAFP Easier to ask for forgiveness than permission. This common Python coding style assumes the existence of valid keys or attributes and catches exceptions if the assumption proves false. This clean and fast style is characterized by the presence of many :keyword:`try` and :keyword:`except` statements. The technique contrasts with the :term:`LBYL` style common to many other languages such as C. evaluate function A function that can be called to evaluate a lazily evaluated attribute of an object, such as the value of type aliases created with the :keyword:`type` statement. exhausted An :term:`iterator` that has produced all of its values is said to be :dfn:`exhausted`. Further attempts to get the next value (for example, calls to :func:`next`) raise :exc:`StopIteration` (or :exc:`StopAsyncIteration` in the case of an :term:`asynchronous iterator`). expression A piece of syntax which can be evaluated to some value. In other words, an expression is an accumulation of expression elements like literals, names, attribute access, operators or function calls which all return a value. Not all language constructs are expressions. There are also :term:`statement`\s which cannot be used as expressions, such as :keyword:`while`. Assignments are also statements, not expressions. extension module A module written in C or C++, using Python's C API to interact with the core and with user code. f-string f-strings String literals prefixed with ``f`` or ``F`` are commonly called "f-strings" which is short for :ref:`formatted string literals `. See also :pep:`498`. file object An object exposing a file-oriented API (with methods such as :meth:`!read` or :meth:`!write`) to an underlying resource. Depending on the way it was created, a file object can mediate access to a real on-disk file or to another type of storage or communication device (for example standard input/output, in-memory buffers, sockets, pipes, etc.). File objects are also called :dfn:`file-like objects` or :dfn:`streams`. There are actually three categories of file objects: raw :term:`binary files `, buffered :term:`binary files ` and :term:`text files `. Their interfaces are defined in the :mod:`io` module. The canonical way to create a file object is by using the :func:`open` function. file-like object A synonym for :term:`file object`. filesystem encoding and error handler Encoding and error handler used by Python to decode bytes from the operating system and encode Unicode to the operating system. The filesystem encoding must guarantee to successfully decode all bytes below 128. If the file system encoding fails to provide this guarantee, API functions can raise :exc:`UnicodeError`. The :func:`sys.getfilesystemencoding` and :func:`sys.getfilesystemencodeerrors` functions can be used to get the filesystem encoding and error handler. The :term:`filesystem encoding and error handler` are configured at Python startup by the :c:func:`PyConfig_Read` function: see :c:member:`~PyConfig.filesystem_encoding` and :c:member:`~PyConfig.filesystem_errors` members of :c:type:`PyConfig`. See also the :term:`locale encoding`. finder An object that tries to find the :term:`loader` for a module that is being imported. There are two types of finder: :term:`meta path finders ` for use with :data:`sys.meta_path`, and :term:`path entry finders ` for use with :data:`sys.path_hooks`. See :ref:`finders-and-loaders` and :mod:`importlib` for much more detail. floor division Mathematical division that rounds down to nearest integer. The floor division operator is ``//``. For example, the expression ``11 // 4`` evaluates to ``2`` in contrast to the ``2.75`` returned by float true division. Note that ``(-11) // 4`` is ``-3`` because that is ``-2.75`` rounded *downward*. See :pep:`238`. free threading A threading model where multiple threads can run Python bytecode simultaneously within the same interpreter. This is in contrast to the :term:`global interpreter lock` which allows only one thread to execute Python bytecode at a time. See :pep:`703`. free-threaded build A build of :term:`CPython` that supports :term:`free threading`, configured using the :option:`--disable-gil` option before compilation. See :ref:`freethreading-python-howto`. free variable Formally, as defined in the :ref:`language execution model `, a free variable is any variable used in a namespace which is not a local variable in that namespace. See :term:`closure variable` for an example. Pragmatically, due to the name of the :attr:`codeobject.co_freevars` attribute, the term is also sometimes used as a synonym for :term:`closure variable`. function A series of statements which returns some value to a caller. It can also be passed zero or more :term:`arguments ` which may be used in the execution of the body. See also :term:`parameter`, :term:`method`, and the :ref:`function` section. function annotation An :term:`annotation` of a function parameter or return value. Function annotations are usually used for :term:`type hints `: for example, this function is expected to take two :class:`int` arguments and is also expected to have an :class:`int` return value:: def sum_two_numbers(a: int, b: int) -> int: return a + b Function annotation syntax is explained in section :ref:`function`. See :term:`variable annotation` and :pep:`484`, which describe this functionality. Also see :ref:`annotations-howto` for best practices on working with annotations. __future__ A :ref:`future statement `, ``from __future__ import ``, directs the compiler to compile the current module using syntax or semantics that will become standard in a future release of Python. The :mod:`__future__` module documents the possible values of *feature*. By importing this module and evaluating its variables, you can see when a new feature was first added to the language and when it will (or did) become the default:: >>> import __future__ >>> __future__.division _Feature((2, 2, 0, 'alpha', 2), (3, 0, 0, 'alpha', 0), 8192) garbage collection The process of freeing memory when it is not used anymore. Python performs garbage collection via reference counting and a cyclic garbage collector that is able to detect and break reference cycles. The garbage collector can be controlled using the :mod:`gc` module. .. index:: single: generator generator Informally used to mean either a :term:`generator function` or a :term:`generator iterator`, depending on context. The formal terms :term:`generator function` and :term:`generator iterator` are uncommon in practice; "generator" alone is almost always sufficient. .. index:: single: generator function generator function A function which returns a :term:`generator` object. It looks like a normal function except that it contains :keyword:`yield` expressions for producing a series of values usable in a :keyword:`for`\-loop or that can be retrieved one at a time with the :func:`next` function. See :ref:`yieldexpr`. generator iterator An object created by a :term:`generator function` or a :term:`generator expression`. Each :keyword:`yield` temporarily suspends processing, remembering the execution state (including local variables and pending try-statements). When the *generator iterator* resumes, it picks up where it left off (in contrast to functions which start fresh on every invocation). Generator iterators also implement the :meth:`~generator.send` method to send a value into the suspended generator, and the :meth:`~generator.throw` method to raise an exception at the point where the generator was paused. See :ref:`generator-methods`. .. index:: single: generator expression generator expression An :term:`expression` that returns an :term:`iterator`. It looks like a normal expression followed by a :keyword:`!for` clause defining a loop variable, range, and an optional :keyword:`!if` clause. The combined expression generates values for an enclosing function:: >>> sum(i*i for i in range(10)) # sum of squares 0, 1, 4, ... 81 285 generic function A function composed of multiple functions implementing the same operation for different types. Which implementation should be used during a call is determined by the dispatch algorithm. See also the :term:`single dispatch` glossary entry, the :deco:`functools.singledispatch` decorator, and :pep:`443`. generic type A :term:`type` that can be parameterized; typically a :ref:`container class` such as :class:`list` or :class:`dict`. Used for :term:`type hints ` and :term:`annotations `. For more details, see :ref:`generic alias types`, :pep:`483`, :pep:`484`, :pep:`585`, and the :mod:`typing` module. GIL See :term:`global interpreter lock`. global interpreter lock The mechanism used by the :term:`CPython` interpreter to assure that only one thread executes Python :term:`bytecode` at a time. This simplifies the CPython implementation by making the object model (including critical built-in types such as :class:`dict`) implicitly safe against concurrent access. Locking the entire interpreter makes it easier for the interpreter to be multi-threaded, at the expense of much of the parallelism afforded by multi-processor machines. However, some extension modules, either standard or third-party, are designed so as to release the GIL when doing computationally intensive tasks such as compression or hashing. Also, the GIL is always released when doing I/O. As of Python 3.13, the GIL can be disabled using the :option:`--disable-gil` build configuration. After building Python with this option, code must be run with :option:`-X gil=0 <-X>` or after setting the :envvar:`PYTHON_GIL=0 ` environment variable. This feature enables improved performance for multi-threaded applications and makes it easier to use multi-core CPUs efficiently. For more details, see :pep:`703`. In prior versions of Python's C API, a function might declare that it requires the GIL to be held in order to use it. This refers to having an :term:`attached thread state`. global state Data that is accessible throughout a program, such as module-level variables, class variables, or C static variables in :term:`extension modules `. In multi-threaded programs, global state shared between threads typically requires synchronization to avoid :term:`race conditions ` and :term:`data races `. hash-based pyc A bytecode cache file that uses the hash rather than the last-modified time of the corresponding source file to determine its validity. See :ref:`pyc-invalidation`. hashable An object is *hashable* if it has a hash value which never changes during its lifetime (it needs a :meth:`~object.__hash__` method), and can be compared to other objects (it needs an :meth:`~object.__eq__` method). Hashable objects which compare equal must have the same hash value. Hashability makes an object usable as a dictionary key and a set member, because these data structures use the hash value internally. Most of Python's immutable built-in objects are hashable; mutable containers (such as lists or dictionaries) are not; immutable containers (such as tuples and frozensets) are only hashable if their elements are hashable. Objects which are instances of user-defined classes are hashable by default. They all compare unequal (except with themselves), and their hash value is derived from their :func:`id`. IDLE An Integrated Development and Learning Environment for Python. :ref:`idle` is a basic editor and interpreter environment which ships with the standard distribution of Python. immortal *Immortal objects* are a CPython implementation detail introduced in :pep:`683`. If an object is immortal, its :term:`reference count` is never modified, and therefore it is never deallocated while the interpreter is running. For example, :const:`True` and :const:`None` are immortal in CPython. Immortal objects can be identified via :func:`sys._is_immortal`, or via :c:func:`PyUnstable_IsImmortal` in the C API. immutable An object with a fixed value. Immutable objects include numbers, strings and tuples. Such an object cannot be altered. A new object has to be created if a different value has to be stored. They play an important role in places where a constant hash value is needed, for example as a key in a dictionary. Immutable objects are inherently :term:`thread-safe` because their state cannot be modified after creation, eliminating concerns about improperly synchronized :term:`concurrent modification`. import path A list of locations (or :term:`path entries `) that are searched by the :term:`path based finder` for modules to import. During import, this list of locations usually comes from :data:`sys.path`, but for subpackages it may also come from the parent package's ``__path__`` attribute. importing The process by which Python code in one module is made available to Python code in another module. importer An object that both finds and loads a module; both a :term:`finder` and :term:`loader` object. index A numeric value that represents the position of an element in a :term:`sequence`. In Python, indexing starts at zero. For example, ``things[0]`` names the *first* element of ``things``; ``things[1]`` names the second one. In some contexts, Python allows negative indexes for counting from the end of a sequence, and indexing using :term:`slices `. See also :term:`subscript`. interactive Python has an interactive interpreter which means you can enter statements and expressions at the interpreter prompt, immediately execute them and see their results. Just launch ``python`` with no arguments (possibly by selecting it from your computer's main menu). It is a very powerful way to test out new ideas or inspect modules and packages (remember ``help(x)``). For more on interactive mode, see :ref:`tut-interac`. interpreted Python is an interpreted language, as opposed to a compiled one, though the distinction can be blurry because of the presence of the bytecode compiler. This means that source files can be run directly without explicitly creating an executable which is then run. Interpreted languages typically have a shorter development/debug cycle than compiled ones, though their programs generally also run more slowly. See also :term:`interactive`. interpreter shutdown When asked to shut down, the Python interpreter enters a special phase where it gradually releases all allocated resources, such as modules and various critical internal structures. It also makes several calls to the :term:`garbage collector `. This can trigger the execution of code in user-defined destructors or weakref callbacks. Code executed during the shutdown phase can encounter various exceptions as the resources it relies on may not function anymore (common examples are library modules or the warnings machinery). The main reason for interpreter shutdown is that the ``__main__`` module or the script being run has finished executing. iterable An object capable of returning its members one at a time. Examples of iterables include all sequence types (such as :class:`list`, :class:`str`, and :class:`tuple`) and some non-sequence types like :class:`dict`, :term:`file objects `, and objects of any classes you define with an :meth:`~object.__iter__` method or with a :meth:`~object.__getitem__` method that implements :term:`sequence` semantics. Iterables can be used in a :keyword:`for` loop and in many other places where a sequence is needed (:func:`zip`, :func:`map`, ...). When an iterable object is passed as an argument to the built-in function :func:`iter`, it returns an iterator for the object. This iterator is good for one pass over the set of values. When using iterables, it is usually not necessary to call :func:`iter` or deal with iterator objects yourself. The :keyword:`for` statement does that automatically for you, creating a temporary unnamed variable to hold the iterator for the duration of the loop. See also :term:`iterator`, :term:`sequence`, and :term:`generator`. iterator An object representing a stream of data. Repeated calls to the iterator's :meth:`~iterator.__next__` method (or passing it to the built-in function :func:`next`) return successive items in the stream. When no more data are available a :exc:`StopIteration` exception is raised instead. At this point, the iterator object is :term:`exhausted` and any further calls to its :meth:`!__next__` method just raise :exc:`StopIteration` again. Iterators are required to have an :meth:`~iterator.__iter__` method that returns the iterator object itself so every iterator is also iterable and may be used in most places where other iterables are accepted. One notable exception is code which attempts multiple iteration passes. A container object (such as a :class:`list`) produces a fresh new iterator each time you pass it to the :func:`iter` function or use it in a :keyword:`for` loop. Attempting this with an iterator will just return the same exhausted iterator object used in the previous iteration pass, making it appear like an empty container. More information can be found in :ref:`typeiter`. .. impl-detail:: CPython does not consistently apply the requirement that an iterator define :meth:`~iterator.__iter__`. And also please note that :term:`free-threaded ` CPython does not guarantee :term:`thread-safe` behavior of iterator operations. key A value that identifies an entry in a :term:`mapping`. See also :term:`subscript`. key function A key function or collation function is a callable that returns a value used for sorting or ordering. For example, :func:`locale.strxfrm` is used to produce a sort key that is aware of locale specific sort conventions. A number of tools in Python accept key functions to control how elements are ordered or grouped. They include :func:`min`, :func:`max`, :func:`sorted`, :meth:`list.sort`, :func:`heapq.merge`, :func:`heapq.nsmallest`, :func:`heapq.nlargest`, and :func:`itertools.groupby`. There are several ways to create a key function. For example. the :meth:`str.casefold` method can serve as a key function for case insensitive sorts. Alternatively, a key function can be built from a :keyword:`lambda` expression such as ``lambda r: (r[0], r[2])``. Also, :func:`operator.attrgetter`, :func:`operator.itemgetter`, and :func:`operator.methodcaller` are three key function constructors. See the :ref:`Sorting HOW TO ` for examples of how to create and use key functions. keyword argument See :term:`argument`. lambda An anonymous inline function consisting of a single :term:`expression` which is evaluated when the function is called. The syntax to create a lambda function is ``lambda [parameters]: expression`` LBYL Look before you leap. This coding style explicitly tests for pre-conditions before making calls or lookups. This style contrasts with the :term:`EAFP` approach and is characterized by the presence of many :keyword:`if` statements. In a multi-threaded environment, the LBYL approach can risk introducing a :term:`race condition` between "the looking" and "the leaping". For example, the code, ``if key in mapping: return mapping[key]`` can fail if another thread removes *key* from *mapping* after the test, but before the lookup. This issue can be solved with :term:`locks ` or by using the :term:`EAFP` approach. See also :term:`thread-safe`. lexical analyzer Formal name for the *tokenizer*; see :term:`token`. list A built-in Python :term:`sequence`. Despite its name it is more akin to an array in other languages than to a linked list since access to elements is *O*\ (1). See :ref:`time-complexity`. list comprehension A compact way to process all or part of the elements in a sequence and return a list with the results. ``result = ['{:#04x}'.format(x) for x in range(256) if x % 2 == 0]`` generates a list of strings containing even hex numbers (0x..) in the range from 0 to 255. The :keyword:`if` clause is optional. If omitted, all elements in ``range(256)`` are processed. lock A :term:`synchronization primitive` that allows only one thread at a time to access a shared resource. A thread must acquire a lock before accessing the protected resource and release it afterward. If a thread attempts to acquire a lock that is already held by another thread, it will block until the lock becomes available. Python's :mod:`threading` module provides :class:`~threading.Lock` (a basic lock) and :class:`~threading.RLock` (a :term:`reentrant` lock). Locks are used to prevent :term:`race conditions ` and ensure :term:`thread-safe` access to shared data. Alternative design patterns to locks exist such as queues, producer/consumer patterns, and thread-local state. See also :term:`deadlock`, and :term:`reentrant`. lock-free An operation that does not acquire any :term:`lock` and uses atomic CPU instructions to ensure correctness. Lock-free operations can execute concurrently without blocking each other and cannot be blocked by operations that hold locks. In :term:`free-threaded ` Python, built-in types like :class:`dict` and :class:`list` provide lock-free read operations, which means other threads may observe intermediate states during multi-step modifications even when those modifications hold the :term:`per-object lock`. loader An object that loads a module. It must define the :meth:`!exec_module` and :meth:`!create_module` methods to implement the :class:`~importlib.abc.Loader` interface. A loader is typically returned by a :term:`finder`. See also: * :ref:`finders-and-loaders` * :class:`importlib.abc.Loader` * :pep:`302` locale encoding On Unix, it is the encoding of the LC_CTYPE locale. It can be set with :func:`locale.setlocale(locale.LC_CTYPE, new_locale) `. On Windows, it is the ANSI code page (ex: ``"cp1252"``). On Android and VxWorks, Python uses ``"utf-8"`` as the locale encoding. :func:`locale.getencoding` can be used to get the locale encoding. See also the :term:`filesystem encoding and error handler`. magic method .. index:: pair: magic; method An informal synonym for :term:`special method`. mapping A container object that supports arbitrary key lookups and implements the methods specified in the :class:`collections.abc.Mapping` or :class:`collections.abc.MutableMapping` :ref:`abstract base classes `. Examples include :class:`dict`, :class:`collections.defaultdict`, :class:`collections.OrderedDict` and :class:`collections.Counter`. meta path finder A :term:`finder` returned by a search of :data:`sys.meta_path`. Meta path finders are related to, but different from :term:`path entry finders `. See :class:`importlib.abc.MetaPathFinder` for the methods that meta path finders implement. metaclass The class of a class. Class definitions create a class name, a class dictionary, and a list of base classes. The metaclass is responsible for taking those three arguments and creating the class. Most object oriented programming languages provide a default implementation. What makes Python special is that it is possible to create custom metaclasses. Most users never need this tool, but when the need arises, metaclasses can provide powerful, elegant solutions. They have been used for logging attribute access, adding thread-safety, tracking object creation, implementing singletons, and many other tasks. More information can be found in :ref:`metaclasses`. method A function which is defined inside a class body. If called as an attribute of an instance of that class, the method will get the instance object as its first :term:`argument` (which is usually called ``self``). See :term:`function` and :term:`nested scope`. method resolution order Method Resolution Order is the order in which base classes are searched for a member during lookup. See :ref:`python_2.3_mro` for details of the algorithm used by the Python interpreter since the 2.3 release. module An object that serves as an organizational unit of Python code. Modules have a namespace containing arbitrary Python objects. Modules are loaded into Python by the process of :term:`importing`. See also :term:`package`. module spec A namespace containing the import-related information used to load a module. An instance of :class:`importlib.machinery.ModuleSpec`. See also :ref:`module-specs`. MRO See :term:`method resolution order`. mutable An :term:`object` with state that is allowed to change during the course of the program. In multi-threaded programs, mutable objects that are shared between threads require careful synchronization to avoid :term:`race conditions `. See also :term:`immutable`, :term:`thread-safe`, and :term:`concurrent modification`. named tuple The term "named tuple" applies to any type or class that inherits from tuple and whose indexable elements are also accessible using named attributes. The type or class may have other features as well. Several built-in types are named tuples, including the values returned by :func:`time.localtime` and :func:`os.stat`. Another example is :data:`sys.float_info`:: >>> sys.float_info[1] # indexed access 1024 >>> sys.float_info.max_exp # named field access 1024 >>> isinstance(sys.float_info, tuple) # kind of tuple True Some named tuples are built-in types (such as the above examples). Alternatively, a named tuple can be created from a regular class definition that inherits from :class:`tuple` and that defines named fields. Such a class can be written by hand, or it can be created by inheriting :class:`typing.NamedTuple`, or with the factory function :func:`collections.namedtuple`. The latter techniques also add some extra methods that may not be found in hand-written or built-in named tuples. namespace The place where a variable is stored. Namespaces are implemented as dictionaries. There are the local, global and built-in namespaces as well as nested namespaces in objects (in methods). Namespaces support modularity by preventing naming conflicts. For instance, the functions :func:`builtins.open <.open>` and :func:`os.open` are distinguished by their namespaces. Namespaces also aid readability and maintainability by making it clear which module implements a function. For instance, writing :func:`random.seed` or :func:`itertools.islice` makes it clear that those functions are implemented by the :mod:`random` and :mod:`itertools` modules, respectively. namespace package A :term:`package` which serves only as a container for subpackages. Namespace packages may have no physical representation, and specifically are not like a :term:`regular package` because they have no ``__init__.py`` file. Namespace packages allow several individually installable packages to have a common parent package. Otherwise, it is recommended to use a :term:`regular package`. For more information, see :pep:`420` and :ref:`reference-namespace-package`. See also :term:`module`. native code Code that is compiled to machine instructions and runs directly on the processor, as opposed to code that is interpreted or runs in a virtual machine. In the context of Python, native code typically refers to C, C++, Rust or Fortran code in :term:`extension modules ` that can be called from Python. See also :term:`extension module`. nested scope The ability to refer to a variable in an enclosing definition. For instance, a function defined inside another function can refer to variables in the outer function. Note that nested scopes by default work only for reference and not for assignment. Local variables both read and write in the innermost scope. Likewise, global variables read and write to the global namespace. The :keyword:`nonlocal` allows writing to outer scopes. new-style class Old name for the flavor of classes now used for all class objects. In earlier Python versions, only new-style classes could use Python's newer, versatile features like :attr:`~object.__slots__`, descriptors, properties, :meth:`~object.__getattribute__`, class methods, and static methods. non-deterministic Behavior where the outcome of a program can vary between executions with the same inputs. In multi-threaded programs, non-deterministic behavior often results from :term:`race conditions ` where the relative timing or interleaving of threads affects the result. Proper synchronization using :term:`locks ` and other :term:`synchronization primitives ` helps ensure deterministic behavior. object Any data with state (attributes or value) and defined behavior (methods). Also the ultimate base class of any :term:`new-style class`. optimized scope A scope where target local variable names are reliably known to the compiler when the code is compiled, allowing optimization of read and write access to these names. The local namespaces for functions, generators, coroutines, comprehensions, and generator expressions are optimized in this fashion. Note: most interpreter optimizations are applied to all scopes, only those relying on a known set of local and nonlocal variable names are restricted to optimized scopes. optional module An :term:`extension module` that is part of the :term:`standard library`, but may be absent in some builds of :term:`CPython`, usually due to missing third-party libraries or because the module is not available for a given platform. See :ref:`optional-module-requirements` for a list of optional modules that require third-party libraries. package A Python :term:`module` which can contain submodules or recursively, subpackages. Technically, a package is a Python module with a ``__path__`` attribute. See also :term:`regular package` and :term:`namespace package`. parallelism Executing multiple operations at the same time (e.g. on multiple CPU cores). In Python builds with the :term:`global interpreter lock (GIL) `, only one thread runs Python bytecode at a time, so taking advantage of multiple CPU cores typically involves multiple processes (e.g. :mod:`multiprocessing`) or native extensions that release the GIL. In :term:`free-threaded ` Python, multiple Python threads can run Python code simultaneously on different cores. parameter A named entity in a :term:`function` (or method) definition that specifies an :term:`argument` (or in some cases, arguments) that the function can accept. There are five kinds of parameter: * :dfn:`positional-or-keyword`: specifies an argument that can be passed either :term:`positionally ` or as a :term:`keyword argument `. This is the default kind of parameter, for example *foo* and *bar* in the following:: def func(foo, bar=None): ... .. _positional-only_parameter: * :dfn:`positional-only`: specifies an argument that can be supplied only by position. Positional-only parameters can be defined by including a ``/`` character in the parameter list of the function definition after them, for example *posonly1* and *posonly2* in the following:: def func(posonly1, posonly2, /, positional_or_keyword): ... .. _keyword-only_parameter: * :dfn:`keyword-only`: specifies an argument that can be supplied only by keyword. Keyword-only parameters can be defined by including a single var-positional parameter or bare ``*`` in the parameter list of the function definition before them, for example *kw_only1* and *kw_only2* in the following:: def func(arg, *, kw_only1, kw_only2): ... * :dfn:`var-positional`: specifies that an arbitrary sequence of positional arguments can be provided (in addition to any positional arguments already accepted by other parameters). Such a parameter can be defined by prepending the parameter name with ``*``, for example *args* in the following:: def func(*args, **kwargs): ... * :dfn:`var-keyword`: specifies that arbitrarily many keyword arguments can be provided (in addition to any keyword arguments already accepted by other parameters). Such a parameter can be defined by prepending the parameter name with ``**``, for example *kwargs* in the example above. Parameters can specify both optional and required arguments, as well as default values for some optional arguments. See also the :term:`argument` glossary entry, the FAQ question on :ref:`the difference between arguments and parameters `, the :class:`inspect.Parameter` class, the :ref:`function` section, and :pep:`362`. per-object lock A :term:`lock` associated with an individual object instance rather than a global lock shared across all objects. In :term:`free-threaded ` Python, built-in types like :class:`dict` and :class:`list` use per-object locks to allow concurrent operations on different objects while serializing operations on the same object. Operations that hold the per-object lock prevent other locking operations on the same object from proceeding, but do not block :term:`lock-free` operations. path entry A single location on the :term:`import path` which the :term:`path based finder` consults to find modules for importing. path entry finder A :term:`finder` returned by a callable on :data:`sys.path_hooks` (i.e. a :term:`path entry hook`) which knows how to locate modules given a :term:`path entry`. See :class:`importlib.abc.PathEntryFinder` for the methods that path entry finders implement. path entry hook A callable on the :data:`sys.path_hooks` list which returns a :term:`path entry finder` if it knows how to find modules on a specific :term:`path entry`. path based finder One of the default :term:`meta path finders ` which searches an :term:`import path` for modules. path-like object An object representing a file system path. A path-like object is either a :class:`str` or :class:`bytes` object representing a path, or an object implementing the :class:`os.PathLike` protocol. An object that supports the :class:`os.PathLike` protocol can be converted to a :class:`str` or :class:`bytes` file system path by calling the :func:`os.fspath` function; :func:`os.fsdecode` and :func:`os.fsencode` can be used to guarantee a :class:`str` or :class:`bytes` result instead, respectively. Introduced by :pep:`519`. PEP Python Enhancement Proposal. A PEP is a design document providing information to the Python community, or describing a new feature for Python or its processes or environment. PEPs should provide a concise technical specification and a rationale for proposed features. PEPs are intended to be the primary mechanisms for proposing major new features, for collecting community input on an issue, and for documenting the design decisions that have gone into Python. The PEP author is responsible for building consensus within the community and documenting dissenting opinions. See :pep:`1`. portion A set of files in a single directory (possibly stored in a zip file) that contribute to a namespace package, as defined in :pep:`420`. positional argument See :term:`argument`. provisional API A provisional API is one which has been deliberately excluded from the standard library's backwards compatibility guarantees. While major changes to such interfaces are not expected, as long as they are marked provisional, backwards incompatible changes (up to and including removal of the interface) may occur if deemed necessary by core developers. Such changes will not be made gratuitously -- they will occur only if serious fundamental flaws are uncovered that were missed prior to the inclusion of the API. Even for provisional APIs, backwards incompatible changes are seen as a "solution of last resort" - every attempt will still be made to find a backwards compatible resolution to any identified problems. This process allows the standard library to continue to evolve over time, without locking in problematic design errors for extended periods of time. See :pep:`411` for more details. provisional package See :term:`provisional API`. Python 3000 Nickname for the Python 3.x release line (coined long ago when the release of version 3 was something in the distant future.) This is also abbreviated "Py3k". Pythonic An idea or piece of code which closely follows the most common idioms of the Python language, rather than implementing code using concepts common to other languages. For example, a common idiom in Python is to loop over all elements of an iterable using a :keyword:`for` statement. Many other languages don't have this type of construct, so people unfamiliar with Python sometimes use a numerical counter instead:: for i in range(len(food)): print(food[i]) As opposed to the cleaner, Pythonic method:: for piece in food: print(piece) qualified name A dotted name showing the "path" from a module's global scope to a class, function or method defined in that module, as defined in :pep:`3155`. For top-level functions and classes, the qualified name is the same as the object's name:: >>> class C: ... class D: ... def meth(self): ... pass ... >>> C.__qualname__ 'C' >>> C.D.__qualname__ 'C.D' >>> C.D.meth.__qualname__ 'C.D.meth' When used to refer to modules, the *fully qualified name* means the entire dotted path to the module, including any parent packages, e.g. ``email.mime.text``:: >>> import email.mime.text >>> email.mime.text.__name__ 'email.mime.text' race condition A condition of a program where the behavior depends on the relative timing or ordering of events, particularly in multi-threaded programs. Race conditions can lead to :term:`non-deterministic` behavior and bugs that are difficult to reproduce. A :term:`data race` is a specific type of race condition involving unsynchronized access to shared memory. The :term:`LBYL` coding style is particularly susceptible to race conditions in multi-threaded code. Using :term:`locks ` and other :term:`synchronization primitives ` helps prevent race conditions. reference count The number of references to an object. When the reference count of an object drops to zero, it is deallocated. Some objects are :term:`immortal` and have reference counts that are never modified, and therefore the objects are never deallocated. Reference counting is generally not visible to Python code, but it is a key element of the :term:`CPython` implementation. Programmers can call the :func:`sys.getrefcount` function to return the reference count for a particular object. In :term:`CPython`, reference counts are not considered to be stable or well-defined values; the number of references to an object, and how that number is affected by Python code, may be different between versions. regular package A traditional :term:`package`, such as a directory containing an ``__init__.py`` file. See also :term:`namespace package`. reentrant A property of a function or :term:`lock` that allows it to be called or acquired multiple times by the same thread without causing errors or a :term:`deadlock`. For functions, reentrancy means the function can be safely called again before a previous invocation has completed, which is important when functions may be called recursively or from signal handlers. Thread-unsafe functions may be :term:`non-deterministic` if they're called reentrantly in a multithreaded program. For locks, Python's :class:`threading.RLock` (reentrant lock) is reentrant, meaning a thread that already holds the lock can acquire it again without blocking. In contrast, :class:`threading.Lock` is not reentrant - attempting to acquire it twice from the same thread will cause a deadlock. See also :term:`lock` and :term:`deadlock`. REPL An acronym for the "read–eval–print loop", another name for the :term:`interactive` interpreter shell. __slots__ A declaration inside a class that saves memory by pre-declaring space for instance attributes and eliminating instance dictionaries. Though popular, the technique is somewhat tricky to get right and is best reserved for rare cases where there are large numbers of instances in a memory-critical application. sequence An :term:`iterable` which supports efficient element access using integer indices via the :meth:`~object.__getitem__` special method and defines a :meth:`~object.__len__` method that returns the length of the sequence. Some built-in sequence types are :class:`list`, :class:`str`, :class:`tuple`, and :class:`bytes`. Note that :class:`dict` also supports :meth:`~object.__getitem__` and :meth:`!__len__`, but is considered a mapping rather than a sequence because the lookups use arbitrary :term:`hashable` keys rather than integers. The :class:`collections.abc.Sequence` abstract base class defines a much richer interface that goes beyond just :meth:`~object.__getitem__` and :meth:`~object.__len__`, adding :meth:`~sequence.count`, :meth:`~sequence.index`, :meth:`~object.__contains__`, and :meth:`~object.__reversed__`. Types that implement this expanded interface can be registered explicitly using :func:`~abc.ABCMeta.register`. For more documentation on sequence methods generally, see :ref:`Common Sequence Operations `. set comprehension A compact way to process all or part of the elements in an iterable and return a set with the results. ``results = {c for c in 'abracadabra' if c not in 'abc'}`` generates the set of strings ``{'r', 'd'}``. See :ref:`comprehensions`. single dispatch A form of :term:`generic function` dispatch where the implementation is chosen based on the type of a single argument. slice An object of type :class:`slice`, used to describe a portion of a :term:`sequence`. A slice object is created when using the :ref:`slicing ` form of :ref:`subscript notation `, with colons inside square brackets, such as in ``variable_name[1:3:5]``. soft deprecated A soft deprecated API should not be used in new code, but it is safe for already existing code to use it. The API remains documented and tested, but will not be enhanced further. Soft deprecation, unlike normal deprecation, does not plan on removing the API and will not emit warnings. See `PEP 387: Soft Deprecation `_. special method .. index:: pair: special; method A method that is called implicitly by Python to execute a certain operation on a type, such as addition. Such methods have names starting and ending with double underscores. Special methods are documented in :ref:`specialnames`. standard library The collection of :term:`packages `, :term:`modules ` and :term:`extension modules ` distributed as a part of the official Python interpreter package. The exact membership of the collection may vary based on platform, available system libraries, or other criteria. Documentation can be found at :ref:`library-index`. See also :data:`sys.stdlib_module_names` for a list of all possible standard library module names. statement A statement is part of a suite (a "block" of code). A statement is either an :term:`expression` or one of several constructs with a keyword, such as :keyword:`if`, :keyword:`while` or :keyword:`for`. static type checker An external tool that reads Python code and analyzes it, looking for issues such as incorrect types. See also :term:`type hints ` and the :mod:`typing` module. stdlib An abbreviation of :term:`standard library`. steal In Python's C API, "*stealing*" an argument means that ownership of the argument is transferred to the called function. The caller must not use that reference after the call. Generally, functions that "steal" an argument do so even if they fail. See :ref:`api-refcountdetails` for a full explanation. strong reference In Python's C API, a strong reference is a reference to an object which is owned by the code holding the reference. The strong reference is taken by calling :c:func:`Py_INCREF` when the reference is created and released with :c:func:`Py_DECREF` when the reference is deleted. The :c:func:`Py_NewRef` function can be used to create a strong reference to an object. Usually, the :c:func:`Py_DECREF` function must be called on the strong reference before exiting the scope of the strong reference, to avoid leaking one reference. See also :term:`borrowed reference`. subscript The expression in square brackets of a :ref:`subscription expression `, for example, the ``3`` in ``items[3]``. Usually used to select an element of a container. Also called a :term:`key` when subscripting a :term:`mapping`, or an :term:`index` when subscripting a :term:`sequence`. synchronization primitive A basic building block for coordinating (synchronizing) the execution of multiple threads to ensure :term:`thread-safe` access to shared resources. Python's :mod:`threading` module provides several synchronization primitives including :class:`~threading.Lock`, :class:`~threading.RLock`, :class:`~threading.Semaphore`, :class:`~threading.Condition`, :class:`~threading.Event`, and :class:`~threading.Barrier`. Additionally, the :mod:`queue` module provides multi-producer, multi-consumer queues that are especially useful in multithreaded programs. These primitives help prevent :term:`race conditions ` and coordinate thread execution. See also :term:`lock`. t-string t-strings String literals prefixed with ``t`` or ``T`` are commonly called "t-strings" which is short for :ref:`template string literals `. text encoding A string in Python is a sequence of Unicode code points (in range ``U+0000``--``U+10FFFF``). To store or transfer a string, it needs to be serialized as a sequence of bytes. Serializing a string into a sequence of bytes is known as "encoding", and recreating the string from the sequence of bytes is known as "decoding". There are a variety of different text serialization :ref:`codecs `, which are collectively referred to as "text encodings". text file A :term:`file object` able to read and write :class:`str` objects. Often, a text file actually accesses a byte-oriented datastream and handles the :term:`text encoding` automatically. Examples of text files are files opened in text mode (``'r'`` or ``'w'``), :data:`sys.stdin`, :data:`sys.stdout`, and instances of :class:`io.StringIO`. See also :term:`binary file` for a file object able to read and write :term:`bytes-like objects `. thread state The information used by the :term:`CPython` runtime to run in an OS thread. For example, this includes the current exception, if any, and the state of the bytecode interpreter. Each thread state is bound to a single OS thread, but threads may have many thread states available. At most, one of them may be :term:`attached ` at once. An :term:`attached thread state` is required to call most of Python's C API, unless a function explicitly documents otherwise. The bytecode interpreter only runs under an attached thread state. Each thread state belongs to a single interpreter, but each interpreter may have many thread states, including multiple for the same OS thread. Thread states from multiple interpreters may be bound to the same thread, but only one can be :term:`attached ` in that thread at any given moment. See :ref:`Thread State and the Global Interpreter Lock ` for more information. thread-safe A module, function, or class that behaves correctly when used by multiple threads concurrently. Thread-safe code uses appropriate :term:`synchronization primitives ` like :term:`locks ` to protect shared mutable state, or is designed to avoid shared mutable state entirely. In the :term:`free-threaded ` build, built-in types like :class:`dict`, :class:`list`, and :class:`set` use internal locking to make many operations thread-safe, although thread safety is not necessarily guaranteed. Code that is not thread-safe may experience :term:`race conditions ` and :term:`data races ` when used in multi-threaded programs. token A small unit of source code, generated by the :ref:`lexical analyzer ` (also called the *tokenizer*). Names, numbers, strings, operators, newlines and similar are represented by tokens. The :mod:`tokenize` module exposes Python's lexical analyzer. The :mod:`token` module contains information on the various types of tokens. triple-quoted string A string which is bound by three instances of either a quotation mark (") or an apostrophe ('). While they don't provide any functionality not available with single-quoted strings, they are useful for a number of reasons. They allow you to include unescaped single and double quotes within a string and they can span multiple lines without the use of the continuation character, making them especially useful when writing docstrings. type The type of a Python object determines what kind of object it is; every object has a type. An object's type is accessible as its :attr:`~object.__class__` attribute or can be retrieved with ``type(obj)``. type alias A synonym for a type, created by assigning the type to an identifier. Type aliases are useful for simplifying :term:`type hints `. For example:: def remove_gray_shades( colors: list[tuple[int, int, int]]) -> list[tuple[int, int, int]]: pass could be made more readable like this:: Color = tuple[int, int, int] def remove_gray_shades(colors: list[Color]) -> list[Color]: pass See :mod:`typing` and :pep:`484`, which describe this functionality. type hint An :term:`annotation` that specifies the expected type for a variable, a class attribute, or a function parameter or return value. Type hints are optional and are not enforced by Python but they are useful to :term:`static type checkers `. They can also aid IDEs with code completion and refactoring. Type hints of global variables, class attributes, and functions, but not local variables, can be accessed using :func:`typing.get_type_hints`. See :mod:`typing` and :pep:`484`, which describe this functionality. universal newlines A manner of interpreting text streams in which all of the following are recognized as ending a line: the Unix end-of-line convention ``'\n'``, the Windows convention ``'\r\n'``, and the old Macintosh convention ``'\r'``. See :pep:`278` and :pep:`3116`, as well as :func:`bytes.splitlines` for an additional use. variable annotation An :term:`annotation` of a variable or a class attribute. When annotating a variable or a class attribute, assignment is optional:: class C: field: 'annotation' Variable annotations are usually used for :term:`type hints `: for example this variable is expected to take :class:`int` values:: count: int = 0 Variable annotation syntax is explained in section :ref:`annassign`. See :term:`function annotation`, :pep:`484` and :pep:`526`, which describe this functionality. Also see :ref:`annotations-howto` for best practices on working with annotations. virtual environment A cooperatively isolated runtime environment that allows Python users and applications to install and upgrade Python distribution packages without interfering with the behaviour of other Python applications running on the same system. See also :mod:`venv`. virtual machine A computer defined entirely in software. Python's virtual machine executes the :term:`bytecode` emitted by the bytecode compiler. walrus operator A light-hearted way to refer to the :ref:`assignment expression ` operator ``:=`` because it looks a bit like a walrus if you turn your head. Zen of Python Listing of Python design principles and philosophies that are helpful in understanding and using the language. The listing can be found by typing "``import this``" at the interactive prompt. --- ### Doc Improve Page Nojs (`Doc/improve-page-nojs.rst`) :orphan: **************************** Improve a documentation page **************************** .. This is the no-javascript version of this page. The one most people will see (with JavaScript enabled) is improve-page.rst. If you edit this page, please also edit that one, and vice versa. .. only:: html and not epub We are always interested to hear ideas about improvements to the documentation. .. only:: translation If the bug or suggested improvement concerns the translation of this documentation, open an issue or edit the page in `translation's repository `_ instead. You have a few ways to ask questions or suggest changes: - You can start a discussion about the page on the Python discussion forum. This link will start a topic in the Documentation category: `New Documentation topic `_. - You can open an issue on the Python GitHub issue tracker. This link will create a new issue with the "docs" label: `New docs issue `_. --- ### Doc Improve Page (`Doc/improve-page.rst`) :orphan: **************************** Improve a documentation page **************************** .. This is the JavaScript-enabled version of this page. Another version (for those with JavaScript disabled) is improve-page-nojs.rst. If you edit this page, please also edit that one, and vice versa. .. only:: html and not epub .. raw:: html We are always interested to hear ideas about improvements to the documentation. You were reading "PAGETITLE" at ``_. The source for that page is on `GitHub `_. .. only:: translation If the bug or suggested improvement concerns the translation of this documentation, open an issue or edit the page in `translation's repository `_ instead. You have a few ways to ask questions or suggest changes: - You can start a discussion about the page on the Python discussion forum. This link will start a pre-populated topic: `Question about page "PAGETITLE" `_. - You can open an issue on the Python GitHub issue tracker. This link will create a new pre-populated issue: `Docs: problem with page "PAGETITLE" `_. - You can `edit the page on GitHub `_ to open a pull request and begin the contribution process. --- ### Doc Requirements (`Doc/requirements.txt`) # Requirements to build the Python documentation # # Note that when updating this file, you will likely also have to update # the Doc/constraints.txt file. # The Sphinx version is pinned so that new versions that introduce new warnings # won't suddenly cause build failures. Updating the version is fine as long # as no warnings are raised by doing so. # Keep this version in sync with ``Doc/conf.py``. sphinx<9.0.0 pygments>=2.21 blurb sphinx-linklint sphinx-notfound-page~=1.0.0 sphinxext-opengraph~=0.13.0 sphinxext-rediraffe # The theme used by the documentation is stored separately, so we need # to install that as well. python-docs-theme>=2023.3.1,!=2023.7 -c constraints.txt --- ### Doc Builtins Constants (`Doc/builtins/constants.rst`) .. _built-in-consts: Built-in Constants ================== A small number of constants live in the built-in namespace. They are: .. data:: False The false value of the :class:`bool` type. Assignments to ``False`` are illegal and raise a :exc:`SyntaxError`. .. data:: True The true value of the :class:`bool` type. Assignments to ``True`` are illegal and raise a :exc:`SyntaxError`. .. data:: None An object frequently used to represent the absence of a value, as when default arguments are not passed to a function. Assignments to ``None`` are illegal and raise a :exc:`SyntaxError`. ``None`` is the sole instance of the :class:`~types.NoneType` type. .. data:: NotImplemented A special value which should be returned by the binary special methods (e.g. :meth:`~object.__eq__`, :meth:`~object.__lt__`, :meth:`~object.__add__`, :meth:`~object.__rsub__`, etc.) to indicate that the operation is not implemented with respect to the other type; may be returned by the in-place binary special methods (e.g. :meth:`~object.__imul__`, :meth:`~object.__iand__`, etc.) for the same purpose. It should not be evaluated in a boolean context. :data:`!NotImplemented` is the sole instance of the :class:`types.NotImplementedType` type. .. note:: When a binary (or in-place) method returns :data:`!NotImplemented` the interpreter will try the reflected operation on the other type (or some other fallback, depending on the operator). If all attempts return :data:`!NotImplemented`, the interpreter will raise an appropriate exception. Incorrectly returning :data:`!NotImplemented` will result in a misleading error message or the :data:`!NotImplemented` value being returned to Python code. See :ref:`implementing-the-arithmetic-operations` for examples. .. caution:: :data:`!NotImplemented` and :exc:`!NotImplementedError` are not interchangeable. This constant should only be used as described above; see :exc:`NotImplementedError` for details on correct usage of the exception. .. versionchanged:: 3.9 Evaluating :data:`!NotImplemented` in a boolean context was deprecated. .. versionchanged:: 3.14 Evaluating :data:`!NotImplemented` in a boolean context now raises a :exc:`TypeError`. It previously evaluated to :const:`True` and emitted a :exc:`DeprecationWarning` since Python 3.9. .. index:: single: ...; ellipsis literal .. data:: Ellipsis The same as the ellipsis literal "``...``", an object frequently used to indicate that something is omitted. Assignment to ``Ellipsis`` is possible, but assignment to ``...`` raises a :exc:`SyntaxError`. ``Ellipsis`` is the sole instance of the :class:`types.EllipsisType` type. .. data:: __debug__ This constant is true if Python was not started with an :option:`-O` option. See also the :keyword:`assert` statement. .. note:: The names :data:`None`, :data:`False`, :data:`True` and :data:`__debug__` cannot be reassigned (assignments to them, even as an attribute name, raise :exc:`SyntaxError`), so they can be considered "true" constants. .. _site-consts: Constants added by the :mod:`site` module ----------------------------------------- The :mod:`site` module (which is imported automatically during startup, except if the :option:`-S` command-line option is given) adds several constants to the built-in namespace. They are useful for the interactive interpreter shell and should not be used in programs. .. data:: quit(code=None) exit(code=None) Objects that when printed, print a message like "Use quit() or Ctrl-D (i.e. EOF) to exit", and when accessed directly in the interactive interpreter or called as functions, raise :exc:`SystemExit` with the specified exit code. .. data:: help :noindex: Object that when printed, prints the message "Type help() for interactive help, or help(object) for help about object.", and when accessed directly in the interactive interpreter, invokes the built-in help system (see :func:`help`). .. data:: copyright credits Objects that when printed or called, print the text of copyright or credits, respectively. .. data:: license Object that when printed, prints the message "Type license() to see the full license text", and when called, displays the full license text in a pager-like fashion (one screen at a time). --- ### Doc Builtins Exceptions (`Doc/builtins/exceptions.rst`) .. _bltin-exceptions: Built-in Exceptions =================== .. index:: pair: statement; try pair: statement; except In Python, all exceptions must be instances of a class that derives from :class:`BaseException`. In a :keyword:`try` statement with an :keyword:`except` clause that mentions a particular class, that clause also handles any exception classes derived from that class (but not exception classes from which *it* is derived). Two exception classes that are not related via subclassing are never equivalent, even if they have the same name. .. index:: pair: statement; raise The built-in exceptions listed in this chapter can be generated by the interpreter or built-in functions. Except where mentioned, they have an "associated value" indicating the detailed cause of the error. This may be a string or a tuple of several items of information (e.g., an error code and a string explaining the code). The associated value is usually passed as arguments to the exception class's constructor. User code can raise built-in exceptions. This can be used to test an exception handler or to report an error condition "just like" the situation in which the interpreter raises the same exception; but beware that there is nothing to prevent user code from raising an inappropriate error. The built-in exception classes can be subclassed to define new exceptions; programmers are encouraged to derive new exceptions from the :exc:`Exception` class or one of its subclasses, and not from :exc:`BaseException`. More information on defining exceptions is available in the Python Tutorial under :ref:`tut-userexceptions`. Exception context ----------------- .. index:: pair: exception; chaining __cause__ (exception attribute) __context__ (exception attribute) __suppress_context__ (exception attribute) Three attributes on exception objects provide information about the context in which the exception was raised: .. attribute:: BaseException.__context__ BaseException.__cause__ BaseException.__suppress_context__ When raising a new exception while another exception is already being handled, the new exception's :attr:`!__context__` attribute is automatically set to the handled exception. An exception may be handled when an :keyword:`except` or :keyword:`finally` clause, or a :keyword:`with` statement, is used. This implicit exception context can be supplemented with an explicit cause by using :keyword:`!from` with :keyword:`raise`:: raise new_exc from original_exc The expression following :keyword:`from` must be an exception or ``None``. It will be set as :attr:`!__cause__` on the raised exception. Setting :attr:`!__cause__` also implicitly sets the :attr:`!__suppress_context__` attribute to ``True``, so that using ``raise new_exc from None`` effectively replaces the old exception with the new one for display purposes (e.g. converting :exc:`KeyError` to :exc:`AttributeError`), while leaving the old exception available in :attr:`!__context__` for introspection when debugging. The default traceback display code shows these chained exceptions in addition to the traceback for the exception itself. An explicitly chained exception in :attr:`!__cause__` is always shown when present. An implicitly chained exception in :attr:`!__context__` is shown only if :attr:`!__cause__` is :const:`None` and :attr:`!__suppress_context__` is false. In either case, the exception itself is always shown after any chained exceptions so that the final line of the traceback always shows the last exception that was raised. Inheriting from built-in exceptions ----------------------------------- User code can create subclasses that inherit from an exception type. It's recommended to only subclass one exception type at a time to avoid any possible conflicts between how the bases handle the ``args`` attribute, as well as due to possible memory layout incompatibilities. .. impl-detail:: Most built-in exceptions are implemented in C for efficiency, see: :source:`Objects/exceptions.c`. Some have custom memory layouts which makes it impossible to create a subclass that inherits from multiple exception types. The memory layout of a type is an implementation detail and might change between Python versions, leading to new conflicts in the future. Therefore, it's recommended to avoid subclassing multiple exception types altogether. Base classes ------------ The following exceptions are used mostly as base classes for other exceptions. .. exception:: BaseException The base class for all built-in exceptions. It is not meant to be directly inherited by user-defined classes (for that, use :exc:`Exception`). If :func:`str` is called on an instance of this class, the representation of the argument(s) to the instance are returned, or the empty string when there were no arguments. .. attribute:: args The tuple of arguments given to the exception constructor. Some built-in exceptions (like :exc:`OSError`) expect a certain number of arguments and assign a special meaning to the elements of this tuple, while others are usually called only with a single string giving an error message. .. method:: with_traceback(tb) This method sets *tb* as the new traceback for the exception and returns the exception object. It was more commonly used before the exception chaining features of :pep:`3134` became available. The following example shows how we can convert an instance of ``SomeException`` into an instance of ``OtherException`` while preserving the traceback. Once raised, the current frame is pushed onto the traceback of the ``OtherException``, as would have happened to the traceback of the original ``SomeException`` had we allowed it to propagate to the caller. :: try: ... except SomeException: tb = sys.exception().__traceback__ raise OtherException(...).with_traceback(tb) .. attribute:: __traceback__ A writable field that holds the :ref:`traceback object ` associated with this exception. See also: :ref:`raise`. .. method:: add_note(note) Add the string ``note`` to the exception's notes which appear in the standard traceback after the exception string. A :exc:`TypeError` is raised if ``note`` is not a string. .. versionadded:: 3.11 .. attribute:: __notes__ A list of the notes of this exception, which were added with :meth:`add_note`. This attribute is created when :meth:`add_note` is called. .. versionadded:: 3.11 .. exception:: Exception All built-in, non-system-exiting exceptions are derived from this class. All user-defined exceptions should also be derived from this class. .. exception:: ArithmeticError The base class for those built-in exceptions that are raised for various arithmetic errors: :exc:`OverflowError`, :exc:`ZeroDivisionError`, :exc:`FloatingPointError`. .. exception:: BufferError Raised when a :ref:`buffer ` related operation cannot be performed. .. exception:: LookupError The base class for the exceptions that are raised when a key or index used on a mapping or sequence is invalid: :exc:`IndexError`, :exc:`KeyError`. This can be raised directly by :func:`codecs.lookup`. Concrete exceptions ------------------- The following exceptions are the exceptions that are usually raised. .. exception:: AssertionError .. index:: pair: statement; assert Raised when an :keyword:`assert` statement fails. .. exception:: AttributeError Raised when an attribute reference (see :ref:`attribute-references`) or assignment fails. (When an object does not support attribute references or attribute assignments at all, :exc:`TypeError` is raised.) The optional *name* and *obj* keyword-only arguments set the corresponding attributes: .. attribute:: name The name of the attribute that was attempted to be accessed. .. attribute:: obj The object that was accessed for the named attribute. When possible, :attr:`name` and :attr:`obj` are set automatically. .. versionchanged:: 3.10 Added the :attr:`name` and :attr:`obj` attributes. .. exception:: EOFError Raised when the :func:`input` function hits an end-of-file condition (EOF) without reading any data. (Note: the :meth:`io.TextIOBase.read` and :meth:`io.IOBase.readline` methods return an empty string when they hit EOF.) .. exception:: FloatingPointError Not currently used. .. exception:: GeneratorExit Raised when a :term:`generator` or :term:`coroutine` is closed; see :meth:`generator.close` and :meth:`coroutine.close`. It directly inherits from :exc:`BaseException` instead of :exc:`Exception` since it is technically not an error. .. exception:: ImportError Raised when the :keyword:`import` statement has troubles trying to load a module. Also raised when the "from list" in ``from ... import`` has a name that cannot be found. The optional *name* and *path* keyword-only arguments set the corresponding attributes: .. attribute:: name The name of the module that was attempted to be imported. .. attribute:: path The path to any file which triggered the exception. .. versionchanged:: 3.3 Added the :attr:`name` and :attr:`path` attributes. .. exception:: ModuleNotFoundError A subclass of :exc:`ImportError` which is raised by :keyword:`import` when a module could not be located. It is also raised when ``None`` is found in :data:`sys.modules`. .. versionadded:: 3.6 .. exception:: ImportCycleError A subclass of :exc:`ImportError` which is raised when a lazy import fails because it (directly or indirectly) tries to import itself. .. versionadded:: 3.15 .. exception:: IndexError Raised when a sequence subscript is out of range. (Slice indices are silently truncated to fall in the allowed range; if an index is not an integer, :exc:`TypeError` is raised.) .. XXX xref to sequences .. exception:: KeyError Raised when a mapping (dictionary) key is not found in the set of existing keys. .. XXX xref to mapping objects? .. exception:: KeyboardInterrupt Raised when the user hits the interrupt key (normally :kbd:`Control-C` or :kbd:`Delete`). During execution, a check for interrupts is made regularly. The exception inherits from :exc:`BaseException` so as to not be accidentally caught by code that catches :exc:`Exception` and thus prevent the interpreter from exiting. .. note:: Catching a :exc:`KeyboardInterrupt` requires special consideration. Because it can be raised at unpredictable points, it may, in some circumstances, leave the running program in an inconsistent state. It is generally best to allow :exc:`KeyboardInterrupt` to end the program as quickly as possible or avoid raising it entirely. (See :ref:`handlers-and-exceptions`.) .. exception:: MemoryError Raised when an operation runs out of memory but the situation may still be rescued (by deleting some objects). The associated value is a string indicating what kind of (internal) operation ran out of memory. Note that because of the underlying memory management architecture (C's :c:func:`malloc` function), the interpreter may not always be able to completely recover from this situation; it nevertheless raises an exception so that a stack traceback can be printed, in case a run-away program was the cause. .. exception:: NameError Raised when a local or global name is not found. This applies only to unqualified names. The associated value is an error message that includes the name that could not be found. The optional *name* keyword-only argument sets the attribute: .. attribute:: name The name of the variable that was attempted to be accessed. .. versionchanged:: 3.10 Added the :attr:`name` attribute. .. exception:: NotImplementedError This exception is derived from :exc:`RuntimeError`. In user defined base classes, abstract methods should raise this exception when they require derived classes to override the method, or while the class is being developed to indicate that the real implementation still needs to be added. .. note:: It should not be used to indicate that an operator or method is not meant to be supported at all -- in that case either leave the operator / method undefined or, if a subclass, set it to :data:`None`. .. caution:: :exc:`!NotImplementedError` and :data:`!NotImplemented` are not interchangeable. This exception should only be used as described above; see :data:`NotImplemented` for details on correct usage of the built-in constant. .. exception:: OSError([arg]) OSError(errno, strerror[, filename[, winerror[, filename2]]]) .. index:: pair: module; errno This exception is raised when a system function returns a system-related error, including I/O failures such as "file not found" or "disk full" (not for illegal argument types or other incidental errors). The second form of the constructor sets the corresponding attributes, described below. The attributes default to :const:`None` if not specified. For backwards compatibility, if three arguments are passed, the :attr:`~BaseException.args` attribute contains only a 2-tuple of the first two constructor arguments. The constructor often actually returns a subclass of :exc:`OSError`, as described in `OS exceptions`_ below. The particular subclass depends on the final :attr:`.errno` value. This behaviour only occurs when constructing :exc:`OSError` directly or via an alias, and is not inherited when subclassing. .. attribute:: errno A numeric error code from the C variable :c:data:`errno`. .. attribute:: winerror Under Windows, this gives you the native Windows error code. The :attr:`.errno` attribute is then an approximate translation, in POSIX terms, of that native error code. Under Windows, if the *winerror* constructor argument is an integer, the :attr:`.errno` attribute is determined from the Windows error code, and the *errno* argument is ignored. On other platforms, the *winerror* argument is ignored, and the :attr:`winerror` attribute does not exist. .. attribute:: strerror The corresponding error message, as provided by the operating system. It is formatted by the C functions :c:func:`!perror` under POSIX, and :c:func:`!FormatMessage` under Windows. .. attribute:: filename filename2 For exceptions that involve a file system path (such as :func:`open` or :func:`os.unlink`), :attr:`filename` is the file name passed to the function. For functions that involve two file system paths (such as :func:`os.rename`), :attr:`filename2` corresponds to the second file name passed to the function. .. versionchanged:: 3.3 :exc:`EnvironmentError`, :exc:`IOError`, :exc:`WindowsError`, :exc:`socket.error`, :exc:`select.error` and :exc:`!mmap.error` have been merged into :exc:`OSError`, and the constructor may return a subclass. .. versionchanged:: 3.4 The :attr:`filename` attribute is now the original file name passed to the function, instead of the name encoded to or decoded from the :term:`filesystem encoding and error handler`. Also, the *filename2* constructor argument and attribute was added. .. exception:: OverflowError Raised when the result of an arithmetic operation is too large to be represented. This cannot occur for integers (which would rather raise :exc:`MemoryError` than give up). However, for historical reasons, OverflowError is sometimes raised for integers that are outside a required range. Because of the lack of standardization of floating-point exception handling in C, most floating-point operations are not checked. .. exception:: PythonFinalizationError This exception is derived from :exc:`RuntimeError`. It is raised when an operation is blocked during interpreter shutdown also known as :term:`Python finalization `. Examples of operations which can be blocked with a :exc:`PythonFinalizationError` during the Python finalization: * Creating a new Python thread. * :meth:`Joining ` a running daemon thread. * :func:`os.fork`, * acquiring a lock such as :class:`threading.Lock`, when it is known that the operation would otherwise deadlock. See also the :func:`sys.is_finalizing` function. .. versionadded:: 3.13 Previously, a plain :exc:`RuntimeError` was raised. .. versionchanged:: 3.14 :meth:`threading.Thread.join` can now raise this exception. .. versionchanged:: 3.15 This exception may be raised when acquiring :meth:`threading.Lock` or :meth:`threading.RLock`. .. exception:: RecursionError This exception is derived from :exc:`RuntimeError`. It is raised when the interpreter detects that the maximum recursion depth (see :func:`sys.getrecursionlimit`) is exceeded. .. versionadded:: 3.5 Previously, a plain :exc:`RuntimeError` was raised. .. exception:: ReferenceError This exception is raised when a weak reference proxy, created by the :func:`weakref.proxy` function, is used to access an attribute of the referent after it has been garbage collected. For more information on weak references, see the :mod:`weakref` module. .. exception:: RuntimeError Raised when an error is detected that doesn't fall in any of the other categories. The associated value is a string indicating what precisely went wrong. .. exception:: StopIteration Raised by built-in function :func:`next` and an :term:`iterator`\'s :meth:`~iterator.__next__` method to signal that there are no further items produced by the iterator. .. attribute:: StopIteration.value The exception object has a single attribute :attr:`!value`, which is given as an argument when constructing the exception, and defaults to :const:`None`. When a :term:`generator` or :term:`coroutine` function returns, a new :exc:`StopIteration` instance is raised, and the value returned by the function is used as the :attr:`value` parameter to the constructor of the exception. If a generator code directly or indirectly raises :exc:`StopIteration`, it is converted into a :exc:`RuntimeError` (retaining the :exc:`StopIteration` as the new exception's cause). .. versionchanged:: 3.3 Added ``value`` attribute and the ability for generator functions to use it to return a value. .. versionchanged:: 3.5 Introduced the RuntimeError transformation via ``from __future__ import generator_stop``, see :pep:`479`. .. versionchanged:: 3.7 Enable :pep:`479` for all code by default: a :exc:`StopIteration` error raised in a generator is transformed into a :exc:`RuntimeError`. .. exception:: StopAsyncIteration Must be raised by :meth:`~object.__anext__` method of an :term:`asynchronous iterator` object to stop the iteration. .. versionadded:: 3.5 .. exception:: SyntaxError(message, details) Raised when the parser encounters a syntax error. This may occur in an :keyword:`import` statement, in a call to the built-in functions :func:`compile`, :func:`exec`, or :func:`eval`, or when reading the initial script or standard input (also interactively). The :func:`str` of the exception instance returns only the error message. Details is a tuple whose members are also available as separate attributes. .. attribute:: filename The name of the file the syntax error occurred in. .. attribute:: lineno Which line number in the file the error occurred in. This is 1-indexed: the first line in the file has a ``lineno`` of 1. .. attribute:: offset The column in the line where the error occurred. This is 1-indexed: the first character in the line has an ``offset`` of 1. .. attribute:: text The source code text involved in the error. .. attribute:: end_lineno Which line number in the file the error occurred ends in. This is 1-indexed: the first line in the file has a ``lineno`` of 1. .. attribute:: end_offset The column in the end line where the error occurred finishes. This is 1-indexed: the first character in the line has an ``offset`` of 1. For errors in f-string fields, the message is prefixed by "f-string: " and the offsets are offsets in a text constructed from the replacement expression. For example, compiling f'Bad {a b} field' results in this args attribute: ('f-string: ...', ('', 1, 2, '(a b)\n', 1, 5)). .. versionchanged:: 3.10 Added the :attr:`end_lineno` and :attr:`end_offset` attributes. .. exception:: IndentationError Base class for syntax errors related to incorrect indentation. This is a subclass of :exc:`SyntaxError`. .. exception:: TabError Raised when indentation contains an inconsistent use of tabs and spaces. This is a subclass of :exc:`IndentationError`. .. exception:: SystemError Raised when the interpreter finds an internal error, but the situation does not look so serious to cause it to abandon all hope. The associated value is a string indicating what went wrong (in low-level terms). In :term:`CPython`, this could be raised by incorrectly using Python's C API, such as returning a ``NULL`` value without an exception set. If you're confident that this exception wasn't your fault, or the fault of a package you're using, you should report this to the author or maintainer of your Python interpreter. Be sure to report the version of the Python interpreter (``sys.version``; it is also printed at the start of an interactive Python session), the exact error message (the exception's associated value) and if possible the source of the program that triggered the error. .. exception:: SystemExit This exception is raised by the :func:`sys.exit` function. It inherits from :exc:`BaseException` instead of :exc:`Exception` so that it is not accidentally caught by code that catches :exc:`Exception`. This allows the exception to properly propagate up and cause the interpreter to exit. When it is not handled, the Python interpreter exits; no stack traceback is printed. The constructor accepts the same optional argument passed to :func:`sys.exit`. If the value is an integer, it specifies the system exit status (passed to C's :c:func:`!exit` function); if it is ``None``, the exit status is zero; if it has another type (such as a string), the object's value is printed and the exit status is one. A call to :func:`sys.exit` is translated into an exception so that clean-up handlers (:keyword:`finally` clauses of :keyword:`try` statements) can be executed, and so that a debugger can execute a script without running the risk of losing control. The :func:`os._exit` function can be used if it is absolutely positively necessary to exit immediately (for example, in the child process after a call to :func:`os.fork`). .. attribute:: code The exit status or error message that is passed to the constructor. (Defaults to ``None``.) .. exception:: TypeError Raised when an operation or function is applied to an object of inappropriate type. The associated value is a string giving details about the type mismatch. This exception may be raised by user code to indicate that an attempted operation on an object is not supported, and is not meant to be. If an object is meant to support a given operation but has not yet provided an implementation, :exc:`NotImplementedError` is the proper exception to raise. Passing arguments of the wrong type (e.g. passing a :class:`list` when an :class:`int` is expected) should result in a :exc:`TypeError`, but passing arguments with the wrong value (e.g. a number outside expected boundaries) should result in a :exc:`ValueError`. .. exception:: UnboundLocalError Raised when a reference is made to a local variable in a function or method, but no value has been bound to that variable. This is a subclass of :exc:`NameError`. .. exception:: UnicodeError Raised when a Unicode-related encoding or decoding error occurs. It is a subclass of :exc:`ValueError`. :exc:`UnicodeError` has attributes that describe the encoding or decoding error. For example, ``err.object[err.start:err.end]`` gives the particular invalid input that the codec failed on. .. attribute:: encoding The name of the encoding that raised the error. .. attribute:: reason A string describing the specific codec error. .. attribute:: object The object the codec was attempting to encode or decode. .. attribute:: start The first index of invalid data in :attr:`object`. This value should not be negative as it is interpreted as an absolute offset but this constraint is not enforced at runtime. .. attribute:: end The index after the last invalid data in :attr:`object`. This value should not be negative as it is interpreted as an absolute offset but this constraint is not enforced at runtime. .. exception:: UnicodeEncodeError Raised when a Unicode-related error occurs during encoding. It is a subclass of :exc:`UnicodeError`. .. exception:: UnicodeDecodeError Raised when a Unicode-related error occurs during decoding. It is a subclass of :exc:`UnicodeError`. .. exception:: UnicodeTranslateError Raised when a Unicode-related error occurs during translating. It is a subclass of :exc:`UnicodeError`. .. exception:: ValueError Raised when an operation or function receives an argument that has the right type but an inappropriate value, and the situation is not described by a more precise exception such as :exc:`IndexError`. .. exception:: ZeroDivisionError Raised when the second argument of a division or modulo operation is zero. The associated value is a string indicating the type of the operands and the operation. The following exceptions are kept for compatibility with previous versions; starting from Python 3.3, they are aliases of :exc:`OSError`. .. exception:: EnvironmentError .. exception:: IOError .. exception:: WindowsError Only available on Windows. OS exceptions ^^^^^^^^^^^^^ The following exceptions are subclasses of :exc:`OSError`, they get raised depending on the system error code. .. exception:: BlockingIOError Raised when an operation would block on an object (e.g. socket) set for non-blocking operation. Corresponds to :c:data:`errno` :py:const:`~errno.EAGAIN`, :py:const:`~errno.EALREADY`, :py:const:`~errno.EWOULDBLOCK` and :py:const:`~errno.EINPROGRESS`. In addition to those of :exc:`OSError`, :exc:`BlockingIOError` can have one more attribute: .. attribute:: characters_written An integer containing the number of **bytes** written to the stream before it blocked. This attribute is available when using the buffered I/O classes from the :mod:`io` module. .. exception:: ChildProcessError Raised when an operation on a child process failed. Corresponds to :c:data:`errno` :py:const:`~errno.ECHILD`. .. exception:: ConnectionError A base class for connection-related issues. Subclasses are :exc:`BrokenPipeError`, :exc:`ConnectionAbortedError`, :exc:`ConnectionRefusedError` and :exc:`ConnectionResetError`. .. exception:: BrokenPipeError A subclass of :exc:`ConnectionError`, raised when trying to write on a pipe while the other end has been closed, or trying to write on a socket which has been shutdown for writing. Corresponds to :c:data:`errno` :py:const:`~errno.EPIPE` and :py:const:`~errno.ESHUTDOWN`. .. exception:: ConnectionAbortedError A subclass of :exc:`ConnectionError`, raised when a connection attempt is aborted by the peer. Corresponds to :c:data:`errno` :py:const:`~errno.ECONNABORTED`. .. exception:: ConnectionRefusedError A subclass of :exc:`ConnectionError`, raised when a connection attempt is refused by the peer. Corresponds to :c:data:`errno` :py:const:`~errno.ECONNREFUSED`. .. exception:: ConnectionResetError A subclass of :exc:`ConnectionError`, raised when a connection is reset by the peer. Corresponds to :c:data:`errno` :py:const:`~errno.ECONNRESET`. .. exception:: FileExistsError Raised when trying to create a file or directory which already exists. Corresponds to :c:data:`errno` :py:const:`~errno.EEXIST`. .. exception:: FileNotFoundError Raised when a file or directory is requested but doesn't exist. Corresponds to :c:data:`errno` :py:const:`~errno.ENOENT`. .. exception:: InterruptedError Raised when a system call is interrupted by an incoming signal. Corresponds to :c:data:`errno` :py:const:`~errno.EINTR`. .. versionchanged:: 3.5 Python now retries system calls when a syscall is interrupted by a signal, except if the signal handler raises an exception (see :pep:`475` for the rationale), instead of raising :exc:`InterruptedError`. .. exception:: IsADirectoryError Raised when a file operation (such as :func:`os.remove`) is requested on a directory. Corresponds to :c:data:`errno` :py:const:`~errno.EISDIR`. .. exception:: NotADirectoryError Raised when a directory operation (such as :func:`os.listdir`) is requested on something which is not a directory. On most POSIX platforms, it may also be raised if an operation attempts to open or traverse a non-directory file as if it were a directory. Corresponds to :c:data:`errno` :py:const:`~errno.ENOTDIR`. .. exception:: PermissionError Raised when trying to run an operation without the adequate access rights - for example filesystem permissions. Corresponds to :c:data:`errno` :py:const:`~errno.EACCES`, :py:const:`~errno.EPERM`, and :py:const:`~errno.ENOTCAPABLE`. .. versionchanged:: 3.11.1 WASI's :py:const:`~errno.ENOTCAPABLE` is now mapped to :exc:`PermissionError`. .. exception:: ProcessLookupError Raised when a given process doesn't exist. Corresponds to :c:data:`errno` :py:const:`~errno.ESRCH`. .. exception:: TimeoutError Raised when a system function timed out at the system level. Corresponds to :c:data:`errno` :py:const:`~errno.ETIMEDOUT`. .. versionadded:: 3.3 All the above :exc:`OSError` subclasses were added. .. seealso:: :pep:`3151` - Reworking the OS and IO exception hierarchy .. _warning-categories-as-exceptions: Warnings -------- The following exceptions are used as warning categories; see the :ref:`warning-categories` documentation for more details. .. exception:: Warning Base class for warning categories. .. exception:: UserWarning Base class for warnings generated by user code. .. exception:: DeprecationWarning Base class for warnings about deprecated features when those warnings are intended for other Python developers. Ignored by the default warning filters, except in the ``__main__`` module (:pep:`565`). Enabling the :ref:`Python Development Mode ` shows this warning. The deprecation policy is described in :pep:`387`. .. exception:: PendingDeprecationWarning Base class for warnings about features which are obsolete and expected to be deprecated in the future, but are not deprecated at the moment. This class is rarely used as emitting a warning about a possible upcoming deprecation is unusual, and :exc:`DeprecationWarning` is preferred for already active deprecations. Ignored by the default warning filters. Enabling the :ref:`Python Development Mode ` shows this warning. The deprecation policy is described in :pep:`387`. .. exception:: SyntaxWarning Base class for warnings about dubious syntax. This warning is typically emitted when compiling Python source code, and usually won't be reported when running already compiled code. .. exception:: RuntimeWarning Base class for warnings about dubious runtime behavior. .. exception:: FutureWarning Base class for warnings about deprecated features when those warnings are intended for end users of applications that are written in Python. .. exception:: ImportWarning Base class for warnings about probable mistakes in module imports. Ignored by the default warning filters. Enabling the :ref:`Python Development Mode ` shows this warning. .. exception:: UnicodeWarning Base class for warnings related to Unicode. .. exception:: EncodingWarning Base class for warnings related to encodings. See :ref:`io-encoding-warning` for details. .. versionadded:: 3.10 .. exception:: BytesWarning Base class for warnings related to :class:`bytes` and :class:`bytearray`. .. exception:: ResourceWarning Base class for warnings related to resource usage. Ignored by the default warning filters. Enabling the :ref:`Python Development Mode ` shows this warning. .. versionadded:: 3.2 .. _lib-exception-groups: Exception groups ---------------- The following are used when it is necessary to raise multiple unrelated exceptions. They are part of the exception hierarchy so they can be handled with :keyword:`except` like all other exceptions. In addition, they are recognised by :keyword:`except*`, which matches their subgroups based on the types of the contained exceptions. .. exception:: ExceptionGroup(msg, excs) .. exception:: BaseExceptionGroup(msg, excs) Both of these exception types wrap the exceptions in the sequence ``excs``. The ``msg`` parameter must be a string. The difference between the two classes is that :exc:`BaseExceptionGroup` extends :exc:`BaseException` and it can wrap any exception, while :exc:`ExceptionGroup` extends :exc:`Exception` and it can only wrap subclasses of :exc:`Exception`. This design is so that ``except Exception`` catches an :exc:`ExceptionGroup` but not :exc:`BaseExceptionGroup`. The :exc:`BaseExceptionGroup` constructor returns an :exc:`ExceptionGroup` rather than a :exc:`BaseExceptionGroup` if all contained exceptions are :exc:`Exception` instances, so it can be used to make the selection automatic. The :exc:`ExceptionGroup` constructor, on the other hand, raises a :exc:`TypeError` if any contained exception is not an :exc:`Exception` subclass. Exception groups are :ref:`generic ` over the type of their contained exceptions. .. impl-detail:: The ``excs`` parameter may be any sequence, but lists and tuples are specifically processed more efficiently here. For optimal performance, pass a tuple as ``excs``. .. attribute:: message The ``msg`` argument to the constructor. This is a read-only attribute. .. attribute:: exceptions A tuple of the exceptions in the ``excs`` sequence given to the constructor. This is a read-only attribute. .. method:: subgroup(condition) Returns an exception group that contains only the exceptions from the current group that match *condition*, or ``None`` if the result is empty. The condition can be an exception type or tuple of exception types, in which case each exception is checked for a match using the same check that is used in an ``except`` clause. The condition can also be a callable (other than a type object) that accepts an exception as its single argument and returns true for the exceptions that should be in the subgroup. The nesting structure of the current exception is preserved in the result, as are the values of its :attr:`message`, :attr:`~BaseException.__traceback__`, :attr:`~BaseException.__cause__`, :attr:`~BaseException.__context__` and :attr:`~BaseException.__notes__` fields. Empty nested groups are omitted from the result. The condition is checked for all exceptions in the nested exception group, including the top-level and any nested exception groups. If the condition is true for such an exception group, it is included in the result in full. .. versionadded:: 3.13 ``condition`` can be any callable which is not a type object. .. method:: split(condition) Like :meth:`subgroup`, but returns the pair ``(match, rest)`` where ``match`` is ``subgroup(condition)`` and ``rest`` is the remaining non-matching part. .. method:: derive(excs) Returns an exception group with the same :attr:`message`, but which wraps the exceptions in ``excs``. This method is used by :meth:`subgroup` and :meth:`split`, which are used in various contexts to break up an exception group. A subclass needs to override it in order to make :meth:`subgroup` and :meth:`split` return instances of the subclass rather than :exc:`ExceptionGroup`. :meth:`subgroup` and :meth:`split` copy the :attr:`~BaseException.__traceback__`, :attr:`~BaseException.__cause__`, :attr:`~BaseException.__context__` and :attr:`~BaseException.__notes__` fields from the original exception group to the one returned by :meth:`derive`, so these fields do not need to be updated by :meth:`derive`. .. doctest:: >>> class MyGroup(ExceptionGroup): ... def derive(self, excs): ... return MyGroup(self.message, excs) ... >>> e = MyGroup("eg", [ValueError(1), TypeError(2)]) >>> e.add_note("a note") >>> e.__context__ = Exception("context") >>> e.__cause__ = Exception("cause") >>> try: ... raise e ... except Exception as e: ... exc = e ... >>> match, rest = exc.split(ValueError) >>> exc, exc.__context__, exc.__cause__, exc.__notes__ (MyGroup('eg', [ValueError(1), TypeError(2)]), Exception('context'), Exception('cause'), ['a note']) >>> match, match.__context__, match.__cause__, match.__notes__ (MyGroup('eg', [ValueError(1)]), Exception('context'), Exception('cause'), ['a note']) >>> rest, rest.__context__, rest.__cause__, rest.__notes__ (MyGroup('eg', [TypeError(2)]), Exception('context'), Exception('cause'), ['a note']) >>> exc.__traceback__ is match.__traceback__ is rest.__traceback__ True Note that :exc:`BaseExceptionGroup` defines :meth:`~object.__new__`, so subclasses that need a different constructor signature need to override that rather than :meth:`~object.__init__`. For example, the following defines an exception group subclass which accepts an exit_code and constructs the group's message from it. :: class Errors(ExceptionGroup): def __new__(cls, errors, exit_code): self = super().__new__(Errors, f"exit code: {exit_code}", errors) self.exit_code = exit_code return self def derive(self, excs): return Errors(excs, self.exit_code) Like :exc:`ExceptionGroup`, any subclass of :exc:`BaseExceptionGroup` which is also a subclass of :exc:`Exception` can only wrap instances of :exc:`Exception`. .. versionadded:: 3.11 Exception hierarchy ------------------- The class hierarchy for built-in exceptions is: .. literalinclude:: ../../Lib/test/exception_hierarchy.txt :language: text --- ### Doc Builtins Functions (`Doc/builtins/functions.rst`) .. XXX document all delegations to __special__ methods .. _built-in-funcs: Built-in functions ================== The Python interpreter has a number of functions and types built into it that are always available. They are listed here in alphabetical order. +---------------------------------------------------------------------------------------------------+ | Built-in Functions | +=========================+=======================+=======================+=========================+ | | **A** | | **E** | | **L** | | **R** | | | :func:`abs` | | :func:`enumerate` | | :func:`len` | | |func-range|_ | | | :func:`aiter` | | :func:`eval` | | |func-list|_ | | :func:`repr` | | | :func:`all` | | :func:`exec` | | :func:`locals` | | :func:`reversed` | | | :func:`anext` | | | | | | :func:`round` | | | :func:`any` | | **F** | | **M** | | | | | :func:`ascii` | | :func:`filter` | | :func:`map` | | **S** | | | | | :func:`float` | | :func:`max` | | |func-set|_ | | | **B** | | :func:`format` | | |func-memoryview|_ | | :func:`setattr` | | | :func:`bin` | | |func-frozendict|_ | | :func:`min` | | :func:`sentinel` | | | :func:`bool` | | |func-frozenset|_ | | | | :func:`slice` | | | :func:`breakpoint` | | | | **N** | | :func:`sorted` | | | |func-bytearray|_ | | **G** | | :func:`next` | | :func:`staticmethod` | | | |func-bytes|_ | | :func:`getattr` | | | | |func-str|_ | | | | | :func:`globals` | | **O** | | :func:`sum` | | | **C** | | | | :func:`object` | | :func:`super` | | | :func:`callable` | | **H** | | :func:`oct` | | | | | :func:`chr` | | :func:`hasattr` | | :func:`open` | | **T** | | | :func:`classmethod` | | :func:`hash` | | :func:`ord` | | |func-tuple|_ | | | :func:`compile` | | :func:`help` | | | | :func:`type` | | | :func:`complex` | | :func:`hex` | | **P** | | | | | | | | | :func:`pow` | | **V** | | | **D** | | **I** | | :func:`print` | | :func:`vars` | | | :func:`delattr` | | :func:`id` | | :func:`property` | | | | | |func-dict|_ | | :func:`input` | | | | **Z** | | | :func:`dir` | | :func:`int` | | | | :func:`zip` | | | :func:`divmod` | | :func:`isinstance` | | | | | | | | | :func:`issubclass` | | | | **_** | | | | | :func:`iter` | | | | :func:`__import__` | +-------------------------+-----------------------+-----------------------+-------------------------+ .. using :func:`dict` would create a link to another page, so local targets are used, with replacement texts to make the output in the table consistent .. |func-dict| replace:: ``dict()`` .. |func-frozendict| replace:: ``frozendict()`` .. |func-frozenset| replace:: ``frozenset()`` .. |func-memoryview| replace:: ``memoryview()`` .. |func-set| replace:: ``set()`` .. |func-list| replace:: ``list()`` .. |func-str| replace:: ``str()`` .. |func-tuple| replace:: ``tuple()`` .. |func-range| replace:: ``range()`` .. |func-bytearray| replace:: ``bytearray()`` .. |func-bytes| replace:: ``bytes()`` .. function:: abs(number, /) Return the absolute value of a number. The argument may be an integer, a floating-point number, or an object implementing :meth:`~object.__abs__`. If the argument is a complex number, its magnitude is returned. .. function:: aiter(async_iterable, /) aiter(callable, /, stop_value, *, stop_exception=StopAsyncIteration) aiter(callable, /, *, stop_exception) Return an :term:`asynchronous iterator` object. The first argument is interpreted very differently depending on the presence of the other arguments. Without other arguments, the single argument must be an :term:`asynchronous iterable`, and the result is equivalent to calling ``x.__aiter__()``. If *stop_value* or *stop_exception* is given, then the first argument must be a callable object. The asynchronous iterator created in this case calls *callable* with no arguments and awaits the result for each call to its :meth:`~object.__anext__` method; if the awaited value is equal to *stop_value*, or if the call raises an exception matching *stop_exception*, :exc:`StopAsyncIteration` will be raised, otherwise the value will be returned. The callable is only called when the result of :meth:`~object.__anext__` is awaited. *stop_exception* is an exception class or a tuple of exception classes. If *stop_value* is not specified, the iteration stops only when the callable raises an exception. If the callable raises :exc:`StopAsyncIteration` which does not match *stop_exception*, it is replaced with a :exc:`RuntimeError`, as for asynchronous generators (see :pep:`525`). For example, reading fixed-size chunks from an asynchronous stream until the end of file is reached:: from functools import partial async for chunk in aiter(partial(reader.read, 1024), b''): process_chunk(chunk) Or consuming an :class:`asyncio.Queue` until it is shut down:: from asyncio import QueueShutDown async for item in aiter(queue.get, stop_exception=QueueShutDown): process_item(item) .. versionadded:: 3.10 .. versionchanged:: 3.16 Added the *stop_value* and *stop_exception* parameters. .. function:: all(iterable, /) Return ``True`` if all elements of the *iterable* are true (or if the iterable is empty). Equivalent to:: def all(iterable): for element in iterable: if not element: return False return True .. awaitablefunction:: anext(async_iterator, /) anext(async_iterator, default, /) When awaited, return the next item from the given :term:`asynchronous iterator`, or *default* if given and the iterator is :term:`exhausted`. This is the async variant of the :func:`next` builtin, and behaves similarly. This calls the :meth:`~object.__anext__` method of *async_iterator*, returning an :term:`awaitable`. Awaiting this returns the next value of the iterator. If *default* is given, it is returned if the iterator is exhausted, otherwise :exc:`StopAsyncIteration` is raised. .. versionadded:: 3.10 .. function:: any(iterable, /) Return ``True`` if any element of the *iterable* is true. If the iterable is empty, return ``False``. Equivalent to:: def any(iterable): for element in iterable: if element: return True return False .. function:: ascii(object, /) As :func:`repr`, return a string containing a printable representation of an object, but escape the non-ASCII characters in the string returned by :func:`repr` using ``\x``, ``\u``, or ``\U`` escapes. This generates a string similar to that returned by :func:`repr` in Python 2. .. function:: bin(integer, /) Convert an integer number to a binary string prefixed with "0b". The result is a valid Python expression. If *integer* is not a Python :class:`int` object, it has to define an :meth:`~object.__index__` method that returns an integer. Some examples: >>> bin(3) '0b11' >>> bin(-10) '-0b1010' If the prefix "0b" is desired or not, you can use either of the following ways. >>> format(14, '#b'), format(14, 'b') ('0b1110', '1110') >>> f'{14:#b}', f'{14:b}' ('0b1110', '1110') See also :func:`enum.bin` to represent negative values as twos-complement. See also :func:`format` for more information. .. class:: bool(object=False, /) Return a Boolean value, i.e. one of ``True`` or ``False``. The argument is converted using the standard :ref:`truth testing procedure `. If the argument is false or omitted, this returns ``False``; otherwise, it returns ``True``. The :class:`bool` class is a subclass of :class:`int` (see :ref:`typesnumeric`). It cannot be subclassed further. Its only instances are ``False`` and ``True`` (see :ref:`typebool`). .. index:: pair: Boolean; type .. versionchanged:: 3.7 The parameter is now positional-only. .. function:: breakpoint(*args, **kws) This function drops you into the debugger at the call site. Specifically, it calls :func:`sys.breakpointhook`, passing ``args`` and ``kws`` straight through. By default, ``sys.breakpointhook()`` calls :func:`pdb.set_trace` expecting no arguments. In this case, it is purely a convenience function so you don't have to explicitly import :mod:`pdb` or type as much code to enter the debugger. However, :func:`sys.breakpointhook` can be set to some other function and :func:`breakpoint` will automatically call that, allowing you to drop into the debugger of choice. If :func:`sys.breakpointhook` is not accessible, this function will raise :exc:`RuntimeError`. By default, the behavior of :func:`breakpoint` can be changed with the :envvar:`PYTHONBREAKPOINT` environment variable. See :func:`sys.breakpointhook` for usage details. Note that this is not guaranteed if :func:`sys.breakpointhook` has been replaced. .. audit-event:: builtins.breakpoint breakpointhook breakpoint .. versionadded:: 3.7 .. _func-bytearray: .. class:: bytearray(source=b'') bytearray(source, encoding, errors='strict') :noindex: Return a new array of bytes. The :class:`bytearray` class is a mutable sequence of integers in the range 0 <= x < 256. It has most of the usual methods of mutable sequences, described in :ref:`typesseq-mutable`, as well as most methods that the :class:`bytes` type has, see :ref:`bytes-methods`. The optional *source* parameter can be used to initialize the array in a few different ways: * If it is a *string*, you must also give the *encoding* (and optionally, *errors*) parameters; :func:`bytearray` then converts the string to bytes using :meth:`str.encode`. * If it is an *integer*, the array will have that size and will be initialized with null bytes. * If it is an object conforming to the :ref:`buffer interface `, a read-only buffer of the object will be used to initialize the bytes array. * If it is an *iterable*, it must be an iterable of integers in the range ``0 <= x < 256``, which are used as the initial contents of the array. Without an argument, an array of size 0 is created. See also :ref:`binaryseq` and :ref:`typebytearray`. .. _func-bytes: .. class:: bytes(source=b'') bytes(source, encoding, errors='strict') :noindex: Return a new "bytes" object which is an immutable sequence of integers in the range ``0 <= x < 256``. :class:`bytes` is an immutable version of :class:`bytearray` -- it has the same non-mutating methods and the same indexing and slicing behavior. Accordingly, constructor arguments are interpreted as for :func:`bytearray`. Bytes objects can also be created with literals, see :ref:`strings`. See also :ref:`binaryseq`, :ref:`typebytes`, and :ref:`bytes-methods`. .. function:: callable(object, /) Return :const:`True` if the *object* argument appears callable, :const:`False` if not. If this returns ``True``, it is still possible that a call fails, but if it is ``False``, calling *object* will never succeed. Note that classes are callable (calling a class returns a new instance); instances are callable if their class has a :meth:`~object.__call__` method. .. versionadded:: 3.2 This function was first removed in Python 3.0 and then brought back in Python 3.2. .. function:: chr(codepoint, /) Return the string representing a character with the specified Unicode code point. For example, ``chr(97)`` returns the string ``'a'``, while ``chr(8364)`` returns the string ``'€'``. This is the inverse of :func:`ord`. The valid range for the argument is from 0 through 1,114,111 (0x10FFFF in base 16). :exc:`ValueError` will be raised if it is outside that range. .. decorator:: classmethod Transform a method into a class method. A class method receives the class as an implicit first argument, just like an instance method receives the instance. To declare a class method, use this idiom:: class C: @classmethod def f(cls, arg1, arg2): ... The ``@classmethod`` form is a function :term:`decorator` -- see :ref:`function` for details. A class method can be called either on the class (such as ``C.f()``) or on an instance (such as ``C().f()``). The instance is ignored except for its class. If a class method is called for a derived class, the derived class object is passed as the implied first argument. Class methods are different than C++ or Java static methods. If you want those, see :func:`staticmethod` in this section. For more information on class methods, see :ref:`types`. .. versionchanged:: 3.9 Class methods can now wrap other :term:`descriptors ` such as :func:`property`. .. versionchanged:: 3.10 Class methods now inherit the method attributes (:attr:`~function.__module__`, :attr:`~function.__name__`, :attr:`~function.__qualname__`, :attr:`~function.__doc__` and :attr:`~function.__annotations__`) and have a new ``__wrapped__`` attribute. .. deprecated-removed:: 3.11 3.13 Class methods can no longer wrap other :term:`descriptors ` such as :func:`property`. .. function:: compile(source, filename, mode, flags=0, \ dont_inherit=False, optimize=-1, \ *, module=None) Compile the *source* into a code or AST object. Code objects can be executed by :func:`exec` or :func:`eval`. *source* can either be a normal string, a byte string, or an AST object. Refer to the :mod:`ast` module documentation for information on how to work with AST objects. The *filename* argument should give the file from which the code was read; pass some recognizable value if it wasn't read from a file (``''`` is commonly used). The *mode* argument specifies what kind of code must be compiled; it can be ``'exec'`` if *source* consists of a sequence of statements, ``'eval'`` if it consists of a single expression, or ``'single'`` if it consists of a single interactive statement (in the latter case, expression statements that evaluate to something other than ``None`` will be printed). The optional arguments *flags* and *dont_inherit* control which :ref:`compiler options ` should be activated and which :ref:`future features ` should be allowed. If neither is present (or both are zero) the code is compiled with the same flags that affect the code that is calling :func:`compile`. If the *flags* argument is given and *dont_inherit* is not (or is zero) then the compiler options and the future statements specified by the *flags* argument are used in addition to those that would be used anyway. If *dont_inherit* is a non-zero integer then the *flags* argument is it -- the flags (future features and compiler options) in the surrounding code are ignored. Compiler options and future statements are specified by bits which can be bitwise ORed together to specify multiple options. The bitfield required to specify a given future feature can be found as the :attr:`~__future__._Feature.compiler_flag` attribute on the :class:`~__future__._Feature` instance in the :mod:`__future__` module. :ref:`Compiler flags ` can be found in :mod:`ast` module, with ``PyCF_`` prefix. The argument *optimize* specifies the optimization level of the compiler; the default value of ``-1`` selects the optimization level of the interpreter as given by :option:`-O` options. Explicit levels are ``0`` (no optimization; ``__debug__`` is true), ``1`` (asserts are removed, ``__debug__`` is false) or ``2`` (docstrings are removed too). The optional argument *module* specifies the module name. It is needed to unambiguous :ref:`filter ` syntax warnings by module name. This function raises :exc:`SyntaxError` if the compiled source is invalid, including a *source* containing a null character or that cannot be decoded; :exc:`ValueError` if *mode* or *flags* is invalid, or if a string *source* contains surrogate characters; :exc:`MemoryError` or :exc:`RecursionError` if *source* is too complex to parse or compile, for example an expression with many thousands of nested operators; and :exc:`OverflowError` if *source* is too large. If you want to parse Python code into its AST representation, see :func:`ast.parse`. .. audit-event:: compile source,filename compile Raises an :ref:`auditing event ` ``compile`` with arguments ``source`` and ``filename``. This event may also be raised by implicit compilation. .. note:: When compiling a string with multi-line code in ``'single'`` or ``'eval'`` mode, input must be terminated by at least one newline character. This is to facilitate detection of incomplete and complete statements in the :mod:`code` module. .. warning:: It is possible to crash the Python interpreter with a sufficiently large/complex string when compiling to an AST object due to stack depth limitations in Python's AST compiler. .. versionchanged:: 3.2 Allowed use of Windows and Mac newlines. Also, input in ``'exec'`` mode does not have to end in a newline anymore. Added the *optimize* parameter. .. versionchanged:: 3.5 Previously, :exc:`TypeError` was raised when null bytes were encountered in *source*. .. versionchanged:: 3.8 ``ast.PyCF_ALLOW_TOP_LEVEL_AWAIT`` can now be passed in flags to enable support for top-level ``await``, ``async for``, and ``async with``. .. versionchanged:: 3.12 :exc:`SyntaxError` is raised instead of :exc:`ValueError` when null bytes are encountered in *source*. .. versionchanged:: 3.15 Added the *module* parameter. .. class:: complex(number=0, /) complex(string, /) complex(real=0, imag=0) Convert a single string or number to a complex number, or create a complex number from real and imaginary parts. Examples: .. doctest:: >>> complex('+1.23') (1.23+0j) >>> complex('-4.5j') -4.5j >>> complex('-1.23+4.5j') (-1.23+4.5j) >>> complex('\t( -1.23+4.5J )\n') (-1.23+4.5j) >>> complex('-Infinity+NaNj') (-inf+nanj) >>> complex(1.23) (1.23+0j) >>> complex(imag=-4.5) -4.5j >>> complex(-1.23, 4.5) (-1.23+4.5j) If the argument is a string, it must contain either a real part (in the same format as for :func:`float`) or an imaginary part (in the same format but with a ``'j'`` or ``'J'`` suffix), or both real and imaginary parts (the sign of the imaginary part is mandatory in this case). The string can optionally be surrounded by whitespaces and the round parentheses ``'('`` and ``')'``, which are ignored. The string must not contain whitespace between ``'+'``, ``'-'``, the ``'j'`` or ``'J'`` suffix, and the decimal number. For example, ``complex('1+2j')`` is fine, but ``complex('1 + 2j')`` raises :exc:`ValueError`. More precisely, the input must conform to the :token:`~float:complexvalue` production rule in the following grammar, after parentheses and leading and trailing whitespace characters are removed: .. productionlist:: float complexvalue: `floatvalue` | : `floatvalue` ("j" | "J") | : `floatvalue` `sign` `absfloatvalue` ("j" | "J") If the argument is a number, the constructor serves as a numeric conversion like :class:`int` and :class:`float`. For a general Python object ``x``, ``complex(x)`` delegates to ``x.__complex__()``. If :meth:`~object.__complex__` is not defined then it falls back to :meth:`~object.__float__`. If :meth:`!__float__` is not defined then it falls back to :meth:`~object.__index__`. If two arguments are provided or keyword arguments are used, each argument may be any numeric type (including complex). If both arguments are real numbers, return a complex number with the real component *real* and the imaginary component *imag*. If both arguments are complex numbers, return a complex number with the real component ``real.real-imag.imag`` and the imaginary component ``real.imag+imag.real``. If one of arguments is a real number, only its real component is used in the above expressions. See also :meth:`complex.from_number` which only accepts a single numeric argument. If all arguments are omitted, returns ``0j``. The complex type is described in :ref:`typesnumeric`. .. versionchanged:: 3.6 Grouping digits with underscores as in code literals is allowed. .. versionchanged:: 3.8 Falls back to :meth:`~object.__index__` if :meth:`~object.__complex__` and :meth:`~object.__float__` are not defined. .. deprecated:: 3.14 Passing a complex number as the *real* or *imag* argument is now deprecated; it should only be passed as a single positional argument. .. function:: delattr(object, name, /) This is a relative of :func:`setattr`. The arguments are an object and a string. The string must be the name of one of the object's attributes. The function deletes the named attribute, provided the object allows it. For example, ``delattr(x, 'foobar')`` is equivalent to ``del x.foobar``. *name* need not be a Python identifier (see :func:`setattr`). .. _func-dict: .. class:: dict(**kwargs) dict(mapping, /, **kwargs) dict(iterable, /, **kwargs) :noindex: Create a new dictionary. The :class:`dict` object is the dictionary class. See also :ref:`typesmapping` for documentation about this class. For other containers see the built-in :class:`frozendict`, :class:`list`, :class:`set`, and :class:`tuple` classes, as well as the :mod:`collections` module. .. function:: dir() dir(object, /) Without arguments, return the list of names in the current local scope. With an argument, attempt to return a list of valid attributes for that object. If the object has a method named :meth:`~object.__dir__`, this method will be called and must return the list of attributes. This allows objects that implement a custom :func:`~object.__getattr__` or :func:`~object.__getattribute__` function to customize the way :func:`dir` reports their attributes. If the object does not provide :meth:`~object.__dir__`, the function tries its best to gather information from the object's :attr:`~object.__dict__` attribute, if defined, and from its type object. The resulting list is not necessarily complete and may be inaccurate when the object has a custom :func:`~object.__getattr__`. The default :func:`dir` mechanism behaves differently with different types of objects, as it attempts to produce the most relevant, rather than complete, information: * If the object is a module object, the list contains the names of the module's attributes. * If the object is a type or class object, the list contains the names of its attributes, and recursively of the attributes of its bases. * Otherwise, the list contains the object's attributes' names, the names of its class's attributes, and recursively of the attributes of its class's base classes. The resulting list is sorted alphabetically. For example: >>> import struct >>> dir() # show the names in the module namespace # doctest: +SKIP ['__builtins__', '__name__', 'struct'] >>> dir(struct) # show the names in the struct module # doctest: +SKIP ['Struct', '__all__', '__builtins__', '__doc__', '__file__', '__initializing__', '__loader__', '__name__', '__package__', '_clearcache', 'calcsize', 'error', 'pack', 'pack_into', 'unpack', 'unpack_from'] >>> class Shape: ... def __dir__(self): ... return ['area', 'perimeter', 'location'] ... >>> s = Shape() >>> dir(s) ['area', 'location', 'perimeter'] .. note:: Because :func:`dir` is supplied primarily as a convenience for use at an interactive prompt, it tries to supply an interesting set of names more than it tries to supply a rigorously or consistently defined set of names, and its detailed behavior may change across releases. For example, metaclass attributes are not in the result list when the argument is a class. .. function:: divmod(a, b, /) Take two (non-complex) numbers as arguments and return a pair of numbers consisting of their quotient and remainder when using integer division. With mixed operand types, the rules for binary arithmetic operators apply. For integers, the result is the same as ``(a // b, a % b)``. For floating-point numbers the result is ``(q, a % b)``, where *q* is usually ``math.floor(a / b)`` but may be 1 less than that. In any case ``q * b + a % b`` is very close to *a*, if ``a % b`` is non-zero it has the same sign as *b*, and ``0 <= abs(a % b) < abs(b)``. .. function:: enumerate(iterable, start=0) Return an enumerate object. *iterable* must be a sequence, an :term:`iterator`, or some other object which supports iteration. The :meth:`~iterator.__next__` method of the iterator returned by :func:`enumerate` returns a tuple containing a count (from *start* which defaults to 0) and the values obtained from iterating over *iterable*. >>> seasons = ['Spring', 'Summer', 'Fall', 'Winter'] >>> list(enumerate(seasons)) [(0, 'Spring'), (1, 'Summer'), (2, 'Fall'), (3, 'Winter')] >>> list(enumerate(seasons, start=1)) [(1, 'Spring'), (2, 'Summer'), (3, 'Fall'), (4, 'Winter')] Equivalent to:: def enumerate(iterable, start=0): n = start for elem in iterable: yield n, elem n += 1 .. _func-eval: .. function:: eval(source, /, globals=None, locals=None) :param source: A Python expression. :type source: :class:`str` | :ref:`code object ` :param globals: The global namespace (default: ``None``). :type globals: :class:`dict` | :class:`frozendict` | ``None`` :param locals: The local namespace (default: ``None``). :type locals: :term:`mapping` | ``None`` :returns: The result of the evaluated expression. :raises: Syntax errors are reported as exceptions. .. warning:: This function executes arbitrary code. Calling it with untrusted user-supplied input will lead to security vulnerabilities. The *source* argument is parsed and evaluated as a Python expression (technically speaking, an :ref:`expression list `) using the *globals* and *locals* mappings as global and local namespace. If the *globals* dictionary is present and does not contain a value for the key ``__builtins__``, a reference to the dictionary of the built-in module :mod:`builtins` is inserted under that key before *source* is parsed. Overriding ``__builtins__`` can be used to restrict or change the available names, but this is **not** a security mechanism: the executed code can still access all builtins. If the *locals* mapping is omitted it defaults to the *globals* dictionary. If both mappings are omitted, the source is executed with the *globals* and *locals* in the environment where :func:`eval` is called. Note, *eval()* will only have access to the :term:`nested scopes ` (non-locals) in the enclosing environment if they are already referenced in the scope that is calling :func:`eval` (e.g. via a :keyword:`nonlocal` statement). Example: >>> x = 1 >>> eval('x+1') 2 >>> eval("1, 2") (1, 2) This function can also be used to execute arbitrary code objects (such as those created by :func:`compile`). In this case, pass a code object instead of a string. If the code object has been compiled with ``'exec'`` as the *mode* argument, :func:`eval`\'s return value will be ``None``. Hints: dynamic execution of statements is supported by the :func:`exec` function. The :func:`globals` and :func:`locals` functions return the current global and local dictionary, respectively, which may be useful to pass around for use by :func:`eval` or :func:`exec`. If the given source is a string, then leading and trailing spaces and tabs are stripped. See :func:`ast.literal_eval` for a function to evaluate strings with expressions containing only literals. .. audit-event:: exec code_object eval Raises an :ref:`auditing event ` ``exec`` with the code object as the argument. Code compilation events may also be raised. .. versionchanged:: 3.13 The *globals* and *locals* arguments can now be passed as keywords. .. versionchanged:: 3.13 The semantics of the default *locals* namespace have been adjusted as described for the :func:`locals` builtin. .. versionchanged:: 3.15 *globals* can now be a :class:`frozendict`. .. index:: pair: built-in function; exec .. function:: exec(source, /, globals=None, locals=None, *, closure=None) .. warning:: This function executes arbitrary code. Calling it with untrusted user-supplied input will lead to security vulnerabilities. This function supports dynamic execution of Python code. *source* must be either a string or a code object. If it is a string, the string is parsed as a suite of Python statements which is then executed (unless a syntax error occurs). [#]_ If it is a code object, it is simply executed. In all cases, the code that's executed is expected to be valid as file input (see the section :ref:`file-input` in the Reference Manual). Be aware that the :keyword:`nonlocal`, :keyword:`yield`, and :keyword:`return` statements may not be used outside of function definitions even within the context of code passed to the :func:`exec` function. The return value is ``None``. In all cases, if the optional parts are omitted, the code is executed in the current scope. If only *globals* is provided, it must be a dictionary (and not a subclass of dictionary), which will be used for both the global and the local variables. If *globals* and *locals* are given, they are used for the global and local variables, respectively. If provided, *locals* can be any mapping object. Remember that at the module level, globals and locals are the same dictionary. .. note:: When ``exec`` gets two separate objects as *globals* and *locals*, the code will be executed as if it were embedded in a class definition. This means functions and classes defined in the executed code will not be able to access variables assigned at the top level (as the "top level" variables are treated as class variables in a class definition). If the *globals* dictionary does not contain a value for the key ``__builtins__``, a reference to the dictionary of the built-in module :mod:`builtins` is inserted under that key. Overriding ``__builtins__`` can be used to restrict or change the available names, but this is **not** a security mechanism: the executed code can still access all builtins. The *closure* argument specifies a closure--a tuple of cellvars. It's only valid when the *object* is a code object containing :term:`free (closure) variables `. The length of the tuple must exactly match the length of the code object's :attr:`~codeobject.co_freevars` attribute. .. audit-event:: exec code_object exec Raises an :ref:`auditing event ` ``exec`` with the code object as the argument. Code compilation events may also be raised. .. note:: The built-in functions :func:`globals` and :func:`locals` return the current global and local namespace, respectively, which may be useful to pass around for use as the second and third argument to :func:`exec`. .. note:: The default *locals* act as described for function :func:`locals` below. Pass an explicit *locals* dictionary if you need to see effects of the code on *locals* after function :func:`exec` returns. .. versionchanged:: 3.11 Added the *closure* parameter. .. versionchanged:: 3.13 The *globals* and *locals* arguments can now be passed as keywords. .. versionchanged:: 3.13 The semantics of the default *locals* namespace have been adjusted as described for the :func:`locals` builtin. .. versionchanged:: 3.15 *globals* can now be a :class:`frozendict`. .. function:: filter(function, iterable, /) Construct an iterator from those elements of *iterable* for which *function* is true. *iterable* may be either a sequence, a container which supports iteration, or an iterator. If *function* is ``None``, the identity function is assumed, that is, all elements of *iterable* that are false are removed. Note that ``filter(function, iterable)`` is equivalent to the generator expression ``(item for item in iterable if function(item))`` if function is not ``None`` and ``(item for item in iterable if item)`` if function is ``None``. See :func:`itertools.filterfalse` for the complementary function that returns elements of *iterable* for which *function* is false. .. class:: float(number=0.0, /) float(string, /) .. index:: single: NaN single: Infinity Return a floating-point number constructed from a number or a string. Examples: .. doctest:: >>> float('+1.23') 1.23 >>> float(' -12345\n') -12345.0 >>> float('1e-003') 0.001 >>> float('+1E6') 1000000.0 >>> float('-Infinity') -inf If the argument is a string, it should contain a decimal number, optionally preceded by a sign, and optionally embedded in whitespace. The optional sign may be ``'+'`` or ``'-'``; a ``'+'`` sign has no effect on the value produced. The argument may also be a string representing a NaN (not-a-number), or positive or negative infinity. More precisely, the input must conform to the :token:`~float:floatvalue` production rule in the following grammar, after leading and trailing whitespace characters are removed: .. productionlist:: float sign: "+" | "-" infinity: "Infinity" | "inf" nan: "nan" digit: digitpart: `digit` (["_"] `digit`)* number: [`digitpart`] "." `digitpart` | `digitpart` ["."] exponent: ("e" | "E") [`sign`] `digitpart` floatnumber: `number` [`exponent`] absfloatvalue: `floatnumber` | `infinity` | `nan` floatvalue: [`sign`] `absfloatvalue` Case is not significant, so, for example, "inf", "Inf", "INFINITY", and "iNfINity" are all acceptable spellings for positive infinity. Otherwise, if the argument is an integer or a floating-point number, a floating-point number with the same value (within Python's floating-point precision) is returned. If the argument is outside the range of a Python float, an :exc:`OverflowError` will be raised. For a general Python object ``x``, ``float(x)`` delegates to ``x.__float__()``. If :meth:`~object.__float__` is not defined then it falls back to :meth:`~object.__index__`. See also :meth:`float.from_number` which only accepts a numeric argument. If no argument is given, ``0.0`` is returned. The float type is described in :ref:`typesnumeric`. .. versionchanged:: 3.6 Grouping digits with underscores as in code literals is allowed. .. versionchanged:: 3.7 The parameter is now positional-only. .. versionchanged:: 3.8 Falls back to :meth:`~object.__index__` if :meth:`~object.__float__` is not defined. .. index:: single: __format__ single: string; format() (built-in function) .. function:: format(value, format_spec="", /) Convert a *value* to a "formatted" representation, as controlled by *format_spec*. The interpretation of *format_spec* will depend on the type of the *value* argument; however, there is a standard formatting syntax that is used by most built-in types: :ref:`formatspec`. The default *format_spec* is an empty string which usually gives the same effect as calling :func:`str(value) `. A call to ``format(value, format_spec)`` is translated to ``type(value).__format__(value, format_spec)`` which bypasses the instance dictionary when searching for the value's :meth:`~object.__format__` method. A :exc:`TypeError` exception is raised if the method search reaches :mod:`object` and the *format_spec* is non-empty, or if either the *format_spec* or the return value are not strings. .. versionchanged:: 3.4 ``object().__format__(format_spec)`` raises :exc:`TypeError` if *format_spec* is not an empty string. .. _func-frozendict: .. class:: frozendict(**kwargs) frozendict(mapping, /, **kwargs) frozendict(iterable, /, **kwargs) :noindex: Create a new frozen dictionary. The :class:`frozendict` object is a built-in class. See also :ref:`typesmapping` for documentation about this class. For other containers see the built-in :class:`dict`, :class:`list`, :class:`set`, and :class:`tuple` classes, as well as the :mod:`collections` module. .. versionadded:: 3.15 .. _func-frozenset: .. class:: frozenset(iterable=(), /) :noindex: Return a new :class:`frozenset` object, optionally with elements taken from *iterable*. :class:`frozenset` is a built-in class. See also :ref:`types-set` for documentation about this class. For other containers see the built-in :class:`set`, :class:`list`, :class:`tuple`, and :class:`dict` classes, as well as the :mod:`collections` module. .. function:: getattr(object, name, /) getattr(object, name, default, /) Return the value of the named attribute of *object*. *name* must be a string. If the string is the name of one of the object's attributes, the result is the value of that attribute. For example, ``getattr(x, 'foobar')`` is equivalent to ``x.foobar``. If the named attribute does not exist, *default* is returned if provided, otherwise :exc:`AttributeError` is raised. *name* need not be a Python identifier (see :func:`setattr`). .. note:: Since :ref:`private name mangling ` happens at compilation time, one must manually mangle a private attribute's (attributes with two leading underscores) name in order to retrieve it with :func:`getattr`. .. function:: globals() Return the dictionary implementing the current module namespace. For code within functions, this is set when the function is defined and remains the same regardless of where the function is called. .. function:: hasattr(object, name, /) The arguments are an object and a string. The result is ``True`` if the string is the name of one of the object's attributes, ``False`` if not. (This is implemented by calling ``getattr(object, name)`` and seeing whether it raises an :exc:`AttributeError` or not.) .. function:: hash(object, /) Return the hash value of the object (if it has one). Hash values are integers. They are used to quickly compare dictionary keys during a dictionary lookup. Numeric values that compare equal have the same hash value (even if they are of different types, as is the case for 1 and 1.0). .. note:: For objects with custom :meth:`~object.__hash__` methods, note that :func:`hash` truncates the return value based on the bit width of the host machine. .. function:: help() help(request) Invoke the built-in help system. (This function is intended for interactive use.) If no argument is given, the interactive help system starts on the interpreter console. If the argument is a string, then the string is looked up as the name of a module, function, class, method, keyword, or documentation topic, and a help page is printed on the console. If the argument is any other kind of object, a help page on the object is generated. Note that if a slash(/) appears in the parameter list of a function when invoking :func:`help`, it means that the parameters prior to the slash are positional-only. For more info, see :ref:`the FAQ entry on positional-only parameters `. This function is added to the built-in namespace by the :mod:`site` module. .. versionchanged:: 3.4 Changes to :mod:`pydoc` and :mod:`inspect` mean that the reported signatures for callables are now more comprehensive and consistent. .. function:: hex(integer, /) Convert an integer number to a lowercase hexadecimal string prefixed with "0x". If *integer* is not a Python :class:`int` object, it has to define an :meth:`~object.__index__` method that returns an integer. Some examples: >>> hex(255) '0xff' >>> hex(-42) '-0x2a' If you want to convert an integer number to an uppercase or lower hexadecimal string with prefix or not, you can use either of the following ways: >>> '%#x' % 255, '%x' % 255, '%X' % 255 ('0xff', 'ff', 'FF') >>> format(255, '#x'), format(255, 'x'), format(255, 'X') ('0xff', 'ff', 'FF') >>> f'{255:#x}', f'{255:x}', f'{255:X}' ('0xff', 'ff', 'FF') See also :func:`format` for more information. See also :func:`int` for converting a hexadecimal string to an integer using a base of 16. .. note:: To obtain a hexadecimal string representation for a float, use the :meth:`float.hex` method. .. function:: id(object, /) Return the "identity" of an object. This is an integer which is guaranteed to be unique and constant for this object during its lifetime. Two objects with non-overlapping lifetimes may have the same :func:`id` value. .. impl-detail:: This is the address of the object in memory. .. audit-event:: builtins.id id id .. function:: input() input(prompt, /) If the *prompt* argument is present, it is written to standard output without a trailing newline. The function then reads a line from input, converts it to a string (stripping a trailing newline), and returns that. When EOF is read, :exc:`EOFError` is raised. Example:: >>> s = input('--> ') # doctest: +SKIP --> Monty Python's Flying Circus >>> s # doctest: +SKIP "Monty Python's Flying Circus" If the :mod:`readline` module was loaded, then :func:`input` will use it to provide elaborate line editing and history features. .. audit-event:: builtins.input prompt input Raises an :ref:`auditing event ` ``builtins.input`` with argument ``prompt`` before reading input .. audit-event:: builtins.input/result result input Raises an :ref:`auditing event ` ``builtins.input/result`` with the result after successfully reading input. .. class:: int(number=0, /) int(string, /, base=10) Return an integer object constructed from a number or a string, or return ``0`` if no arguments are given. Examples: .. doctest:: >>> int(123.45) 123 >>> int('123') 123 >>> int(' -12_345\n') -12345 >>> int('FACE', 16) 64206 >>> int('0xface', 0) 64206 >>> int('01110011', base=2) 115 If the argument defines :meth:`~object.__int__`, ``int(x)`` returns ``x.__int__()``. If the argument defines :meth:`~object.__index__`, it returns ``x.__index__()``. For floating-point numbers, this truncates towards zero. If the argument is not a number or if *base* is given, then it must be a string, :class:`bytes`, or :class:`bytearray` instance representing an integer in radix *base*. Optionally, the string can be preceded by ``+`` or ``-`` (with no space in between), have leading zeros, be surrounded by whitespace, and have single underscores interspersed between digits. A base-n integer string contains digits, each representing a value from 0 to n-1. The values 0--9 can be represented by any Unicode decimal digit. The values 10--35 can be represented by ``a`` to ``z`` (or ``A`` to ``Z``). The default *base* is 10. The allowed bases are 0 and 2--36. Base-2, -8, and -16 strings can be optionally prefixed with ``0b``/``0B``, ``0o``/``0O``, or ``0x``/``0X``, as with integer literals in code. For base 0, the string is interpreted in a similar way to an :ref:`integer literal in code `, in that the actual base is 2, 8, 10, or 16 as determined by the prefix. Base 0 also disallows leading zeros: ``int('010', 0)`` is not legal, while ``int('010')`` and ``int('010', 8)`` are. The integer type is described in :ref:`typesnumeric`. .. versionchanged:: 3.4 If *base* is not an instance of :class:`int` and the *base* object has a :meth:`base.__index__ ` method, that method is called to obtain an integer for the base. Previous versions used :meth:`base.__int__ ` instead of :meth:`base.__index__ `. .. versionchanged:: 3.6 Grouping digits with underscores as in code literals is allowed. .. versionchanged:: 3.7 The first parameter is now positional-only. .. versionchanged:: 3.8 Falls back to :meth:`~object.__index__` if :meth:`~object.__int__` is not defined. .. versionchanged:: 3.11 :class:`int` string inputs and string representations can be limited to help avoid denial of service attacks. A :exc:`ValueError` is raised when the limit is exceeded while converting a string to an :class:`int` or when converting an :class:`int` into a string would exceed the limit. See the :ref:`integer string conversion length limitation ` documentation. .. versionchanged:: 3.14 :func:`int` no longer delegates to the :meth:`~object.__trunc__` method. .. function:: isinstance(object, classinfo, /) Return ``True`` if the *object* argument is an instance of the *classinfo* argument, or of a (direct, indirect, or :term:`virtual `) subclass thereof. If *object* is not an object of the given type, the function always returns ``False``. If *classinfo* is a tuple of type objects (or recursively, other such tuples) or a :ref:`types-union` of multiple types, return ``True`` if *object* is an instance of any of the types. If *classinfo* is not a type or tuple of types and such tuples, a :exc:`TypeError` exception is raised. :exc:`TypeError` may not be raised for an invalid type if an earlier check succeeds. .. versionchanged:: 3.10 *classinfo* can be a :ref:`types-union`. .. function:: issubclass(cls, classinfo, /) Return ``True`` if *cls* is a subclass (direct, indirect, or :term:`virtual `) of *classinfo*. A class is considered a subclass of itself. *classinfo* may be a tuple of class objects (or recursively, other such tuples) or a :ref:`types-union`, in which case return ``True`` if *cls* is a subclass of any entry in *classinfo*. In any other case, a :exc:`TypeError` exception is raised. .. versionchanged:: 3.10 *classinfo* can be a :ref:`types-union`. .. function:: iter(iterable, /) iter(callable, /, stop_value, *, stop_exception=StopIteration) iter(callable, /, *, stop_exception) Return an :term:`iterator` object. The first argument is interpreted very differently depending on the presence of the other arguments. Without other arguments, the single argument must be a collection object which supports the :term:`iterable` protocol (the :meth:`~object.__iter__` method), or it must support the sequence protocol (the :meth:`~object.__getitem__` method with integer arguments starting at ``0``). If it does not support either of those protocols, :exc:`TypeError` is raised. If *stop_value* or *stop_exception* is given, then the first argument must be a callable object. The iterator created in this case will call *callable* with no arguments for each call to its :meth:`~iterator.__next__` method; if the value returned is equal to *stop_value*, or if the call raises an exception matching *stop_exception*, :exc:`StopIteration` will be raised, otherwise the value will be returned. *stop_exception* is an exception class or a tuple of exception classes. If *stop_value* is not specified, the iteration stops only when the callable raises an exception. If the callable raises :exc:`StopIteration` which does not match *stop_exception*, it is replaced with a :exc:`RuntimeError`, as for generators (see :pep:`479`). See also :ref:`typeiter`. One useful application of the second form of :func:`iter` is to build a block-reader. For example, reading fixed-width blocks from a binary database file until the end of file is reached:: from functools import partial with open('mydata.db', 'rb') as f: for block in iter(partial(f.read, 64), b''): process_block(block) *stop_exception* is useful for callables which report :term:`exhaustion ` by raising an exception instead of returning a special value. For example, draining a queue:: import queue for item in iter(input_queue.get_nowait, stop_exception=queue.Empty): process_item(item) .. versionchanged:: 3.16 Added the *stop_exception* parameter and allowed passing *stop_value* by keyword. .. function:: len(object, /) Return the length (the number of items) of an object. The argument may be a sequence (such as a string, bytes, tuple, list, or range) or a collection (such as a dictionary, set, or frozen set). .. impl-detail:: ``len`` raises :exc:`OverflowError` on lengths larger than :data:`sys.maxsize`, such as :class:`range(2 ** 100) `. .. _func-list: .. class:: list(iterable=(), /) :noindex: Rather than being a function, :class:`list` is actually a mutable sequence type, as documented in :ref:`typesseq-list` and :ref:`typesseq`. .. function:: locals() Return a mapping object representing the current local symbol table, with variable names as the keys, and their currently bound references as the values. At module scope, as well as when using :func:`exec` or :func:`eval` with a single namespace, this function returns the same namespace as :func:`globals`. At class scope, it returns the namespace that will be passed to the metaclass constructor. When using ``exec()`` or ``eval()`` with separate local and global arguments, it returns the local namespace passed in to the function call. In all of the above cases, each call to ``locals()`` in a given frame of execution will return the *same* mapping object. Changes made through the mapping object returned from ``locals()`` will be visible as assigned, reassigned, or deleted local variables, and assigning, reassigning, or deleting local variables will immediately affect the contents of the returned mapping object. In an :term:`optimized scope` (including functions, generators, and coroutines), each call to ``locals()`` instead returns a fresh dictionary containing the current bindings of the function's local variables and any nonlocal cell references. In this case, name binding changes made via the returned dict are *not* written back to the corresponding local variables or nonlocal cell references, and assigning, reassigning, or deleting local variables and nonlocal cell references does *not* affect the contents of previously returned dictionaries. Calling ``locals()`` as part of a comprehension in a function, generator, or coroutine is equivalent to calling it in the containing scope, except that the comprehension's initialised iteration variables will be included. In other scopes, it behaves as if the comprehension were running as a nested function. Calling ``locals()`` as part of a generator expression is equivalent to calling it in a nested generator function. .. versionchanged:: 3.12 The behaviour of ``locals()`` in a comprehension has been updated as described in :pep:`709`. .. versionchanged:: 3.13 As part of :pep:`667`, the semantics of mutating the mapping objects returned from this function are now defined. The behavior in :term:`optimized scopes ` is now as described above. Aside from being defined, the behaviour in other scopes remains unchanged from previous versions. .. function:: map(function, iterable, /, *iterables, strict=False) Return an iterator that applies *function* to every item of *iterable*, yielding the results. If additional *iterables* arguments are passed, *function* must take that many arguments and is applied to the items from all iterables in parallel. With multiple iterables, the iterator stops when the shortest iterable is :term:`exhausted`. If *strict* is ``True`` and one of the iterables is exhausted before the others, a :exc:`ValueError` is raised. For cases where the function inputs are already arranged into argument tuples, see :func:`itertools.starmap`. .. versionchanged:: 3.14 Added the *strict* parameter. .. function:: max(iterable, /, *, key=None) max(iterable, /, *, default, key=None) max(arg1, arg2, /, *args, key=None) Return the largest item in an iterable or the largest of two or more arguments. If one positional argument is provided, it should be an :term:`iterable`. The largest item in the iterable is returned. If two or more positional arguments are provided, the largest of the positional arguments is returned. There are two optional keyword-only arguments. The *key* argument specifies a one-argument ordering function like that used for :meth:`list.sort`. The *default* argument specifies an object to return if the provided iterable is empty. If the iterable is empty and *default* is not provided, a :exc:`ValueError` is raised. If multiple items are maximal, the function returns the first one encountered. This is consistent with other sort-stability preserving tools such as ``sorted(iterable, key=keyfunc, reverse=True)[0]`` and ``heapq.nlargest(1, iterable, key=keyfunc)``. .. versionchanged:: 3.4 Added the *default* keyword-only parameter. .. versionchanged:: 3.8 The *key* can be ``None``. .. _func-memoryview: .. class:: memoryview(object) :noindex: Return a "memory view" object created from the given argument. See :ref:`typememoryview` for more information. .. function:: min(iterable, /, *, key=None) min(iterable, /, *, default, key=None) min(arg1, arg2, /, *args, key=None) Return the smallest item in an iterable or the smallest of two or more arguments. If one positional argument is provided, it should be an :term:`iterable`. The smallest item in the iterable is returned. If two or more positional arguments are provided, the smallest of the positional arguments is returned. There are two optional keyword-only arguments. The *key* argument specifies a one-argument ordering function like that used for :meth:`list.sort`. The *default* argument specifies an object to return if the provided iterable is empty. If the iterable is empty and *default* is not provided, a :exc:`ValueError` is raised. If multiple items are minimal, the function returns the first one encountered. This is consistent with other sort-stability preserving tools such as ``sorted(iterable, key=keyfunc)[0]`` and ``heapq.nsmallest(1, iterable, key=keyfunc)``. .. versionchanged:: 3.4 Added the *default* keyword-only parameter. .. versionchanged:: 3.8 The *key* can be ``None``. .. function:: next(iterator, /) next(iterator, default, /) Retrieve the next item from the :term:`iterator` by calling its :meth:`~iterator.__next__` method. If *default* is given, it is returned if the iterator is :term:`exhausted`, otherwise :exc:`StopIteration` is raised. .. class:: object() This is the ultimate base class of all other classes. It has methods that are common to all instances of Python classes. When the constructor is called, it returns a new featureless object. The constructor does not accept any arguments. .. note:: :class:`object` instances do *not* have :attr:`~object.__dict__` attributes, so you can't assign arbitrary attributes to an instance of :class:`object`. .. function:: oct(integer, /) Convert an integer number to an octal string prefixed with "0o". The result is a valid Python expression. If *integer* is not a Python :class:`int` object, it has to define an :meth:`~object.__index__` method that returns an integer. For example: >>> oct(8) '0o10' >>> oct(-56) '-0o70' If you want to convert an integer number to an octal string either with the prefix "0o" or not, you can use either of the following ways. >>> '%#o' % 10, '%o' % 10 ('0o12', '12') >>> format(10, '#o'), format(10, 'o') ('0o12', '12') >>> f'{10:#o}', f'{10:o}' ('0o12', '12') See also :func:`format` for more information. .. index:: single: file object; open() built-in function .. function:: open(file, mode='r', buffering=-1, encoding=None, errors=None, newline=None, closefd=True, opener=None) Open *file* and return a corresponding :term:`file object`. If the file cannot be opened, an :exc:`OSError` is raised. See :ref:`tut-files` for more examples of how to use this function. *file* is a :term:`path-like object` giving the pathname (absolute or relative to the current working directory) of the file to be opened or an integer file descriptor of the file to be wrapped. (If a file descriptor is given, it is closed when the returned I/O object is closed unless *closefd* is set to ``False``.) *mode* is an optional string that specifies the mode in which the file is opened. It defaults to ``'r'`` which means open for reading in text mode. Other common values are ``'w'`` for writing (truncating the file if it already exists), ``'x'`` for exclusive creation, and ``'a'`` for appending (which on *some* Unix systems, means that *all* writes append to the end of the file regardless of the current seek position). In text mode, if *encoding* is not specified, UTF-8 is used by default; if :ref:`Python UTF-8 Mode ` is disabled, :func:`locale.getencoding` is called to get the current locale encoding. (For reading and writing raw bytes use binary mode and leave *encoding* unspecified.) The available modes are: .. _filemodes: .. index:: pair: file; modes ========= =============================================================== Character Meaning ========= =============================================================== ``'r'`` open for reading (default) ``'w'`` open for writing, truncating the file first ``'x'`` open for exclusive creation, failing if the file already exists ``'a'`` open for writing, appending to the end of file if it exists ``'b'`` binary mode ``'t'`` text mode (default) ``'+'`` open for updating (reading and writing) ========= =============================================================== The default mode is ``'r'`` (open for reading text, a synonym of ``'rt'``). Modes ``'w+'`` and ``'w+b'`` open and truncate the file. Modes ``'r+'`` and ``'r+b'`` open the file with no truncation. As mentioned in the :ref:`io-overview`, Python distinguishes between binary and text I/O. Files opened in binary mode (including ``'b'`` in the *mode* argument) return contents as :class:`bytes` objects without any decoding. In text mode (the default, or when ``'t'`` is included in the *mode* argument), the contents of the file are returned as :class:`str`, the bytes having been first decoded using the default encoding or using the specified *encoding* if given. .. note:: Python doesn't depend on the underlying operating system's notion of text files; all the processing is done by Python itself, and is therefore platform-independent. *buffering* is an optional integer used to set the buffering policy. Pass 0 to switch buffering off (only allowed in binary mode), 1 to select line buffering (only usable when writing in text mode), and an integer > 1 to indicate the size in bytes of a fixed-size chunk buffer. Note that specifying a buffer size this way applies for binary buffered I/O, but ``TextIOWrapper`` (i.e., files opened with ``mode='r+'``) would have another buffering. To disable buffering in ``TextIOWrapper``, consider using the ``write_through`` flag for :func:`io.TextIOWrapper.reconfigure`. When no *buffering* argument is given, the default buffering policy works as follows: * Binary files are buffered in fixed-size chunks; the size of the buffer is ``max(min(blocksize, 8 MiB), DEFAULT_BUFFER_SIZE)`` when the device block size is available. On most systems, the buffer will typically be 128 kilobytes long. * "Interactive" text files (files for which :meth:`~io.IOBase.isatty` returns ``True``) use line buffering. Other text files use the policy described above for binary files. *encoding* is the name of the encoding used to decode or encode the file. This should only be used in text mode. The default encoding is UTF-8; if :ref:`Python UTF-8 Mode ` is disabled, the default is platform-dependent (whatever :func:`locale.getencoding` returns). Any :term:`text encoding` supported by Python can be used, and ``encoding="locale"`` specifies the current locale encoding explicitly. See the :mod:`codecs` module for the list of supported encodings. *errors* is an optional string that specifies how encoding and decoding errors are to be handled—this cannot be used in binary mode. A variety of standard error handlers are available, though any error handling name that has been registered with :func:`codecs.register_error` is also valid. The standard names can be found in :ref:`error-handlers`. .. index:: single: universal newlines; open() built-in function .. _open-newline-parameter: *newline* determines how to parse newline characters from the stream. It can be ``None``, ``''``, ``'\n'``, ``'\r'``, and ``'\r\n'``. It works as follows: * When reading input from the stream, if *newline* is ``None``, universal newlines mode is enabled. Lines in the input can end in ``'\n'``, ``'\r'``, or ``'\r\n'``, and these are translated into ``'\n'`` before being returned to the caller. If it is ``''``, universal newlines mode is enabled, but line endings are returned to the caller untranslated. If it has any of the other legal values, input lines are only terminated by the given string, and the line ending is returned to the caller untranslated. * When writing output to the stream, if *newline* is ``None``, any ``'\n'`` characters written are translated to the system default line separator, :data:`os.linesep`. If *newline* is ``''`` or ``'\n'``, no translation takes place. If *newline* is any of the other legal values, any ``'\n'`` characters written are translated to the given string. If *closefd* is ``False`` and a file descriptor rather than a filename was given, the underlying file descriptor will be kept open when the file is closed. If a filename is given *closefd* must be ``True`` (the default); otherwise, an error will be raised. A custom opener can be used by passing a callable as *opener*. The underlying file descriptor for the file object is then obtained by calling *opener* with (*file*, *flags*). *opener* must return an open file descriptor (passing :mod:`os.open` as *opener* results in functionality similar to passing ``None``). The newly created file is :ref:`non-inheritable `. The following example uses the :ref:`dir_fd ` parameter of the :func:`os.open` function to open a file relative to a given directory:: >>> import os >>> dir_fd = os.open('somedir', os.O_RDONLY) >>> def opener(path, flags): ... return os.open(path, flags, dir_fd=dir_fd) ... >>> with open('spamspam.txt', 'w', opener=opener) as f: ... print('This will be written to somedir/spamspam.txt', file=f) ... >>> os.close(dir_fd) # don't leak a file descriptor The type of :term:`file object` returned by the :func:`open` function depends on the mode. When :func:`open` is used to open a file in a text mode (``'w'``, ``'r'``, ``'wt'``, ``'rt'``, etc.), it returns a subclass of :class:`io.TextIOBase` (specifically :class:`io.TextIOWrapper`). When used to open a file in a binary mode with buffering, the returned class is a subclass of :class:`io.BufferedIOBase`. The exact class varies: in read binary mode, it returns an :class:`io.BufferedReader`; in write binary and append binary modes, it returns an :class:`io.BufferedWriter`, and in read/write mode, it returns an :class:`io.BufferedRandom`. When buffering is disabled, the raw stream, a subclass of :class:`io.RawIOBase`, :class:`io.FileIO`, is returned. .. index:: single: line-buffered I/O single: unbuffered I/O single: buffer size, I/O single: I/O control; buffering single: binary mode single: text mode pair: module; sys See also the file handling modules, such as :mod:`fileinput`, :mod:`io` (where :func:`open` is declared), :mod:`os`, :mod:`os.path`, :mod:`tempfile`, and :mod:`shutil`. .. audit-event:: open path,mode,flags open The ``mode`` and ``flags`` arguments may have been modified or inferred from the original call. .. versionchanged:: 3.3 * The *opener* parameter was added. * The ``'x'`` mode was added. * :exc:`IOError` used to be raised, it is now an alias of :exc:`OSError`. * :exc:`FileExistsError` is now raised if the file opened in exclusive creation mode (``'x'``) already exists. .. versionchanged:: 3.4 * The file is now non-inheritable. .. versionchanged:: 3.5 * If the system call is interrupted and the signal handler does not raise an exception, the function now retries the system call instead of raising an :exc:`InterruptedError` exception (see :pep:`475` for the rationale). * The ``'namereplace'`` error handler was added. .. versionchanged:: 3.6 * Support added to accept objects implementing :class:`os.PathLike`. * On Windows, opening a console buffer may return a subclass of :class:`io.RawIOBase` other than :class:`io.FileIO`. .. versionchanged:: 3.11 The ``'U'`` mode has been removed. .. versionchanged:: 3.15 UTF-8 is now the default encoding, instead of the platform-dependent locale encoding (:pep:`686`). .. function:: ord(character, /) Return the ordinal value of a character. If the argument is a one-character string, return the Unicode code point of that character. For example, ``ord('a')`` returns the integer ``97`` and ``ord('€')`` (Euro sign) returns ``8364``. This is the inverse of :func:`chr`. If the argument is a :class:`bytes` or :class:`bytearray` object of length 1, return its single byte value. For example, ``ord(b'a')`` returns the integer ``97``. .. function:: pow(base, exp, mod=None) Return *base* to the power *exp*; if *mod* is present, return *base* to the power *exp*, modulo *mod* (computed more efficiently than ``pow(base, exp) % mod``). The two-argument form ``pow(base, exp)`` is equivalent to using the power operator: ``base**exp``. When arguments are builtin numeric types with mixed operand types, the coercion rules for binary arithmetic operators apply. For :class:`int` operands, the result has the same type as the operands (after coercion) unless the second argument is negative; in that case, all arguments are converted to float and a float result is delivered. For example, ``pow(10, 2)`` returns ``100``, but ``pow(10, -2)`` returns ``0.01``. For a negative base of type :class:`int` or :class:`float` and a non-integral exponent, a complex result is delivered. For example, ``pow(-9, 0.5)`` returns a value close to ``3j``. Whereas, for a negative base of type :class:`int` or :class:`float` with an integral exponent, a float result is delivered. For example, ``pow(-9, 2.0)`` returns ``81.0``. For :class:`int` operands *base* and *exp*, if *mod* is present, *mod* must also be of integer type and *mod* must be nonzero. If *mod* is present and *exp* is negative, *base* must be relatively prime to *mod*. In that case, ``pow(inv_base, -exp, mod)`` is returned, where *inv_base* is an inverse to *base* modulo *mod*. Here's an example of computing an inverse for ``38`` modulo ``97``:: >>> pow(38, -1, mod=97) 23 >>> 23 * 38 % 97 == 1 True .. versionchanged:: 3.8 For :class:`int` operands, the three-argument form of ``pow`` now allows the second argument to be negative, permitting computation of modular inverses. .. versionchanged:: 3.8 Allow keyword arguments. Formerly, only positional arguments were supported. .. function:: print(*objects, sep=' ', end='\n', file=None, flush=False) Print *objects* to the text stream *file*, separated by *sep* and followed by *end*. *sep*, *end*, *file*, and *flush*, if present, must be given as keyword arguments. All non-keyword arguments are converted to strings like :func:`str` does and written to the stream, separated by *sep* and followed by *end*. Both *sep* and *end* must be strings; they can also be ``None``, which means to use the default values. If no *objects* are given, :func:`print` will just write *end*. The *file* argument must be an object with a ``write(string)`` method; if it is not present or ``None``, :data:`sys.stdout` will be used. Since printed arguments are converted to text strings, :func:`print` cannot be used with binary mode file objects. For these, use ``file.write(...)`` instead. Output buffering is usually determined by *file*. However, if *flush* is true, the stream is forcibly flushed. .. versionchanged:: 3.3 Added the *flush* keyword argument. .. class:: property(fget=None, fset=None, fdel=None, doc=None) Return a property attribute. *fget* is a function for getting an attribute value. *fset* is a function for setting an attribute value. *fdel* is a function for deleting an attribute value. And *doc* creates a docstring for the attribute. A typical use is to define a managed attribute ``x``:: class C: def __init__(self): self._x = None def getx(self): return self._x def setx(self, value): self._x = value def delx(self): del self._x x = property(getx, setx, delx, "I'm the 'x' property.") If *c* is an instance of *C*, ``c.x`` will invoke the getter, ``c.x = value`` will invoke the setter, and ``del c.x`` the deleter. If given, *doc* will be the docstring of the property attribute. Otherwise, the property will copy *fget*'s docstring (if it exists). This makes it possible to create read-only properties easily using :deco:`property` as a :term:`decorator`:: class Parrot: def __init__(self): self._voltage = 100000 @property def voltage(self): """Get the current voltage.""" return self._voltage The ``@property`` decorator turns the :meth:`!voltage` method into a "getter" for a read-only attribute with the same name, and it sets the docstring for *voltage* to "Get the current voltage." .. decorator:: property.getter .. decorator:: property.setter .. decorator:: property.deleter A property object has ``getter``, ``setter``, and ``deleter`` methods usable as decorators that create a copy of the property with the corresponding accessor function set to the decorated function. This is best explained with an example: .. testcode:: class C: def __init__(self): self._x = None @property def x(self): """I'm the 'x' property.""" return self._x @x.setter def x(self, value): self._x = value @x.deleter def x(self): del self._x This code is exactly equivalent to the first example. Be sure to give the additional functions the same name as the original property (``x`` in this case.) The returned property object also has the attributes ``fget``, ``fset``, and ``fdel`` corresponding to the constructor arguments. .. versionchanged:: 3.5 The docstrings of property objects are now writeable. .. attribute:: __name__ Attribute holding the name of the property. The name of the property can be changed at runtime. .. versionadded:: 3.13 .. _func-range: .. class:: range(stop, /) range(start, stop, step=1, /) :noindex: Rather than being a function, :class:`range` is actually an immutable sequence type, as documented in :ref:`typesseq-range` and :ref:`typesseq`. .. function:: repr(object, /) Return a string containing a printable representation of an object. For many types, this function makes an attempt to return a string that would yield an object with the same value when passed to :func:`eval`; otherwise, the representation is a string enclosed in angle brackets that contains the name of the type of the object together with additional information often including the name and address of the object. A class can control what this function returns for its instances by defining a :meth:`~object.__repr__` method. If :func:`sys.displayhook` is not accessible, this function will raise :exc:`RuntimeError`. This class has a custom representation that can be evaluated:: class Person: def __init__(self, name, age): self.name = name self.age = age def __repr__(self): return f"Person({self.name!r}, {self.age!r})" .. function:: reversed(object, /) Return a reverse :term:`iterator`. The argument must be an object which has a :meth:`~object.__reversed__` method or supports the sequence protocol (the :meth:`~object.__len__` method and the :meth:`~object.__getitem__` method with integer arguments starting at ``0``). .. function:: round(number, ndigits=None) Return *number* rounded to *ndigits* precision after the decimal point. If *ndigits* is omitted or is ``None``, it returns the nearest integer to its input. For the built-in types supporting :func:`round`, values are rounded to the closest multiple of 10 to the power minus *ndigits*; if two multiples are equally close, rounding is done toward the even choice (so, for example, both ``round(0.5)`` and ``round(-0.5)`` are ``0``, and ``round(1.5)`` is ``2``). Any integer value is valid for *ndigits* (positive, zero, or negative). The return value is an integer if *ndigits* is omitted or ``None``. Otherwise, the return value has the same type as *number*. For a general Python object ``number``, ``round`` delegates to ``number.__round__``. .. note:: The behavior of :func:`round` for floats can be surprising: for example, ``round(2.675, 2)`` gives ``2.67`` instead of the expected ``2.68``. This is not a bug: it's a result of the fact that most decimal fractions can't be represented exactly as a float. See :ref:`tut-fp-issues` for more information. .. _func-set: .. class:: set(iterable=(), /) :noindex: Return a new :class:`set` object, optionally with elements taken from *iterable*. :class:`set` is a built-in class. See also :ref:`types-set` for documentation about this class. For other containers see the built-in :class:`frozenset`, :class:`list`, :class:`tuple`, and :class:`dict` classes, as well as the :mod:`collections` module. .. function:: setattr(object, name, value, /) This is the counterpart of :func:`getattr`. The arguments are an object, a string, and an arbitrary value. The string may name an existing attribute or a new attribute. The function assigns the value to the attribute, provided the object allows it. For example, ``setattr(x, 'foobar', 123)`` is equivalent to ``x.foobar = 123``. *name* need not be a Python identifier as defined in :ref:`identifiers` unless the object chooses to enforce that, for example in a custom :meth:`~object.__getattribute__` or via :attr:`~object.__slots__`. An attribute whose name is not an identifier will not be accessible using the dot notation, but is accessible through :func:`getattr` etc.. .. note:: Since :ref:`private name mangling ` happens at compilation time, one must manually mangle a private attribute's (attributes with two leading underscores) name in order to set it with :func:`setattr`. .. class:: sentinel(name, /, *, repr=None) Return a new unique sentinel object. *name* must be a :class:`str`, and is used by default as the returned object's representation:: >>> MISSING = sentinel("MISSING") >>> MISSING MISSING The optional *repr* argument can be used to specify a different representation:: >>> MISSING = sentinel("MISSING", repr="") >>> MISSING Sentinel objects are truthy and compare equal only to themselves. They are intended to be compared with the :keyword:`is` operator. ``sentinel`` does not support subclassing. Shallow and deep copies of a sentinel object return the object itself. Sentinels are conventionally assigned to a variable with a matching name. Sentinels defined in this way can be used in :term:`type hints `:: MISSING = sentinel("MISSING") def next_value(default: int | MISSING = MISSING): ... Sentinel objects support the :ref:`| ` operator for use in type expressions. :mod:`Pickling ` is supported for sentinel objects that are placed in the global scope of a module under a name matching the sentinel's name, and for sentinels placed in class scopes with a name matching the :term:`qualified name` of the sentinel. Other sentinels, such as those defined in a function scope, are not picklable. The identity of the sentinel is preserved after pickling:: import pickle PICKLABLE = sentinel("PICKLABLE") assert pickle.loads(pickle.dumps(PICKLABLE)) is PICKLABLE class Cls: PICKLABLE = sentinel("Cls.PICKLABLE") assert pickle.loads(pickle.dumps(Cls.PICKLABLE)) is Cls.PICKLABLE Sentinel objects have the following attributes: .. attribute:: __name__ The sentinel's name. .. attribute:: __module__ The name of the module where the sentinel was created. This attribute is writable. .. versionadded:: 3.15 .. class:: slice(stop, /) slice(start, stop, step=None, /) Return a :term:`slice` object representing the set of indices specified by ``range(start, stop, step)``. The *start* and *step* arguments default to ``None``. Slice objects are also generated when :ref:`slicing syntax ` is used. For example: ``a[start:stop:step]`` or ``a[start:stop, i]``. See :func:`itertools.islice` for an alternate version that returns an :term:`iterator`. .. attribute:: slice.start slice.stop slice.step These read-only attributes are set to the argument values (or their default). They have no other explicit functionality; however, they are used by NumPy and other third-party packages. .. versionchanged:: 3.12 Slice objects are now :term:`hashable` (provided :attr:`~slice.start`, :attr:`~slice.stop`, and :attr:`~slice.step` are hashable). .. function:: sorted(iterable, /, *, key=None, reverse=False) Return a new sorted list from the items in *iterable*. Has two optional arguments which must be specified as keyword arguments. *key* specifies a function of one argument that is used to extract a comparison key from each element in *iterable* (for example, ``key=str.lower``). The default value is ``None`` (compare the elements directly). *reverse* is a boolean value. If set to ``True``, then the list elements are sorted as if each comparison were reversed. Use :func:`functools.cmp_to_key` to convert an old-style *cmp* function to a *key* function. The built-in :func:`sorted` function is guaranteed to be stable. A sort is stable if it guarantees not to change the relative order of elements that compare equal --- this is helpful for sorting in multiple passes (for example, sort by department, then by salary grade). The sort algorithm uses only ``<`` comparisons between items. While defining an :meth:`~object.__lt__` method will suffice for sorting, :PEP:`8` recommends that all six :ref:`rich comparisons ` be implemented. This will help avoid bugs when using the same data with other ordering tools such as :func:`max` that rely on a different underlying method. Implementing all six comparisons also helps avoid confusion for mixed type comparisons which can call the reflected :meth:`~object.__gt__` method. For sorting examples and a brief sorting tutorial, see :ref:`sortinghowto`. .. decorator:: staticmethod Transform a method into a static method. A static method does not receive an implicit first argument. To declare a static method, use this idiom:: class C: @staticmethod def f(arg1, arg2, argN): ... The ``@staticmethod`` form is a function :term:`decorator` -- see :ref:`function` for details. A static method can be called either on the class (such as ``C.f()``) or on an instance (such as ``C().f()``). Moreover, the static method :term:`descriptor` is also callable, so it can be used in the class definition (such as ``f()``). Static methods in Python are similar to those found in Java or C++. Also, see :deco:`classmethod` for a variant that is useful for creating alternate class constructors. Like all decorators, it is also possible to call ``staticmethod`` as a regular function and do something with its result. This is needed in some cases where you need a reference to a function from a class body and you want to avoid the automatic transformation to instance method. For these cases, use this idiom:: def regular_function(): ... class C: method = staticmethod(regular_function) For more information on static methods, see :ref:`types`. .. versionchanged:: 3.10 Static methods now inherit the method attributes (:attr:`~function.__module__`, :attr:`~function.__name__`, :attr:`~function.__qualname__`, :attr:`~function.__doc__` and :attr:`~function.__annotations__`), have a new ``__wrapped__`` attribute, and are now callable as regular functions. .. index:: single: string; str() (built-in function) .. _func-str: .. class:: str(*, encoding='utf-8', errors='strict') str(object) str(object, encoding, errors='strict') str(object, *, errors) :noindex: Return a :class:`str` version of *object*. See :func:`str` for details. ``str`` is the built-in string :term:`class`. For general information about strings, see :ref:`textseq`. .. function:: sum(iterable, /, start=0) Sums *start* and the items of an *iterable* from left to right and returns the total. The *iterable*'s items are normally numbers, and the start value is not allowed to be a string. For some use cases, there are good alternatives to :func:`sum`. The preferred, fast way to concatenate a sequence of strings is by calling ``''.join(sequence)``. To add floating-point values with extended precision, see :func:`math.fsum`\. To concatenate a series of iterables, consider using :func:`itertools.chain`. .. versionchanged:: 3.8 The *start* parameter can be specified as a keyword argument. .. versionchanged:: 3.12 Summation of floats switched to an algorithm that gives higher accuracy and better commutativity on most builds. .. versionchanged:: 3.14 Added specialization for summation of complexes, using same algorithm as for summation of floats. .. class:: super() super(type, object_or_type=None, /) Return a proxy object that delegates method calls to a parent or sibling class of *type*. This is useful for accessing inherited methods that have been overridden in a class. The *object_or_type* determines the :term:`method resolution order` to be searched. The search starts from the class right after the *type*. For example, if :attr:`~type.__mro__` of *object_or_type* is ``D -> B -> C -> A -> object`` and the value of *type* is ``B``, then :func:`super` searches ``C -> A -> object``. The :attr:`~type.__mro__` attribute of the class corresponding to *object_or_type* lists the method resolution search order used by both :func:`getattr` and :func:`super`. The attribute is dynamic and can change whenever the inheritance hierarchy is updated. If the second argument is omitted, the super object returned is unbound. If the second argument is an object, ``isinstance(obj, type)`` must be true. If the second argument is a type, ``issubclass(type2, type)`` must be true (this is useful for classmethods). When called directly within an ordinary method of a class, both arguments may be omitted ("zero-argument :func:`!super`"). In this case, *type* will be the enclosing class, and *obj* will be the first argument of the immediately enclosing function (typically ``self``). (This means that zero-argument :func:`!super` will not work as expected within nested functions, including generator expressions, which implicitly create nested functions.) There are two typical use cases for *super*. In a class hierarchy with single inheritance, *super* can be used to refer to parent classes without naming them explicitly, thus making the code more maintainable. This use closely parallels the use of *super* in other programming languages. The second use case is to support cooperative multiple inheritance in a dynamic execution environment. This use case is unique to Python and is not found in statically compiled languages or languages that only support single inheritance. This makes it possible to implement "diamond diagrams" where multiple base classes implement the same method. Good design dictates that such implementations have the same calling signature in every case (because the order of calls is determined at runtime, because that order adapts to changes in the class hierarchy, and because that order can include sibling classes that are unknown prior to runtime). For both use cases, a typical superclass call looks like this:: class C(B): def method(self, arg): super().method(arg) # This does the same thing as: # super(C, self).method(arg) In addition to method lookups, :func:`super` also works for attribute lookups. One possible use case for this is calling :term:`descriptors ` in a parent or sibling class. Note that :func:`super` is implemented as part of the binding process for explicit dotted attribute lookups such as ``super().__getitem__(name)``. It does so by implementing its own :meth:`~object.__getattribute__` method for searching classes in a predictable order that supports cooperative multiple inheritance. Accordingly, :func:`super` is undefined for implicit lookups using statements or operators such as ``super()[name]``. Also note that, aside from the zero argument form, :func:`super` is not limited to use inside methods. The two argument form specifies the arguments exactly and makes the appropriate references. The zero argument form only works inside a class definition, as the compiler fills in the necessary details to correctly retrieve the class being defined, as well as accessing the current instance for ordinary methods. For practical suggestions on how to design cooperative classes using :func:`super`, see `guide to using super() `_. .. versionchanged:: 3.14 :class:`super` objects are now :mod:`pickleable ` and :mod:`copyable `. .. _func-tuple: .. class:: tuple(iterable=(), /) :noindex: Rather than being a function, :class:`tuple` is actually an immutable sequence type, as documented in :ref:`typesseq-tuple` and :ref:`typesseq`. .. class:: type(object, /) type(name, bases, dict, /, **kwargs) .. index:: pair: object; type With one argument, return the type of an *object*. The return value is a type object and generally the same object as returned by :attr:`object.__class__`. The :func:`isinstance` built-in function is recommended for testing the type of an object, because it takes subclasses into account. With three arguments, return a new type object. This is essentially a dynamic form of the :keyword:`class` statement. The *name* string is the class name and becomes the :attr:`~type.__name__` attribute. The *bases* tuple contains the base classes and becomes the :attr:`~type.__bases__` attribute; if empty, :class:`object`, the ultimate base of all classes, is added. The *dict* dictionary contains attribute and method definitions for the class body; it may be copied or wrapped before becoming the :attr:`~type.__dict__` attribute. The following two statements create identical :class:`!type` objects: >>> class X: ... a = 1 ... >>> X = type('X', (), dict(a=1)) See also: * :ref:`Documentation on attributes and methods on classes `. * :ref:`bltin-type-objects` Keyword arguments provided to the three argument form are passed to the appropriate metaclass machinery (usually :meth:`~object.__init_subclass__`) in the same way that keywords in a class definition (besides *metaclass*) would. Unlike a :keyword:`class` statement, the three argument form does not call the metaclass ``__prepare__`` method (see :ref:`prepare`). Use :func:`types.new_class` to dynamically create a class using the appropriate metaclass. See also :ref:`class-customization`. .. versionchanged:: 3.6 Subclasses of :class:`!type` which don't override ``type.__new__`` may no longer use the one-argument form to get the type of an object. .. versionchanged:: 3.15 *dict* can now be a :class:`frozendict`. .. function:: vars() vars(object, /) Return the :attr:`~object.__dict__` attribute for a module, class, instance, or any other object with a :attr:`!__dict__` attribute. Objects such as modules and instances have an updateable :attr:`~object.__dict__` attribute; however, other objects may have write restrictions on their :attr:`!__dict__` attributes (for example, classes use a :class:`types.MappingProxyType` to prevent direct dictionary updates). Without an argument, :func:`vars` acts like :func:`locals`. A :exc:`TypeError` exception is raised if an object is specified but it doesn't have a :attr:`~object.__dict__` attribute (for example, if its class defines the :attr:`~object.__slots__` attribute). .. versionchanged:: 3.13 The result of calling this function without an argument has been updated as described for the :func:`locals` builtin. .. function:: zip(*iterables, strict=False) Iterate over several iterables in parallel, producing tuples with an item from each one. Example:: >>> for item in zip([1, 2, 3], ['sugar', 'spice', 'everything nice']): ... print(item) ... (1, 'sugar') (2, 'spice') (3, 'everything nice') More formally: :func:`zip` returns an iterator of tuples, where the *i*-th tuple contains the *i*-th element from each of the argument iterables. Another way to think of :func:`zip` is that it turns rows into columns, and columns into rows. This is similar to `transposing a matrix `_. :func:`zip` is lazy: The elements won't be processed until the iterable is iterated on, e.g. by a :keyword:`!for` loop or by wrapping in a :class:`list`. One thing to consider is that the iterables passed to :func:`zip` could have different lengths; sometimes by design, and sometimes because of a bug in the code that prepared these iterables. Python offers three different approaches to dealing with this issue: * By default, :func:`zip` stops when the shortest iterable is :term:`exhausted`. It will ignore the remaining items in the longer iterables, cutting off the result to the length of the shortest iterable:: >>> list(zip(range(3), ['fee', 'fi', 'fo', 'fum'])) [(0, 'fee'), (1, 'fi'), (2, 'fo')] * :func:`zip` is often used in cases where the iterables are assumed to be of equal length. In such cases, it's recommended to use the ``strict=True`` option. Its output is the same as regular :func:`zip`:: >>> list(zip(('a', 'b', 'c'), (1, 2, 3), strict=True)) [('a', 1), ('b', 2), ('c', 3)] Unlike the default behavior, it raises a :exc:`ValueError` if one iterable is :term:`exhausted` before the others: >>> for item in zip(range(3), ['fee', 'fi', 'fo', 'fum'], strict=True): # doctest: +SKIP ... print(item) ... (0, 'fee') (1, 'fi') (2, 'fo') Traceback (most recent call last): ... ValueError: zip() argument 2 is longer than argument 1 .. This doctest is disabled because doctest does not support capturing output and exceptions in the same code unit. https://github.com/python/cpython/issues/65382 Without the ``strict=True`` argument, any bug that results in iterables of different lengths will be silenced, possibly manifesting as a hard-to-find bug in another part of the program. * Shorter iterables can be padded with a constant value to make all the iterables have the same length. This is done by :func:`itertools.zip_longest`. Edge cases: With a single iterable argument, :func:`zip` returns an iterator of 1-tuples. With no arguments, it returns an empty iterator. Tips and tricks: * The left-to-right evaluation order of the iterables is guaranteed. This makes possible an idiom for clustering a data series into n-length groups using ``zip(*[iter(s)]*n, strict=True)``. This repeats the *same* iterator ``n`` times so that each output tuple has the result of ``n`` calls to the iterator. This has the effect of dividing the input into n-length chunks. * :func:`zip` in conjunction with the ``*`` operator can be used to unzip a list:: >>> x = [1, 2, 3] >>> y = [4, 5, 6] >>> list(zip(x, y)) [(1, 4), (2, 5), (3, 6)] >>> x2, y2 = zip(*zip(x, y)) >>> x == list(x2) and y == list(y2) True .. versionchanged:: 3.10 Added the ``strict`` argument. .. function:: __import__(name, globals=None, locals=None, fromlist=(), level=0) .. index:: pair: statement; import pair: module; builtins .. note:: This is an advanced function that is not needed in everyday Python programming, unlike :func:`importlib.import_module`. This function is invoked by the :keyword:`import` statement. It can be replaced (by importing the :mod:`builtins` module and assigning to ``builtins.__import__``) in order to change semantics of the :keyword:`!import` statement, but doing so is **strongly** discouraged as it is usually simpler to use import hooks (see :pep:`302`) to attain the same goals and does not cause issues with code which assumes the default import implementation is in use. Direct use of :func:`__import__` is also discouraged in favor of :func:`importlib.import_module`. The function imports the module *name*, potentially using the given *globals* and *locals* to determine how to interpret the name in a package context. The *fromlist* gives the names of objects or submodules that should be imported from the module given by *name*. The standard implementation does not use its *locals* argument at all and uses its *globals* only to determine the package context of the :keyword:`import` statement. *level* specifies whether to use absolute or relative imports. ``0`` (the default) means only perform absolute imports. Positive values for *level* indicate the number of parent directories to search relative to the directory of the module calling :func:`__import__` (see :pep:`328` for the details). When the *name* variable is of the form ``package.module``, normally, the top-level package (the name up till the first dot) is returned, *not* the module named by *name*. However, when a non-empty *fromlist* argument is given, the module named by *name* is returned. For example, the statement ``import spam`` results in bytecode resembling the following code:: spam = __import__('spam', globals(), locals(), [], 0) The statement ``import spam.ham`` results in this call:: spam = __import__('spam.ham', globals(), locals(), [], 0) Note how :func:`__import__` returns the toplevel module here because this is the object that is bound to a name by the :keyword:`import` statement. On the other hand, the statement ``from spam.ham import eggs, sausage as saus`` results in :: _temp = __import__('spam.ham', globals(), locals(), ['eggs', 'sausage'], 0) eggs = _temp.eggs saus = _temp.sausage Here, the ``spam.ham`` module is returned from :func:`__import__`. From this object, the names to import are retrieved and assigned to their respective names. If you simply want to import a module (potentially within a package) by name, use :func:`importlib.import_module`. .. versionchanged:: 3.3 Negative values for *level* are no longer supported (which also changes the default value to 0). .. versionchanged:: 3.9 When the command line options :option:`-E` or :option:`-I` are being used, the environment variable :envvar:`PYTHONCASEOK` is now ignored. .. rubric:: Footnotes .. [#] Note that the parser only accepts the Unix-style end of line convention. If you are reading the code from a file, make sure to use newline conversion mode to convert Windows or Mac-style newlines. --- ### Doc Builtins Index (`Doc/builtins/index.rst`) .. _builtins-index: ############################## Python built-ins reference ############################## Python comes with a number of built-in functions and classes. The built-in classes include data types that would normally be considered part of the "core" of a language, such as numbers and lists. For these types, the Python language core defines the form of literals and places some constraints on their semantics, but does not fully define the semantics. The built-ins also include functions and exceptions --- objects that can be used by all Python code without the need of an :keyword:`import` statement. Some of these are defined by the core language, but many are not essential for the core semantics and are only described here. .. seealso:: In addition to the built-ins, Python provides an extensive importable standard library, see :ref:`library-index`. .. We don't use :numbered: option for the TOC below as it enforces numbered sections for the entire builtin docs. If desired, :numbered: can be enabled on a per-page basis. .. toctree:: :maxdepth: 2 stdtypes.rst constants.rst functions.rst exceptions.rst threadsafety.rst time-complexity.rst --- ### Doc Builtins Stdtypes (`Doc/builtins/stdtypes.rst`) .. XXX: reference/datamodel and this have quite a few overlaps! .. _bltin-types: ************** Built-in Types ************** The following sections describe the standard types that are built into the interpreter. .. index:: pair: built-in; types The principal built-in types are numerics, sequences, mappings, classes, instances and exceptions. Some collection classes are mutable. The methods that add, subtract, or rearrange their members in place, and don't return a specific item, never return the collection instance itself but ``None``. Some operations are supported by several object types; in particular, practically all objects can be compared for equality, tested for truth value, and converted to a string (with the :func:`repr` function or the slightly different :func:`str` function). The latter function is implicitly used when an object is written by the :func:`print` function. .. _truth: Truth Value Testing =================== .. index:: pair: statement; if pair: statement; while pair: truth; value pair: Boolean; operations single: false Any object can be tested for truth value, for use in an :keyword:`if` or :keyword:`while` condition or as operand of the Boolean operations below. .. index:: single: true By default, an object is considered true unless its class defines either a :meth:`~object.__bool__` method that returns ``False`` or a :meth:`~object.__len__` method that returns zero, when called with the object. [1]_ If one of the methods raises an exception when called, the exception is propagated and the object does not have a truth value (for example, :data:`NotImplemented`). Here are most of the built-in objects considered false: .. index:: single: None (Built-in object) single: False (Built-in object) * constants defined to be false: ``None`` and ``False`` * zero of any numeric type: ``0``, ``0.0``, ``0j``, ``Decimal(0)``, ``Fraction(0, 1)`` * empty sequences and collections: ``''``, ``()``, ``[]``, ``{}``, ``set()``, ``range(0)`` .. index:: pair: operator; or pair: operator; and single: False single: True Operations and built-in functions that have a Boolean result always return ``0`` or ``False`` for false and ``1`` or ``True`` for true, unless otherwise stated. (Important exception: the Boolean operations ``or`` and ``and`` always return one of their operands.) .. _boolean: Boolean Operations --- :keyword:`!and`, :keyword:`!or`, :keyword:`!not` ======================================================================= .. index:: pair: Boolean; operations These are the Boolean operations, ordered by ascending priority: +-------------+---------------------------------+-------+ | Operation | Result | Notes | +=============+=================================+=======+ | ``x or y`` | if *x* is true, then *x*, else | \(1) | | | *y* | | +-------------+---------------------------------+-------+ | ``x and y`` | if *x* is false, then *x*, else | \(2) | | | *y* | | +-------------+---------------------------------+-------+ | ``not x`` | if *x* is false, then ``True``, | \(3) | | | else ``False`` | | +-------------+---------------------------------+-------+ .. index:: pair: operator; and pair: operator; or pair: operator; not Notes: (1) This is a short-circuit operator, so it only evaluates the second argument if the first one is false. (2) This is a short-circuit operator, so it only evaluates the second argument if the first one is true. (3) ``not`` has a lower priority than non-Boolean operators, so ``not a == b`` is interpreted as ``not (a == b)``, and ``a == not b`` is a syntax error. .. _stdcomparisons: Comparisons =========== .. index:: pair: chaining; comparisons pair: operator; comparison pair: operator; == pair: operator; < (less) pair: operator; <= pair: operator; > (greater) pair: operator; >= pair: operator; != pair: operator; is pair: operator; is not There are eight comparison operations in Python. They all have the same priority (which is higher than that of the Boolean operations). Comparisons can be chained arbitrarily; for example, ``x < y <= z`` is equivalent to ``x < y and y <= z``, except that *y* is evaluated only once (but in both cases *z* is not evaluated at all when ``x < y`` is found to be false). This table summarizes the comparison operations: +------------+-------------------------+ | Operation | Meaning | +============+=========================+ | ``<`` | strictly less than | +------------+-------------------------+ | ``<=`` | less than or equal | +------------+-------------------------+ | ``>`` | strictly greater than | +------------+-------------------------+ | ``>=`` | greater than or equal | +------------+-------------------------+ | ``==`` | equal | +------------+-------------------------+ | ``!=`` | not equal | +------------+-------------------------+ | ``is`` | object identity | +------------+-------------------------+ | ``is not`` | negated object identity | +------------+-------------------------+ .. index:: pair: object; numeric pair: objects; comparing Unless stated otherwise, objects of different types never compare equal. The ``==`` operator is always defined but for some object types (for example, class objects) is equivalent to :keyword:`is`. The ``<``, ``<=``, ``>`` and ``>=`` operators are only defined where they make sense; for example, they raise a :exc:`TypeError` exception when one of the arguments is a complex number. .. index:: single: __eq__() (instance method) single: __ne__() (instance method) single: __lt__() (instance method) single: __le__() (instance method) single: __gt__() (instance method) single: __ge__() (instance method) Non-identical instances of a class normally compare as non-equal unless the class defines the :meth:`~object.__eq__` method. Instances of a class cannot be ordered with respect to other instances of the same class, or other types of object, unless the class defines enough of the methods :meth:`~object.__lt__`, :meth:`~object.__le__`, :meth:`~object.__gt__`, and :meth:`~object.__ge__` (in general, :meth:`~object.__lt__` and :meth:`~object.__eq__` are sufficient, if you want the conventional meanings of the comparison operators). The behavior of the :keyword:`is` and :keyword:`is not` operators cannot be customized; also they can be applied to any two objects and never raise an exception. .. index:: pair: operator; in pair: operator; not in Two more operations with the same syntactic priority, :keyword:`in` and :keyword:`not in`, are supported by types that are :term:`iterable` or implement the :meth:`~object.__contains__` method. .. _typesnumeric: Numeric Types --- :class:`int`, :class:`float`, :class:`complex` ================================================================ .. index:: pair: object; numeric pair: object; Boolean pair: object; integer pair: object; floating-point pair: object; complex number pair: C; language There are three distinct numeric types: :dfn:`integers`, :dfn:`floating-point numbers`, and :dfn:`complex numbers`. In addition, Booleans are a subtype of integers. Integers have unlimited precision. Floating-point numbers are usually implemented using :c:expr:`double` in C; information about the precision and internal representation of floating-point numbers for the machine on which your program is running is available in :data:`sys.float_info`. Complex numbers have a real and imaginary part, which are each a floating-point number. To extract these parts from a complex number *z*, use ``z.real`` and ``z.imag``. (The standard library includes the additional numeric types :mod:`fractions.Fraction`, for rationals, and :mod:`decimal.Decimal`, for floating-point numbers with user-definable precision.) .. index:: pair: numeric; literals pair: integer; literals pair: floating-point; literals pair: complex number; literals pair: hexadecimal; literals pair: octal; literals pair: binary; literals Numbers are created by numeric literals or as the result of built-in functions and operators. Unadorned integer literals (including hex, octal and binary numbers) yield integers. Numeric literals containing a decimal point or an exponent sign yield floating-point numbers. Appending ``'j'`` or ``'J'`` to a numeric literal yields an imaginary number (a complex number with a zero real part) which you can add to an integer or float to get a complex number with real and imaginary parts. The constructors :func:`int`, :func:`float`, and :func:`complex` can be used to produce numbers of a specific type. .. index:: single: arithmetic pair: built-in function; int pair: built-in function; float pair: built-in function; complex single: operator; + (plus) single: + (plus); unary operator single: + (plus); binary operator single: operator; - (minus) single: - (minus); unary operator single: - (minus); binary operator pair: operator; * (asterisk) pair: operator; / (slash) pair: operator; // pair: operator; % (percent) pair: operator; ** .. _stdtypes-mixed-arithmetic: Python fully supports mixed arithmetic: when a binary arithmetic operator has operands of different built-in numeric types, the operand with the "narrower" type is widened to that of the other: * If both arguments are complex numbers, no conversion is performed; * if either argument is a complex or a floating-point number, the other is converted to a floating-point number; * otherwise, both must be integers and no conversion is necessary. Arithmetic with complex and real operands is defined by the usual mathematical formula, for example:: x + complex(u, v) = complex(x + u, v) x * complex(u, v) = complex(x * u, x * v) A comparison between numbers of different types behaves as though the exact values of those numbers were being compared. [2]_ All numeric types (except complex) support the following operations (for priorities of the operations, see :ref:`operator-summary`): +---------------------+---------------------------------+---------+--------------------+ | Operation | Result | Notes | Full documentation | +=====================+=================================+=========+====================+ | ``x + y`` | sum of *x* and *y* | | | +---------------------+---------------------------------+---------+--------------------+ | ``x - y`` | difference of *x* and *y* | | | +---------------------+---------------------------------+---------+--------------------+ | ``x * y`` | product of *x* and *y* | | | +---------------------+---------------------------------+---------+--------------------+ | ``x / y`` | quotient of *x* and *y* | | | +---------------------+---------------------------------+---------+--------------------+ | ``x // y`` | floored quotient of *x* and | \(1)\(2)| | | | *y* | | | +---------------------+---------------------------------+---------+--------------------+ | ``x % y`` | remainder of ``x / y`` | \(2) | | +---------------------+---------------------------------+---------+--------------------+ | ``-x`` | *x* negated | | | +---------------------+---------------------------------+---------+--------------------+ | ``+x`` | *x* unchanged | | | +---------------------+---------------------------------+---------+--------------------+ | ``abs(x)`` | absolute value or magnitude of | | :func:`abs` | | | *x* | | | +---------------------+---------------------------------+---------+--------------------+ | ``int(x)`` | *x* converted to integer | \(3)\(6)| :func:`int` | +---------------------+---------------------------------+---------+--------------------+ | ``float(x)`` | *x* converted to floating point | \(4)\(6)| :func:`float` | +---------------------+---------------------------------+---------+--------------------+ | ``complex(re, im)`` | a complex number with real part | \(6) | :func:`complex` | | | *re*, imaginary part *im*. | | | | | *im* defaults to zero. | | | +---------------------+---------------------------------+---------+--------------------+ | ``c.conjugate()`` | conjugate of the complex number | | | | | *c* | | | +---------------------+---------------------------------+---------+--------------------+ | ``divmod(x, y)`` | the pair ``(x // y, x % y)`` | \(2) | :func:`divmod` | +---------------------+---------------------------------+---------+--------------------+ | ``pow(x, y)`` | *x* to the power *y* | \(5) | :func:`pow` | +---------------------+---------------------------------+---------+--------------------+ | ``x ** y`` | *x* to the power *y* | \(5) | | +---------------------+---------------------------------+---------+--------------------+ .. index:: triple: operations on; numeric; types single: conjugate() (complex number method) Notes: (1) Also referred to as integer division. For operands of type :class:`int`, the result has type :class:`int`. For operands of type :class:`float`, the result has type :class:`float`. In general, the result is a whole integer, though the result's type is not necessarily :class:`int`. The result is always rounded towards minus infinity: ``1//2`` is ``0``, ``(-1)//2`` is ``-1``, ``1//(-2)`` is ``-1``, and ``(-1)//(-2)`` is ``0``. (2) Not for complex numbers. Instead convert to floats using :func:`abs` if appropriate. (3) .. index:: pair: module; math single: floor() (in module math) single: ceil() (in module math) single: trunc() (in module math) pair: numeric; conversions Conversion from :class:`float` to :class:`int` truncates, discarding the fractional part. See functions :func:`math.floor` and :func:`math.ceil` for alternative conversions. (4) float also accepts the strings "nan" and "inf" with an optional prefix "+" or "-" for Not a Number (NaN) and positive or negative infinity. (5) Python defines ``pow(0, 0)`` and ``0 ** 0`` to be ``1``, as is common for programming languages. (6) The numeric literals accepted include the digits ``0`` to ``9`` or any Unicode equivalent (code points with the ``Nd`` property). See `the Unicode Standard `_ for a complete list of code points with the ``Nd`` property. All :class:`numbers.Real` types (:class:`int` and :class:`float`) also include the following operations: +--------------------+---------------------------------------------+ | Operation | Result | +====================+=============================================+ | :func:`math.trunc(\| *x* truncated to :class:`~numbers.Integral` | | x) ` | | +--------------------+---------------------------------------------+ | :func:`round(x[, | *x* rounded to *n* digits, | | n]) ` | rounding half to even. If *n* is | | | omitted, it defaults to 0. | +--------------------+---------------------------------------------+ | :func:`math.floor(\| the greatest :class:`~numbers.Integral` | | x) ` | <= *x* | +--------------------+---------------------------------------------+ | :func:`math.ceil(x)| the least :class:`~numbers.Integral` >= *x* | | ` | | +--------------------+---------------------------------------------+ For additional numeric operations see the :mod:`math` and :mod:`cmath` modules. .. XXXJH exceptions: overflow (when? what operations?) zerodivision .. _bitstring-ops: Bitwise Operations on Integer Types ----------------------------------- .. index:: triple: operations on; integer; types pair: bitwise; operations pair: shifting; operations pair: masking; operations pair: operator; | (vertical bar) pair: operator; ^ (caret) pair: operator; & (ampersand) pair: operator; << pair: operator; >> pair: operator; ~ (tilde) Bitwise operations only make sense for integers. The result of bitwise operations is calculated as though carried out in two's complement with an infinite number of sign bits. The priorities of the binary bitwise operations are all lower than the numeric operations and higher than the comparisons; the unary operation ``~`` has the same priority as the other unary numeric operations (``+`` and ``-``). This table lists the bitwise operations sorted in ascending priority: +------------+--------------------------------+----------+ | Operation | Result | Notes | +============+================================+==========+ | ``x | y`` | bitwise :dfn:`or` of *x* and | \(4) | | | *y* | | +------------+--------------------------------+----------+ | ``x ^ y`` | bitwise :dfn:`exclusive or` of | \(4) | | | *x* and *y* | | +------------+--------------------------------+----------+ | ``x & y`` | bitwise :dfn:`and` of *x* and | \(4) | | | *y* | | +------------+--------------------------------+----------+ | ``x << n`` | *x* shifted left by *n* bits | (1)(2) | +------------+--------------------------------+----------+ | ``x >> n`` | *x* shifted right by *n* bits | (1)(3) | +------------+--------------------------------+----------+ | ``~x`` | the bits of *x* inverted | | +------------+--------------------------------+----------+ Notes: (1) Negative shift counts are illegal and cause a :exc:`ValueError` to be raised. (2) A left shift by *n* bits is equivalent to multiplication by ``pow(2, n)``. (3) A right shift by *n* bits is equivalent to floor division by ``pow(2, n)``. (4) Performing these calculations with at least one extra sign extension bit in a finite two's complement representation (a working bit-width of ``1 + max(x.bit_length(), y.bit_length())`` or more) is sufficient to get the same result as if there were an infinite number of sign bits. Additional Methods on Integer Types ----------------------------------- The int type implements the :class:`numbers.Integral` :term:`abstract base class`. In addition, it provides a few more methods: .. method:: int.bit_length() Return the number of bits necessary to represent an integer in binary, excluding the sign and leading zeros:: >>> n = -37 >>> bin(n) '-0b100101' >>> n.bit_length() 6 More precisely, if ``x`` is nonzero, then ``x.bit_length()`` is the unique positive integer ``k`` such that ``2**(k-1) <= abs(x) < 2**k``. Equivalently, when ``abs(x)`` is small enough to have a correctly rounded logarithm, then ``k = 1 + int(log(abs(x), 2))``. If ``x`` is zero, then ``x.bit_length()`` returns ``0``. Equivalent to:: def bit_length(self): s = bin(self) # binary representation: bin(-37) --> '-0b100101' s = s.lstrip('-0b') # remove leading zeros and minus sign return len(s) # len('100101') --> 6 .. versionadded:: 3.1 .. method:: int.bit_count() Return the number of ones in the binary representation of the absolute value of the integer. This is also known as the population count. Example:: >>> n = 19 >>> bin(n) '0b10011' >>> n.bit_count() 3 >>> (-n).bit_count() 3 Equivalent to:: def bit_count(self): return bin(self).count("1") .. versionadded:: 3.10 .. method:: int.to_bytes(length=1, byteorder='big', *, signed=False) Return an array of bytes representing an integer. >>> (1024).to_bytes(2, byteorder='big') b'\x04\x00' >>> (1024).to_bytes(10, byteorder='big') b'\x00\x00\x00\x00\x00\x00\x00\x00\x04\x00' >>> (-1024).to_bytes(10, byteorder='big', signed=True) b'\xff\xff\xff\xff\xff\xff\xff\xff\xfc\x00' >>> x = 1000 >>> x.to_bytes((x.bit_length() + 7) // 8, byteorder='little') b'\xe8\x03' The integer is represented using *length* bytes, and defaults to 1. An :exc:`OverflowError` is raised if the integer is not representable with the given number of bytes. The *byteorder* argument determines the byte order used to represent the integer, and defaults to ``"big"``. If *byteorder* is ``"big"``, the most significant byte is at the beginning of the byte array. If *byteorder* is ``"little"``, the most significant byte is at the end of the byte array. The *signed* argument determines whether two's complement is used to represent the integer. If *signed* is ``False`` and a negative integer is given, an :exc:`OverflowError` is raised. The default value for *signed* is ``False``. The default values can be used to conveniently turn an integer into a single byte object:: >>> (65).to_bytes() b'A' However, when using the default arguments, don't try to convert a value greater than 255 or you'll get an :exc:`OverflowError`. Equivalent to:: def to_bytes(n, length=1, byteorder='big', signed=False): if byteorder == 'little': order = range(length) elif byteorder == 'big': order = reversed(range(length)) else: raise ValueError("byteorder must be either 'little' or 'big'") return bytes((n >> i*8) & 0xff for i in order) .. versionadded:: 3.2 .. versionchanged:: 3.11 Added default argument values for ``length`` and ``byteorder``. .. classmethod:: int.from_bytes(bytes, byteorder='big', *, signed=False) Return the integer represented by the given array of bytes. >>> int.from_bytes(b'\x00\x10', byteorder='big') 16 >>> int.from_bytes(b'\x00\x10', byteorder='little') 4096 >>> int.from_bytes(b'\xfc\x00', byteorder='big', signed=True) -1024 >>> int.from_bytes(b'\xfc\x00', byteorder='big', signed=False) 64512 >>> int.from_bytes([255, 0, 0], byteorder='big') 16711680 The argument *bytes* must either be a :term:`bytes-like object` or an iterable producing bytes. The *byteorder* argument determines the byte order used to represent the integer, and defaults to ``"big"``. If *byteorder* is ``"big"``, the most significant byte is at the beginning of the byte array. If *byteorder* is ``"little"``, the most significant byte is at the end of the byte array. To request the native byte order of the host system, use :data:`sys.byteorder` as the byte order value. The *signed* argument indicates whether two's complement is used to represent the integer. Equivalent to:: def from_bytes(bytes, byteorder='big', signed=False): if byteorder == 'little': little_ordered = list(bytes) elif byteorder == 'big': little_ordered = list(reversed(bytes)) else: raise ValueError("byteorder must be either 'little' or 'big'") n = sum(b << i*8 for i, b in enumerate(little_ordered)) if signed and little_ordered and (little_ordered[-1] & 0x80): n -= 1 << 8*len(little_ordered) return n .. versionadded:: 3.2 .. versionchanged:: 3.11 Added default argument value for ``byteorder``. .. method:: int.as_integer_ratio() Return a pair of integers whose ratio is equal to the original integer and has a positive denominator. The integer ratio of integers (whole numbers) is always the integer as the numerator and ``1`` as the denominator. .. versionadded:: 3.8 .. method:: int.is_integer() Returns ``True``. Exists for duck type compatibility with :meth:`float.is_integer`. .. versionadded:: 3.12 Additional Methods on Float --------------------------- The float type implements the :class:`numbers.Real` :term:`abstract base class`. float also has the following additional methods. .. classmethod:: float.from_number(x) Class method to return a floating-point number constructed from a number *x*. If the argument is an integer or a floating-point number, a floating-point number with the same value (within Python's floating-point precision) is returned. If the argument is outside the range of a Python float, an :exc:`OverflowError` will be raised. For a general Python object ``x``, ``float.from_number(x)`` delegates to ``x.__float__()``. If :meth:`~object.__float__` is not defined then it falls back to :meth:`~object.__index__`. .. versionadded:: 3.14 .. method:: float.as_integer_ratio() Return a pair of integers whose ratio is exactly equal to the original float. The ratio is in lowest terms and has a positive denominator. Raises :exc:`OverflowError` on infinities and a :exc:`ValueError` on NaNs. .. method:: float.is_integer() Return ``True`` if the float instance is finite with integral value, and ``False`` otherwise:: >>> (-2.0).is_integer() True >>> (3.2).is_integer() False Two methods support conversion to and from hexadecimal strings. Since Python's floats are stored internally as binary numbers, converting a float to or from a *decimal* string usually involves a small rounding error. In contrast, hexadecimal strings allow exact representation and specification of floating-point numbers. This can be useful when debugging, and in numerical work. .. method:: float.hex() Return a representation of a floating-point number as a hexadecimal string. For finite floating-point numbers, this representation will always include a leading ``0x`` and a trailing ``p`` and exponent. .. classmethod:: float.fromhex(s) Class method to return the float represented by a hexadecimal string *s*. The string *s* may have leading and trailing whitespace. Note that :meth:`float.hex` is an instance method, while :meth:`float.fromhex` is a class method. A hexadecimal string takes the form:: [sign] ['0x'] integer ['.' fraction] ['p' exponent] where the optional ``sign`` may be either ``+`` or ``-``, ``integer`` and ``fraction`` are strings of hexadecimal digits, and ``exponent`` is a decimal integer with an optional leading sign. Case is not significant, and there must be at least one hexadecimal digit in either the integer or the fraction. This syntax is similar to the syntax specified in section 6.4.4.2 of the C99 standard, and also to the syntax used in Java 1.5 onwards. In particular, the output of :meth:`float.hex` is usable as a hexadecimal floating-point literal in C or Java code, and hexadecimal strings produced by C's ``%a`` format character or Java's ``Double.toHexString`` are accepted by :meth:`float.fromhex`. Note that the exponent is written in decimal rather than hexadecimal, and that it gives the power of 2 by which to multiply the coefficient. For example, the hexadecimal string ``0x3.a7p10`` represents the floating-point number ``(3 + 10./16 + 7./16**2) * 2.0**10``, or ``3740.0``:: >>> float.fromhex('0x3.a7p10') 3740.0 Applying the reverse conversion to ``3740.0`` gives a different hexadecimal string representing the same number:: >>> float.hex(3740.0) '0x1.d380000000000p+11' Additional Methods on Complex ----------------------------- The :class:`!complex` type implements the :class:`numbers.Complex` :term:`abstract base class`. :class:`!complex` also has the following additional methods. .. classmethod:: complex.from_number(x) Class method to convert a number to a complex number. For a general Python object ``x``, ``complex.from_number(x)`` delegates to ``x.__complex__()``. If :meth:`~object.__complex__` is not defined then it falls back to :meth:`~object.__float__`. If :meth:`!__float__` is not defined then it falls back to :meth:`~object.__index__`. .. versionadded:: 3.14 .. _numeric-hash: Hashing of numeric types ------------------------ For numbers ``x`` and ``y``, possibly of different types, it's a requirement that ``hash(x) == hash(y)`` whenever ``x == y`` (see the :meth:`~object.__hash__` method documentation for more details). For ease of implementation and efficiency across a variety of numeric types (including :class:`int`, :class:`float`, :class:`decimal.Decimal` and :class:`fractions.Fraction`) Python's hash for numeric types is based on a single mathematical function that's defined for any rational number, and hence applies to all instances of :class:`int` and :class:`fractions.Fraction`, and all finite instances of :class:`float` and :class:`decimal.Decimal`. Essentially, this function is given by reduction modulo ``P`` for a fixed prime ``P``. The value of ``P`` is made available to Python as the :attr:`~sys.hash_info.modulus` attribute of :data:`sys.hash_info`. .. impl-detail:: Currently, the prime used is ``P = 2**31 - 1`` on machines with 32-bit C longs and ``P = 2**61 - 1`` on machines with 64-bit C longs. Here are the rules in detail: - If ``x = m / n`` is a nonnegative rational number and ``n`` is not divisible by ``P``, define ``hash(x)`` as ``m * invmod(n, P) % P``, where ``invmod(n, P)`` gives the inverse of ``n`` modulo ``P``. - If ``x = m / n`` is a nonnegative rational number and ``n`` is divisible by ``P`` (but ``m`` is not) then ``n`` has no inverse modulo ``P`` and the rule above doesn't apply; in this case define ``hash(x)`` to be the constant value ``sys.hash_info.inf``. - If ``x = m / n`` is a negative rational number define ``hash(x)`` as ``-hash(-x)``. If the resulting hash is ``-1``, replace it with ``-2``. - The particular values ``sys.hash_info.inf`` and ``-sys.hash_info.inf`` are used as hash values for positive infinity or negative infinity (respectively). - For a :class:`complex` number ``z``, the hash values of the real and imaginary parts are combined by computing ``hash(z.real) + sys.hash_info.imag * hash(z.imag)``, reduced modulo ``2**sys.hash_info.width`` so that it lies in ``range(-2**(sys.hash_info.width - 1), 2**(sys.hash_info.width - 1))``. Again, if the result is ``-1``, it's replaced with ``-2``. To clarify the above rules, here's some example Python code, equivalent to the built-in hash, for computing the hash of a rational number, :class:`float`, or :class:`complex`:: import sys, math def hash_fraction(m, n): """Compute the hash of a rational number m / n. Assumes m and n are integers, with n positive. Equivalent to hash(fractions.Fraction(m, n)). """ P = sys.hash_info.modulus # Remove common factors of P. (Unnecessary if m and n already coprime.) while m % P == n % P == 0: m, n = m // P, n // P if n % P == 0: hash_value = sys.hash_info.inf else: # Fermat's Little Theorem: pow(n, P-1, P) is 1, so # pow(n, P-2, P) gives the inverse of n modulo P. hash_value = (abs(m) % P) * pow(n, P - 2, P) % P if m < 0: hash_value = -hash_value if hash_value == -1: hash_value = -2 return hash_value def hash_float(x): """Compute the hash of a float x.""" if math.isnan(x): return object.__hash__(x) elif math.isinf(x): return sys.hash_info.inf if x > 0 else -sys.hash_info.inf else: return hash_fraction(*x.as_integer_ratio()) def hash_complex(z): """Compute the hash of a complex number z.""" hash_value = hash_float(z.real) + sys.hash_info.imag * hash_float(z.imag) # do a signed reduction modulo 2**sys.hash_info.width M = 2**(sys.hash_info.width - 1) hash_value = (hash_value & (M - 1)) - (hash_value & M) if hash_value == -1: hash_value = -2 return hash_value .. _bltin-boolean-values: .. _typebool: Boolean Type - :class:`bool` ============================ Booleans represent truth values. The :class:`bool` type has exactly two constant instances: ``True`` and ``False``. .. index:: single: False single: True pair: Boolean; values The built-in function :func:`bool` converts any value to a boolean, if the value can be interpreted as a truth value (see section :ref:`truth` above). For logical operations, use the :ref:`boolean operators ` ``and``, ``or`` and ``not``. When applying the bitwise operators ``&``, ``|``, ``^`` to two booleans, they return a bool equivalent to the logical operations "and", "or", "xor". However, the logical operators ``and``, ``or`` and ``!=`` should be preferred over ``&``, ``|`` and ``^``. .. deprecated:: 3.12 The use of the bitwise inversion operator ``~`` is deprecated and will raise an error in Python 3.16. :class:`bool` is a subclass of :class:`int` (see :ref:`typesnumeric`). In many numeric contexts, ``False`` and ``True`` behave like the integers 0 and 1, respectively. However, relying on this is discouraged; explicitly convert using :func:`int` instead. .. _iterator-types: .. _typeiter: Iteration-related types ======================= .. index:: single: iterator protocol single: protocol; iterator single: sequence; iteration single: container; iteration over Python supports a concept of iteration over containers. This is implemented using two distinct methods; these are used to allow user-defined classes to support iteration. Sequences, described below in more detail, always support the iteration methods. Iterables --------- One method needs to be defined for container objects to provide :term:`iterable` support: .. XXX duplicated in reference/datamodel! .. method:: container.__iter__() Return an :term:`iterator` object. The object is required to support the iterator protocol described below. If a container supports different types of iteration, additional methods can be provided to specifically request iterators for those iteration types. (An example of an object supporting multiple forms of iteration would be a tree structure which supports both breadth-first and depth-first traversal.) This method corresponds to the :c:member:`~PyTypeObject.tp_iter` slot of the type structure for Python objects in the Python/C API. .. _stdtypes-iterators: Iterators --------- The iterator objects themselves are required to support the following two methods, which together form the :dfn:`iterator protocol`: .. method:: iterator.__iter__() Return the :term:`iterator` object itself. This is required to allow both containers and iterators to be used with the :keyword:`for` and :keyword:`in` statements. This method corresponds to the :c:member:`~PyTypeObject.tp_iter` slot of the type structure for Python objects in the Python/C API. .. method:: iterator.__next__() Return the next item from the :term:`iterator`. If there are no further items, raise the :exc:`StopIteration` exception. This method corresponds to the :c:member:`~PyTypeObject.tp_iternext` slot of the type structure for Python objects in the Python/C API. Python defines several iterator objects to support iteration over general and specific sequence types, dictionaries, and other more specialized forms. The specific types are not important beyond their implementation of the iterator protocol. Once an iterator's :meth:`~iterator.__next__` method raises :exc:`StopIteration`, it must continue to do so on subsequent calls. Implementations that do not obey this property are deemed broken. .. _generator-types: Generator types --------------- Python's :term:`generators ` -- or more precisely, :term:`generator functions ` and :term:`generator iterators ` -- provide a convenient way to implement the iterator protocol. A function that contains one or more :ref:`yield expressions ` is a :term:`generator function`. For example:: >>> def count_to_three(): ... yield 0 ... yield 1 ... yield 2 ... yield 3 Generator functions behave as regular :ref:`user-defined functions ` (for example, they have the same attributes), except that calling a generator function returns a :ref:`generator iterator `:: >>> count_to_three() Iterating a generator iterator executes code of the underlying generator function, producing each :keyword:`yield`\ed value in turn:: >>> for number in count_to_three(): ... print(number) 0 1 2 3 >>> list(count_to_three()) [0, 1, 2, 3] One common use for generator functions is implementing the :meth:`~object.__iter__` method of custom iterable objects. For example:: >>> class CardDeck: ... def __iter__(self): ... yield 'three of clubs' ... yield 'ace of hearts' >>> list(CardDeck()) ['three of clubs', 'ace of hearts'] .. index:: pair: object; generator .. _generator-methods: Generator iterators ^^^^^^^^^^^^^^^^^^^ Generator iterators implement the :ref:`iterator protocol `. Iterating them drives execution of the underlying generator function. .. index:: pair: exception; StopIteration .. method:: generator.__next__() Starts the execution of a generator function or resumes it at the :ref:`yield expression ` where the function is currently suspended. When a generator function is resumed with a :meth:`~generator.__next__` method, the current yield expression always evaluates to :const:`None`. The execution then continues to the next yield expression, where the generator is suspended again, and the value of the expression after the :keyword:`yield` keyword is returned to :meth:`~generator.__next__`'s caller. If the generator exits without yielding another value, :meth:`~generator.__next__` raises a :exc:`StopIteration` exception, signalling that iteration has completed. This method is normally called implicitly, for example by a :keyword:`for` loop, or by the built-in :func:`next` function. Generator iterators have a few more methods than generic iterators, which can be used to control the execution of the underlying generator function: .. method:: generator.send(value) "Sends" a value into the generator function: the *value* argument becomes the result of the current yield expression. Otherwise, this method behaves like :meth:`~generator.__next__`: it resumes the underlying function and either returns the next yielded value or raises :exc:`StopIteration`. When :meth:`send` is called to start the generator, it must be called with :const:`None` as the argument, because there is no current yield expression that could receive the value. .. method:: generator.throw(value) generator.throw(type[, value[, traceback]]) Raises an exception at the point where the generator is currently suspended. Otherwise, this method behaves like :meth:`~generator.__next__`: it resumes the underlying function and either returns the next yielded value or raises :exc:`StopIteration`. If the generator function does not catch the passed-in exception, or raises a different exception, then that exception propagates to the caller. When :meth:`throw` is called to start the generator, the generator immediately exits (that is, subsequent calls to :meth:`~generator.__next__` will raise :exc:`StopIteration`) and the thrown exception is propagated to :meth:`throw`'s caller. In typical use, this is called with a single argument, an exception instance, similar to the way the :keyword:`raise` keyword is used. For backwards compatibility, however, the second signature is supported, following a convention from older versions of Python. The *type* argument should be an exception class, and *value* should be an exception instance. If the *value* is not provided, the *type* constructor is called to get an instance. If *traceback* is provided, it is set on the exception, otherwise any existing :attr:`~BaseException.__traceback__` attribute stored in *value* may be cleared. .. versionchanged:: 3.12 The second signature \(type\[, value\[, traceback\]\]\) is deprecated and may be removed in a future version of Python. .. index:: pair: exception; GeneratorExit .. method:: generator.close() Raises a :exc:`GeneratorExit` exception at the point where the generator function is currently suspended (equivalent to calling ``throw(GeneratorExit)``). If the generator function has already exited (due to an exception or normal return), or raises :exc:`GeneratorExit` (by not catching the exception), :meth:`close` returns :const:`None`. If the generator yields a value, a :exc:`RuntimeError` is raised. If the generator raises any other exception, it is propagated to the caller. If a generator returns a value upon being closed, that value is returned by :meth:`close`. When a generator iterator is garbage collected before it has exited, :meth:`~generator.close` is called automatically. .. versionchanged:: 3.13 If a generator returns a value upon being closed, the value is returned by :meth:`close`. Previously, it returned ``None``. Calling any of the generator methods (:meth:`~generator.__next__`, :meth:`~generator.send`, :meth:`~generator.throw`, :meth:`~generator.close`) while one of these methods is already executing raises a :exc:`ValueError` exception. .. index:: pair: object; asynchronous-generator .. _asynchronous-generator-methods: Asynchronous generator iterators ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ This subsection describes the methods of an asynchronous generator iterator, which are used to control the execution of an asynchronous generator function. .. index:: pair: exception; StopAsyncIteration .. method:: agen.__anext__() :async: Returns an :term:`awaitable` which when run starts to execute the asynchronous generator function or resumes it at the :ref:`yield expression ` where the function is currently suspended. When an asynchronous generator function is resumed with an :meth:`~agen.__anext__` method, the current yield expression always evaluates to :const:`None` in the returned awaitable, which when run will continue to the next yield expression. The value of the expression after the :keyword:`yield` keyword is the value of the :exc:`StopIteration` exception raised by the completing coroutine. If the asynchronous generator exits without yielding another value, the awaitable instead raises a :exc:`StopAsyncIteration` exception, signalling that the asynchronous iteration has completed. This method is normally called implicitly by an :keyword:`async for` loop, or by the built-in :func:`anext` function. Asynchronous generator-iterators have a few more methods than generic asynchronous iterators, which can be used to control the execution of the underlying generator function: .. method:: agen.asend(value) :async: Returns an awaitable which, when run, "sends" a value into the underlying asynchronous generator function: the *value* argument becomes the result of the current yield expression. Otherwise, this method behaves like :meth:`~agen.__anext__`: when the returned awaitable runs, it resumes the underlying function and either returns the next yielded value as the value of the raised :exc:`StopIteration`, or raises :exc:`StopAsyncIteration`. When :meth:`asend` is called to start the asynchronous generator, it must be called with :const:`None` as the argument, because there is no yield expression that could receive the value. .. method:: agen.athrow(value) agen.athrow(type[, value[, traceback]]) :async: Returns an awaitable that, when run, raises an exception at the point where the underlying asynchronous generator function is currently suspended. Otherwise, this method behaves like :meth:`~agen.__anext__`: when the returned awaitable runs, it resumes the underlying function (with an exception raised) and either returns the next yielded value as the value of the raised :exc:`StopIteration`, or raises :exc:`StopAsyncIteration`. If the underlying function does not catch the passed-in exception, or raises a different exception, then when the awaitable is run, that exception propagates to the caller of the awaitable. When :meth:`~agen.athrow` is called to start the generator, the generator exits when the awaitable runs (that is, subsequent results from :meth:`~agen.__anext__` will raise :exc:`StopAsyncIteration` when run) and the thrown exception is propagated to the awaitable's caller. In typical use, this is called with a single argument, an exception instance, similar to the way the :keyword:`raise` keyword is used. For backwards compatibility, however, the second signature is supported. An exception instance is created from three arguments in the same way as in :meth:`generator.throw`. .. versionchanged:: 3.12 The second signature \(type\[, value\[, traceback\]\]\) is deprecated and may be removed in a future version of Python. .. index:: pair: exception; GeneratorExit .. method:: agen.aclose() :async: Returns an awaitable that when run will throw a :exc:`GeneratorExit` into the underlying asynchronous generator function at the point where it is currently suspended (equivalent to calling ``athrow(GeneratorExit)``). If the asynchronous generator function then exits gracefully, is already closed, or raises :exc:`GeneratorExit` (by not catching the exception), then the returned awaitable will raise a :exc:`StopIteration` exception. Any further awaitables returned by subsequent calls to the asynchronous generator will raise a :exc:`StopAsyncIteration` exception. If the asynchronous generator yields a value, a :exc:`RuntimeError` is raised by the awaitable. If the asynchronous generator raises any other exception, that exception is propagated to the caller of the awaitable. If the asynchronous generator has already exited due to an exception or normal exit, then further calls to :meth:`aclose` will return an awaitable that does nothing. .. _typesseq: Sequence Types --- :class:`list`, :class:`tuple`, :class:`range` ================================================================ There are three basic sequence types: lists, tuples, and range objects. Additional sequence types tailored for processing of :ref:`binary data ` and :ref:`text strings ` are described in dedicated sections. .. _typesseq-common: Common Sequence Operations -------------------------- .. index:: pair: object; sequence The operations in the following table are supported by most sequence types, both mutable and immutable. The :class:`collections.abc.Sequence` ABC is provided to make it easier to correctly implement these operations on custom sequence types. This table lists the sequence operations sorted in ascending priority. In the table, *s* and *t* are sequences of the same type, *n*, *i*, *j* and *k* are integers and *x* is an arbitrary object that meets any type and value restrictions imposed by *s*. The ``in`` and ``not in`` operations have the same priorities as the comparison operations. The ``+`` (concatenation) and ``*`` (repetition) operations have the same priority as the corresponding numeric operations. [3]_ See :ref:`time-complexity` for the costs of the various sequence operations. .. index:: triple: operations on; sequence; types pair: built-in function; len pair: built-in function; min pair: built-in function; max pair: concatenation; operation pair: repetition; operation pair: subscript; operation pair: slice; operation pair: operator; in pair: operator; not in +--------------------------+--------------------------------+----------+ | Operation | Result | Notes | +==========================+================================+==========+ | ``x in s`` | ``True`` if an item of *s* is | \(1) | | | equal to *x*, else ``False`` | | +--------------------------+--------------------------------+----------+ | ``x not in s`` | ``False`` if an item of *s* is | \(1) | | | equal to *x*, else ``True`` | | +--------------------------+--------------------------------+----------+ | ``s + t`` | the concatenation of *s* and | (6)(7) | | | *t* | | +--------------------------+--------------------------------+----------+ | ``s * n`` or | equivalent to adding *s* to | (2)(7) | | ``n * s`` | itself *n* times | | +--------------------------+--------------------------------+----------+ | ``s[i]`` | *i*\ th item of *s*, origin 0 | (3)(8) | +--------------------------+--------------------------------+----------+ | ``s[i:j]`` | slice of *s* from *i* to *j* | (3)(4) | +--------------------------+--------------------------------+----------+ | ``s[i:j:k]`` | slice of *s* from *i* to *j* | (3)(5) | | | with step *k* | | +--------------------------+--------------------------------+----------+ | ``len(s)`` | length of *s* | | +--------------------------+--------------------------------+----------+ | ``min(s)`` | smallest item of *s* | | +--------------------------+--------------------------------+----------+ | ``max(s)`` | largest item of *s* | | +--------------------------+--------------------------------+----------+ Sequences of the same type also support comparisons. In particular, tuples and lists are compared lexicographically by comparing corresponding elements. This means that to compare equal, every element must compare equal and the two sequences must be of the same type and have the same length. (For full details see :ref:`comparisons` in the language reference.) .. index:: single: loop; over mutable sequence single: mutable sequence; loop over Forward and reversed iterators over mutable sequences access values using an index. That index will continue to march forward (or backward) even if the underlying sequence is mutated. The iterator terminates only when an :exc:`IndexError` or a :exc:`StopIteration` is encountered (or when the index drops below zero). Notes: (1) While the ``in`` and ``not in`` operations are used only for simple containment testing in the general case, some specialised sequences (such as :class:`str`, :class:`bytes` and :class:`bytearray`) also use them for subsequence testing:: >>> "gg" in "eggs" True (2) Values of *n* less than ``0`` are treated as ``0`` (which yields an empty sequence of the same type as *s*). Note that items in the sequence *s* are not copied; they are referenced multiple times. This often haunts new Python programmers; consider:: >>> lists = [[]] * 3 >>> lists [[], [], []] >>> lists[0].append(3) >>> lists [[3], [3], [3]] What has happened is that ``[[]]`` is a one-element list containing an empty list, so all three elements of ``[[]] * 3`` are references to this single empty list. Modifying any of the elements of ``lists`` modifies this single list. You can create a list of different lists this way:: >>> lists = [[] for i in range(3)] >>> lists[0].append(3) >>> lists[1].append(5) >>> lists[2].append(7) >>> lists [[3], [5], [7]] Further explanation is available in the FAQ entry :ref:`faq-multidimensional-list`. (3) If *i* or *j* is negative, the index is relative to the end of sequence *s*: ``len(s) + i`` or ``len(s) + j`` is substituted. But note that ``-0`` is still ``0``. (4) The slice of *s* from *i* to *j* is defined as the sequence of items with index *k* such that ``i <= k < j``. * If *i* is omitted or ``None``, use ``0``. * If *j* is omitted or ``None``, use ``len(s)``. * If *i* or *j* is less than ``-len(s)``, use ``0``. * If *i* or *j* is greater than ``len(s)``, use ``len(s)``. * If *i* is greater than or equal to *j*, the slice is empty. (5) The slice of *s* from *i* to *j* with step *k* is defined as the sequence of items with index ``x = i + n*k`` such that ``0 <= n < (j-i)/k``. In other words, the indices are ``i``, ``i+k``, ``i+2*k``, ``i+3*k`` and so on, stopping when *j* is reached (but never including *j*). When *k* is positive, *i* and *j* are reduced to ``len(s)`` if they are greater. When *k* is negative, *i* and *j* are reduced to ``len(s) - 1`` if they are greater. If *i* or *j* are omitted or ``None``, they become "end" values (which end depends on the sign of *k*). Note, *k* cannot be zero. If *k* is ``None``, it is treated like ``1``. .. _typesseq-repeated-concatenation: (6) Concatenating immutable sequences always results in a new object. This means that building up a sequence by repeated concatenation will have a quadratic runtime cost in the total sequence length. To get a linear runtime cost, you must switch to one of the alternatives below: * if concatenating :class:`str` objects, you can build a list and use :meth:`str.join` at the end or else write to an :class:`io.StringIO` instance and retrieve its value when complete * if concatenating :class:`bytes` objects, you can similarly use :meth:`bytes.join` or :class:`io.BytesIO`, or you can do in-place concatenation with a :class:`bytearray` object. :class:`bytearray` objects are mutable and have an efficient overallocation mechanism * if concatenating :class:`tuple` objects, extend a :class:`list` instead * for other types, investigate the relevant class documentation (7) Some sequence types (such as :class:`range`) only support item sequences that follow specific patterns, and hence don't support sequence concatenation or repetition. (8) An :exc:`IndexError` is raised if *i* is outside the sequence range. .. rubric:: Sequence Methods Sequence types also support the following methods: .. method:: list.count(value, /) range.count(value, /) tuple.count(value, /) :no-contents-entry: :no-index-entry: :no-typesetting: .. method:: sequence.count(value, /) Return the total number of occurrences of *value* in *sequence*. .. method:: list.index(value[, start[, stop]]) range.index(value[, start[, stop]]) tuple.index(value[, start[, stop]]) :no-contents-entry: :no-index-entry: :no-typesetting: .. method:: sequence.index(value[, start[, stop]]) Return the index of the first occurrence of *value* in *sequence*. Raises :exc:`ValueError` if *value* is not found in *sequence*. The *start* or *stop* arguments allow for efficient searching of subsections of the sequence, beginning at *start* and ending at *stop*. This is roughly equivalent to ``start + sequence[start:stop].index(value)``, only without copying any data. .. caution:: Not all sequence types support passing the *start* and *stop* arguments. .. _typesseq-immutable: Immutable Sequence Types ------------------------ .. index:: triple: immutable; sequence; types pair: object; tuple pair: built-in function; hash The only operation that immutable sequence types generally implement that is not also implemented by mutable sequence types is support for the :func:`hash` built-in. This support allows immutable sequences, such as :class:`tuple` instances, to be used as :class:`dict` keys and stored in :class:`set` and :class:`frozenset` instances. Attempting to hash an immutable sequence that contains unhashable values will result in :exc:`TypeError`. .. _typesseq-mutable: Mutable Sequence Types ---------------------- .. index:: triple: mutable; sequence; types pair: object; list pair: object; bytearray The operations in the following table are defined on mutable sequence types. The :class:`collections.abc.MutableSequence` ABC is provided to make it easier to correctly implement these operations on custom sequence types. In the table *s* is an instance of a mutable sequence type, *t* is any iterable object and *x* is an arbitrary object that meets any type and value restrictions imposed by *s* (for example, :class:`bytearray` only accepts integers that meet the value restriction ``0 <= x <= 255``). .. index:: triple: operations on; sequence; types triple: operations on; list; type pair: subscript; assignment pair: slice; assignment pair: statement; del +------------------------------+--------------------------------+---------------------+ | Operation | Result | Notes | +==============================+================================+=====================+ | ``s[i] = x`` | item *i* of *s* is replaced by | | | | *x* | | +------------------------------+--------------------------------+---------------------+ | ``del s[i]`` | removes item *i* of *s* | | +------------------------------+--------------------------------+---------------------+ | ``s[i:j] = t`` | slice of *s* from *i* to *j* | | | | is replaced by the contents of | | | | the iterable *t* | | +------------------------------+--------------------------------+---------------------+ | ``del s[i:j]`` | removes the elements of | | | | ``s[i:j]`` from the list | | | | (same as ``s[i:j] = []``) | | +------------------------------+--------------------------------+---------------------+ | ``s[i:j:k] = t`` | the elements of ``s[i:j:k]`` | \(1) | | | are replaced by those of *t* | | +------------------------------+--------------------------------+---------------------+ | ``del s[i:j:k]`` | removes the elements of | | | | ``s[i:j:k]`` from the list | | +------------------------------+--------------------------------+---------------------+ | ``s += t`` | extends *s* with the | | | | contents of *t* (for the | | | | most part the same as | | | | ``s[len(s):len(s)] = t``) | | +------------------------------+--------------------------------+---------------------+ | ``s *= n`` | updates *s* with its contents | \(2) | | | repeated *n* times | | +------------------------------+--------------------------------+---------------------+ Notes: (1) If *k* is not equal to ``1``, *t* must have the same length as the slice it is replacing. (2) The value *n* is an integer, or an object implementing :meth:`~object.__index__`. Zero and negative values of *n* clear the sequence. Items in the sequence are not copied; they are referenced multiple times, as explained for ``s * n`` under :ref:`typesseq-common`. .. rubric:: Mutable Sequence Methods Mutable sequence types also support the following methods: .. method:: bytearray.append(value, /) list.append(value, /) :no-contents-entry: :no-index-entry: :no-typesetting: .. method:: sequence.append(value, /) Append *value* to the end of the sequence. This is equivalent to writing ``seq[len(seq):len(seq)] = [value]``. .. method:: bytearray.clear() list.clear() :no-contents-entry: :no-index-entry: :no-typesetting: .. method:: sequence.clear() .. versionadded:: 3.3 Remove all items from *sequence*. This is equivalent to writing ``del sequence[:]``. .. method:: bytearray.copy() list.copy() :no-contents-entry: :no-index-entry: :no-typesetting: .. method:: sequence.copy() .. versionadded:: 3.3 Create a shallow copy of *sequence*. This is equivalent to writing ``sequence[:]``. .. hint:: The :meth:`!copy` method is not part of the :class:`~collections.abc.MutableSequence` :class:`~abc.ABC`, but most concrete mutable sequence types provide it. .. method:: bytearray.extend(iterable, /) list.extend(iterable, /) :no-contents-entry: :no-index-entry: :no-typesetting: .. method:: sequence.extend(iterable, /) Extend *sequence* with the contents of *iterable*. For the most part, this is the same as writing ``seq[len(seq):len(seq)] = iterable``. .. method:: bytearray.insert(index, value, /) list.insert(index, value, /) :no-contents-entry: :no-index-entry: :no-typesetting: .. method:: sequence.insert(index, value, /) Insert *value* into *sequence* at the given *index*. This is equivalent to writing ``sequence[index:index] = [value]``. .. method:: bytearray.pop(index=-1, /) list.pop(index=-1, /) :no-contents-entry: :no-index-entry: :no-typesetting: .. method:: sequence.pop(index=-1, /) Retrieve the item at *index* and also remove it from *sequence*. By default, the last item in *sequence* is removed and returned. .. method:: bytearray.remove(value, /) list.remove(value, /) :no-contents-entry: :no-index-entry: :no-typesetting: .. method:: sequence.remove(value, /) Remove the first item from *sequence* where ``sequence[i] == value``. Raises :exc:`ValueError` if *value* is not found in *sequence*. .. method:: bytearray.reverse() list.reverse() :no-contents-entry: :no-index-entry: :no-typesetting: .. method:: sequence.reverse() Reverse the items of *sequence* in place. This method maintains economy of space when reversing a large sequence. To remind users that it operates by side-effect, it returns ``None``. .. _typesseq-list: Lists ----- .. index:: pair: object; list Lists are mutable sequences, typically used to store collections of homogeneous items (where the precise degree of similarity will vary by application). .. class:: list(iterable=(), /) Lists may be constructed in several ways: * Using a pair of square brackets to denote the empty list: ``[]`` * Using square brackets, separating items with commas: ``[a]``, ``[a, b, c]`` * Using a list comprehension: ``[x for x in iterable]`` * Using the type constructor: ``list()`` or ``list(iterable)`` The constructor builds a list whose items are the same and in the same order as *iterable*'s items. *iterable* may be either a sequence, a container that supports iteration, or an iterator object. If *iterable* is already a list, a copy is made and returned, similar to ``iterable[:]``. For example, ``list('abc')`` returns ``['a', 'b', 'c']`` and ``list( (1, 2, 3) )`` returns ``[1, 2, 3]``. If no argument is given, the constructor creates a new empty list, ``[]``. Many other operations also produce lists, including the :func:`sorted` built-in. Lists are :ref:`generic ` over the types of their items. Lists implement all of the :ref:`common ` and :ref:`mutable ` sequence operations. Lists also provide the following additional method: .. method:: list.sort(*, key=None, reverse=False) This method sorts the list in place, using only ``<`` comparisons between items. Exceptions are not suppressed - if any comparison operations fail, the entire sort operation will fail (and the list will likely be left in a partially modified state). :meth:`sort` accepts two arguments that can only be passed by keyword (:ref:`keyword-only arguments `): *key* specifies a function of one argument that is used to extract a comparison key from each list element (for example, ``key=str.lower``). The key corresponding to each item in the list is calculated once and then used for the entire sorting process. The default value of ``None`` means that list items are sorted directly without calculating a separate key value. The :func:`functools.cmp_to_key` utility is available to convert a 2.x style *cmp* function to a *key* function. *reverse* is a boolean value. If set to ``True``, then the list elements are sorted as if each comparison were reversed. This method modifies the sequence in place for economy of space when sorting a large sequence. To remind users that it operates by side effect, it does not return the sorted sequence (use :func:`sorted` to explicitly request a new sorted list instance). The :meth:`sort` method is guaranteed to be stable. A sort is stable if it guarantees not to change the relative order of elements that compare equal --- this is helpful for sorting in multiple passes (for example, sort by department, then by salary grade). For sorting examples and a brief sorting tutorial, see :ref:`sortinghowto`. .. impl-detail:: While a list is being sorted, the effect of attempting to mutate, or even inspect, the list is undefined. The C implementation of Python makes the list appear empty for the duration, and raises :exc:`ValueError` if it can detect that the list has been mutated during a sort. .. seealso:: For detailed information on thread-safety guarantees for :class:`list` objects, see :ref:`thread-safety-list`. .. _typesseq-tuple: Tuples ------ .. index:: pair: object; tuple Tuples are immutable sequences, typically used to store collections of heterogeneous data (such as the 2-tuples produced by the :func:`enumerate` built-in). Tuples are also used for cases where an immutable sequence of homogeneous data is needed (such as allowing storage in a :class:`set` or :class:`dict` instance). .. class:: tuple(iterable=(), /) Tuples may be constructed in a number of ways: * Using a pair of parentheses to denote the empty tuple: ``()`` * Using a trailing comma for a singleton tuple: ``a,`` or ``(a,)`` * Separating items with commas: ``a, b, c`` or ``(a, b, c)`` * Using the :func:`tuple` built-in: ``tuple()`` or ``tuple(iterable)`` The constructor builds a tuple whose items are the same and in the same order as *iterable*'s items. *iterable* may be either a sequence, a container that supports iteration, or an iterator object. If *iterable* is already a tuple, it is returned unchanged. For example, ``tuple('abc')`` returns ``('a', 'b', 'c')`` and ``tuple( [1, 2, 3] )`` returns ``(1, 2, 3)``. If no argument is given, the constructor creates a new empty tuple, ``()``. Note that it is actually the comma which makes a tuple, not the parentheses. The parentheses are optional, except in the empty tuple case, or when they are needed to avoid syntactic ambiguity. For example, ``f(a, b, c)`` is a function call with three arguments, while ``f((a, b, c))`` is a function call with a 3-tuple as the sole argument. Tuples implement all of the :ref:`common ` sequence operations. Tuples are :ref:`generic ` over the types of their contents. For more information, refer to :ref:`the typing documentation on annotating tuples `. For heterogeneous collections of data where access by name is clearer than access by index, :func:`collections.namedtuple` may be a more appropriate choice than a simple tuple object. .. _typesseq-range: Ranges ------ .. index:: pair: object; range The :class:`range` type represents an immutable sequence of numbers and is commonly used for looping a specific number of times in :keyword:`for` loops. .. class:: range(stop, /) range(start, stop, step=1, /) The arguments to the range constructor must be integers (either built-in :class:`int` or any object that implements the :meth:`~object.__index__` special method). If the *step* argument is omitted, it defaults to ``1``. If the *start* argument is omitted, it defaults to ``0``. If *step* is zero, :exc:`ValueError` is raised. For a positive *step*, the contents of a range ``r`` are determined by the formula ``r[i] = start + step*i`` where ``i >= 0`` and ``r[i] < stop``. For a negative *step*, the contents of the range are still determined by the formula ``r[i] = start + step*i``, but the constraints are ``i >= 0`` and ``r[i] > stop``. A range object will be empty if ``r[0]`` does not meet the value constraint. Ranges do support negative indices, but these are interpreted as indexing from the end of the sequence determined by the positive indices. Ranges containing absolute values larger than :data:`sys.maxsize` are permitted but some features (such as :func:`len`) may raise :exc:`OverflowError`. Range examples:: >>> list(range(10)) [0, 1, 2, 3, 4, 5, 6, 7, 8, 9] >>> list(range(1, 11)) [1, 2, 3, 4, 5, 6, 7, 8, 9, 10] >>> list(range(0, 30, 5)) [0, 5, 10, 15, 20, 25] >>> list(range(0, 10, 3)) [0, 3, 6, 9] >>> list(range(0, -10, -1)) [0, -1, -2, -3, -4, -5, -6, -7, -8, -9] >>> list(range(0)) [] >>> list(range(1, 0)) [] Ranges implement all of the :ref:`common ` sequence operations except concatenation and repetition (due to the fact that range objects can only represent sequences that follow a strict pattern and repetition and concatenation will usually violate that pattern). .. attribute:: start The value of the *start* parameter (or ``0`` if the parameter was not supplied) .. attribute:: stop The value of the *stop* parameter .. attribute:: step The value of the *step* parameter (or ``1`` if the parameter was not supplied) The advantage of the :class:`range` type over a regular :class:`list` or :class:`tuple` is that a :class:`range` object will always take the same (small) amount of memory, no matter the size of the range it represents (as it only stores the ``start``, ``stop`` and ``step`` values, calculating individual items and subranges as needed). Range objects implement the :class:`collections.abc.Sequence` ABC, and provide features such as containment tests, element index lookup, slicing and support for negative indices (see :ref:`typesseq`): >>> r = range(0, 20, 2) >>> r range(0, 20, 2) >>> 11 in r False >>> 10 in r True >>> r.index(10) 5 >>> r[5] 10 >>> r[:5] range(0, 10, 2) >>> r[-1] 18 Testing range objects for equality with ``==`` and ``!=`` compares them as sequences. That is, two range objects are considered equal if they represent the same sequence of values. (Note that two range objects that compare equal might have different :attr:`~range.start`, :attr:`~range.stop` and :attr:`~range.step` attributes, for example ``range(0) == range(2, 1, 3)`` or ``range(0, 3, 2) == range(0, 4, 2)``.) .. versionchanged:: 3.2 Implement the Sequence ABC. Support slicing and negative indices. Test :class:`int` objects for membership in constant time instead of iterating through all items. .. versionchanged:: 3.3 Define '==' and '!=' to compare range objects based on the sequence of values they define (instead of comparing based on object identity). Added the :attr:`~range.start`, :attr:`~range.stop` and :attr:`~range.step` attributes. .. seealso:: * The `linspace recipe `_ shows how to implement a lazy version of range suitable for floating-point applications. .. index:: single: string; text sequence type single: str (built-in class); (see also string) pair: object; string .. _text-methods-summary: Text and Binary Sequence Type Methods Summary ============================================= The following table summarizes the text and binary sequence types methods by category. +--------------------------+-------------------------------------------+---------------------------------------------------+ | Category | :class:`str` methods | :class:`bytes` and :class:`bytearray` methods | +==========================+===========================================+===================================================+ | Formatting | :meth:`str.format` | | | +-------------------------------------------+---------------------------------------------------+ | | :meth:`str.format_map` | | | +-------------------------------------------+---------------------------------------------------+ | | :ref:`f-strings` | | | +-------------------------------------------+---------------------------------------------------+ | | :ref:`old-string-formatting` | :ref:`bytes-formatting` | +--------------------------+------------------+------------------------+--------------------+------------------------------+ | Searching and Replacing | :meth:`str.find` | :meth:`str.rfind` | :meth:`bytes.find` | :meth:`bytes.rfind` | | +------------------+------------------------+--------------------+------------------------------+ | | :meth:`str.index`| :meth:`str.rindex` | :meth:`bytes.index`| :meth:`bytes.rindex` | | +------------------+------------------------+--------------------+------------------------------+ | | :meth:`str.startswith` | :meth:`bytes.startswith` | | +-------------------------------------------+---------------------------------------------------+ | | :meth:`str.endswith` | :meth:`bytes.endswith` | | +-------------------------------------------+---------------------------------------------------+ | | :meth:`str.count` | :meth:`bytes.count` | | +-------------------------------------------+---------------------------------------------------+ | | :meth:`str.replace` | :meth:`bytes.replace` | +--------------------------+-------------------+-----------------------+---------------------+-----------------------------+ | Splitting and Joining | :meth:`str.split` | :meth:`str.rsplit` | :meth:`bytes.split` | :meth:`bytes.rsplit` | | +-------------------+-----------------------+---------------------+-----------------------------+ | | :meth:`str.splitlines` | :meth:`bytes.splitlines` | | +-------------------------------------------+---------------------------------------------------+ | | :meth:`str.partition` | :meth:`bytes.partition` | | +-------------------------------------------+---------------------------------------------------+ | | :meth:`str.rpartition` | :meth:`bytes.rpartition` | | +-------------------------------------------+---------------------------------------------------+ | | :meth:`str.join` | :meth:`bytes.join` | +--------------------------+-------------------------------------------+---------------------------------------------------+ | String Classification | :meth:`str.isalpha` | :meth:`bytes.isalpha` | | +-------------------------------------------+---------------------------------------------------+ | | :meth:`str.isdecimal` | | | +-------------------------------------------+---------------------------------------------------+ | | :meth:`str.isdigit` | :meth:`bytes.isdigit` | | +-------------------------------------------+---------------------------------------------------+ | | :meth:`str.isnumeric` | | | +-------------------------------------------+---------------------------------------------------+ | | :meth:`str.isalnum` | :meth:`bytes.isalnum` | | +-------------------------------------------+---------------------------------------------------+ | | :meth:`str.isidentifier` | | | +-------------------------------------------+---------------------------------------------------+ | | :meth:`str.islower` | :meth:`bytes.islower` | | +-------------------------------------------+---------------------------------------------------+ | | :meth:`str.isupper` | :meth:`bytes.isupper` | | +-------------------------------------------+---------------------------------------------------+ | | :meth:`str.istitle` | :meth:`bytes.istitle` | | +-------------------------------------------+---------------------------------------------------+ | | :meth:`str.isspace` | :meth:`bytes.isspace` | | +-------------------------------------------+---------------------------------------------------+ | | :meth:`str.isprintable` | | +--------------------------+-------------------------------------------+---------------------------------------------------+ | Case Manipulation | :meth:`str.lower` | :meth:`bytes.lower` | | +-------------------------------------------+---------------------------------------------------+ | | :meth:`str.upper` | :meth:`bytes.upper` | | +-------------------------------------------+---------------------------------------------------+ | | :meth:`str.casefold` | | | +-------------------------------------------+---------------------------------------------------+ | | :meth:`str.capitalize` | :meth:`bytes.capitalize` | | +-------------------------------------------+---------------------------------------------------+ | | :meth:`str.title` | :meth:`bytes.title` | | +-------------------------------------------+---------------------------------------------------+ | | :meth:`str.swapcase` | :meth:`bytes.swapcase` | +--------------------------+-------------------+-----------------------+---------------------+-----------------------------+ | Padding and Stripping | :meth:`str.ljust` | :meth:`str.rjust` | :meth:`bytes.ljust` | :meth:`bytes.rjust` | | +-------------------+-----------------------+---------------------+-----------------------------+ | | :meth:`str.center` | :meth:`bytes.center` | | +-------------------------------------------+---------------------------------------------------+ | | :meth:`str.expandtabs` | :meth:`bytes.expandtabs` | | +-------------------------------------------+---------------------------------------------------+ | | :meth:`str.strip` | :meth:`bytes.strip` | | +--------------------+----------------------+----------------------+----------------------------+ | | :meth:`str.lstrip` | :meth:`str.rstrip` | :meth:`bytes.lstrip` | :meth:`bytes.rstrip` | | +--------------------+----------------------+----------------------+----------------------------+ | | :meth:`str.removeprefix` | :meth:`bytes.removeprefix` | | +-------------------------------------------+---------------------------------------------------+ | | :meth:`str.removesuffix` | :meth:`bytes.removesuffix` | +--------------------------+-------------------------------------------+---------------------------------------------------+ | Translation and Encoding | :meth:`str.translate` | :meth:`bytes.translate` | | +-------------------------------------------+---------------------------------------------------+ | | :meth:`str.maketrans` | :meth:`bytes.maketrans` | | +-------------------------------------------+---------------------------------------------------+ | | :meth:`str.encode` | | | +-------------------------------------------+---------------------------------------------------+ | | | :meth:`bytes.decode` | +--------------------------+-------------------------------------------+---------------------------------------------------+ .. _textseq: Text Sequence Type --- :class:`str` =================================== Textual data in Python is handled with :class:`str` objects, or :dfn:`strings`. Strings are immutable :ref:`sequences ` of Unicode code points. String literals are written in a variety of ways: * Single quotes: ``'allows embedded "double" quotes'`` * Double quotes: ``"allows embedded 'single' quotes"`` * Triple quoted: ``'''Three single quotes'''``, ``"""Three double quotes"""`` Triple quoted strings may span multiple lines - all associated whitespace will be included in the string literal. String literals that are part of a single expression and have only whitespace between them will be implicitly converted to a single string literal. That is, ``("spam " "eggs") == "spam eggs"``. See :ref:`strings` for more about the various forms of string literal, including supported :ref:`escape sequences `, and the ``r`` ("raw") prefix that disables most escape sequence processing. Strings may also be created from other objects using the :class:`str` constructor. Since there is no separate "character" type, indexing a string produces strings of length 1. That is, for a non-empty string *s*, ``s[0] == s[0:1]``. .. index:: pair: object; io.StringIO There is also no mutable string type, but :meth:`str.join` or :class:`io.StringIO` can be used to efficiently construct strings from multiple fragments. .. versionchanged:: 3.3 For backwards compatibility with the Python 2 series, the ``u`` prefix is once again permitted on string literals. It has no effect on the meaning of string literals and cannot be combined with the ``r`` prefix. .. index:: single: string; str (built-in class) .. class:: str(*, encoding='utf-8', errors='strict') str(object) str(object, encoding, errors='strict') str(object, *, errors) Return a :ref:`string ` version of *object*. If *object* is not provided, returns the empty string. Otherwise, the behavior of ``str()`` depends on whether *encoding* or *errors* is given, as follows. If neither *encoding* nor *errors* is given, ``str(object)`` returns :meth:`type(object).__str__(object) `, which is the "informal" or nicely printable string representation of *object*. For string objects, this is the string itself. If *object* does not have a :meth:`~object.__str__` method, then :func:`str` falls back to returning :func:`repr(object) `. .. index:: single: buffer protocol; str (built-in class) single: bytes; str (built-in class) If at least one of *encoding* or *errors* is given, *object* should be a :term:`bytes-like object` (e.g. :class:`bytes` or :class:`bytearray`). In this case, if *object* is a :class:`bytes` (or :class:`bytearray`) object, then ``str(bytes, encoding, errors)`` is equivalent to :meth:`bytes.decode(encoding, errors) `. Otherwise, the bytes object underlying the buffer object is obtained before calling :meth:`bytes.decode`. See :ref:`binaryseq` and :ref:`bufferobjects` for information on buffer objects. Passing a :class:`bytes` object to :func:`str` without the *encoding* or *errors* arguments falls under the first case of returning the informal string representation (see also the :option:`-b` command-line option to Python). For example:: >>> str(b'Zoot!') "b'Zoot!'" For more information on the ``str`` class and its methods, see :ref:`textseq` and the :ref:`string-methods` section below. To output formatted strings, see the :ref:`f-strings` and :ref:`formatstrings` sections. In addition, see the :ref:`stringservices` section. .. index:: pair: string; methods .. _string-methods: String Methods -------------- .. index:: pair: module; re Strings implement all of the :ref:`common ` sequence operations, along with the additional methods described below. Strings also support two styles of string formatting, one providing a large degree of flexibility and customization (see :meth:`str.format`, :ref:`formatstrings` and :ref:`string-formatting`) and the other based on C ``printf`` style formatting that handles a narrower range of types and is slightly harder to use correctly, but is often faster for the cases it can handle (:ref:`old-string-formatting`). The :ref:`textservices` section of the standard library covers a number of other modules that provide various text related utilities (including regular expression support in the :mod:`re` module). .. method:: str.capitalize() Return a copy of the string with its first character capitalized and the rest lowercased. .. versionchanged:: 3.8 The first character is now put into titlecase rather than uppercase. This means that characters like digraphs will only have their first letter capitalized, instead of the full character. .. method:: str.casefold() Return a casefolded copy of the string. Casefolded strings may be used for caseless matching. Casefolding is similar to lowercasing but more aggressive because it is intended to remove all case distinctions in a string. For example, the German lowercase letter ``'ß'`` is equivalent to ``"ss"``. Since it is already lowercase, :meth:`lower` would do nothing to ``'ß'``; :meth:`casefold` converts it to ``"ss"``. For example: .. doctest:: >>> 'straße'.lower() 'straße' >>> 'straße'.casefold() 'strasse' The casefolding algorithm is `described in section 3.13.3 'Default Case Folding' of the Unicode Standard `__. .. versionadded:: 3.3 .. method:: str.center(width, fillchar=' ', /) Return centered in a string of length *width*. Padding is done using the specified *fillchar* (default is an ASCII space). The original string is returned if *width* is less than or equal to ``len(s)``. For example:: >>> 'Python'.center(10) ' Python ' >>> 'Python'.center(10, '-') '--Python--' >>> 'Python'.center(4) 'Python' .. method:: str.count(sub[, start[, end]]) Return the number of non-overlapping occurrences of substring *sub* in the range [*start*, *end*]. Optional arguments *start* and *end* are interpreted as in slice notation. If *sub* is empty, returns the number of empty strings between characters which is the length of the string plus one. For example:: >>> 'spam, spam, spam'.count('spam') 3 >>> 'spam, spam, spam'.count('spam', 5) 2 >>> 'spam, spam, spam'.count('spam', 5, 10) 1 >>> 'spam, spam, spam'.count('eggs') 0 >>> 'spam, spam, spam'.count('') 17 .. method:: str.encode(encoding="utf-8", errors="strict") Return the string encoded to :class:`bytes`. *encoding* defaults to ``'utf-8'``; see :ref:`standard-encodings` for possible values. *errors* controls how encoding errors are handled. If ``'strict'`` (the default), a :exc:`UnicodeError` exception is raised. Other possible values are ``'ignore'``, ``'replace'``, ``'xmlcharrefreplace'``, ``'backslashreplace'`` and any other name registered via :func:`codecs.register_error`. See :ref:`error-handlers` for details. For performance reasons, the value of *errors* is not checked for validity unless an encoding error actually occurs, :ref:`devmode` is enabled or a :ref:`debug build ` is used. For example:: >>> encoded_str_to_bytes = 'Python'.encode() >>> type(encoded_str_to_bytes) >>> encoded_str_to_bytes b'Python' .. versionchanged:: 3.1 Added support for keyword arguments. .. versionchanged:: 3.9 The value of the *errors* argument is now checked in :ref:`devmode` and in :ref:`debug mode `. .. method:: str.endswith(suffix[, start[, end]]) Return ``True`` if the string ends with the specified *suffix*, otherwise return ``False``. *suffix* can also be a tuple of suffixes to look for. With optional *start*, test beginning at that position. With optional *end*, stop comparing at that position. Using *start* and *end* is equivalent to ``str[start:end].endswith(suffix)``. For example:: >>> 'Python'.endswith('on') True >>> 'a tuple of suffixes'.endswith(('at', 'in')) False >>> 'a tuple of suffixes'.endswith(('at', 'es')) True >>> 'Python is amazing'.endswith('is', 0, 9) True See also :meth:`startswith` and :meth:`removesuffix`. .. method:: str.expandtabs(tabsize=8) Return a copy of the string where all tab characters are replaced by one or more spaces, depending on the current column and the given tab size. Tab positions occur every *tabsize* characters (default is 8, giving tab positions at columns 0, 8, 16 and so on). To expand the string, the current column is set to zero and the string is examined character by character. If the character is a tab (``\t``), one or more space characters are inserted in the result until the current column is equal to the next tab position. (The tab character itself is not copied.) If the character is a newline (``\n``) or return (``\r``), it is copied and the current column is reset to zero. Any other character is copied unchanged and the current column is incremented by one regardless of how the character is represented when printed. For example:: >>> '01\t012\t0123\t01234'.expandtabs() '01 012 0123 01234' >>> '01\t012\t0123\t01234'.expandtabs(4) '01 012 0123 01234' >>> print('01\t012\n0123\t01234'.expandtabs(4)) 01 012 0123 01234 .. method:: str.find(sub[, start[, end]]) Return the lowest index in the string where substring *sub* is found within the slice ``s[start:end]``. Optional arguments *start* and *end* are interpreted as in slice notation. Return ``-1`` if *sub* is not found. For example:: >>> 'spam, spam, spam'.find('sp') 0 >>> 'spam, spam, spam'.find('sp', 5) 6 See also :meth:`rfind` and :meth:`index`. .. note:: The :meth:`~str.find` method should be used only if you need to know the position of *sub*. To check if *sub* is a substring or not, use the :keyword:`in` operator:: >>> 'Py' in 'Python' True .. method:: str.format(*args, **kwargs) Perform a string formatting operation. The string on which this method is called can contain literal text or replacement fields delimited by braces ``{}``. Each replacement field contains either the numeric index of a positional argument, or the name of a keyword argument. Returns a copy of the string where each replacement field is replaced with the string value of the corresponding argument. For example: .. doctest:: >>> "The sum of 1 + 2 is {0}".format(1+2) 'The sum of 1 + 2 is 3' >>> "The sum of {a} + {b} is {answer}".format(answer=1+2, a=1, b=2) 'The sum of 1 + 2 is 3' >>> "{1} expects the {0} Inquisition!".format("Spanish", "Nobody") 'Nobody expects the Spanish Inquisition!' See :ref:`formatstrings` for a description of the various formatting options that can be specified in format strings. .. note:: When formatting a number (:class:`int`, :class:`float`, :class:`complex`, :class:`decimal.Decimal` and subclasses) with the ``n`` type (ex: ``'{:n}'.format(1234)``), the function temporarily sets the ``LC_CTYPE`` locale to the ``LC_NUMERIC`` locale to decode ``decimal_point`` and ``thousands_sep`` fields of :c:func:`localeconv` if they are non-ASCII or longer than 1 byte, and the ``LC_NUMERIC`` locale is different than the ``LC_CTYPE`` locale. This temporary change affects other threads. .. versionchanged:: 3.7 When formatting a number with the ``n`` type, the function sets temporarily the ``LC_CTYPE`` locale to the ``LC_NUMERIC`` locale in some cases. .. method:: str.format_map(mapping, /) Similar to ``str.format(**mapping)``, except that ``mapping`` is used directly and not copied to a :class:`dict`. This is useful if for example ``mapping`` is a dict subclass: >>> class Default(dict): ... def __missing__(self, key): ... return key ... >>> '{name} was born in {country}'.format_map(Default(name='Guido')) 'Guido was born in country' .. versionadded:: 3.2 .. method:: str.index(sub[, start[, end]]) Like :meth:`~str.find`, but raise :exc:`ValueError` when the substring is not found. For example: .. doctest:: >>> 'spam, spam, spam'.index('spam') 0 >>> 'spam, spam, spam'.index('eggs') Traceback (most recent call last): File "", line 1, in 'spam, spam, spam'.index('eggs') ~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ ValueError: substring not found See also :meth:`rindex`. .. method:: str.isalnum() Return ``True`` if all characters in the string are alphanumeric and there is at least one character, ``False`` otherwise. A character ``c`` is alphanumeric if one of the following returns ``True``: ``c.isalpha()``, ``c.isdecimal()``, ``c.isdigit()``, or ``c.isnumeric()``. For example: .. doctest:: >>> 'abc123'.isalnum() True >>> 'abc123!@#'.isalnum() False >>> ''.isalnum() False >>> ' '.isalnum() False .. method:: str.isalpha() Return ``True`` if all characters in the string are alphabetic and there is at least one character, ``False`` otherwise. Alphabetic characters are those characters defined in the Unicode character database as "Letter", i.e., those with general category property being one of "Lm", "Lt", "Lu", "Ll", or "Lo". Note that this is different from the `Alphabetic property defined in section 4.10 'Letters, Alphabetic, and Ideographic' of the Unicode Standard `__. For example: .. doctest:: >>> 'Letters and spaces'.isalpha() False >>> 'LettersOnly'.isalpha() True >>> 'µ'.isalpha() # non-ASCII characters can be considered alphabetical too True See :ref:`unicode-properties`. .. method:: str.isascii() Return ``True`` if the string is empty or all characters in the string are ASCII, ``False`` otherwise. ASCII characters have code points in the range U+0000-U+007F. For example: .. doctest:: >>> 'ASCII characters'.isascii() True >>> 'µ'.isascii() False .. versionadded:: 3.7 .. method:: str.isdecimal() Return ``True`` if all characters in the string are decimal characters and there is at least one character, ``False`` otherwise. Decimal characters are those that can be used to form numbers in base 10, such as U+0660, ARABIC-INDIC DIGIT ZERO. Formally a decimal character is a character in the Unicode General Category "Nd". For example: .. doctest:: >>> '0123456789'.isdecimal() True >>> '٠١٢٣٤٥٦٧٨٩'.isdecimal() # Arabic-Indic digits zero to nine True >>> 'alphabetic'.isdecimal() False .. method:: str.isdigit() Return ``True`` if all characters in the string are digits and there is at least one character, ``False`` otherwise. Digits include decimal characters and digits that need special handling, such as the compatibility superscript digits. This covers digits which cannot be used to form numbers in base 10, like the `Kharosthi numbers `__. Formally, a digit is a character that has the property value Numeric_Type=Digit or Numeric_Type=Decimal. For example: .. doctest:: >>> '0123456789'.isdigit() True >>> '٠١٢٣٤٥٦٧٨٩'.isdigit() # Arabic-Indic digits zero to nine True >>> '⅕'.isdigit() # Vulgar fraction one fifth False >>> '²'.isdecimal(), '²'.isdigit(), '²'.isnumeric() (False, True, True) See also :meth:`isdecimal` and :meth:`isnumeric`. .. method:: str.isidentifier() Return ``True`` if the string is a valid identifier according to the language definition, section :ref:`identifiers`. :func:`keyword.iskeyword` can be used to test whether string ``s`` is a reserved identifier, such as :keyword:`def` and :keyword:`class`. Example: :: >>> from keyword import iskeyword >>> 'hello'.isidentifier(), iskeyword('hello') (True, False) >>> 'def'.isidentifier(), iskeyword('def') (True, True) .. method:: str.islower() Return ``True`` if all cased characters [4]_ in the string are lowercase and there is at least one cased character, ``False`` otherwise. .. method:: str.isnumeric() Return ``True`` if all characters in the string are numeric characters, and there is at least one character, ``False`` otherwise. Numeric characters include digit characters, and all characters that have the Unicode numeric value property, e.g. U+2155, VULGAR FRACTION ONE FIFTH. Formally, numeric characters are those with the property value Numeric_Type=Digit, Numeric_Type=Decimal or Numeric_Type=Numeric. For example: .. doctest:: >>> '0123456789'.isnumeric() True >>> '٠١٢٣٤٥٦٧٨٩'.isnumeric() # Arabic-Indic digits zero to nine True >>> '⅕'.isnumeric() # Vulgar fraction one fifth True >>> '²'.isdecimal(), '²'.isdigit(), '²'.isnumeric() (False, True, True) See also :meth:`isdecimal` and :meth:`isdigit`. .. method:: str.isprintable() Return ``True`` if all characters in the string are printable, ``False`` if it contains at least one non-printable character. Here "printable" means the character is suitable for :func:`repr` to use in its output; "non-printable" means that :func:`repr` on built-in types will hex-escape the character. It has no bearing on the handling of strings written to :data:`sys.stdout` or :data:`sys.stderr`. The printable characters are those which in the Unicode character database (see :mod:`unicodedata`) have a general category in group Letter, Mark, Number, Punctuation, or Symbol (L, M, N, P, or S); plus the ASCII space 0x20. Nonprintable characters are those in group Separator or Other (Z or C), except the ASCII space. For example: .. doctest:: >>> ''.isprintable(), ' '.isprintable() (True, True) >>> '\t'.isprintable(), '\n'.isprintable() (False, False) See also :meth:`isspace`. .. method:: str.isspace() Return ``True`` if there are only whitespace characters in the string and there is at least one character, ``False`` otherwise. For example: .. doctest:: >>> ''.isspace() False >>> ' '.isspace() True >>> '\t\n'.isspace() # TAB and BREAK LINE True >>> '\u3000'.isspace() # IDEOGRAPHIC SPACE True A character is *whitespace* if in the Unicode character database (see :mod:`unicodedata`), either its general category is ``Zs`` ("Separator, space"), or its bidirectional class is one of ``WS``, ``B``, or ``S``. See also :meth:`isprintable`. .. method:: str.istitle() Return ``True`` if the string is a titlecased string and there is at least one character, for example uppercase characters may only follow uncased characters and lowercase characters only cased ones. Return ``False`` otherwise. For example: .. doctest:: >>> 'Spam, Spam, Spam'.istitle() True >>> 'spam, spam, spam'.istitle() False >>> 'SPAM, SPAM, SPAM'.istitle() False See also :meth:`title`. .. method:: str.isupper() Return ``True`` if all cased characters [4]_ in the string are uppercase and there is at least one cased character, ``False`` otherwise. >>> 'BANANA'.isupper() True >>> 'banana'.isupper() False >>> 'baNana'.isupper() False >>> ' '.isupper() False .. _meth-str-join: .. method:: str.join(iterable, /) Return a string which is the concatenation of the strings in *iterable*. A :exc:`TypeError` will be raised if there are any non-string values in *iterable*, including :class:`bytes` objects. The separator between elements is the string providing this method. For example: .. doctest:: >>> ', '.join(['spam', 'spam', 'spam']) 'spam, spam, spam' >>> '-'.join('Python') 'P-y-t-h-o-n' See also :meth:`split`. .. method:: str.ljust(width, fillchar=' ', /) Return the string left justified in a string of length *width*. Padding is done using the specified *fillchar* (default is an ASCII space). The original string is returned if *width* is less than or equal to ``len(s)``. For example: .. doctest:: >>> 'Python'.ljust(10) 'Python ' >>> 'Python'.ljust(10, '.') 'Python....' >>> 'Monty Python'.ljust(10, '.') 'Monty Python' See also :meth:`rjust`. .. method:: str.lower() Return a copy of the string with all the cased characters [4]_ converted to lowercase. For example: .. doctest:: >>> 'Lower Method Example'.lower() 'lower method example' The lowercasing algorithm used is `described in section 3.13.2 'Default Case Conversion' of the Unicode Standard `__. .. method:: str.lstrip(chars=None, /) Return a copy of the string with leading characters removed. The *chars* argument is a string specifying the set of characters to be removed. If omitted or ``None``, the *chars* argument defaults to removing whitespace, that is characters for which :meth:`str.isspace` is true. The *chars* argument is not a prefix; rather, all combinations of its values are stripped:: >>> ' spacious '.lstrip() 'spacious ' >>> 'www.example.com'.lstrip('cmowz.') 'example.com' See :meth:`str.removeprefix` for a method that will remove a single prefix string rather than all of a set of characters. For example:: >>> 'Arthur: three!'.lstrip('Arthur: ') 'ee!' >>> 'Arthur: three!'.removeprefix('Arthur: ') 'three!' .. staticmethod:: str.maketrans(dict, /) str.maketrans(from, to, remove='', /) This static method returns a translation table usable for :meth:`str.translate`. If there is only one argument, it must be a dictionary mapping Unicode ordinals (integers) or characters (strings of length 1) to Unicode ordinals, strings (of arbitrary lengths) or ``None``. Character keys will then be converted to ordinals. If there are two arguments, they must be strings of equal length, and in the resulting dictionary, each character in *from* will be mapped to the character at the same position in *to*. If there is a third argument, it must be a string, whose characters will be mapped to ``None`` in the result. .. versionchanged:: 3.15 *dict* can now be a :class:`frozendict`. .. method:: str.partition(sep, /) Split the string at the first occurrence of *sep*, and return a 3-tuple containing the part before the separator, the separator itself, and the part after the separator. If the separator is not found, return a 3-tuple containing the string itself, followed by two empty strings. For example: .. doctest:: >>> 'Monty Python'.partition(' ') ('Monty', ' ', 'Python') >>> "Monty Python's Flying Circus".partition(' ') ('Monty', ' ', "Python's Flying Circus") >>> 'Monty Python'.partition('-') ('Monty Python', '', '') See also :meth:`rpartition`. .. method:: str.removeprefix(prefix, /) If the string starts with the *prefix* string, return ``string[len(prefix):]``. Otherwise, return a copy of the original string: .. doctest:: >>> 'TestHook'.removeprefix('Test') 'Hook' >>> 'BaseTestCase'.removeprefix('Test') 'BaseTestCase' .. versionadded:: 3.9 See also :meth:`removesuffix` and :meth:`startswith`. .. method:: str.removesuffix(suffix, /) If the string ends with the *suffix* string and that *suffix* is not empty, return ``string[:-len(suffix)]``. Otherwise, return a copy of the original string: .. doctest:: >>> 'MiscTests'.removesuffix('Tests') 'Misc' >>> 'TmpDirMixin'.removesuffix('Tests') 'TmpDirMixin' .. versionadded:: 3.9 See also :meth:`removeprefix` and :meth:`endswith`. .. method:: str.replace(old, new, /, count=-1) Return a copy of the string with all occurrences of substring *old* replaced by *new*. If *count* is given, only the first *count* occurrences are replaced. If *count* is not specified or ``-1``, then all occurrences are replaced. For example: .. doctest:: >>> 'spam, spam, spam'.replace('spam', 'eggs') 'eggs, eggs, eggs' >>> 'spam, spam, spam'.replace('spam', 'eggs', 1) 'eggs, spam, spam' .. versionchanged:: 3.13 *count* is now supported as a keyword argument. .. method:: str.rfind(sub[, start[, end]]) Return the highest index in the string where substring *sub* is found, such that *sub* is contained within ``s[start:end]``. Optional arguments *start* and *end* are interpreted as in slice notation. Return ``-1`` on failure. For example: .. doctest:: >>> 'spam, spam, spam'.rfind('sp') 12 >>> 'spam, spam, spam'.rfind('sp', 0, 10) 6 See also :meth:`find` and :meth:`rindex`. .. method:: str.rindex(sub[, start[, end]]) Like :meth:`rfind` but raises :exc:`ValueError` when the substring *sub* is not found. For example: .. doctest:: >>> 'spam, spam, spam'.rindex('spam') 12 >>> 'spam, spam, spam'.rindex('eggs') Traceback (most recent call last): File "", line 1, in 'spam, spam, spam'.rindex('eggs') ~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ ValueError: substring not found See also :meth:`index` and :meth:`find`. .. method:: str.rjust(width, fillchar=' ', /) Return the string right justified in a string of length *width*. Padding is done using the specified *fillchar* (default is an ASCII space). The original string is returned if *width* is less than or equal to ``len(s)``. For example: .. doctest:: >>> 'Python'.rjust(10) ' Python' >>> 'Python'.rjust(10, '.') '....Python' >>> 'Monty Python'.rjust(10, '.') 'Monty Python' See also :meth:`ljust` and :meth:`zfill`. .. method:: str.rpartition(sep, /) Split the string at the last occurrence of *sep*, and return a 3-tuple containing the part before the separator, the separator itself, and the part after the separator. If the separator is not found, return a 3-tuple containing two empty strings, followed by the string itself. For example: .. doctest:: >>> 'Monty Python'.rpartition(' ') ('Monty', ' ', 'Python') >>> "Monty Python's Flying Circus".rpartition(' ') ("Monty Python's Flying", ' ', 'Circus') >>> 'Monty Python'.rpartition('-') ('', '', 'Monty Python') See also :meth:`partition`. .. method:: str.rsplit(sep=None, maxsplit=-1) Return a list of the words in the string, using *sep* as the delimiter string. If *maxsplit* is given, at most *maxsplit* splits are done, the *rightmost* ones. If *sep* is not specified or ``None``, any :meth:`whitespace ` string is a separator. Except for splitting from the right, :meth:`rsplit` behaves like :meth:`split` which is described in detail below. .. method:: str.rstrip(chars=None, /) Return a copy of the string with trailing characters removed. The *chars* argument is a string specifying the set of characters to be removed. If omitted or ``None``, the *chars* argument defaults to removing whitespace, that is characters for which :meth:`str.isspace` is true. The *chars* argument is not a suffix; rather, all combinations of its values are stripped. For example: .. doctest:: >>> ' spacious '.rstrip() ' spacious' >>> 'mississippi'.rstrip('ipz') 'mississ' See :meth:`removesuffix` for a method that will remove a single suffix string rather than all of a set of characters. For example:: >>> 'Monty Python'.rstrip(' Python') 'M' >>> 'Monty Python'.removesuffix(' Python') 'Monty' See also :meth:`strip`. .. method:: str.split(sep=None, maxsplit=-1) Return a list of the words in the string, using *sep* as the delimiter string. If *maxsplit* is given, at most *maxsplit* splits are done (thus, the list will have at most ``maxsplit+1`` elements). If *maxsplit* is not specified or ``-1``, then there is no limit on the number of splits (all possible splits are made). If *sep* is given, consecutive delimiters are not grouped together and are deemed to delimit empty strings (for example, ``'1,,2'.split(',')`` returns ``['1', '', '2']``). The *sep* argument may consist of multiple characters as a single delimiter (to split with multiple delimiters, use :func:`re.split`). Splitting an empty string with a specified separator returns ``['']``. For example: .. doctest:: >>> '1,2,3'.split(',') ['1', '2', '3'] >>> '1,2,3'.split(',', maxsplit=1) ['1', '2,3'] >>> '1,2,,3,'.split(',') ['1', '2', '', '3', ''] >>> '1<>2<>3<4'.split('<>') ['1', '2', '3<4'] If *sep* is not specified or is ``None``, a different splitting algorithm is applied: runs of consecutive :meth:`whitespace ` are regarded as a single separator, and the result will contain no empty strings at the start or end if the string has leading or trailing whitespace. Consequently, splitting an empty string or a string consisting of just whitespace with a ``None`` separator returns ``[]``. For example: .. doctest:: >>> '1 2 3'.split() ['1', '2', '3'] >>> '1 2 3'.split(maxsplit=1) ['1', '2 3'] >>> ' 1 2 3 '.split() ['1', '2', '3'] If *sep* is not specified or is ``None`` and *maxsplit* is ``0``, only leading runs of consecutive whitespace are considered. For example: .. doctest:: >>> "".split(None, 0) [] >>> " ".split(None, 0) [] >>> " foo ".split(maxsplit=0) ['foo '] See also :meth:`join` and :meth:`rsplit`. .. index:: single: universal newlines; str.splitlines method .. method:: str.splitlines(keepends=False) Return a list of the lines in the string, breaking at line boundaries. Line breaks are not included in the resulting list unless *keepends* is given and true. This method splits on the following line boundaries. In particular, the boundaries are a superset of :term:`universal newlines`. +-----------------------+-----------------------------+ | Representation | Description | +=======================+=============================+ | ``\n`` | Line Feed | +-----------------------+-----------------------------+ | ``\r`` | Carriage Return | +-----------------------+-----------------------------+ | ``\r\n`` | Carriage Return + Line Feed | +-----------------------+-----------------------------+ | ``\v`` or ``\x0b`` | Line Tabulation | +-----------------------+-----------------------------+ | ``\f`` or ``\x0c`` | Form Feed | +-----------------------+-----------------------------+ | ``\x1c`` | File Separator | +-----------------------+-----------------------------+ | ``\x1d`` | Group Separator | +-----------------------+-----------------------------+ | ``\x1e`` | Record Separator | +-----------------------+-----------------------------+ | ``\x85`` | Next Line (C1 Control Code) | +-----------------------+-----------------------------+ | ``\u2028`` | Line Separator | +-----------------------+-----------------------------+ | ``\u2029`` | Paragraph Separator | +-----------------------+-----------------------------+ .. versionchanged:: 3.2 ``\v`` and ``\f`` added to list of line boundaries. For example:: >>> 'ab c\n\nde fg\rkl\r\n'.splitlines() ['ab c', '', 'de fg', 'kl'] >>> 'ab c\n\nde fg\rkl\r\n'.splitlines(keepends=True) ['ab c\n', '\n', 'de fg\r', 'kl\r\n'] Unlike :meth:`~str.split` when a delimiter string *sep* is given, this method returns an empty list for the empty string, and a terminal line break does not result in an extra line:: >>> "".splitlines() [] >>> "One line\n".splitlines() ['One line'] For comparison, ``split('\n')`` gives:: >>> ''.split('\n') [''] >>> 'Two lines\n'.split('\n') ['Two lines', ''] .. method:: str.startswith(prefix[, start[, end]]) Return ``True`` if string starts with the *prefix*, otherwise return ``False``. *prefix* can also be a tuple of prefixes to look for. With optional *start*, test string beginning at that position. With optional *end*, stop comparing string at that position. For example: .. doctest:: >>> 'Python'.startswith('Py') True >>> 'a tuple of prefixes'.startswith(('at', 'a')) True >>> 'Python is amazing'.startswith('is', 7) True See also :meth:`endswith` and :meth:`removeprefix`. .. method:: str.strip(chars=None, /) Return a copy of the string with the leading and trailing characters removed. The *chars* argument is a string specifying the set of characters to be removed. If omitted or ``None``, the *chars* argument defaults to removing whitespace, that is characters for which :meth:`str.isspace` is true. The *chars* argument is not a prefix or suffix; rather, all combinations of its values are stripped. For example: .. doctest:: >>> ' spacious '.strip() 'spacious' >>> 'www.example.com'.strip('cmowz.') 'example' The outermost leading and trailing *chars* argument values are stripped from the string. Characters are removed from the leading end until reaching a string character that is not contained in the set of characters in *chars*. A similar action takes place on the trailing end. For example: .. doctest:: >>> comment_string = '#....... Section 3.2.1 Issue #32 .......' >>> comment_string.strip('.#! ') 'Section 3.2.1 Issue #32' See also :meth:`rstrip`. .. method:: str.swapcase() Return a copy of the string with uppercase characters converted to lowercase and vice versa. For example: .. doctest:: >>> 'Hello World'.swapcase() 'hELLO wORLD' Note that it is not necessarily true that ``s.swapcase().swapcase() == s``. For example: .. doctest:: >>> 'straße'.swapcase().swapcase() 'strasse' See also :meth:`str.lower` and :meth:`str.upper`. .. method:: str.title() Return a titlecased version of the string where words start with an uppercase character and the remaining characters are lowercase. For example:: >>> 'Hello world'.title() 'Hello World' The algorithm uses a simple language-independent definition of a word as groups of consecutive letters. The definition works in many contexts but it means that apostrophes in contractions and possessives form word boundaries, which may not be the desired result:: >>> "they're bill's friends from the UK".title() "They'Re Bill'S Friends From The Uk" The :func:`string.capwords` function does not have this problem, as it splits words on spaces only. Alternatively, a workaround for apostrophes can be constructed using regular expressions:: >>> import re >>> def titlecase(s): ... return re.sub(r"[A-Za-z]+('[A-Za-z]+)?", ... lambda mo: mo.group(0).capitalize(), ... s) ... >>> titlecase("they're bill's friends.") "They're Bill's Friends." See also :meth:`istitle`. .. method:: str.translate(table, /) Return a copy of the string in which each character has been mapped through the given translation table. The table must be an object that implements indexing via :meth:`~object.__getitem__`, typically a :term:`mapping` or :term:`sequence`. When indexed by a Unicode ordinal (an integer), the table object can do any of the following: return a Unicode ordinal or a string, to map the character to one or more other characters; return ``None``, to delete the character from the return string; or raise a :exc:`LookupError` exception, to map the character to itself. You can use :meth:`str.maketrans` to create a translation map from character-to-character mappings in different formats. The following example uses a mapping to replace ``'a'`` with ``'X'``, ``'b'`` with ``'Y'``, and delete ``'c'``: .. doctest:: >>> 'abc123'.translate({ord('a'): 'X', ord('b'): 'Y', ord('c'): None}) 'XY123' See also the :mod:`codecs` module for a more flexible approach to custom character mappings. .. method:: str.upper() Return a copy of the string with all the cased characters [4]_ converted to uppercase. Note that ``s.upper().isupper()`` might be ``False`` if ``s`` contains uncased characters or if the Unicode category of the resulting character(s) is not "Lu" (Letter, uppercase), but e.g. "Lt" (Letter, titlecase). The uppercasing algorithm used is `described in section 3.13.2 'Default Case Conversion' of the Unicode Standard `__. .. method:: str.zfill(width, /) Return a copy of the string left filled with ASCII ``'0'`` digits to make a string of length *width*. A leading sign prefix (``'+'``/``'-'``) is handled by inserting the padding *after* the sign character rather than before. The original string is returned if *width* is less than or equal to ``len(s)``. For example: .. doctest:: >>> "42".zfill(5) '00042' >>> "-42".zfill(5) '-0042' See also :meth:`rjust`. .. index:: single: ! formatted string literal single: formatted string literals single: ! f-string single: f-strings single: fstring single: interpolated string literal single: string; formatted literal single: string; interpolated literal single: {} (curly brackets); in formatted string literal single: ! (exclamation mark); in formatted string literal single: : (colon); in formatted string literal single: = (equals); for help in debugging using string literals .. _stdtypes-fstrings: Formatted String Literals (f-strings) ------------------------------------- .. versionadded:: 3.6 .. versionchanged:: 3.7 The :keyword:`await` and :keyword:`async for` can be used in expressions within f-strings. .. versionchanged:: 3.8 Added the debug specifier (``=``) .. versionchanged:: 3.12 Many restrictions on expressions within f-strings have been removed. Notably, nested strings, comments, and backslashes are now permitted. An :dfn:`f-string` (formally a :dfn:`formatted string literal`) is a string literal that is prefixed with ``f`` or ``F``. This type of string literal allows embedding the results of arbitrary Python expressions within *replacement fields*, which are delimited by curly brackets (``{}``). Each replacement field must contain an expression, optionally followed by: * a *debug specifier* -- an equal sign (``=``); * a *conversion specifier* -- ``!s``, ``!r`` or ``!a``; and/or * a *format specifier* prefixed with a colon (``:``). See the :ref:`Lexical Analysis section on f-strings ` for details on the syntax of these fields. Debug specifier ^^^^^^^^^^^^^^^ .. versionadded:: 3.8 If a debug specifier -- an equal sign (``=``) -- appears after the replacement field expression, the resulting f-string will contain the expression's source, the equal sign, and the value of the expression. This is often useful for debugging:: >>> number = 14.3 >>> f'{number=}' 'number=14.3' Whitespace before, inside and after the expression, as well as whitespace after the equal sign, is significant --- it is retained in the result:: >>> f'{ number - 4 = }' ' number - 4 = 10.3' Conversion specifier ^^^^^^^^^^^^^^^^^^^^ By default, the value of a replacement field expression is converted to a string using :func:`str`:: >>> from fractions import Fraction >>> one_third = Fraction(1, 3) >>> f'{one_third}' '1/3' When a debug specifier but no format specifier is used, the default conversion instead uses :func:`repr`:: >>> f'{one_third = }' 'one_third = Fraction(1, 3)' The conversion can be specified explicitly using one of these specifiers: * ``!s`` for :func:`str` * ``!r`` for :func:`repr` * ``!a`` for :func:`ascii` For example:: >>> str(one_third) '1/3' >>> repr(one_third) 'Fraction(1, 3)' >>> f'{one_third!s} is {one_third!r}' '1/3 is Fraction(1, 3)' >>> string = "¡kočka 😸!" >>> ascii(string) "'\\xa1ko\\u010dka \\U0001f638!'" >>> f'{string = !a}' "string = '\\xa1ko\\u010dka \\U0001f638!'" Format specifier ^^^^^^^^^^^^^^^^ After the expression has been evaluated, and possibly converted using an explicit conversion specifier, it is formatted using the :func:`format` function. If the replacement field includes a *format specifier* introduced by a colon (``:``), the specifier is passed to :func:`!format` as the second argument. The result of :func:`!format` is then used as the final value for the replacement field. For example:: >>> from fractions import Fraction >>> one_third = Fraction(1, 3) >>> f'{one_third:.6f}' '0.333333' >>> f'{one_third:_^+10}' '___+1/3___' >>> f'{one_third!r:_^20}' '___Fraction(1, 3)___' >>> f'{one_third = :~>10}~' 'one_third = ~~~~~~~1/3~' .. _stdtypes-tstrings: Template String Literals (t-strings) ------------------------------------ A :dfn:`t-string` (formally a :dfn:`template string literal`) is a string literal that is prefixed with ``t`` or ``T``. These strings follow the same syntax and evaluation rules as :ref:`formatted string literals `, with the following differences: * Rather than evaluating to a ``str`` object, template string literals evaluate to a :class:`string.templatelib.Template` object. * The :func:`format` protocol is not used. Instead, the format specifier and conversions (if any) are passed to a new :class:`~string.templatelib.Interpolation` object that is created for each evaluated expression. It is up to code that processes the resulting :class:`~string.templatelib.Template` object to decide how to handle format specifiers and conversions. * Format specifiers containing nested replacement fields are evaluated eagerly, prior to being passed to the :class:`~string.templatelib.Interpolation` object. For instance, an interpolation of the form ``{amount:.{precision}f}`` will evaluate the inner expression ``{precision}`` to determine the value of the ``format_spec`` attribute. If ``precision`` were to be ``2``, the resulting format specifier would be ``'.2f'``. * When the equals sign ``'='`` is provided in an interpolation expression, the text of the expression is appended to the literal string that precedes the relevant interpolation. This includes the equals sign and any surrounding whitespace. The :class:`!Interpolation` instance for the expression will be created as normal, except that :attr:`~string.templatelib.Interpolation.conversion` will be set to '``r``' (:func:`repr`) by default. If an explicit conversion or format specifier is provided, this will override the default behaviour. .. _old-string-formatting: ``printf``-style String Formatting ---------------------------------- .. index:: single: formatting, string (%) single: interpolation, string (%) single: string; formatting, printf single: string; interpolation, printf single: printf-style formatting single: sprintf-style formatting single: % (percent); printf-style formatting .. note:: The formatting operations described here exhibit a variety of quirks that lead to a number of common errors (such as failing to display tuples and dictionaries correctly). Using :ref:`formatted string literals `, the :meth:`str.format` interface, or :class:`string.Template` may help avoid these errors. Each of these alternatives provides their own trade-offs and benefits of simplicity, flexibility, and/or extensibility. String objects have one unique built-in operation: the ``%`` operator (modulo). This is also known as the string *formatting* or *interpolation* operator. Given ``format % values`` (where *format* is a string), ``%`` conversion specifications in *format* are replaced with zero or more elements of *values*. The effect is similar to using the :c:func:`sprintf` function in the C language. For example: .. doctest:: >>> print('%s has %d quote types.' % ('Python', 2)) Python has 2 quote types. If *format* requires a single argument, *values* may be a single non-tuple object. [5]_ Otherwise, *values* must be a tuple with exactly the number of items specified by the format string, or a single mapping object (for example, a dictionary). .. index:: single: () (parentheses); in printf-style formatting single: * (asterisk); in printf-style formatting single: . (dot); in printf-style formatting A conversion specifier contains two or more characters and has the following components, which must occur in this order: #. The ``'%'`` character, which marks the start of the specifier. #. Mapping key (optional), consisting of a parenthesised sequence of characters (for example, ``(somename)``). #. Conversion flags (optional), which affect the result of some conversion types. #. Minimum field width (optional). If specified as an ``'*'`` (asterisk), the actual width is read from the next element of the tuple in *values*, and the object to convert comes after the minimum field width and optional precision. #. Precision (optional), given as a ``'.'`` (dot) followed by the precision. If specified as ``'*'`` (an asterisk), the actual precision is read from the next element of the tuple in *values*, and the value to convert comes after the precision. #. Length modifier (optional). #. Conversion type. When the right argument is a dictionary (or other mapping type), then the formats in the string *must* include a parenthesised mapping key into that dictionary inserted immediately after the ``'%'`` character. The mapping key selects the value to be formatted from the mapping. For example: >>> print('%(language)s has %(number)03d quote types.' % ... {'language': "Python", "number": 2}) Python has 002 quote types. In this case no ``*`` specifiers may occur in a format (since they require a sequential parameter list). The conversion flag characters are: .. index:: single: # (hash); in printf-style formatting single: - (minus); in printf-style formatting single: + (plus); in printf-style formatting single: space; in printf-style formatting +---------+---------------------------------------------------------------------+ | Flag | Meaning | +=========+=====================================================================+ | ``'#'`` | The value conversion will use the "alternate form" (where defined | | | below). | +---------+---------------------------------------------------------------------+ | ``'0'`` | The conversion will be zero padded for numeric values. | +---------+---------------------------------------------------------------------+ | ``'-'`` | The converted value is left adjusted (overrides the ``'0'`` | | | conversion if both are given). | +---------+---------------------------------------------------------------------+ | ``' '`` | (a space) A blank should be left before a positive number (or empty | | | string) produced by a signed conversion. | +---------+---------------------------------------------------------------------+ | ``'+'`` | A sign character (``'+'`` or ``'-'``) will precede the conversion | | | (overrides a "space" flag). | +---------+---------------------------------------------------------------------+ A length modifier (``h``, ``l``, or ``L``) may be present, but is ignored as it is not necessary for Python -- so e.g. ``%ld`` is identical to ``%d``. The conversion types are: +------------+-----------------------------------------------------+-------+ | Conversion | Meaning | Notes | +============+=====================================================+=======+ | ``'d'`` | Signed integer decimal. | | +------------+-----------------------------------------------------+-------+ | ``'i'`` | Signed integer decimal. | | +------------+-----------------------------------------------------+-------+ | ``'o'`` | Signed octal value. | \(1) | +------------+-----------------------------------------------------+-------+ | ``'u'`` | Obsolete type -- it is identical to ``'d'``. | \(6) | +------------+-----------------------------------------------------+-------+ | ``'x'`` | Signed hexadecimal (lowercase). | \(2) | +------------+-----------------------------------------------------+-------+ | ``'X'`` | Signed hexadecimal (uppercase). | \(2) | +------------+-----------------------------------------------------+-------+ | ``'e'`` | Floating-point exponential format (lowercase). | \(3) | +------------+-----------------------------------------------------+-------+ | ``'E'`` | Floating-point exponential format (uppercase). | \(3) | +------------+-----------------------------------------------------+-------+ | ``'f'`` | Floating-point decimal format. | \(3) | +------------+-----------------------------------------------------+-------+ | ``'F'`` | Floating-point decimal format. | \(3) | +------------+-----------------------------------------------------+-------+ | ``'g'`` | Floating-point format. Uses lowercase exponential | \(4) | | | format if exponent is less than -4 or not less than | | | | precision, decimal format otherwise. | | +------------+-----------------------------------------------------+-------+ | ``'G'`` | Floating-point format. Uses uppercase exponential | \(4) | | | format if exponent is less than -4 or not less than | | | | precision, decimal format otherwise. | | +------------+-----------------------------------------------------+-------+ | ``'c'`` | Single character (accepts integer or single | | | | character string). | | +------------+-----------------------------------------------------+-------+ | ``'r'`` | String (converts any Python object using | \(5) | | | :func:`repr`). | | +------------+-----------------------------------------------------+-------+ | ``'s'`` | String (converts any Python object using | \(5) | | | :func:`str`). | | +------------+-----------------------------------------------------+-------+ | ``'a'`` | String (converts any Python object using | \(5) | | | :func:`ascii`). | | +------------+-----------------------------------------------------+-------+ | ``'%'`` | No argument is converted, results in a ``'%'`` | | | | character in the result. | | +------------+-----------------------------------------------------+-------+ For floating-point formats, the result should be correctly rounded to a given precision ``p`` of digits after the decimal point. The rounding mode matches that of the :func:`round` builtin. Notes: (1) The alternate form causes a leading octal specifier (``'0o'``) to be inserted before the first digit. (2) The alternate form causes a leading ``'0x'`` or ``'0X'`` (depending on whether the ``'x'`` or ``'X'`` format was used) to be inserted before the first digit. (3) The alternate form causes the result to always contain a decimal point, even if no digits follow it. The precision determines the number of digits after the decimal point and defaults to 6. (4) The alternate form causes the result to always contain a decimal point, and trailing zeroes are not removed as they would otherwise be. The precision determines the number of significant digits before and after the decimal point and defaults to 6. (5) If precision is ``N``, the output is truncated to ``N`` characters. (6) See :pep:`237`. Since Python strings have an explicit length, ``%s`` conversions do not assume that ``'\0'`` is the end of the string. .. XXX Examples? .. versionchanged:: 3.1 ``%f`` conversions for numbers whose absolute value is over 1e50 are no longer replaced by ``%g`` conversions. .. index:: single: buffer protocol; binary sequence types .. _binaryseq: Binary Sequence Types --- :class:`bytes`, :class:`bytearray`, :class:`memoryview` ================================================================================= .. index:: pair: object; bytes pair: object; bytearray pair: object; memoryview pair: module; array The core built-in types for manipulating binary data are :class:`bytes` and :class:`bytearray`. They are supported by :class:`memoryview` which uses the :ref:`buffer protocol ` to access the memory of other binary objects without needing to make a copy. The :mod:`array` module supports efficient storage of basic data types like 32-bit integers and IEEE754 double-precision floating values. .. _typebytes: Bytes Objects ------------- .. index:: pair: object; bytes Bytes objects are immutable sequences of single bytes. Since many major binary protocols are based on the ASCII text encoding, bytes objects offer several methods that are only valid when working with ASCII compatible data and are closely related to string objects in a variety of other ways. .. class:: bytes(source=b'') bytes(source, encoding, errors='strict') Firstly, the syntax for bytes literals is largely the same as that for string literals, except that a ``b`` prefix is added: * Single quotes: ``b'still allows embedded "double" quotes'`` * Double quotes: ``b"still allows embedded 'single' quotes"`` * Triple quoted: ``b'''3 single quotes'''``, ``b"""3 double quotes"""`` Only ASCII characters are permitted in bytes literals (regardless of the declared source code encoding). Any binary values over 127 must be entered into bytes literals using the appropriate escape sequence. As with string literals, bytes literals may also use a ``r`` prefix to disable processing of escape sequences. See :ref:`strings` for more about the various forms of bytes literal, including supported escape sequences. While bytes literals and representations are based on ASCII text, bytes objects actually behave like immutable sequences of integers, with each value in the sequence restricted such that ``0 <= x < 256`` (attempts to violate this restriction will trigger :exc:`ValueError`). This is done deliberately to emphasise that while many binary formats include ASCII based elements and can be usefully manipulated with some text-oriented algorithms, this is not generally the case for arbitrary binary data (blindly applying text processing algorithms to binary data formats that are not ASCII compatible will usually lead to data corruption). In addition to the literal forms, bytes objects can be created in a number of other ways: * A zero-filled bytes object of a specified length: ``bytes(10)`` * From an iterable of integers: ``bytes(range(20))`` * Copying existing binary data via the buffer protocol: ``bytes(obj)`` Also see the :ref:`bytes ` built-in. Since 2 hexadecimal digits correspond precisely to a single byte, hexadecimal numbers are a commonly used format for describing binary data. Accordingly, the bytes type has an additional class method to read data in that format: .. classmethod:: fromhex(string, /) This :class:`bytes` class method returns a bytes object, decoding the given string object. The string must contain two hexadecimal digits per byte, with ASCII whitespace being ignored. >>> bytes.fromhex('2Ef0 F1f2 ') b'.\xf0\xf1\xf2' .. versionchanged:: 3.7 :meth:`bytes.fromhex` now skips all ASCII whitespace in the string, not just spaces. .. versionchanged:: 3.14 :meth:`bytes.fromhex` now accepts ASCII :class:`bytes` and :term:`bytes-like objects ` as input. A reverse conversion function exists to transform a bytes object into its hexadecimal representation. .. method:: hex(*, bytes_per_sep=1) hex(sep, bytes_per_sep=1) Return a string object containing two hexadecimal digits for each byte in the instance. >>> b'\xf0\xf1\xf2'.hex() 'f0f1f2' If you want to make the hex string easier to read, you can specify a single character separator *sep* parameter to include in the output. By default, this separator will be included between each byte. A second optional *bytes_per_sep* parameter controls the spacing. Positive values calculate the separator position from the right, negative values from the left. >>> value = b'\xf0\xf1\xf2' >>> value.hex('-') 'f0-f1-f2' >>> value.hex('_', 2) 'f0_f1f2' >>> b'UUDDLRLRAB'.hex(' ', -4) '55554444 4c524c52 4142' .. versionadded:: 3.5 .. versionchanged:: 3.8 :meth:`bytes.hex` now supports optional *sep* and *bytes_per_sep* parameters to insert separators between bytes in the hex output. Since bytes objects are sequences of integers (akin to a tuple), for a bytes object *b*, ``b[0]`` will be an integer, while ``b[0:1]`` will be a bytes object of length 1. (This contrasts with text strings, where both indexing and slicing will produce a string of length 1) The representation of bytes objects uses the literal format (``b'...'``) since it is often more useful than e.g. ``bytes([46, 46, 46])``. You can always convert a bytes object into a list of integers using ``list(b)``. .. _typebytearray: Bytearray Objects ----------------- .. index:: pair: object; bytearray :class:`bytearray` objects are a mutable counterpart to :class:`bytes` objects. .. class:: bytearray(source=b'') bytearray(source, encoding, errors='strict') There is no dedicated literal syntax for bytearray objects, instead they are always created by calling the constructor: * Creating an empty instance: ``bytearray()`` * Creating a zero-filled instance with a given length: ``bytearray(10)`` * From an iterable of integers: ``bytearray(range(20))`` * Copying existing binary data via the buffer protocol: ``bytearray(b'Hi!')`` As bytearray objects are mutable, they support the :ref:`mutable ` sequence operations in addition to the common bytes and bytearray operations described in :ref:`bytes-methods`. Also see the :ref:`bytearray ` built-in. Since 2 hexadecimal digits correspond precisely to a single byte, hexadecimal numbers are a commonly used format for describing binary data. Accordingly, the bytearray type has an additional class method to read data in that format: .. classmethod:: fromhex(string, /) This :class:`bytearray` class method returns a bytearray object, decoding the given string object. The string must contain two hexadecimal digits per byte, with ASCII whitespace being ignored. >>> bytearray.fromhex('2Ef0 F1f2 ') bytearray(b'.\xf0\xf1\xf2') .. versionchanged:: 3.7 :meth:`bytearray.fromhex` now skips all ASCII whitespace in the string, not just spaces. .. versionchanged:: 3.14 :meth:`bytearray.fromhex` now accepts ASCII :class:`bytes` and :term:`bytes-like objects ` as input. A reverse conversion function exists to transform a bytearray object into its hexadecimal representation. .. method:: hex(*, bytes_per_sep=1) hex(sep, bytes_per_sep=1) Return a string object containing two hexadecimal digits for each byte in the instance. >>> bytearray(b'\xf0\xf1\xf2').hex() 'f0f1f2' .. versionadded:: 3.5 .. versionchanged:: 3.8 Similar to :meth:`bytes.hex`, :meth:`bytearray.hex` now supports optional *sep* and *bytes_per_sep* parameters to insert separators between bytes in the hex output. .. method:: resize(size, /) Resize the :class:`bytearray` to contain *size* bytes. *size* must be greater than or equal to 0. If the :class:`bytearray` needs to shrink, bytes beyond *size* are truncated. If the :class:`bytearray` needs to grow, all new bytes, those beyond *size*, will be set to null bytes. This is equivalent to: >>> def resize(ba, size): ... if len(ba) > size: ... del ba[size:] ... else: ... ba += b'\0' * (size - len(ba)) Examples: >>> shrink = bytearray(b'abc') >>> shrink.resize(1) >>> (shrink, len(shrink)) (bytearray(b'a'), 1) >>> grow = bytearray(b'abc') >>> grow.resize(5) >>> (grow, len(grow)) (bytearray(b'abc\x00\x00'), 5) .. versionadded:: 3.14 .. method:: take_bytes(n=None, /) Remove the first *n* bytes from the bytearray and return them as an immutable :class:`bytes`. By default (if *n* is ``None``), return all bytes and clear the bytearray. If *n* is negative, index from the end and take the first :func:`len` plus *n* bytes. If *n* is out of bounds, raise :exc:`IndexError`. Taking less than the full length will leave remaining bytes in the :class:`bytearray`, which requires a copy. If the remaining bytes should be discarded, use :func:`~bytearray.resize` or :keyword:`del` to truncate then :func:`~bytearray.take_bytes` without a size. .. impl-detail:: Taking all bytes is a zero-copy operation. .. versionadded:: 3.15 See the :ref:`What's New ` entry for common code patterns which can be optimized with :meth:`bytearray.take_bytes`. Since bytearray objects are sequences of integers (akin to a list), for a bytearray object *b*, ``b[0]`` will be an integer, while ``b[0:1]`` will be a bytearray object of length 1. (This contrasts with text strings, where both indexing and slicing will produce a string of length 1) The representation of bytearray objects uses the bytes literal format (``bytearray(b'...')``) since it is often more useful than e.g. ``bytearray([46, 46, 46])``. You can always convert a bytearray object into a list of integers using ``list(b)``. .. seealso:: For detailed information on thread-safety guarantees for :class:`bytearray` objects, see :ref:`thread-safety-bytearray`. .. _bytes-methods: Bytes and Bytearray Operations ------------------------------ .. index:: pair: bytes; methods pair: bytearray; methods Both bytes and bytearray objects support the :ref:`common ` sequence operations. They interoperate not just with operands of the same type, but with any :term:`bytes-like object`. Due to this flexibility, they can be freely mixed in operations without causing errors. However, the return type of the result may depend on the order of operands. .. note:: The methods on bytes and bytearray objects don't accept strings as their arguments, just as the methods on strings don't accept bytes as their arguments. For example, you have to write:: a = "abc" b = a.replace("a", "f") and:: a = b"abc" b = a.replace(b"a", b"f") Some bytes and bytearray operations assume the use of ASCII compatible binary formats, and hence should be avoided when working with arbitrary binary data. These restrictions are covered below. .. note:: Using these ASCII based operations to manipulate binary data that is not stored in an ASCII based format may lead to data corruption. The following methods on bytes and bytearray objects can be used with arbitrary binary data. .. method:: bytes.count(sub[, start[, end]]) bytearray.count(sub[, start[, end]]) Return the number of non-overlapping occurrences of subsequence *sub* in the range [*start*, *end*]. Optional arguments *start* and *end* are interpreted as in slice notation. The subsequence to search for may be any :term:`bytes-like object` or an integer in the range 0 to 255. If *sub* is empty, returns the number of empty slices between characters which is the length of the bytes object plus one. .. versionchanged:: 3.3 Also accept an integer in the range 0 to 255 as the subsequence. .. method:: bytes.removeprefix(prefix, /) bytearray.removeprefix(prefix, /) If the binary data starts with the *prefix* string, return ``bytes[len(prefix):]``. Otherwise, return a copy of the original binary data:: >>> b'TestHook'.removeprefix(b'Test') b'Hook' >>> b'BaseTestCase'.removeprefix(b'Test') b'BaseTestCase' The *prefix* may be any :term:`bytes-like object`. .. note:: The bytearray version of this method does *not* operate in place - it always produces a new object, even if no changes were made. .. versionadded:: 3.9 .. method:: bytes.removesuffix(suffix, /) bytearray.removesuffix(suffix, /) If the binary data ends with the *suffix* string and that *suffix* is not empty, return ``bytes[:-len(suffix)]``. Otherwise, return a copy of the original binary data:: >>> b'MiscTests'.removesuffix(b'Tests') b'Misc' >>> b'TmpDirMixin'.removesuffix(b'Tests') b'TmpDirMixin' The *suffix* may be any :term:`bytes-like object`. .. note:: The bytearray version of this method does *not* operate in place - it always produces a new object, even if no changes were made. .. versionadded:: 3.9 .. method:: bytes.decode(encoding="utf-8", errors="strict") bytearray.decode(encoding="utf-8", errors="strict") Return the bytes decoded to a :class:`str`. *encoding* defaults to ``'utf-8'``; see :ref:`standard-encodings` for possible values. *errors* controls how decoding errors are handled. If ``'strict'`` (the default), a :exc:`UnicodeError` exception is raised. Other possible values are ``'ignore'``, ``'replace'``, and any other name registered via :func:`codecs.register_error`. See :ref:`error-handlers` for details. For performance reasons, the value of *errors* is not checked for validity unless a decoding error actually occurs, :ref:`devmode` is enabled or a :ref:`debug build ` is used. .. note:: Passing the *encoding* argument to :class:`str` allows decoding any :term:`bytes-like object` directly, without needing to make a temporary :class:`!bytes` or :class:`!bytearray` object. .. versionchanged:: 3.1 Added support for keyword arguments. .. versionchanged:: 3.9 The value of the *errors* argument is now checked in :ref:`devmode` and in :ref:`debug mode `. .. method:: bytes.endswith(suffix[, start[, end]]) bytearray.endswith(suffix[, start[, end]]) Return ``True`` if the binary data ends with the specified *suffix*, otherwise return ``False``. *suffix* can also be a tuple of suffixes to look for. With optional *start*, test beginning at that position. With optional *end*, stop comparing at that position. The suffix(es) to search for may be any :term:`bytes-like object`. .. method:: bytes.find(sub[, start[, end]]) bytearray.find(sub[, start[, end]]) Return the lowest index in the data where the subsequence *sub* is found, such that *sub* is contained in the slice ``s[start:end]``. Optional arguments *start* and *end* are interpreted as in slice notation. Return ``-1`` if *sub* is not found. The subsequence to search for may be any :term:`bytes-like object` or an integer in the range 0 to 255. .. note:: The :meth:`~bytes.find` method should be used only if you need to know the position of *sub*. To check if *sub* is a substring or not, use the :keyword:`in` operator:: >>> b'Py' in b'Python' True .. versionchanged:: 3.3 Also accept an integer in the range 0 to 255 as the subsequence. .. method:: bytes.index(sub[, start[, end]]) bytearray.index(sub[, start[, end]]) Like :meth:`~bytes.find`, but raise :exc:`ValueError` when the subsequence is not found. The subsequence to search for may be any :term:`bytes-like object` or an integer in the range 0 to 255. .. versionchanged:: 3.3 Also accept an integer in the range 0 to 255 as the subsequence. .. method:: bytes.join(iterable, /) bytearray.join(iterable, /) Return a bytes or bytearray object which is the concatenation of the binary data sequences in *iterable*. A :exc:`TypeError` will be raised if there are any values in *iterable* that are not :term:`bytes-like objects `, including :class:`str` objects. The separator between elements is the contents of the bytes or bytearray object providing this method. .. staticmethod:: bytes.maketrans(from, to, /) bytearray.maketrans(from, to, /) This static method returns a translation table usable for :meth:`bytes.translate` that will map each character in *from* into the character at the same position in *to*; *from* and *to* must both be :term:`bytes-like objects ` and have the same length. .. versionadded:: 3.1 .. method:: bytes.partition(sep, /) bytearray.partition(sep, /) Split the sequence at the first occurrence of *sep*, and return a 3-tuple containing the part before the separator, the separator itself or its bytearray copy, and the part after the separator. If the separator is not found, return a 3-tuple containing a copy of the original sequence, followed by two empty bytes or bytearray objects. The separator to search for may be any :term:`bytes-like object`. .. method:: bytes.replace(old, new, /, count=-1) bytearray.replace(old, new, /, count=-1) Return a copy of the sequence with all occurrences of subsequence *old* replaced by *new*. If *count* is given, only the first *count* occurrences are replaced. If *count* is not specified or ``-1``, then all occurrences are replaced. The subsequence to search for and its replacement may be any :term:`bytes-like object`. .. note:: The bytearray version of this method does *not* operate in place - it always produces a new object, even if no changes were made. .. versionchanged:: 3.15 *count* is now supported as a keyword argument. .. method:: bytes.rfind(sub[, start[, end]]) bytearray.rfind(sub[, start[, end]]) Return the highest index in the sequence where the subsequence *sub* is found, such that *sub* is contained within ``s[start:end]``. Optional arguments *start* and *end* are interpreted as in slice notation. Return ``-1`` on failure. The subsequence to search for may be any :term:`bytes-like object` or an integer in the range 0 to 255. .. versionchanged:: 3.3 Also accept an integer in the range 0 to 255 as the subsequence. .. method:: bytes.rindex(sub[, start[, end]]) bytearray.rindex(sub[, start[, end]]) Like :meth:`~bytes.rfind` but raises :exc:`ValueError` when the subsequence *sub* is not found. The subsequence to search for may be any :term:`bytes-like object` or an integer in the range 0 to 255. .. versionchanged:: 3.3 Also accept an integer in the range 0 to 255 as the subsequence. .. method:: bytes.rpartition(sep, /) bytearray.rpartition(sep, /) Split the sequence at the last occurrence of *sep*, and return a 3-tuple containing the part before the separator, the separator itself or its bytearray copy, and the part after the separator. If the separator is not found, return a 3-tuple containing two empty bytes or bytearray objects, followed by a copy of the original sequence. The separator to search for may be any :term:`bytes-like object`. .. method:: bytes.startswith(prefix[, start[, end]]) bytearray.startswith(prefix[, start[, end]]) Return ``True`` if the binary data starts with the specified *prefix*, otherwise return ``False``. *prefix* can also be a tuple of prefixes to look for. With optional *start*, test beginning at that position. With optional *end*, stop comparing at that position. The prefix(es) to search for may be any :term:`bytes-like object`. .. method:: bytes.translate(table, /, delete=b'') bytearray.translate(table, /, delete=b'') Return a copy of the bytes or bytearray object where all bytes occurring in the optional argument *delete* are removed, and the remaining bytes have been mapped through the given translation table, which must be a bytes object of length 256. You can use the :func:`bytes.maketrans` method to create a translation table. Set the *table* argument to ``None`` for translations that only delete characters:: >>> b'read this short text'.translate(None, b'aeiou') b'rd ths shrt txt' .. versionchanged:: 3.6 *delete* is now supported as a keyword argument. The following methods on bytes and bytearray objects have default behaviours that assume the use of ASCII compatible binary formats, but can still be used with arbitrary binary data by passing appropriate arguments. Note that all of the bytearray methods in this section do *not* operate in place, and instead produce new objects. .. method:: bytes.center(width, fillbyte=b' ', /) bytearray.center(width, fillbyte=b' ', /) Return a copy of the object centered in a sequence of length *width*. Padding is done using the specified *fillbyte* (default is an ASCII space). For :class:`bytes` objects, the original sequence is returned if *width* is less than or equal to ``len(s)``. .. note:: The bytearray version of this method does *not* operate in place - it always produces a new object, even if no changes were made. .. method:: bytes.ljust(width, fillbyte=b' ', /) bytearray.ljust(width, fillbyte=b' ', /) Return a copy of the object left justified in a sequence of length *width*. Padding is done using the specified *fillbyte* (default is an ASCII space). For :class:`bytes` objects, the original sequence is returned if *width* is less than or equal to ``len(s)``. .. note:: The bytearray version of this method does *not* operate in place - it always produces a new object, even if no changes were made. .. method:: bytes.lstrip(bytes=None, /) bytearray.lstrip(bytes=None, /) Return a copy of the sequence with specified leading bytes removed. The *bytes* argument is a binary sequence specifying the set of byte values to be removed. If omitted or ``None``, the *bytes* argument defaults to removing :meth:`ASCII whitespace `. The *bytes* argument is not a prefix; rather, all combinations of its values are stripped:: >>> b' spacious '.lstrip() b'spacious ' >>> b'www.example.com'.lstrip(b'cmowz.') b'example.com' The binary sequence of byte values to remove may be any :term:`bytes-like object`. See :meth:`~bytes.removeprefix` for a method that will remove a single prefix string rather than all of a set of characters. For example:: >>> b'Arthur: three!'.lstrip(b'Arthur: ') b'ee!' >>> b'Arthur: three!'.removeprefix(b'Arthur: ') b'three!' .. note:: The bytearray version of this method does *not* operate in place - it always produces a new object, even if no changes were made. .. method:: bytes.rjust(width, fillbyte=b' ', /) bytearray.rjust(width, fillbyte=b' ', /) Return a copy of the object right justified in a sequence of length *width*. Padding is done using the specified *fillbyte* (default is an ASCII space). For :class:`bytes` objects, the original sequence is returned if *width* is less than or equal to ``len(s)``. .. note:: The bytearray version of this method does *not* operate in place - it always produces a new object, even if no changes were made. .. method:: bytes.rsplit(sep=None, maxsplit=-1) bytearray.rsplit(sep=None, maxsplit=-1) Split the binary sequence into subsequences of the same type, using *sep* as the delimiter string. If *maxsplit* is given, at most *maxsplit* splits are done, the *rightmost* ones. If *sep* is not specified or ``None``, any subsequence consisting solely of :meth:`ASCII whitespace ` is a separator. Except for splitting from the right, :meth:`rsplit` behaves like :meth:`split` which is described in detail below. .. method:: bytes.rstrip(bytes=None, /) bytearray.rstrip(bytes=None, /) Return a copy of the sequence with specified trailing bytes removed. The *bytes* argument is a binary sequence specifying the set of byte values to be removed. If omitted or ``None``, the *bytes* argument defaults to removing :meth:`ASCII whitespace `. The *bytes* argument is not a suffix; rather, all combinations of its values are stripped:: >>> b' spacious '.rstrip() b' spacious' >>> b'mississippi'.rstrip(b'ipz') b'mississ' The binary sequence of byte values to remove may be any :term:`bytes-like object`. See :meth:`~bytes.removesuffix` for a method that will remove a single suffix string rather than all of a set of characters. For example:: >>> b'Monty Python'.rstrip(b' Python') b'M' >>> b'Monty Python'.removesuffix(b' Python') b'Monty' .. note:: The bytearray version of this method does *not* operate in place - it always produces a new object, even if no changes were made. .. method:: bytes.split(sep=None, maxsplit=-1) bytearray.split(sep=None, maxsplit=-1) Split the binary sequence into subsequences of the same type, using *sep* as the delimiter string. If *maxsplit* is given and non-negative, at most *maxsplit* splits are done (thus, the list will have at most ``maxsplit+1`` elements). If *maxsplit* is not specified or is ``-1``, then there is no limit on the number of splits (all possible splits are made). If *sep* is given, consecutive delimiters are not grouped together and are deemed to delimit empty subsequences (for example, ``b'1,,2'.split(b',')`` returns ``[b'1', b'', b'2']``). The *sep* argument may consist of a multibyte sequence as a single delimiter. Splitting an empty sequence with a specified separator returns ``[b'']`` or ``[bytearray(b'')]`` depending on the type of object being split. The *sep* argument may be any :term:`bytes-like object`. For example:: >>> b'1,2,3'.split(b',') [b'1', b'2', b'3'] >>> b'1,2,3'.split(b',', maxsplit=1) [b'1', b'2,3'] >>> b'1,2,,3,'.split(b',') [b'1', b'2', b'', b'3', b''] >>> b'1<>2<>3<4'.split(b'<>') [b'1', b'2', b'3<4'] If *sep* is not specified or is ``None``, a different splitting algorithm is applied: runs of consecutive :meth:`ASCII whitespace ` are regarded as a single separator, and the result will contain no empty strings at the start or end if the sequence has leading or trailing whitespace. Consequently, splitting an empty sequence or a sequence consisting solely of ASCII whitespace without a specified separator returns ``[]``. For example:: >>> b'1 2 3'.split() [b'1', b'2', b'3'] >>> b'1 2 3'.split(maxsplit=1) [b'1', b'2 3'] >>> b' 1 2 3 '.split() [b'1', b'2', b'3'] .. method:: bytes.strip(bytes=None, /) bytearray.strip(bytes=None, /) Return a copy of the sequence with specified leading and trailing bytes removed. The *bytes* argument is a binary sequence specifying the set of byte values to be removed. If omitted or ``None``, the *bytes* argument defaults to removing :meth:`ASCII whitespace `. The *bytes* argument is not a prefix or suffix; rather, all combinations of its values are stripped:: >>> b' spacious '.strip() b'spacious' >>> b'www.example.com'.strip(b'cmowz.') b'example' The binary sequence of byte values to remove may be any :term:`bytes-like object`. .. note:: The bytearray version of this method does *not* operate in place - it always produces a new object, even if no changes were made. The following methods on bytes and bytearray objects assume the use of ASCII compatible binary formats and should not be applied to arbitrary binary data. Note that all of the bytearray methods in this section do *not* operate in place, and instead produce new objects. .. method:: bytes.capitalize() bytearray.capitalize() Return a copy of the sequence with each byte interpreted as an ASCII character, and the first byte capitalized and the rest lowercased. Non-ASCII byte values are passed through unchanged. .. note:: The bytearray version of this method does *not* operate in place - it always produces a new object, even if no changes were made. .. method:: bytes.expandtabs(tabsize=8) bytearray.expandtabs(tabsize=8) Return a copy of the sequence where all ASCII tab characters are replaced by one or more ASCII spaces, depending on the current column and the given tab size. Tab positions occur every *tabsize* bytes (default is 8, giving tab positions at columns 0, 8, 16 and so on). To expand the sequence, the current column is set to zero and the sequence is examined byte by byte. If the byte is an ASCII tab character (``b'\t'``), one or more space characters are inserted in the result until the current column is equal to the next tab position. (The tab character itself is not copied.) If the current byte is an ASCII newline (``b'\n'``) or carriage return (``b'\r'``), it is copied and the current column is reset to zero. Any other byte value is copied unchanged and the current column is incremented by one regardless of how the byte value is represented when printed:: >>> b'01\t012\t0123\t01234'.expandtabs() b'01 012 0123 01234' >>> b'01\t012\t0123\t01234'.expandtabs(4) b'01 012 0123 01234' .. note:: The bytearray version of this method does *not* operate in place - it always produces a new object, even if no changes were made. .. method:: bytes.isalnum() bytearray.isalnum() Return ``True`` if all bytes in the sequence are alphabetical ASCII characters or ASCII decimal digits and the sequence is not empty, ``False`` otherwise. Alphabetic ASCII characters are those byte values in the sequence ``b'abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ'``. ASCII decimal digits are those byte values in the sequence ``b'0123456789'``. For example:: >>> b'ABCabc1'.isalnum() True >>> b'ABC abc1'.isalnum() False .. method:: bytes.isalpha() bytearray.isalpha() Return ``True`` if all bytes in the sequence are alphabetic ASCII characters and the sequence is not empty, ``False`` otherwise. Alphabetic ASCII characters are those byte values in the sequence ``b'abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ'``. For example:: >>> b'ABCabc'.isalpha() True >>> b'ABCabc1'.isalpha() False .. method:: bytes.isascii() bytearray.isascii() Return ``True`` if the sequence is empty or all bytes in the sequence are ASCII, ``False`` otherwise. ASCII bytes are in the range 0-0x7F. .. versionadded:: 3.7 .. method:: bytes.isdigit() bytearray.isdigit() Return ``True`` if all bytes in the sequence are ASCII decimal digits and the sequence is not empty, ``False`` otherwise. ASCII decimal digits are those byte values in the sequence ``b'0123456789'``. For example:: >>> b'1234'.isdigit() True >>> b'1.23'.isdigit() False .. method:: bytes.islower() bytearray.islower() Return ``True`` if there is at least one lowercase ASCII character in the sequence and no uppercase ASCII characters, ``False`` otherwise. For example:: >>> b'hello world'.islower() True >>> b'Hello world'.islower() False Lowercase ASCII characters are those byte values in the sequence ``b'abcdefghijklmnopqrstuvwxyz'``. Uppercase ASCII characters are those byte values in the sequence ``b'ABCDEFGHIJKLMNOPQRSTUVWXYZ'``. .. method:: bytes.isspace() bytearray.isspace() Return ``True`` if all bytes in the sequence are ASCII whitespace and the sequence is not empty, ``False`` otherwise. ASCII whitespace characters are those byte values in the sequence ``b' \t\n\r\x0b\f'`` (space, tab, newline, carriage return, vertical tab, form feed). .. method:: bytes.istitle() bytearray.istitle() Return ``True`` if the sequence is ASCII titlecase and the sequence is not empty, ``False`` otherwise. See :meth:`bytes.title` for more details on the definition of "titlecase". For example:: >>> b'Hello World'.istitle() True >>> b'Hello world'.istitle() False .. method:: bytes.isupper() bytearray.isupper() Return ``True`` if there is at least one uppercase alphabetic ASCII character in the sequence and no lowercase ASCII characters, ``False`` otherwise. For example:: >>> b'HELLO WORLD'.isupper() True >>> b'Hello world'.isupper() False Lowercase ASCII characters are those byte values in the sequence ``b'abcdefghijklmnopqrstuvwxyz'``. Uppercase ASCII characters are those byte values in the sequence ``b'ABCDEFGHIJKLMNOPQRSTUVWXYZ'``. .. method:: bytes.lower() bytearray.lower() Return a copy of the sequence with all the uppercase ASCII characters converted to their corresponding lowercase counterpart. For example:: >>> b'Hello World'.lower() b'hello world' Lowercase ASCII characters are those byte values in the sequence ``b'abcdefghijklmnopqrstuvwxyz'``. Uppercase ASCII characters are those byte values in the sequence ``b'ABCDEFGHIJKLMNOPQRSTUVWXYZ'``. .. note:: The bytearray version of this method does *not* operate in place - it always produces a new object, even if no changes were made. .. index:: single: universal newlines; bytes.splitlines method single: universal newlines; bytearray.splitlines method .. method:: bytes.splitlines(keepends=False) bytearray.splitlines(keepends=False) Return a list of the lines in the binary sequence, breaking at ASCII line boundaries. This method uses the :term:`universal newlines` approach to splitting lines. Line breaks are not included in the resulting list unless *keepends* is given and true. For example:: >>> b'ab c\n\nde fg\rkl\r\n'.splitlines() [b'ab c', b'', b'de fg', b'kl'] >>> b'ab c\n\nde fg\rkl\r\n'.splitlines(keepends=True) [b'ab c\n', b'\n', b'de fg\r', b'kl\r\n'] Unlike :meth:`~bytes.split` when a delimiter string *sep* is given, this method returns an empty list for the empty string, and a terminal line break does not result in an extra line:: >>> b"".split(b'\n'), b"Two lines\n".split(b'\n') ([b''], [b'Two lines', b'']) >>> b"".splitlines(), b"One line\n".splitlines() ([], [b'One line']) .. method:: bytes.swapcase() bytearray.swapcase() Return a copy of the sequence with all the lowercase ASCII characters converted to their corresponding uppercase counterpart and vice-versa. For example:: >>> b'Hello World'.swapcase() b'hELLO wORLD' Lowercase ASCII characters are those byte values in the sequence ``b'abcdefghijklmnopqrstuvwxyz'``. Uppercase ASCII characters are those byte values in the sequence ``b'ABCDEFGHIJKLMNOPQRSTUVWXYZ'``. Unlike :func:`str.swapcase`, it is always the case that ``bin.swapcase().swapcase() == bin`` for the binary versions. Case conversions are symmetrical in ASCII, even though that is not generally true for arbitrary Unicode code points. .. note:: The bytearray version of this method does *not* operate in place - it always produces a new object, even if no changes were made. .. method:: bytes.title() bytearray.title() Return a titlecased version of the binary sequence where words start with an uppercase ASCII character and the remaining characters are lowercase. Uncased byte values are left unmodified. For example:: >>> b'Hello world'.title() b'Hello World' Lowercase ASCII characters are those byte values in the sequence ``b'abcdefghijklmnopqrstuvwxyz'``. Uppercase ASCII characters are those byte values in the sequence ``b'ABCDEFGHIJKLMNOPQRSTUVWXYZ'``. All other byte values are uncased. The algorithm uses a simple language-independent definition of a word as groups of consecutive letters. The definition works in many contexts but it means that apostrophes in contractions and possessives form word boundaries, which may not be the desired result:: >>> b"they're bill's friends from the UK".title() b"They'Re Bill'S Friends From The Uk" A workaround for apostrophes can be constructed using regular expressions:: >>> import re >>> def titlecase(s): ... return re.sub(rb"[A-Za-z]+('[A-Za-z]+)?", ... lambda mo: mo.group(0)[0:1].upper() + ... mo.group(0)[1:].lower(), ... s) ... >>> titlecase(b"they're bill's friends.") b"They're Bill's Friends." .. note:: The bytearray version of this method does *not* operate in place - it always produces a new object, even if no changes were made. .. method:: bytes.upper() bytearray.upper() Return a copy of the sequence with all the lowercase ASCII characters converted to their corresponding uppercase counterpart. For example:: >>> b'Hello World'.upper() b'HELLO WORLD' Lowercase ASCII characters are those byte values in the sequence ``b'abcdefghijklmnopqrstuvwxyz'``. Uppercase ASCII characters are those byte values in the sequence ``b'ABCDEFGHIJKLMNOPQRSTUVWXYZ'``. .. note:: The bytearray version of this method does *not* operate in place - it always produces a new object, even if no changes were made. .. method:: bytes.zfill(width, /) bytearray.zfill(width, /) Return a copy of the sequence left filled with ASCII ``b'0'`` digits to make a sequence of length *width*. A leading sign prefix (``b'+'``/ ``b'-'``) is handled by inserting the padding *after* the sign character rather than before. For :class:`bytes` objects, the original sequence is returned if *width* is less than or equal to ``len(seq)``. For example:: >>> b"42".zfill(5) b'00042' >>> b"-42".zfill(5) b'-0042' .. note:: The bytearray version of this method does *not* operate in place - it always produces a new object, even if no changes were made. .. _bytes-formatting: ``printf``-style Bytes Formatting ---------------------------------- .. index:: single: formatting; bytes (%) single: formatting; bytearray (%) single: interpolation; bytes (%) single: interpolation; bytearray (%) single: bytes; formatting single: bytearray; formatting single: bytes; interpolation single: bytearray; interpolation single: printf-style formatting single: sprintf-style formatting single: % (percent); printf-style formatting .. note:: The formatting operations described here exhibit a variety of quirks that lead to a number of common errors (such as failing to display tuples and dictionaries correctly). If the value being printed may be a tuple or dictionary, wrap it in a tuple. Bytes objects (``bytes``/``bytearray``) have one unique built-in operation: the ``%`` operator (modulo). This is also known as the bytes *formatting* or *interpolation* operator. Given ``format % values`` (where *format* is a bytes object), ``%`` conversion specifications in *format* are replaced with zero or more elements of *values*. The effect is similar to using the :c:func:`sprintf` function in the C language. If *format* requires a single argument, *values* may be a single non-tuple object. [5]_ Otherwise, *values* must be a tuple with exactly the number of items specified by the format bytes object, or a single mapping object (for example, a dictionary). .. index:: single: () (parentheses); in printf-style formatting single: * (asterisk); in printf-style formatting single: . (dot); in printf-style formatting A conversion specifier contains two or more characters and has the following components, which must occur in this order: #. The ``'%'`` character, which marks the start of the specifier. #. Mapping key (optional), consisting of a parenthesised sequence of characters (for example, ``(somename)``). #. Conversion flags (optional), which affect the result of some conversion types. #. Minimum field width (optional). If specified as an ``'*'`` (asterisk), the actual width is read from the next element of the tuple in *values*, and the object to convert comes after the minimum field width and optional precision. #. Precision (optional), given as a ``'.'`` (dot) followed by the precision. If specified as ``'*'`` (an asterisk), the actual precision is read from the next element of the tuple in *values*, and the value to convert comes after the precision. #. Length modifier (optional). #. Conversion type. When the right argument is a dictionary (or other mapping type), then the formats in the bytes object *must* include a parenthesised mapping key into that dictionary inserted immediately after the ``'%'`` character. The mapping key selects the value to be formatted from the mapping. For example: >>> print(b'%(language)s has %(number)03d quote types.' % ... {b'language': b"Python", b"number": 2}) b'Python has 002 quote types.' In this case no ``*`` specifiers may occur in a format (since they require a sequential parameter list). The conversion flag characters are: .. index:: single: # (hash); in printf-style formatting single: - (minus); in printf-style formatting single: + (plus); in printf-style formatting single: space; in printf-style formatting +---------+---------------------------------------------------------------------+ | Flag | Meaning | +=========+=====================================================================+ | ``'#'`` | The value conversion will use the "alternate form" (where defined | | | below). | +---------+---------------------------------------------------------------------+ | ``'0'`` | The conversion will be zero padded for numeric values. | +---------+---------------------------------------------------------------------+ | ``'-'`` | The converted value is left adjusted (overrides the ``'0'`` | | | conversion if both are given). | +---------+---------------------------------------------------------------------+ | ``' '`` | (a space) A blank should be left before a positive number (or empty | | | string) produced by a signed conversion. | +---------+---------------------------------------------------------------------+ | ``'+'`` | A sign character (``'+'`` or ``'-'``) will precede the conversion | | | (overrides a "space" flag). | +---------+---------------------------------------------------------------------+ A length modifier (``h``, ``l``, or ``L``) may be present, but is ignored as it is not necessary for Python -- so e.g. ``%ld`` is identical to ``%d``. The conversion types are: +------------+-----------------------------------------------------+-------+ | Conversion | Meaning | Notes | +============+=====================================================+=======+ | ``'d'`` | Signed integer decimal. | | +------------+-----------------------------------------------------+-------+ | ``'i'`` | Signed integer decimal. | | +------------+-----------------------------------------------------+-------+ | ``'o'`` | Signed octal value. | \(1) | +------------+-----------------------------------------------------+-------+ | ``'u'`` | Obsolete type -- it is identical to ``'d'``. | \(8) | +------------+-----------------------------------------------------+-------+ | ``'x'`` | Signed hexadecimal (lowercase). | \(2) | +------------+-----------------------------------------------------+-------+ | ``'X'`` | Signed hexadecimal (uppercase). | \(2) | +------------+-----------------------------------------------------+-------+ | ``'e'`` | Floating-point exponential format (lowercase). | \(3) | +------------+-----------------------------------------------------+-------+ | ``'E'`` | Floating-point exponential format (uppercase). | \(3) | +------------+-----------------------------------------------------+-------+ | ``'f'`` | Floating-point decimal format. | \(3) | +------------+-----------------------------------------------------+-------+ | ``'F'`` | Floating-point decimal format. | \(3) | +------------+-----------------------------------------------------+-------+ | ``'g'`` | Floating-point format. Uses lowercase exponential | \(4) | | | format if exponent is less than -4 or not less than | | | | precision, decimal format otherwise. | | +------------+-----------------------------------------------------+-------+ | ``'G'`` | Floating-point format. Uses uppercase exponential | \(4) | | | format if exponent is less than -4 or not less than | | | | precision, decimal format otherwise. | | +------------+-----------------------------------------------------+-------+ | ``'c'`` | Single byte (accepts integer or single | | | | byte objects). | | +------------+-----------------------------------------------------+-------+ | ``'b'`` | Bytes (any object that follows the | \(5) | | | :ref:`buffer protocol ` or has | | | | :meth:`~object.__bytes__`). | | +------------+-----------------------------------------------------+-------+ | ``'s'`` | ``'s'`` is an alias for ``'b'`` and should only | \(6) | | | be used for Python2/3 code bases. | | +------------+-----------------------------------------------------+-------+ | ``'a'`` | Bytes (converts any Python object using | \(5) | | | ``repr(obj).encode('ascii', 'backslashreplace')``). | | +------------+-----------------------------------------------------+-------+ | ``'r'`` | ``'r'`` is an alias for ``'a'`` and should only | \(7) | | | be used for Python2/3 code bases. | | +------------+-----------------------------------------------------+-------+ | ``'%'`` | No argument is converted, results in a ``'%'`` | | | | character in the result. | | +------------+-----------------------------------------------------+-------+ Notes: (1) The alternate form causes a leading octal specifier (``'0o'``) to be inserted before the first digit. (2) The alternate form causes a leading ``'0x'`` or ``'0X'`` (depending on whether the ``'x'`` or ``'X'`` format was used) to be inserted before the first digit. (3) The alternate form causes the result to always contain a decimal point, even if no digits follow it. The precision determines the number of digits after the decimal point and defaults to 6. (4) The alternate form causes the result to always contain a decimal point, and trailing zeroes are not removed as they would otherwise be. The precision determines the number of significant digits before and after the decimal point and defaults to 6. (5) If precision is ``N``, the output is truncated to ``N`` characters. (6) ``b'%s'`` is deprecated, but will not be removed during the 3.x series. (7) ``b'%r'`` is deprecated, but will not be removed during the 3.x series. (8) See :pep:`237`. .. note:: The bytearray version of this method does *not* operate in place - it always produces a new object, even if no changes were made. .. seealso:: :pep:`461` - Adding % formatting to bytes and bytearray .. versionadded:: 3.5 .. _typememoryview: Memory Views ------------ :class:`memoryview` objects allow Python code to access the internal data of an object that supports the :ref:`buffer protocol ` without copying. .. class:: memoryview(object) Create a :class:`memoryview` that references *object*. *object* must support the buffer protocol. Built-in objects that support the buffer protocol include :class:`bytes` and :class:`bytearray`. A :class:`memoryview` has the notion of an *element*, which is the atomic memory unit handled by the originating *object*. For many simple types such as :class:`bytes` and :class:`bytearray`, an element is a single byte, but other types such as :class:`array.array` may have bigger elements. :class:`!memoryview`\s are :ref:`generic ` over the type of their underlying data. ``len(view)`` is equal to the length of :meth:`~memoryview.tolist`, which is the nested list representation of the view. If ``view.ndim == 1``, this is equal to the number of elements in the view. .. versionchanged:: 3.12 If ``view.ndim == 0``, ``len(view)`` now raises :exc:`TypeError` instead of returning 1. The :class:`~memoryview.itemsize` attribute will give you the number of bytes in a single element. A :class:`memoryview` supports slicing and indexing to expose its data. One-dimensional slicing will result in a subview:: >>> v = memoryview(b'abcefg') >>> v[1] 98 >>> v[-1] 103 >>> v[1:4] >>> bytes(v[1:4]) b'bce' If :class:`~memoryview.format` is one of the native format specifiers from the :mod:`struct` module, indexing with an integer or a tuple of integers is also supported and returns a single *element* with the correct type. One-dimensional memoryviews can be indexed with an integer or a one-integer tuple. Multi-dimensional memoryviews can be indexed with tuples of exactly *ndim* integers where *ndim* is the number of dimensions. Zero-dimensional memoryviews can be indexed with the empty tuple. Here is an example with a non-byte format:: >>> import array >>> a = array.array('l', [-11111111, 22222222, -33333333, 44444444]) >>> m = memoryview(a) >>> m[0] -11111111 >>> m[-1] 44444444 >>> m[::2].tolist() [-11111111, -33333333] If the underlying object is writable, the memoryview supports one-dimensional slice assignment. Resizing is not allowed:: >>> data = bytearray(b'abcefg') >>> v = memoryview(data) >>> v.readonly False >>> v[0] = ord(b'z') >>> data bytearray(b'zbcefg') >>> v[1:4] = b'123' >>> data bytearray(b'z123fg') >>> v[2:3] = b'spam' Traceback (most recent call last): File "", line 1, in ValueError: memoryview assignment: lvalue and rvalue have different structures >>> v[2:6] = b'spam' >>> data bytearray(b'z1spam') One-dimensional memoryviews of :term:`hashable` (read-only) types with formats 'B', 'b' or 'c' are also hashable. The hash is defined as ``hash(m) == hash(m.tobytes())``:: >>> v = memoryview(b'abcefg') >>> hash(v) == hash(b'abcefg') True >>> hash(v[2:4]) == hash(b'ce') True >>> hash(v[::-2]) == hash(b'abcefg'[::-2]) True .. versionchanged:: 3.3 One-dimensional memoryviews can now be sliced. One-dimensional memoryviews with formats 'B', 'b' or 'c' are now :term:`hashable`. .. versionchanged:: 3.4 memoryview is now registered automatically with :class:`collections.abc.Sequence` .. versionchanged:: 3.5 memoryviews can now be indexed with a tuple of integers. .. versionchanged:: 3.14 memoryview is now a :term:`generic type`. :class:`memoryview` has several methods: .. method:: __eq__(exporter) A memoryview and a :pep:`3118` exporter are equal if their shapes are equivalent and if all corresponding values are equal when the operands' respective format codes are interpreted using :mod:`struct` syntax. For the subset of :mod:`struct` format strings currently supported by :meth:`tolist`, ``v`` and ``w`` are equal if ``v.tolist() == w.tolist()``:: >>> import array >>> a = array.array('I', [1, 2, 3, 4, 5]) >>> b = array.array('d', [1.0, 2.0, 3.0, 4.0, 5.0]) >>> c = array.array('b', [5, 3, 1]) >>> x = memoryview(a) >>> y = memoryview(b) >>> x == a == y == b True >>> x.tolist() == a.tolist() == y.tolist() == b.tolist() True >>> z = y[::-2] >>> z == c True >>> z.tolist() == c.tolist() True If either format string is not supported by the :mod:`struct` module, then the objects will always compare as unequal (even if the format strings and buffer contents are identical):: >>> from ctypes import BigEndianStructure, c_long >>> class BEPoint(BigEndianStructure): ... _fields_ = [("x", c_long), ("y", c_long)] ... >>> point = BEPoint(100, 200) >>> a = memoryview(point) >>> b = memoryview(point) >>> a == point False >>> a == b False Note that, as with floating-point numbers, ``v is w`` does *not* imply ``v == w`` for memoryview objects. .. versionchanged:: 3.3 Previous versions compared the raw memory disregarding the item format and the logical array structure. .. method:: tobytes(order='C') Return the data in the buffer as a bytestring. This is equivalent to calling the :class:`bytes` constructor on the memoryview. :: >>> m = memoryview(b"abc") >>> m.tobytes() b'abc' >>> bytes(m) b'abc' For non-contiguous arrays the result is equal to the flattened list representation with all elements converted to bytes. :meth:`tobytes` supports all format strings, including those that are not in :mod:`struct` module syntax. .. versionadded:: 3.8 *order* can be {'C', 'F', 'A'}. When *order* is 'C' or 'F', the data of the original array is converted to C or Fortran order. For contiguous views, 'A' returns an exact copy of the physical memory. In particular, in-memory Fortran order is preserved. For non-contiguous views, the data is converted to C first. *order=None* is the same as *order='C'*. .. method:: hex(*, bytes_per_sep=1) hex(sep, bytes_per_sep=1) Return a string object containing two hexadecimal digits for each byte in the buffer. :: >>> m = memoryview(b"abc") >>> m.hex() '616263' .. versionadded:: 3.5 .. versionchanged:: 3.8 Similar to :meth:`bytes.hex`, :meth:`memoryview.hex` now supports optional *sep* and *bytes_per_sep* parameters to insert separators between bytes in the hex output. .. method:: tolist() Return the data in the buffer as a list of elements. :: >>> memoryview(b'abc').tolist() [97, 98, 99] >>> import array >>> a = array.array('d', [1.1, 2.2, 3.3]) >>> m = memoryview(a) >>> m.tolist() [1.1, 2.2, 3.3] .. versionchanged:: 3.3 :meth:`tolist` now supports all single character native formats in :mod:`struct` module syntax as well as multi-dimensional representations. .. method:: toreadonly() Return a readonly version of the memoryview object. The original memoryview object is unchanged. :: >>> m = memoryview(bytearray(b'abc')) >>> mm = m.toreadonly() >>> mm.tolist() [97, 98, 99] >>> mm[0] = 42 Traceback (most recent call last): File "", line 1, in TypeError: cannot modify read-only memory >>> m[0] = 43 >>> mm.tolist() [43, 98, 99] .. versionadded:: 3.8 .. method:: release() Release the underlying buffer exposed by the memoryview object. Many objects take special actions when a view is held on them (for example, a :class:`bytearray` would temporarily forbid resizing); therefore, calling release() is handy to remove these restrictions (and free any dangling resources) as soon as possible. After this method has been called, any further operation on the view raises a :class:`ValueError` (except :meth:`release` itself which can be called multiple times):: >>> m = memoryview(b'abc') >>> m.release() >>> m[0] Traceback (most recent call last): File "", line 1, in ValueError: operation forbidden on released memoryview object The context management protocol can be used for a similar effect, using the ``with`` statement:: >>> with memoryview(b'abc') as m: ... m[0] ... 97 >>> m[0] Traceback (most recent call last): File "", line 1, in ValueError: operation forbidden on released memoryview object .. versionadded:: 3.2 .. method:: cast(format, /) cast(format, shape, /, *, order='C') Cast a memoryview to a new format or shape. *shape* defaults to ``[byte_length//new_itemsize]``, which means that the result view will be one-dimensional. The return value is a new memoryview, but the buffer itself is not copied. Supported casts are 1D -> C-:term:`contiguous`, C-contiguous -> 1D, and F-contiguous -> 1D. With a multidimensional *shape*, *order* selects the memory layout of the result: ``'C'`` for C-contiguous (row-major, the default) or ``'F'`` for Fortran-contiguous (column-major). The buffer is still not copied, so ``order='F'`` gives a zero-copy view over a buffer holding column-major data. The destination format is restricted to a single element native format in :mod:`struct` syntax. One of the formats must be a byte format ('B', 'b' or 'c'). The byte length of the result must be the same as the original length. Note that all byte lengths may depend on the operating system. Cast 1D/long to 1D/unsigned bytes:: >>> import array >>> a = array.array('l', [1,2,3]) >>> x = memoryview(a) >>> x.format 'l' >>> x.itemsize 8 >>> len(x) 3 >>> x.nbytes 24 >>> y = x.cast('B') >>> y.format 'B' >>> y.itemsize 1 >>> len(y) 24 >>> y.nbytes 24 Cast 1D/unsigned bytes to 1D/char:: >>> b = bytearray(b'zyz') >>> x = memoryview(b) >>> x[0] = b'a' Traceback (most recent call last): ... TypeError: memoryview: invalid type for format 'B' >>> y = x.cast('c') >>> y[0] = b'a' >>> b bytearray(b'ayz') Cast 1D/bytes to 3D/ints to 1D/signed char:: >>> import struct >>> buf = struct.pack("i"*12, *list(range(12))) >>> x = memoryview(buf) >>> y = x.cast('i', shape=[2,2,3]) >>> y.tolist() [[[0, 1, 2], [3, 4, 5]], [[6, 7, 8], [9, 10, 11]]] >>> y.format 'i' >>> y.itemsize 4 >>> len(y) 2 >>> y.nbytes 48 >>> z = y.cast('b') >>> z.format 'b' >>> z.itemsize 1 >>> len(z) 48 >>> z.nbytes 48 Cast 1D/unsigned long to 2D/unsigned long:: >>> buf = struct.pack("L"*6, *list(range(6))) >>> x = memoryview(buf) >>> y = x.cast('L', shape=[2,3]) >>> len(y) 2 >>> y.nbytes 48 >>> y.tolist() [[0, 1, 2], [3, 4, 5]] .. versionadded:: 3.3 .. versionchanged:: 3.5 The source format is no longer restricted when casting to a byte view. .. versionchanged:: 3.16 Casting a multi-dimensional F-contiguous view to a one-dimensional view is now supported. .. method:: count(value, /) Count the number of occurrences of *value*. .. versionadded:: 3.14 .. method:: index(value, start=0, stop=sys.maxsize, /) Return the index of the first occurrence of *value* (at or after index *start* and before index *stop*). Raises a :exc:`ValueError` if *value* cannot be found. .. versionadded:: 3.14 There are also several readonly attributes available: .. attribute:: obj The underlying object of the memoryview:: >>> b = bytearray(b'xyz') >>> m = memoryview(b) >>> m.obj is b True .. versionadded:: 3.3 .. attribute:: nbytes ``nbytes == product(shape) * itemsize == len(m.tobytes())``. This is the amount of space in bytes that the array would use in a contiguous representation. It is not necessarily equal to ``len(m)``:: >>> import array >>> a = array.array('i', [1,2,3,4,5]) >>> m = memoryview(a) >>> len(m) 5 >>> m.nbytes 20 >>> y = m[::2] >>> len(y) 3 >>> y.nbytes 12 >>> len(y.tobytes()) 12 Multi-dimensional arrays:: >>> import struct >>> buf = struct.pack("d"*12, *[1.5*x for x in range(12)]) >>> x = memoryview(buf) >>> y = x.cast('d', shape=[3,4]) >>> y.tolist() [[0.0, 1.5, 3.0, 4.5], [6.0, 7.5, 9.0, 10.5], [12.0, 13.5, 15.0, 16.5]] >>> len(y) 3 >>> y.nbytes 96 Interpret a flat buffer as a Fortran-contiguous (column-major) array:: >>> buf = bytes(range(6)) >>> y = memoryview(buf).cast('B', shape=[3, 2], order='F') >>> y.f_contiguous True >>> y.tolist() [[0, 3], [1, 4], [2, 5]] .. versionadded:: 3.3 .. versionchanged:: 3.16 Added the *order* parameter. .. attribute:: readonly A bool indicating whether the memory is read only. .. attribute:: format A string containing the format (in :mod:`struct` module style) for each element in the view. A memoryview can be created from exporters with arbitrary format strings, but some methods (e.g. :meth:`tolist`) are restricted to native single element formats. .. versionchanged:: 3.3 format ``'B'`` is now handled according to the struct module syntax. This means that ``memoryview(b'abc')[0] == b'abc'[0] == 97``. .. attribute:: itemsize The size in bytes of each element of the memoryview:: >>> import array, struct >>> m = memoryview(array.array('H', [32000, 32001, 32002])) >>> m.itemsize 2 >>> m[0] 32000 >>> struct.calcsize('H') == m.itemsize True .. attribute:: ndim An integer indicating how many dimensions of a multi-dimensional array the memory represents. .. attribute:: shape A tuple of integers the length of :attr:`ndim` giving the shape of the memory as an N-dimensional array. .. versionchanged:: 3.3 An empty tuple instead of ``None`` when ndim = 0. .. attribute:: strides A tuple of integers the length of :attr:`ndim` giving the size in bytes to access each element for each dimension of the array. .. versionchanged:: 3.3 An empty tuple instead of ``None`` when ndim = 0. .. attribute:: suboffsets Used internally for PIL-style arrays. The value is informational only. .. attribute:: c_contiguous A bool indicating whether the memory is C-:term:`contiguous`. .. versionadded:: 3.3 .. attribute:: f_contiguous A bool indicating whether the memory is Fortran :term:`contiguous`. .. versionadded:: 3.3 .. attribute:: contiguous A bool indicating whether the memory is :term:`contiguous`. .. versionadded:: 3.3 For information on the thread safety of :class:`memoryview` objects in the :term:`free-threaded build`, see :ref:`thread-safety-memoryview`. .. _types-set: Set Types --- :class:`set`, :class:`frozenset` ============================================== .. index:: pair: object; set A :dfn:`set` object is an unordered collection of distinct :term:`hashable` objects. Common uses include membership testing, removing duplicates from a sequence, and computing mathematical operations such as intersection, union, difference, and symmetric difference. (For other containers see the built-in :class:`dict`, :class:`list`, and :class:`tuple` classes, and the :mod:`collections` module.) See :ref:`time-complexity` for the costs of the various set operations. Like other collections, sets support ``x in set``, ``len(set)``, and ``for x in set``. Being an unordered collection, sets do not record element position or order of insertion. Accordingly, sets do not support indexing, slicing, or other sequence-like behavior. There are currently two built-in set types, :class:`set` and :class:`frozenset`. The :class:`set` type is mutable --- the contents can be changed using methods like :meth:`~set.add` and :meth:`~set.remove`. Since it is mutable, it has no hash value and cannot be used as either a dictionary key or as an element of another set. The :class:`frozenset` type is immutable and :term:`hashable` --- its contents cannot be altered after it is created; it can therefore be used as a dictionary key or as an element of another set. Non-empty sets (not frozensets) can be created by placing a comma-separated list of elements within braces, for example: ``{'jack', 'sjoerd'}``, in addition to the :class:`set` constructor. The constructors for both classes work the same: .. class:: set(iterable=(), /) frozenset(iterable=(), /) Return a new set or frozenset object whose elements are taken from *iterable*. The elements of a set must be :term:`hashable`. To represent sets of sets, the inner sets must be :class:`frozenset` objects. If *iterable* is not specified, a new empty set is returned. Sets can be created by several means: * Use a comma-separated list of elements within braces: ``{'jack', 'sjoerd'}`` * Use a set comprehension: ``{c for c in 'abracadabra' if c not in 'abc'}`` * Use the type constructor: ``set()``, ``set('foobar')``, ``set(['a', 'b', 'foo'])`` Instances of :class:`set` and :class:`frozenset` provide the following operations: .. describe:: len(s) Return the number of elements in set *s* (cardinality of *s*). .. describe:: x in s Test *x* for membership in *s*. .. describe:: x not in s Test *x* for non-membership in *s*. .. method:: frozenset.isdisjoint(other, /) set.isdisjoint(other, /) Return ``True`` if the set has no elements in common with *other*. Sets are disjoint if and only if their intersection is the empty set. .. method:: frozenset.issubset(other, /) set.issubset(other, /) .. describe:: set <= other Test whether every element in the set is in *other*. .. describe:: set < other Test whether the set is a proper subset of *other*, that is, ``set <= other and set != other``. .. method:: frozenset.issuperset(other, /) set.issuperset(other, /) .. describe:: set >= other Test whether every element in *other* is in the set. .. describe:: set > other Test whether the set is a proper superset of *other*, that is, ``set >= other and set != other``. .. method:: frozenset.union(*others) set.union(*others) .. describe:: set | other | ... Return a new set with elements from the set and all others. .. method:: frozenset.intersection(*others) set.intersection(*others) .. describe:: set & other & ... Return a new set with elements common to the set and all others. .. method:: frozenset.difference(*others) set.difference(*others) .. describe:: set - other - ... Return a new set with elements in the set that are not in the others. .. method:: frozenset.symmetric_difference(other, /) set.symmetric_difference(other, /) .. describe:: set ^ other Return a new set with elements in either the set or *other* but not both. .. method:: frozenset.copy() set.copy() Return a shallow copy of the set. Note, the non-operator versions of :meth:`~frozenset.union`, :meth:`~frozenset.intersection`, :meth:`~frozenset.difference`, :meth:`~frozenset.symmetric_difference`, :meth:`~frozenset.issubset`, and :meth:`~frozenset.issuperset` methods will accept any iterable as an argument. In contrast, their operator based counterparts require their arguments to be sets. This precludes error-prone constructions like ``set('abc') & 'cbs'`` in favor of the more readable ``set('abc').intersection('cbs')``. Both :class:`set` and :class:`frozenset` support set to set comparisons. Two sets are equal if and only if every element of each set is contained in the other (each is a subset of the other). A set is less than another set if and only if the first set is a proper subset of the second set (is a subset, but is not equal). A set is greater than another set if and only if the first set is a proper superset of the second set (is a superset, but is not equal). Instances of :class:`set` are compared to instances of :class:`frozenset` based on their members. For example, ``set('abc') == frozenset('abc')`` returns ``True`` and so does ``set('abc') in set([frozenset('abc')])``. The subset and equality comparisons do not generalize to a total ordering function. For example, any two nonempty disjoint sets are not equal and are not subsets of each other, so *all* of the following return ``False``: ``ab``. Since sets only define partial ordering (subset relationships), the output of the :meth:`list.sort` method is undefined for lists of sets. Set elements, like dictionary keys, must be :term:`hashable`. Binary operations that mix :class:`set` instances with :class:`frozenset` return the type of the first operand. For example: ``frozenset('ab') | set('bc')`` returns an instance of :class:`frozenset`. The following table lists operations available for :class:`set` that do not apply to immutable instances of :class:`frozenset`: .. method:: set.update(*others) .. describe:: set |= other | ... Update the set, adding elements from all others. .. method:: set.intersection_update(*others) .. describe:: set &= other & ... Update the set, keeping only elements found in it and all others. .. method:: set.difference_update(*others) .. describe:: set -= other | ... Update the set, removing elements found in others. .. method:: set.symmetric_difference_update(other, /) .. describe:: set ^= other Update the set, keeping only elements found in either set, but not in both. .. method:: set.add(elem, /) Add element *elem* to the set. .. method:: set.remove(elem, /) Remove element *elem* from the set. Raises :exc:`KeyError` if *elem* is not contained in the set. .. method:: set.discard(elem, /) Remove element *elem* from the set if it is present. .. method:: set.pop() Remove and return an arbitrary element from the set. Raises :exc:`KeyError` if the set is empty. .. method:: set.clear() Remove all elements from the set. Note, the non-operator versions of the :meth:`~set.update`, :meth:`~set.intersection_update`, :meth:`~set.difference_update`, and :meth:`~set.symmetric_difference_update` methods will accept any iterable as an argument. Note, the *elem* argument to the :meth:`~object.__contains__`, :meth:`~set.remove`, and :meth:`~set.discard` methods may be a set. To support searching for an equivalent frozenset, a temporary one is created from *elem*. Sets and frozensets are :ref:`generic ` over the type of their elements. .. seealso:: For detailed information on thread-safety guarantees for :class:`set` objects, see :ref:`thread-safety-set`. .. _typesmapping: Mapping types --- :class:`!dict`, :class:`!frozendict` ====================================================== .. index:: pair: object; mapping pair: object; dictionary triple: operations on; mapping; types triple: operations on; dictionary; type pair: statement; del pair: built-in function; len A :term:`mapping` object maps :term:`hashable` values to arbitrary objects. There are currently two standard mapping types, the :dfn:`dictionary` and :class:`frozendict`. (For other containers see the built-in :class:`list`, :class:`set`, and :class:`tuple` classes, and the :mod:`collections` module.) See :ref:`time-complexity` for the costs of the various dictionary operations. A dictionary's keys are *almost* arbitrary values. Values that are not :term:`hashable`, that is, values containing lists, dictionaries or other mutable types (that are compared by value rather than by object identity) may not be used as keys. Values that compare equal (such as ``1``, ``1.0``, and ``True``) can be used interchangeably to index the same dictionary entry. .. class:: dict(**kwargs) dict(mapping, /, **kwargs) dict(iterable, /, **kwargs) Return a new dictionary initialized from an optional positional argument and a possibly empty set of keyword arguments. Dictionaries can be created by several means: * Use a comma-separated list of ``key: value`` pairs within braces: ``{'jack': 4098, 'sjoerd': 4127}`` or ``{4098: 'jack', 4127: 'sjoerd'}`` * Use a dict comprehension: ``{}``, ``{x: x ** 2 for x in range(10)}`` * Use the type constructor: ``dict()``, ``dict([('foo', 100), ('bar', 200)])``, ``dict(foo=100, bar=200)`` If no positional argument is given, an empty dictionary is created. If a positional argument is given and it defines a ``keys()`` method, a dictionary is created by calling :meth:`~object.__getitem__` on the argument with each returned key from the method. Otherwise, the positional argument must be an :term:`iterable` object. Each item in the iterable must itself be an iterable with exactly two elements. The first element of each item becomes a key in the new dictionary, and the second element the corresponding value. If a key occurs more than once, the last value for that key becomes the corresponding value in the new dictionary. If keyword arguments are given, the keyword arguments and their values are added to the dictionary created from the positional argument. If a key being added is already present, the value from the keyword argument replaces the value from the positional argument. Dictionaries compare equal if and only if they have the same ``(key, value)`` pairs (regardless of ordering). Order comparisons ('<', '<=', '>=', '>') raise :exc:`TypeError`. To illustrate dictionary creation and equality, the following examples all return a dictionary equal to ``{"one": 1, "two": 2, "three": 3}``:: >>> a = dict(one=1, two=2, three=3) >>> b = {'one': 1, 'two': 2, 'three': 3} >>> c = dict(zip(['one', 'two', 'three'], [1, 2, 3])) >>> d = dict([('two', 2), ('one', 1), ('three', 3)]) >>> e = dict({'three': 3, 'one': 1, 'two': 2}) >>> f = dict({'one': 1, 'three': 3}, two=2) >>> a == b == c == d == e == f True Providing keyword arguments as in the first example only works for keys that are valid Python identifiers. Otherwise, any valid keys can be used. Dictionaries preserve insertion order. Note that updating a key does not affect the order. Keys added after deletion are inserted at the end. :: >>> d = {"one": 1, "two": 2, "three": 3, "four": 4} >>> d {'one': 1, 'two': 2, 'three': 3, 'four': 4} >>> list(d) ['one', 'two', 'three', 'four'] >>> list(d.values()) [1, 2, 3, 4] >>> d["one"] = 42 >>> d {'one': 42, 'two': 2, 'three': 3, 'four': 4} >>> del d["two"] >>> d["two"] = None >>> d {'one': 42, 'three': 3, 'four': 4, 'two': None} .. versionchanged:: 3.7 Dictionary order is guaranteed to be insertion order. This behavior was an implementation detail of CPython from 3.6. Dictionaries are :ref:`generic ` over two types, signifying (respectively) the types of the dictionary's keys and values. These are the operations that dictionaries support (and therefore, custom mapping types should support too): .. describe:: list(d) Return a list of all the keys used in the dictionary *d*. .. describe:: len(d) Return the number of items in the dictionary *d*. .. describe:: d[key] Return the item of *d* with key *key*. Raises a :exc:`KeyError` if *key* is not in the map. .. index:: __missing__() If a subclass of dict defines a method :meth:`~object.__missing__` and *key* is not present, the ``d[key]`` operation calls that method with the key *key* as argument. The ``d[key]`` operation then returns or raises whatever is returned or raised by the ``__missing__(key)`` call. No other operations or methods invoke :meth:`~object.__missing__`. If :meth:`~object.__missing__` is not defined, :exc:`KeyError` is raised. :meth:`~object.__missing__` must be a method; it cannot be an instance variable:: >>> class Counter(dict): ... def __missing__(self, key): ... return 0 ... >>> c = Counter() >>> c['red'] 0 >>> c['red'] += 1 >>> c['red'] 1 The example above shows part of the implementation of :class:`collections.Counter`. A different :meth:`!__missing__` method is used by :class:`collections.defaultdict`. .. describe:: d[key] = value Set ``d[key]`` to *value*. .. describe:: del d[key] Remove ``d[key]`` from *d*. Raises a :exc:`KeyError` if *key* is not in the map. .. describe:: key in d Return ``True`` if *d* has a key *key*, else ``False``. .. describe:: key not in d Equivalent to ``not key in d``. .. describe:: iter(d) Return an iterator over the keys of the dictionary. This is a shortcut for ``iter(d.keys())``. .. method:: clear() Remove all items from the dictionary. .. method:: copy() Return a shallow copy of the dictionary. .. classmethod:: fromkeys(iterable, value=None, /) Create a new dictionary with keys from *iterable* and values set to *value*. :meth:`fromkeys` is a class method that returns a new dictionary. *value* defaults to ``None``. All of the values refer to just a single instance, so it generally doesn't make sense for *value* to be a mutable object such as an empty list. To get distinct values, use a :ref:`dict comprehension ` instead. .. method:: get(key, default=None, /) Return the value for *key* if *key* is in the dictionary, else *default*. If *default* is not given, it defaults to ``None``, so that this method never raises a :exc:`KeyError`. .. method:: items() Return a new view of the dictionary's items (``(key, value)`` pairs). See the :ref:`documentation of view objects `. .. method:: keys() Return a new view of the dictionary's keys. See the :ref:`documentation of view objects `. .. method:: pop(key, /) pop(key, default, /) If *key* is in the dictionary, remove it and return its value, else return *default*. If *default* is not given and *key* is not in the dictionary, a :exc:`KeyError` is raised. .. method:: popitem() Remove and return a ``(key, value)`` pair from the dictionary. Pairs are returned in :abbr:`LIFO (last-in, first-out)` order. :meth:`popitem` is useful to destructively iterate over a dictionary, as often used in set algorithms. If the dictionary is empty, calling :meth:`popitem` raises a :exc:`KeyError`. .. versionchanged:: 3.7 LIFO order is now guaranteed. In prior versions, :meth:`popitem` would return an arbitrary key/value pair. .. describe:: reversed(d) Return a reverse iterator over the keys of the dictionary. This is a shortcut for ``reversed(d.keys())``. .. versionadded:: 3.8 .. method:: setdefault(key, default=None, /) If *key* is in the dictionary, return its value. If not, insert *key* with a value of *default* and return *default*. *default* defaults to ``None``. .. method:: update(**kwargs) update(mapping, /, **kwargs) update(iterable, /, **kwargs) Update the dictionary with the key/value pairs from *mapping* or *iterable* and *kwargs*, overwriting existing keys. Return ``None``. :meth:`update` accepts either another object with a ``keys()`` method (in which case :meth:`~object.__getitem__` is called with every key returned from the method) or an iterable of key/value pairs (as tuples or other iterables of length two). If keyword arguments are specified, the dictionary is then updated with those key/value pairs: ``d.update(red=1, blue=2)``. .. method:: values() Return a new view of the dictionary's values. See the :ref:`documentation of view objects `. An equality comparison between one ``dict.values()`` view and another will always return ``False``. This also applies when comparing ``dict.values()`` to itself:: >>> d = {'a': 1} >>> d.values() == d.values() False .. describe:: d | other Create a new dictionary with the merged keys and values of *d* and *other*, which must both be dictionaries. The values of *other* take priority when *d* and *other* share keys. .. versionadded:: 3.9 .. describe:: d |= other Update the dictionary *d* with keys and values from *other*, which may be either a :term:`mapping` or an :term:`iterable` of key/value pairs. The values of *other* take priority when *d* and *other* share keys. .. versionadded:: 3.9 Dictionaries and dictionary views are reversible. :: >>> d = {"one": 1, "two": 2, "three": 3, "four": 4} >>> d {'one': 1, 'two': 2, 'three': 3, 'four': 4} >>> list(reversed(d)) ['four', 'three', 'two', 'one'] >>> list(reversed(d.values())) [4, 3, 2, 1] >>> list(reversed(d.items())) [('four', 4), ('three', 3), ('two', 2), ('one', 1)] .. versionchanged:: 3.8 Dictionaries are now reversible. .. seealso:: :class:`frozendict` and :class:`types.MappingProxyType` can be used to create a read-only view of a :class:`dict`. .. seealso:: For detailed information on thread-safety guarantees for :class:`dict` objects, see :ref:`thread-safety-dict`. .. _dict-views: Dictionary view objects ----------------------- The objects returned by :meth:`dict.keys`, :meth:`dict.values` and :meth:`dict.items` are *view objects*. They provide a dynamic view on the dictionary's entries, which means that when the dictionary changes, the view reflects these changes. Dictionary views can be iterated over to yield their respective data, and support membership tests: .. describe:: len(dictview) Return the number of entries in the dictionary. .. describe:: iter(dictview) Return an iterator over the keys, values or items (represented as tuples of ``(key, value)``) in the dictionary. Keys and values are iterated over in insertion order. This allows the creation of ``(value, key)`` pairs using :func:`zip`: ``pairs = zip(d.values(), d.keys())``. Another way to create the same list is ``pairs = [(v, k) for (k, v) in d.items()]``. Iterating views while adding or deleting entries in the dictionary may raise a :exc:`RuntimeError` or fail to iterate over all entries. .. versionchanged:: 3.7 Dictionary order is guaranteed to be insertion order. .. describe:: x in dictview Return ``True`` if *x* is in the underlying dictionary's keys, values or items (in the latter case, *x* should be a ``(key, value)`` tuple). .. describe:: reversed(dictview) Return a reverse iterator over the keys, values or items of the dictionary. The view will be iterated in reverse order of the insertion. .. versionchanged:: 3.8 Dictionary views are now reversible. .. describe:: dictview.mapping Return a :class:`types.MappingProxyType` that wraps the original dictionary to which the view refers. .. versionadded:: 3.10 Keys views are set-like since their entries are unique and :term:`hashable`. Items views also have set-like operations since the (key, value) pairs are unique and the keys are hashable. If all values in an items view are hashable as well, then the items view can interoperate with other sets. (Values views are not treated as set-like since the entries are generally not unique.) For set-like views, all of the operations defined for the abstract base class :class:`collections.abc.Set` are available (for example, ``==``, ``<``, or ``^``). While using set operators, set-like views accept any iterable as the other operand, unlike sets which only accept sets as the input. An example of dictionary view usage:: >>> dishes = {'eggs': 2, 'sausage': 1, 'bacon': 1, 'spam': 500} >>> keys = dishes.keys() >>> values = dishes.values() >>> # iteration >>> n = 0 >>> for val in values: ... n += val ... >>> print(n) 504 >>> # keys and values are iterated over in the same order (insertion order) >>> list(keys) ['eggs', 'sausage', 'bacon', 'spam'] >>> list(values) [2, 1, 1, 500] >>> # view objects are dynamic and reflect dict changes >>> del dishes['eggs'] >>> del dishes['sausage'] >>> list(keys) ['bacon', 'spam'] >>> # set operations >>> keys & {'eggs', 'bacon', 'salad'} {'bacon'} >>> keys ^ {'sausage', 'juice'} == {'juice', 'sausage', 'bacon', 'spam'} True >>> keys | ['juice', 'juice', 'juice'] == {'bacon', 'spam', 'juice'} True >>> # get back a read-only proxy for the original dictionary >>> values.mapping mappingproxy({'bacon': 1, 'spam': 500}) >>> values.mapping['spam'] 500 Frozen dictionaries ------------------- .. class:: frozendict(**kwargs) frozendict(mapping, /, **kwargs) frozendict(iterable, /, **kwargs) Return a new frozen dictionary initialized from an optional positional argument and a possibly empty set of keyword arguments. A :class:`!frozendict` has a similar API to the :class:`dict` API, with the following differences: * :class:`!dict` has more methods than :class:`!frozendict`: * :meth:`!__delitem__` * :meth:`!__setitem__` * :meth:`~dict.clear` * :meth:`~dict.pop` * :meth:`~dict.popitem` * :meth:`~dict.setdefault` * :meth:`~dict.update` * A :class:`!frozendict` can be hashed with ``hash(frozendict)`` if all keys and values can be hashed. * ``frozendict |= other`` does not modify the :class:`!frozendict` in-place but creates a new frozen dictionary. :class:`!frozendict` is not a :class:`!dict` subclass but inherits directly from ``object``. Like dictionaries, frozendicts are :ref:`generic ` over two types, signifying (respectively) the types of the frozendict's keys and values. .. classmethod:: fromkeys(iterable, value=None, /) Similar to :meth:`dict.fromkeys`, but call again the type constructor with an initialized :class:`frozendict` if the type is a :class:`frozendict` subclass or if the constructor returned a :class:`frozendict`. .. versionadded:: 3.15 .. _typecontextmanager: Context Manager Types ===================== .. index:: single: context manager single: context management protocol single: protocol; context management Python's :keyword:`with` statement supports the concept of a runtime context defined by a context manager. This is implemented using a pair of methods that allow user-defined classes to define a runtime context that is entered before the statement body is executed and exited when the statement ends: .. method:: contextmanager.__enter__() Enter the runtime context and return either this object or another object related to the runtime context. The value returned by this method is bound to the identifier in the :keyword:`!as` clause of :keyword:`with` statements using this context manager. An example of a context manager that returns itself is a :term:`file object`. File objects return themselves from __enter__() to allow :func:`open` to be used as the context expression in a :keyword:`with` statement. An example of a context manager that returns a related object is the one returned by :func:`decimal.localcontext`. These managers set the active decimal context to a copy of the original decimal context and then return the copy. This allows changes to be made to the current decimal context in the body of the :keyword:`with` statement without affecting code outside the :keyword:`!with` statement. .. method:: contextmanager.__exit__(exc_type, exc_val, exc_tb) Exit the runtime context and return a Boolean flag indicating if any exception that occurred should be suppressed. If an exception occurred while executing the body of the :keyword:`with` statement, the arguments contain the exception type, value and traceback information. Otherwise, all three arguments are ``None``. Returning a true value from this method will cause the :keyword:`with` statement to suppress the exception and continue execution with the statement immediately following the :keyword:`!with` statement. Otherwise the exception continues propagating after this method has finished executing. If this method raises an exception while handling an earlier exception from the :keyword:`with` block, the new exception is raised, and the original exception is stored in its :attr:`~BaseException.__context__` attribute. The exception passed in should never be reraised explicitly - instead, this method should return a false value to indicate that the method completed successfully and does not want to suppress the raised exception. This allows context management code to easily detect whether or not an :meth:`~object.__exit__` method has actually failed. Python defines several context managers to support easy thread synchronisation, prompt closure of files or other objects, and simpler manipulation of the active decimal arithmetic context. The specific types are not treated specially beyond their implementation of the context management protocol. See the :mod:`contextlib` module for some examples. Python's :term:`generator`\s and the :class:`contextlib.contextmanager` decorator provide a convenient way to implement these protocols. If a generator function is decorated with the :class:`contextlib.contextmanager` decorator, it will return a context manager implementing the necessary :meth:`~contextmanager.__enter__` and :meth:`~contextmanager.__exit__` methods, rather than the iterator produced by an undecorated generator function. Note that there is no specific slot for any of these methods in the type structure for Python objects in the Python/C API. Extension types wanting to define these methods must provide them as a normal Python accessible method. Compared to the overhead of setting up the runtime context, the overhead of a single class dictionary lookup is negligible. Type Annotation Types --- :ref:`Generic Alias `, :ref:`Union ` =============================================================================================== .. index:: single: annotation; type annotation; type hint The core built-in types for :term:`type annotations ` are :ref:`Generic Alias ` and :ref:`Union `. .. _types-genericalias: Generic Alias Type ------------------ .. index:: pair: object; GenericAlias pair: Generic; Alias ``GenericAlias`` objects are generally created by :ref:`subscripting ` a class. They are most often used with :ref:`container classes `, such as :class:`list` or :class:`dict`. For example, ``list[int]`` is a ``GenericAlias`` object created by subscripting the ``list`` class with the argument :class:`int`. ``GenericAlias`` objects are intended primarily for use with :term:`type annotations `. .. note:: It is generally only possible to subscript a class if the class implements the special method :meth:`~object.__class_getitem__`. A ``GenericAlias`` object acts as a proxy for a :term:`generic type`, implementing *parameterized generics*. For a container class, the argument(s) supplied to a :ref:`subscription ` of the class may indicate the type(s) of the elements an object contains. For example, ``set[bytes]`` can be used in type annotations to signify a :class:`set` in which all the elements are of type :class:`bytes`. For a class which defines :meth:`~object.__class_getitem__` but is not a container, the argument(s) supplied to a subscription of the class will often indicate the return type(s) of one or more methods defined on an object. For example, :mod:`regular expressions ` can be used on both the :class:`str` data type and the :class:`bytes` data type: * If ``x = re.search('foo', 'foo')``, ``x`` will be a :ref:`re.Match ` object where the return values of ``x.group(0)`` and ``x[0]`` will both be of type :class:`str`. We can represent this kind of object in type annotations with the ``GenericAlias`` ``re.Match[str]``. * If ``y = re.search(b'bar', b'bar')``, (note the ``b`` for :class:`bytes`), ``y`` will also be an instance of ``re.Match``, but the return values of ``y.group(0)`` and ``y[0]`` will both be of type :class:`bytes`. In type annotations, we would represent this variety of :ref:`re.Match ` objects with ``re.Match[bytes]``. ``GenericAlias`` objects are instances of the class :class:`types.GenericAlias`, which can also be used to create ``GenericAlias`` objects directly. Specializations of user-defined :ref:`generic classes ` may not be instances of :class:`types.GenericAlias`, but they provide similar functionality. .. describe:: T[X, Y, ...] Creates a ``GenericAlias`` representing a type ``T`` parameterized by types *X*, *Y*, and more depending on the ``T`` used. For example, a function expecting a :class:`list` containing :class:`float` elements:: def average(values: list[float]) -> float: return sum(values) / len(values) Another example for :term:`mapping` objects, using a :class:`dict`, which is a generic type expecting two type parameters representing the key type and the value type. In this example, the function expects a ``dict`` with keys of type :class:`str` and values of type :class:`int`:: def send_post_request(url: str, body: dict[str, int]) -> None: ... The builtin functions :func:`isinstance` and :func:`issubclass` do not accept ``GenericAlias`` types for their second argument:: >>> isinstance([1, 2], list[str]) Traceback (most recent call last): File "", line 1, in TypeError: isinstance() argument 2 cannot be a parameterized generic The Python runtime does not enforce :term:`type annotations `. This extends to generic types and their type parameters. When creating a container object from a ``GenericAlias``, the elements in the container are not checked against their type. For example, the following code is discouraged, but will run without errors:: >>> t = list[str] >>> t([1, 2, 3]) [1, 2, 3] Furthermore, parameterized generics erase type parameters during object creation:: >>> t = list[str] >>> type(t) >>> l = t() >>> type(l) Instances of ``GenericAlias`` are not classes at runtime, even though they behave like classes (they can be instantiated and subclassed):: >>> import inspect >>> inspect.isclass(list[int]) False This is true for :ref:`user-defined generics ` also. Calling :func:`repr` or :func:`str` on a generic shows the parameterized type:: >>> repr(list[int]) 'list[int]' >>> str(list[int]) 'list[int]' The :meth:`~object.__getitem__` method of generic containers will raise an exception to disallow mistakes like ``dict[str][str]``:: >>> dict[str][str] Traceback (most recent call last): ... TypeError: dict[str] is not a generic class However, such expressions are valid when :ref:`type variables ` are used. The index must have as many elements as there are type variable items in the ``GenericAlias`` object's :attr:`~genericalias.__args__`. :: >>> from typing import TypeVar >>> Y = TypeVar('Y') >>> dict[str, Y][int] dict[str, int] Standard Generic Classes ^^^^^^^^^^^^^^^^^^^^^^^^ The following standard library classes support parameterized generics. This list is non-exhaustive. * :class:`tuple` * :class:`list` * :class:`dict` * :class:`set` * :class:`frozendict` * :class:`frozenset` * :class:`type` * :class:`asyncio.Future` * :class:`asyncio.Task` * :class:`collections.deque` * :class:`collections.defaultdict` * :class:`collections.OrderedDict` * :class:`collections.Counter` * :class:`collections.ChainMap` * :class:`collections.abc.Awaitable` * :class:`collections.abc.Coroutine` * :class:`collections.abc.AsyncIterable` * :class:`collections.abc.AsyncIterator` * :class:`collections.abc.AsyncGenerator` * :class:`collections.abc.Iterable` * :class:`collections.abc.Iterator` * :class:`collections.abc.Generator` * :class:`collections.abc.Reversible` * :class:`collections.abc.Container` * :class:`collections.abc.Collection` * :class:`collections.abc.Callable` * :class:`collections.abc.Set` * :class:`collections.abc.MutableSet` * :class:`collections.abc.Mapping` * :class:`collections.abc.MutableMapping` * :class:`collections.abc.Sequence` * :class:`collections.abc.MutableSequence` * :class:`collections.abc.ByteString` * :class:`collections.abc.MappingView` * :class:`collections.abc.KeysView` * :class:`collections.abc.ItemsView` * :class:`collections.abc.ValuesView` * :class:`contextlib.AbstractContextManager` * :class:`contextlib.AbstractAsyncContextManager` * :class:`dataclasses.Field` * :class:`functools.cached_property` * :class:`functools.partialmethod` * :class:`os.PathLike` * :class:`queue.LifoQueue` * :class:`queue.Queue` * :class:`queue.PriorityQueue` * :class:`queue.SimpleQueue` * :ref:`re.Pattern ` * :ref:`re.Match ` * :class:`shelve.BsdDbShelf` * :class:`shelve.DbfilenameShelf` * :class:`shelve.Shelf` * :class:`types.MappingProxyType` * :class:`weakref.WeakKeyDictionary` * :class:`weakref.WeakMethod` * :class:`weakref.WeakSet` * :class:`weakref.WeakValueDictionary` Special Attributes of ``GenericAlias`` objects ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ All parameterized generics implement special read-only attributes. .. attribute:: genericalias.__origin__ This attribute points at the non-parameterized generic class:: >>> list[int].__origin__ .. attribute:: genericalias.__args__ This attribute is a :class:`tuple` (possibly of length 1) of generic types passed to the original :meth:`~object.__class_getitem__` of the generic class:: >>> dict[str, list[int]].__args__ (, list[int]) .. attribute:: genericalias.__parameters__ This attribute is a lazily computed tuple (possibly empty) of unique type variables found in ``__args__``:: >>> from typing import TypeVar >>> T = TypeVar('T') >>> list[T].__parameters__ (~T,) .. note:: A ``GenericAlias`` object with :class:`typing.ParamSpec` parameters may not have correct ``__parameters__`` after substitution because :class:`typing.ParamSpec` is intended primarily for static type checking. .. attribute:: genericalias.__unpacked__ A boolean that is true if the alias has been unpacked using the ``*`` operator (see :data:`~typing.TypeVarTuple`). .. versionadded:: 3.11 .. seealso:: :pep:`484` - Type Hints Introducing Python's framework for type annotations. :pep:`585` - Type Hinting Generics In Standard Collections Introducing the ability to natively parameterize standard-library classes, provided they implement the special class method :meth:`~object.__class_getitem__`. :ref:`Generics`, :ref:`user-defined generics ` and :class:`typing.Generic` Documentation on how to implement generic classes that can be parameterized at runtime and understood by static type-checkers. .. versionadded:: 3.9 .. _types-union: Union Type ---------- .. index:: pair: object; Union pair: union; type A union object holds the value of the ``|`` (bitwise or) operation on multiple :ref:`type objects `. These types are intended primarily for :term:`type annotations `. The union type expression enables cleaner type hinting syntax compared to subscripting :class:`typing.Union`. .. describe:: X | Y | ... Defines a union object which holds types *X*, *Y*, and so forth. ``X | Y`` means either X or Y. It is equivalent to ``typing.Union[X, Y]``. For example, the following function expects an argument of type :class:`int` or :class:`float`:: def square(number: int | float) -> int | float: return number ** 2 .. note:: The ``|`` operator cannot be used at runtime to define unions where one or more members is a forward reference. For example, ``int | "Foo"``, where ``"Foo"`` is a reference to a class not yet defined, will fail at runtime. For unions which include forward references, present the whole expression as a string, e.g. ``"int | Foo"``. .. describe:: union_object == other Union objects can be tested for equality with other union objects. Details: * Unions of unions are flattened:: (int | str) | float == int | str | float * Redundant types are removed:: int | str | int == int | str * When comparing unions, the order is ignored:: int | str == str | int * It creates instances of :class:`typing.Union`:: int | str == typing.Union[int, str] type(int | str) is typing.Union * Optional types can be spelled as a union with ``None``:: str | None == typing.Optional[str] .. describe:: isinstance(obj, union_object) .. describe:: issubclass(obj, union_object) Calls to :func:`isinstance` and :func:`issubclass` are also supported with a union object:: >>> isinstance("", int | str) True However, :ref:`parameterized generics ` in union objects cannot be checked:: >>> isinstance(1, int | list[int]) # short-circuit evaluation True >>> isinstance([1], int | list[int]) Traceback (most recent call last): ... TypeError: isinstance() argument 2 cannot be a parameterized generic The user-exposed type for the union object can be accessed from :class:`typing.Union` and used for :func:`isinstance` checks:: >>> import typing >>> isinstance(int | str, typing.Union) True >>> typing.Union() Traceback (most recent call last): File "", line 1, in TypeError: cannot create 'typing.Union' instances .. note:: The :meth:`!__or__` method for type objects was added to support the syntax ``X | Y``. If a metaclass implements :meth:`!__or__`, the Union may override it: .. doctest:: >>> class M(type): ... def __or__(self, other): ... return "Hello" ... >>> class C(metaclass=M): ... pass ... >>> C | int 'Hello' >>> int | C int | C .. seealso:: :pep:`604` -- PEP proposing the ``X | Y`` syntax and the Union type. .. versionadded:: 3.10 .. versionchanged:: 3.14 Union objects are now instances of :class:`typing.Union`. Previously, they were instances of :class:`types.UnionType`, which remains an alias for :class:`typing.Union`. .. _typesother: Other Built-in Types ==================== The interpreter supports several other kinds of objects. Most of these support only one or two operations. .. _typesmodules: Modules ------- The only special operation on a module is attribute access: ``m.name``, where *m* is a module and *name* accesses a name defined in *m*'s symbol table. Module attributes can be assigned to. (Note that the :keyword:`import` statement is not, strictly speaking, an operation on a module object; ``import foo`` does not require a module object named *foo* to exist, rather it requires an (external) *definition* for a module named *foo* somewhere.) A special attribute of every module is :attr:`~object.__dict__`. This is the dictionary containing the module's symbol table. Modifying this dictionary will actually change the module's symbol table, but direct assignment to the :attr:`~object.__dict__` attribute is not possible (you can write ``m.__dict__['a'] = 1``, which defines ``m.a`` to be ``1``, but you can't write ``m.__dict__ = {}``). Modifying :attr:`~object.__dict__` directly is not recommended. Modules built into the interpreter are written like this: ````. If loaded from a file, they are written as ````. .. _typesobjects: Classes and Class Instances --------------------------- See :ref:`objects` and :ref:`class` for these. .. _typesfunctions: Functions --------- Function objects are created by function definitions. The only operation on a function object is to call it: ``func(argument-list)``. There are really two flavors of function objects: built-in functions and user-defined functions. Both support the same operation (to call the function), but the implementation is different, hence the different object types. See :ref:`function` for more information. .. _typesmethods: Methods ------- .. index:: pair: object; method Methods are functions that are called using the attribute notation. There are two flavors: :ref:`built-in methods ` (such as :meth:`~list.append` on lists) and :ref:`class instance methods `. Built-in methods are described with the types that support them. If you access a method (a function defined in a class namespace) through an instance, you get a special object: a :dfn:`bound method` (also called :ref:`instance method `) object. When called, it will add the ``self`` argument to the argument list. Bound methods have two special read-only attributes: :attr:`m.__self__ ` is the object on which the method operates, and :attr:`m.__func__ ` is the function implementing the method. Calling ``m(arg-1, arg-2, ..., arg-n)`` is completely equivalent to calling ``m.__func__(m.__self__, arg-1, arg-2, ..., arg-n)``. Like :ref:`function objects `, bound method objects support getting arbitrary attributes. However, since method attributes are actually stored on the underlying function object (:attr:`method.__func__`), setting method attributes on bound methods is disallowed. Attempting to set an attribute on a method results in an :exc:`AttributeError` being raised. In order to set a method attribute, you need to explicitly set it on the underlying function object: .. doctest:: >>> class C: ... def method(self): ... pass ... >>> c = C() >>> c.method.whoami = 'my name is method' # can't set on the method Traceback (most recent call last): File "", line 1, in AttributeError: 'method' object has no attribute 'whoami' >>> c.method.__func__.whoami = 'my name is method' >>> c.method.whoami 'my name is method' See :ref:`instance-methods` for more information. .. index:: object; code, code object .. _bltin-code-objects: Code Objects ------------ .. index:: pair: built-in function; compile single: __code__ (function object attribute) Code objects are used by the implementation to represent "pseudo-compiled" executable Python code such as a function body. They differ from function objects because they don't contain a reference to their global execution environment. Code objects are returned by the built-in :func:`compile` function and can be extracted from function objects through their :attr:`~function.__code__` attribute. See also the :mod:`code` module. Accessing :attr:`~function.__code__` raises an :ref:`auditing event ` ``object.__getattr__`` with arguments ``obj`` and ``"__code__"``. .. index:: pair: built-in function; exec pair: built-in function; eval A code object can be executed or evaluated by passing it (instead of a source string) to the :func:`exec` or :func:`eval` built-in functions. See :ref:`types` for more information. .. _bltin-type-objects: Type Objects ------------ .. index:: pair: built-in function; type pair: module; types Type objects represent the various object types. An object's type is accessed by the built-in function :func:`type`. There are no special operations on types. The standard module :mod:`types` defines names for all standard built-in types. Types are written like this: ````. .. _bltin-null-object: The Null Object --------------- This object is returned by functions that don't explicitly return a value. It supports no special operations. There is exactly one null object, named ``None`` (a built-in name). ``type(None)()`` produces the same singleton. It is written as ``None``. .. index:: single: ...; ellipsis literal .. _bltin-ellipsis-object: The Ellipsis Object ------------------- This object is commonly used to indicate that something is omitted. It supports no special operations. There is exactly one ellipsis object, named :const:`Ellipsis` (a built-in name). ``type(Ellipsis)()`` produces the :const:`Ellipsis` singleton. It is written as ``Ellipsis`` or ``...``. In typical use, ``...`` as the ``Ellipsis`` object appears in a few different places, for instance: - In type annotations, such as :ref:`callable arguments ` or :ref:`tuple elements `. - As the body of a function instead of a :ref:`pass statement `. - In third-party libraries, such as `Numpy's slicing and striding `_. Python also uses three dots in ways that are not ``Ellipsis`` objects, for instance: - Doctest's :const:`ELLIPSIS `, as a pattern for missing content. - The default Python prompt of the :term:`interactive` shell when partial input is incomplete. Lastly, the Python documentation often uses three dots in conventional English usage to mean omitted content, even in code examples that also use them as the ``Ellipsis``. .. _bltin-notimplemented-object: The NotImplemented Object ------------------------- This object is returned from comparisons and binary operations when they are asked to operate on types they don't support. See :ref:`comparisons` for more information. There is exactly one :data:`NotImplemented` object. :code:`type(NotImplemented)()` produces the singleton instance. It is written as :code:`NotImplemented`. .. _typesinternal: Internal Objects ---------------- See :ref:`types` for this information. It describes :ref:`stack frame objects `, :ref:`traceback objects `, and slice objects. .. _specialattrs: Special Attributes ================== The implementation adds a few special read-only attributes to several object types, where they are relevant. Some of these are not reported by the :func:`dir` built-in function. .. attribute:: definition.__name__ The name of the class, function, method, descriptor, or generator instance. .. attribute:: definition.__qualname__ The :term:`qualified name` of the class, function, method, descriptor, or generator instance. .. versionadded:: 3.3 .. attribute:: definition.__module__ The name of the module in which a class or function was defined. .. attribute:: definition.__doc__ The documentation string of a class or function, or ``None`` if undefined. .. attribute:: definition.__type_params__ The :ref:`type parameters ` of generic classes, functions, and :ref:`type aliases `. For classes and functions that are not generic, this will be an empty tuple. .. versionadded:: 3.12 .. _int_max_str_digits: Integer string conversion length limitation =========================================== CPython has a global limit for converting between :class:`int` and :class:`str` to mitigate denial of service attacks. This limit *only* applies to decimal or other non-power-of-two number bases. Hexadecimal, octal, and binary conversions are unlimited. The limit can be configured. The :class:`int` type in CPython is an arbitrary length number stored in binary form (commonly known as a "bignum"). There exists no algorithm that can convert a string to a binary integer or a binary integer to a string in linear time, *unless* the base is a power of 2. Even the best known algorithms for base 10 have sub-quadratic complexity. Converting a large value such as ``int('1' * 500_000)`` can take over a second on a fast CPU. Limiting conversion size offers a practical way to avoid :cve:`2020-10735`. The limit is applied to the number of digit characters in the input or output string when a non-linear conversion algorithm would be involved. Underscores and the sign are not counted towards the limit. When an operation would exceed the limit, a :exc:`ValueError` is raised: .. doctest:: >>> import sys >>> sys.set_int_max_str_digits(4300) # Illustrative, this is the default. >>> _ = int('2' * 5432) Traceback (most recent call last): ... ValueError: Exceeds the limit (4300 digits) for integer string conversion: value has 5432 digits; use sys.set_int_max_str_digits() to increase the limit >>> i = int('2' * 4300) >>> len(str(i)) 4300 >>> i_squared = i*i >>> len(str(i_squared)) Traceback (most recent call last): ... ValueError: Exceeds the limit (4300 digits) for integer string conversion; use sys.set_int_max_str_digits() to increase the limit >>> len(hex(i_squared)) 7144 >>> assert int(hex(i_squared), base=16) == i*i # Hexadecimal is unlimited. The default limit is 4300 digits as provided in :data:`sys.int_info.default_max_str_digits `. The lowest limit that can be configured is 640 digits as provided in :data:`sys.int_info.str_digits_check_threshold `. Verification: .. doctest:: >>> import sys >>> assert sys.int_info.default_max_str_digits == 4300, sys.int_info >>> assert sys.int_info.str_digits_check_threshold == 640, sys.int_info >>> msg = int('578966293710682886880994035146873798396722250538762761564' ... '9252925514383915483333812743580549779436104706260696366600' ... '571186405732').to_bytes(53, 'big') ... .. versionadded:: 3.11 Affected APIs ------------- The limitation only applies to potentially slow conversions between :class:`int` and :class:`str` or :class:`bytes`: * ``int(string)`` with default base 10. * ``int(string, base)`` for all bases that are not a power of 2. * ``str(integer)``. * ``repr(integer)``. * any other string conversion to base 10, for example ``f"{integer}"``, ``"{}".format(integer)``, or ``b"%d" % integer``. The limitations do not apply to functions with a linear algorithm: * ``int(string, base)`` with base 2, 4, 8, 16, or 32. * :func:`int.from_bytes` and :func:`int.to_bytes`. * :func:`hex`, :func:`oct`, :func:`bin`. * :ref:`formatspec` for hex, octal, and binary numbers. * :class:`str` to :class:`float`. * :class:`str` to :class:`decimal.Decimal`. Configuring the limit --------------------- Before Python starts up you can use an environment variable or an interpreter command line flag to configure the limit: * :envvar:`PYTHONINTMAXSTRDIGITS`, e.g. ``PYTHONINTMAXSTRDIGITS=640 python3`` to set the limit to 640 or ``PYTHONINTMAXSTRDIGITS=0 python3`` to disable the limitation. * :option:`-X int_max_str_digits <-X>`, e.g. ``python3 -X int_max_str_digits=640`` * :data:`sys.flags.int_max_str_digits` contains the value of :envvar:`PYTHONINTMAXSTRDIGITS` or :option:`-X int_max_str_digits <-X>`. If both the env var and the ``-X`` option are set, the ``-X`` option takes precedence. A value of *-1* indicates that both were unset, thus a value of :data:`sys.int_info.default_max_str_digits` was used during initialization. From code, you can inspect the current limit and set a new one using these :mod:`sys` APIs: * :func:`sys.get_int_max_str_digits` and :func:`sys.set_int_max_str_digits` are a getter and setter for the interpreter-wide limit. Subinterpreters have their own limit. Information about the default and minimum can be found in :data:`sys.int_info`: * :data:`sys.int_info.default_max_str_digits ` is the compiled-in default limit. * :data:`sys.int_info.str_digits_check_threshold ` is the lowest accepted value for the limit (other than 0 which disables it). .. versionadded:: 3.11 .. caution:: Setting a low limit *can* lead to problems. While rare, code exists that contains integer constants in decimal in their source that exceed the minimum threshold. A consequence of setting the limit is that Python source code containing decimal integer literals longer than the limit will encounter an error during parsing, usually at startup time or import time or even at installation time - anytime an up to date ``.pyc`` does not already exist for the code. A workaround for source that contains such large constants is to convert them to ``0x`` hexadecimal form as it has no limit. Test your application thoroughly if you use a low limit. Ensure your tests run with the limit set early via the environment or flag so that it applies during startup and even during any installation step that may invoke Python to precompile ``.py`` sources to ``.pyc`` files. Recommended configuration ------------------------- The default :data:`sys.int_info.default_max_str_digits` is expected to be reasonable for most applications. If your application requires a different limit, set it from your main entry point using Python version agnostic code as these APIs were added in security patch releases in versions before 3.12. Example:: >>> import sys >>> if hasattr(sys, "set_int_max_str_digits"): ... upper_bound = 68000 ... lower_bound = 4004 ... current_limit = sys.get_int_max_str_digits() ... if current_limit == 0 or current_limit > upper_bound: ... sys.set_int_max_str_digits(upper_bound) ... elif current_limit < lower_bound: ... sys.set_int_max_str_digits(lower_bound) If you need to disable it entirely, set it to ``0``. .. rubric:: Footnotes .. [1] Additional information on these special methods may be found in the Python Reference Manual (:ref:`customization`). .. [2] As a consequence, the list ``[1, 2]`` is considered equal to ``[1.0, 2.0]``, and similarly for tuples. .. [3] They must have since the parser can't tell the type of the operands. .. [4] Cased characters are those with general category property being one of "Lu" (Letter, uppercase), "Ll" (Letter, lowercase), or "Lt" (Letter, titlecase). .. [5] To format only a tuple you should therefore provide a singleton tuple whose only element is the tuple to be formatted. --- ### Doc Builtins Threadsafety (`Doc/builtins/threadsafety.rst`) .. _threadsafety: ************************ Thread Safety Guarantees ************************ This page documents thread-safety guarantees for built-in types in Python's free-threaded build. The guarantees described here apply when using Python with the :term:`GIL` disabled (free-threaded mode). When the GIL is enabled, most operations are implicitly serialized. For general guidance on writing thread-safe code in free-threaded Python, see :ref:`freethreading-python-howto`. .. _threadsafety-levels: Thread safety levels ==================== The C API documentation uses the following levels to describe the thread safety guarantees of each function. The levels are listed from least to most safe. .. _threadsafety-level-incompatible: Incompatible ------------ A function or operation that cannot be made safe for concurrent use even with external synchronization. Incompatible code typically accesses global state in an unsynchronized way and must only be called from a single thread throughout the program's lifetime. Example: a function that modifies process-wide state such as signal handlers or environment variables, where concurrent calls from any threads, even with external locking, can conflict with the runtime or other libraries. .. _threadsafety-level-compatible: Compatible ---------- A function or operation that is safe to call from multiple threads *provided* the caller supplies appropriate external synchronization, for example by holding a :term:`lock` for the duration of each call. Without such synchronization, concurrent calls may produce :term:`race conditions ` or :term:`data races `. Example: a function that reads from or writes to an object whose internal state is not protected by a lock. Callers must ensure that no two threads access the same object at the same time. .. _threadsafety-level-distinct: Safe on distinct objects ------------------------ A function or operation that is safe to call from multiple threads without external synchronization, as long as each thread operates on a **different** object. Two threads may call the function at the same time, but they must not pass the same object (or objects that share underlying state) as arguments. Example: a function that modifies fields of a struct using non-atomic writes. Two threads can each call the function on their own struct instance safely, but concurrent calls on the *same* instance require external synchronization. .. _threadsafety-level-shared: Safe on shared objects ---------------------- A function or operation that is safe for concurrent use on the **same** object. The implementation uses internal synchronization (such as :term:`per-object locks ` or :ref:`critical sections `) to protect shared mutable state, so callers do not need to supply their own locking. Example: :c:func:`PyList_GetItemRef` can be called from multiple threads on the same :c:type:`PyListObject` - it uses internal synchronization to serialize access. .. _threadsafety-level-atomic: Atomic ------ A function or operation that appears :term:`atomic ` with respect to other threads - it executes instantaneously from the perspective of other threads. This is the strongest form of thread safety. Example: :c:func:`PyMutex_IsLocked` performs an atomic read of the mutex state and can be called from any thread at any time. .. _thread-safety-list: Thread safety for list objects ============================== Reading a single element from a :class:`list` is :term:`atomic `: .. code-block:: :class: good lst[i] # list.__getitem__ The following methods traverse the list and use :term:`atomic ` reads of each item to perform their function. That means that they may return results affected by concurrent modifications: .. code-block:: :class: maybe item in lst lst.index(item) lst.count(item) All of the above operations avoid acquiring :term:`per-object locks `. They do not block concurrent modifications. Other operations that hold a lock will not block these from observing intermediate states. All other operations from here on block using the :term:`per-object lock`. Writing a single item via ``lst[i] = x`` is safe to call from multiple threads and will not corrupt the list. The following operations return new objects and appear :term:`atomic ` to other threads: .. code-block:: :class: good lst1 + lst2 # concatenates two lists into a new list x * lst # repeats lst x times into a new list lst.copy() # returns a shallow copy of the list The following methods that only operate on a single element with no shifting required are :term:`atomic `: .. code-block:: :class: good lst.append(x) # append to the end of the list, no shifting required lst.pop() # pop element from the end of the list, no shifting required The :meth:`~list.clear` method is also :term:`atomic `. Other threads cannot observe elements being removed. The :meth:`~list.sort` method is not :term:`atomic `. Other threads cannot observe intermediate states during sorting, but the list appears empty for the duration of the sort. The following operations may allow :term:`lock-free` operations to observe intermediate states since they modify multiple elements in place: .. code-block:: :class: maybe lst.insert(idx, item) # shifts elements lst.pop(idx) # idx not at the end of the list, shifts elements lst *= x # copies elements in place The :meth:`~list.remove` method may allow concurrent modifications since element comparison may execute arbitrary Python code (via :meth:`~object.__eq__`). :meth:`~list.extend` is safe to call from multiple threads. However, its guarantees depend on the iterable passed to it. If it is a :class:`list`, a :class:`tuple`, a :class:`set`, a :class:`frozenset`, a :class:`dict` or a :ref:`dictionary view object ` (but not their subclasses), the ``extend`` operation is safe from concurrent modifications to the iterable. Otherwise, an iterator is created which can be concurrently modified by another thread. The same applies to inplace concatenation of a list with other iterables when using ``lst += iterable``. Similarly, assigning to a list slice with ``lst[i:j] = iterable`` is safe to call from multiple threads, but ``iterable`` is only locked when it is also a :class:`list` (but not its subclasses). Operations that involve multiple accesses, as well as iteration, are never atomic. For example: .. code-block:: :class: bad # NOT atomic: read-modify-write lst[i] = lst[i] + 1 # NOT atomic: check-then-act if lst: item = lst.pop() # NOT thread-safe: iteration while modifying for item in lst: process(item) # another thread may modify lst Consider external synchronization when sharing :class:`list` instances across threads. .. _thread-safety-dict: Thread safety for dict objects ============================== Creating a dictionary with the :class:`dict` constructor is atomic when the argument to it is a :class:`dict` or a :class:`tuple`. When using the :meth:`dict.fromkeys` method, dictionary creation is atomic when the argument is a :class:`dict`, :class:`tuple`, :class:`set` or :class:`frozenset`. The following operations and functions are :term:`lock-free` and :term:`atomic `. .. code-block:: :class: good d[key] # dict.__getitem__ d.get(key) # dict.get key in d # dict.__contains__ len(d) # dict.__len__ All other operations from here on hold the :term:`per-object lock`. Writing or removing a single item is safe to call from multiple threads and will not corrupt the dictionary: .. code-block:: :class: good d[key] = value # write del d[key] # delete d.pop(key) # remove and return d.popitem() # remove and return last item d.setdefault(key, v) # insert if missing These operations may compare keys using :meth:`~object.__eq__`, which can execute arbitrary Python code. During such comparisons, the dictionary may be modified by another thread. For built-in types like :class:`str`, :class:`int`, and :class:`float`, that implement :meth:`~object.__eq__` in C, the underlying lock is not released during comparisons and this is not a concern. The following operations return new objects and hold the :term:`per-object lock` for the duration of the operation: .. code-block:: :class: good d.copy() # returns a shallow copy of the dictionary d | other # merges two dicts into a new dict d.keys() # returns a new dict_keys view object d.values() # returns a new dict_values view object d.items() # returns a new dict_items view object The :meth:`~dict.clear` method holds the lock for its duration. Other threads cannot observe elements being removed. The following operations lock both dictionaries. For :meth:`~dict.update` and ``|=``, this applies only when the other operand is a :class:`dict` that uses the standard dict iterator (but not subclasses that override iteration). For equality comparison, this applies to :class:`dict` and its subclasses: .. code-block:: :class: good d.update(other_dict) # both locked when other_dict is a dict d |= other_dict # both locked when other_dict is a dict d == other_dict # both locked for dict and subclasses All comparison operations also compare values using :meth:`~object.__eq__`, so for non-built-in types the lock may be released during comparison. :meth:`~dict.fromkeys` locks both the new dictionary and the iterable when the iterable is exactly a :class:`dict`, :class:`set`, or :class:`frozenset` (not subclasses): .. code-block:: :class: good dict.fromkeys(a_dict) # locks both dict.fromkeys(a_set) # locks both dict.fromkeys(a_frozenset) # locks both When updating from a non-dict iterable, only the target dictionary is locked. The iterable may be concurrently modified by another thread: .. code-block:: :class: maybe d.update(iterable) # iterable is not a dict: only d locked d |= iterable # iterable is not a dict: only d locked dict.fromkeys(iterable) # iterable is not a dict/set/frozenset: only result locked Operations that involve multiple accesses, as well as iteration, are never atomic: .. code-block:: :class: bad # NOT atomic: read-modify-write d[key] = d[key] + 1 # NOT atomic: check-then-act (TOCTOU) if key in d: del d[key] # NOT thread-safe: iteration while modifying for key, value in d.items(): process(key) # another thread may modify d To avoid time-of-check to time-of-use (TOCTOU) issues, use atomic operations or handle exceptions: .. code-block:: :class: good # Use pop() with default instead of check-then-delete d.pop(key, None) # Or handle the exception try: del d[key] except KeyError: pass To safely iterate over a dictionary that may be modified by another thread, iterate over a copy: .. code-block:: :class: good # Make a copy to iterate safely for key, value in d.copy().items(): process(key) Consider external synchronization when sharing :class:`dict` instances across threads. .. _thread-safety-set: Thread safety for set objects ============================== The :func:`len` function is lock-free and :term:`atomic `. The following read operation is lock-free. It does not block concurrent modifications and may observe intermediate states from operations that hold the per-object lock: .. code-block:: :class: good elem in s # set.__contains__ This operation may compare elements using :meth:`~object.__eq__`, which can execute arbitrary Python code. During such comparisons, the set may be modified by another thread. For built-in types like :class:`str`, :class:`int`, and :class:`float`, :meth:`!__eq__` does not release the underlying lock during comparisons and this is not a concern. All other operations from here on hold the per-object lock. Adding or removing a single element is safe to call from multiple threads and will not corrupt the set: .. code-block:: :class: good s.add(elem) # add element s.remove(elem) # remove element, raise if missing s.discard(elem) # remove element if present s.pop() # remove and return arbitrary element These operations also compare elements, so the same :meth:`~object.__eq__` considerations as above apply. The :meth:`~set.copy` method returns a new object and holds the per-object lock for the duration so that it is always atomic. The :meth:`~set.clear` method holds the lock for its duration. Other threads cannot observe elements being removed. The following operations only accept :class:`set` or :class:`frozenset` as operands and always lock both objects: .. code-block:: :class: good s |= other # other must be set/frozenset s &= other # other must be set/frozenset s -= other # other must be set/frozenset s ^= other # other must be set/frozenset s & other # other must be set/frozenset s | other # other must be set/frozenset s - other # other must be set/frozenset s ^ other # other must be set/frozenset :meth:`set.update`, :meth:`set.union`, :meth:`set.intersection` and :meth:`set.difference` can take multiple iterables as arguments. They all iterate through all the passed iterables and do the following: * :meth:`set.update` and :meth:`set.union` lock both objects only when the other operand is a :class:`set`, :class:`frozenset`, or :class:`dict`. * :meth:`set.intersection` and :meth:`set.difference` always try to lock all objects. :meth:`set.symmetric_difference` tries to lock both objects. The update variants of the above methods also have some differences between them: * :meth:`set.difference_update` and :meth:`set.intersection_update` try to lock all objects one-by-one. * :meth:`set.symmetric_difference_update` only locks the arguments if it is of type :class:`set`, :class:`frozenset`, or :class:`dict`. The following methods always try to lock both objects: .. code-block:: :class: good s.isdisjoint(other) # both locked s.issubset(other) # both locked s.issuperset(other) # both locked Operations that involve multiple accesses, as well as iteration, are never atomic: .. code-block:: :class: bad # NOT atomic: check-then-act if elem in s: s.remove(elem) # NOT thread-safe: iteration while modifying for elem in s: process(elem) # another thread may modify s Consider external synchronization when sharing :class:`set` instances across threads. See :ref:`freethreading-python-howto` for more information. .. _thread-safety-bytearray: Thread safety for bytearray objects =================================== The :func:`len` function is lock-free and :term:`atomic `. Concatenation and comparisons use the buffer protocol, which prevents resizing but does not hold the per-object lock. These operations may observe intermediate states from concurrent modifications: .. code-block:: :class: maybe ba + other # may observe concurrent writes ba == other # may observe concurrent writes ba < other # may observe concurrent writes All other operations from here on hold the per-object lock. Reading a single element or slice is safe to call from multiple threads: .. code-block:: :class: good ba[i] # bytearray.__getitem__ ba[i:j] # slice The following operations are safe to call from multiple threads and will not corrupt the bytearray: .. code-block:: :class: good ba[i] = x # write single byte ba[i:j] = values # write slice ba.append(x) # append single byte ba.extend(other) # extend with iterable ba.insert(i, x) # insert single byte ba.pop() # remove and return last byte ba.pop(i) # remove and return byte at index ba.remove(x) # remove first occurrence ba.reverse() # reverse in place ba.clear() # remove all bytes Slice assignment locks both objects when *values* is a :class:`bytearray`: .. code-block:: :class: good ba[i:j] = other_bytearray # both locked The following operations return new objects and hold the per-object lock for the duration: .. code-block:: :class: good ba.copy() # returns a shallow copy ba * n # repeat into new bytearray The membership test holds the lock for its duration: .. code-block:: :class: good x in ba # bytearray.__contains__ All other bytearray methods (such as :meth:`~bytearray.find`, :meth:`~bytearray.replace`, :meth:`~bytearray.split`, :meth:`~bytearray.decode`, etc.) hold the per-object lock for their duration. Operations that involve multiple accesses, as well as iteration, are never atomic: .. code-block:: :class: bad # NOT atomic: check-then-act if x in ba: ba.remove(x) # NOT thread-safe: iteration while modifying for byte in ba: process(byte) # another thread may modify ba To safely iterate over a bytearray that may be modified by another thread, iterate over a copy: .. code-block:: :class: good # Make a copy to iterate safely for byte in ba.copy(): process(byte) Consider external synchronization when sharing :class:`bytearray` instances across threads. See :ref:`freethreading-python-howto` for more information. .. _thread-safety-memoryview: Thread safety for memoryview objects ==================================== :class:`memoryview` objects provide access to the internal data of an underlying object without copying. Thread safety depends on both the memoryview itself and the underlying buffer exporter. The memoryview implementation uses atomic operations to track its own exports in the :term:`free-threaded build`. Creating and releasing a memoryview are thread-safe. Attribute access (e.g., :attr:`~memoryview.shape`, :attr:`~memoryview.format`) reads fields that are immutable for the lifetime of the memoryview, so concurrent reads are safe as long as the memoryview has not been released. However, the actual data accessed through the memoryview is owned by the underlying object. Concurrent access to this data is only safe if the underlying object supports it: * For immutable objects like :class:`bytes`, concurrent reads through multiple memoryviews are safe. * For mutable objects like :class:`bytearray`, reading and writing the same memory region from multiple threads without external synchronization is not safe and may result in data corruption. Note that even read-only memoryviews of mutable objects do not prevent data races if the underlying object is modified from another thread. .. code-block:: :class: bad # NOT safe: concurrent writes to the same buffer data = bytearray(1000) view = memoryview(data) # Thread 1: view[0:500] = b'x' * 500 # Thread 2: view[0:500] = b'y' * 500 .. code-block:: :class: good # Safe: use a lock for concurrent access import threading lock = threading.Lock() data = bytearray(1000) view = memoryview(data) with lock: view[0:500] = b'x' * 500 Resizing or reallocating the underlying object (such as calling :meth:`bytearray.resize`) while a memoryview is exported raises :exc:`BufferError`. This is enforced regardless of threading. --- ### Doc Builtins Time Complexity (`Doc/builtins/time-complexity.rst`) .. _time-complexity: =============================================== Time complexity of operations on built-in types =============================================== This page documents the time complexity of various operations on built-in types in CPython. Other Python implementations may have different performance characteristics. Additionally, the listed costs assume exact built-in types, as instances of subclasses may have different costs. We use |big O notation|_ to describe how the running time of an operation grows with the size of its inputs. Unless stated otherwise, *n* denotes the number of elements currently in the container, and *k* is the value of a numeric parameter, such as an index or a repeat count. .. |big O notation| replace:: Big *O* notation .. _big O notation: https://en.wikipedia.org/wiki/Big_O_notation :class:`!list` ============== Lists are mutable sequences; for more detail on the implementation see :ref:`how-are-lists-implemented`. The largest costs come from growing beyond the current allocation size (because everything must move), or from inserting or deleting somewhere near the beginning (because everything after that must move). If you need to add or remove at both ends, consider using a :class:`collections.deque` instead. .. list-table:: :header-rows: 1 * - Operation - Complexity * - Copy (``l.copy()``) - *O*\ (*n*) * - Append (``l.append(x)``) [1]_ - *O*\ (1) * - Pop (``l.pop(k)``) [1]_ [2]_ - *O*\ (*n* - *k*) * - Insert (``l.insert(k, x)``) [1]_ [2]_ - *O*\ (*n* - *k*) * - Get item (``l[k]``) - *O*\ (1) * - Set item (``l[k] = x``) - *O*\ (1) * - Delete item (``del l[k]``) [2]_ - *O*\ (*n* - *k*) * - Iteration - *O*\ (*n*) * - Get slice (``l[i:j]``) - *O*\ (*j* - *i*) * - Set slice (``l[i:j] = t``) [1]_ - *O*\ (*j* - *i*) if len(*t*) == *j* - *i*, otherwise *O*\ (*n* - *i* + len(*t*)) * - Delete slice (``del l[i:j]``) - *O*\ (*n* - *i*) * - Extend (``l.extend(t)``) [1]_ [3]_ - *O*\ (len(*t*)) * - Sort (``l.sort()``) [4]_ - *O*\ (*n* log *n*) * - Concatenate (``l1 + l2``) - *O*\ (len(*l1*) + len(*l2*)) * - Multiply (``l * k``) - *O*\ (*nk*) * - ``x in l`` - *O*\ (*n*) * - ``min(l)``, ``max(l)`` - *O*\ (*n*) * - Get length (``len(l)``) [5]_ - *O*\ (1) :class:`!tuple` =============== A :class:`tuple` is an :term:`immutable` sequence. Because a tuple can never change, there are no insertion or deletion costs, and making a copy simply returns the same object, so is constant time (*O*\ (1)). .. list-table:: :header-rows: 1 * - Operation - Complexity * - Copy (``tuple(t)``) - *O*\ (1) * - Get item (``t[k]``) - *O*\ (1) * - Get slice (``t[i:j]``) - *O*\ (*j* - *i*) * - Concatenate (``t1 + t2``) - *O*\ (len(*t1*) + len(*t2*)) * - Multiply (``t * k``) - *O*\ (*nk*) * - Iteration - *O*\ (*n*) * - ``x in t`` - *O*\ (*n*) * - ``min(t)``, ``max(t)`` - *O*\ (*n*) * - Get length (``len(t)``) [5]_ - *O*\ (1) :class:`!dict`, :class:`!frozendict` ==================================== The times listed for dict objects are average-case times, as they assume the hash function for the objects is sufficiently robust to make collisions uncommon. They also assume the keys are well-distributed among the set of possible keys. In the worst case, when every key hashes to the same value, each of the *O*\ (1) operations below instead takes *O*\ (*n*) time. They also assume that hashing and comparing a key is *O*\ (1). For more detail on the implementation, see :ref:`how-are-dictionaries-implemented`. A :class:`frozendict` is immutable, so it does not support setting, deleting, or updating items. The other operations below apply to it at the same costs. .. list-table:: :header-rows: 1 * - Operation - Complexity * - ``key in d`` - *O*\ (1) * - Copy (``d.copy()``) [6]_ [7]_ - *O*\ (*n*) * - Get item (``d[key]``, ``d.get(key)``) - *O*\ (1) * - Set item (``d[key] = value``) [1]_ - *O*\ (1) * - Delete item (``del d[key]``, ``d.pop(key)``) - *O*\ (1) * - Update (``d.update(t)``, ``d |= t``) [1]_ [3]_ [7]_ - *O*\ (len(*t*)) * - Iteration [7]_ - *O*\ (*n*) * - Get length (``len(d)``) [5]_ - *O*\ (1) :class:`!set`, :class:`!frozenset` ================================== See :class:`dict` as the :class:`set` and :class:`frozenset` implementations are similar, and the same caveats apply. In the worst case, *O*\ (1) operations instead take *O*\ (*n*) time, and operations that look up every element degrade accordingly. A :class:`frozenset` is :term:`immutable`, so it does not support adding, discarding, or the in-place update operations. The others below apply to it at the same costs. .. list-table:: :header-rows: 1 * - Operation - Complexity * - ``x in s`` - *O*\ (1) * - Copy (``s.copy()``) [6]_ [7]_ - *O*\ (*n*) * - Add (``s.add(x)``) [1]_ - *O*\ (1) * - Discard (``s.discard(x)``, ``s.remove(x)``) - *O*\ (1) * - Union (``s1 | s2``, ``s1.union(s2)``) [7]_ - *O*\ (len(*s1*) + len(*s2*)) * - Update (``s1 |= s2``, ``s1.update(s2)``) [1]_ [7]_ - *O*\ (len(*s2*)) * - Intersection (``s1 & s2``, ``s1.intersection(s2)``) [7]_ [8]_ - *O*\ (min(len(*s1*), len(*s2*))) * - Intersection update (``s1 &= s2``, ``s1.intersection_update(s2)``) [1]_ [7]_ [8]_ - *O*\ (min(len(*s1*), len(*s2*))) * - Difference (``s1 - s2``, ``s1.difference(s2)``) [7]_ [9]_ - *O*\ (len(*s1*)) * - Difference update (``s1 -= s2``, ``s1.difference_update(s2)``) [1]_ [7]_ [8]_ - *O*\ (min(len(*s1*), len(*s2*))) * - Symmetric difference (``s1 ^ s2``, ``s1.symmetric_difference(s2)``) [7]_ - *O*\ (len(*s1*) + len(*s2*)) * - Symmetric difference update (``s1 ^= s2``, ``s1.symmetric_difference_update(s2)``) [1]_ [7]_ - *O*\ (len(*s2*)) * - Get length (``len(s)``) [5]_ - *O*\ (1) :class:`!str`, :class:`!bytes`, :class:`!bytearray` =================================================== :class:`str` and :class:`bytes` objects are immutable sequences of characters and bytes, respectively. As with tuples, copying one returns the original object. A :class:`bytearray` is mutable, and additionally supports the mutating operations of :class:`list` (except :meth:`!sort`), at the same costs. However, deleting at the front with ``del`` (``del b[0]``, ``del b[:k]``) only advances the start of the buffer instead of moving the remaining bytes, and is amortized *O*\ (1). .. list-table:: :header-rows: 1 * - Operation - Complexity * - Get item (``s[k]``) - *O*\ (1) * - Get slice (``s[i:j]``) - *O*\ (*j* - *i*) * - Concatenate (``s + t``) [10]_ - *O*\ (len(*s*) + len(*t*)) * - Multiply (``s * k``) - *O*\ (*nk*) * - Substring search (``x in s``, ``s.find(x)``, ``s.index(x)``) [11]_ - *O*\ (*n*) * - Reverse substring search (``s.rfind(x)``, ``s.rindex(x)``) [11]_ [12]_ - *O*\ (*n* × len(*x*)) * - Encode or decode [13]_ - *O*\ (*n*) * - Iteration - *O*\ (*n*) * - Get length (``len(s)``) [5]_ - *O*\ (1) :class:`!memoryview` ==================== :class:`memoryview` objects allow Python code to access the internal data of an object that supports the :ref:`buffer protocol ` without copying. In particular, slicing a memory view returns a new view onto the same buffer. .. list-table:: :header-rows: 1 * - Operation - Complexity * - Create (``memoryview(obj)``) - *O*\ (1) * - Get item (``v[k]``) - *O*\ (1) * - Get slice (``v[i:j]``) - *O*\ (1) * - Index (``v.index(x)``) [11]_ [14]_ - *O*\ (*n*) * - Count (``v.count(x)``) [14]_ - *O*\ (*n*) * - Convert to bytes (``v.tobytes()``, ``bytes(v)``) - *O*\ (*n*) * - Get length (``len(v)``) [5]_ - *O*\ (1) :class:`!range` =============== A :class:`range` object computes its items on demand from its *start*, *stop* and *step* values, so most operations do not depend on the length of the range. .. list-table:: :header-rows: 1 * - Operation - Complexity * - Get item (``r[k]``) - *O*\ (1) * - Get slice (``r[i:j]``) - *O*\ (1) * - ``x in r`` [15]_ - *O*\ (1) * - Index and count (``r.index(x)``, ``r.count(x)``) [15]_ - *O*\ (1) * - Iteration - *O*\ (*n*) * - ``min(r)``, ``max(r)`` - *O*\ (*n*) * - Get length (``len(r)``) [5]_ - *O*\ (1) Notes ===== .. [1] Amortized. An individual operation may occasionally be *O*\ (*n*) when the underlying storage is resized, but this cost is spread over many operations, depending on the history of the container. .. [2] Popping or deleting the element at index *k* of a list of size *n* shifts all elements after *k* one slot to the left, moving *n* - *k* - 1 elements; inserting at index *k* shifts the elements from *k* onwards one slot to the right, moving *n* - *k* elements. The worst case is index 0, where the whole rest of the list has to be moved; the average case, an index in the middle of the list, takes *O*\ (*n*/2) = *O*\ (*n*) operations; and operating at the end of the list moves nothing and is *O*\ (1). .. [3] Plus the cost of iterating over *t*, which may be expensive for an arbitrary iterable. .. [4] This is the worst case scenario. Sorting is adaptive and input that is already sorted or reverse-sorted takes only *O*\ (*n*) comparisons. See :source:`Objects/listsort.txt` for more information. .. [5] The number of elements is stored in the object, so ``len()`` does not need to count them. .. [6] Copying a :class:`frozendict` or a :class:`frozenset` is *O*\ (1) as it returns the original object. .. [7] These operations scan the container's internal hash table, which is not shrunk when elements are removed. After removing most elements, they still take time proportional to the container's former size, until a later insertion triggers a resize. .. [8] *O*\ (len(*t*)) if *t* is not a set. .. [9] *O*\ (len(*s*) + len(*t*)) if *t* is not a set. .. [10] Each concatenation builds a new object, so building a string by concatenating many pieces in a loop is quadratic in the total length. See the :ref:`note on concatenating immutable sequences ` for alternatives. .. [11] With *start* and *end* arguments, *n* is the length of the region searched rather than of *s*, and unlike slicing nothing is copied. .. [12] This is the worst case. Reverse searches are *O*\ (*n*) on typical input. Forward searches instead use a more elaborate algorithm with a linear worst case, described in :source:`Objects/stringlib/stringlib_find_two_way_notes.txt`. .. [13] This assumes a codec that does a constant amount of work per character. .. [14] These unpack and compare each element individually, so they are much slower than the equivalent :class:`bytes` methods. .. [15] Assuming :class:`int` or :class:`bool` arguments. For other types, the range is searched like any other sequence in *O*\ (*n*) time. --- ### Doc C Api Abstract (`Doc/c-api/abstract.rst`) .. highlight:: c .. _abstract: ********************** Abstract Objects Layer ********************** The functions in this chapter interact with Python objects regardless of their type, or with wide classes of object types (e.g. all numerical types, or all sequence types). When used on object types for which they do not apply, they will raise a Python exception. It is not possible to use these functions on objects that are not properly initialized, such as a list object that has been created by :c:func:`PyList_New`, but whose items have not been set to some non-\ ``NULL`` value yet. .. toctree:: object.rst call.rst number.rst sequence.rst mapping.rst iter.rst buffer.rst --- ### Doc C Api Allocation (`Doc/c-api/allocation.rst`) .. highlight:: c .. _allocating-objects: Allocating objects on the heap ============================== .. c:function:: PyObject* _PyObject_New(PyTypeObject *type) .. c:function:: PyVarObject* _PyObject_NewVar(PyTypeObject *type, Py_ssize_t size) .. c:function:: PyObject* PyObject_Init(PyObject *op, PyTypeObject *type) Initialize a newly allocated object *op* with its type and initial reference. Returns the initialized object. Other fields of the object are not initialized. Despite its name, this function is unrelated to the object's :meth:`~object.__init__` method (:c:member:`~PyTypeObject.tp_init` slot). Specifically, this function does **not** call the object's :meth:`!__init__` method. In general, consider this function to be a low-level routine. Use :c:member:`~PyTypeObject.tp_alloc` where possible. For implementing :c:member:`!tp_alloc` for your type, prefer :c:func:`PyType_GenericAlloc` or :c:func:`PyObject_New`. .. note:: This function only initializes the object's memory corresponding to the initial :c:type:`PyObject` structure. It does not zero the rest. .. c:function:: PyVarObject* PyObject_InitVar(PyVarObject *op, PyTypeObject *type, Py_ssize_t size) This does everything :c:func:`PyObject_Init` does, and also initializes the length information for a variable-size object. .. note:: This function only initializes some of the object's memory. It does not zero the rest. .. c:macro:: PyObject_New(TYPE, typeobj) Allocates a new Python object using the C structure type *TYPE* and the Python type object *typeobj* (``PyTypeObject*``) by calling :c:func:`PyObject_Malloc` to allocate memory and initializing it like :c:func:`PyObject_Init`. The caller will own the only reference to the object (i.e. its reference count will be one). Avoid calling this directly to allocate memory for an object; call the type's :c:member:`~PyTypeObject.tp_alloc` slot instead. When populating a type's :c:member:`~PyTypeObject.tp_alloc` slot, :c:func:`PyType_GenericAlloc` is preferred over a custom function that simply calls this macro. This macro does not call :c:member:`~PyTypeObject.tp_alloc`, :c:member:`~PyTypeObject.tp_new` (:meth:`~object.__new__`), or :c:member:`~PyTypeObject.tp_init` (:meth:`~object.__init__`). This cannot be used for objects with :c:macro:`Py_TPFLAGS_HAVE_GC` set in :c:member:`~PyTypeObject.tp_flags`; use :c:macro:`PyObject_GC_New` instead. Memory allocated by this macro must be freed with :c:func:`PyObject_Free` (usually called via the object's :c:member:`~PyTypeObject.tp_free` slot). .. note:: The returned memory is not guaranteed to have been completely zeroed before it was initialized. .. note:: This macro does not construct a fully initialized object of the given type; it merely allocates memory and prepares it for further initialization by :c:member:`~PyTypeObject.tp_init`. To construct a fully initialized object, call *typeobj* instead. For example:: PyObject *foo = PyObject_CallNoArgs((PyObject *)&PyFoo_Type); .. seealso:: * :c:func:`PyObject_Free` * :c:macro:`PyObject_GC_New` * :c:func:`PyType_GenericAlloc` * :c:member:`~PyTypeObject.tp_alloc` .. c:macro:: PyObject_NewVar(TYPE, typeobj, size) Like :c:macro:`PyObject_New` except: * It allocates enough memory for the *TYPE* structure plus *size* (``Py_ssize_t``) fields of the size given by the :c:member:`~PyTypeObject.tp_itemsize` field of *typeobj*. * The memory is initialized like :c:func:`PyObject_InitVar`. This is useful for implementing objects like tuples, which are able to determine their size at construction time. Embedding the array of fields into the same allocation decreases the number of allocations, improving the memory management efficiency. Avoid calling this directly to allocate memory for an object; call the type's :c:member:`~PyTypeObject.tp_alloc` slot instead. When populating a type's :c:member:`~PyTypeObject.tp_alloc` slot, :c:func:`PyType_GenericAlloc` is preferred over a custom function that simply calls this macro. This cannot be used for objects with :c:macro:`Py_TPFLAGS_HAVE_GC` set in :c:member:`~PyTypeObject.tp_flags`; use :c:macro:`PyObject_GC_NewVar` instead. Memory allocated by this function must be freed with :c:func:`PyObject_Free` (usually called via the object's :c:member:`~PyTypeObject.tp_free` slot). .. note:: The returned memory is not guaranteed to have been completely zeroed before it was initialized. .. note:: This macro does not construct a fully initialized object of the given type; it merely allocates memory and prepares it for further initialization by :c:member:`~PyTypeObject.tp_init`. To construct a fully initialized object, call *typeobj* instead. For example:: PyObject *list_instance = PyObject_CallNoArgs((PyObject *)&PyList_Type); .. seealso:: * :c:func:`PyObject_Free` * :c:macro:`PyObject_GC_NewVar` * :c:func:`PyType_GenericAlloc` * :c:member:`~PyTypeObject.tp_alloc` .. c:var:: PyObject _Py_NoneStruct Object which is visible in Python as ``None``. This should only be accessed using the :c:macro:`Py_None` macro, which evaluates to a pointer to this object. .. seealso:: :ref:`moduleobjects` To allocate and create extension modules. Soft-deprecated aliases ^^^^^^^^^^^^^^^^^^^^^^^ .. soft-deprecated:: 3.15 These are aliases to existing functions and macros. They exist solely for backwards compatibility. .. list-table:: :widths: auto :header-rows: 1 * * Soft-deprecated alias * Function * * .. c:macro:: PyObject_NEW(type, typeobj) * :c:macro:`PyObject_New` * * .. c:macro:: PyObject_NEW_VAR(type, typeobj, n) * :c:macro:`PyObject_NewVar` * * .. c:macro:: PyObject_INIT(op, typeobj) * :c:func:`PyObject_Init` * * .. c:macro:: PyObject_INIT_VAR(op, typeobj, n) * :c:func:`PyObject_InitVar` * * .. c:macro:: PyObject_MALLOC(n) * :c:func:`PyObject_Malloc` * * .. c:macro:: PyObject_REALLOC(p, n) * :c:func:`PyObject_Realloc` * * .. c:macro:: PyObject_FREE(p) * :c:func:`PyObject_Free` * * .. c:macro:: PyObject_DEL(p) * :c:func:`PyObject_Free` * * .. c:macro:: PyObject_Del(p) * :c:func:`PyObject_Free` --- ### Doc C Api Apiabiversion (`Doc/c-api/apiabiversion.rst`) .. highlight:: c .. _apiabiversion: *********************** API and ABI Versioning *********************** Build-time version constants ---------------------------- CPython exposes its version number in the following macros. Note that these correspond to the version code is **built** with. See :c:var:`Py_Version` for the version used at **run time**. See :ref:`stable` for a discussion of API and ABI stability across versions. .. c:macro:: PY_MAJOR_VERSION The ``3`` in ``3.4.1a2``. .. c:macro:: PY_MINOR_VERSION The ``4`` in ``3.4.1a2``. .. c:macro:: PY_MICRO_VERSION The ``1`` in ``3.4.1a2``. .. c:macro:: PY_RELEASE_LEVEL The ``a`` in ``3.4.1a2``. This can be ``0xA`` for alpha, ``0xB`` for beta, ``0xC`` for release candidate or ``0xF`` for final. .. c:namespace:: NULL .. c:macro:: PY_RELEASE_LEVEL_ALPHA :no-typesetting: .. c:macro:: PY_RELEASE_LEVEL_BETA :no-typesetting: .. c:macro:: PY_RELEASE_LEVEL_GAMMA :no-typesetting: .. c:macro:: PY_RELEASE_LEVEL_FINAL :no-typesetting: For completeness, the values are available as macros: :c:macro:`!PY_RELEASE_LEVEL_ALPHA` (``0xA``), :c:macro:`!PY_RELEASE_LEVEL_BETA` (``0xB``), :c:macro:`!PY_RELEASE_LEVEL_GAMMA` (``0xC``), and :c:macro:`!PY_RELEASE_LEVEL_FINAL` (``0xF``). .. c:macro:: PY_RELEASE_SERIAL The ``2`` in ``3.4.1a2``. Zero for final releases. .. c:macro:: PY_VERSION_HEX The Python version number encoded in a single integer. See :c:func:`Py_PACK_FULL_VERSION` for the encoding details. Use this for numeric comparisons, for example, ``#if PY_VERSION_HEX >= ...``. .. c:macro:: PY_VERSION The Python version as a string, for example, ``"3.4.1a2"``. These macros are defined in :source:`Include/patchlevel.h`. Run-time version ---------------- .. c:var:: const unsigned long Py_Version The Python runtime version number encoded in a single constant integer. See :c:func:`Py_PACK_FULL_VERSION` for the encoding details. This contains the Python version used at run time. Use this for numeric comparisons, for example, ``if (Py_Version >= ...)``. .. versionadded:: 3.11 Bit-packing macros ------------------ .. c:function:: uint32_t Py_PACK_FULL_VERSION(int major, int minor, int micro, int release_level, int release_serial) Return the given version, encoded as a single 32-bit integer with the following structure: +------------------+-------+----------------+-----------+--------------------------+ | | No. | | | Example values | | | of | | +-------------+------------+ | Argument | bits | Bit mask | Bit shift | ``3.4.1a2`` | ``3.10.0`` | +==================+=======+================+===========+=============+============+ | *major* | 8 | ``0xFF000000`` | 24 | ``0x03`` | ``0x03`` | +------------------+-------+----------------+-----------+-------------+------------+ | *minor* | 8 | ``0x00FF0000`` | 16 | ``0x04`` | ``0x0A`` | +------------------+-------+----------------+-----------+-------------+------------+ | *micro* | 8 | ``0x0000FF00`` | 8 | ``0x01`` | ``0x00`` | +------------------+-------+----------------+-----------+-------------+------------+ | *release_level* | 4 | ``0x000000F0`` | 4 | ``0xA`` | ``0xF`` | +------------------+-------+----------------+-----------+-------------+------------+ | *release_serial* | 4 | ``0x0000000F`` | 0 | ``0x2`` | ``0x0`` | +------------------+-------+----------------+-----------+-------------+------------+ For example: +-------------+------------------------------------+-----------------+ | Version | ``Py_PACK_FULL_VERSION`` arguments | Encoded version | +=============+====================================+=================+ | ``3.4.1a2`` | ``(3, 4, 1, 0xA, 2)`` | ``0x030401a2`` | +-------------+------------------------------------+-----------------+ | ``3.10.0`` | ``(3, 10, 0, 0xF, 0)`` | ``0x030a00f0`` | +-------------+------------------------------------+-----------------+ Out-of range bits in the arguments are ignored. That is, the macro can be defined as: .. code-block:: c #ifndef Py_PACK_FULL_VERSION #define Py_PACK_FULL_VERSION(X, Y, Z, LEVEL, SERIAL) ( \ (((X) & 0xff) << 24) | \ (((Y) & 0xff) << 16) | \ (((Z) & 0xff) << 8) | \ (((LEVEL) & 0xf) << 4) | \ (((SERIAL) & 0xf) << 0)) #endif ``Py_PACK_FULL_VERSION`` is primarily a macro, intended for use in ``#if`` directives, but it is also available as an exported function. .. versionadded:: 3.14 .. c:function:: uint32_t Py_PACK_VERSION(int major, int minor) Equivalent to ``Py_PACK_FULL_VERSION(major, minor, 0, 0, 0)``. The result does not correspond to any Python release, but is useful in numeric comparisons. .. versionadded:: 3.14 --- ### Doc C Api Arg (`Doc/c-api/arg.rst`) .. highlight:: c .. _arg-parsing: Parsing arguments and building values ===================================== These functions are useful when creating your own extension functions and methods. Additional information and examples are available in :ref:`extending-index`. The first three of these functions described, :c:func:`PyArg_ParseTuple`, :c:func:`PyArg_ParseTupleAndKeywords`, and :c:func:`PyArg_Parse`, all use *format strings* which are used to tell the function about the expected arguments. The format strings use the same syntax for each of these functions. ----------------- Parsing arguments ----------------- A format string consists of zero or more "format units." A format unit describes one Python object; it is usually a single character or a parenthesized sequence of format units. With a few exceptions, a format unit that is not a parenthesized sequence normally corresponds to a single address argument to these functions. In the following description, the quoted form is the format unit; the entry in (round) parentheses is the Python object type that matches the format unit; and the entry in [square] brackets is the type of the C variable(s) whose address should be passed. .. _arg-parsing-string-and-buffers: Strings and buffers ------------------- .. note:: On Python 3.12 and older, the macro :c:macro:`!PY_SSIZE_T_CLEAN` must be defined before including :file:`Python.h` to use all ``#`` variants of formats (``s#``, ``y#``, etc.) explained below. This is not necessary on Python 3.13 and later. These formats allow accessing an object as a contiguous chunk of memory. You don't have to provide raw storage for the returned unicode or bytes area. Unless otherwise stated, buffers are not NUL-terminated. There are three ways strings and buffers can be converted to C: * Formats such as ``y*`` and ``s*`` fill a :c:type:`Py_buffer` structure. This locks the underlying buffer so that the caller can subsequently use the buffer even inside a :c:type:`Py_BEGIN_ALLOW_THREADS` block without the risk of mutable data being resized or destroyed. As a result, **you have to call** :c:func:`PyBuffer_Release` after you have finished processing the data (or in any early abort case). * The ``es``, ``es#``, ``et`` and ``et#`` formats allocate the result buffer. **You have to call** :c:func:`PyMem_Free` after you have finished processing the data (or in any early abort case). * .. _c-arg-borrowed-buffer: Other formats take a :class:`str` or a read-only :term:`bytes-like object`, such as :class:`bytes`, and provide a ``const char *`` pointer to its buffer. In this case the buffer is "borrowed": it is managed by the corresponding Python object, and shares the lifetime of this object. You won't have to release any memory yourself. To ensure that the underlying buffer may be safely borrowed, the object's :c:member:`PyBufferProcs.bf_releasebuffer` field must be ``NULL``. This disallows common mutable objects such as :class:`bytearray`, but also some read-only objects such as :class:`memoryview` of :class:`bytes`. Besides this ``bf_releasebuffer`` requirement, there is no check to verify whether the input object is immutable (e.g. whether it would honor a request for a writable buffer, or whether another thread can mutate the data). ``s`` (:class:`str`) [const char \*] Convert a Unicode object to a C pointer to a character string. A pointer to an existing string is stored in the character pointer variable whose address you pass. The C string is NUL-terminated. The Python string must not contain embedded null code points; if it does, a :exc:`ValueError` exception is raised. Unicode objects are converted to C strings using ``'utf-8'`` encoding. If this conversion fails, a :exc:`UnicodeError` is raised. .. note:: This format does not accept :term:`bytes-like objects `. If you want to accept filesystem paths and convert them to C character strings, it is preferable to use the ``O&`` format with :c:func:`PyUnicode_FSConverter` as *converter*. .. versionchanged:: 3.5 Previously, :exc:`TypeError` was raised when embedded null code points were encountered in the Python string. ``s*`` (:class:`str` or :term:`bytes-like object`) [Py_buffer] This format accepts Unicode objects as well as bytes-like objects. It fills a :c:type:`Py_buffer` structure provided by the caller. In this case the resulting C string may contain embedded NUL bytes. Unicode objects are converted to C strings using ``'utf-8'`` encoding. ``s#`` (:class:`str`, read-only :term:`bytes-like object`) [const char \*, :c:type:`Py_ssize_t`] Like ``s*``, except that it provides a :ref:`borrowed buffer `. The result is stored into two C variables, the first one a pointer to a C string, the second one its length. The string may contain embedded null bytes. Unicode objects are converted to C strings using ``'utf-8'`` encoding. ``z`` (:class:`str` or ``None``) [const char \*] Like ``s``, but the Python object may also be ``None``, in which case the C pointer is set to ``NULL``. ``z*`` (:class:`str`, :term:`bytes-like object` or ``None``) [Py_buffer] Like ``s*``, but the Python object may also be ``None``, in which case the ``buf`` member of the :c:type:`Py_buffer` structure is set to ``NULL``. ``z#`` (:class:`str`, read-only :term:`bytes-like object` or ``None``) [const char \*, :c:type:`Py_ssize_t`] Like ``s#``, but the Python object may also be ``None``, in which case the C pointer is set to ``NULL``. ``y`` (read-only :term:`bytes-like object`) [const char \*] This format converts a bytes-like object to a C pointer to a :ref:`borrowed ` character string; it does not accept Unicode objects. The bytes buffer must not contain embedded null bytes; if it does, a :exc:`ValueError` exception is raised. .. versionchanged:: 3.5 Previously, :exc:`TypeError` was raised when embedded null bytes were encountered in the bytes buffer. ``y*`` (:term:`bytes-like object`) [Py_buffer] This variant on ``s*`` doesn't accept Unicode objects, only bytes-like objects. **This is the recommended way to accept binary data.** ``y#`` (read-only :term:`bytes-like object`) [const char \*, :c:type:`Py_ssize_t`] This variant on ``s#`` doesn't accept Unicode objects, only bytes-like objects. ``S`` (:class:`bytes`) [PyBytesObject \*] Requires that the Python object is a :class:`bytes` object, without attempting any conversion. Raises :exc:`TypeError` if the object is not a bytes object. The C variable may also be declared as :c:expr:`PyObject*`. ``Y`` (:class:`bytearray`) [PyByteArrayObject \*] Requires that the Python object is a :class:`bytearray` object, without attempting any conversion. Raises :exc:`TypeError` if the object is not a :class:`bytearray` object. The C variable may also be declared as :c:expr:`PyObject*`. ``U`` (:class:`str`) [PyObject \*] Requires that the Python object is a Unicode object, without attempting any conversion. Raises :exc:`TypeError` if the object is not a Unicode object. The C variable may also be declared as :c:expr:`PyObject*`. ``w*`` (read-write :term:`bytes-like object`) [Py_buffer] This format accepts any object which implements the read-write buffer interface. It fills a :c:type:`Py_buffer` structure provided by the caller. The buffer may contain embedded null bytes. The caller has to call :c:func:`PyBuffer_Release` when it is done with the buffer. ``es`` (:class:`str`) [const char \*encoding, char \*\*buffer] This variant on ``s`` is used for encoding Unicode into a character buffer. It only works for encoded data without embedded NUL bytes. This format requires two arguments. The first is only used as input, and must be a :c:expr:`const char*` which points to the name of an encoding as a NUL-terminated string, or ``NULL``, in which case ``'utf-8'`` encoding is used. An exception is raised if the named encoding is not known to Python. The second argument must be a :c:expr:`char**`; the value of the pointer it references will be set to a buffer with the contents of the argument text. The text will be encoded in the encoding specified by the first argument. :c:func:`PyArg_ParseTuple` will allocate a buffer of the needed size, copy the encoded data into this buffer and adjust *\*buffer* to reference the newly allocated storage. The caller is responsible for calling :c:func:`PyMem_Free` to free the allocated buffer after use. ``et`` (:class:`str`, :class:`bytes` or :class:`bytearray`) [const char \*encoding, char \*\*buffer] Same as ``es`` except that byte string objects are passed through without recoding them. Instead, the implementation assumes that the byte string object uses the encoding passed in as parameter. ``es#`` (:class:`str`) [const char \*encoding, char \*\*buffer, :c:type:`Py_ssize_t` \*buffer_length] This variant on ``s#`` is used for encoding Unicode into a character buffer. Unlike the ``es`` format, this variant allows input data which contains NUL characters. It requires three arguments. The first is only used as input, and must be a :c:expr:`const char*` which points to the name of an encoding as a NUL-terminated string, or ``NULL``, in which case ``'utf-8'`` encoding is used. An exception is raised if the named encoding is not known to Python. The second argument must be a :c:expr:`char**`; the value of the pointer it references will be set to a buffer with the contents of the argument text. The text will be encoded in the encoding specified by the first argument. The third argument must be a pointer to an integer; the referenced integer will be set to the number of bytes in the output buffer. There are two modes of operation: If *\*buffer* points a ``NULL`` pointer, the function will allocate a buffer of the needed size, copy the encoded data into this buffer and set *\*buffer* to reference the newly allocated storage. The caller is responsible for calling :c:func:`PyMem_Free` to free the allocated buffer after usage. If *\*buffer* points to a non-``NULL`` pointer (an already allocated buffer), :c:func:`PyArg_ParseTuple` will use this location as the buffer and interpret the initial value of *\*buffer_length* as the buffer size. It will then copy the encoded data into the buffer and NUL-terminate it. If the buffer is not large enough, a :exc:`ValueError` will be set. In both cases, *\*buffer_length* is set to the length of the encoded data without the trailing NUL byte. ``et#`` (:class:`str`, :class:`bytes` or :class:`bytearray`) [const char \*encoding, char \*\*buffer, :c:type:`Py_ssize_t` \*buffer_length] Same as ``es#`` except that byte string objects are passed through without recoding them. Instead, the implementation assumes that the byte string object uses the encoding passed in as parameter. .. versionchanged:: 3.12 ``u``, ``u#``, ``Z``, and ``Z#`` are removed because they used a legacy ``Py_UNICODE*`` representation. Numbers ------- These formats allow representing Python numbers or single characters as C numbers. Formats that require :class:`int`, :class:`float` or :class:`complex` can also use the corresponding special methods :meth:`~object.__index__`, :meth:`~object.__float__` or :meth:`~object.__complex__` to convert the Python object to the required type. For signed integer formats, :exc:`OverflowError` is raised if the value is out of range for the C type. For unsigned integer formats, the most significant bits are silently truncated when the receiving field is too small to receive the value, and :exc:`DeprecationWarning` is emitted when the value is larger than the maximal value for the C type or less than the minimal value for the corresponding signed integer type of the same size. ``b`` (:class:`int`) [unsigned char] Convert a nonnegative Python integer to an unsigned tiny integer, stored in a C :c:expr:`unsigned char`. ``B`` (:class:`int`) [unsigned char] Convert a Python integer to a tiny integer without overflow checking, stored in a C :c:expr:`unsigned char`. Convert a Python integer to a C :c:expr:`unsigned char`. ``h`` (:class:`int`) [short int] Convert a Python integer to a C :c:expr:`short int`. ``H`` (:class:`int`) [unsigned short int] Convert a Python integer to a C :c:expr:`unsigned short int`. ``i`` (:class:`int`) [int] Convert a Python integer to a plain C :c:expr:`int`. ``I`` (:class:`int`) [unsigned int] Convert a Python integer to a C :c:expr:`unsigned int`. ``l`` (:class:`int`) [long int] Convert a Python integer to a C :c:expr:`long int`. ``k`` (:class:`int`) [unsigned long] Convert a Python integer to a C :c:expr:`unsigned long`. .. versionchanged:: 3.14 Use :meth:`~object.__index__` if available. ``L`` (:class:`int`) [long long] Convert a Python integer to a C :c:expr:`long long`. ``K`` (:class:`int`) [unsigned long long] Convert a Python integer to a C :c:expr:`unsigned long long`. .. versionchanged:: 3.14 Use :meth:`~object.__index__` if available. ``n`` (:class:`int`) [:c:type:`Py_ssize_t`] Convert a Python integer to a C :c:type:`Py_ssize_t`. ``c`` (:class:`bytes` or :class:`bytearray` of length 1) [char] Convert a Python byte, represented as a :class:`bytes` or :class:`bytearray` object of length 1, to a C :c:expr:`char`. .. versionchanged:: 3.3 Allow :class:`bytearray` objects. ``C`` (:class:`str` of length 1) [int] Convert a Python character, represented as a :class:`str` object of length 1, to a C :c:expr:`int`. ``f`` (:class:`float`) [float] Convert a Python floating-point number to a C :c:expr:`float`. ``d`` (:class:`float`) [double] Convert a Python floating-point number to a C :c:expr:`double`. ``D`` (:class:`complex`) [Py_complex] Convert a Python complex number to a C :c:type:`Py_complex` structure. .. deprecated:: 3.15 For unsigned integer formats ``B``, ``H``, ``I``, ``k`` and ``K``, :exc:`DeprecationWarning` is emitted when the value is larger than the maximal value for the C type or less than the minimal value for the corresponding signed integer type of the same size. Other objects ------------- ``O`` (object) [PyObject \*] Store a Python object (without any conversion) in a C object pointer. The C program thus receives the actual object that was passed. A new :term:`strong reference` to the object is not created (i.e. its reference count is not increased). The pointer stored is not ``NULL``. ``O!`` (object) [*typeobject*, PyObject \*] Store a Python object in a C object pointer. This is similar to ``O``, but takes two C arguments: the first is the address of a Python type object, the second is the address of the C variable (of type :c:expr:`PyObject*`) into which the object pointer is stored. If the Python object does not have the required type, :exc:`TypeError` is raised. .. _o_ampersand: ``O&`` (object) [*converter*, *address*] Convert a Python object to a C variable through a *converter* function. This takes two arguments: the first is a function, the second is the address of a C variable (of arbitrary type), converted to :c:expr:`void *`. The *converter* function in turn is called as follows:: status = converter(object, address); where *object* is the Python object to be converted and *address* is the :c:expr:`void*` argument that was passed to the ``PyArg_Parse*`` function. The returned *status* should be ``1`` for a successful conversion and ``0`` if the conversion has failed. When the conversion fails, the *converter* function should raise an exception and leave the content of *address* unmodified. .. c:macro:: Py_CLEANUP_SUPPORTED :no-typesetting: If the *converter* returns :c:macro:`!Py_CLEANUP_SUPPORTED`, it may get called a second time if the argument parsing eventually fails, giving the converter a chance to release any memory that it had already allocated. In this second call, the *object* parameter will be ``NULL``; *address* will have the same value as in the original call. Examples of converters: :c:func:`PyUnicode_FSConverter` and :c:func:`PyUnicode_FSDecoder`. .. versionchanged:: 3.1 :c:macro:`!Py_CLEANUP_SUPPORTED` was added. ``p`` (:class:`bool`) [int] Tests the value passed in for truth (a boolean **p**\ redicate) and converts the result to its equivalent C true/false integer value. Sets the int to ``1`` if the expression was true and ``0`` if it was false. This accepts any valid Python value. See :ref:`truth` for more information about how Python tests values for truth. .. versionadded:: 3.3 ``(items)`` (sequence) [*matching-items*] The object must be a Python sequence (except :class:`str`, :class:`bytes` or :class:`bytearray`) whose length is the number of format units in *items*. The C arguments must correspond to the individual format units in *items*. Format units for sequences may be nested. If *items* contains format units which store a :ref:`borrowed buffer ` (``s``, ``s#``, ``z``, ``z#``, ``y``, or ``y#``) or a :term:`borrowed reference` (``S``, ``Y``, ``U``, ``O``, or ``O!``), the object must be a Python tuple. The *converter* for the ``O&`` format unit in *items* must not store a borrowed buffer or a borrowed reference. .. versionchanged:: 3.14 :class:`str` and :class:`bytearray` objects no longer accepted as a sequence. .. deprecated:: 3.14 Non-tuple sequences are deprecated if *items* contains format units which store a borrowed buffer or a borrowed reference. A few other characters have a meaning in a format string. These may not occur inside nested parentheses. They are: ``|`` Indicates that the remaining arguments in the Python argument list are optional. The C variables corresponding to optional arguments should be initialized to their default value --- when an optional argument is not specified, :c:func:`PyArg_ParseTuple` does not touch the contents of the corresponding C variable(s). For example, the format string ``"OO|OO"`` corresponds to the Python signature ``f(a, b, c=None, d=None)``. ``$`` :c:func:`PyArg_ParseTupleAndKeywords` only: Indicates that the remaining arguments in the Python argument list are keyword-only. They are optional if ``|`` was specified before ``$``, and required otherwise. ``|`` cannot be specified after ``$``. For example, the format string ``"O|O$O"`` corresponds to the Python signature ``f(a, b=None, *, c=None)``, and the format string ``"OO$OO"`` corresponds to ``f(a, b, *, c, d)``. .. versionadded:: 3.3 ``:`` The list of format units ends here; the string after the colon is used as the function name in error messages (the "associated value" of the exception that :c:func:`PyArg_ParseTuple` raises). ``;`` The list of format units ends here; the string after the semicolon is used as the error message *instead* of the default error message. ``:`` and ``;`` mutually exclude each other. Note that any Python object references which are provided to the caller are *borrowed* references; do not release them (i.e. do not decrement their reference count)! Additional arguments passed to these functions must be addresses of variables whose type is determined by the format string; these are used to store values from the input tuple. There are a few cases, as described in the list of format units above, where these parameters are used as input values; they should match what is specified for the corresponding format unit in that case. For the conversion to succeed, the *arg* object must match the format and the format must be exhausted. On success, the ``PyArg_Parse*`` functions return true, otherwise they return false and raise an appropriate exception. When the ``PyArg_Parse*`` functions fail due to conversion failure in one of the format units, the variables at the addresses corresponding to that and the following format units are left untouched. API Functions ------------- .. c:function:: int PyArg_ParseTuple(PyObject *args, const char *format, ...) Parse the parameters of a function that takes only positional parameters into local variables. Returns true on success; on failure, it returns false and raises the appropriate exception. .. c:function:: int PyArg_VaParse(PyObject *args, const char *format, va_list vargs) Identical to :c:func:`PyArg_ParseTuple`, except that it accepts a va_list rather than a variable number of arguments. .. c:function:: int PyArg_ParseTupleAndKeywords(PyObject *args, PyObject *kw, const char *format, char * const *keywords, ...) Parse the parameters of a function that takes both positional and keyword parameters into local variables. The *keywords* argument is a ``NULL``-terminated array of keyword parameter names specified as null-terminated ASCII or UTF-8 encoded C strings. Empty names denote :ref:`positional-only parameters `. Returns true on success; on failure, it returns false and raises the appropriate exception. .. note:: The *keywords* parameter declaration is :c:expr:`char * const *` in C and :c:expr:`const char * const *` in C++. This can be overridden with the :c:macro:`PY_CXX_CONST` macro. .. versionchanged:: 3.6 Added support for :ref:`positional-only parameters `. .. versionchanged:: 3.13 The *keywords* parameter has now type :c:expr:`char * const *` in C and :c:expr:`const char * const *` in C++, instead of :c:expr:`char **`. Added support for non-ASCII keyword parameter names. .. c:function:: int PyArg_VaParseTupleAndKeywords(PyObject *args, PyObject *kw, const char *format, char * const *keywords, va_list vargs) Identical to :c:func:`PyArg_ParseTupleAndKeywords`, except that it accepts a va_list rather than a variable number of arguments. .. c:function:: int PyArg_ValidateKeywordArguments(PyObject *) Ensure that the keys in the keywords argument dictionary are strings. This is only needed if :c:func:`PyArg_ParseTupleAndKeywords` is not used, since the latter already does this check. .. versionadded:: 3.2 .. c:function:: int PyArg_Parse(PyObject *args, const char *format, ...) Parse the parameter of a function that takes a single positional parameter into a local variable. Returns true on success; on failure, it returns false and raises the appropriate exception. Example:: // Function using METH_O calling convention static PyObject* my_function(PyObject *module, PyObject *arg) { int value; if (!PyArg_Parse(arg, "i:my_function", &value)) { return NULL; } // ... use value ... } .. c:function:: int PyArg_ParseArray(PyObject *const *args, Py_ssize_t nargs, const char *format, ...) Parse the parameters of a function that takes only array parameters into local variables (that is, a function using the :c:macro:`METH_FASTCALL` calling convention). Returns true on success; on failure, it returns false and raises the appropriate exception. .. versionadded:: 3.15 .. c:function:: int PyArg_ParseArrayAndKeywords(PyObject *const *args, Py_ssize_t nargs, PyObject *kwnames, const char *format, const char * const *kwlist, ...) Parse the parameters of a function that takes both array and keyword parameters into local variables (that is, a function using the :c:macro:`METH_FASTCALL` ``|`` :c:macro:`METH_KEYWORDS` calling convention). Returns true on success; on failure, it returns false and raises the appropriate exception. .. versionadded:: 3.15 .. c:function:: int PyArg_UnpackTuple(PyObject *args, const char *name, Py_ssize_t min, Py_ssize_t max, ...) A simpler form of parameter retrieval which does not use a format string to specify the types of the arguments. Functions which use this method to retrieve their parameters should be declared as :c:macro:`METH_VARARGS` in function or method tables. The tuple containing the actual parameters should be passed as *args*; it must actually be a tuple. The length of the tuple must be at least *min* and no more than *max*; *min* and *max* may be equal. Additional arguments must be passed to the function, each of which should be a pointer to a :c:expr:`PyObject*` variable; these will be filled in with the values from *args*; they will contain :term:`borrowed references `. The variables which correspond to optional parameters not given by *args* will not be filled in; these should be initialized by the caller. This function returns true on success and false if *args* is not a tuple or contains the wrong number of elements; an exception will be set if there was a failure. This is an example of the use of this function, taken from the sources for the :mod:`!_weakref` helper module for weak references:: static PyObject * weakref_ref(PyObject *self, PyObject *args) { PyObject *object; PyObject *callback = NULL; PyObject *result = NULL; if (PyArg_UnpackTuple(args, "ref", 1, 2, &object, &callback)) { result = PyWeakref_NewRef(object, callback); } return result; } The call to :c:func:`PyArg_UnpackTuple` in this example is entirely equivalent to this call to :c:func:`PyArg_ParseTuple`:: PyArg_ParseTuple(args, "O|O:ref", &object, &callback) .. c:macro:: PY_CXX_CONST The value to be inserted, if any, before :c:expr:`char * const *` in the *keywords* parameter declaration of :c:func:`PyArg_ParseTupleAndKeywords` and :c:func:`PyArg_VaParseTupleAndKeywords`. Default empty for C and ``const`` for C++ (:c:expr:`const char * const *`). To override, define it to the desired value before including :file:`Python.h`. .. versionadded:: 3.13 --------------- Building values --------------- .. c:function:: PyObject* Py_BuildValue(const char *format, ...) Create a new value based on a format string similar to those accepted by the ``PyArg_Parse*`` family of functions and a sequence of values. Returns the value or ``NULL`` in the case of an error; an exception will be raised if ``NULL`` is returned. :c:func:`Py_BuildValue` does not always build a tuple. It builds a tuple only if its format string contains two or more format units. If the format string is empty, it returns ``None``; if it contains exactly one format unit, it returns whatever object is described by that format unit. To force it to return a tuple of size 0 or one, parenthesize the format string. When memory buffers are passed as parameters to supply data to build objects, as for the ``s`` and ``s#`` formats, the required data is copied. Buffers provided by the caller are never referenced by the objects created by :c:func:`Py_BuildValue`. In other words, if your code invokes :c:func:`malloc` and passes the allocated memory to :c:func:`Py_BuildValue`, your code is responsible for calling :c:func:`free` for that memory once :c:func:`Py_BuildValue` returns. In the following description, the quoted form is the format unit; the entry in (round) parentheses is the Python object type that the format unit will return; and the entry in [square] brackets is the type of the C value(s) to be passed. The characters space, tab, colon and comma are ignored in format strings (but not within format units such as ``s#``). This can be used to make long format strings a tad more readable. ``s`` (:class:`str` or ``None``) [const char \*] Convert a null-terminated C string to a Python :class:`str` object using ``'utf-8'`` encoding. If the C string pointer is ``NULL``, ``None`` is used. ``s#`` (:class:`str` or ``None``) [const char \*, :c:type:`Py_ssize_t`] Convert a C string and its length to a Python :class:`str` object using ``'utf-8'`` encoding. If the C string pointer is ``NULL``, the length is ignored and ``None`` is returned. ``y`` (:class:`bytes`) [const char \*] This converts a C string to a Python :class:`bytes` object. If the C string pointer is ``NULL``, ``None`` is returned. ``y#`` (:class:`bytes`) [const char \*, :c:type:`Py_ssize_t`] This converts a C string and its lengths to a Python object. If the C string pointer is ``NULL``, ``None`` is returned. ``z`` (:class:`str` or ``None``) [const char \*] Same as ``s``. ``z#`` (:class:`str` or ``None``) [const char \*, :c:type:`Py_ssize_t`] Same as ``s#``. ``u`` (:class:`str`) [const wchar_t \*] Convert a null-terminated :c:type:`wchar_t` buffer of Unicode (UTF-16 or UCS-4) data to a Python Unicode object. If the Unicode buffer pointer is ``NULL``, ``None`` is returned. ``u#`` (:class:`str`) [const wchar_t \*, :c:type:`Py_ssize_t`] Convert a Unicode (UTF-16 or UCS-4) data buffer and its length to a Python Unicode object. If the Unicode buffer pointer is ``NULL``, the length is ignored and ``None`` is returned. ``U`` (:class:`str` or ``None``) [const char \*] Same as ``s``. ``U#`` (:class:`str` or ``None``) [const char \*, :c:type:`Py_ssize_t`] Same as ``s#``. ``i`` (:class:`int`) [int] Convert a plain C :c:expr:`int` to a Python integer object. ``b`` (:class:`int`) [char] Convert a plain C :c:expr:`char` to a Python integer object. ``h`` (:class:`int`) [short int] Convert a plain C :c:expr:`short int` to a Python integer object. ``l`` (:class:`int`) [long int] Convert a C :c:expr:`long int` to a Python integer object. ``B`` (:class:`int`) [unsigned char] Convert a C :c:expr:`unsigned char` to a Python integer object. ``H`` (:class:`int`) [unsigned short int] Convert a C :c:expr:`unsigned short int` to a Python integer object. ``I`` (:class:`int`) [unsigned int] Convert a C :c:expr:`unsigned int` to a Python integer object. ``k`` (:class:`int`) [unsigned long] Convert a C :c:expr:`unsigned long` to a Python integer object. ``L`` (:class:`int`) [long long] Convert a C :c:expr:`long long` to a Python integer object. .. _capi-py-buildvalue-format-K: ``K`` (:class:`int`) [unsigned long long] Convert a C :c:expr:`unsigned long long` to a Python integer object. ``n`` (:class:`int`) [:c:type:`Py_ssize_t`] Convert a C :c:type:`Py_ssize_t` to a Python integer. ``p`` (:class:`bool`) [int] Convert a C :c:expr:`int` to a Python :class:`bool` object. Be aware that this format requires an ``int`` argument. Unlike most other contexts in C, variadic arguments are not coerced to a suitable type automatically. You can convert another type (for example, a pointer or a float) to a suitable ``int`` value using ``(x) ? 1 : 0`` or ``!!x``. .. versionadded:: 3.14 ``c`` (:class:`bytes` of length 1) [char] Convert a C :c:expr:`int` representing a byte to a Python :class:`bytes` object of length 1. ``C`` (:class:`str` of length 1) [int] Convert a C :c:expr:`int` representing a character to Python :class:`str` object of length 1. ``d`` (:class:`float`) [double] Convert a C :c:expr:`double` to a Python floating-point number. ``f`` (:class:`float`) [float] Convert a C :c:expr:`float` to a Python floating-point number. ``D`` (:class:`complex`) [Py_complex \*] Convert a C :c:type:`Py_complex` structure to a Python complex number. ``O`` (object) [PyObject \*] Pass a Python object untouched but create a new :term:`strong reference` to it (i.e. its reference count is incremented by one). If the object passed in is a ``NULL`` pointer, it is assumed that this was caused because the call producing the argument found an error and set an exception. Therefore, :c:func:`Py_BuildValue` will return ``NULL`` but won't raise an exception. If no exception has been raised yet, :exc:`SystemError` is set. ``S`` (object) [PyObject \*] Same as ``O``. ``N`` (object) [PyObject \*] Same as ``O``, except it doesn't create a new :term:`strong reference`. Useful when the object is created by a call to an object constructor in the argument list. ``O&`` (object) [*converter*, *anything*] Convert *anything* to a Python object through a *converter* function. The function is called with *anything* (which should be compatible with :c:expr:`void*`) as its argument and should return a "new" Python object, or ``NULL`` if an error occurred. ``(items)`` (:class:`tuple`) [*matching-items*] Convert a sequence of C values to a Python tuple with the same number of items. ``[items]`` (:class:`list`) [*matching-items*] Convert a sequence of C values to a Python list with the same number of items. ``{items}`` (:class:`dict`) [*matching-items*] Convert a sequence of C values to a Python dictionary. Each pair of consecutive C values adds one item to the dictionary, serving as key and value, respectively. If there is an error in the format string, the :exc:`SystemError` exception is set and ``NULL`` returned. .. c:function:: PyObject* Py_VaBuildValue(const char *format, va_list vargs) Identical to :c:func:`Py_BuildValue`, except that it accepts a va_list rather than a variable number of arguments. --- ### Doc C Api Bool (`Doc/c-api/bool.rst`) .. highlight:: c .. _boolobjects: Boolean Objects --------------- Booleans in Python are implemented as a subclass of integers. There are only two booleans, :c:data:`Py_False` and :c:data:`Py_True`. As such, the normal creation and deletion functions don't apply to booleans. The following macros are available, however. .. c:var:: PyTypeObject PyBool_Type This instance of :c:type:`PyTypeObject` represents the Python boolean type; it is the same object as :class:`bool` in the Python layer. .. c:function:: int PyBool_Check(PyObject *o) Return true if *o* is of type :c:data:`PyBool_Type`. This function always succeeds. .. c:var:: PyObject* Py_False The Python ``False`` object. This object has no methods and is :term:`immortal`. .. versionchanged:: 3.12 :c:data:`Py_False` is :term:`immortal`. .. c:var:: PyObject* Py_True The Python ``True`` object. This object has no methods and is :term:`immortal`. .. versionchanged:: 3.12 :c:data:`Py_True` is :term:`immortal`. .. c:macro:: Py_RETURN_FALSE Return :c:data:`Py_False` from a function. .. c:macro:: Py_RETURN_TRUE Return :c:data:`Py_True` from a function. .. c:function:: PyObject* PyBool_FromLong(long v) Return :c:data:`Py_True` or :c:data:`Py_False`, depending on the truth value of *v*. --- ### Doc C Api Buffer (`Doc/c-api/buffer.rst`) .. highlight:: c .. index:: single: buffer protocol single: buffer interface; (see buffer protocol) single: buffer object; (see buffer protocol) .. _bufferobjects: Buffer Protocol --------------- Certain objects available in Python wrap access to an underlying memory array or *buffer*. Such objects include the built-in :class:`bytes` and :class:`bytearray`, and some extension types like :class:`array.array`. Third-party libraries may define their own types for special purposes, such as image processing or numeric analysis. While each of these types have their own semantics, they share the common characteristic of being backed by a possibly large memory buffer. It is then desirable, in some situations, to access that buffer directly and without intermediate copying. Python provides such a facility at the C and Python level in the form of the :ref:`buffer protocol `. This protocol has two sides: .. index:: single: PyBufferProcs (C type) - on the producer side, a type can export a "buffer interface" which allows objects of that type to expose information about their underlying buffer. This interface is described in the section :ref:`buffer-structs`; for Python see :ref:`python-buffer-protocol`. - on the consumer side, several means are available to obtain a pointer to the raw underlying data of an object (for example a method parameter). For Python see :class:`memoryview`. Simple objects such as :class:`bytes` and :class:`bytearray` expose their underlying buffer in byte-oriented form. Other forms are possible; for example, the elements exposed by an :class:`array.array` can be multi-byte values. An example consumer of the buffer interface is the :meth:`~io.BufferedIOBase.write` method of file objects: any object that can export a series of bytes through the buffer interface can be written to a file. While :meth:`!write` only needs read-only access to the internal contents of the object passed to it, other methods such as :meth:`~io.BufferedIOBase.readinto` need write access to the contents of their argument. The buffer interface allows objects to selectively allow or reject exporting of read-write and read-only buffers. There are two ways for a consumer of the buffer interface to acquire a buffer over a target object: * call :c:func:`PyObject_GetBuffer` with the right parameters; * call :c:func:`PyArg_ParseTuple` (or one of its siblings) with one of the ``y*``, ``w*`` or ``s*`` :ref:`format codes `. In both cases, :c:func:`PyBuffer_Release` must be called when the buffer isn't needed anymore. Failure to do so could lead to various issues such as resource leaks. .. versionadded:: 3.12 The buffer protocol is now accessible in Python, see :ref:`python-buffer-protocol` and :class:`memoryview`. .. _buffer-structure: Buffer structure ================ Buffer structures (or simply "buffers") are useful as a way to expose the binary data from another object to the Python programmer. They can also be used as a zero-copy slicing mechanism. Using their ability to reference a block of memory, it is possible to expose any data to the Python programmer quite easily. The memory could be a large, constant array in a C extension, it could be a raw block of memory for manipulation before passing to an operating system library, or it could be used to pass around structured data in its native, in-memory format. Contrary to most data types exposed by the Python interpreter, buffers are not :c:type:`PyObject` pointers but rather simple C structures. This allows them to be created and copied very simply. When a generic wrapper around a buffer is needed, a :ref:`memoryview ` object can be created. For short instructions how to write an exporting object, see :ref:`Buffer Object Structures `. For obtaining a buffer, see :c:func:`PyObject_GetBuffer`. .. c:type:: Py_buffer .. c:member:: void *buf A pointer to the start of the logical structure described by the buffer fields. This can be any location within the underlying physical memory block of the exporter. For example, with negative :c:member:`~Py_buffer.strides` the value may point to the end of the memory block. For :term:`contiguous` arrays, the value points to the beginning of the memory block. .. c:member:: PyObject *obj A new reference to the exporting object. The reference is owned by the consumer and automatically released (i.e. reference count decremented) and set to ``NULL`` by :c:func:`PyBuffer_Release`. The field is the equivalent of the return value of any standard C-API function. As a special case, for *temporary* buffers that are wrapped by :c:func:`PyMemoryView_FromBuffer` or :c:func:`PyBuffer_FillInfo` this field is ``NULL``. In general, exporting objects MUST NOT use this scheme. .. c:member:: Py_ssize_t len ``product(shape) * itemsize``. For contiguous arrays, this is the length of the underlying memory block. For non-contiguous arrays, it is the length that the logical structure would have if it were copied to a contiguous representation. Accessing ``((char *)buf)[0] up to ((char *)buf)[len-1]`` is only valid if the buffer has been obtained by a request that guarantees contiguity. In most cases such a request will be :c:macro:`PyBUF_SIMPLE` or :c:macro:`PyBUF_WRITABLE`. .. c:member:: int readonly An indicator of whether the buffer is read-only. This field is controlled by the :c:macro:`PyBUF_WRITABLE` flag. .. c:member:: Py_ssize_t itemsize Item size in bytes of a single element. Same as the value of :func:`struct.calcsize` called on non-``NULL`` :c:member:`~Py_buffer.format` values. Important exception: If a consumer requests a buffer without the :c:macro:`PyBUF_FORMAT` flag, :c:member:`~Py_buffer.format` will be set to ``NULL``, but :c:member:`~Py_buffer.itemsize` still has the value for the original format. If :c:member:`~Py_buffer.shape` is present, the equality ``product(shape) * itemsize == len`` still holds and the consumer can use :c:member:`~Py_buffer.itemsize` to navigate the buffer. If :c:member:`~Py_buffer.shape` is ``NULL`` as a result of a :c:macro:`PyBUF_SIMPLE` or a :c:macro:`PyBUF_WRITABLE` request, the consumer must disregard :c:member:`~Py_buffer.itemsize` and assume ``itemsize == 1``. .. c:member:: char *format A *NULL* terminated string in :mod:`struct` module style syntax describing the contents of a single item. If this is ``NULL``, ``"B"`` (unsigned bytes) is assumed. This field is controlled by the :c:macro:`PyBUF_FORMAT` flag. .. c:member:: int ndim The number of dimensions the memory represents as an n-dimensional array. If it is ``0``, :c:member:`~Py_buffer.buf` points to a single item representing a scalar. In this case, :c:member:`~Py_buffer.shape`, :c:member:`~Py_buffer.strides` and :c:member:`~Py_buffer.suboffsets` MUST be ``NULL``. The maximum number of dimensions is given by :c:macro:`PyBUF_MAX_NDIM`. .. c:member:: Py_ssize_t *shape An array of :c:type:`Py_ssize_t` of length :c:member:`~Py_buffer.ndim` indicating the shape of the memory as an n-dimensional array. Note that ``shape[0] * ... * shape[ndim-1] * itemsize`` MUST be equal to :c:member:`~Py_buffer.len`. Shape values are restricted to ``shape[n] >= 0``. The case ``shape[n] == 0`` requires special attention. See `complex arrays`_ for further information. The shape array is read-only for the consumer. .. c:member:: Py_ssize_t *strides An array of :c:type:`Py_ssize_t` of length :c:member:`~Py_buffer.ndim` giving the number of bytes to skip to get to a new element in each dimension. Stride values can be any integer. For regular arrays, strides are usually positive, but a consumer MUST be able to handle the case ``strides[n] <= 0``. See `complex arrays`_ for further information. The strides array is read-only for the consumer. .. c:member:: Py_ssize_t *suboffsets An array of :c:type:`Py_ssize_t` of length :c:member:`~Py_buffer.ndim`. If ``suboffsets[n] >= 0``, the values stored along the nth dimension are pointers and the suboffset value dictates how many bytes to add to each pointer after de-referencing. A suboffset value that is negative indicates that no de-referencing should occur (striding in a contiguous memory block). If all suboffsets are negative (i.e. no de-referencing is needed), then this field must be ``NULL`` (the default value). This type of array representation is used by the Python Imaging Library (PIL). See `complex arrays`_ for further information how to access elements of such an array. The suboffsets array is read-only for the consumer. .. c:member:: void *internal This is for use internally by the exporting object. For example, this might be re-cast as an integer by the exporter and used to store flags about whether or not the shape, strides, and suboffsets arrays must be freed when the buffer is released. The consumer MUST NOT alter this value. Constants: .. c:macro:: PyBUF_MAX_NDIM The maximum number of dimensions the memory represents. Exporters MUST respect this limit, consumers of multi-dimensional buffers SHOULD be able to handle up to :c:macro:`!PyBUF_MAX_NDIM` dimensions. Currently set to 64. .. _buffer-request-types: Buffer request types ==================== Buffers are usually obtained by sending a buffer request to an exporting object via :c:func:`PyObject_GetBuffer`. Since the complexity of the logical structure of the memory can vary drastically, the consumer uses the *flags* argument to specify the exact buffer type it can handle. All :c:type:`Py_buffer` fields are unambiguously defined by the request type. request-independent fields ~~~~~~~~~~~~~~~~~~~~~~~~~~ The following fields are not influenced by *flags* and must always be filled in with the correct values: :c:member:`~Py_buffer.obj`, :c:member:`~Py_buffer.buf`, :c:member:`~Py_buffer.len`, :c:member:`~Py_buffer.itemsize`, :c:member:`~Py_buffer.ndim`. readonly, format ~~~~~~~~~~~~~~~~ .. c:macro:: PyBUF_WRITABLE Controls the :c:member:`~Py_buffer.readonly` field. If set, the exporter MUST provide a writable buffer or else report failure. Otherwise, the exporter MAY provide either a read-only or writable buffer, but the choice MUST be consistent for all consumers. For example, :c:expr:`PyBUF_SIMPLE | PyBUF_WRITABLE` can be used to request a simple writable buffer. .. c:macro:: PyBUF_WRITEABLE This is an alias to :c:macro:`PyBUF_WRITABLE`. .. soft-deprecated:: 3.13 .. c:macro:: PyBUF_FORMAT Controls the :c:member:`~Py_buffer.format` field. If set, this field MUST be filled in correctly. Otherwise, this field MUST be ``NULL``. :c:macro:`PyBUF_WRITABLE` can be \|'d to any of the flags in the next section. Since :c:macro:`PyBUF_SIMPLE` is defined as 0, :c:macro:`PyBUF_WRITABLE` can be used as a stand-alone flag to request a simple writable buffer. :c:macro:`PyBUF_FORMAT` must be \|'d to any of the flags except :c:macro:`PyBUF_SIMPLE`, because the latter already implies format ``B`` (unsigned bytes). :c:macro:`!PyBUF_FORMAT` cannot be used on its own. shape, strides, suboffsets ~~~~~~~~~~~~~~~~~~~~~~~~~~ The flags that control the logical structure of the memory are listed in decreasing order of complexity. Note that each flag contains all bits of the flags below it. .. tabularcolumns:: |p{0.35\linewidth}|l|l|l| +-----------------------------+-------+---------+------------+ | Request | shape | strides | suboffsets | +=============================+=======+=========+============+ | .. c:macro:: PyBUF_INDIRECT | yes | yes | if needed | +-----------------------------+-------+---------+------------+ | .. c:macro:: PyBUF_STRIDES | yes | yes | NULL | +-----------------------------+-------+---------+------------+ | .. c:macro:: PyBUF_ND | yes | NULL | NULL | +-----------------------------+-------+---------+------------+ | .. c:macro:: PyBUF_SIMPLE | NULL | NULL | NULL | +-----------------------------+-------+---------+------------+ .. index:: contiguous, C-contiguous, Fortran contiguous contiguity requests ~~~~~~~~~~~~~~~~~~~ C or Fortran :term:`contiguity ` can be explicitly requested, with and without stride information. Without stride information, the buffer must be C-contiguous. .. tabularcolumns:: |p{0.35\linewidth}|l|l|l|l| +-----------------------------------+-------+---------+------------+--------+ | Request | shape | strides | suboffsets | contig | +===================================+=======+=========+============+========+ | .. c:macro:: PyBUF_C_CONTIGUOUS | yes | yes | NULL | C | +-----------------------------------+-------+---------+------------+--------+ | .. c:macro:: PyBUF_F_CONTIGUOUS | yes | yes | NULL | F | +-----------------------------------+-------+---------+------------+--------+ | .. c:macro:: PyBUF_ANY_CONTIGUOUS | yes | yes | NULL | C or F | +-----------------------------------+-------+---------+------------+--------+ | :c:macro:`PyBUF_ND` | yes | NULL | NULL | C | +-----------------------------------+-------+---------+------------+--------+ compound requests ~~~~~~~~~~~~~~~~~ All possible requests are fully defined by some combination of the flags in the previous section. For convenience, the buffer protocol provides frequently used combinations as single flags. In the following table *U* stands for undefined contiguity. The consumer would have to call :c:func:`PyBuffer_IsContiguous` to determine contiguity. .. tabularcolumns:: |p{0.35\linewidth}|l|l|l|l|l|l| +-------------------------------+-------+---------+------------+--------+----------+--------+ | Request | shape | strides | suboffsets | contig | readonly | format | +===============================+=======+=========+============+========+==========+========+ | .. c:macro:: PyBUF_FULL | yes | yes | if needed | U | 0 | yes | +-------------------------------+-------+---------+------------+--------+----------+--------+ | .. c:macro:: PyBUF_FULL_RO | yes | yes | if needed | U | 1 or 0 | yes | +-------------------------------+-------+---------+------------+--------+----------+--------+ | .. c:macro:: PyBUF_RECORDS | yes | yes | NULL | U | 0 | yes | +-------------------------------+-------+---------+------------+--------+----------+--------+ | .. c:macro:: PyBUF_RECORDS_RO | yes | yes | NULL | U | 1 or 0 | yes | +-------------------------------+-------+---------+------------+--------+----------+--------+ | .. c:macro:: PyBUF_STRIDED | yes | yes | NULL | U | 0 | NULL | +-------------------------------+-------+---------+------------+--------+----------+--------+ | .. c:macro:: PyBUF_STRIDED_RO | yes | yes | NULL | U | 1 or 0 | NULL | +-------------------------------+-------+---------+------------+--------+----------+--------+ | .. c:macro:: PyBUF_CONTIG | yes | NULL | NULL | C | 0 | NULL | +-------------------------------+-------+---------+------------+--------+----------+--------+ | .. c:macro:: PyBUF_CONTIG_RO | yes | NULL | NULL | C | 1 or 0 | NULL | +-------------------------------+-------+---------+------------+--------+----------+--------+ Complex arrays ============== NumPy-style: shape and strides ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ The logical structure of NumPy-style arrays is defined by :c:member:`~Py_buffer.itemsize`, :c:member:`~Py_buffer.ndim`, :c:member:`~Py_buffer.shape` and :c:member:`~Py_buffer.strides`. If ``ndim == 0``, the memory location pointed to by :c:member:`~Py_buffer.buf` is interpreted as a scalar of size :c:member:`~Py_buffer.itemsize`. In that case, both :c:member:`~Py_buffer.shape` and :c:member:`~Py_buffer.strides` are ``NULL``. If :c:member:`~Py_buffer.strides` is ``NULL``, the array is interpreted as a standard n-dimensional C-array. Otherwise, the consumer must access an n-dimensional array as follows: .. code-block:: c ptr = (char *)buf + indices[0] * strides[0] + ... + indices[n-1] * strides[n-1]; item = *((typeof(item) *)ptr); As noted above, :c:member:`~Py_buffer.buf` can point to any location within the actual memory block. An exporter can check the validity of a buffer with this function: .. code-block:: python def verify_structure(memlen, itemsize, ndim, shape, strides, offset): """Verify that the parameters represent a valid array within the bounds of the allocated memory: char *mem: start of the physical memory block memlen: length of the physical memory block offset: (char *)buf - mem """ if offset % itemsize: return False if offset < 0 or offset+itemsize > memlen: return False if any(v % itemsize for v in strides): return False if ndim <= 0: return ndim == 0 and not shape and not strides if 0 in shape: return True imin = sum(strides[j]*(shape[j]-1) for j in range(ndim) if strides[j] <= 0) imax = sum(strides[j]*(shape[j]-1) for j in range(ndim) if strides[j] > 0) return 0 <= offset+imin and offset+imax+itemsize <= memlen PIL-style: shape, strides and suboffsets ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ In addition to the regular items, PIL-style arrays can contain pointers that must be followed in order to get to the next element in a dimension. For example, the regular three-dimensional C-array ``char v[2][2][3]`` can also be viewed as an array of 2 pointers to 2 two-dimensional arrays: ``char (*v[2])[2][3]``. In suboffsets representation, those two pointers can be embedded at the start of :c:member:`~Py_buffer.buf`, pointing to two ``char x[2][3]`` arrays that can be located anywhere in memory. Here is a function that returns a pointer to the element in an N-D array pointed to by an N-dimensional index when there are both non-``NULL`` strides and suboffsets:: void *get_item_pointer(int ndim, void *buf, Py_ssize_t *strides, Py_ssize_t *suboffsets, Py_ssize_t *indices) { char *pointer = (char*)buf; int i; for (i = 0; i < ndim; i++) { pointer += strides[i] * indices[i]; if (suboffsets[i] >=0 ) { pointer = *((char**)pointer) + suboffsets[i]; } } return (void*)pointer; } Buffer-related functions ======================== .. c:function:: int PyObject_CheckBuffer(PyObject *obj) Return ``1`` if *obj* supports the buffer interface otherwise ``0``. When ``1`` is returned, it doesn't guarantee that :c:func:`PyObject_GetBuffer` will succeed. This function always succeeds. .. c:function:: int PyObject_GetBuffer(PyObject *exporter, Py_buffer *view, int flags) Send a request to *exporter* to fill in *view* as specified by *flags*. If the exporter cannot provide a buffer of the exact type, it MUST raise :exc:`BufferError`, set ``view->obj`` to ``NULL`` and return ``-1``. On success, fill in *view*, set ``view->obj`` to a new reference to *exporter* and return 0. In the case of chained buffer providers that redirect requests to a single object, ``view->obj`` MAY refer to this object instead of *exporter* (See :ref:`Buffer Object Structures `). Successful calls to :c:func:`PyObject_GetBuffer` must be paired with calls to :c:func:`PyBuffer_Release`, similar to :c:func:`malloc` and :c:func:`free`. Thus, after the consumer is done with the buffer, :c:func:`PyBuffer_Release` must be called exactly once. .. c:function:: void PyBuffer_Release(Py_buffer *view) Release the buffer *view* and release the :term:`strong reference` (i.e. decrement the reference count) to the view's supporting object, ``view->obj``. This function MUST be called when the buffer is no longer being used, otherwise reference leaks may occur. It is an error to call this function on a buffer that was not obtained via :c:func:`PyObject_GetBuffer`. .. c:function:: Py_ssize_t PyBuffer_SizeFromFormat(const char *format) Return the implied :c:member:`~Py_buffer.itemsize` from :c:member:`~Py_buffer.format`. On error, raise an exception and return -1. .. versionadded:: 3.9 .. c:function:: int PyBuffer_IsContiguous(const Py_buffer *view, char order) Return ``1`` if the memory defined by the *view* is C-style (*order* is ``'C'``) or Fortran-style (*order* is ``'F'``) :term:`contiguous` or either one (*order* is ``'A'``). Return ``0`` otherwise. This function always succeeds. .. c:function:: void* PyBuffer_GetPointer(const Py_buffer *view, const Py_ssize_t *indices) Get the memory area pointed to by the *indices* inside the given *view*. *indices* must point to an array of ``view->ndim`` indices. .. c:function:: int PyBuffer_FromContiguous(const Py_buffer *view, const void *buf, Py_ssize_t len, char order) Copy contiguous *len* bytes from *buf* to *view*. *order* can be ``'C'`` or ``'F'`` or ``'A'`` (for C-style or Fortran-style ordering or either one). ``0`` is returned on success, ``-1`` on error. .. c:function:: int PyBuffer_ToContiguous(void *buf, const Py_buffer *src, Py_ssize_t len, char order) Copy *len* bytes from *src* to its contiguous representation in *buf*. *order* can be ``'C'`` or ``'F'`` or ``'A'`` (for C-style or Fortran-style ordering or either one). ``0`` is returned on success, ``-1`` on error. This function fails if *len* != *src->len*. .. c:function:: int PyObject_CopyData(PyObject *dest, PyObject *src) Copy data from *src* to *dest* buffer. Can convert between C-style and or Fortran-style buffers. ``0`` is returned on success, ``-1`` on error. .. c:function:: void PyBuffer_FillContiguousStrides(int ndims, Py_ssize_t *shape, Py_ssize_t *strides, int itemsize, char order) Fill the *strides* array with byte-strides of a :term:`contiguous` (C-style if *order* is ``'C'`` or Fortran-style if *order* is ``'F'``) array of the given shape with the given number of bytes per element. .. c:function:: int PyBuffer_FillInfo(Py_buffer *view, PyObject *exporter, void *buf, Py_ssize_t len, int readonly, int flags) Handle buffer requests for an exporter that wants to expose *buf* of size *len* with writability set according to *readonly*. *buf* is interpreted as a sequence of unsigned bytes. The *flags* argument indicates the request type. This function always fills in *view* as specified by flags, unless *buf* has been designated as read-only and :c:macro:`PyBUF_WRITABLE` is set in *flags*. On success, set ``view->obj`` to a new reference to *exporter* and return 0. Otherwise, raise :exc:`BufferError`, set ``view->obj`` to ``NULL`` and return ``-1``; If this function is used as part of a :ref:`getbufferproc `, *exporter* MUST be set to the exporting object and *flags* must be passed unmodified. Otherwise, *exporter* MUST be ``NULL``. --- ### Doc C Api Bytearray (`Doc/c-api/bytearray.rst`) .. highlight:: c .. _bytearrayobjects: Byte Array Objects ------------------ .. index:: pair: object; bytearray .. c:type:: PyByteArrayObject This subtype of :c:type:`PyObject` represents a Python bytearray object. .. impl-detail:: The internal buffer of :c:type:`PyByteArrayObject` always includes an extra trailing null byte for compatibility with null terminated C strings. This extra byte is not counted in :c:func:`PyByteArray_Size` nor in the *len* arguments of the functions below. .. c:var:: PyTypeObject PyByteArray_Type This instance of :c:type:`PyTypeObject` represents the Python bytearray type; it is the same object as :class:`bytearray` in the Python layer. Type check macros ^^^^^^^^^^^^^^^^^ .. c:function:: int PyByteArray_Check(PyObject *o) Return true if the object *o* is a bytearray object or an instance of a subtype of the bytearray type. This function always succeeds. .. c:function:: int PyByteArray_CheckExact(PyObject *o) Return true if the object *o* is a bytearray object, but not an instance of a subtype of the bytearray type. This function always succeeds. Direct API functions ^^^^^^^^^^^^^^^^^^^^ .. c:function:: PyObject* PyByteArray_FromObject(PyObject *o) Return a new bytearray object from any object, *o*, that implements the :ref:`buffer protocol `. On failure, return ``NULL`` with an exception set. .. note:: If the object implements the buffer protocol, then the buffer must not be mutated while the bytearray object is being created. .. c:function:: PyObject* PyByteArray_FromStringAndSize(const char *string, Py_ssize_t len) Create a new bytearray object from *string* and its length, *len*. On failure, return ``NULL`` with an exception set. .. c:function:: PyObject* PyByteArray_Concat(PyObject *a, PyObject *b) Concat bytearrays *a* and *b* and return a new bytearray with the result. On failure, return ``NULL`` with an exception set. .. note:: If the object implements the buffer protocol, then the buffer must not be mutated while the bytearray object is being created. .. c:function:: Py_ssize_t PyByteArray_Size(PyObject *bytearray) Return the size of *bytearray* after checking for a ``NULL`` pointer. .. c:function:: char* PyByteArray_AsString(PyObject *bytearray) Return the contents of *bytearray* as a char array after checking for a ``NULL`` pointer. The returned array always has an extra null byte appended. .. note:: It is not thread-safe to mutate the bytearray object while using the returned char array. .. c:function:: int PyByteArray_Resize(PyObject *bytearray, Py_ssize_t len) Resize the internal buffer of *bytearray* to *len*. Failure is a ``-1`` return with an exception set. .. versionchanged:: 3.14 A negative *len* will now result in an exception being set and -1 returned. Macros ^^^^^^ These macros trade safety for speed and they don't check pointers. .. c:function:: char* PyByteArray_AS_STRING(PyObject *bytearray) Similar to :c:func:`PyByteArray_AsString`, but without error checking. .. note:: It is not thread-safe to mutate the bytearray object while using the returned char array. .. c:function:: Py_ssize_t PyByteArray_GET_SIZE(PyObject *bytearray) Similar to :c:func:`PyByteArray_Size`, but without error checking. --- ### Doc C Api Bytes (`Doc/c-api/bytes.rst`) .. highlight:: c .. _bytesobjects: Bytes Objects ------------- These functions raise :exc:`TypeError` when expecting a bytes parameter and called with a non-bytes parameter. .. impl-detail:: The internal buffer of :c:type:`PyBytesObject` always includes an extra trailing null byte for compatibility with null terminated C strings. This extra byte is not counted in :c:func:`PyBytes_Size` nor in the various *length* and *size* arguments of the functions below. .. index:: pair: object; bytes .. c:type:: PyBytesObject This subtype of :c:type:`PyObject` represents a Python bytes object. .. c:var:: PyTypeObject PyBytes_Type This instance of :c:type:`PyTypeObject` represents the Python bytes type; it is the same object as :class:`bytes` in the Python layer. .. c:function:: int PyBytes_Check(PyObject *o) Return true if the object *o* is a bytes object or an instance of a subtype of the bytes type. This function always succeeds. .. c:function:: int PyBytes_CheckExact(PyObject *o) Return true if the object *o* is a bytes object, but not an instance of a subtype of the bytes type. This function always succeeds. .. c:function:: PyObject* PyBytes_FromString(const char *v) Return a new bytes object with a copy of the string *v* as value on success, and ``NULL`` on failure. The parameter *v* must not be ``NULL``; it will not be checked. .. c:function:: PyObject* PyBytes_FromStringAndSize(const char *v, Py_ssize_t len) Return a new bytes object with a copy of the string *v* as value and length *len* on success, and ``NULL`` on failure. If *v* is ``NULL``, the contents of the bytes object are uninitialized. .. soft-deprecated:: 3.15 Use the :c:type:`PyBytesWriter` API instead of ``PyBytes_FromStringAndSize(NULL, len)``. .. c:function:: PyObject* PyBytes_FromFormat(const char *format, ...) Take a C :c:func:`printf`\ -style *format* string and a variable number of arguments, calculate the size of the resulting Python bytes object and return a bytes object with the values formatted into it. The variable arguments must be C types and must correspond exactly to the format characters in the *format* string. The following format characters are allowed: .. % XXX: This should be exactly the same as the table in PyErr_Format. .. % One should just refer to the other. .. tabularcolumns:: |l|l|L| +-------------------+---------------+--------------------------------+ | Format Characters | Type | Comment | +===================+===============+================================+ | ``%%`` | *n/a* | The literal % character. | +-------------------+---------------+--------------------------------+ | ``%c`` | int | A single byte, | | | | represented as a C int. | +-------------------+---------------+--------------------------------+ | ``%d`` | int | Equivalent to | | | | ``printf("%d")``. [1]_ | +-------------------+---------------+--------------------------------+ | ``%u`` | unsigned int | Equivalent to | | | | ``printf("%u")``. [1]_ | +-------------------+---------------+--------------------------------+ | ``%ld`` | long | Equivalent to | | | | ``printf("%ld")``. [1]_ | +-------------------+---------------+--------------------------------+ | ``%lu`` | unsigned long | Equivalent to | | | | ``printf("%lu")``. [1]_ | +-------------------+---------------+--------------------------------+ | ``%zd`` | :c:type:`\ | Equivalent to | | | Py_ssize_t` | ``printf("%zd")``. [1]_ | +-------------------+---------------+--------------------------------+ | ``%zu`` | size_t | Equivalent to | | | | ``printf("%zu")``. [1]_ | +-------------------+---------------+--------------------------------+ | ``%i`` | int | Equivalent to | | | | ``printf("%i")``. [1]_ | +-------------------+---------------+--------------------------------+ | ``%x`` | int | Equivalent to | | | | ``printf("%x")``. [1]_ | +-------------------+---------------+--------------------------------+ | ``%s`` | const char\* | A null-terminated C character | | | | array. | +-------------------+---------------+--------------------------------+ | ``%p`` | const void\* | The hex representation of a C | | | | pointer. Mostly equivalent to | | | | ``printf("%p")`` except that | | | | it is guaranteed to start with | | | | the literal ``0x`` regardless | | | | of what the platform's | | | | ``printf`` yields. | +-------------------+---------------+--------------------------------+ An unrecognized format character causes all the rest of the format string to be copied as-is to the result object, and any extra arguments discarded. .. [1] For integer specifiers (d, u, ld, lu, zd, zu, i, x): the 0-conversion flag has effect even when a precision is given. .. c:function:: PyObject* PyBytes_FromFormatV(const char *format, va_list vargs) Identical to :c:func:`PyBytes_FromFormat` except that it takes exactly two arguments. .. c:function:: PyObject* PyBytes_FromObject(PyObject *o) Return the bytes representation of object *o* that implements the buffer protocol. .. note:: If the object implements the buffer protocol, then the buffer must not be mutated while the bytes object is being created. .. c:function:: Py_ssize_t PyBytes_Size(PyObject *o) Return the length of the bytes in bytes object *o*. .. c:function:: Py_ssize_t PyBytes_GET_SIZE(PyObject *o) Similar to :c:func:`PyBytes_Size`, but without error checking. .. c:function:: char* PyBytes_AsString(PyObject *o) Return a pointer to the contents of *o*. The pointer refers to the internal buffer of *o*, which consists of ``len(o) + 1`` bytes. The last byte in the buffer is always null, regardless of whether there are any other null bytes. The data must not be modified in any way, unless the object was just created using ``PyBytes_FromStringAndSize(NULL, size)``. It must not be deallocated. If *o* is not a bytes object at all, :c:func:`PyBytes_AsString` returns ``NULL`` and raises :exc:`TypeError`. .. c:function:: char* PyBytes_AS_STRING(PyObject *string) Similar to :c:func:`PyBytes_AsString`, but without error checking. .. c:function:: int PyBytes_AsStringAndSize(PyObject *obj, char **buffer, Py_ssize_t *length) Return the null-terminated contents of the object *obj* through the output variables *buffer* and *length*. Returns ``0`` on success. If *length* is ``NULL``, the bytes object may not contain embedded null bytes; if it does, the function returns ``-1`` and a :exc:`ValueError` is raised. The buffer refers to an internal buffer of *obj*, which includes an additional null byte at the end (not counted in *length*). The data must not be modified in any way, unless the object was just created using ``PyBytes_FromStringAndSize(NULL, size)``. It must not be deallocated. If *obj* is not a bytes object at all, :c:func:`PyBytes_AsStringAndSize` returns ``-1`` and raises :exc:`TypeError`. .. versionchanged:: 3.5 Previously, :exc:`TypeError` was raised when embedded null bytes were encountered in the bytes object. .. c:function:: void PyBytes_Concat(PyObject **bytes, PyObject *newpart) Create a new bytes object in *\*bytes* containing the contents of *newpart* appended to *bytes*; the caller will own the new reference. The reference to the old value of *bytes* will be ":term:`stolen `". If the new object cannot be created, the old reference to *bytes* will still be "stolen", the value of *\*bytes* will be set to ``NULL``, and the appropriate exception will be set. .. note:: If *newpart* implements the buffer protocol, then the buffer must not be mutated while the new bytes object is being created. .. c:function:: void PyBytes_ConcatAndDel(PyObject **bytes, PyObject *newpart) Create a new bytes object in *\*bytes* containing the contents of *newpart* appended to *bytes*. This version releases the :term:`strong reference` to *newpart* (i.e. decrements its reference count). .. note:: If *newpart* implements the buffer protocol, then the buffer must not be mutated while the new bytes object is being created. .. c:function:: PyObject* PyBytes_Join(PyObject *sep, PyObject *iterable) Similar to ``sep.join(iterable)`` in Python. *sep* must be Python :class:`bytes` object. (Note that :c:func:`PyUnicode_Join` accepts ``NULL`` separator and treats it as a space, whereas :c:func:`PyBytes_Join` doesn't accept ``NULL`` separator.) *iterable* must be an iterable object yielding objects that implement the :ref:`buffer protocol `. On success, return a new :class:`bytes` object. On error, set an exception and return ``NULL``. .. versionadded:: 3.14 .. note:: If *iterable* objects implement the buffer protocol, then the buffers must not be mutated while the new bytes object is being created. .. c:function:: int _PyBytes_Resize(PyObject **bytes, Py_ssize_t newsize) Resize a bytes object. *newsize* will be the new length of the bytes object. You can think of it as creating a new bytes object and destroying the old one, only more efficiently. Pass the address of an existing bytes object as an lvalue (it may be written into), and the new size desired. On success, *\*bytes* holds the resized bytes object and ``0`` is returned; the address in *\*bytes* may differ from its input value. If the reallocation fails, the original bytes object at *\*bytes* is deallocated, *\*bytes* is set to ``NULL``, :exc:`MemoryError` is set, and ``-1`` is returned. While bytes objects are usually immutable in Python, this special C API allows mutating a bytes object in-place. The returned bytes object can still be mutated using :c:func:`PyBytes_AsString`; except if *newsize* is zero in which case it returns the immutable empty bytes string. .. soft-deprecated:: 3.15 Use the :c:type:`PyBytesWriter` API instead. .. c:function:: PyObject *PyBytes_Repr(PyObject *bytes, int smartquotes) Get the string representation of *bytes*. This function is currently used to implement :meth:`!bytes.__repr__` in Python. This function does not do type checking; it is undefined behavior to pass *bytes* as a non-bytes object or ``NULL``. If *smartquotes* is true, the representation will use a double-quoted string instead of single-quoted string when single-quotes are present in *bytes*. For example, the byte string ``'Python'`` would be represented as ``b"'Python'"`` when *smartquotes* is true, or ``b'\'Python\''`` when it is false. On success, this function returns a :term:`strong reference` to a :class:`str` object containing the representation. On failure, this returns ``NULL`` with an exception set. .. c:function:: PyObject *PyBytes_DecodeEscape(const char *s, Py_ssize_t len, const char *errors, Py_ssize_t unicode, const char *recode_encoding) Unescape a backslash-escaped string *s*. *s* must not be ``NULL``. *len* must be the size of *s*. *errors* must be one of ``"strict"``, ``"replace"``, or ``"ignore"``. If *errors* is ``NULL``, then ``"strict"`` is used by default. On success, this function returns a :term:`strong reference` to a Python :class:`bytes` object containing the unescaped string. On failure, this function returns ``NULL`` with an exception set. .. versionchanged:: 3.9 *unicode* and *recode_encoding* are now unused. .. _pybyteswriter: PyBytesWriter ------------- The :c:type:`PyBytesWriter` API can be used to create a Python :class:`bytes` object. .. versionadded:: 3.15 .. c:type:: PyBytesWriter A bytes writer object. The API is **not thread safe**. To share a writer with multiple threads, a critical section or a lock is needed. The instance must be destroyed by :c:func:`PyBytesWriter_Finish` on success, or :c:func:`PyBytesWriter_Discard` on error. Create, Finish, Discard ^^^^^^^^^^^^^^^^^^^^^^^ .. c:function:: PyBytesWriter* PyBytesWriter_Create(Py_ssize_t size) Create a :c:type:`PyBytesWriter` to write *size* bytes. If *size* is greater than zero, allocate *size* bytes, and set the writer size to *size*. The caller is responsible to write *size* bytes using :c:func:`PyBytesWriter_GetData`. This function does not overallocate. On error, set an exception and return ``NULL``. *size* must be positive or zero. .. c:function:: PyObject* PyBytesWriter_Finish(PyBytesWriter *writer) Finish a :c:type:`PyBytesWriter` created by :c:func:`PyBytesWriter_Create`. On success, return a Python :class:`bytes` object. On error, set an exception and return ``NULL``. The writer instance is invalid after the call in any case. No API can be called on the writer after :c:func:`PyBytesWriter_Finish`. .. c:function:: PyObject* PyBytesWriter_FinishWithSize(PyBytesWriter *writer, Py_ssize_t size) Similar to :c:func:`PyBytesWriter_Finish`, but resize the writer to *size* bytes before creating the :class:`bytes` object. .. c:function:: PyObject* PyBytesWriter_FinishWithPointer(PyBytesWriter *writer, void *buf) Similar to :c:func:`PyBytesWriter_Finish`, but resize the writer using *buf* pointer before creating the :class:`bytes` object. Set an exception and return ``NULL`` if *buf* pointer is outside the internal buffer bounds. Function pseudo-code:: Py_ssize_t size = (char*)buf - (char*)PyBytesWriter_GetData(writer); return PyBytesWriter_FinishWithSize(writer, size); .. c:function:: void PyBytesWriter_Discard(PyBytesWriter *writer) Discard a :c:type:`PyBytesWriter` created by :c:func:`PyBytesWriter_Create`. Do nothing if *writer* is ``NULL``. The writer instance is invalid after the call. No API can be called on the writer after :c:func:`PyBytesWriter_Discard`. High-level API ^^^^^^^^^^^^^^ .. c:function:: int PyBytesWriter_WriteBytes(PyBytesWriter *writer, const void *bytes, Py_ssize_t size) Grow the *writer* internal buffer by *size* bytes, write *size* bytes of *bytes* at the *writer* end, and add *size* to the *writer* size. If *size* is equal to ``-1``, call ``strlen(bytes)`` to get the string length. On success, return ``0``. On error, set an exception and return ``-1``. .. c:function:: int PyBytesWriter_Format(PyBytesWriter *writer, const char *format, ...) Similar to :c:func:`PyBytes_FromFormat`, but write the output directly at the writer end. Grow the writer internal buffer on demand. Then add the written size to the writer size. On success, return ``0``. On error, set an exception and return ``-1``. Getters ^^^^^^^ .. c:function:: Py_ssize_t PyBytesWriter_GetSize(PyBytesWriter *writer) Get the writer size. The function does not invalidate pointers returned by :c:func:`PyBytesWriter_GetData`. The function cannot fail. .. c:function:: void* PyBytesWriter_GetData(PyBytesWriter *writer) Get the writer data: start of the internal buffer. The pointer remains valid until a :c:type:`PyBytesWriter` function other than :c:func:`PyBytesWriter_GetData` or :c:func:`PyBytesWriter_GetSize` is called on *writer*. The function cannot fail. Low-level API ^^^^^^^^^^^^^ .. c:function:: int PyBytesWriter_Resize(PyBytesWriter *writer, Py_ssize_t size) Resize the writer to *size* bytes. It can be used to enlarge or to shrink the writer. This function typically overallocates to achieve amortized performance when resizing multiple times. Newly allocated bytes are left uninitialized. On success, return ``0``. On error, set an exception and return ``-1``. *size* must be positive or zero. .. c:function:: int PyBytesWriter_Grow(PyBytesWriter *writer, Py_ssize_t grow) Resize the writer by adding *grow* bytes to the current writer size. This function typically overallocates to achieve amortized performance when resizing multiple times. Newly allocated bytes are left uninitialized. On success, return ``0``. On error, set an exception and return ``-1``. *grow* can be negative to shrink the writer. .. c:function:: void* PyBytesWriter_GrowAndUpdatePointer(PyBytesWriter *writer, Py_ssize_t size, void *buf) Similar to :c:func:`PyBytesWriter_Grow`, but update also the *buf* pointer. The *buf* pointer is moved if the internal buffer is moved in memory. The *buf* relative position within the internal buffer is left unchanged. On error, set an exception and return ``NULL``. *buf* must not be ``NULL``. Function pseudo-code:: Py_ssize_t pos = (char*)buf - (char*)PyBytesWriter_GetData(writer); if (PyBytesWriter_Grow(writer, size) < 0) { return NULL; } return (char*)PyBytesWriter_GetData(writer) + pos; Example creating the string ``b'abc'`` ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ Example creating the bytes string ``b"abc"``, with a fixed size of 3 bytes:: PyObject* create_abc(void) { PyBytesWriter *writer = PyBytesWriter_Create(3); if (writer == NULL) { return NULL; } char *str = PyBytesWriter_GetData(writer); memcpy(str, "abc", 3); return PyBytesWriter_Finish(writer); } Variant using a pointer to compute the final size:: PyObject* create_abc_ptr(void) { PyBytesWriter *writer = PyBytesWriter_Create(3); if (writer == NULL) { return NULL; } char *str = PyBytesWriter_GetData(writer); memcpy(str, "abc", 3); str += 1; return PyBytesWriter_FinishWithPointer(writer, str); } Update ``PyBytes_FromStringAndSize()`` code ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ Example of code using the soft deprecated ``PyBytes_FromStringAndSize(NULL, size)`` API:: int copy_bytes(char *dest, const char *src, Py_ssize_t num_bytes) { ... } PyObject* create_string(const char *start, Py_ssize_t num_bytes) { PyObject *result = PyBytes_FromStringAndSize(NULL, num_bytes); if (result == NULL) { return NULL; } if (copy_bytes(PyBytes_AS_STRING(result), start, num_bytes) < 0) { Py_CLEAR(result); } return result; } It can now be updated to:: PyObject* create_string(const char *start, Py_ssize_t num_bytes) { PyBytesWriter *writer = PyBytesWriter_Create(num_bytes); if (writer == NULL) { return NULL; } if (copy_bytes(PyBytesWriter_GetData(writer), start, num_bytes) < 0) { PyBytesWriter_Discard(writer); return NULL; } return PyBytesWriter_Finish(writer); } Update ``_PyBytes_Resize()`` code ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ Example of code using the soft deprecated ``_PyBytes_Resize()`` API:: void create_string(Py_ssize_t size) { PyObject *bytes = PyBytes_FromStringAndSize(NULL, size); if (bytes == NULL) { return NULL; } char *str = PyBytes_AS_STRING(bytes); // ... write data into str ... if (_PyBytes_Resize(&bytes, (str - PyBytes_AS_STRING(bytes)))) { return NULL; } return bytes; } It can now be updated to:: void create_string(Py_ssize_t size) { PyBytesWriter *writer = PyBytesWriter_Create(size); if (writer == NULL) { return NULL; } char *str = PyBytesWriter_GetData(writer); // ... write data into str ... return PyBytesWriter_FinishWithPointer(writer, str); } --- ### Doc C Api Call (`Doc/c-api/call.rst`) .. highlight:: c .. _call: Call Protocol ============= CPython supports two different calling protocols: *tp_call* and vectorcall. The *tp_call* Protocol ---------------------- Instances of classes that set :c:member:`~PyTypeObject.tp_call` are callable. The signature of the slot is:: PyObject *tp_call(PyObject *callable, PyObject *args, PyObject *kwargs); A call is made using a tuple for the positional arguments and a dict for the keyword arguments, similarly to ``callable(*args, **kwargs)`` in Python code. *args* must be non-NULL (use an empty tuple if there are no arguments) but *kwargs* may be *NULL* if there are no keyword arguments. This convention is not only used by *tp_call*: :c:member:`~PyTypeObject.tp_new` and :c:member:`~PyTypeObject.tp_init` also pass arguments this way. To call an object, use :c:func:`PyObject_Call` or another :ref:`call API `. .. _vectorcall: The Vectorcall Protocol ----------------------- .. versionadded:: 3.9 The vectorcall protocol was introduced in :pep:`590` as an additional protocol for making calls more efficient. As rule of thumb, CPython will prefer the vectorcall for internal calls if the callable supports it. However, this is not a hard rule. Additionally, some third-party extensions use *tp_call* directly (rather than using :c:func:`PyObject_Call`). Therefore, a class supporting vectorcall must also implement :c:member:`~PyTypeObject.tp_call`. Moreover, the callable must behave the same regardless of which protocol is used. The recommended way to achieve this is by setting :c:member:`~PyTypeObject.tp_call` to :c:func:`PyVectorcall_Call`. This bears repeating: .. warning:: A class supporting vectorcall **must** also implement :c:member:`~PyTypeObject.tp_call` with the same semantics. .. versionchanged:: 3.12 The :c:macro:`Py_TPFLAGS_HAVE_VECTORCALL` flag is now removed from a class when the class's :py:meth:`~object.__call__` method is reassigned. (This internally sets :c:member:`~PyTypeObject.tp_call` only, and thus may make it behave differently than the vectorcall function.) In earlier Python versions, vectorcall should only be used with :c:macro:`immutable ` or static types. A class should not implement vectorcall if that would be slower than *tp_call*. For example, if the callee needs to convert the arguments to an args tuple and kwargs dict anyway, then there is no point in implementing vectorcall. Classes can implement the vectorcall protocol by enabling the :c:macro:`Py_TPFLAGS_HAVE_VECTORCALL` flag and setting :c:member:`~PyTypeObject.tp_vectorcall_offset` to the offset inside the object structure where a *vectorcallfunc* appears. This is a pointer to a function with the following signature: .. c:type:: PyObject *(*vectorcallfunc)(PyObject *callable, PyObject *const *args, size_t nargsf, PyObject *kwnames) - *callable* is the object being called. - *args* is a C array consisting of the positional arguments followed by the values of the keyword arguments. This can be *NULL* if there are no arguments. - *nargsf* is the number of positional arguments plus possibly the :c:macro:`PY_VECTORCALL_ARGUMENTS_OFFSET` flag. To get the actual number of positional arguments from *nargsf*, use :c:func:`PyVectorcall_NARGS`. - *kwnames* is a tuple containing the names of the keyword arguments; in other words, the keys of the kwargs dict. These names must be strings (instances of ``str`` or a subclass) and they must be unique. If there are no keyword arguments, then *kwnames* can instead be *NULL*. .. c:macro:: PY_VECTORCALL_ARGUMENTS_OFFSET If this flag is set in a vectorcall *nargsf* argument, the callee is allowed to temporarily change ``args[-1]``. In other words, *args* points to argument 1 (not 0) in the allocated vector. The callee must restore the value of ``args[-1]`` before returning. For :c:func:`PyObject_VectorcallMethod`, this flag means instead that ``args[0]`` may be changed. Whenever they can do so cheaply (without additional allocation), callers are encouraged to use :c:macro:`PY_VECTORCALL_ARGUMENTS_OFFSET`. Doing so will allow callables such as bound methods to make their onward calls (which include a prepended *self* argument) very efficiently. .. versionadded:: 3.8 To call an object that implements vectorcall, use a :ref:`call API ` function as with any other callable. :c:func:`PyObject_Vectorcall` will usually be most efficient. Recursion Control ................. When using *tp_call*, callees do not need to worry about :ref:`recursion `: CPython uses :c:func:`Py_EnterRecursiveCall` and :c:func:`Py_LeaveRecursiveCall` for calls made using *tp_call*. For efficiency, this is not the case for calls done using vectorcall: the callee should use *Py_EnterRecursiveCall* and *Py_LeaveRecursiveCall* if needed. Vectorcall Support API ...................... .. c:function:: Py_ssize_t PyVectorcall_NARGS(size_t nargsf) Given a vectorcall *nargsf* argument, return the actual number of arguments. Currently equivalent to:: (Py_ssize_t)(nargsf & ~PY_VECTORCALL_ARGUMENTS_OFFSET) However, the function ``PyVectorcall_NARGS`` should be used to allow for future extensions. .. versionadded:: 3.8 .. c:function:: vectorcallfunc PyVectorcall_Function(PyObject *op) If *op* does not support the vectorcall protocol (either because the type does not or because the specific instance does not), return *NULL*. Otherwise, return the vectorcall function pointer stored in *op*. This function never raises an exception. This is mostly useful to check whether or not *op* supports vectorcall, which can be done by checking ``PyVectorcall_Function(op) != NULL``. .. versionadded:: 3.9 .. c:function:: PyObject* PyVectorcall_Call(PyObject *callable, PyObject *tuple, PyObject *dict) Call *callable*'s :c:type:`vectorcallfunc` with positional and keyword arguments given in a tuple and dict, respectively. This is a specialized function, intended to be put in the :c:member:`~PyTypeObject.tp_call` slot or be used in an implementation of ``tp_call``. It does not check the :c:macro:`Py_TPFLAGS_HAVE_VECTORCALL` flag and it does not fall back to ``tp_call``. .. versionadded:: 3.8 .. _capi-call: Object Calling API ------------------ Various functions are available for calling a Python object. Each converts its arguments to a convention supported by the called object – either *tp_call* or vectorcall. In order to do as little conversion as possible, pick one that best fits the format of data you have available. The following table summarizes the available functions; please see individual documentation for details. +------------------------------------------+------------------+--------------------+---------------+ | Function | callable | args | kwargs | +==========================================+==================+====================+===============+ | :c:func:`PyObject_Call` | ``PyObject *`` | tuple | dict/``NULL`` | +------------------------------------------+------------------+--------------------+---------------+ | :c:func:`PyObject_CallNoArgs` | ``PyObject *`` | --- | --- | +------------------------------------------+------------------+--------------------+---------------+ | :c:func:`PyObject_CallOneArg` | ``PyObject *`` | 1 object | --- | +------------------------------------------+------------------+--------------------+---------------+ | :c:func:`PyObject_CallObject` | ``PyObject *`` | tuple/``NULL`` | --- | +------------------------------------------+------------------+--------------------+---------------+ | :c:func:`PyObject_CallFunction` | ``PyObject *`` | format | --- | +------------------------------------------+------------------+--------------------+---------------+ | :c:func:`PyObject_CallMethod` | obj + ``char*`` | format | --- | +------------------------------------------+------------------+--------------------+---------------+ | :c:func:`PyObject_CallFunctionObjArgs` | ``PyObject *`` | variadic | --- | +------------------------------------------+------------------+--------------------+---------------+ | :c:func:`PyObject_CallMethodObjArgs` | obj + name | variadic | --- | +------------------------------------------+------------------+--------------------+---------------+ | :c:func:`PyObject_CallMethodNoArgs` | obj + name | --- | --- | +------------------------------------------+------------------+--------------------+---------------+ | :c:func:`PyObject_CallMethodOneArg` | obj + name | 1 object | --- | +------------------------------------------+------------------+--------------------+---------------+ | :c:func:`PyObject_Vectorcall` | ``PyObject *`` | vectorcall | vectorcall | +------------------------------------------+------------------+--------------------+---------------+ | :c:func:`PyObject_VectorcallDict` | ``PyObject *`` | vectorcall | dict/``NULL`` | +------------------------------------------+------------------+--------------------+---------------+ | :c:func:`PyObject_VectorcallMethod` | arg + name | vectorcall | vectorcall | +------------------------------------------+------------------+--------------------+---------------+ .. c:function:: PyObject* PyObject_Call(PyObject *callable, PyObject *args, PyObject *kwargs) Call a callable Python object *callable*, with arguments given by the tuple *args*, and named arguments given by the dictionary *kwargs*. *args* must not be *NULL*; use an empty tuple if no arguments are needed. If no named arguments are needed, *kwargs* can be *NULL*. Return the result of the call on success, or raise an exception and return *NULL* on failure. This is the equivalent of the Python expression: ``callable(*args, **kwargs)``. .. c:function:: PyObject* PyObject_CallNoArgs(PyObject *callable) Call a callable Python object *callable* without any arguments. It is the most efficient way to call a callable Python object without any argument. Return the result of the call on success, or raise an exception and return *NULL* on failure. .. versionadded:: 3.9 .. c:function:: PyObject* PyObject_CallOneArg(PyObject *callable, PyObject *arg) Call a callable Python object *callable* with exactly 1 positional argument *arg* and no keyword arguments. Return the result of the call on success, or raise an exception and return *NULL* on failure. .. versionadded:: 3.9 .. c:function:: PyObject* PyObject_CallObject(PyObject *callable, PyObject *args) Call a callable Python object *callable*, with arguments given by the tuple *args*. If no arguments are needed, then *args* can be *NULL*. Return the result of the call on success, or raise an exception and return *NULL* on failure. This is the equivalent of the Python expression: ``callable(*args)``. .. c:function:: PyObject* PyObject_CallFunction(PyObject *callable, const char *format, ...) Call a callable Python object *callable*, with a variable number of C arguments. The C arguments are described using a :c:func:`Py_BuildValue` style format string. The format can be *NULL*, indicating that no arguments are provided. Return the result of the call on success, or raise an exception and return *NULL* on failure. This is the equivalent of the Python expression: ``callable(*args)``. Note that if you only pass :c:expr:`PyObject *` args, :c:func:`PyObject_CallFunctionObjArgs` is a faster alternative. .. versionchanged:: 3.4 The type of *format* was changed from ``char *``. .. c:function:: PyObject* PyObject_CallMethod(PyObject *obj, const char *name, const char *format, ...) Call the method named *name* of object *obj* with a variable number of C arguments. The C arguments are described by a :c:func:`Py_BuildValue` format string that should produce a tuple. The format can be *NULL*, indicating that no arguments are provided. Return the result of the call on success, or raise an exception and return *NULL* on failure. This is the equivalent of the Python expression: ``obj.name(arg1, arg2, ...)``. Note that if you only pass :c:expr:`PyObject *` args, :c:func:`PyObject_CallMethodObjArgs` is a faster alternative. .. versionchanged:: 3.4 The types of *name* and *format* were changed from ``char *``. .. c:function:: PyObject* PyObject_CallFunctionObjArgs(PyObject *callable, ...) Call a callable Python object *callable*, with a variable number of :c:expr:`PyObject *` arguments. The arguments are provided as a variable number of parameters followed by *NULL*. Return the result of the call on success, or raise an exception and return *NULL* on failure. This is the equivalent of the Python expression: ``callable(arg1, arg2, ...)``. .. c:function:: PyObject* PyObject_CallMethodObjArgs(PyObject *obj, PyObject *name, ...) Call a method of the Python object *obj*, where the name of the method is given as a Python string object in *name*. It is called with a variable number of :c:expr:`PyObject *` arguments. The arguments are provided as a variable number of parameters followed by *NULL*. Return the result of the call on success, or raise an exception and return *NULL* on failure. .. c:function:: PyObject* PyObject_CallMethodNoArgs(PyObject *obj, PyObject *name) Call a method of the Python object *obj* without arguments, where the name of the method is given as a Python string object in *name*. Return the result of the call on success, or raise an exception and return *NULL* on failure. .. versionadded:: 3.9 .. c:function:: PyObject* PyObject_CallMethodOneArg(PyObject *obj, PyObject *name, PyObject *arg) Call a method of the Python object *obj* with a single positional argument *arg*, where the name of the method is given as a Python string object in *name*. Return the result of the call on success, or raise an exception and return *NULL* on failure. .. versionadded:: 3.9 .. c:function:: PyObject* _PyObject_Vectorcall(PyObject *callable, PyObject *const *args, size_t nargsf, PyObject *kwnames) :no-typesetting: .. c:function:: PyObject* PyObject_Vectorcall(PyObject *callable, PyObject *const *args, size_t nargsf, PyObject *kwnames) Call a callable Python object *callable*. The arguments are the same as for :c:type:`vectorcallfunc`. If *callable* supports vectorcall_, this directly calls the vectorcall function stored in *callable*. Return the result of the call on success, or raise an exception and return *NULL* on failure. .. versionadded:: 3.8 as ``_PyObject_Vectorcall`` .. versionchanged:: 3.9 Renamed to the current name, without the leading underscore. The old provisional name is :term:`soft deprecated`. .. c:function:: PyObject* PyObject_VectorcallDict(PyObject *callable, PyObject *const *args, size_t nargsf, PyObject *kwdict) Call *callable* with positional arguments passed exactly as in the vectorcall_ protocol, but with keyword arguments passed as a dictionary *kwdict*. The *args* array contains only the positional arguments. Regardless of which protocol is used internally, a conversion of arguments needs to be done. Therefore, this function should only be used if the caller already has a dictionary ready to use for the keyword arguments, but not a tuple for the positional arguments. .. versionadded:: 3.9 .. c:function:: PyObject* PyObject_VectorcallMethod(PyObject *name, PyObject *const *args, size_t nargsf, PyObject *kwnames) Call a method using the vectorcall calling convention. The name of the method is given as a Python string *name*. The object whose method is called is *args[0]*, and the *args* array starting at *args[1]* represents the arguments of the call. There must be at least one positional argument. *nargsf* is the number of positional arguments including *args[0]*, plus :c:macro:`PY_VECTORCALL_ARGUMENTS_OFFSET` if the value of ``args[0]`` may temporarily be changed. Keyword arguments can be passed just like in :c:func:`PyObject_Vectorcall`. If the object has the :c:macro:`Py_TPFLAGS_METHOD_DESCRIPTOR` feature, this will call the unbound method object with the full *args* vector as arguments. Return the result of the call on success, or raise an exception and return *NULL* on failure. .. versionadded:: 3.9 Call Support API ---------------- .. c:function:: int PyCallable_Check(PyObject *o) Determine if the object *o* is callable. Return ``1`` if the object is callable and ``0`` otherwise. This function always succeeds. --- ### Doc C Api Capsule (`Doc/c-api/capsule.rst`) .. highlight:: c .. _capsules: Capsules -------- .. index:: pair: object; Capsule Refer to :ref:`using-capsules` for more information on using these objects. .. versionadded:: 3.1 .. c:type:: PyCapsule This subtype of :c:type:`PyObject` represents an opaque value, useful for C extension modules which need to pass an opaque value (as a :c:expr:`void*` pointer) through Python code to other C code. It is often used to make a C function pointer defined in one module available to other modules, so the regular import mechanism can be used to access C APIs defined in dynamically loaded modules. .. c:var:: PyTypeObject PyCapsule_Type The type object corresponding to capsule objects. This is the same object as :class:`types.CapsuleType` in the Python layer. .. c:type:: PyCapsule_Destructor The type of a destructor callback for a capsule. Defined as:: typedef void (*PyCapsule_Destructor)(PyObject *); See :c:func:`PyCapsule_New` for the semantics of PyCapsule_Destructor callbacks. .. c:function:: int PyCapsule_CheckExact(PyObject *p) Return true if its argument is a :c:type:`PyCapsule`. This function always succeeds. .. c:function:: PyObject* PyCapsule_New(void *pointer, const char *name, PyCapsule_Destructor destructor) Create a :c:type:`PyCapsule` encapsulating the *pointer*. The *pointer* argument may not be ``NULL``. On failure, set an exception and return ``NULL``. The *name* string may either be ``NULL`` or a pointer to a valid C string. If non-``NULL``, this string must outlive the capsule. (Though it is permitted to free it inside the *destructor*.) If the *destructor* argument is not ``NULL``, it will be called with the capsule as its argument when it is destroyed. If this capsule will be stored as an attribute of a module, the *name* should be specified as ``modulename.attributename``. This will enable other modules to import the capsule using :c:func:`PyCapsule_Import`. .. c:function:: void* PyCapsule_GetPointer(PyObject *capsule, const char *name) Retrieve the *pointer* stored in the capsule. On failure, set an exception and return ``NULL``. The *name* parameter must compare exactly to the name stored in the capsule. If the name stored in the capsule is ``NULL``, the *name* passed in must also be ``NULL``. Python uses the C function :c:func:`!strcmp` to compare capsule names. .. c:function:: PyCapsule_Destructor PyCapsule_GetDestructor(PyObject *capsule) Return the current destructor stored in the capsule. On failure, set an exception and return ``NULL``. It is legal for a capsule to have a ``NULL`` destructor. This makes a ``NULL`` return code somewhat ambiguous; use :c:func:`PyCapsule_IsValid` or :c:func:`PyErr_Occurred` to disambiguate. .. c:function:: void* PyCapsule_GetContext(PyObject *capsule) Return the current context stored in the capsule. On failure, set an exception and return ``NULL``. It is legal for a capsule to have a ``NULL`` context. This makes a ``NULL`` return code somewhat ambiguous; use :c:func:`PyCapsule_IsValid` or :c:func:`PyErr_Occurred` to disambiguate. .. c:function:: const char* PyCapsule_GetName(PyObject *capsule) Return the current name stored in the capsule. On failure, set an exception and return ``NULL``. It is legal for a capsule to have a ``NULL`` name. This makes a ``NULL`` return code somewhat ambiguous; use :c:func:`PyCapsule_IsValid` or :c:func:`PyErr_Occurred` to disambiguate. .. c:function:: void* PyCapsule_Import(const char *name, int no_block) Import a pointer to a C object from a capsule attribute in a module. The *name* parameter should specify the full name to the attribute, as in ``package.module.attribute``. Modules are imported if needed, other components are looked up as attributes. The *name* stored in the capsule must match this string exactly. Return the capsule's internal *pointer* on success. On failure, set an exception and return ``NULL``. .. versionchanged:: 3.3 *no_block* has no effect anymore. .. versionchanged:: 3.16 Submodules are now imported if needed. .. c:function:: int PyCapsule_IsValid(PyObject *capsule, const char *name) Determines whether or not *capsule* is a valid capsule. A valid capsule is non-``NULL``, passes :c:func:`PyCapsule_CheckExact`, has a non-``NULL`` pointer stored in it, and its internal name matches the *name* parameter. (See :c:func:`PyCapsule_GetPointer` for information on how capsule names are compared.) In other words, if :c:func:`PyCapsule_IsValid` returns a true value, calls to any of the accessors (any function starting with ``PyCapsule_Get``) are guaranteed to succeed. Return a nonzero value if the object is valid and matches the name passed in. Return ``0`` otherwise. This function will not fail. .. c:function:: int PyCapsule_SetContext(PyObject *capsule, void *context) Set the context pointer inside *capsule* to *context*. Return ``0`` on success. Return nonzero and set an exception on failure. .. c:function:: int PyCapsule_SetDestructor(PyObject *capsule, PyCapsule_Destructor destructor) Set the destructor inside *capsule* to *destructor*. Return ``0`` on success. Return nonzero and set an exception on failure. .. c:function:: int PyCapsule_SetName(PyObject *capsule, const char *name) Set the name inside *capsule* to *name*. If non-``NULL``, the name must outlive the capsule. If the previous *name* stored in the capsule was not ``NULL``, no attempt is made to free it. Return ``0`` on success. Return nonzero and set an exception on failure. .. c:function:: int PyCapsule_SetPointer(PyObject *capsule, void *pointer) Set the void pointer inside *capsule* to *pointer*. The pointer may not be ``NULL``. Return ``0`` on success. Return nonzero and set an exception on failure. --- ### Doc C Api Cell (`Doc/c-api/cell.rst`) .. highlight:: c .. _cell-objects: Cell Objects ------------ "Cell" objects are used to implement variables referenced by multiple scopes. For each such variable, a cell object is created to store the value; the local variables of each stack frame that references the value contain a reference to the cells from outer scopes which also use that variable. When the value is accessed, the value contained in the cell is used instead of the cell object itself. This de-referencing of the cell object requires support from the generated byte-code; these are not automatically de-referenced when accessed. Cell objects are not likely to be useful elsewhere. .. c:type:: PyCellObject The C structure used for cell objects. .. c:var:: PyTypeObject PyCell_Type The type object corresponding to cell objects. .. c:function:: int PyCell_Check(PyObject *ob) Return true if *ob* is a cell object; *ob* must not be ``NULL``. This function always succeeds. .. c:function:: PyObject* PyCell_New(PyObject *ob) Create and return a new cell object containing the value *ob*. The parameter may be ``NULL``. .. c:function:: PyObject* PyCell_Get(PyObject *cell) Return the contents of the cell *cell*, which can be ``NULL``. If *cell* is not a cell object, returns ``NULL`` with an exception set. .. c:function:: PyObject* PyCell_GET(PyObject *cell) Return the contents of the cell *cell*, but without checking that *cell* is non-``NULL`` and a cell object. .. c:function:: int PyCell_Set(PyObject *cell, PyObject *value) Set the contents of the cell object *cell* to *value*. This releases the reference to any current content of the cell. *value* may be ``NULL``. *cell* must be non-``NULL``. On success, return ``0``. If *cell* is not a cell object, set an exception and return ``-1``. .. c:function:: void PyCell_SET(PyObject *cell, PyObject *value) Sets the value of the cell object *cell* to *value*. No reference counts are adjusted, and no checks are made for safety; *cell* must be non-``NULL`` and must be a cell object. --- ### Doc C Api Code (`Doc/c-api/code.rst`) .. highlight:: c .. index:: object; code, code object .. _codeobjects: Code Objects ------------ Code objects are a low-level detail of the CPython implementation. Each one represents a chunk of executable code that hasn't yet been bound into a function. .. c:type:: PyCodeObject The C structure of the objects used to describe code objects. The fields of this type are subject to change at any time. .. c:var:: PyTypeObject PyCode_Type This is an instance of :c:type:`PyTypeObject` representing the Python :ref:`code object `. .. c:function:: int PyCode_Check(PyObject *co) Return true if *co* is a :ref:`code object `. This function always succeeds. .. c:function:: Py_ssize_t PyCode_GetNumFree(PyCodeObject *co) Return the number of :term:`free (closure) variables ` in a code object. .. c:function:: int PyUnstable_Code_GetFirstFree(PyCodeObject *co) Return the position of the first :term:`free (closure) variable ` in a code object. .. versionchanged:: 3.13 Renamed from ``PyCode_GetFirstFree`` as part of :ref:`unstable-c-api`. The old name is deprecated, but will remain available until the signature changes again. .. c:function:: PyCodeObject* PyUnstable_Code_New(int argcount, int kwonlyargcount, int nlocals, int stacksize, int flags, PyObject *code, PyObject *consts, PyObject *names, PyObject *varnames, PyObject *freevars, PyObject *cellvars, PyObject *filename, PyObject *name, PyObject *qualname, int firstlineno, PyObject *linetable, PyObject *exceptiontable) Return a new code object. If you need a dummy code object to create a frame, use :c:func:`PyCode_NewEmpty` instead. Since the definition of the bytecode changes often, calling :c:func:`PyUnstable_Code_New` directly can bind you to a precise Python version. The many arguments of this function are inter-dependent in complex ways, meaning that subtle changes to values are likely to result in incorrect execution or VM crashes. Use this function only with extreme care. .. versionchanged:: 3.11 Added ``qualname`` and ``exceptiontable`` parameters. .. index:: single: PyCode_New (C function) .. versionchanged:: 3.12 Renamed from ``PyCode_New`` as part of :ref:`unstable-c-api`. The old name is deprecated, but will remain available until the signature changes again. .. c:function:: PyCodeObject* PyCode_NewWithPosOnlyArgs(...) :no-typesetting: .. c:function:: PyCodeObject* PyUnstable_Code_NewWithPosOnlyArgs(int argcount, int posonlyargcount, int kwonlyargcount, int nlocals, int stacksize, int flags, PyObject *code, PyObject *consts, PyObject *names, PyObject *varnames, PyObject *freevars, PyObject *cellvars, PyObject *filename, PyObject *name, PyObject *qualname, int firstlineno, PyObject *linetable, PyObject *exceptiontable) Similar to :c:func:`PyUnstable_Code_New`, but with an extra "posonlyargcount" for positional-only arguments. The same caveats that apply to ``PyUnstable_Code_New`` also apply to this function. .. versionadded:: 3.8 as ``PyCode_NewWithPosOnlyArgs`` .. versionchanged:: 3.11 Added ``qualname`` and ``exceptiontable`` parameters. .. versionchanged:: 3.12 Renamed to ``PyUnstable_Code_NewWithPosOnlyArgs``. The old name is deprecated, but will remain available until the signature changes again. .. c:function:: PyCodeObject* PyCode_NewEmpty(const char *filename, const char *funcname, int firstlineno) Return a new empty code object with the specified filename, function name, and first line number. The resulting code object will raise an ``Exception`` if executed. .. c:function:: int PyCode_Addr2Line(PyCodeObject *co, int byte_offset) Return the line number of the instruction that occurs on or before ``byte_offset`` and ends after it. If you just need the line number of a frame, use :c:func:`PyFrame_GetLineNumber` instead. For efficiently iterating over the line numbers in a code object, use :pep:`the API described in PEP 626 <0626#out-of-process-debuggers-and-profilers>`. .. c:function:: int PyCode_Addr2Location(PyObject *co, int byte_offset, int *start_line, int *start_column, int *end_line, int *end_column) Sets the passed ``int`` pointers to the source code line and column numbers for the instruction at ``byte_offset``. Sets the value to ``0`` when information is not available for any particular element. Returns ``1`` if the function succeeds and 0 otherwise. .. versionadded:: 3.11 .. c:function:: PyObject* PyCode_GetCode(PyCodeObject *co) Equivalent to the Python code ``getattr(co, 'co_code')``. Returns a strong reference to a :c:type:`PyBytesObject` representing the bytecode in a code object. On error, ``NULL`` is returned and an exception is raised. This ``PyBytesObject`` may be created on-demand by the interpreter and does not necessarily represent the bytecode actually executed by CPython. The primary use case for this function is debuggers and profilers. .. versionadded:: 3.11 .. c:function:: PyObject* PyCode_GetVarnames(PyCodeObject *co) Equivalent to the Python code ``getattr(co, 'co_varnames')``. Returns a new reference to a :c:type:`PyTupleObject` containing the names of the local variables. On error, ``NULL`` is returned and an exception is raised. .. versionadded:: 3.11 .. c:function:: PyObject* PyCode_GetCellvars(PyCodeObject *co) Equivalent to the Python code ``getattr(co, 'co_cellvars')``. Returns a new reference to a :c:type:`PyTupleObject` containing the names of the local variables that are referenced by nested functions. On error, ``NULL`` is returned and an exception is raised. .. versionadded:: 3.11 .. c:function:: PyObject* PyCode_GetFreevars(PyCodeObject *co) Equivalent to the Python code ``getattr(co, 'co_freevars')``. Returns a new reference to a :c:type:`PyTupleObject` containing the names of the :term:`free (closure) variables `. On error, ``NULL`` is returned and an exception is raised. .. versionadded:: 3.11 .. c:function:: int PyCode_AddWatcher(PyCode_WatchCallback callback) Register *callback* as a code object watcher for the current interpreter. Return an ID which may be passed to :c:func:`PyCode_ClearWatcher`. In case of error (e.g. no more watcher IDs available), return ``-1`` and set an exception. .. versionadded:: 3.12 .. c:function:: int PyCode_ClearWatcher(int watcher_id) Clear watcher identified by *watcher_id* previously returned from :c:func:`PyCode_AddWatcher` for the current interpreter. Return ``0`` on success, or ``-1`` and set an exception on error (e.g. if the given *watcher_id* was never registered.) .. versionadded:: 3.12 .. c:type:: PyCodeEvent Enumeration of possible code object watcher events: - ``PY_CODE_EVENT_CREATE`` - ``PY_CODE_EVENT_DESTROY`` .. versionadded:: 3.12 .. c:type:: int (*PyCode_WatchCallback)(PyCodeEvent event, PyCodeObject* co) Type of a code object watcher callback function. If *event* is ``PY_CODE_EVENT_CREATE``, then the callback is invoked after *co* has been fully initialized. Otherwise, the callback is invoked before the destruction of *co* takes place, so the prior state of *co* can be inspected. If *event* is ``PY_CODE_EVENT_DESTROY``, taking a reference in the callback to the about-to-be-destroyed code object will resurrect it and prevent it from being freed at this time. When the resurrected object is destroyed later, any watcher callbacks active at that time will be called again. Users of this API should not rely on internal runtime implementation details. Such details may include, but are not limited to, the exact order and timing of creation and destruction of code objects. While changes in these details may result in differences observable by watchers (including whether a callback is invoked or not), it does not change the semantics of the Python code being executed. If the callback sets an exception, it must return ``-1``; this exception will be printed as an unraisable exception using :c:func:`PyErr_WriteUnraisable`. Otherwise it should return ``0``. There may already be a pending exception set on entry to the callback. In this case, the callback should return ``0`` with the same exception still set. This means the callback may not call any other API that can set an exception unless it saves and clears the exception state first, and restores it before returning. .. versionadded:: 3.12 .. c:function:: PyObject *PyCode_Optimize(PyObject *code, PyObject *consts, PyObject *names, PyObject *lnotab_obj) This is a function that does nothing. Prior to Python 3.10, this function would perform basic optimizations to a code object. .. versionchanged:: 3.10 This function now does nothing. .. soft-deprecated:: 3.13 .. _c_codeobject_flags: Code Object Flags ----------------- Code objects contain a bit-field of flags, which can be retrieved as the :attr:`~codeobject.co_flags` Python attribute (for example using :c:func:`PyObject_GetAttrString`), and set using a *flags* argument to :c:func:`PyUnstable_Code_New` and similar functions. Flags whose names start with ``CO_FUTURE_`` correspond to features normally selectable by :ref:`future statements `. These flags can be used in :c:member:`PyCompilerFlags.cf_flags`. Note that many ``CO_FUTURE_`` flags are mandatory in current versions of Python, and setting them has no effect. The following flags are available. For their meaning, see the linked documentation of their Python equivalents. .. list-table:: :widths: auto :header-rows: 1 * * Flag * Meaning * * .. c:macro:: CO_OPTIMIZED * :py:data:`inspect.CO_OPTIMIZED` * * .. c:macro:: CO_NEWLOCALS * :py:data:`inspect.CO_NEWLOCALS` * * .. c:macro:: CO_VARARGS * :py:data:`inspect.CO_VARARGS` * * .. c:macro:: CO_VARKEYWORDS * :py:data:`inspect.CO_VARKEYWORDS` * * .. c:macro:: CO_NESTED * :py:data:`inspect.CO_NESTED` * * .. c:macro:: CO_GENERATOR * :py:data:`inspect.CO_GENERATOR` * * .. c:macro:: CO_COROUTINE * :py:data:`inspect.CO_COROUTINE` * * .. c:macro:: CO_ITERABLE_COROUTINE * :py:data:`inspect.CO_ITERABLE_COROUTINE` * * .. c:macro:: CO_ASYNC_GENERATOR * :py:data:`inspect.CO_ASYNC_GENERATOR` * * .. c:macro:: CO_HAS_DOCSTRING * :py:data:`inspect.CO_HAS_DOCSTRING` * * .. c:macro:: CO_METHOD * :py:data:`inspect.CO_METHOD` * * .. c:macro:: CO_FUTURE_DIVISION * no effect (:py:data:`__future__.division`) * * .. c:macro:: CO_FUTURE_ABSOLUTE_IMPORT * no effect (:py:data:`__future__.absolute_import`) * * .. c:macro:: CO_FUTURE_WITH_STATEMENT * no effect (:py:data:`__future__.with_statement`) * * .. c:macro:: CO_FUTURE_PRINT_FUNCTION * no effect (:py:data:`__future__.print_function`) * * .. c:macro:: CO_FUTURE_UNICODE_LITERALS * no effect (:py:data:`__future__.unicode_literals`) * * .. c:macro:: CO_FUTURE_GENERATOR_STOP * no effect (:py:data:`__future__.generator_stop`) * * .. c:macro:: CO_FUTURE_ANNOTATIONS * :py:data:`__future__.annotations` Extra information ----------------- To support low-level extensions to frame evaluation, such as external just-in-time compilers, it is possible to attach arbitrary extra data to code objects. These functions are part of the unstable C API tier: this functionality is a CPython implementation detail, and the API may change without deprecation warnings. .. c:function:: Py_ssize_t _PyEval_RequestCodeExtraIndex(freefunc free) :no-typesetting: .. c:function:: Py_ssize_t PyUnstable_Eval_RequestCodeExtraIndex(freefunc free) Return a new opaque index value used to adding data to code objects. You generally call this function once (per interpreter) and use the result with ``PyCode_GetExtra`` and ``PyCode_SetExtra`` to manipulate data on individual code objects. If *free* is not ``NULL``: when a code object is deallocated, *free* will be called on non-``NULL`` data stored under the new index. Use :c:func:`Py_DecRef` when storing :c:type:`PyObject`. .. versionadded:: 3.6 as ``_PyEval_RequestCodeExtraIndex`` .. versionchanged:: 3.12 Renamed to ``PyUnstable_Eval_RequestCodeExtraIndex``. The old private name is deprecated, but will be available until the API changes. .. c:function:: int _PyCode_GetExtra(PyObject *code, Py_ssize_t index, void **extra) :no-typesetting: .. c:function:: int PyUnstable_Code_GetExtra(PyObject *code, Py_ssize_t index, void **extra) Set *extra* to the extra data stored under the given index. Return 0 on success. Set an exception and return -1 on failure. If no data was set under the index, set *extra* to ``NULL`` and return 0 without setting an exception. .. versionadded:: 3.6 as ``_PyCode_GetExtra`` .. versionchanged:: 3.12 Renamed to ``PyUnstable_Code_GetExtra``. The old private name is deprecated, but will be available until the API changes. .. c:function:: int _PyCode_SetExtra(PyObject *code, Py_ssize_t index, void *extra) :no-typesetting: .. c:function:: int PyUnstable_Code_SetExtra(PyObject *code, Py_ssize_t index, void *extra) Set the extra data stored under the given index to *extra*. Return 0 on success. Set an exception and return -1 on failure. .. versionadded:: 3.6 as ``_PyCode_SetExtra`` .. versionchanged:: 3.12 Renamed to ``PyUnstable_Code_SetExtra``. The old private name is deprecated, but will be available until the API changes. --- ### Doc C Api Codec (`Doc/c-api/codec.rst`) .. _codec-registry: Codec registry and support functions ==================================== .. c:function:: int PyCodec_Register(PyObject *search_function) Register a new codec search function. As a side effect, this tries to load the :mod:`!encodings` package, if not yet done, to make sure that it is always first in the list of search functions. .. c:function:: int PyCodec_Unregister(PyObject *search_function) Unregister a codec search function and clear the registry's cache. If the search function is not registered, do nothing. Return 0 on success. Raise an exception and return -1 on error. .. versionadded:: 3.10 .. c:function:: int PyCodec_KnownEncoding(const char *encoding) Return ``1`` or ``0`` depending on whether there is a registered codec for the given *encoding*. This function always succeeds. .. c:function:: PyObject* PyCodec_Encode(PyObject *object, const char *encoding, const char *errors) Generic codec based encoding API. *object* is passed through the encoder function found for the given *encoding* using the error handling method defined by *errors*. *errors* may be ``NULL`` to use the default method defined for the codec. Raises a :exc:`LookupError` if no encoder can be found. .. c:function:: PyObject* PyCodec_Decode(PyObject *object, const char *encoding, const char *errors) Generic codec based decoding API. *object* is passed through the decoder function found for the given *encoding* using the error handling method defined by *errors*. *errors* may be ``NULL`` to use the default method defined for the codec. Raises a :exc:`LookupError` if no decoder can be found. Codec lookup API ---------------- In the following functions, the *encoding* string is looked up converted to all lower-case characters, which makes encodings looked up through this mechanism effectively case-insensitive. If no codec is found, a :exc:`KeyError` is set and ``NULL`` returned. .. c:function:: PyObject* PyCodec_Encoder(const char *encoding) Get an encoder function for the given *encoding*. .. c:function:: PyObject* PyCodec_Decoder(const char *encoding) Get a decoder function for the given *encoding*. .. c:function:: PyObject* PyCodec_IncrementalEncoder(const char *encoding, const char *errors) Get an :class:`~codecs.IncrementalEncoder` object for the given *encoding*. .. c:function:: PyObject* PyCodec_IncrementalDecoder(const char *encoding, const char *errors) Get an :class:`~codecs.IncrementalDecoder` object for the given *encoding*. .. c:function:: PyObject* PyCodec_StreamReader(const char *encoding, PyObject *stream, const char *errors) Get a :class:`~codecs.StreamReader` factory function for the given *encoding*. .. c:function:: PyObject* PyCodec_StreamWriter(const char *encoding, PyObject *stream, const char *errors) Get a :class:`~codecs.StreamWriter` factory function for the given *encoding*. Registry API for Unicode encoding error handlers ------------------------------------------------ .. c:function:: int PyCodec_RegisterError(const char *name, PyObject *error) Register the error handling callback function *error* under the given *name*. This callback function will be called by a codec when it encounters unencodable characters/undecodable bytes and *name* is specified as the error parameter in the call to the encode/decode function. The callback gets a single argument, an instance of :exc:`UnicodeEncodeError`, :exc:`UnicodeDecodeError` or :exc:`UnicodeTranslateError` that holds information about the problematic sequence of characters or bytes and their offset in the original string (see :ref:`unicodeexceptions` for functions to extract this information). The callback must either raise the given exception, or return a two-item tuple containing the replacement for the problematic sequence, and an integer giving the offset in the original string at which encoding/decoding should be resumed. Return ``0`` on success, ``-1`` on error. .. c:function:: PyObject* PyCodec_LookupError(const char *name) Lookup the error handling callback function registered under *name*. As a special case ``NULL`` can be passed, in which case the error handling callback for "strict" will be returned. .. c:function:: PyObject* PyCodec_StrictErrors(PyObject *exc) Raise *exc* as an exception. .. c:function:: PyObject* PyCodec_IgnoreErrors(PyObject *exc) Ignore the unicode error, skipping the faulty input. .. c:function:: PyObject* PyCodec_ReplaceErrors(PyObject *exc) Replace the unicode encode error with ``?`` or ``U+FFFD``. .. c:function:: PyObject* PyCodec_XMLCharRefReplaceErrors(PyObject *exc) Replace the unicode encode error with XML character references. .. c:function:: PyObject* PyCodec_BackslashReplaceErrors(PyObject *exc) Replace the unicode encode error with backslash escapes (``\x``, ``\u`` and ``\U``). .. c:function:: PyObject* PyCodec_NameReplaceErrors(PyObject *exc) Replace the unicode encode error with ``\N{...}`` escapes. .. versionadded:: 3.5 Codec utility variables ----------------------- .. c:var:: const char *Py_hexdigits A string constant containing the lowercase hexadecimal digits: ``"0123456789abcdef"``. .. versionadded:: 3.3 --- ### Doc C Api Complex (`Doc/c-api/complex.rst`) .. highlight:: c .. _complexobjects: Complex Number Objects ====================== .. index:: pair: object; complex number .. c:type:: PyComplexObject This subtype of :c:type:`PyObject` represents a Python complex number object. .. c:member:: Py_complex cval The complex number value, using the C :c:type:`Py_complex` representation. .. deprecated-removed:: 3.15 3.20 Use :c:func:`PyComplex_AsCComplex` and :c:func:`PyComplex_FromCComplex` to convert a Python complex number to/from the C :c:type:`Py_complex` representation. .. c:var:: PyTypeObject PyComplex_Type This instance of :c:type:`PyTypeObject` represents the Python complex number type. It is the same object as :class:`complex` in the Python layer. .. c:function:: int PyComplex_Check(PyObject *p) Return true if its argument is a :c:type:`PyComplexObject` or a subtype of :c:type:`PyComplexObject`. This function always succeeds. .. c:function:: int PyComplex_CheckExact(PyObject *p) Return true if its argument is a :c:type:`PyComplexObject`, but not a subtype of :c:type:`PyComplexObject`. This function always succeeds. .. c:function:: PyObject* PyComplex_FromDoubles(double real, double imag) Return a new :c:type:`PyComplexObject` object from *real* and *imag*. Return ``NULL`` with an exception set on error. .. c:function:: double PyComplex_RealAsDouble(PyObject *op) Return the real part of *op* as a C :c:expr:`double`. If *op* is not a Python complex number object but has a :meth:`~object.__complex__` method, this method will first be called to convert *op* to a Python complex number object. If :meth:`!__complex__` is not defined then it falls back to call :c:func:`PyFloat_AsDouble` and returns its result. Upon failure, this method returns ``-1.0`` with an exception set, so one should call :c:func:`PyErr_Occurred` to check for errors. .. versionchanged:: 3.13 Use :meth:`~object.__complex__` if available. .. c:function:: double PyComplex_ImagAsDouble(PyObject *op) Return the imaginary part of *op* as a C :c:expr:`double`. If *op* is not a Python complex number object but has a :meth:`~object.__complex__` method, this method will first be called to convert *op* to a Python complex number object. If :meth:`!__complex__` is not defined then it falls back to call :c:func:`PyFloat_AsDouble` and returns ``0.0`` on success. Upon failure, this method returns ``-1.0`` with an exception set, so one should call :c:func:`PyErr_Occurred` to check for errors. .. versionchanged:: 3.13 Use :meth:`~object.__complex__` if available. .. c:type:: Py_complex This C structure defines an export format for a Python complex number object. .. c:member:: double real double imag The structure is defined as:: typedef struct { double real; double imag; } Py_complex; .. c:function:: PyObject* PyComplex_FromCComplex(Py_complex v) Create a new Python complex number object from a C :c:type:`Py_complex` value. Return ``NULL`` with an exception set on error. .. c:function:: Py_complex PyComplex_AsCComplex(PyObject *op) Return the :c:type:`Py_complex` value of the complex number *op*. If *op* is not a Python complex number object but has a :meth:`~object.__complex__` method, this method will first be called to convert *op* to a Python complex number object. If :meth:`!__complex__` is not defined then it falls back to :meth:`~object.__float__`. If :meth:`!__float__` is not defined then it falls back to :meth:`~object.__index__`. Upon failure, this method returns :c:type:`Py_complex` with :c:member:`~Py_complex.real` set to ``-1.0`` and with an exception set, so one should call :c:func:`PyErr_Occurred` to check for errors. .. versionchanged:: 3.8 Use :meth:`~object.__index__` if available. Complex Numbers as C Structures ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ The API also provides functions for working with complex numbers, using the :c:type:`Py_complex` representation. Note that the functions which accept these structures as parameters and return them as results do so *by value* rather than dereferencing them through pointers. Please note, that these functions are :term:`soft deprecated` since Python 3.15. Avoid using this API in a new code to do complex arithmetic: either use the :ref:`Number Protocol ` API or use native complex types, like :c:expr:`double complex`. .. c:function:: Py_complex _Py_c_sum(Py_complex left, Py_complex right) Return the sum of two complex numbers, using the C :c:type:`Py_complex` representation. .. deprecated:: 3.15 .. c:function:: Py_complex _Py_c_diff(Py_complex left, Py_complex right) Return the difference between two complex numbers, using the C :c:type:`Py_complex` representation. .. deprecated:: 3.15 .. c:function:: Py_complex _Py_c_neg(Py_complex num) Return the negation of the complex number *num*, using the C :c:type:`Py_complex` representation. .. deprecated:: 3.15 .. c:function:: Py_complex _Py_c_prod(Py_complex left, Py_complex right) Return the product of two complex numbers, using the C :c:type:`Py_complex` representation. .. deprecated:: 3.15 .. c:function:: Py_complex _Py_c_quot(Py_complex dividend, Py_complex divisor) Return the quotient of two complex numbers, using the C :c:type:`Py_complex` representation. If *divisor* is null, this method returns zero and sets :c:data:`errno` to :c:macro:`!EDOM`. .. deprecated:: 3.15 .. c:function:: Py_complex _Py_c_pow(Py_complex num, Py_complex exp) Return the exponentiation of *num* by *exp*, using the C :c:type:`Py_complex` representation. If *num* is null and *exp* is not a positive real number, this method returns zero and sets :c:data:`errno` to :c:macro:`!EDOM`. Set :c:data:`errno` to :c:macro:`!ERANGE` on overflows. .. deprecated:: 3.15 .. c:function:: double _Py_c_abs(Py_complex num) Return the absolute value of the complex number *num*. Set :c:data:`errno` to :c:macro:`!ERANGE` on overflows. .. deprecated:: 3.15 .. versionchanged:: 3.16 This function leaves :c:data:`errno` unchanged on success. --- ### Doc C Api Concrete (`Doc/c-api/concrete.rst`) .. highlight:: c .. _concrete: ********************** Concrete Objects Layer ********************** The functions in this chapter are specific to certain Python object types. Passing them an object of the wrong type is not a good idea; if you receive an object from a Python program and you are not sure that it has the right type, you must perform a type check first; for example, to check that an object is a dictionary, use :c:func:`PyDict_Check`. The chapter is structured like the "family tree" of Python object types. .. warning:: While the functions described in this chapter carefully check the type of the objects which are passed in, many of them do not check for ``NULL`` being passed instead of a valid object. Allowing ``NULL`` to be passed in can cause memory access violations and immediate termination of the interpreter. .. _fundamental: Fundamental Objects =================== This section describes Python type objects and the singleton object ``None``. .. toctree:: type.rst none.rst .. _numericobjects: Numeric Objects =============== .. index:: pair: object; numeric .. toctree:: long.rst bool.rst float.rst complex.rst .. _sequenceobjects: Sequence Objects ================ .. index:: pair: object; sequence Generic operations on sequence objects were discussed in the previous chapter; this section deals with the specific kinds of sequence objects that are intrinsic to the Python language. .. XXX sort out unicode, str, bytes and bytearray .. toctree:: bytes.rst bytearray.rst unicode.rst tuple.rst list.rst .. _mapobjects: Container Objects ================= .. index:: pair: object; mapping .. toctree:: dict.rst set.rst .. _otherobjects: Function Objects ================ .. toctree:: function.rst method.rst cell.rst code.rst Other Objects ============= .. toctree:: file.rst module.rst iterator.rst descriptor.rst slice.rst memoryview.rst picklebuffer.rst weakref.rst capsule.rst sentinel.rst frame.rst gen.rst coro.rst contextvars.rst typehints.rst C API for extension modules =========================== .. toctree:: curses.rst datetime.rst --- ### Doc C Api Contextvars (`Doc/c-api/contextvars.rst`) .. highlight:: c .. _contextvarsobjects: Context Variables Objects ------------------------- .. _contextvarsobjects_pointertype_change: .. versionadded:: 3.7 .. versionchanged:: 3.7.1 .. note:: In Python 3.7.1 the signatures of all context variables C APIs were **changed** to use :c:type:`PyObject` pointers instead of :c:type:`PyContext`, :c:type:`PyContextVar`, and :c:type:`PyContextToken`, e.g.:: // in 3.7.0: PyContext *PyContext_New(void); // in 3.7.1+: PyObject *PyContext_New(void); See :issue:`34762` for more details. This section details the public C API for the :mod:`contextvars` module. .. c:type:: PyContext The C structure used to represent a :class:`contextvars.Context` object. .. c:type:: PyContextVar The C structure used to represent a :class:`contextvars.ContextVar` object. .. c:type:: PyContextToken The C structure used to represent a :class:`contextvars.Token` object. .. c:var:: PyTypeObject PyContext_Type The type object representing the *context* type. .. c:var:: PyTypeObject PyContextVar_Type The type object representing the *context variable* type. .. c:var:: PyTypeObject PyContextToken_Type The type object representing the *context variable token* type. Type-check macros: .. c:function:: int PyContext_CheckExact(PyObject *o) Return true if *o* is of type :c:data:`PyContext_Type`. *o* must not be ``NULL``. This function always succeeds. .. c:function:: int PyContextVar_CheckExact(PyObject *o) Return true if *o* is of type :c:data:`PyContextVar_Type`. *o* must not be ``NULL``. This function always succeeds. .. c:function:: int PyContextToken_CheckExact(PyObject *o) Return true if *o* is of type :c:data:`PyContextToken_Type`. *o* must not be ``NULL``. This function always succeeds. Context object management functions: .. c:function:: PyObject *PyContext_New(void) Create a new empty context object. Returns ``NULL`` if an error has occurred. .. c:function:: PyObject *PyContext_Copy(PyObject *ctx) Create a shallow copy of the passed *ctx* context object. Returns ``NULL`` if an error has occurred. .. c:function:: PyObject *PyContext_CopyCurrent(void) Create a shallow copy of the current thread context. Returns ``NULL`` if an error has occurred. .. c:function:: int PyContext_Enter(PyObject *ctx) Set *ctx* as the current context for the current thread. Returns ``0`` on success, and ``-1`` on error. .. c:function:: int PyContext_Exit(PyObject *ctx) Deactivate the *ctx* context and restore the previous context as the current context for the current thread. Returns ``0`` on success, and ``-1`` on error. .. c:function:: int PyContext_AddWatcher(PyContext_WatchCallback callback) Register *callback* as a context object watcher for the current interpreter. Return an ID which may be passed to :c:func:`PyContext_ClearWatcher`. In case of error (e.g. no more watcher IDs available), return ``-1`` and set an exception. .. versionadded:: 3.14 .. c:function:: int PyContext_ClearWatcher(int watcher_id) Clear watcher identified by *watcher_id* previously returned from :c:func:`PyContext_AddWatcher` for the current interpreter. Return ``0`` on success, or ``-1`` and set an exception on error (e.g. if the given *watcher_id* was never registered.) .. versionadded:: 3.14 .. c:type:: PyContextEvent Enumeration of possible context object watcher events: - ``Py_CONTEXT_SWITCHED``: The :term:`current context` has switched to a different context. The object passed to the watch callback is the now-current :class:`contextvars.Context` object, or None if no context is current. .. versionadded:: 3.14 .. c:type:: int (*PyContext_WatchCallback)(PyContextEvent event, PyObject *obj) Context object watcher callback function. The object passed to the callback is event-specific; see :c:type:`PyContextEvent` for details. If the callback returns with an exception set, it must return ``-1``; this exception will be printed as an unraisable exception using :c:func:`PyErr_FormatUnraisable`. Otherwise it should return ``0``. There may already be a pending exception set on entry to the callback. In this case, the callback should return ``0`` with the same exception still set. This means the callback may not call any other API that can set an exception unless it saves and clears the exception state first, and restores it before returning. .. versionadded:: 3.14 Context variable functions: .. c:function:: PyObject *PyContextVar_New(const char *name, PyObject *def) Create a new ``ContextVar`` object. The *name* parameter is used for introspection and debug purposes. The *def* parameter specifies a default value for the context variable, or ``NULL`` for no default. If an error has occurred, this function returns ``NULL``. .. c:function:: int PyContextVar_Get(PyObject *var, PyObject *default_value, PyObject **value) Get the value of a context variable. Returns ``-1`` if an error has occurred during lookup, and ``0`` if no error occurred, whether or not a value was found. If the context variable was found, *value* will be a pointer to it. If the context variable was *not* found, *value* will point to: - *default_value*, if not ``NULL``; - the default value of *var*, if not ``NULL``; - ``NULL`` Except for ``NULL``, the function returns a new reference. .. c:function:: PyObject *PyContextVar_Set(PyObject *var, PyObject *value) Set the value of *var* to *value* in the current context. Returns a new token object for this change, or ``NULL`` if an error has occurred. .. c:function:: int PyContextVar_Reset(PyObject *var, PyObject *token) Reset the state of the *var* context variable to that it was in before :c:func:`PyContextVar_Set` that returned the *token* was called. This function returns ``0`` on success and ``-1`` on error. --- ### Doc C Api Conversion (`Doc/c-api/conversion.rst`) .. highlight:: c .. _string-conversion: String conversion and formatting ================================ Functions for number conversion and formatted string output. .. c:function:: int PyOS_snprintf(char *str, size_t size, const char *format, ...) Output not more than *size* bytes to *str* according to the format string *format* and the extra arguments. See the Unix man page :manpage:`snprintf(3)`. .. c:function:: int PyOS_vsnprintf(char *str, size_t size, const char *format, va_list va) Output not more than *size* bytes to *str* according to the format string *format* and the variable argument list *va*. Unix man page :manpage:`vsnprintf(3)`. :c:func:`PyOS_snprintf` and :c:func:`PyOS_vsnprintf` wrap the Standard C library functions :c:func:`snprintf` and :c:func:`vsnprintf`. Their purpose is to guarantee consistent behavior in corner cases, which the Standard C functions do not. The wrappers ensure that ``str[size-1]`` is always ``'\0'`` upon return. They never write more than *size* bytes (including the trailing ``'\0'``) into str. Both functions require that ``str != NULL``, ``size > 0``, ``format != NULL`` and ``size < INT_MAX``. Note that this means there is no equivalent to the C99 ``n = snprintf(NULL, 0, ...)`` which would determine the necessary buffer size. The return value (*rv*) for these functions should be interpreted as follows: * When ``0 <= rv < size``, the output conversion was successful and *rv* characters were written to *str* (excluding the trailing ``'\0'`` byte at ``str[rv]``). * When ``rv >= size``, the output conversion was truncated and a buffer with ``rv + 1`` bytes would have been needed to succeed. ``str[size-1]`` is ``'\0'`` in this case. * When ``rv < 0``, the output conversion failed and ``str[size-1]`` is ``'\0'`` in this case too, but the rest of *str* is undefined. The exact cause of the error depends on the underlying platform. The following functions provide locale-independent string to number conversions. .. c:function:: unsigned long PyOS_strtoul(const char *str, char **ptr, int base) Convert the initial part of the string in ``str`` to an :c:expr:`unsigned long` value according to the given ``base``, which must be between ``2`` and ``36`` inclusive, or be the special value ``0``. Leading white space and case of characters are ignored. If ``base`` is zero it looks for a leading ``0b``, ``0o`` or ``0x`` to tell which base. If these are absent it defaults to ``10``. Base must be 0 or between 2 and 36 (inclusive). If ``ptr`` is non-``NULL`` it will contain a pointer to the end of the scan. If the converted value falls out of range of corresponding return type, range error occurs (:c:data:`errno` is set to :c:macro:`!ERANGE`) and :c:macro:`!ULONG_MAX` is returned. If no conversion can be performed, ``0`` is returned. See also the Unix man page :manpage:`strtoul(3)`. .. versionadded:: 3.2 .. c:function:: long PyOS_strtol(const char *str, char **ptr, int base) Convert the initial part of the string in ``str`` to an :c:expr:`long` value according to the given ``base``, which must be between ``2`` and ``36`` inclusive, or be the special value ``0``. Same as :c:func:`PyOS_strtoul`, but return a :c:expr:`long` value instead and :c:macro:`LONG_MAX` on overflows. See also the Unix man page :manpage:`strtol(3)`. .. versionadded:: 3.2 .. c:function:: double PyOS_string_to_double(const char *s, char **endptr, PyObject *overflow_exception) Convert a string ``s`` to a :c:expr:`double`, raising a Python exception on failure. The set of accepted strings corresponds to the set of strings accepted by Python's :func:`float` constructor, except that ``s`` must not have leading or trailing whitespace. The conversion is independent of the current locale. If ``endptr`` is ``NULL``, convert the whole string. Raise :exc:`ValueError` and return ``-1.0`` if the string is not a valid representation of a floating-point number. If endptr is not ``NULL``, convert as much of the string as possible and set ``*endptr`` to point to the first unconverted character. If no initial segment of the string is the valid representation of a floating-point number, set ``*endptr`` to point to the beginning of the string, raise ValueError, and return ``-1.0``. If ``s`` represents a value that is too large to store in a float (for example, ``"1e500"`` is such a string on many platforms) then if ``overflow_exception`` is ``NULL`` return :c:macro:`!INFINITY` (with an appropriate sign) and don't set any exception. Otherwise, ``overflow_exception`` must point to a Python exception object; raise that exception and return ``-1.0``. In both cases, set ``*endptr`` to point to the first character after the converted value. If any other error occurs during the conversion (for example an out-of-memory error), set the appropriate Python exception and return ``-1.0``. .. versionadded:: 3.1 .. c:function:: char* PyOS_double_to_string(double val, char format_code, int precision, int flags, int *ptype) Convert a :c:expr:`double` *val* to a string using supplied *format_code*, *precision*, and *flags*. *format_code* must be one of ``'e'``, ``'E'``, ``'f'``, ``'F'``, ``'g'``, ``'G'`` or ``'r'``. For ``'r'``, the supplied *precision* must be 0 and is ignored. The ``'r'`` format code specifies the standard :func:`repr` format. *flags* can be zero or more of the following values or-ed together: .. c:namespace:: NULL .. c:macro:: Py_DTSF_SIGN Always precede the returned string with a sign character, even if *val* is non-negative. .. c:macro:: Py_DTSF_ADD_DOT_0 Ensure that the returned string will not look like an integer. .. c:macro:: Py_DTSF_ALT Apply "alternate" formatting rules. See the documentation for the :c:func:`PyOS_snprintf` ``'#'`` specifier for details. .. c:macro:: Py_DTSF_NO_NEG_0 Negative zero is converted to positive zero. .. versionadded:: 3.11 If *ptype* is non-``NULL``, then the value it points to will be set to one of the following constants depending on the type of *val*: .. list-table:: :header-rows: 1 :align: left * - *\*ptype* - type of *val* * - .. c:macro:: Py_DTST_FINITE - finite number * - .. c:macro:: Py_DTST_INFINITE - infinite number * - .. c:macro:: Py_DTST_NAN - not a number The return value is a pointer to *buffer* with the converted string or ``NULL`` if the conversion failed. The caller is responsible for freeing the returned string by calling :c:func:`PyMem_Free`. .. versionadded:: 3.1 .. c:function:: int PyOS_mystricmp(const char *str1, const char *str2) int PyOS_mystrnicmp(const char *str1, const char *str2, Py_ssize_t size) Case insensitive comparison of strings. These functions work almost identically to :c:func:`!strcmp` and :c:func:`!strncmp` (respectively), except that they ignore the case of ASCII characters. Return ``0`` if the strings are equal, a negative value if *str1* sorts lexicographically before *str2*, or a positive value if it sorts after. In the *str1* or *str2* arguments, a NUL byte marks the end of the string. For :c:func:`!PyOS_mystrnicmp`, the *size* argument gives the maximum size of the string, as if NUL was present at the index given by *size*. These functions do not use the locale. .. c:function:: int PyOS_stricmp(const char *str1, const char *str2) int PyOS_strnicmp(const char *str1, const char *str2, Py_ssize_t size) Case insensitive comparison of strings. On Windows, these are aliases of :c:func:`!stricmp` and :c:func:`!strnicmp`, respectively. On other platforms, they are aliases of :c:func:`PyOS_mystricmp` and :c:func:`PyOS_mystrnicmp`, respectively. Character classification and conversion ======================================= The following macros provide locale-independent (unlike the C standard library ``ctype.h``) character classification and conversion. The argument must be a signed or unsigned :c:expr:`char`. .. c:macro:: Py_ISALNUM(c) Return true if the character *c* is an alphanumeric character. .. c:macro:: Py_ISALPHA(c) Return true if the character *c* is an alphabetic character (``a-z`` and ``A-Z``). .. c:macro:: Py_ISDIGIT(c) Return true if the character *c* is a decimal digit (``0-9``). .. c:macro:: Py_ISLOWER(c) Return true if the character *c* is a lowercase ASCII letter (``a-z``). .. c:macro:: Py_ISUPPER(c) Return true if the character *c* is an uppercase ASCII letter (``A-Z``). .. c:macro:: Py_ISSPACE(c) Return true if the character *c* is a whitespace character (space, tab, carriage return, newline, vertical tab, or form feed). .. c:macro:: Py_ISXDIGIT(c) Return true if the character *c* is a hexadecimal digit (``0-9``, ``a-f``, and ``A-F``). .. c:macro:: Py_TOLOWER(c) Return the lowercase equivalent of the character *c*. .. c:macro:: Py_TOUPPER(c) Return the uppercase equivalent of the character *c*. --- ### Doc C Api Coro (`Doc/c-api/coro.rst`) .. highlight:: c .. _coro-objects: Coroutine Objects ----------------- .. versionadded:: 3.5 Coroutine objects are what functions declared with an ``async`` keyword return. .. c:type:: PyCoroObject The C structure used for coroutine objects. .. c:var:: PyTypeObject PyCoro_Type The type object corresponding to coroutine objects. .. c:function:: int PyCoro_CheckExact(PyObject *ob) Return true if *ob*'s type is :c:type:`PyCoro_Type`; *ob* must not be ``NULL``. This function always succeeds. .. c:function:: PyObject* PyCoro_New(PyFrameObject *frame, PyObject *name, PyObject *qualname) Create and return a new coroutine object based on the *frame* object, with ``__name__`` and ``__qualname__`` set to *name* and *qualname*. A reference to *frame* is stolen by this function. The *frame* argument must not be ``NULL``. .. deprecated-removed:: 3.16 3.18 This function has not been used since 3.10. It is also impossible to construct a proper *frame* object to call this function. --- ### Doc C Api Curses (`Doc/c-api/curses.rst`) .. highlight:: c Curses C API ------------ :mod:`curses` exposes a small C interface for extension modules. Consumers must include the header file :file:`py_curses.h` (which is not included by default by :file:`Python.h`) and :c:func:`import_curses` must be invoked, usually as part of the module initialisation function, to populate :c:var:`PyCurses_API`. .. warning:: Neither the C API nor the pure Python :mod:`curses` module are compatible with subinterpreters. .. c:macro:: import_curses() Import the curses C API. The macro does not need a semi-colon to be called. On success, populate the :c:var:`PyCurses_API` pointer. On failure, set :c:var:`PyCurses_API` to NULL and set an exception. The caller must check if an error occurred via :c:func:`PyErr_Occurred`: .. code-block:: import_curses(); // semi-colon is optional but recommended if (PyErr_Occurred()) { /* cleanup */ } .. c:var:: void **PyCurses_API Dynamically allocated object containing the curses C API. This variable is only available once :c:macro:`import_curses` succeeds. ``PyCurses_API[0]`` corresponds to :c:data:`PyCursesWindow_Type`. ``PyCurses_API[1]``, ``PyCurses_API[2]``, and ``PyCurses_API[3]`` are pointers to predicate functions of type ``int (*)(void)``. When called, these predicates return whether :func:`curses.setupterm`, :func:`curses.initscr`, and :func:`curses.start_color` have been called respectively. See also the convenience macros :c:macro:`PyCursesSetupTermCalled`, :c:macro:`PyCursesInitialised`, and :c:macro:`PyCursesInitialisedColor`. .. note:: The number of entries in this structure is subject to changes. Consider using :c:macro:`PyCurses_API_pointers` to check if new fields are available or not. .. c:macro:: PyCurses_API_pointers The number of accessible fields (``4``) in :c:var:`PyCurses_API`. This number is incremented whenever new fields are added. .. c:var:: PyTypeObject PyCursesWindow_Type The :ref:`heap type ` corresponding to :class:`curses.window`. .. c:function:: int PyCursesWindow_Check(PyObject *op) Return true if *op* is a :class:`curses.window` instance, false otherwise. The following macros are convenience macros expanding into C statements. In particular, they can only be used as ``macro;`` or ``macro``, but not ``macro()`` or ``macro();``. .. c:macro:: PyCursesSetupTermCalled Macro checking if :func:`curses.setupterm` has been called. The macro expansion is roughly equivalent to: .. code-block:: { typedef int (*predicate_t)(void); predicate_t was_setupterm_called = (predicate_t)PyCurses_API[1]; if (!was_setupterm_called()) { return NULL; } } .. c:macro:: PyCursesInitialised Macro checking if :func:`curses.initscr` has been called. The macro expansion is roughly equivalent to: .. code-block:: { typedef int (*predicate_t)(void); predicate_t was_initscr_called = (predicate_t)PyCurses_API[2]; if (!was_initscr_called()) { return NULL; } } .. c:macro:: PyCursesInitialisedColor Macro checking if :func:`curses.start_color` has been called. The macro expansion is roughly equivalent to: .. code-block:: { typedef int (*predicate_t)(void); predicate_t was_start_color_called = (predicate_t)PyCurses_API[3]; if (!was_start_color_called()) { return NULL; } } Internal data ------------- The following objects are exposed by the C API but should be considered internal-only. .. c:macro:: PyCurses_CAPSULE_NAME Name of the curses capsule to pass to :c:func:`PyCapsule_Import`. Internal usage only. Use :c:macro:`import_curses` instead. --- ### Doc C Api Datetime (`Doc/c-api/datetime.rst`) .. highlight:: c .. _datetimeobjects: DateTime Objects ---------------- Various date and time objects are supplied by the :mod:`datetime` module. Before using any of these functions, the header file :file:`datetime.h` must be included in your source (note that this is not included by :file:`Python.h`), and the macro :c:macro:`PyDateTime_IMPORT` must be invoked, usually as part of the module initialisation function. The macro puts a pointer to a C structure into a static variable, :c:data:`PyDateTimeAPI`, that is used by the following macros. .. c:macro:: PyDateTime_IMPORT() Import the datetime C API. On success, populate the :c:var:`PyDateTimeAPI` pointer. On failure, set :c:var:`PyDateTimeAPI` to ``NULL`` and set an exception. The caller must check if an error occurred via :c:func:`PyErr_Occurred`: .. code-block:: PyDateTime_IMPORT; if (PyErr_Occurred()) { /* cleanup */ } .. warning:: This is not compatible with subinterpreters. .. versionchanged:: 3.15 This macro is now thread safe. .. c:type:: PyDateTime_CAPI Structure containing the fields for the datetime C API. The fields of this structure are private and subject to change. Do not use this directly; prefer ``PyDateTime_*`` APIs instead. .. c:var:: PyDateTime_CAPI *PyDateTimeAPI Dynamically allocated object containing the datetime C API. This variable is only available once :c:macro:`PyDateTime_IMPORT` succeeds. .. versionchanged:: 3.15 This variable should not be accessed directly as direct access is not thread-safe. Use :c:func:`PyDateTime_IMPORT` instead. .. c:type:: PyDateTime_Date This subtype of :c:type:`PyObject` represents a Python date object. .. c:type:: PyDateTime_DateTime This subtype of :c:type:`PyObject` represents a Python datetime object. .. c:type:: PyDateTime_Time This subtype of :c:type:`PyObject` represents a Python time object. .. c:type:: PyDateTime_Delta This subtype of :c:type:`PyObject` represents the difference between two datetime values. .. c:var:: PyTypeObject PyDateTime_DateType This instance of :c:type:`PyTypeObject` represents the Python date type; it is the same object as :class:`datetime.date` in the Python layer. .. c:var:: PyTypeObject PyDateTime_DateTimeType This instance of :c:type:`PyTypeObject` represents the Python datetime type; it is the same object as :class:`datetime.datetime` in the Python layer. .. c:var:: PyTypeObject PyDateTime_TimeType This instance of :c:type:`PyTypeObject` represents the Python time type; it is the same object as :class:`datetime.time` in the Python layer. .. c:var:: PyTypeObject PyDateTime_DeltaType This instance of :c:type:`PyTypeObject` represents the Python type for the difference between two datetime values; it is the same object as :class:`datetime.timedelta` in the Python layer. .. c:var:: PyTypeObject PyDateTime_TZInfoType This instance of :c:type:`PyTypeObject` represents the Python time zone info type; it is the same object as :class:`datetime.tzinfo` in the Python layer. Macro for access to the UTC singleton: .. c:var:: PyObject* PyDateTime_TimeZone_UTC Returns the time zone singleton representing UTC, the same object as :attr:`datetime.timezone.utc`. .. versionadded:: 3.7 Type-check macros: .. c:function:: int PyDate_Check(PyObject *ob) Return true if *ob* is of type :c:data:`PyDateTime_DateType` or a subtype of :c:data:`!PyDateTime_DateType`. *ob* must not be ``NULL``. This function always succeeds. .. c:function:: int PyDate_CheckExact(PyObject *ob) Return true if *ob* is of type :c:data:`PyDateTime_DateType`. *ob* must not be ``NULL``. This function always succeeds. .. c:function:: int PyDateTime_Check(PyObject *ob) Return true if *ob* is of type :c:data:`PyDateTime_DateTimeType` or a subtype of :c:data:`!PyDateTime_DateTimeType`. *ob* must not be ``NULL``. This function always succeeds. .. c:function:: int PyDateTime_CheckExact(PyObject *ob) Return true if *ob* is of type :c:data:`PyDateTime_DateTimeType`. *ob* must not be ``NULL``. This function always succeeds. .. c:function:: int PyTime_Check(PyObject *ob) Return true if *ob* is of type :c:data:`PyDateTime_TimeType` or a subtype of :c:data:`!PyDateTime_TimeType`. *ob* must not be ``NULL``. This function always succeeds. .. c:function:: int PyTime_CheckExact(PyObject *ob) Return true if *ob* is of type :c:data:`PyDateTime_TimeType`. *ob* must not be ``NULL``. This function always succeeds. .. c:function:: int PyDelta_Check(PyObject *ob) Return true if *ob* is of type :c:data:`PyDateTime_DeltaType` or a subtype of :c:data:`!PyDateTime_DeltaType`. *ob* must not be ``NULL``. This function always succeeds. .. c:function:: int PyDelta_CheckExact(PyObject *ob) Return true if *ob* is of type :c:data:`PyDateTime_DeltaType`. *ob* must not be ``NULL``. This function always succeeds. .. c:function:: int PyTZInfo_Check(PyObject *ob) Return true if *ob* is of type :c:data:`PyDateTime_TZInfoType` or a subtype of :c:data:`!PyDateTime_TZInfoType`. *ob* must not be ``NULL``. This function always succeeds. .. c:function:: int PyTZInfo_CheckExact(PyObject *ob) Return true if *ob* is of type :c:data:`PyDateTime_TZInfoType`. *ob* must not be ``NULL``. This function always succeeds. Macros to create objects: .. c:function:: PyObject* PyDate_FromDate(int year, int month, int day) Return a :class:`datetime.date` object with the specified year, month and day. .. c:function:: PyObject* PyDateTime_FromDateAndTime(int year, int month, int day, int hour, int minute, int second, int usecond) Return a :class:`datetime.datetime` object with the specified year, month, day, hour, minute, second and microsecond. .. c:function:: PyObject* PyDateTime_FromDateAndTimeAndFold(int year, int month, int day, int hour, int minute, int second, int usecond, int fold) Return a :class:`datetime.datetime` object with the specified year, month, day, hour, minute, second, microsecond and fold. .. versionadded:: 3.6 .. c:function:: PyObject* PyTime_FromTime(int hour, int minute, int second, int usecond) Return a :class:`datetime.time` object with the specified hour, minute, second and microsecond. .. c:function:: PyObject* PyTime_FromTimeAndFold(int hour, int minute, int second, int usecond, int fold) Return a :class:`datetime.time` object with the specified hour, minute, second, microsecond and fold. .. versionadded:: 3.6 .. c:function:: PyObject* PyDelta_FromDSU(int days, int seconds, int useconds) Return a :class:`datetime.timedelta` object representing the given number of days, seconds and microseconds. Normalization is performed so that the resulting number of microseconds and seconds lie in the ranges documented for :class:`datetime.timedelta` objects. .. c:function:: PyObject* PyTimeZone_FromOffset(PyObject *offset) Return a :class:`datetime.timezone` object with an unnamed fixed offset represented by the *offset* argument. .. versionadded:: 3.7 .. c:function:: PyObject* PyTimeZone_FromOffsetAndName(PyObject *offset, PyObject *name) Return a :class:`datetime.timezone` object with a fixed offset represented by the *offset* argument and with tzname *name*. .. versionadded:: 3.7 Macros to extract fields from date objects. The argument must be an instance of :c:type:`PyDateTime_Date`, including subclasses (such as :c:type:`PyDateTime_DateTime`). The argument must not be ``NULL``, and the type is not checked: .. c:function:: int PyDateTime_GET_YEAR(PyDateTime_Date *o) Return the year, as a positive int. .. c:function:: int PyDateTime_GET_MONTH(PyDateTime_Date *o) Return the month, as an int from 1 through 12. .. c:function:: int PyDateTime_GET_DAY(PyDateTime_Date *o) Return the day, as an int from 1 through 31. Macros to extract fields from datetime objects. The argument must be an instance of :c:type:`PyDateTime_DateTime`, including subclasses. The argument must not be ``NULL``, and the type is not checked: .. c:function:: int PyDateTime_DATE_GET_HOUR(PyDateTime_DateTime *o) Return the hour, as an int from 0 through 23. .. c:function:: int PyDateTime_DATE_GET_MINUTE(PyDateTime_DateTime *o) Return the minute, as an int from 0 through 59. .. c:function:: int PyDateTime_DATE_GET_SECOND(PyDateTime_DateTime *o) Return the second, as an int from 0 through 59. .. c:function:: int PyDateTime_DATE_GET_MICROSECOND(PyDateTime_DateTime *o) Return the microsecond, as an int from 0 through 999999. .. c:function:: int PyDateTime_DATE_GET_FOLD(PyDateTime_DateTime *o) Return the fold, as an int from 0 through 1. .. versionadded:: 3.6 .. c:function:: PyObject* PyDateTime_DATE_GET_TZINFO(PyDateTime_DateTime *o) Return the tzinfo (which may be ``None``). .. versionadded:: 3.10 Macros to extract fields from time objects. The argument must be an instance of :c:type:`PyDateTime_Time`, including subclasses. The argument must not be ``NULL``, and the type is not checked: .. c:function:: int PyDateTime_TIME_GET_HOUR(PyDateTime_Time *o) Return the hour, as an int from 0 through 23. .. c:function:: int PyDateTime_TIME_GET_MINUTE(PyDateTime_Time *o) Return the minute, as an int from 0 through 59. .. c:function:: int PyDateTime_TIME_GET_SECOND(PyDateTime_Time *o) Return the second, as an int from 0 through 59. .. c:function:: int PyDateTime_TIME_GET_MICROSECOND(PyDateTime_Time *o) Return the microsecond, as an int from 0 through 999999. .. c:function:: int PyDateTime_TIME_GET_FOLD(PyDateTime_Time *o) Return the fold, as an int from 0 through 1. .. versionadded:: 3.6 .. c:function:: PyObject* PyDateTime_TIME_GET_TZINFO(PyDateTime_Time *o) Return the tzinfo (which may be ``None``). .. versionadded:: 3.10 Macros to extract fields from time delta objects. The argument must be an instance of :c:type:`PyDateTime_Delta`, including subclasses. The argument must not be ``NULL``, and the type is not checked: .. c:function:: int PyDateTime_DELTA_GET_DAYS(PyDateTime_Delta *o) Return the number of days, as an int from -999999999 to 999999999. .. versionadded:: 3.3 .. c:function:: int PyDateTime_DELTA_GET_SECONDS(PyDateTime_Delta *o) Return the number of seconds, as an int from 0 through 86399. .. versionadded:: 3.3 .. c:function:: int PyDateTime_DELTA_GET_MICROSECONDS(PyDateTime_Delta *o) Return the number of microseconds, as an int from 0 through 999999. .. versionadded:: 3.3 Macros for the convenience of modules implementing the DB API: .. c:function:: PyObject* PyDateTime_FromTimestamp(PyObject *args) Create and return a new :class:`datetime.datetime` object given an argument tuple suitable for passing to :meth:`datetime.datetime.fromtimestamp`. .. c:function:: PyObject* PyDate_FromTimestamp(PyObject *args) Create and return a new :class:`datetime.date` object given an argument tuple suitable for passing to :meth:`datetime.date.fromtimestamp`. Internal data ------------- The following symbols are exposed by the C API but should be considered internal-only. .. c:macro:: PyDateTime_CAPSULE_NAME Name of the datetime capsule to pass to :c:func:`PyCapsule_Import`. Internal usage only. Use :c:macro:`PyDateTime_IMPORT` instead. --- ### Doc C Api Descriptor (`Doc/c-api/descriptor.rst`) .. highlight:: c .. _descriptor-objects: Descriptor Objects ------------------ "Descriptors" are objects that describe some attribute of an object. They are found in the dictionary of type objects. .. c:function:: PyObject* PyDescr_NewGetSet(PyTypeObject *type, struct PyGetSetDef *getset) Create a new get-set descriptor for extension type *type* from the :c:type:`PyGetSetDef` structure *getset*. Get-set descriptors expose attributes implemented by C getter and setter functions rather than stored directly in the instance. This is the same kind of descriptor created for entries in :c:member:`~PyTypeObject.tp_getset`, and it appears in Python as a :class:`types.GetSetDescriptorType` object. On success, return a :term:`strong reference` to the descriptor. Return ``NULL`` with an exception set on failure. .. c:function:: PyObject* PyDescr_NewMember(PyTypeObject *type, struct PyMemberDef *member) Create a new member descriptor for extension type *type* from the :c:type:`PyMemberDef` structure *member*. Member descriptors expose fields in the type's C struct as Python attributes. This is the same kind of descriptor created for entries in :c:member:`~PyTypeObject.tp_members`, and it appears in Python as a :class:`types.MemberDescriptorType` object. On success, return a :term:`strong reference` to the descriptor. Return ``NULL`` with an exception set on failure. .. c:var:: PyTypeObject PyMemberDescr_Type The type object for member descriptor objects created from :c:type:`PyMemberDef` structures. These descriptors expose fields of a C struct as attributes on a type, and correspond to :class:`types.MemberDescriptorType` objects in Python. .. c:var:: PyTypeObject PyGetSetDescr_Type The type object for get/set descriptor objects created from :c:type:`PyGetSetDef` structures. These descriptors implement attributes whose value is computed by C getter and setter functions, and are used for many built-in type attributes. They correspond to :class:`types.GetSetDescriptorType` objects in Python. .. c:function:: PyObject* PyDescr_NewMethod(PyTypeObject *type, struct PyMethodDef *meth) Create a new method descriptor for extension type *type* from the :c:type:`PyMethodDef` structure *meth*. Method descriptors expose C functions as methods on a type. This is the same kind of descriptor created for entries in :c:member:`~PyTypeObject.tp_methods`, and it appears in Python as a :class:`types.MethodDescriptorType` object. On success, return a :term:`strong reference` to the descriptor. Return ``NULL`` with an exception set on failure. .. c:var:: PyTypeObject PyMethodDescr_Type The type object for method descriptor objects created from :c:type:`PyMethodDef` structures. These descriptors expose C functions as methods on a type, and correspond to :class:`types.MethodDescriptorType` objects in Python. .. c:struct:: wrapperbase Describes a slot wrapper used by :c:func:`PyDescr_NewWrapper`. Each ``wrapperbase`` record stores the Python-visible name and metadata for a special method implemented by a type slot, together with the wrapper function used to adapt that slot to Python's calling convention. .. c:function:: PyObject* PyDescr_NewWrapper(PyTypeObject *type, struct wrapperbase *base, void *wrapped) Create a new wrapper descriptor for extension type *type* from the :c:struct:`wrapperbase` structure *base* and the wrapped slot function pointer *wrapped*. Wrapper descriptors expose special methods implemented by type slots. This is the same kind of descriptor that CPython creates for slot-based special methods such as ``__repr__`` or ``__add__``, and it appears in Python as a :class:`types.WrapperDescriptorType` object. On success, return a :term:`strong reference` to the descriptor. Return ``NULL`` with an exception set on failure. .. c:var:: PyTypeObject PyWrapperDescr_Type The type object for wrapper descriptor objects created by :c:func:`PyDescr_NewWrapper` and :c:func:`PyWrapper_New`. Wrapper descriptors are used internally to expose special methods implemented via wrapper structures, and appear in Python as :class:`types.WrapperDescriptorType` objects. .. c:function:: PyObject* PyDescr_NewClassMethod(PyTypeObject *type, PyMethodDef *method) Create a new class method descriptor for extension type *type* from the :c:type:`PyMethodDef` structure *method*. Class method descriptors expose C methods that receive the class rather than an instance when accessed. This is the same kind of descriptor created for ``METH_CLASS`` entries in :c:member:`~PyTypeObject.tp_methods`, and it appears in Python as a :class:`types.ClassMethodDescriptorType` object. On success, return a :term:`strong reference` to the descriptor. Return ``NULL`` with an exception set on failure. .. c:function:: int PyDescr_IsData(PyObject *descr) Return non-zero if the descriptor object *descr* describes a data attribute, or ``0`` if it describes a method. *descr* must be a descriptor object; there is no error checking. .. c:function:: PyObject* PyWrapper_New(PyObject *d, PyObject *self) Create a new bound wrapper object from the wrapper descriptor *d* and the instance *self*. This is the bound form of a wrapper descriptor created by :c:func:`PyDescr_NewWrapper`. CPython creates these objects when a slot wrapper is accessed through an instance, and they appear in Python as :class:`types.MethodWrapperType` objects. On success, return a :term:`strong reference` to the wrapper object. Return ``NULL`` with an exception set on failure. .. c:macro:: PyDescr_COMMON This is a macro including the common fields for a descriptor object. This was included in Python's C API by mistake; do not use it in extensions. For creating custom descriptor objects, create a class implementing the descriptor protocol (:c:member:`~PyTypeObject.tp_descr_get` and :c:member:`~PyTypeObject.tp_descr_set`). .. soft-deprecated:: 3.15 Built-in descriptors ^^^^^^^^^^^^^^^^^^^^ .. c:var:: PyTypeObject PyProperty_Type The type object for property objects. This is the same object as :class:`property` in the Python layer. .. c:var:: PyTypeObject PySuper_Type The type object for super objects. This is the same object as :class:`super` in the Python layer. .. c:var:: PyTypeObject PyClassMethod_Type The type of class method objects. This is the same object as :class:`classmethod` in the Python layer. .. c:var:: PyTypeObject PyClassMethodDescr_Type The type object for C-level class method descriptor objects. This is the type of the descriptors created for :func:`classmethod` defined in C extension types, and corresponds to :class:`types.ClassMethodDescriptorType` objects in Python. .. c:function:: PyObject *PyClassMethod_New(PyObject *callable) Create a new :class:`classmethod` object wrapping *callable*. *callable* must be a callable object and must not be ``NULL``. On success, this function returns a :term:`strong reference` to a new class method descriptor. On failure, this function returns ``NULL`` with an exception set. .. c:var:: PyTypeObject PyStaticMethod_Type The type of static method objects. This is the same object as :class:`staticmethod` in the Python layer. .. c:function:: PyObject *PyStaticMethod_New(PyObject *callable) Create a new :class:`staticmethod` object wrapping *callable*. *callable* must be a callable object and must not be ``NULL``. On success, this function returns a :term:`strong reference` to a new static method descriptor. On failure, this function returns ``NULL`` with an exception set. --- ### Doc C Api Dict (`Doc/c-api/dict.rst`) .. highlight:: c .. _dictobjects: Dictionary objects ------------------ .. index:: pair: object; dictionary .. c:type:: PyDictObject This subtype of :c:type:`PyObject` represents a Python dictionary object. .. c:var:: PyTypeObject PyDict_Type This instance of :c:type:`PyTypeObject` represents the Python dictionary type. This is the same object as :class:`dict` in the Python layer. .. c:function:: int PyDict_Check(PyObject *p) Return true if *p* is a dict object or an instance of a subtype of the dict type. This function always succeeds. .. c:function:: int PyDict_CheckExact(PyObject *p) Return true if *p* is a dict object, but not an instance of a subtype of the dict type. This function always succeeds. .. c:function:: PyObject* PyDict_New() Return a new empty dictionary, or ``NULL`` on failure. .. c:function:: PyObject* PyDictProxy_New(PyObject *mapping) Return a :class:`types.MappingProxyType` object for a mapping which enforces read-only behavior. This is normally used to create a view to prevent modification of the dictionary for non-dynamic class types. The first argument can be a :class:`dict`, a :class:`frozendict`, or a mapping. .. versionchanged:: 3.15 Also accept :class:`frozendict`. .. c:var:: PyTypeObject PyDictProxy_Type The type object for mapping proxy objects created by :c:func:`PyDictProxy_New` and for the read-only ``__dict__`` attribute of many built-in types. A :c:type:`PyDictProxy_Type` instance provides a dynamic, read-only view of an underlying dictionary: changes to the underlying dictionary are reflected in the proxy, but the proxy itself does not support mutation operations. This corresponds to :class:`types.MappingProxyType` in Python. .. c:function:: void PyDict_Clear(PyObject *p) Empty an existing dictionary of all key-value pairs. Do nothing if the argument is not a :class:`dict` or a :class:`!dict` subclass. .. c:function:: int PyDict_Contains(PyObject *p, PyObject *key) Determine if dictionary *p* contains *key*. If an item in *p* matches *key*, return ``1``, otherwise return ``0``. On error, return ``-1``. This is equivalent to the Python expression ``key in p``. The first argument can be a :class:`dict` or a :class:`frozendict`. .. note:: The operation is atomic on :term:`free threading ` when *key* is :class:`str`, :class:`int`, :class:`float`, :class:`bool` or :class:`bytes`. .. versionchanged:: 3.15 Also accept :class:`frozendict`. .. c:function:: int PyDict_ContainsString(PyObject *p, const char *key) This is the same as :c:func:`PyDict_Contains`, but *key* is specified as a :c:expr:`const char*` UTF-8 encoded bytes string, rather than a :c:expr:`PyObject*`. The first argument can be a :class:`dict` or a :class:`frozendict`. .. versionadded:: 3.13 .. versionchanged:: 3.15 Also accept :class:`frozendict`. .. c:function:: PyObject* PyDict_Copy(PyObject *p) Return a new dictionary that contains the same key-value pairs as *p*. .. c:function:: int PyDict_SetItem(PyObject *p, PyObject *key, PyObject *val) Insert *val* into the dictionary *p* with a key of *key*. *key* must be :term:`hashable`; if it isn't, :exc:`TypeError` will be raised. Return ``0`` on success or ``-1`` on failure. This function *does not* ":term:`steal`" a reference to *val*. .. note:: The operation is atomic on :term:`free threading ` when *key* is :class:`str`, :class:`int`, :class:`float`, :class:`bool` or :class:`bytes`. .. c:function:: int PyDict_SetItemString(PyObject *p, const char *key, PyObject *val) This is the same as :c:func:`PyDict_SetItem`, but *key* is specified as a :c:expr:`const char*` UTF-8 encoded bytes string, rather than a :c:expr:`PyObject*`. .. c:function:: int PyDict_DelItem(PyObject *p, PyObject *key) Remove the entry in dictionary *p* with key *key*. *key* must be :term:`hashable`; if it isn't, :exc:`TypeError` is raised. If *key* is not in the dictionary, :exc:`KeyError` is raised. Return ``0`` on success or ``-1`` on failure. .. note:: The operation is atomic on :term:`free threading ` when *key* is :class:`str`, :class:`int`, :class:`float`, :class:`bool` or :class:`bytes`. .. c:function:: int PyDict_DelItemString(PyObject *p, const char *key) This is the same as :c:func:`PyDict_DelItem`, but *key* is specified as a :c:expr:`const char*` UTF-8 encoded bytes string, rather than a :c:expr:`PyObject*`. .. c:function:: int PyDict_GetItemRef(PyObject *p, PyObject *key, PyObject **result) Return a new :term:`strong reference` to the object from dictionary *p* which has a key *key*: * If the key is present, set *\*result* to a new :term:`strong reference` to the value and return ``1``. * If the key is missing, set *\*result* to ``NULL`` and return ``0``. * On error, raise an exception, set *\*result* to ``NULL`` and return ``-1``. The first argument can be a :class:`dict` or a :class:`frozendict`. .. note:: The operation is atomic on :term:`free threading ` when *key* is :class:`str`, :class:`int`, :class:`float`, :class:`bool` or :class:`bytes`. .. versionadded:: 3.13 .. versionchanged:: 3.15 Also accept :class:`frozendict`. See also the :c:func:`PyObject_GetItem` function. .. c:function:: PyObject* PyDict_GetItem(PyObject *p, PyObject *key) Return a :term:`borrowed reference` to the object from dictionary *p* which has a key *key*. Return ``NULL`` if the key *key* is missing *without* setting an exception. The first argument can be a :class:`dict` or a :class:`frozendict`. .. note:: Exceptions that occur while this calls :meth:`~object.__hash__` and :meth:`~object.__eq__` methods are silently ignored. Prefer the :c:func:`PyDict_GetItemWithError` function instead. .. note:: In the :term:`free-threaded build`, the returned :term:`borrowed reference` may become invalid if another thread modifies the dictionary concurrently. Prefer :c:func:`PyDict_GetItemRef`, which returns a :term:`strong reference`. .. versionchanged:: 3.10 Calling this API without an :term:`attached thread state` had been allowed for historical reason. It is no longer allowed. .. versionchanged:: 3.15 Also accept :class:`frozendict`. .. c:function:: PyObject* PyDict_GetItemWithError(PyObject *p, PyObject *key) Variant of :c:func:`PyDict_GetItem` that does not suppress exceptions. Return ``NULL`` **with** an exception set if an exception occurred. Return ``NULL`` **without** an exception set if the key wasn't present. .. note:: In the :term:`free-threaded build`, the returned :term:`borrowed reference` may become invalid if another thread modifies the dictionary concurrently. Prefer :c:func:`PyDict_GetItemRef`, which returns a :term:`strong reference`. .. versionchanged:: 3.15 Also accept :class:`frozendict`. .. c:function:: PyObject* PyDict_GetItemString(PyObject *p, const char *key) This is the same as :c:func:`PyDict_GetItem`, but *key* is specified as a :c:expr:`const char*` UTF-8 encoded bytes string, rather than a :c:expr:`PyObject*`. .. note:: Exceptions that occur while this calls :meth:`~object.__hash__` and :meth:`~object.__eq__` methods or while creating the temporary :class:`str` object are silently ignored. Prefer using the :c:func:`PyDict_GetItemWithError` function with your own :c:func:`PyUnicode_FromString` *key* instead. .. note:: In the :term:`free-threaded build`, the returned :term:`borrowed reference` may become invalid if another thread modifies the dictionary concurrently. Prefer :c:func:`PyDict_GetItemStringRef`, which returns a :term:`strong reference`. .. versionchanged:: 3.15 Also accept :class:`frozendict`. .. c:function:: int PyDict_GetItemStringRef(PyObject *p, const char *key, PyObject **result) Similar to :c:func:`PyDict_GetItemRef`, but *key* is specified as a :c:expr:`const char*` UTF-8 encoded bytes string, rather than a :c:expr:`PyObject*`. .. versionadded:: 3.13 .. versionchanged:: 3.15 Also accept :class:`frozendict`. .. c:function:: PyObject* PyDict_SetDefault(PyObject *p, PyObject *key, PyObject *defaultobj) This is the same as the Python-level :meth:`dict.setdefault`. If present, it returns the value corresponding to *key* from the dictionary *p*. If the key is not in the dict, it is inserted with value *defaultobj* and *defaultobj* is returned. This function evaluates the hash function of *key* only once, instead of evaluating it independently for the lookup and the insertion. .. versionadded:: 3.4 .. note:: In the :term:`free-threaded build`, the returned :term:`borrowed reference` may become invalid if another thread modifies the dictionary concurrently. Prefer :c:func:`PyDict_SetDefaultRef`, which returns a :term:`strong reference`. .. c:function:: int PyDict_SetDefaultRef(PyObject *p, PyObject *key, PyObject *default_value, PyObject **result) Inserts *default_value* into the dictionary *p* with a key of *key* if the key is not already present in the dictionary. If *result* is not ``NULL``, then *\*result* is set to a :term:`strong reference` to either *default_value*, if the key was not present, or the existing value, if *key* was already present in the dictionary. Returns ``1`` if the key was present and *default_value* was not inserted, or ``0`` if the key was not present and *default_value* was inserted. On failure, returns ``-1``, sets an exception, and sets ``*result`` to ``NULL``. For clarity: if you have a strong reference to *default_value* before calling this function, then after it returns, you hold a strong reference to both *default_value* and *\*result* (if it's not ``NULL``). These may refer to the same object: in that case you hold two separate references to it. .. note:: The operation is atomic on :term:`free threading ` when *key* is :class:`str`, :class:`int`, :class:`float`, :class:`bool` or :class:`bytes`. .. versionadded:: 3.13 .. c:function:: int PyDict_Pop(PyObject *p, PyObject *key, PyObject **result) Remove *key* from dictionary *p* and optionally return the removed value. Do not raise :exc:`KeyError` if the key is missing. - If the key is present, set *\*result* to a new reference to the removed value if *result* is not ``NULL``, and return ``1``. - If the key is missing, set *\*result* to ``NULL`` if *result* is not ``NULL``, and return ``0``. - On error, raise an exception and return ``-1``. Similar to :meth:`dict.pop`, but without the default value and not raising :exc:`KeyError` if the key is missing. .. note:: The operation is atomic on :term:`free threading ` when *key* is :class:`str`, :class:`int`, :class:`float`, :class:`bool` or :class:`bytes`. .. versionadded:: 3.13 .. c:function:: int PyDict_PopString(PyObject *p, const char *key, PyObject **result) Similar to :c:func:`PyDict_Pop`, but *key* is specified as a :c:expr:`const char*` UTF-8 encoded bytes string, rather than a :c:expr:`PyObject*`. .. versionadded:: 3.13 .. c:function:: PyObject* PyDict_Items(PyObject *p) Return a :c:type:`PyListObject` containing all the items from the dictionary. The first argument can be a :class:`dict` or a :class:`frozendict`. .. versionchanged:: 3.15 Also accept :class:`frozendict`. .. c:function:: PyObject* PyDict_Keys(PyObject *p) Return a :c:type:`PyListObject` containing all the keys from the dictionary. The first argument can be a :class:`dict` or a :class:`frozendict`. .. versionchanged:: 3.15 Also accept :class:`frozendict`. .. c:function:: PyObject* PyDict_Values(PyObject *p) Return a :c:type:`PyListObject` containing all the values from the dictionary *p*. The first argument can be a :class:`dict` or a :class:`frozendict`. .. versionchanged:: 3.15 Also accept :class:`frozendict`. .. c:function:: Py_ssize_t PyDict_Size(PyObject *p) .. index:: pair: built-in function; len Return the number of items in the dictionary. This is equivalent to ``len(p)`` on a dictionary. The argument can be a :class:`dict` or a :class:`frozendict`. .. versionchanged:: 3.15 Also accept :class:`frozendict`. .. c:function:: Py_ssize_t PyDict_GET_SIZE(PyObject *p) Similar to :c:func:`PyDict_Size`, but without error checking. .. versionchanged:: 3.15 Also accept :class:`frozendict`. .. c:function:: int PyDict_Next(PyObject *p, Py_ssize_t *ppos, PyObject **pkey, PyObject **pvalue) Iterate over all key-value pairs in the dictionary *p*. The :c:type:`Py_ssize_t` referred to by *ppos* must be initialized to ``0`` prior to the first call to this function to start the iteration; the function returns true for each pair in the dictionary, and false once all pairs have been reported. The parameters *pkey* and *pvalue* should either point to :c:expr:`PyObject*` variables that will be filled in with each key and value, respectively, or may be ``NULL``. Any references returned through them are borrowed. *ppos* should not be altered during iteration. Its value represents offsets within the internal dictionary structure, and since the structure is sparse, the offsets are not consecutive. The first argument can be a :class:`dict` or a :class:`frozendict`. For example:: PyObject *key, *value; Py_ssize_t pos = 0; while (PyDict_Next(self->dict, &pos, &key, &value)) { /* do something interesting with the values... */ ... } The dictionary *p* should not be mutated during iteration. It is safe to modify the values of the keys as you iterate over the dictionary, but only so long as the set of keys does not change. For example:: PyObject *key, *value; Py_ssize_t pos = 0; while (PyDict_Next(self->dict, &pos, &key, &value)) { long i = PyLong_AsLong(value); if (i == -1 && PyErr_Occurred()) { return -1; } PyObject *o = PyLong_FromLong(i + 1); if (o == NULL) return -1; if (PyDict_SetItem(self->dict, key, o) < 0) { Py_DECREF(o); return -1; } Py_DECREF(o); } The function is not thread-safe in the :term:`free-threaded ` build without external synchronization for a mutable :class:`dict`. You can use :c:macro:`Py_BEGIN_CRITICAL_SECTION` to lock the dictionary while iterating over it:: Py_BEGIN_CRITICAL_SECTION(self->dict); while (PyDict_Next(self->dict, &pos, &key, &value)) { ... } Py_END_CRITICAL_SECTION(); The function is thread-safe on a :class:`frozendict`. .. note:: On the free-threaded build, this function can be used safely inside a critical section. However, the references returned for *pkey* and *pvalue* are :term:`borrowed ` and are only valid while the critical section is held. If you need to use these objects outside the critical section or when the critical section can be suspended, create a :term:`strong reference ` (for example, using :c:func:`Py_NewRef`). .. versionchanged:: 3.15 Also accept :class:`frozendict`. .. c:function:: int PyDict_Merge(PyObject *a, PyObject *b, int override) Iterate over mapping object *b* adding key-value pairs to dictionary *a*. *b* may be a dictionary, or any object supporting :c:func:`PyMapping_Keys` and :c:func:`PyObject_GetItem`. If *override* is true, existing pairs in *a* will be replaced if a matching key is found in *b*, otherwise pairs will only be added if there is not a matching key in *a*. Return ``0`` on success or ``-1`` if an exception was raised. .. note:: In the :term:`free-threaded build`, when *b* is a :class:`dict` (with the standard iterator), both *a* and *b* are locked for the duration of the operation. When *b* is a non-dict mapping, only *a* is locked; *b* may be concurrently modified by another thread. .. c:function:: int PyDict_Update(PyObject *a, PyObject *b) This is the same as ``PyDict_Merge(a, b, 1)`` in C, and is similar to ``a.update(b)`` in Python except that :c:func:`PyDict_Update` doesn't fall back to the iterating over a sequence of key value pairs if the second argument has no "keys" attribute. Return ``0`` on success or ``-1`` if an exception was raised. .. note:: In the :term:`free-threaded build`, when *b* is a :class:`dict` (with the standard iterator), both *a* and *b* are locked for the duration of the operation. When *b* is a non-dict mapping, only *a* is locked; *b* may be concurrently modified by another thread. .. c:function:: int PyDict_MergeFromSeq2(PyObject *a, PyObject *seq2, int override) Update or merge into dictionary *a*, from the key-value pairs in *seq2*. *seq2* must be an iterable object producing iterable objects of length 2, viewed as key-value pairs. In case of duplicate keys, the last wins if *override* is true, else the first wins. Return ``0`` on success or ``-1`` if an exception was raised. Equivalent Python (except for the return value):: def PyDict_MergeFromSeq2(a, seq2, override): for key, value in seq2: if override or key not in a: a[key] = value .. note:: In the :term:`free-threaded ` build, only *a* is locked. The iteration over *seq2* is not synchronized; *seq2* may be concurrently modified by another thread. .. c:function:: int PyDict_AddWatcher(PyDict_WatchCallback callback) Register *callback* as a dictionary watcher. Return a non-negative integer id which must be passed to future calls to :c:func:`PyDict_Watch`. In case of error (e.g. no more watcher IDs available), return ``-1`` and set an exception. .. note:: This function is not internally synchronized. In the :term:`free-threaded ` build, callers should ensure no concurrent calls to :c:func:`PyDict_AddWatcher` or :c:func:`PyDict_ClearWatcher` are in progress. .. versionadded:: 3.12 .. c:function:: int PyDict_ClearWatcher(int watcher_id) Clear watcher identified by *watcher_id* previously returned from :c:func:`PyDict_AddWatcher`. Return ``0`` on success, ``-1`` on error (e.g. if the given *watcher_id* was never registered.) .. note:: This function is not internally synchronized. In the :term:`free-threaded ` build, callers should ensure no concurrent calls to :c:func:`PyDict_AddWatcher` or :c:func:`PyDict_ClearWatcher` are in progress. .. versionadded:: 3.12 .. c:function:: int PyDict_Watch(int watcher_id, PyObject *dict) Mark dictionary *dict* as watched. The callback granted *watcher_id* by :c:func:`PyDict_AddWatcher` will be called when *dict* is modified or deallocated. Return ``0`` on success or ``-1`` on error. .. versionadded:: 3.12 .. c:function:: int PyDict_Unwatch(int watcher_id, PyObject *dict) Mark dictionary *dict* as no longer watched. The callback granted *watcher_id* by :c:func:`PyDict_AddWatcher` will no longer be called when *dict* is modified or deallocated. The dict must previously have been watched by this watcher. Return ``0`` on success or ``-1`` on error. .. versionadded:: 3.12 .. c:type:: PyDict_WatchEvent Enumeration of possible dictionary watcher events: ``PyDict_EVENT_ADDED``, ``PyDict_EVENT_MODIFIED``, ``PyDict_EVENT_DELETED``, ``PyDict_EVENT_CLONED``, ``PyDict_EVENT_CLEARED``, or ``PyDict_EVENT_DEALLOCATED``. .. versionadded:: 3.12 .. c:type:: int (*PyDict_WatchCallback)(PyDict_WatchEvent event, PyObject *dict, PyObject *key, PyObject *new_value) Type of a dict watcher callback function. If *event* is ``PyDict_EVENT_CLEARED`` or ``PyDict_EVENT_DEALLOCATED``, both *key* and *new_value* will be ``NULL``. If *event* is ``PyDict_EVENT_ADDED`` or ``PyDict_EVENT_MODIFIED``, *new_value* will be the new value for *key*. If *event* is ``PyDict_EVENT_DELETED``, *key* is being deleted from the dictionary and *new_value* will be ``NULL``. ``PyDict_EVENT_CLONED`` occurs when *dict* was previously empty and another dict is merged into it. To maintain efficiency of this operation, per-key ``PyDict_EVENT_ADDED`` events are not issued in this case; instead a single ``PyDict_EVENT_CLONED`` is issued, and *key* will be the source dictionary. The callback may inspect but must not modify *dict*; doing so could have unpredictable effects, including infinite recursion. Do not trigger Python code execution in the callback, as it could modify the dict as a side effect. If *event* is ``PyDict_EVENT_DEALLOCATED``, taking a new reference in the callback to the about-to-be-destroyed dictionary will resurrect it and prevent it from being freed at this time. When the resurrected object is destroyed later, any watcher callbacks active at that time will be called again. Callbacks occur before the notified modification to *dict* takes place, so the prior state of *dict* can be inspected. If the callback sets an exception, it must return ``-1``; this exception will be printed as an unraisable exception using :c:func:`PyErr_WriteUnraisable`. Otherwise it should return ``0``. There may already be a pending exception set on entry to the callback. In this case, the callback should return ``0`` with the same exception still set. This means the callback may not call any other API that can set an exception unless it saves and clears the exception state first, and restores it before returning. .. versionadded:: 3.12 Dictionary view objects ^^^^^^^^^^^^^^^^^^^^^^^ .. c:function:: int PyDictViewSet_Check(PyObject *op) Return true if *op* is a view of a set inside a dictionary. This is currently equivalent to :c:expr:`PyDictKeys_Check(op) || PyDictItems_Check(op)`. This function always succeeds. .. c:var:: PyTypeObject PyDictKeys_Type Type object for a view of dictionary keys. In Python, this is the type of the object returned by :meth:`dict.keys`. .. c:function:: int PyDictKeys_Check(PyObject *op) Return true if *op* is an instance of a dictionary keys view. This function always succeeds. .. c:var:: PyTypeObject PyDictValues_Type Type object for a view of dictionary values. In Python, this is the type of the object returned by :meth:`dict.values`. .. c:function:: int PyDictValues_Check(PyObject *op) Return true if *op* is an instance of a dictionary values view. This function always succeeds. .. c:var:: PyTypeObject PyDictItems_Type Type object for a view of dictionary items. In Python, this is the type of the object returned by :meth:`dict.items`. .. c:function:: int PyDictItems_Check(PyObject *op) Return true if *op* is an instance of a dictionary items view. This function always succeeds. Frozen dictionary objects ^^^^^^^^^^^^^^^^^^^^^^^^^ .. versionadded:: 3.15 .. c:var:: PyTypeObject PyFrozenDict_Type This instance of :c:type:`PyTypeObject` represents the Python frozen dictionary type. This is the same object as :class:`frozendict` in the Python layer. .. c:function:: int PyAnyDict_Check(PyObject *p) Return true if *p* is a :class:`dict` object, a :class:`frozendict` object, or an instance of a subtype of the :class:`!dict` or :class:`!frozendict` type. This function always succeeds. .. c:function:: int PyAnyDict_CheckExact(PyObject *p) Return true if *p* is a :class:`dict` object or a :class:`frozendict` object, but not an instance of a subtype of the :class:`!dict` or :class:`!frozendict` type. This function always succeeds. .. c:function:: int PyFrozenDict_Check(PyObject *p) Return true if *p* is a :class:`frozendict` object or an instance of a subtype of the :class:`!frozendict` type. This function always succeeds. .. c:function:: int PyFrozenDict_CheckExact(PyObject *p) Return true if *p* is a :class:`frozendict` object, but not an instance of a subtype of the :class:`!frozendict` type. This function always succeeds. .. c:function:: PyObject* PyFrozenDict_New(PyObject *iterable) Return a new :class:`frozendict` from an iterable, or ``NULL`` on failure with an exception set. Create an empty dictionary if *iterable* is ``NULL``. Ordered dictionaries ^^^^^^^^^^^^^^^^^^^^ Python's C API provides interface for :class:`collections.OrderedDict` from C. Since Python 3.7, dictionaries are ordered by default, so there is usually little need for these functions; prefer ``PyDict*`` where possible. .. c:var:: PyTypeObject PyODict_Type Type object for ordered dictionaries. This is the same object as :class:`collections.OrderedDict` in the Python layer. .. c:function:: int PyODict_Check(PyObject *od) Return true if *od* is an ordered dictionary object or an instance of a subtype of the :class:`~collections.OrderedDict` type. This function always succeeds. .. c:function:: int PyODict_CheckExact(PyObject *od) Return true if *od* is an ordered dictionary object, but not an instance of a subtype of the :class:`~collections.OrderedDict` type. This function always succeeds. .. c:var:: PyTypeObject PyODictKeys_Type Analogous to :c:type:`PyDictKeys_Type` for ordered dictionaries. .. c:var:: PyTypeObject PyODictValues_Type Analogous to :c:type:`PyDictValues_Type` for ordered dictionaries. .. c:var:: PyTypeObject PyODictItems_Type Analogous to :c:type:`PyDictItems_Type` for ordered dictionaries. .. c:function:: PyObject *PyODict_New(void) Return a new empty ordered dictionary, or ``NULL`` on failure. This is analogous to :c:func:`PyDict_New`. .. c:function:: int PyODict_SetItem(PyObject *od, PyObject *key, PyObject *value) Insert *value* into the ordered dictionary *od* with a key of *key*. Return ``0`` on success or ``-1`` with an exception set on failure. This is analogous to :c:func:`PyDict_SetItem`. .. c:function:: int PyODict_DelItem(PyObject *od, PyObject *key) Remove the entry in the ordered dictionary *od* with key *key*. Return ``0`` on success or ``-1`` with an exception set on failure. This is analogous to :c:func:`PyDict_DelItem`. These are :term:`soft deprecated` aliases to ``PyDict`` APIs: .. list-table:: :widths: auto :header-rows: 1 * * ``PyODict`` * ``PyDict`` * * .. c:macro:: PyODict_GetItem(od, key) * :c:func:`PyDict_GetItem` * * .. c:macro:: PyODict_GetItemWithError(od, key) * :c:func:`PyDict_GetItemWithError` * * .. c:macro:: PyODict_GetItemString(od, key) * :c:func:`PyDict_GetItemString` * * .. c:macro:: PyODict_Contains(od, key) * :c:func:`PyDict_Contains` * * .. c:macro:: PyODict_Size(od) * :c:func:`PyDict_Size` * * .. c:macro:: PyODict_SIZE(od) * :c:func:`PyDict_GET_SIZE` --- ### Doc C Api Exceptions (`Doc/c-api/exceptions.rst`) .. highlight:: c .. _exceptionhandling: ****************** Exception Handling ****************** The functions described in this chapter will let you handle and raise Python exceptions. It is important to understand some of the basics of Python exception handling. It works somewhat like the POSIX :c:data:`errno` variable: there is a global indicator (per thread) of the last error that occurred. Most C API functions don't clear this on success, but will set it to indicate the cause of the error on failure. Most C API functions also return an error indicator, usually ``NULL`` if they are supposed to return a pointer, or ``-1`` if they return an integer (exception: the ``PyArg_*`` functions return ``1`` for success and ``0`` for failure). Concretely, the error indicator consists of three object pointers: the exception's type, the exception's value, and the traceback object. Any of those pointers can be ``NULL`` if non-set (although some combinations are forbidden, for example you can't have a non-``NULL`` traceback if the exception type is ``NULL``). When a function must fail because some function it called failed, it generally doesn't set the error indicator; the function it called already set it. It is responsible for either handling the error and clearing the exception or returning after cleaning up any resources it holds (such as object references or memory allocations); it should *not* continue normally if it is not prepared to handle the error. If returning due to an error, it is important to indicate to the caller that an error has been set. If the error is not handled or carefully propagated, additional calls into the Python/C API may not behave as intended and may fail in mysterious ways. .. note:: The error indicator is **not** the result of :func:`sys.exc_info`. The former corresponds to an exception that is not yet caught (and is therefore still propagating), while the latter returns an exception after it is caught (and has therefore stopped propagating). Printing and clearing ===================== .. c:function:: void PyErr_Clear() Clear the error indicator. If the error indicator is not set, there is no effect. .. c:function:: void PyErr_PrintEx(int set_sys_last_vars) Print a standard traceback to ``sys.stderr`` and clear the error indicator. **Unless** the error is a ``SystemExit``, in that case no traceback is printed and the Python process will exit with the error code specified by the ``SystemExit`` instance. Call this function **only** when the error indicator is set. Otherwise it will cause a fatal error! If *set_sys_last_vars* is nonzero, the variable :data:`sys.last_exc` is set to the printed exception. For backwards compatibility, the deprecated variables :data:`sys.last_type`, :data:`sys.last_value` and :data:`sys.last_traceback` are also set to the type, value and traceback of this exception, respectively. .. versionchanged:: 3.12 The setting of :data:`sys.last_exc` was added. .. c:function:: void PyErr_Print() Alias for ``PyErr_PrintEx(1)``. .. c:function:: void PyErr_WriteUnraisable(PyObject *obj) Call :func:`sys.unraisablehook` using the current exception and *obj* argument. This utility function prints a warning message to ``sys.stderr`` when an exception has been set but it is impossible for the interpreter to actually raise the exception. It is used, for example, when an exception occurs in an :meth:`~object.__del__` method. The function is called with a single argument *obj* that identifies the context in which the unraisable exception occurred. If possible, the repr of *obj* will be printed in the warning message. If *obj* is ``NULL``, only the traceback is printed. An exception must be set when calling this function. .. versionchanged:: 3.4 Print a traceback. Print only traceback if *obj* is ``NULL``. .. versionchanged:: 3.8 Use :func:`sys.unraisablehook`. .. c:function:: void PyErr_FormatUnraisable(const char *format, ...) Similar to :c:func:`PyErr_WriteUnraisable`, but the *format* and subsequent parameters help format the warning message; they have the same meaning and values as in :c:func:`PyUnicode_FromFormat`. ``PyErr_WriteUnraisable(obj)`` is roughly equivalent to ``PyErr_FormatUnraisable("Exception ignored in: %R", obj)``. If *format* is ``NULL``, only the traceback is printed. .. versionadded:: 3.13 .. c:function:: void PyErr_DisplayException(PyObject *exc) Print the standard traceback display of ``exc`` to ``sys.stderr``, including chained exceptions and notes. .. versionadded:: 3.12 .. c:function:: void PyErr_Display(PyObject *unused, PyObject *value, PyObject *tb) Legacy variant of :c:func:`PyErr_DisplayException`. Print the exception *value* with its traceback to :data:`sys.stderr`. If *value* has no traceback set, *tb* is used as its traceback. The first argument is ignored. If :data:`sys.stderr` is ``None``, nothing is printed. If :data:`sys.stderr` is not set, the exception is dumped to the C ``stderr`` stream instead. .. deprecated:: 3.12 Use :c:func:`PyErr_DisplayException` instead. Raising exceptions ================== These functions help you set the current thread's error indicator. For convenience, some of these functions will always return a ``NULL`` pointer for use in a ``return`` statement. .. c:function:: void PyErr_SetString(PyObject *type, const char *message) This is the most common way to set the error indicator. The first argument specifies the exception type; it is normally one of the standard exceptions, e.g. :c:data:`PyExc_RuntimeError`. You need not create a new :term:`strong reference` to it (e.g. with :c:func:`Py_INCREF`). The second argument is an error message; it is decoded from ``'utf-8'``. .. c:function:: void PyErr_SetObject(PyObject *type, PyObject *value) This function is similar to :c:func:`PyErr_SetString` but lets you specify an arbitrary Python object for the "value" of the exception. .. c:function:: PyObject* PyErr_Format(PyObject *exception, const char *format, ...) This function sets the error indicator and returns ``NULL``. *exception* should be a Python exception class. The *format* and subsequent parameters help format the error message; they have the same meaning and values as in :c:func:`PyUnicode_FromFormat`. *format* is an ASCII-encoded string. .. c:function:: PyObject* PyErr_FormatV(PyObject *exception, const char *format, va_list vargs) Same as :c:func:`PyErr_Format`, but taking a :c:type:`va_list` argument rather than a variable number of arguments. .. versionadded:: 3.5 .. c:function:: void PyErr_SetNone(PyObject *type) This is a shorthand for ``PyErr_SetObject(type, Py_None)``. .. c:function:: int PyErr_BadArgument() This is a shorthand for ``PyErr_SetString(PyExc_TypeError, message)``, where *message* indicates that a built-in operation was invoked with an illegal argument. It is mostly for internal use. .. c:function:: PyObject* PyErr_NoMemory() This is a shorthand for ``PyErr_SetNone(PyExc_MemoryError)``; it returns ``NULL`` so an object allocation function can write ``return PyErr_NoMemory();`` when it runs out of memory. .. c:function:: PyObject* PyErr_SetFromErrno(PyObject *type) .. index:: single: strerror (C function) This is a convenience function to raise an exception when a C library function has returned an error and set the C variable :c:data:`errno`. It constructs a tuple object whose first item is the integer :c:data:`errno` value and whose second item is the corresponding error message (gotten from :c:func:`!strerror`), and then calls ``PyErr_SetObject(type, object)``. On Unix, when the :c:data:`errno` value is :c:macro:`!EINTR`, indicating an interrupted system call, this calls :c:func:`PyErr_CheckSignals`, and if that set the error indicator, leaves it set to that. The function always returns ``NULL``, so a wrapper function around a system call can write ``return PyErr_SetFromErrno(type);`` when the system call returns an error. .. c:function:: PyObject* PyErr_SetFromErrnoWithFilenameObject(PyObject *type, PyObject *filenameObject) Similar to :c:func:`PyErr_SetFromErrno`, with the additional behavior that if *filenameObject* is not ``NULL``, it is passed to the constructor of *type* as a third parameter. In the case of :exc:`OSError` exception, this is used to define the :attr:`!filename` attribute of the exception instance. .. c:function:: PyObject* PyErr_SetFromErrnoWithFilenameObjects(PyObject *type, PyObject *filenameObject, PyObject *filenameObject2) Similar to :c:func:`PyErr_SetFromErrnoWithFilenameObject`, but takes a second filename object, for raising errors when a function that takes two filenames fails. .. versionadded:: 3.4 .. c:function:: PyObject* PyErr_SetFromErrnoWithFilename(PyObject *type, const char *filename) Similar to :c:func:`PyErr_SetFromErrnoWithFilenameObject`, but the filename is given as a C string. *filename* is decoded from the :term:`filesystem encoding and error handler`. .. c:function:: PyObject* PyErr_SetFromWindowsErr(int ierr) This is a convenience function to raise :exc:`OSError`. If called with *ierr* of ``0``, the error code returned by a call to :c:func:`!GetLastError` is used instead. It calls the Win32 function :c:func:`!FormatMessage` to retrieve the Windows description of error code given by *ierr* or :c:func:`!GetLastError`, then it constructs a :exc:`OSError` object with the :attr:`~OSError.winerror` attribute set to the error code, the :attr:`~OSError.strerror` attribute set to the corresponding error message (gotten from :c:func:`!FormatMessage`), and then calls ``PyErr_SetObject(PyExc_OSError, object)``. This function always returns ``NULL``. .. availability:: Windows. .. c:function:: PyObject* PyErr_SetExcFromWindowsErr(PyObject *type, int ierr) Similar to :c:func:`PyErr_SetFromWindowsErr`, with an additional parameter specifying the exception type to be raised. .. availability:: Windows. .. c:function:: PyObject* PyErr_SetFromWindowsErrWithFilename(int ierr, const char *filename) Similar to :c:func:`PyErr_SetFromWindowsErr`, with the additional behavior that if *filename* is not ``NULL``, it is decoded from the filesystem encoding (:func:`os.fsdecode`) and passed to the constructor of :exc:`OSError` as a third parameter to be used to define the :attr:`!filename` attribute of the exception instance. .. availability:: Windows. .. c:function:: PyObject* PyErr_SetExcFromWindowsErrWithFilenameObject(PyObject *type, int ierr, PyObject *filename) Similar to :c:func:`PyErr_SetExcFromWindowsErr`, with the additional behavior that if *filename* is not ``NULL``, it is passed to the constructor of :exc:`OSError` as a third parameter to be used to define the :attr:`!filename` attribute of the exception instance. .. availability:: Windows. .. c:function:: PyObject* PyErr_SetExcFromWindowsErrWithFilenameObjects(PyObject *type, int ierr, PyObject *filename, PyObject *filename2) Similar to :c:func:`PyErr_SetExcFromWindowsErrWithFilenameObject`, but accepts a second filename object. .. availability:: Windows. .. versionadded:: 3.4 .. c:function:: PyObject* PyErr_SetExcFromWindowsErrWithFilename(PyObject *type, int ierr, const char *filename) Similar to :c:func:`PyErr_SetFromWindowsErrWithFilename`, with an additional parameter specifying the exception type to be raised. .. availability:: Windows. .. c:function:: PyObject* PyErr_SetImportError(PyObject *msg, PyObject *name, PyObject *path) This is a convenience function to raise :exc:`ImportError`. *msg* will be set as the exception's message string. *name* and *path*, both of which can be ``NULL``, will be set as the :exc:`ImportError`'s respective ``name`` and ``path`` attributes. .. versionadded:: 3.3 .. c:function:: PyObject* PyErr_SetImportErrorSubclass(PyObject *exception, PyObject *msg, PyObject *name, PyObject *path) Much like :c:func:`PyErr_SetImportError` but this function allows for specifying a subclass of :exc:`ImportError` to raise. .. versionadded:: 3.6 .. c:function:: void PyErr_SyntaxLocationObject(PyObject *filename, int lineno, int col_offset) Set file, line, and offset information for the current exception. If the current exception is not a :exc:`SyntaxError`, then it sets additional attributes, which make the exception printing subsystem think the exception is a :exc:`SyntaxError`. .. versionadded:: 3.4 .. c:function:: void PyErr_RangedSyntaxLocationObject(PyObject *filename, int lineno, int col_offset, int end_lineno, int end_col_offset) Similar to :c:func:`PyErr_SyntaxLocationObject`, but also sets the *end_lineno* and *end_col_offset* information for the current exception. .. versionadded:: 3.10 .. c:function:: void PyErr_SyntaxLocationEx(const char *filename, int lineno, int col_offset) Like :c:func:`PyErr_SyntaxLocationObject`, but *filename* is a byte string decoded from the :term:`filesystem encoding and error handler`. .. versionadded:: 3.2 .. c:function:: void PyErr_SyntaxLocation(const char *filename, int lineno) Like :c:func:`PyErr_SyntaxLocationEx`, but the *col_offset* parameter is omitted. .. c:function:: void PyErr_BadInternalCall() This is a shorthand for ``PyErr_SetString(PyExc_SystemError, message)``, where *message* indicates that an internal operation (e.g. a Python/C API function) was invoked with an illegal argument. It is mostly for internal use. .. c:function:: PyObject *PyErr_ProgramTextObject(PyObject *filename, int lineno) Get the source line in *filename* at line *lineno*. *filename* should be a Python :class:`str` object. On success, this function returns a Python string object with the found line. On failure, this function returns ``NULL`` without an exception set. .. c:function:: PyObject *PyErr_ProgramText(const char *filename, int lineno) Similar to :c:func:`PyErr_ProgramTextObject`, but *filename* is a :c:expr:`const char *`, which is decoded with the :term:`filesystem encoding and error handler`, instead of a Python object reference. Issuing warnings ================ Use these functions to issue warnings from C code. They mirror similar functions exported by the Python :mod:`warnings` module. They normally print a warning message to *sys.stderr*; however, it is also possible that the user has specified that warnings are to be turned into errors, and in that case they will raise an exception. It is also possible that the functions raise an exception because of a problem with the warning machinery. The return value is ``0`` if no exception is raised, or ``-1`` if an exception is raised. (It is not possible to determine whether a warning message is actually printed, nor what the reason is for the exception; this is intentional.) If an exception is raised, the caller should do its normal exception handling (for example, :c:func:`Py_DECREF` owned references and return an error value). .. c:function:: int PyErr_WarnEx(PyObject *category, const char *message, Py_ssize_t stack_level) Issue a warning message. The *category* argument is a warning category (see below) or ``NULL``; the *message* argument is a UTF-8 encoded string. *stack_level* is a positive number giving a number of stack frames; the warning will be issued from the currently executing line of code in that stack frame. A *stack_level* of 1 is the function calling :c:func:`PyErr_WarnEx`, 2 is the function above that, and so forth. Warning categories must be subclasses of :c:data:`PyExc_Warning`; :c:data:`PyExc_Warning` is a subclass of :c:data:`PyExc_Exception`; the default warning category is :c:data:`PyExc_RuntimeWarning`. The standard Python warning categories are available as global variables whose names are enumerated at :ref:`standardwarningcategories`. For information about warning control, see the documentation for the :mod:`warnings` module and the :option:`-W` option in the command line documentation. There is no C API for warning control. .. c:function:: int PyErr_WarnExplicitObject(PyObject *category, PyObject *message, PyObject *filename, int lineno, PyObject *module, PyObject *registry) Issue a warning message with explicit control over all warning attributes. This is a straightforward wrapper around the Python function :func:`warnings.warn_explicit`; see there for more information. The *module* and *registry* arguments may be set to ``NULL`` to get the default effect described there. .. versionadded:: 3.4 .. c:function:: int PyErr_WarnExplicit(PyObject *category, const char *message, const char *filename, int lineno, const char *module, PyObject *registry) Similar to :c:func:`PyErr_WarnExplicitObject` except that *message* and *module* are UTF-8 encoded strings, and *filename* is decoded from the :term:`filesystem encoding and error handler`. .. c:function:: int PyErr_WarnFormat(PyObject *category, Py_ssize_t stack_level, const char *format, ...) Function similar to :c:func:`PyErr_WarnEx`, but uses :c:func:`PyUnicode_FromFormat` to format the warning message. *format* is an ASCII-encoded string. .. versionadded:: 3.2 .. c:function:: int PyErr_WarnExplicitFormat(PyObject *category, const char *filename, int lineno, const char *module, PyObject *registry, const char *format, ...) Similar to :c:func:`PyErr_WarnExplicit`, but uses :c:func:`PyUnicode_FromFormat` to format the warning message. *format* is an ASCII-encoded string. .. versionadded:: 3.2 .. c:function:: int PyErr_ResourceWarning(PyObject *source, Py_ssize_t stack_level, const char *format, ...) Function similar to :c:func:`PyErr_WarnFormat`, but *category* is :exc:`ResourceWarning` and it passes *source* to :class:`!warnings.WarningMessage`. .. versionadded:: 3.6 Querying the error indicator ============================ .. c:function:: PyObject* PyErr_Occurred() Test whether the error indicator is set. If set, return the exception *type* (the first argument to the last call to one of the ``PyErr_Set*`` functions or to :c:func:`PyErr_Restore`). If not set, return ``NULL``. You do not own a reference to the return value, so you do not need to :c:func:`Py_DECREF` it. The caller must have an :term:`attached thread state`. .. note:: Do not compare the return value to a specific exception; use :c:func:`PyErr_ExceptionMatches` instead, shown below. (The comparison could easily fail since the exception may be an instance instead of a class, in the case of a class exception, or it may be a subclass of the expected exception.) .. c:function:: int PyErr_ExceptionMatches(PyObject *exc) Equivalent to ``PyErr_GivenExceptionMatches(PyErr_Occurred(), exc)``. This should only be called when an exception is actually set; a memory access violation will occur if no exception has been raised. .. c:function:: int PyErr_GivenExceptionMatches(PyObject *given, PyObject *exc) Return true if the *given* exception matches the exception type in *exc*. If *exc* is a class object, this also returns true when *given* is an instance of a subclass. If *exc* is a tuple, all exception types in the tuple (and recursively in subtuples) are searched for a match. .. c:function:: PyObject *PyErr_GetRaisedException(void) Return the exception currently being raised, clearing the error indicator at the same time. Return ``NULL`` if the error indicator is not set. This function is used by code that needs to catch exceptions, or code that needs to save and restore the error indicator temporarily. For example:: { PyObject *exc = PyErr_GetRaisedException(); /* ... code that might produce other errors ... */ PyErr_SetRaisedException(exc); } .. seealso:: :c:func:`PyErr_GetHandledException`, to save the exception currently being handled. .. versionadded:: 3.12 .. c:function:: void PyErr_SetRaisedException(PyObject *exc) Set *exc* as the exception currently being raised, clearing the existing exception if one is set. If *exc* is ``NULL``, just clear the existing exception. *exc* must be a valid exception or ``NULL``. This call ":term:`steals `" a reference to *exc*. .. versionadded:: 3.12 .. c:function:: void PyErr_Fetch(PyObject **ptype, PyObject **pvalue, PyObject **ptraceback) .. deprecated:: 3.12 Use :c:func:`PyErr_GetRaisedException` instead. Retrieve the error indicator into three variables whose addresses are passed. If the error indicator is not set, set all three variables to ``NULL``. If it is set, it will be cleared and you own a reference to each object retrieved. The value and traceback object may be ``NULL`` even when the type object is not. .. note:: This function is normally only used by legacy code that needs to catch exceptions or save and restore the error indicator temporarily. For example:: { PyObject *type, *value, *traceback; PyErr_Fetch(&type, &value, &traceback); /* ... code that might produce other errors ... */ PyErr_Restore(type, value, traceback); } .. c:function:: void PyErr_Restore(PyObject *type, PyObject *value, PyObject *traceback) .. deprecated:: 3.12 Use :c:func:`PyErr_SetRaisedException` instead. Set the error indicator from the three objects, *type*, *value*, and *traceback*, clearing the existing exception if one is set. If the objects are ``NULL``, the error indicator is cleared. Do not pass a ``NULL`` type and non-``NULL`` value or traceback. The exception type should be a class. Do not pass an invalid exception type or value. (Violating these rules will cause subtle problems later.) This call takes away a reference to each object: you must own a reference to each object before the call and after the call you no longer own these references. (If you don't understand this, don't use this function. I warned you.) .. note:: This function is normally only used by legacy code that needs to save and restore the error indicator temporarily. Use :c:func:`PyErr_Fetch` to save the current error indicator. .. c:function:: void PyErr_NormalizeException(PyObject **exc, PyObject **val, PyObject **tb) .. deprecated:: 3.12 Use :c:func:`PyErr_GetRaisedException` instead, to avoid any possible de-normalization. Under certain circumstances, the values returned by :c:func:`PyErr_Fetch` below can be "unnormalized", meaning that ``*exc`` is a class object but ``*val`` is not an instance of the same class. This function can be used to instantiate the class in that case. If the values are already normalized, nothing happens. The delayed normalization is implemented to improve performance. .. note:: This function *does not* implicitly set the :attr:`~BaseException.__traceback__` attribute on the exception value. If setting the traceback appropriately is desired, the following additional snippet is needed:: if (tb != NULL) { PyException_SetTraceback(val, tb); } .. c:function:: PyObject* PyErr_GetHandledException(void) Retrieve the active exception instance, as would be returned by :func:`sys.exception`. This refers to an exception that was *already caught*, not to an exception that was freshly raised. Returns a new reference to the exception or ``NULL``. Does not modify the interpreter's exception state. .. note:: This function is not normally used by code that wants to handle exceptions. Rather, it can be used when code needs to save and restore the exception state temporarily. Use :c:func:`PyErr_SetHandledException` to restore or clear the exception state. .. versionadded:: 3.11 .. c:function:: void PyErr_SetHandledException(PyObject *exc) Set the active exception, as known from ``sys.exception()``. This refers to an exception that was *already caught*, not to an exception that was freshly raised. To clear the exception state, pass ``NULL``. .. note:: This function is not normally used by code that wants to handle exceptions. Rather, it can be used when code needs to save and restore the exception state temporarily. Use :c:func:`PyErr_GetHandledException` to get the exception state. .. versionadded:: 3.11 .. c:function:: void PyErr_GetExcInfo(PyObject **ptype, PyObject **pvalue, PyObject **ptraceback) Retrieve the old-style representation of the exception info, as known from :func:`sys.exc_info`. This refers to an exception that was *already caught*, not to an exception that was freshly raised. Returns new references for the three objects, any of which may be ``NULL``. Does not modify the exception info state. This function is kept for backwards compatibility. Prefer using :c:func:`PyErr_GetHandledException`. .. note:: This function is not normally used by code that wants to handle exceptions. Rather, it can be used when code needs to save and restore the exception state temporarily. Use :c:func:`PyErr_SetExcInfo` to restore or clear the exception state. .. versionadded:: 3.3 .. c:function:: void PyErr_SetExcInfo(PyObject *type, PyObject *value, PyObject *traceback) Set the exception info, as known from ``sys.exc_info()``. This refers to an exception that was *already caught*, not to an exception that was freshly raised. This function ":term:`steals `" the references of the arguments. To clear the exception state, pass ``NULL`` for all three arguments. This function is kept for backwards compatibility. Prefer using :c:func:`PyErr_SetHandledException`. .. note:: This function is not normally used by code that wants to handle exceptions. Rather, it can be used when code needs to save and restore the exception state temporarily. Use :c:func:`PyErr_GetExcInfo` to read the exception state. .. versionadded:: 3.3 .. versionchanged:: 3.11 The ``type`` and ``traceback`` arguments are no longer used and can be NULL. The interpreter now derives them from the exception instance (the ``value`` argument). The function still ":term:`steals `" references of all three arguments. Signal Handling =============== .. c:function:: int PyErr_CheckSignals() .. index:: pair: module; signal single: SIGINT (C macro) single: KeyboardInterrupt (built-in exception) Handle external interruptions, such as signals or activating a debugger, whose processing has been delayed until it is safe to run Python code and/or raise exceptions. For example, pressing :kbd:`Ctrl-C` causes a terminal to send the :py:data:`signal.SIGINT` signal. This function executes the corresponding Python signal handler, which, by default, raises the :exc:`KeyboardInterrupt` exception. :c:func:`!PyErr_CheckSignals` should be called by long-running C code frequently enough so that the response appears immediate to humans. Handlers invoked by this function currently include: - Signal handlers, including Python functions registered using the :mod:`signal` module. Signal handlers are only run in the main thread of the main interpreter. (This is where the function got the name: originally, signals were the only way to interrupt the interpreter.) - Running the garbage collector, if necessary. - Executing a pending :ref:`remote debugger ` script. - Raise the exception set by :c:func:`PyThreadState_SetAsyncExc`. If any handler raises an exception, immediately return ``-1`` with that exception set. Any remaining interruptions are left to be processed on the next :c:func:`PyErr_CheckSignals()` invocation, if appropriate. If all handlers finish successfully, or there are no handlers to run, return ``0``. .. versionchanged:: 3.12 This function may now invoke the garbage collector. .. versionchanged:: 3.14 This function may now execute a remote debugger script, if remote debugging is enabled. .. versionchanged:: 3.15 The exception set by :c:func:`PyThreadState_SetAsyncExc` is now raised. .. c:function:: void PyErr_SetInterrupt() .. index:: pair: module; signal single: SIGINT (C macro) single: KeyboardInterrupt (built-in exception) Simulate the effect of a :c:macro:`!SIGINT` signal arriving. This is equivalent to ``PyErr_SetInterruptEx(SIGINT)``. .. note:: This function is async-signal-safe. It can be called without an :term:`attached thread state` and from a C signal handler. .. c:function:: int PyErr_SetInterruptEx(int signum) .. index:: pair: module; signal single: KeyboardInterrupt (built-in exception) Simulate the effect of a signal arriving. The next time :c:func:`PyErr_CheckSignals` is called, the Python signal handler for the given signal number will be called. This function can be called by C code that sets up its own signal handling and wants Python signal handlers to be invoked as expected when an interruption is requested (for example when the user presses Ctrl-C to interrupt an operation). If the given signal isn't handled by Python (it was set to :py:const:`signal.SIG_DFL` or :py:const:`signal.SIG_IGN`), it will be ignored. If *signum* is outside of the allowed range of signal numbers, ``-1`` is returned. Otherwise, ``0`` is returned. The error indicator is never changed by this function. .. note:: This function is async-signal-safe. It can be called without an :term:`attached thread state` and from a C signal handler. .. versionadded:: 3.10 .. c:function:: int PySignal_SetWakeupFd(int fd) This utility function specifies a file descriptor to which the signal number is written as a single byte whenever a signal is received. *fd* must be non-blocking. It returns the previous such file descriptor. The value ``-1`` disables the feature; this is the initial state. This is equivalent to :func:`signal.set_wakeup_fd` in Python, but without any error checking. *fd* should be a valid file descriptor. The function should only be called from the main thread. .. versionchanged:: 3.5 On Windows, the function now also supports socket handles. Exception Classes ================= .. c:function:: PyObject* PyErr_NewException(const char *name, PyObject *base, PyObject *dict) This utility function creates and returns a new exception class. The *name* argument must be the name of the new exception, a C string of the form ``module.classname``. The *base* and *dict* arguments are normally ``NULL``. This creates a class object derived from :exc:`Exception` (accessible in C as :c:data:`PyExc_Exception`). The :attr:`~type.__module__` attribute of the new class is set to the first part (up to the last dot) of the *name* argument, and the class name is set to the last part (after the last dot). The *base* argument can be used to specify alternate base classes; it can either be only one class or a tuple of classes. The *dict* argument can be used to specify a dictionary of class variables and methods. .. c:function:: PyObject* PyErr_NewExceptionWithDoc(const char *name, const char *doc, PyObject *base, PyObject *dict) Same as :c:func:`PyErr_NewException`, except that the new exception class can easily be given a docstring: If *doc* is non-``NULL``, it will be used as the docstring for the exception class. .. versionadded:: 3.2 .. c:function:: int PyExceptionClass_Check(PyObject *ob) Return non-zero if *ob* is an exception class, zero otherwise. This function always succeeds. .. c:function:: const char *PyExceptionClass_Name(PyObject *ob) Return :c:member:`~PyTypeObject.tp_name` of the exception class *ob*. .. c:macro:: PyException_HEAD This is a macro including the base fields for an exception object. This was included in Python's C API by mistake and is not designed for use in extensions. For creating custom exception objects, use :c:func:`PyErr_NewException` or otherwise create a class inheriting from :c:data:`PyExc_BaseException`. .. soft-deprecated:: 3.15 Exception Objects ================= .. c:function:: int PyExceptionInstance_Check(PyObject *op) Return true if *op* is an instance of :class:`BaseException`, false otherwise. This function always succeeds. .. c:macro:: PyExceptionInstance_Class(op) Equivalent to :c:func:`Py_TYPE(op) `. .. c:function:: PyObject* PyException_GetTraceback(PyObject *ex) Return the traceback associated with the exception as a new reference, as accessible from Python through the :attr:`~BaseException.__traceback__` attribute. If there is no traceback associated, this returns ``NULL``. .. c:function:: int PyException_SetTraceback(PyObject *ex, PyObject *tb) Set the traceback associated with the exception to *tb*. Use ``Py_None`` to clear it. .. c:function:: PyObject* PyException_GetContext(PyObject *ex) Return the context (another exception instance during whose handling *ex* was raised) associated with the exception as a new reference, as accessible from Python through the :attr:`~BaseException.__context__` attribute. If there is no context associated, this returns ``NULL``. .. c:function:: void PyException_SetContext(PyObject *ex, PyObject *ctx) Set the context associated with the exception to *ctx*. Use ``NULL`` to clear it. There is no type check to make sure that *ctx* is an exception instance. This ":term:`steals `" a reference to *ctx*. .. c:function:: PyObject* PyException_GetCause(PyObject *ex) Return the cause (either an exception instance, or ``None``, set by ``raise ... from ...``) associated with the exception as a new reference, as accessible from Python through the :attr:`~BaseException.__cause__` attribute. .. c:function:: void PyException_SetCause(PyObject *ex, PyObject *cause) Set the cause associated with the exception to *cause*. Use ``NULL`` to clear it. There is no type check to make sure that *cause* is either an exception instance or ``None``. This ":term:`steals `" a reference to *cause*. The :attr:`~BaseException.__suppress_context__` attribute is implicitly set to ``True`` by this function. .. c:function:: PyObject* PyException_GetArgs(PyObject *ex) Return :attr:`~BaseException.args` of exception *ex*. .. c:function:: void PyException_SetArgs(PyObject *ex, PyObject *args) Set :attr:`~BaseException.args` of exception *ex* to *args*. .. c:function:: PyObject* PyUnstable_Exc_PrepReraiseStar(PyObject *orig, PyObject *excs) Implement part of the interpreter's implementation of :keyword:`!except*`. *orig* is the original exception that was caught, and *excs* is the list of the exceptions that need to be raised. This list contains the unhandled part of *orig*, if any, as well as the exceptions that were raised from the :keyword:`!except*` clauses (so they have a different traceback from *orig*) and those that were reraised (and have the same traceback as *orig*). Return the :exc:`ExceptionGroup` that needs to be reraised in the end, or ``None`` if there is nothing to reraise. .. versionadded:: 3.12 .. _unicodeexceptions: Unicode Exception Objects ========================= The following functions are used to create and modify Unicode exceptions from C. .. c:function:: PyObject* PyUnicodeDecodeError_Create(const char *encoding, const char *object, Py_ssize_t length, Py_ssize_t start, Py_ssize_t end, const char *reason) Create a :class:`UnicodeDecodeError` object with the attributes *encoding*, *object*, *length*, *start*, *end* and *reason*. *encoding* and *reason* are UTF-8 encoded strings. .. c:function:: PyObject* PyUnicodeDecodeError_GetEncoding(PyObject *exc) PyObject* PyUnicodeEncodeError_GetEncoding(PyObject *exc) Return the *encoding* attribute of the given exception object. .. c:function:: PyObject* PyUnicodeDecodeError_GetObject(PyObject *exc) PyObject* PyUnicodeEncodeError_GetObject(PyObject *exc) PyObject* PyUnicodeTranslateError_GetObject(PyObject *exc) Return the *object* attribute of the given exception object. .. c:function:: int PyUnicodeDecodeError_GetStart(PyObject *exc, Py_ssize_t *start) int PyUnicodeEncodeError_GetStart(PyObject *exc, Py_ssize_t *start) int PyUnicodeTranslateError_GetStart(PyObject *exc, Py_ssize_t *start) Get the *start* attribute of the given exception object and place it into *\*start*. *start* must not be ``NULL``. Return ``0`` on success, ``-1`` on failure. If the :attr:`UnicodeError.object` is an empty sequence, the resulting *start* is ``0``. Otherwise, it is clipped to ``[0, len(object) - 1]``. .. seealso:: :attr:`UnicodeError.start` .. c:function:: int PyUnicodeDecodeError_SetStart(PyObject *exc, Py_ssize_t start) int PyUnicodeEncodeError_SetStart(PyObject *exc, Py_ssize_t start) int PyUnicodeTranslateError_SetStart(PyObject *exc, Py_ssize_t start) Set the *start* attribute of the given exception object to *start*. Return ``0`` on success, ``-1`` on failure. .. note:: While passing a negative *start* does not raise an exception, the corresponding getters will not consider it as a relative offset. .. c:function:: int PyUnicodeDecodeError_GetEnd(PyObject *exc, Py_ssize_t *end) int PyUnicodeEncodeError_GetEnd(PyObject *exc, Py_ssize_t *end) int PyUnicodeTranslateError_GetEnd(PyObject *exc, Py_ssize_t *end) Get the *end* attribute of the given exception object and place it into *\*end*. *end* must not be ``NULL``. Return ``0`` on success, ``-1`` on failure. If the :attr:`UnicodeError.object` is an empty sequence, the resulting *end* is ``0``. Otherwise, it is clipped to ``[1, len(object)]``. .. c:function:: int PyUnicodeDecodeError_SetEnd(PyObject *exc, Py_ssize_t end) int PyUnicodeEncodeError_SetEnd(PyObject *exc, Py_ssize_t end) int PyUnicodeTranslateError_SetEnd(PyObject *exc, Py_ssize_t end) Set the *end* attribute of the given exception object to *end*. Return ``0`` on success, ``-1`` on failure. .. seealso:: :attr:`UnicodeError.end` .. c:function:: PyObject* PyUnicodeDecodeError_GetReason(PyObject *exc) PyObject* PyUnicodeEncodeError_GetReason(PyObject *exc) PyObject* PyUnicodeTranslateError_GetReason(PyObject *exc) Return the *reason* attribute of the given exception object. .. c:function:: int PyUnicodeDecodeError_SetReason(PyObject *exc, const char *reason) int PyUnicodeEncodeError_SetReason(PyObject *exc, const char *reason) int PyUnicodeTranslateError_SetReason(PyObject *exc, const char *reason) Set the *reason* attribute of the given exception object to *reason*. Return ``0`` on success, ``-1`` on failure. .. _recursion: Recursion Control ================= These two functions provide a way to perform safe recursive calls at the C level, both in the core and in extension modules. They are needed if the recursive code does not necessarily invoke Python code (which tracks its recursion depth automatically). They are also not needed for *tp_call* implementations because the :ref:`call protocol ` takes care of recursion handling. .. c:function:: int Py_EnterRecursiveCall(const char *where) Marks a point where a recursive C-level call is about to be performed. The function then checks if the stack limit is reached. If this is the case, a :exc:`RecursionError` is set and a nonzero value is returned. Otherwise, zero is returned. The check is based on the remaining C stack space of the current thread, not on a count of calls, so it is unaffected by :c:func:`Py_SetRecursionLimit` and :func:`sys.setrecursionlimit`. *where* should be a UTF-8 encoded string such as ``" in instance check"`` to be concatenated to the :exc:`RecursionError` message caused by the recursion depth limit. .. seealso:: The :c:func:`PyUnstable_ThreadState_SetStackProtection` function. .. versionchanged:: 3.9 This function is now also available in the :ref:`limited API `. .. versionchanged:: 3.14 The check is based on the remaining C stack space. Previously, a separate counter of C-level calls was used. .. c:function:: void Py_LeaveRecursiveCall(void) Ends a :c:func:`Py_EnterRecursiveCall`. Must be called once for each *successful* invocation of :c:func:`Py_EnterRecursiveCall`. .. versionchanged:: 3.9 This function is now also available in the :ref:`limited API `. Properly implementing :c:member:`~PyTypeObject.tp_repr` for container types requires special recursion handling. In addition to protecting the stack, :c:member:`~PyTypeObject.tp_repr` also needs to track objects to prevent cycles. The following two functions facilitate this functionality. Effectively, these are the C equivalent to :deco:`reprlib.recursive_repr`. .. c:function:: int Py_ReprEnter(PyObject *object) Called at the beginning of the :c:member:`~PyTypeObject.tp_repr` implementation to detect cycles. If the object has already been processed, the function returns a positive integer. In that case the :c:member:`~PyTypeObject.tp_repr` implementation should return a string object indicating a cycle. As examples, :class:`dict` objects return ``{...}`` and :class:`list` objects return ``[...]``. The function will return a negative integer if the recursion limit is reached. In that case the :c:member:`~PyTypeObject.tp_repr` implementation should typically return ``NULL``. Otherwise, the function returns zero and the :c:member:`~PyTypeObject.tp_repr` implementation can continue normally. .. c:function:: void Py_ReprLeave(PyObject *object) Ends a :c:func:`Py_ReprEnter`. Must be called once for each invocation of :c:func:`Py_ReprEnter` that returns zero. .. c:function:: int Py_GetRecursionLimit(void) Get the recursion limit for the current interpreter. It can be set with :c:func:`Py_SetRecursionLimit`. The recursion limit prevents the Python interpreter stack from growing infinitely. This function cannot fail, and the caller must hold an :term:`attached thread state`. .. seealso:: :py:func:`sys.getrecursionlimit` .. c:function:: void Py_SetRecursionLimit(int new_limit) Set the recursion limit for the current interpreter. This function cannot fail, and the caller must hold an :term:`attached thread state`. .. seealso:: :py:func:`sys.setrecursionlimit` .. _standardexceptions: Exception and warning types =========================== All standard Python exceptions and warning categories are available as global variables whose names are ``PyExc_`` followed by the Python exception name. These have the type :c:expr:`PyObject*`; they are all class objects. For completeness, here are all the variables: Exception types --------------- .. list-table:: :align: left :widths: auto :header-rows: 1 * * C name * Python name * * .. c:var:: PyObject *PyExc_BaseException * :exc:`BaseException` * * .. c:var:: PyObject *PyExc_BaseExceptionGroup * :exc:`BaseExceptionGroup` * * .. c:var:: PyObject *PyExc_Exception * :exc:`Exception` * * .. c:var:: PyObject *PyExc_ArithmeticError * :exc:`ArithmeticError` * * .. c:var:: PyObject *PyExc_AssertionError * :exc:`AssertionError` * * .. c:var:: PyObject *PyExc_AttributeError * :exc:`AttributeError` * * .. c:var:: PyObject *PyExc_BlockingIOError * :exc:`BlockingIOError` * * .. c:var:: PyObject *PyExc_BrokenPipeError * :exc:`BrokenPipeError` * * .. c:var:: PyObject *PyExc_BufferError * :exc:`BufferError` * * .. c:var:: PyObject *PyExc_ChildProcessError * :exc:`ChildProcessError` * * .. c:var:: PyObject *PyExc_ConnectionAbortedError * :exc:`ConnectionAbortedError` * * .. c:var:: PyObject *PyExc_ConnectionError * :exc:`ConnectionError` * * .. c:var:: PyObject *PyExc_ConnectionRefusedError * :exc:`ConnectionRefusedError` * * .. c:var:: PyObject *PyExc_ConnectionResetError * :exc:`ConnectionResetError` * * .. c:var:: PyObject *PyExc_EOFError * :exc:`EOFError` * * .. c:var:: PyObject *PyExc_FileExistsError * :exc:`FileExistsError` * * .. c:var:: PyObject *PyExc_FileNotFoundError * :exc:`FileNotFoundError` * * .. c:var:: PyObject *PyExc_FloatingPointError * :exc:`FloatingPointError` * * .. c:var:: PyObject *PyExc_GeneratorExit * :exc:`GeneratorExit` * * .. c:var:: PyObject *PyExc_ImportCycleError * :exc:`ImportCycleError` * * .. c:var:: PyObject *PyExc_ImportError * :exc:`ImportError` * * .. c:var:: PyObject *PyExc_IndentationError * :exc:`IndentationError` * * .. c:var:: PyObject *PyExc_IndexError * :exc:`IndexError` * * .. c:var:: PyObject *PyExc_InterruptedError * :exc:`InterruptedError` * * .. c:var:: PyObject *PyExc_IsADirectoryError * :exc:`IsADirectoryError` * * .. c:var:: PyObject *PyExc_KeyError * :exc:`KeyError` * * .. c:var:: PyObject *PyExc_KeyboardInterrupt * :exc:`KeyboardInterrupt` * * .. c:var:: PyObject *PyExc_LookupError * :exc:`LookupError` * * .. c:var:: PyObject *PyExc_MemoryError * :exc:`MemoryError` * * .. c:var:: PyObject *PyExc_ModuleNotFoundError * :exc:`ModuleNotFoundError` * * .. c:var:: PyObject *PyExc_NameError * :exc:`NameError` * * .. c:var:: PyObject *PyExc_NotADirectoryError * :exc:`NotADirectoryError` * * .. c:var:: PyObject *PyExc_NotImplementedError * :exc:`NotImplementedError` * * .. c:var:: PyObject *PyExc_OSError * :exc:`OSError` * * .. c:var:: PyObject *PyExc_OverflowError * :exc:`OverflowError` * * .. c:var:: PyObject *PyExc_PermissionError * :exc:`PermissionError` * * .. c:var:: PyObject *PyExc_ProcessLookupError * :exc:`ProcessLookupError` * * .. c:var:: PyObject *PyExc_PythonFinalizationError * :exc:`PythonFinalizationError` * * .. c:var:: PyObject *PyExc_RecursionError * :exc:`RecursionError` * * .. c:var:: PyObject *PyExc_ReferenceError * :exc:`ReferenceError` * * .. c:var:: PyObject *PyExc_RuntimeError * :exc:`RuntimeError` * * .. c:var:: PyObject *PyExc_StopAsyncIteration * :exc:`StopAsyncIteration` * * .. c:var:: PyObject *PyExc_StopIteration * :exc:`StopIteration` * * .. c:var:: PyObject *PyExc_SyntaxError * :exc:`SyntaxError` * * .. c:var:: PyObject *PyExc_SystemError * :exc:`SystemError` * * .. c:var:: PyObject *PyExc_SystemExit * :exc:`SystemExit` * * .. c:var:: PyObject *PyExc_TabError * :exc:`TabError` * * .. c:var:: PyObject *PyExc_TimeoutError * :exc:`TimeoutError` * * .. c:var:: PyObject *PyExc_TypeError * :exc:`TypeError` * * .. c:var:: PyObject *PyExc_UnboundLocalError * :exc:`UnboundLocalError` * * .. c:var:: PyObject *PyExc_UnicodeDecodeError * :exc:`UnicodeDecodeError` * * .. c:var:: PyObject *PyExc_UnicodeEncodeError * :exc:`UnicodeEncodeError` * * .. c:var:: PyObject *PyExc_UnicodeError * :exc:`UnicodeError` * * .. c:var:: PyObject *PyExc_UnicodeTranslateError * :exc:`UnicodeTranslateError` * * .. c:var:: PyObject *PyExc_ValueError * :exc:`ValueError` * * .. c:var:: PyObject *PyExc_ZeroDivisionError * :exc:`ZeroDivisionError` .. versionadded:: 3.3 :c:data:`PyExc_BlockingIOError`, :c:data:`PyExc_BrokenPipeError`, :c:data:`PyExc_ChildProcessError`, :c:data:`PyExc_ConnectionError`, :c:data:`PyExc_ConnectionAbortedError`, :c:data:`PyExc_ConnectionRefusedError`, :c:data:`PyExc_ConnectionResetError`, :c:data:`PyExc_FileExistsError`, :c:data:`PyExc_FileNotFoundError`, :c:data:`PyExc_InterruptedError`, :c:data:`PyExc_IsADirectoryError`, :c:data:`PyExc_NotADirectoryError`, :c:data:`PyExc_PermissionError`, :c:data:`PyExc_ProcessLookupError` and :c:data:`PyExc_TimeoutError` were introduced following :pep:`3151`. .. versionadded:: 3.5 :c:data:`PyExc_StopAsyncIteration` and :c:data:`PyExc_RecursionError`. .. versionadded:: 3.6 :c:data:`PyExc_ModuleNotFoundError`. .. versionadded:: 3.11 :c:data:`PyExc_BaseExceptionGroup`. OSError aliases --------------- The following are a compatibility aliases to :c:data:`PyExc_OSError`. .. versionchanged:: 3.3 These aliases used to be separate exception types. .. list-table:: :align: left :widths: auto :header-rows: 1 * * C name * Python name * Notes * * .. c:var:: PyObject *PyExc_EnvironmentError * :exc:`OSError` * * * .. c:var:: PyObject *PyExc_IOError * :exc:`OSError` * * * .. c:var:: PyObject *PyExc_WindowsError * :exc:`OSError` * [win]_ Notes: .. [win] :c:var:`!PyExc_WindowsError` is only defined on Windows; protect code that uses this by testing that the preprocessor macro ``MS_WINDOWS`` is defined. .. _standardwarningcategories: Warning types ------------- .. list-table:: :align: left :widths: auto :header-rows: 1 * * C name * Python name * * .. c:var:: PyObject *PyExc_Warning * :exc:`Warning` * * .. c:var:: PyObject *PyExc_BytesWarning * :exc:`BytesWarning` * * .. c:var:: PyObject *PyExc_DeprecationWarning * :exc:`DeprecationWarning` * * .. c:var:: PyObject *PyExc_EncodingWarning * :exc:`EncodingWarning` * * .. c:var:: PyObject *PyExc_FutureWarning * :exc:`FutureWarning` * * .. c:var:: PyObject *PyExc_ImportWarning * :exc:`ImportWarning` * * .. c:var:: PyObject *PyExc_PendingDeprecationWarning * :exc:`PendingDeprecationWarning` * * .. c:var:: PyObject *PyExc_ResourceWarning * :exc:`ResourceWarning` * * .. c:var:: PyObject *PyExc_RuntimeWarning * :exc:`RuntimeWarning` * * .. c:var:: PyObject *PyExc_SyntaxWarning * :exc:`SyntaxWarning` * * .. c:var:: PyObject *PyExc_UnicodeWarning * :exc:`UnicodeWarning` * * .. c:var:: PyObject *PyExc_UserWarning * :exc:`UserWarning` .. versionadded:: 3.2 :c:data:`PyExc_ResourceWarning`. .. versionadded:: 3.10 :c:data:`PyExc_EncodingWarning`. Tracebacks ========== .. c:var:: PyTypeObject PyTraceBack_Type Type object for traceback objects. This is available as :class:`types.TracebackType` in the Python layer. .. c:function:: int PyTraceBack_Check(PyObject *op) Return true if *op* is a traceback object, false otherwise. This function does not account for subtypes. .. c:function:: int PyTraceBack_Here(PyFrameObject *f) Replace the :attr:`~BaseException.__traceback__` attribute on the current exception with a new traceback prepending *f* to the existing chain. Calling this function without an exception set is undefined behavior. This function returns ``0`` on success, and returns ``-1`` with an exception set on failure. .. c:function:: int PyTraceBack_Print(PyObject *tb, PyObject *f) Write the traceback *tb* into the file *f*. This function returns ``0`` on success, and returns ``-1`` with an exception set on failure. .. c:function:: const char* PyUnstable_DumpTraceback(int fd, PyThreadState *tstate) Write a trace of the Python stack in *tstate* into the file *fd*. The format looks like:: Traceback (most recent call first): File "xxx", line xxx in File "xxx", line xxx in ... File "xxx", line xxx in This function is meant to debug situations such as segfaults, fatal errors, and similar. The file and function names it outputs are encoded to ASCII with backslashreplace and truncated to 500 characters. It writes only the first 100 frames; further frames are truncated with the line ``...``. This function will return ``NULL`` on success, or an error message on error. This function is intended for use in crash scenarios such as signal handlers for SIGSEGV, where the interpreter may be in an inconsistent state. Given that it reads interpreter data structures that may be partially modified, the function might produce incomplete output or it may even crash itself. The caller does not need to hold an :term:`attached thread state`, nor does *tstate* need to be attached. .. versionadded:: 3.15 .. c:function:: const char* PyUnstable_DumpTracebackThreads(int fd, PyInterpreterState *interp, PyThreadState *current_tstate, Py_ssize_t max_threads) Write the traces of all Python threads in *interp* into the file *fd*. If *interp* is ``NULL`` then this function will try to identify the current interpreter using thread-specific storage. If it cannot, it will return an error. If *current_tstate* is not ``NULL`` then it will be used to identify what the current thread is in the written output. If it is ``NULL`` then this function will identify the current thread using thread-specific storage. It is not an error if the function is unable to get the current Python thread state. This function will return ``NULL`` on success, or an error message on error. This function is meant to debug situations such as segfaults, fatal errors, and similar. It calls :c:func:`PyUnstable_DumpTraceback` for each thread. It only writes the tracebacks of the first *max_threads* threads, further output is truncated with the line ``...``. If *max_threads* is 0, the function will use a default value of 100 for the argument. This function is intended for use in crash scenarios such as signal handlers for SIGSEGV, where the interpreter may be in an inconsistent state. Given that it reads interpreter data structures that may be partially modified, the function might produce incomplete output or it may even crash itself. The caller does not need to hold an :term:`attached thread state`, nor does *current_tstate* need to be attached. .. warning:: On the :term:`free-threaded build`, this function is not thread-safe. If another thread deletes its :term:`thread state` while this function is being called, the process will likely crash. .. versionadded:: 3.15 --- ### Doc C Api Extension Modules (`Doc/c-api/extension-modules.rst`) .. highlight:: c .. _extension-modules: Defining extension modules -------------------------- A C extension for CPython is a shared library (for example, a ``.so`` file on Linux, ``.pyd`` DLL on Windows), which is loadable into the Python process (for example, it is compiled with compatible compiler settings), and which exports an :dfn:`export hook` function (or an old-style :ref:`initialization function `). To be importable by default (that is, by :py:class:`importlib.machinery.ExtensionFileLoader`), the shared library must be available on :py:attr:`sys.path`, and must be named after the module name plus an extension listed in :py:attr:`importlib.machinery.EXTENSION_SUFFIXES`. .. note:: Building, packaging and distributing extension modules is best done with third-party tools, and is out of scope of this document. One suitable tool is Setuptools, whose documentation can be found at https://setuptools.pypa.io/en/latest/setuptools.html. .. _extension-export-hook: Extension export hook ..................... .. versionadded:: 3.15 Support for the :samp:`PyModExport_{}` export hook was added in Python 3.15. The older way of defining modules is still available: consult either the :ref:`extension-pyinit` section or earlier versions of this documentation if you plan to support earlier Python versions. The export hook must be an exported function with the following signature: .. c:function:: PySlot *PyModExport_modulename(void) For modules with ASCII-only names, the :ref:`export hook ` must be named :samp:`PyModExport_{}`, with ```` replaced by the module's name. For non-ASCII module names, the export hook must instead be named :samp:`PyModExportU_{}` (note the ``U``), with ```` encoded using Python's *punycode* encoding with hyphens replaced by underscores. In Python: .. code-block:: python def hook_name(name): try: suffix = b'_' + name.encode('ascii') except UnicodeEncodeError: suffix = b'U_' + name.encode('punycode').replace(b'-', b'_') return b'PyModExport' + suffix The export hook returns an array of :c:type:`PySlot` entries, terminated by an entry with a slot ID of ``0``. These slots describe how the module should be created and initialized. This array must remain valid and constant until interpreter shutdown. Typically, it should use ``static`` storage. Prefer using the :c:macro:`Py_mod_create` and :c:macro:`Py_mod_exec` slots for any dynamic behavior. The export hook may return ``NULL`` with an exception set to signal failure. It is recommended to define the export hook function using a helper macro: .. c:macro:: PyMODEXPORT_FUNC Declare an extension module export hook. This macro: * specifies the :c:expr:`PySlot*` return type, * adds any special linkage declarations required by the platform, and * for C++, declares the function as ``extern "C"``. For example, a module called ``spam`` would be defined like this:: PyABIInfo_VAR(abi_info); static PySlot spam_slots[] = { PySlot_STATIC_DATA(Py_mod_abi, &abi_info), PySlot_STATIC_DATA(Py_mod_name, "spam"), PySlot_FUNC(Py_mod_init, spam_init_function), ... PySlot_END }; PyMODEXPORT_FUNC PyModExport_spam(void) { return spam_slots; } The export hook is typically the only non-\ ``static`` item defined in the module's C source. .. _pymodexport-api-caveats: The hook should be kept short. If it does more than ``return`` a static array, several caveats apply: - If you need to use any Python C API, it is recommended to call :c:func:`PyABIInfo_Check` first to raise an exception, rather than crash, in common cases of ABI mismatch. - Code in the export hook must never rely on the :term:`GIL`: :term:`free-threaded builds ` of Python can only check the :c:macro:`Py_mod_gil` slot (or the lack of it) after the hook returns, - Similarly, the hook may be called in any subinterpreter, since the :c:macro:`Py_mod_multiple_interpreters` slot (or lack of it) is only checked after the hook returns. For example:: PyMODEXPORT_FUNC PyModExport_modulename(void) { if (PyABIInfo_Check(&abi_info, "modulename") < 0) { /* ABI mismatch. It's not safe to examine the raised exception. */ return NULL; } /* use Python API (as little as possible); don't rely on GIL */ return modulename_slots; } .. note:: It is possible to export multiple modules from a single shared library by defining multiple export hooks. However, importing them requires a custom importer or suitably named copies/links of the extension file, because Python's import machinery only finds the function corresponding to the filename. See the `Multiple modules in one library `__ section in :pep:`489` for details. .. _multi-phase-initialization: Multi-phase initialization .......................... The process of creating an extension module follows several phases: - Python finds and calls the export hook to get information on how to create the module. - Before any substantial code is executed, Python can determine which capabilities the module supports, and it can adjust the environment or refuse loading an incompatible extension. Slots like :c:data:`Py_mod_abi`, :c:data:`Py_mod_gil` and :c:data:`Py_mod_multiple_interpreters` influence this step. - By default, Python itself then creates the module object -- that is, it does the equivalent of calling :py:meth:`~object.__new__` when creating an object. This step can be overridden using the :c:data:`Py_mod_create` slot. - Python sets initial module attributes like :attr:`~module.__package__` and :attr:`~module.__loader__`, and inserts the module object into :py:attr:`sys.modules`. - Afterwards, the module object is initialized in an extension-specific way -- the equivalent of :py:meth:`~object.__init__` when creating an object, or of executing top-level code in a Python-language module. The behavior is specified using the :c:data:`Py_mod_exec` slot. This is called *multi-phase initialization* to distinguish it from the legacy (but still supported) :ref:`single-phase initialization `, where an initialization function returns a fully constructed module. .. versionchanged:: 3.5 Added support for multi-phase initialization (:pep:`489`). Multiple module instances ......................... By default, extension modules are not singletons. For example, if the :py:attr:`sys.modules` entry is removed and the module is re-imported, a new module object is created and, typically, populated with fresh method and type objects. The old module is subject to normal garbage collection. This mirrors the behavior of pure-Python modules. Additional module instances may be created in :ref:`sub-interpreters ` or after Python runtime reinitialization (:c:func:`Py_Finalize` and :c:func:`Py_Initialize`). In these cases, sharing Python objects between module instances would likely cause crashes or undefined behavior. To avoid such issues, each instance of an extension module should be *isolated*: changes to one instance should not implicitly affect the others, and all state owned by the module, including references to Python objects, should be specific to a particular module instance. See :ref:`isolating-extensions-howto` for more details and a practical guide. A simpler way to avoid these issues is :ref:`raising an error on repeated initialization `. All modules are expected to support :ref:`sub-interpreters `, or otherwise explicitly signal a lack of support. This is usually achieved by isolation or blocking repeated initialization, as above. A module may also be limited to the main interpreter using the :c:data:`Py_mod_multiple_interpreters` slot. .. _extension-pyinit: ``PyInit`` function ................... .. soft-deprecated:: 3.15 This functionality will not get new features, but there are no plans to remove it. Instead of :c:func:`PyModExport_modulename`, an extension module can define an older-style :dfn:`initialization function` with the signature: .. c:function:: PyObject* PyInit_modulename(void) Its name should be :samp:`PyInit_{}`, with ```` replaced by the name of the module. For non-ASCII module names, use :samp:`PyInitU_{}` instead, with ```` encoded in the same way as for the :ref:`export hook ` (that is, using Punycode with underscores). If a module exports both :samp:`PyInit_{}` and :samp:`PyModExport_{}`, the :samp:`PyInit_{}` function is ignored. Like with :c:macro:`PyMODEXPORT_FUNC`, it is recommended to define the initialization function using a helper macro: .. c:macro:: PyMODINIT_FUNC Declare an extension module initialization function. This macro: * specifies the :c:expr:`PyObject*` return type, * adds any special linkage declarations required by the platform, and * for C++, declares the function as ``extern "C"``. Normally, the initialization function (``PyInit_modulename``) returns a :c:type:`PyModuleDef` instance with non-``NULL`` :c:member:`~PyModuleDef.m_slots`. This allows Python to use :ref:`multi-phase initialization `. Before it is returned, the ``PyModuleDef`` instance must be initialized using the following function: .. c:function:: PyObject* PyModuleDef_Init(PyModuleDef *def) Ensure a module definition is a properly initialized Python object that correctly reports its type and a reference count. Return *def* cast to ``PyObject*``, or ``NULL`` if an error occurred. Calling this function is required before returning a :c:type:`PyModuleDef` from a module initialization function. It should not be used in other contexts. Note that Python assumes that ``PyModuleDef`` structures are statically allocated. This function may return either a new reference or a borrowed one; this reference must not be released. .. versionadded:: 3.5 For example, a module called ``spam`` would be defined like this:: static struct PyModuleDef spam_module = { .m_base = PyModuleDef_HEAD_INIT, .m_name = "spam", ... }; PyMODINIT_FUNC PyInit_spam(void) { return PyModuleDef_Init(&spam_module); } .. _single-phase-initialization: Legacy single-phase initialization ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ .. soft-deprecated:: 3.15 Single-phase initialization is a legacy mechanism to initialize extension modules, with known drawbacks and design flaws. Extension module authors are encouraged to use multi-phase initialization instead. However, there are no plans to remove support for it. In single-phase initialization, the old-style :ref:`initialization function ` (``PyInit_modulename``) should create, populate and return a module object. This is typically done using :c:func:`PyModule_Create` and functions like :c:func:`PyModule_AddObjectRef`. Single-phase initialization differs from the :ref:`default ` in the following ways: * Single-phase modules are, or rather *contain*, “singletons”. When the module is first initialized, Python saves the contents of the module's ``__dict__`` (that is, typically, the module's functions and types). For subsequent imports, Python does not call the initialization function again. Instead, it creates a new module object with a new ``__dict__``, and copies the saved contents to it. For example, given a single-phase module ``_testsinglephase`` [#testsinglephase]_ that defines a function ``sum`` and an exception class ``error``: .. code-block:: python >>> import sys >>> import _testsinglephase as one >>> del sys.modules['_testsinglephase'] >>> import _testsinglephase as two >>> one is two False >>> one.__dict__ is two.__dict__ False >>> one.sum is two.sum True >>> one.error is two.error True The exact behavior should be considered a CPython implementation detail. * To work around the fact that ``PyInit_modulename`` does not take a *spec* argument, some state of the import machinery is saved and applied to the first suitable module created during the ``PyInit_modulename`` call. Specifically, when a sub-module is imported, this mechanism prepends the parent package name to the name of the module. A single-phase ``PyInit_modulename`` function should create “its” module object as soon as possible, before any other module objects can be created. * Non-ASCII module names (``PyInitU_modulename``) are not supported. * Single-phase modules support module lookup functions like :c:func:`PyState_FindModule`. * The module's :c:member:`PyModuleDef.m_slots` must be NULL. .. [#testsinglephase] ``_testsinglephase`` is an internal module used in CPython's self-test suite; your installation may or may not include it. --- ### Doc C Api File (`Doc/c-api/file.rst`) .. highlight:: c .. _fileobjects: File objects ------------ .. index:: pair: object; file These APIs are a minimal emulation of the Python 2 C API for built-in file objects, which used to rely on the buffered I/O (:c:expr:`FILE*`) support from the C standard library. In Python 3, files and streams use the new :mod:`io` module, which defines several layers over the low-level unbuffered I/O of the operating system. The functions described below are convenience C wrappers over these new APIs, and meant mostly for internal error reporting in the interpreter; third-party code is advised to access the :mod:`io` APIs instead. .. c:function:: PyObject* PyFile_FromFd(int fd, const char *name, const char *mode, int buffering, const char *encoding, const char *errors, const char *newline, int closefd) Create a Python file object from the file descriptor of an already opened file *fd*. The arguments *name*, *encoding*, *errors* and *newline* can be ``NULL`` to use the defaults; *buffering* can be *-1* to use the default. *name* is ignored and kept for backward compatibility. Return ``NULL`` on failure. For a more comprehensive description of the arguments, please refer to the :func:`io.open` function documentation. .. warning:: Since Python streams have their own buffering layer, mixing them with OS-level file descriptors can produce various issues (such as unexpected ordering of data). .. versionchanged:: 3.2 Ignore *name* attribute. .. c:function:: int PyObject_AsFileDescriptor(PyObject *p) Return the file descriptor associated with *p* as an :c:expr:`int`. If the object is an integer, its value is returned. If not, the object's :meth:`~io.IOBase.fileno` method is called if it exists; the method must return an integer, which is returned as the file descriptor value. Sets an exception and returns ``-1`` on failure. .. c:function:: PyObject* PyFile_GetLine(PyObject *p, int n) .. index:: single: EOFError (built-in exception) Equivalent to ``p.readline([n])``, this function reads one line from the object *p*. *p* may be a file object or any object with a :meth:`~io.IOBase.readline` method. If *n* is ``0``, exactly one line is read, regardless of the length of the line. If *n* is greater than ``0``, no more than *n* bytes will be read from the file; a partial line can be returned. In both cases, an empty string is returned if the end of the file is reached immediately. If *n* is less than ``0``, however, one line is read regardless of length, but :exc:`EOFError` is raised if the end of the file is reached immediately. .. c:function:: int PyFile_SetOpenCodeHook(Py_OpenCodeHookFunction handler) Overrides the normal behavior of :func:`io.open_code` to pass its parameter through the provided handler. The *handler* is a function of type: .. c:namespace:: NULL .. c:type:: PyObject * (*Py_OpenCodeHookFunction)(PyObject *, void *) Equivalent of :c:expr:`PyObject *(\*)(PyObject *path, void *userData)`, where *path* is guaranteed to be :c:type:`PyUnicodeObject`. The *userData* pointer is passed into the hook function. Since hook functions may be called from different runtimes, this pointer should not refer directly to Python state. As this hook is intentionally used during import, avoid importing new modules during its execution unless they are known to be frozen or available in ``sys.modules``. Once a hook has been set, it cannot be removed or replaced, and later calls to :c:func:`PyFile_SetOpenCodeHook` will fail. On failure, the function returns -1 and sets an exception if the interpreter has been initialized. This function is safe to call before :c:func:`Py_Initialize`. .. audit-event:: setopencodehook "" c.PyFile_SetOpenCodeHook .. versionadded:: 3.8 .. c:function:: PyObject *PyFile_OpenCodeObject(PyObject *path) Open *path* with the mode ``'rb'``. *path* must be a Python :class:`str` object. The behavior of this function may be overridden by :c:func:`PyFile_SetOpenCodeHook` to allow for some preprocessing of the text. This is analogous to :func:`io.open_code` in Python. On success, this function returns a :term:`strong reference` to a Python file object. On failure, this function returns ``NULL`` with an exception set. .. versionadded:: 3.8 .. c:function:: PyObject *PyFile_OpenCode(const char *path) Similar to :c:func:`PyFile_OpenCodeObject`, but *path* is a UTF-8 encoded :c:expr:`const char*`. .. versionadded:: 3.8 .. c:function:: int PyFile_WriteObject(PyObject *obj, PyObject *p, int flags) .. index:: single: Py_PRINT_RAW (C macro) Write object *obj* to file object *p*. The only supported flag for *flags* is :c:macro:`Py_PRINT_RAW`; if given, the :func:`str` of the object is written instead of the :func:`repr`. If *obj* is ``NULL``, write the string ``""``. Return ``0`` on success or ``-1`` on failure; the appropriate exception will be set. .. c:function:: int PyFile_WriteString(const char *s, PyObject *p) Write string *s* to file object *p*. Return ``0`` on success or ``-1`` on failure; the appropriate exception will be set. Soft-deprecated API ^^^^^^^^^^^^^^^^^^^ .. soft-deprecated:: 3.15 These are APIs that were included in Python's C API by mistake. They are documented solely for completeness; use other ``PyFile*`` APIs instead. .. c:function:: PyObject *PyFile_NewStdPrinter(int fd) Use :c:func:`PyFile_FromFd` with defaults (``fd, NULL, "w", -1, NULL, NULL, NULL, 0``) instead. .. c:var:: PyTypeObject PyStdPrinter_Type Type of file-like objects used internally at Python startup when :py:mod:`io` is not yet available. Use Python :py:func:`open` or :c:func:`PyFile_FromFd` to create file objects instead. --- ### Doc C Api Float (`Doc/c-api/float.rst`) .. highlight:: c .. _floatobjects: Floating-Point Objects ====================== .. index:: pair: object; floating-point .. c:type:: PyFloatObject This subtype of :c:type:`PyObject` represents a Python floating-point object. .. c:var:: PyTypeObject PyFloat_Type This instance of :c:type:`PyTypeObject` represents the Python floating-point type. This is the same object as :class:`float` in the Python layer. .. c:function:: int PyFloat_Check(PyObject *p) Return true if its argument is a :c:type:`PyFloatObject` or a subtype of :c:type:`PyFloatObject`. This function always succeeds. .. c:function:: int PyFloat_CheckExact(PyObject *p) Return true if its argument is a :c:type:`PyFloatObject`, but not a subtype of :c:type:`PyFloatObject`. This function always succeeds. .. c:function:: PyObject* PyFloat_FromString(PyObject *str) Create a :c:type:`PyFloatObject` object based on the string value in *str*, or ``NULL`` on failure. .. c:function:: PyObject* PyFloat_FromDouble(double v) Create a :c:type:`PyFloatObject` object from *v*, or ``NULL`` on failure. .. c:function:: double PyFloat_AsDouble(PyObject *pyfloat) Return a C :c:expr:`double` representation of the contents of *pyfloat*. If *pyfloat* is not a Python floating-point object but has a :meth:`~object.__float__` method, this method will first be called to convert *pyfloat* into a float. If :meth:`!__float__` is not defined then it falls back to :meth:`~object.__index__`. This method returns ``-1.0`` upon failure, so one should call :c:func:`PyErr_Occurred` to check for errors. .. versionchanged:: 3.8 Use :meth:`~object.__index__` if available. .. c:function:: double PyFloat_AS_DOUBLE(PyObject *pyfloat) Return a C :c:expr:`double` representation of the contents of *pyfloat*, but without error checking. .. c:function:: PyObject* PyFloat_GetInfo(void) Return a structseq instance which contains information about the precision, minimum and maximum values of a float. It's a thin wrapper around the header file :file:`float.h`. .. c:function:: double PyFloat_GetMax() Return the maximum representable finite float *DBL_MAX* as C :c:expr:`double`. .. c:function:: double PyFloat_GetMin() Return the minimum normalized positive float *DBL_MIN* as C :c:expr:`double`. .. c:macro:: Py_INFINITY This macro expands to a constant expression of type :c:expr:`double`, that represents the positive infinity. It is equivalent to the :c:macro:`!INFINITY` macro from the C11 standard ```` header. .. soft-deprecated:: 3.15 .. c:macro:: Py_NAN This macro expands to a constant expression of type :c:expr:`double`, that represents a quiet not-a-number (qNaN) value. On most platforms, this is equivalent to the :c:macro:`!NAN` macro from the C11 standard ```` header. .. c:macro:: Py_HUGE_VAL Equivalent to :c:macro:`!INFINITY`. .. soft-deprecated:: 3.14 .. c:macro:: Py_MATH_E The definition (accurate for a :c:expr:`double` type) of the :data:`math.e` constant. .. c:macro:: Py_MATH_El High precision (long double) definition of :data:`~math.e` constant. .. deprecated-removed:: 3.15 3.20 .. c:macro:: Py_MATH_PI The definition (accurate for a :c:expr:`double` type) of the :data:`math.pi` constant. .. c:macro:: Py_MATH_PIl High precision (long double) definition of :data:`~math.pi` constant. .. deprecated-removed:: 3.15 3.20 .. c:macro:: Py_MATH_TAU The definition (accurate for a :c:expr:`double` type) of the :data:`math.tau` constant. .. versionadded:: 3.6 .. c:macro:: Py_RETURN_NAN Return :data:`math.nan` from a function. On most platforms, this is equivalent to ``return PyFloat_FromDouble(NAN)``. .. c:macro:: Py_RETURN_INF(sign) Return :data:`math.inf` or :data:`-math.inf ` from a function, depending on the sign of *sign*. On most platforms, this is equivalent to the following:: return PyFloat_FromDouble(copysign(INFINITY, sign)); .. c:macro:: Py_IS_FINITE(X) Return ``1`` if the given floating-point number *X* is finite, that is, it is normal, subnormal or zero, but not infinite or NaN. Return ``0`` otherwise. .. soft-deprecated:: 3.14 Use :c:macro:`!isfinite` instead. .. c:macro:: Py_IS_INFINITY(X) Return ``1`` if the given floating-point number *X* is positive or negative infinity. Return ``0`` otherwise. .. soft-deprecated:: 3.14 Use :c:macro:`!isinf` instead. .. c:macro:: Py_IS_NAN(X) Return ``1`` if the given floating-point number *X* is a not-a-number (NaN) value. Return ``0`` otherwise. .. soft-deprecated:: 3.14 Use :c:macro:`!isnan` instead. Pack and Unpack functions ------------------------- The pack and unpack functions provide an efficient platform-independent way to store floating-point values as byte strings. The Pack routines produce a bytes string from a C :c:expr:`double`, and the Unpack routines produce a C :c:expr:`double` from such a bytes string. The suffix (2, 4 or 8) specifies the number of bytes in the bytes string: * The 2-byte format is the IEEE 754 binary16 half-precision format. * The 4-byte format is the IEEE 754 binary32 single-precision format. * The 8-byte format is the IEEE 754 binary64 double-precision format. The NaN type may not be preserved on some platforms while unpacking (signaling NaNs become quiet NaNs), for example on x86 systems in 32-bit mode. It's assumed that the :c:expr:`double` type has the IEEE 754 binary64 double precision format. What happens if it's not true is partly accidental (alas). On non-IEEE platforms with more precision, or larger dynamic range, than IEEE 754 supports, not all values can be packed; on non-IEEE platforms with less precision, or smaller dynamic range, not all values can be unpacked. The packing of special numbers like INFs and NaNs (if such things exist on the platform) may not be handled correctly, and attempting to unpack a bytes string containing an IEEE INF or NaN may raise an exception. .. versionadded:: 3.11 Pack functions ^^^^^^^^^^^^^^ The pack routines write 2, 4 or 8 bytes, starting at *p*. *le* is an :c:expr:`int` argument, non-zero if you want the bytes string in little-endian format (exponent last, at ``p+1``, ``p+3``, or ``p+6`` and ``p+7``), zero if you want big-endian format (exponent first, at *p*). Use the :c:macro:`!PY_LITTLE_ENDIAN` constant to select the native endian: it is equal to ``0`` on big endian processor, or ``1`` on little endian processor. Return value: ``0`` if all is OK, ``-1`` if error (and an exception is set, most likely :exc:`OverflowError`). .. c:function:: int PyFloat_Pack2(double x, char *p, int le) Pack a C double as the IEEE 754 binary16 half-precision format. .. c:function:: int PyFloat_Pack4(double x, char *p, int le) Pack a C double as the IEEE 754 binary32 single precision format. .. c:function:: int PyFloat_Pack8(double x, char *p, int le) Pack a C double as the IEEE 754 binary64 double precision format. Unpack functions ^^^^^^^^^^^^^^^^ The unpack routines read 2, 4 or 8 bytes, starting at *p*. *le* is an :c:expr:`int` argument, non-zero if the bytes string is in little-endian format (exponent last, at ``p+1``, ``p+3`` or ``p+6`` and ``p+7``), zero if big-endian (exponent first, at *p*). Use the :c:macro:`!PY_LITTLE_ENDIAN` constant to select the native endian: it is equal to ``0`` on big endian processor, or ``1`` on little endian processor. Return value: The unpacked double. On error, this is ``-1.0`` and :c:func:`PyErr_Occurred` is true (and an exception is set, most likely :exc:`OverflowError`). .. c:function:: double PyFloat_Unpack2(const char *p, int le) Unpack the IEEE 754 binary16 half-precision format as a C double. .. c:function:: double PyFloat_Unpack4(const char *p, int le) Unpack the IEEE 754 binary32 single precision format as a C double. .. c:function:: double PyFloat_Unpack8(const char *p, int le) Unpack the IEEE 754 binary64 double precision format as a C double. --- ### Doc C Api Frame (`Doc/c-api/frame.rst`) .. highlight:: c Frame objects ------------- .. c:type:: PyFrameObject The C structure of the objects used to describe frame objects. There are no public members in this structure. .. versionchanged:: 3.11 The members of this structure were removed from the public C API. Refer to the :ref:`What's New entry ` for details. The :c:func:`PyEval_GetFrame` and :c:func:`PyThreadState_GetFrame` functions can be used to get a frame object. See also :ref:`Reflection `. .. c:var:: PyTypeObject PyFrame_Type The type of frame objects. It is the same object as :py:class:`types.FrameType` in the Python layer. .. versionchanged:: 3.11 Previously, this type was only available after including ````. .. c:function:: PyFrameObject *PyFrame_New(PyThreadState *tstate, PyCodeObject *code, PyObject *globals, PyObject *locals) Create a new frame object. This function returns a :term:`strong reference` to the new frame object on success, and returns ``NULL`` with an exception set on failure. .. c:function:: int PyFrame_Check(PyObject *obj) Return non-zero if *obj* is a frame object. .. versionchanged:: 3.11 Previously, this function was only available after including ````. .. c:function:: PyFrameObject* PyFrame_GetBack(PyFrameObject *frame) Get the *frame* next outer frame. Return a :term:`strong reference`, or ``NULL`` if *frame* has no outer frame. This raises no exceptions. .. versionadded:: 3.9 .. c:function:: PyObject* PyFrame_GetBuiltins(PyFrameObject *frame) Get the *frame*'s :attr:`~frame.f_builtins` attribute. Return a :term:`strong reference`. The result cannot be ``NULL``. .. versionadded:: 3.11 .. c:function:: PyCodeObject* PyFrame_GetCode(PyFrameObject *frame) Get the *frame* code. Return a :term:`strong reference`. The result (frame code) cannot be ``NULL``. .. versionadded:: 3.9 .. c:function:: PyObject* PyFrame_GetGenerator(PyFrameObject *frame) Get the generator, coroutine, or async generator that owns this frame, or ``NULL`` if this frame is not owned by a generator. Does not raise an exception, even if the return value is ``NULL``. Return a :term:`strong reference`, or ``NULL``. .. versionadded:: 3.11 .. c:function:: PyObject* PyFrame_GetGlobals(PyFrameObject *frame) Get the *frame*'s :attr:`~frame.f_globals` attribute. Return a :term:`strong reference`. The result cannot be ``NULL``. .. versionadded:: 3.11 .. c:function:: int PyFrame_GetLasti(PyFrameObject *frame) Get the *frame*'s :attr:`~frame.f_lasti` attribute. Returns -1 if ``frame.f_lasti`` is ``None``. .. versionadded:: 3.11 .. c:function:: PyObject* PyFrame_GetVar(PyFrameObject *frame, PyObject *name) Get the variable *name* of *frame*. * Return a :term:`strong reference` to the variable value on success. * Raise :exc:`NameError` and return ``NULL`` if the variable does not exist. * Raise an exception and return ``NULL`` on error. *name* type must be a :class:`str`. .. versionadded:: 3.12 .. c:function:: PyObject* PyFrame_GetVarString(PyFrameObject *frame, const char *name) Similar to :c:func:`PyFrame_GetVar`, but the variable name is a C string encoded in UTF-8. .. versionadded:: 3.12 .. c:function:: PyObject* PyFrame_GetLocals(PyFrameObject *frame) Get the *frame*'s :attr:`~frame.f_locals` attribute. If the frame refers to an :term:`optimized scope`, this returns a write-through proxy object that allows modifying the locals. In all other cases (classes, modules, :func:`exec`, :func:`eval`) it returns the mapping representing the frame locals directly (as described for :func:`locals`). Return a :term:`strong reference`. .. versionadded:: 3.11 .. versionchanged:: 3.13 As part of :pep:`667`, return an instance of :c:var:`PyFrameLocalsProxy_Type`. .. c:function:: int PyFrame_GetLineNumber(PyFrameObject *frame) Return the line number that *frame* is currently executing. Frame locals proxies ^^^^^^^^^^^^^^^^^^^^ .. versionadded:: 3.13 The :attr:`~frame.f_locals` attribute on a :ref:`frame object ` is an instance of a "frame-locals proxy". The proxy object exposes a write-through view of the underlying locals dictionary for the frame. This ensures that the variables exposed by ``f_locals`` are always up to date with the live local variables in the frame itself. See :pep:`667` for more information. .. c:var:: PyTypeObject PyFrameLocalsProxy_Type The type of frame :func:`locals` proxy objects. .. c:function:: int PyFrameLocalsProxy_Check(PyObject *obj) Return non-zero if *obj* is a frame :func:`locals` proxy. Legacy local variable APIs ^^^^^^^^^^^^^^^^^^^^^^^^^^ These APIs are :term:`soft deprecated`. As of Python 3.13, they do nothing. They exist solely for backwards compatibility. .. c:function:: void PyFrame_LocalsToFast(PyFrameObject *f, int clear) Prior to Python 3.13, this function would copy the :attr:`~frame.f_locals` attribute of *f* to the internal "fast" array of local variables, allowing changes in frame objects to be visible to the interpreter. If *clear* was true, this function would process variables that were unset in the locals dictionary. .. soft-deprecated:: 3.13 This function now does nothing. .. c:function:: void PyFrame_FastToLocals(PyFrameObject *f) Prior to Python 3.13, this function would copy the internal "fast" array of local variables (which is used by the interpreter) to the :attr:`~frame.f_locals` attribute of *f*, allowing changes in local variables to be visible to frame objects. .. soft-deprecated:: 3.13 This function now does nothing. .. c:function:: int PyFrame_FastToLocalsWithError(PyFrameObject *f) Prior to Python 3.13, this function was similar to :c:func:`PyFrame_FastToLocals`, but would return ``0`` on success, and ``-1`` with an exception set on failure. .. soft-deprecated:: 3.13 This function now does nothing. .. seealso:: :pep:`667` Internal frames ^^^^^^^^^^^^^^^ Unless using :pep:`523`, you will not need this. .. c:struct:: _PyInterpreterFrame The interpreter's internal frame representation. .. versionadded:: 3.11 .. c:function:: PyObject* PyUnstable_InterpreterFrame_GetCode(struct _PyInterpreterFrame *frame); Return a :term:`strong reference` to the code object for the frame. .. versionadded:: 3.12 .. c:function:: int PyUnstable_InterpreterFrame_GetLasti(struct _PyInterpreterFrame *frame); Return the byte offset into the last executed instruction. .. versionadded:: 3.12 .. c:function:: int PyUnstable_InterpreterFrame_GetLine(struct _PyInterpreterFrame *frame); Return the currently executing line number, or -1 if there is no line number. .. versionadded:: 3.12 --- ### Doc C Api Function (`Doc/c-api/function.rst`) .. highlight:: c .. _function-objects: Function Objects ---------------- .. index:: pair: object; function There are a few functions specific to Python functions. .. c:type:: PyFunctionObject The C structure used for functions. .. c:var:: PyTypeObject PyFunction_Type .. index:: single: MethodType (in module types) This is an instance of :c:type:`PyTypeObject` and represents the Python function type. It is exposed to Python programmers as ``types.FunctionType``. .. c:function:: int PyFunction_Check(PyObject *o) Return true if *o* is a function object (has type :c:data:`PyFunction_Type`). The parameter must not be ``NULL``. This function always succeeds. .. c:function:: PyObject* PyFunction_New(PyObject *code, PyObject *globals) Return a new function object associated with the code object *code*. *globals* must be a dictionary with the global variables accessible to the function. The function's docstring and name are retrieved from the code object. :attr:`~function.__module__` is retrieved from *globals*. The argument defaults, annotations and closure are set to ``NULL``. :attr:`~function.__qualname__` is set to the same value as the code object's :attr:`~codeobject.co_qualname` field. .. c:function:: PyObject* PyFunction_NewWithQualName(PyObject *code, PyObject *globals, PyObject *qualname) As :c:func:`PyFunction_New`, but also allows setting the function object's :attr:`~function.__qualname__` attribute. *qualname* should be a unicode object or ``NULL``; if ``NULL``, the :attr:`!__qualname__` attribute is set to the same value as the code object's :attr:`~codeobject.co_qualname` field. .. versionadded:: 3.3 .. c:function:: PyObject* PyFunction_GetCode(PyObject *op) Return the code object associated with the function object *op*. .. c:function:: PyObject* PyFunction_GetGlobals(PyObject *op) Return the globals dictionary associated with the function object *op*. .. c:function:: PyObject* PyFunction_GetModule(PyObject *op) Return a :term:`borrowed reference` to the :attr:`~function.__module__` attribute of the :ref:`function object ` *op*. It can be *NULL*. This is normally a :class:`string ` containing the module name, but can be set to any other object by Python code. .. c:function:: PyObject* PyFunction_GetDefaults(PyObject *op) Return the argument default values of the function object *op*. This can be a tuple of arguments or ``NULL``. .. c:function:: int PyFunction_SetDefaults(PyObject *op, PyObject *defaults) Set the argument default values for the function object *op*. *defaults* must be ``Py_None`` or a tuple. Raises :exc:`SystemError` and returns ``-1`` on failure. .. c:function:: void PyFunction_SetVectorcall(PyFunctionObject *func, vectorcallfunc vectorcall) Set the vectorcall field of a given function object *func*. Warning: extensions using this API must preserve the behavior of the unaltered (default) vectorcall function! .. versionadded:: 3.12 .. c:function:: PyObject* PyFunction_GetKwDefaults(PyObject *op) Return the keyword-only argument default values of the function object *op*. This can be a dictionary of arguments or ``NULL``. .. c:function:: int PyFunction_SetKwDefaults(PyObject *op, PyObject *defaults) Set the keyword-only argument default values of the function object *op*. *defaults* must be a dictionary of keyword-only arguments or ``Py_None``. This function returns ``0`` on success, and returns ``-1`` with an exception set on failure. .. c:function:: PyObject* PyFunction_GetClosure(PyObject *op) Return the closure associated with the function object *op*. This can be ``NULL`` or a tuple of cell objects. .. c:function:: int PyFunction_SetClosure(PyObject *op, PyObject *closure) Set the closure associated with the function object *op*. *closure* must be ``Py_None`` or a tuple of cell objects. Raises :exc:`SystemError` and returns ``-1`` on failure. .. c:function:: PyObject *PyFunction_GetAnnotations(PyObject *op) Return the annotations of the function object *op*. This can be a mutable dictionary or ``NULL``. .. c:function:: int PyFunction_SetAnnotations(PyObject *op, PyObject *annotations) Set the annotations for the function object *op*. *annotations* must be a dictionary or ``Py_None``. Raises :exc:`SystemError` and returns ``-1`` on failure. .. c:function:: PyObject *PyFunction_GET_CODE(PyObject *op) PyObject *PyFunction_GET_GLOBALS(PyObject *op) PyObject *PyFunction_GET_MODULE(PyObject *op) PyObject *PyFunction_GET_DEFAULTS(PyObject *op) PyObject *PyFunction_GET_KW_DEFAULTS(PyObject *op) PyObject *PyFunction_GET_CLOSURE(PyObject *op) PyObject *PyFunction_GET_ANNOTATIONS(PyObject *op) These functions are similar to their ``PyFunction_Get*`` counterparts, but do not do type checking. Passing anything other than an instance of :c:data:`PyFunction_Type` is undefined behavior. .. c:function:: int PyFunction_AddWatcher(PyFunction_WatchCallback callback) Register *callback* as a function watcher for the current interpreter. Return an ID which may be passed to :c:func:`PyFunction_ClearWatcher`. In case of error (e.g. no more watcher IDs available), return ``-1`` and set an exception. .. versionadded:: 3.12 .. c:function:: int PyFunction_ClearWatcher(int watcher_id) Clear watcher identified by *watcher_id* previously returned from :c:func:`PyFunction_AddWatcher` for the current interpreter. Return ``0`` on success, or ``-1`` and set an exception on error (e.g. if the given *watcher_id* was never registered.) .. versionadded:: 3.12 .. c:type:: PyFunction_WatchEvent Enumeration of possible function watcher events: - ``PyFunction_EVENT_CREATE`` - ``PyFunction_EVENT_DESTROY`` - ``PyFunction_EVENT_MODIFY_CODE`` - ``PyFunction_EVENT_MODIFY_DEFAULTS`` - ``PyFunction_EVENT_MODIFY_KWDEFAULTS`` .. versionadded:: 3.12 - ``PyFunction_PYFUNC_EVENT_MODIFY_QUALNAME`` .. versionadded:: 3.15 .. c:type:: int (*PyFunction_WatchCallback)(PyFunction_WatchEvent event, PyFunctionObject *func, PyObject *new_value) Type of a function watcher callback function. If *event* is ``PyFunction_EVENT_CREATE`` or ``PyFunction_EVENT_DESTROY`` then *new_value* will be ``NULL``. Otherwise, *new_value* will hold a :term:`borrowed reference` to the new value that is about to be stored in *func* for the attribute that is being modified. The callback may inspect but must not modify *func*; doing so could have unpredictable effects, including infinite recursion. If *event* is ``PyFunction_EVENT_CREATE``, then the callback is invoked after *func* has been fully initialized. Otherwise, the callback is invoked before the modification to *func* takes place, so the prior state of *func* can be inspected. The runtime is permitted to optimize away the creation of function objects when possible. In such cases no event will be emitted. Although this creates the possibility of an observable difference of runtime behavior depending on optimization decisions, it does not change the semantics of the Python code being executed. If *event* is ``PyFunction_EVENT_DESTROY``, taking a reference in the callback to the about-to-be-destroyed function will resurrect it, preventing it from being freed at this time. When the resurrected object is destroyed later, any watcher callbacks active at that time will be called again. If the callback sets an exception, it must return ``-1``; this exception will be printed as an unraisable exception using :c:func:`PyErr_WriteUnraisable`. Otherwise it should return ``0``. There may already be a pending exception set on entry to the callback. In this case, the callback should return ``0`` with the same exception still set. This means the callback may not call any other API that can set an exception unless it saves and clears the exception state first, and restores it before returning. .. versionadded:: 3.12 --- ### Doc C Api Gcsupport (`Doc/c-api/gcsupport.rst`) .. highlight:: c .. _supporting-cycle-detection: Supporting Cyclic Garbage Collection ==================================== Python's support for detecting and collecting garbage which involves circular references requires support from object types which are "containers" for other objects which may also be containers. Types which do not store references to other objects, or which only store references to atomic types (such as numbers or strings), do not need to provide any explicit support for garbage collection. To create a container type, the :c:member:`~PyTypeObject.tp_flags` field of the type object must include the :c:macro:`Py_TPFLAGS_HAVE_GC` and provide an implementation of the :c:member:`~PyTypeObject.tp_traverse` handler. If instances of the type are mutable, a :c:member:`~PyTypeObject.tp_clear` implementation must also be provided. :c:macro:`Py_TPFLAGS_HAVE_GC` Objects with a type with this flag set must conform with the rules documented here. For convenience these objects will be referred to as container objects. Constructors for container types must conform to two rules: #. The memory for the object must be allocated using :c:macro:`PyObject_GC_New` or :c:macro:`PyObject_GC_NewVar`. #. Once all the fields which may contain references to other containers are initialized, it must call :c:func:`PyObject_GC_Track`. Similarly, the deallocator for the object must conform to a similar pair of rules: #. Before fields which refer to other containers are invalidated, :c:func:`PyObject_GC_UnTrack` must be called. #. The object's memory must be deallocated using :c:func:`PyObject_GC_Del`. .. warning:: If a type adds the Py_TPFLAGS_HAVE_GC, then it *must* implement at least a :c:member:`~PyTypeObject.tp_traverse` handler or explicitly use one from its subclass or subclasses. When calling :c:func:`PyType_Ready` or some of the APIs that indirectly call it like :c:func:`PyType_FromSpecWithBases` or :c:func:`PyType_FromSpec` the interpreter will automatically populate the :c:member:`~PyTypeObject.tp_flags`, :c:member:`~PyTypeObject.tp_traverse` and :c:member:`~PyTypeObject.tp_clear` fields if the type inherits from a class that implements the garbage collector protocol and the child class does *not* include the :c:macro:`Py_TPFLAGS_HAVE_GC` flag. .. c:macro:: PyObject_GC_New(TYPE, typeobj) Analogous to :c:macro:`PyObject_New` but for container objects with the :c:macro:`Py_TPFLAGS_HAVE_GC` flag set. Do not call this directly to allocate memory for an object; call the type's :c:member:`~PyTypeObject.tp_alloc` slot instead. When populating a type's :c:member:`~PyTypeObject.tp_alloc` slot, :c:func:`PyType_GenericAlloc` is preferred over a custom function that simply calls this macro. Memory allocated by this macro must be freed with :c:func:`PyObject_GC_Del` (usually called via the object's :c:member:`~PyTypeObject.tp_free` slot). .. seealso:: * :c:func:`PyObject_GC_Del` * :c:macro:`PyObject_New` * :c:func:`PyType_GenericAlloc` * :c:member:`~PyTypeObject.tp_alloc` .. c:macro:: PyObject_GC_NewVar(TYPE, typeobj, size) Analogous to :c:macro:`PyObject_NewVar` but for container objects with the :c:macro:`Py_TPFLAGS_HAVE_GC` flag set. Do not call this directly to allocate memory for an object; call the type's :c:member:`~PyTypeObject.tp_alloc` slot instead. When populating a type's :c:member:`~PyTypeObject.tp_alloc` slot, :c:func:`PyType_GenericAlloc` is preferred over a custom function that simply calls this macro. Memory allocated by this macro must be freed with :c:func:`PyObject_GC_Del` (usually called via the object's :c:member:`~PyTypeObject.tp_free` slot). .. seealso:: * :c:func:`PyObject_GC_Del` * :c:macro:`PyObject_NewVar` * :c:func:`PyType_GenericAlloc` * :c:member:`~PyTypeObject.tp_alloc` .. c:function:: PyObject* PyUnstable_Object_GC_NewWithExtraData(PyTypeObject *type, size_t extra_size) Analogous to :c:macro:`PyObject_GC_New` but allocates *extra_size* bytes at the end of the object (at offset :c:member:`~PyTypeObject.tp_basicsize`). The allocated memory is initialized to zeros, except for the :c:type:`Python object header `. The extra data will be deallocated with the object, but otherwise it is not managed by Python. Memory allocated by this function must be freed with :c:func:`PyObject_GC_Del` (usually called via the object's :c:member:`~PyTypeObject.tp_free` slot). .. warning:: The function is marked as unstable because the final mechanism for reserving extra data after an instance is not yet decided. For allocating a variable number of fields, prefer using :c:type:`PyVarObject` and :c:member:`~PyTypeObject.tp_itemsize` instead. .. versionadded:: 3.12 .. c:macro:: PyObject_GC_Resize(TYPE, op, newsize) Resize an object allocated by :c:macro:`PyObject_NewVar`. Returns the resized object of type ``TYPE*`` (refers to any C type) or ``NULL`` on failure. *op* must be of type :c:expr:`PyVarObject *` and must not be tracked by the collector yet. *newsize* must be of type :c:type:`Py_ssize_t`. .. c:function:: void PyObject_GC_Track(PyObject *op) Adds the object *op* to the set of container objects tracked by the collector. The collector can run at unexpected times so objects must be valid while being tracked. This should be called once all the fields followed by the :c:member:`~PyTypeObject.tp_traverse` handler become valid, usually near the end of the constructor. .. c:function:: int PyObject_IS_GC(PyObject *obj) Returns non-zero if the object implements the garbage collector protocol, otherwise returns 0. The object cannot be tracked by the garbage collector if this function returns 0. .. c:function:: int PyObject_GC_IsTracked(PyObject *op) Returns 1 if the object type of *op* implements the GC protocol and *op* is being currently tracked by the garbage collector and 0 otherwise. This is analogous to the Python function :func:`gc.is_tracked`. .. versionadded:: 3.9 .. c:function:: int PyObject_GC_IsFinalized(PyObject *op) Returns 1 if the object type of *op* implements the GC protocol and *op* has been already finalized by the garbage collector and 0 otherwise. This is analogous to the Python function :func:`gc.is_finalized`. .. versionadded:: 3.9 .. c:function:: void PyObject_GC_Del(void *op) Releases memory allocated to an object using :c:macro:`PyObject_GC_New` or :c:macro:`PyObject_GC_NewVar`. Do not call this directly to free an object's memory; call the type's :c:member:`~PyTypeObject.tp_free` slot instead. Do not use this for memory allocated by :c:macro:`PyObject_New`, :c:macro:`PyObject_NewVar`, or related allocation functions; use :c:func:`PyObject_Free` instead. .. seealso:: * :c:func:`PyObject_Free` is the non-GC equivalent of this function. * :c:macro:`PyObject_GC_New` * :c:macro:`PyObject_GC_NewVar` * :c:func:`PyType_GenericAlloc` * :c:member:`~PyTypeObject.tp_free` .. c:function:: void PyObject_GC_UnTrack(void *op) Remove the object *op* from the set of container objects tracked by the collector. Note that :c:func:`PyObject_GC_Track` can be called again on this object to add it back to the set of tracked objects. The deallocator (:c:member:`~PyTypeObject.tp_dealloc` handler) should call this for the object before any of the fields used by the :c:member:`~PyTypeObject.tp_traverse` handler become invalid. .. versionchanged:: 3.8 The :c:func:`!_PyObject_GC_TRACK` and :c:func:`!_PyObject_GC_UNTRACK` macros have been removed from the public C API. The :c:member:`~PyTypeObject.tp_traverse` handler accepts a function parameter of this type: .. c:type:: int (*visitproc)(PyObject *object, void *arg) Type of the visitor function passed to the :c:member:`~PyTypeObject.tp_traverse` handler. The function should be called with an object to traverse as *object* and the third parameter to the :c:member:`~PyTypeObject.tp_traverse` handler as *arg*. The Python core uses several visitor functions to implement cyclic garbage detection; it's not expected that users will need to write their own visitor functions. The :c:member:`~PyTypeObject.tp_clear` handler must be of the :c:type:`inquiry` type, or ``NULL`` if the object is immutable. .. c:type:: int (*inquiry)(PyObject *self) Drop references that may have created reference cycles. Immutable objects do not have to define this method since they can never directly create reference cycles. Note that the object must still be valid after calling this method (don't just call :c:func:`Py_DECREF` on a reference). The collector will call this method if it detects that this object is involved in a reference cycle. .. _gc-traversal: Traversal --------- The :c:member:`~PyTypeObject.tp_traverse` handler must have the following type: .. c:type:: int (*traverseproc)(PyObject *self, visitproc visit, void *arg) Traversal function for a garbage-collected object, used by the garbage collector to detect reference cycles. Implementations must call the *visit* function for each object directly contained by *self*, with the parameters to *visit* being the contained object and the *arg* value passed to the handler. The *visit* function must not be called with a ``NULL`` object argument. If *visit* returns a non-zero value, that value should be returned immediately. A typical :c:member:`!tp_traverse` function calls the :c:func:`Py_VISIT` convenience macro on each of the instance's members that are Python objects that the instance owns. For example, this is a (slightly outdated) traversal function for the :py:class:`threading.local` class:: static int local_traverse(PyObject *op, visitproc visit, void *arg) { localobject *self = (localobject *) op; Py_VISIT(Py_TYPE(self)); Py_VISIT(self->args); Py_VISIT(self->kw); Py_VISIT(self->dict); return 0; } .. note:: :c:func:`Py_VISIT` requires the *visit* and *arg* parameters to :c:func:`!local_traverse` to have these specific names; don't name them just anything. Instances of :ref:`heap-allocated types ` hold a reference to their type. Their traversal function must therefore visit the type:: Py_VISIT(Py_TYPE(self)); Alternately, the type may delegate this responsibility by calling ``tp_traverse`` of a heap-allocated superclass (or another heap-allocated type, if applicable). If they do not, the type object may not be garbage-collected. If the :c:macro:`Py_TPFLAGS_MANAGED_DICT` bit is set in the :c:member:`~PyTypeObject.tp_flags` field, the traverse function must call :c:func:`PyObject_VisitManagedDict` like this:: int err = PyObject_VisitManagedDict((PyObject*)self, visit, arg); if (err) { return err; } Only the members that the instance *owns* (by having :term:`strong references ` to them) must be visited. For instance, if an object supports weak references via the :c:member:`~PyTypeObject.tp_weaklist` slot, the pointer supporting the linked list (what *tp_weaklist* points to) must **not** be visited as the instance does not directly own the weak references to itself. The traversal function has a limitation: .. warning:: The traversal function must not have any side effects. Implementations may not modify the reference counts of any Python objects nor create or destroy any Python objects, directly or indirectly. This means that *most* Python C API functions may not be used, since they can raise a new exception, return a new reference to a result object, have internal logic that uses side effects. Also, unless documented otherwise, functions that happen to not have side effects may start having them in future versions, without warning. For a list of safe functions, see a :ref:`separate section ` below. .. note:: The :c:func:`Py_VISIT` call may be skipped for those members that provably cannot participate in reference cycles. In the ``local_traverse`` example above, there is also a ``self->key`` member, but it can only be ``NULL`` or a Python string and therefore cannot be part of a reference cycle. On the other hand, even if you know a member can never be part of a cycle, as a debugging aid you may want to visit it anyway just so the :mod:`gc` module's :func:`~gc.get_referents` function will include it. .. note:: The :c:member:`~PyTypeObject.tp_traverse` function can be called from any thread. .. impl-detail:: Garbage collection is a "stop-the-world" operation: even in :term:`free threading` builds, only one thread state is :term:`attached ` when :c:member:`!tp_traverse` handlers run. .. versionchanged:: 3.9 Heap-allocated types are expected to visit ``Py_TYPE(self)`` in ``tp_traverse``. In earlier versions of Python, due to `bug 40217 `_, doing this may lead to crashes in subclasses. To simplify writing :c:member:`~PyTypeObject.tp_traverse` handlers, a :c:func:`Py_VISIT` macro is provided. In order to use this macro, the :c:member:`~PyTypeObject.tp_traverse` implementation must name its arguments exactly *visit* and *arg*: .. c:macro:: Py_VISIT(o) If the :c:expr:`PyObject *` *o* is not ``NULL``, call the *visit* callback, with arguments *o* and *arg*. If *visit* returns a non-zero value, then return it. This corresponds roughly to:: #define Py_VISIT(o) \ if (op) { \ int visit_result = visit(o, arg); \ if (visit_result != 0) { \ return visit_result; \ } \ } Traversal-safe functions ^^^^^^^^^^^^^^^^^^^^^^^^ The following functions and macros are safe to use in a :c:member:`~PyTypeObject.tp_traverse` handler: * the *visit* function passed to ``tp_traverse`` * :c:func:`Py_VISIT` * :c:func:`Py_SIZE` * :c:func:`Py_TYPE`: if called from a :c:member:`!tp_traverse` handler, :c:func:`!Py_TYPE`'s result will be valid for the duration of the handler call * :c:func:`PyObject_VisitManagedDict` * :c:func:`PyObject_TypeCheck`, :c:func:`PyType_IsSubtype`, :c:func:`PyType_HasFeature` * :samp:`Py{}_Check` and :samp:`Py{}_CheckExact` -- for example, :c:func:`PyTuple_Check` * :ref:`duringgc-functions` .. _duringgc-functions: "DuringGC" functions ^^^^^^^^^^^^^^^^^^^^ The following functions should *only* be used in a :c:member:`~PyTypeObject.tp_traverse` handler; calling them in other contexts may have unintended consequences. These functions act like their counterparts without the ``_DuringGC`` suffix, but they are guaranteed to not have side effects, they do not set an exception on failure, and they return/set :term:`borrowed references ` as detailed in the individual documentation. Note that these functions may fail (return ``NULL`` or ``-1``), but as they do not set an exception, no error information is available. In some cases, failure is not distinguishable from a successful ``NULL`` result. .. c:function:: void *PyObject_GetTypeData_DuringGC(PyObject *o, PyTypeObject *cls) void *PyObject_GetItemData_DuringGC(PyObject *o) void *PyType_GetModuleState_DuringGC(PyTypeObject *type) void *PyModule_GetState_DuringGC(PyObject *module) int PyModule_GetToken_DuringGC(PyObject *module, void** result) See :ref:`duringgc-functions` for common information. .. versionadded:: 3.15 .. seealso:: :c:func:`PyObject_GetTypeData`, :c:func:`PyObject_GetItemData`, :c:func:`PyType_GetModuleState`, :c:func:`PyModule_GetState`, :c:func:`PyModule_GetToken`, :c:func:`PyType_GetBaseByToken` .. c:function:: int PyType_GetBaseByToken_DuringGC(PyTypeObject *type, void *tp_token, PyTypeObject **result) See :ref:`duringgc-functions` for common information. Sets *\*result* to a :term:`borrowed reference` rather than a strong one. The reference is valid for the duration of the :c:member:`!tp_traverse` handler call. .. versionadded:: 3.15 .. seealso:: :c:func:`PyType_GetBaseByToken` .. c:function:: PyObject* PyType_GetModule_DuringGC(PyTypeObject *type) PyObject* PyType_GetModuleByToken_DuringGC(PyTypeObject *type, const void *mod_token) See :ref:`duringgc-functions` for common information. These functions return a :term:`borrowed reference`, which is valid for the duration of the :c:member:`!tp_traverse` handler call. .. versionadded:: 3.15 .. seealso:: :c:func:`PyType_GetModule`, :c:func:`PyType_GetModuleByToken` Controlling the Garbage Collector State --------------------------------------- The C-API provides the following functions for controlling garbage collection runs. .. c:function:: Py_ssize_t PyGC_Collect(void) Perform a full garbage collection, if the garbage collector is enabled. (Note that :func:`gc.collect` runs it unconditionally.) Returns the number of collected + unreachable objects which cannot be collected. If the garbage collector is disabled or already collecting, returns ``0`` immediately. Errors during garbage collection are passed to :data:`sys.unraisablehook`. This function does not raise exceptions. .. c:function:: int PyGC_Enable(void) Enable the garbage collector: similar to :func:`gc.enable`. Returns the previous state, 0 for disabled and 1 for enabled. .. versionadded:: 3.10 .. c:function:: int PyGC_Disable(void) Disable the garbage collector: similar to :func:`gc.disable`. Returns the previous state, 0 for disabled and 1 for enabled. .. versionadded:: 3.10 .. c:function:: int PyGC_IsEnabled(void) Query the state of the garbage collector: similar to :func:`gc.isenabled`. Returns the current state, 0 for disabled and 1 for enabled. .. versionadded:: 3.10 Querying Garbage Collector State -------------------------------- The C-API provides the following interface for querying information about the garbage collector. .. c:function:: void PyUnstable_GC_VisitObjects(gcvisitobjects_t callback, void *arg) Run supplied *callback* on all live GC-capable objects. *arg* is passed through to all invocations of *callback*. .. warning:: If new objects are (de)allocated by the callback it is undefined if they will be visited. Garbage collection is disabled during operation. Explicitly running a collection in the callback may lead to undefined behaviour e.g. visiting the same objects multiple times or not at all. .. versionadded:: 3.12 .. c:type:: int (*gcvisitobjects_t)(PyObject *object, void *arg) Type of the visitor function to be passed to :c:func:`PyUnstable_GC_VisitObjects`. *arg* is the same as the *arg* passed to ``PyUnstable_GC_VisitObjects``. Return ``1`` to continue iteration, return ``0`` to stop iteration. Other return values are reserved for now so behavior on returning anything else is undefined. .. versionadded:: 3.12 --- ### Doc C Api Gen (`Doc/c-api/gen.rst`) .. highlight:: c .. _gen-objects: Generator Objects ----------------- Generator objects are what Python uses to implement generator iterators. They are normally created by iterating over a function that yields values, rather than explicitly calling :c:func:`PyGen_New` or :c:func:`PyGen_NewWithQualName`. .. c:type:: PyGenObject The C structure used for generator objects. .. c:var:: PyTypeObject PyGen_Type The type object corresponding to generator objects. .. c:function:: int PyGen_Check(PyObject *ob) Return true if *ob* is a generator object; *ob* must not be ``NULL``. This function always succeeds. .. c:function:: int PyGen_CheckExact(PyObject *ob) Return true if *ob*'s type is :c:type:`PyGen_Type`; *ob* must not be ``NULL``. This function always succeeds. .. c:function:: PyObject* PyGen_New(PyFrameObject *frame) Create and return a new generator object based on the *frame* object. A reference to *frame* is ":term:`stolen `" by this function (even on error). The argument must not be ``NULL``. .. deprecated-removed:: 3.16 3.18 This function has not been used since 3.10. It is also impossible to construct a proper *frame* object to call this function. .. c:function:: PyObject* PyGen_NewWithQualName(PyFrameObject *frame, PyObject *name, PyObject *qualname) Create and return a new generator object based on the *frame* object, with ``__name__`` and ``__qualname__`` set to *name* and *qualname*. A reference to *frame* is ":term:`stolen `" by this function (even on error). The *frame* argument must not be ``NULL``. .. deprecated-removed:: 3.16 3.18 This function has not been used since 3.10. It is also impossible to construct a proper *frame* object to call this function. .. c:function:: PyCodeObject* PyGen_GetCode(PyGenObject *gen) Return a new :term:`strong reference` to the code object wrapped by *gen*. This function always succeeds. Asynchronous Generator Objects ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ .. seealso:: :pep:`525` .. c:var:: PyTypeObject PyAsyncGen_Type The type object corresponding to asynchronous generator objects. This is available as :class:`types.AsyncGeneratorType` in the Python layer. .. versionadded:: 3.6 .. c:function:: PyObject *PyAsyncGen_New(PyFrameObject *frame, PyObject *name, PyObject *qualname) Create a new asynchronous generator wrapping *frame*, with ``__name__`` and ``__qualname__`` set to *name* and *qualname*. *frame* is ":term:`stolen `" by this function (even on error) and must not be ``NULL``. On success, this function returns a :term:`strong reference` to the new asynchronous generator. On failure, this function returns ``NULL`` with an exception set. .. versionadded:: 3.6 .. deprecated-removed:: 3.16 3.18 This function has not been used since 3.10. It is also impossible to construct a proper *frame* object to call this function. .. c:function:: int PyAsyncGen_CheckExact(PyObject *op) Return true if *op* is an asynchronous generator object, false otherwise. This function always succeeds. .. versionadded:: 3.6 Deprecated API ^^^^^^^^^^^^^^ .. c:macro:: PyAsyncGenASend_CheckExact(op) This is an API that was included in Python's C API by mistake. It is solely here for completeness; do not use this API. .. soft-deprecated:: 3.14 --- ### Doc C Api Hash (`Doc/c-api/hash.rst`) .. highlight:: c PyHash API ---------- See also the :c:member:`PyTypeObject.tp_hash` member and :ref:`numeric-hash`. .. c:type:: Py_hash_t Hash value type: signed integer. .. versionadded:: 3.2 .. c:type:: Py_uhash_t Hash value type: unsigned integer. .. versionadded:: 3.2 .. c:macro:: Py_HASH_ALGORITHM A numerical value indicating the algorithm for hashing of :class:`str`, :class:`bytes`, and :class:`memoryview`. The algorithm name is exposed by :data:`sys.hash_info.algorithm`. .. versionadded:: 3.4 .. c:macro:: Py_HASH_FNV Py_HASH_SIPHASH24 Py_HASH_SIPHASH13 Numerical values to compare to :c:macro:`Py_HASH_ALGORITHM` to determine which algorithm is used for hashing. The hash algorithm can be configured via the configure :option:`--with-hash-algorithm` option. .. versionadded:: 3.4 Add :c:macro:`!Py_HASH_FNV` and :c:macro:`!Py_HASH_SIPHASH24`. .. versionadded:: 3.11 Add :c:macro:`!Py_HASH_SIPHASH13`. .. c:macro:: Py_HASH_CUTOFF Buffers of length in range ``[1, Py_HASH_CUTOFF)`` are hashed using DJBX33A instead of the algorithm described by :c:macro:`Py_HASH_ALGORITHM`. - A :c:macro:`!Py_HASH_CUTOFF` of 0 disables the optimization. - :c:macro:`!Py_HASH_CUTOFF` must be non-negative and less or equal than 7. 32-bit platforms should use a cutoff smaller than 64-bit platforms because it is easier to create colliding strings. A cutoff of 7 on 64-bit platforms and 5 on 32-bit platforms should provide a decent safety margin. This corresponds to the :data:`sys.hash_info.cutoff` constant. .. versionadded:: 3.4 .. c:macro:: PyHASH_MODULUS The `Mersenne prime `_ ``P = 2**n -1``, used for numeric hash scheme. This corresponds to the :data:`sys.hash_info.modulus` constant. .. versionadded:: 3.13 .. c:macro:: PyHASH_BITS The exponent ``n`` of ``P`` in :c:macro:`PyHASH_MODULUS`. .. versionadded:: 3.13 .. c:macro:: PyHASH_MULTIPLIER Prime multiplier used in string and various other hashes. .. versionadded:: 3.13 .. c:macro:: PyHASH_INF The hash value returned for a positive infinity. This corresponds to the :data:`sys.hash_info.inf` constant. .. versionadded:: 3.13 .. c:macro:: PyHASH_IMAG The multiplier used for the imaginary part of a complex number. This corresponds to the :data:`sys.hash_info.imag` constant. .. versionadded:: 3.13 .. c:type:: PyHash_FuncDef Hash function definition used by :c:func:`PyHash_GetFuncDef`. .. c:member:: Py_hash_t (*const hash)(const void *, Py_ssize_t) Hash function. .. c:member:: const char *name Hash function name (UTF-8 encoded string). This corresponds to the :data:`sys.hash_info.algorithm` constant. .. c:member:: const int hash_bits Internal size of the hash value in bits. This corresponds to the :data:`sys.hash_info.hash_bits` constant. .. c:member:: const int seed_bits Size of seed input in bits. This corresponds to the :data:`sys.hash_info.seed_bits` constant. .. versionadded:: 3.4 .. c:function:: PyHash_FuncDef* PyHash_GetFuncDef(void) Get the hash function definition. .. seealso:: :pep:`456` "Secure and interchangeable hash algorithm". .. versionadded:: 3.4 .. c:function:: Py_hash_t Py_HashPointer(const void *ptr) Hash a pointer value: process the pointer value as an integer (cast it to ``uintptr_t`` internally). The pointer is not dereferenced. The function cannot fail: it cannot return ``-1``. .. versionadded:: 3.13 .. c:function:: Py_hash_t Py_HashBuffer(const void *ptr, Py_ssize_t len) Compute and return the hash value of a buffer of *len* bytes starting at address *ptr*. The hash is guaranteed to match that of :class:`bytes`, :class:`memoryview`, and other built-in objects that implement the :ref:`buffer protocol `. Use this function to implement hashing for immutable objects whose :c:member:`~PyTypeObject.tp_richcompare` function compares to another object's buffer. *len* must be greater than or equal to ``0``. This function always succeeds. .. versionadded:: 3.14 .. c:function:: Py_hash_t PyObject_GenericHash(PyObject *obj) Generic hashing function that is meant to be put into a type object's ``tp_hash`` slot. Its result only depends on the object's identity. .. impl-detail:: In CPython, it is equivalent to :c:func:`Py_HashPointer`. .. versionadded:: 3.13 --- ### Doc C Api Import (`Doc/c-api/import.rst`) .. highlight:: c .. _importing: Importing Modules ================= .. c:function:: PyObject* PyImport_ImportModule(const char *name) .. index:: single: package variable; __all__ single: __all__ (package variable) single: modules (in module sys) This is a wrapper around :c:func:`PyImport_Import()` which takes a :c:expr:`const char *` as an argument instead of a :c:expr:`PyObject *`. .. c:function:: PyObject* PyImport_ImportModuleEx(const char *name, PyObject *globals, PyObject *locals, PyObject *fromlist) .. index:: pair: built-in function; __import__ Import a module. This is best described by referring to the built-in Python function :func:`__import__`. The return value is a new reference to the imported module or top-level package, or ``NULL`` with an exception set on failure. Like for :func:`__import__`, the return value when a submodule of a package was requested is normally the top-level package, unless a non-empty *fromlist* was given. Failing imports remove incomplete module objects, like with :c:func:`PyImport_ImportModule`. .. c:function:: PyObject* PyImport_ImportModuleLevelObject(PyObject *name, PyObject *globals, PyObject *locals, PyObject *fromlist, int level) Import a module. This is best described by referring to the built-in Python function :func:`__import__`, as the standard :func:`__import__` function calls this function directly. The return value is a new reference to the imported module or top-level package, or ``NULL`` with an exception set on failure. Like for :func:`__import__`, the return value when a submodule of a package was requested is normally the top-level package, unless a non-empty *fromlist* was given. .. versionadded:: 3.3 .. c:function:: PyObject* PyImport_ImportModuleLevel(const char *name, PyObject *globals, PyObject *locals, PyObject *fromlist, int level) Similar to :c:func:`PyImport_ImportModuleLevelObject`, but the name is a UTF-8 encoded string instead of a Unicode object. .. versionchanged:: 3.3 Negative values for *level* are no longer accepted. .. c:function:: PyObject* PyImport_Import(PyObject *name) This is a higher-level interface that calls the current "import hook function" (with an explicit *level* of 0, meaning absolute import). It invokes the :func:`__import__` function from the ``__builtins__`` of the current globals. This means that the import is done using whatever import hooks are installed in the current environment. This function always uses absolute imports. .. c:function:: PyObject* PyImport_ReloadModule(PyObject *m) Reload a module. Return a new reference to the reloaded module, or ``NULL`` with an exception set on failure (the module still exists in this case). .. c:function:: PyObject* PyImport_AddModuleRef(const char *name) Return the module object corresponding to a module name. The *name* argument may be of the form ``package.module``. First check the modules dictionary if there's one there, and if not, create a new one and insert it in the modules dictionary. Return a :term:`strong reference` to the module on success. Return ``NULL`` with an exception set on failure. The module name *name* is decoded from UTF-8. This function does not load or import the module; if the module wasn't already loaded, you will get an empty module object. Use :c:func:`PyImport_ImportModule` or one of its variants to import a module. Package structures implied by a dotted name for *name* are not created if not already present. .. versionadded:: 3.13 .. c:function:: PyObject* PyImport_AddModuleObject(PyObject *name) Similar to :c:func:`PyImport_AddModuleRef`, but return a :term:`borrowed reference` and *name* is a Python :class:`str` object. .. versionadded:: 3.3 .. c:function:: PyObject* PyImport_AddModule(const char *name) Similar to :c:func:`PyImport_AddModuleRef`, but return a :term:`borrowed reference`. .. c:function:: PyObject* PyImport_ExecCodeModule(const char *name, PyObject *co) .. index:: pair: built-in function; compile Given a module name (possibly of the form ``package.module``) and a code object read from a Python bytecode file or obtained from the built-in function :func:`compile`, load the module. Return a new reference to the module object, or ``NULL`` with an exception set if an error occurred. *name* is removed from :data:`sys.modules` in error cases, even if *name* was already in :data:`sys.modules` on entry to :c:func:`PyImport_ExecCodeModule`. Leaving incompletely initialized modules in :data:`sys.modules` is dangerous, as imports of such modules have no way to know that the module object is an unknown (and probably damaged with respect to the module author's intents) state. The module's :attr:`~module.__spec__` and :attr:`~module.__loader__` will be set, if not set already, with the appropriate values. The spec's loader will be set to the module's :attr:`!__loader__` (if set) and to an instance of :class:`~importlib.machinery.SourceFileLoader` otherwise. The module's :attr:`~module.__file__` attribute will be set to the code object's :attr:`~codeobject.co_filename`. This function will reload the module if it was already imported. See :c:func:`PyImport_ReloadModule` for the intended way to reload a module. If *name* points to a dotted name of the form ``package.module``, any package structures not already created will still not be created. See also :c:func:`PyImport_ExecCodeModuleEx` and :c:func:`PyImport_ExecCodeModuleWithPathnames`. .. versionchanged:: 3.12 The setting of ``__cached__`` and :attr:`~module.__loader__` is deprecated. See :class:`~importlib.machinery.ModuleSpec` for alternatives. .. versionchanged:: 3.15 The ``__cached__`` attribute is no longer set. .. c:function:: PyObject* PyImport_ExecCodeModuleEx(const char *name, PyObject *co, const char *pathname) Like :c:func:`PyImport_ExecCodeModule`, but the :attr:`~module.__file__` attribute of the module object is set to *pathname* if it is non-``NULL``. See also :c:func:`PyImport_ExecCodeModuleWithPathnames`. .. c:function:: PyObject* PyImport_ExecCodeModuleObject(PyObject *name, PyObject *co, PyObject *pathname, PyObject *cpathname) Like :c:func:`PyImport_ExecCodeModuleEx`, but the path to any compiled file via *cpathname* is used appropriately when non-``NULL``. Of the three functions, this is the preferred one to use. .. versionadded:: 3.3 .. versionchanged:: 3.12 Setting ``__cached__`` is deprecated. See :class:`~importlib.machinery.ModuleSpec` for alternatives. .. versionchanged:: 3.15 The ``__cached__`` attribute no longer set. .. c:function:: PyObject* PyImport_ExecCodeModuleWithPathnames(const char *name, PyObject *co, const char *pathname, const char *cpathname) Like :c:func:`PyImport_ExecCodeModuleObject`, but *name*, *pathname* and *cpathname* are UTF-8 encoded strings. Attempts are also made to figure out what the value for *pathname* should be from *cpathname* if the former is set to ``NULL``. .. versionadded:: 3.2 .. versionchanged:: 3.3 Uses :func:`!imp.source_from_cache` in calculating the source path if only the bytecode path is provided. .. versionchanged:: 3.12 No longer uses the removed :mod:`!imp` module. .. c:function:: long PyImport_GetMagicNumber() Return the magic number for Python bytecode files (a.k.a. :file:`.pyc` file). The magic number should be present in the first four bytes of the bytecode file, in little-endian byte order. Returns ``-1`` on error. .. versionchanged:: 3.3 Return value of ``-1`` upon failure. .. c:function:: const char * PyImport_GetMagicTag() Return the magic tag string for :pep:`3147` format Python bytecode file names. Keep in mind that the value at ``sys.implementation.cache_tag`` is authoritative and should be used instead of this function. .. versionadded:: 3.2 .. c:function:: PyObject* PyImport_GetModuleDict() Return the dictionary used for the module administration (a.k.a. ``sys.modules``). Note that this is a per-interpreter variable. .. c:function:: PyObject* PyImport_GetModule(PyObject *name) Return the already imported module with the given name. If the module has not been imported yet then returns ``NULL`` but does not set an error. Returns ``NULL`` and sets an error if the lookup failed. .. versionadded:: 3.7 .. c:function:: PyObject* PyImport_GetImporter(PyObject *path) Return a finder object for a :data:`sys.path`/:attr:`!pkg.__path__` item *path*, possibly by fetching it from the :data:`sys.path_importer_cache` dict. If it wasn't yet cached, traverse :data:`sys.path_hooks` until a hook is found that can handle the path item. Return ``None`` if no hook could; this tells our caller that the :term:`path based finder` could not find a finder for this path item. Cache the result in :data:`sys.path_importer_cache`. Return a new reference to the finder object. .. c:function:: int PyImport_ImportFrozenModuleObject(PyObject *name) Load a frozen module named *name*. Return ``1`` for success, ``0`` if the module is not found, and ``-1`` with an exception set if the initialization failed. To access the imported module on a successful load, use :c:func:`PyImport_ImportModule`. (Note the misnomer --- this function would reload the module if it was already imported.) .. versionadded:: 3.3 .. versionchanged:: 3.4 The ``__file__`` attribute is no longer set on the module. .. c:function:: int PyImport_ImportFrozenModule(const char *name) Similar to :c:func:`PyImport_ImportFrozenModuleObject`, but the name is a UTF-8 encoded string instead of a Unicode object. .. c:struct:: _frozen .. index:: single: freeze utility This is the structure type definition for frozen module descriptors, as generated by the :program:`freeze` utility (see :file:`Tools/freeze/` in the Python source distribution). Its definition, found in :file:`Include/import.h`, is:: struct _frozen { const char *name; const unsigned char *code; int size; bool is_package; }; .. versionchanged:: 3.11 The new ``is_package`` field indicates whether the module is a package or not. This replaces setting the ``size`` field to a negative value. .. c:var:: const struct _frozen* PyImport_FrozenModules This pointer is initialized to point to an array of :c:struct:`_frozen` records, terminated by one whose members are all ``NULL`` or zero. When a frozen module is imported, it is searched in this table. Third-party code could play tricks with this to provide a dynamically created collection of frozen modules. .. c:function:: int PyImport_AppendInittab(const char *name, PyObject* (*initfunc)(void)) Add a single module to the existing table of built-in modules. This is a convenience wrapper around :c:func:`PyImport_ExtendInittab`, returning ``-1`` if the table could not be extended. The new module can be imported by the name *name*, and uses the function *initfunc* as the initialization function called on the first attempted import. This should be called before :c:func:`Py_Initialize`. .. c:struct:: _inittab Structure describing a single entry in the list of built-in modules. Programs which embed Python may use an array of these structures in conjunction with :c:func:`PyImport_ExtendInittab` to provide additional built-in modules. The structure consists of two members: .. c:member:: const char *name The module name, as an ASCII encoded string. .. c:member:: PyObject* (*initfunc)(void) Initialization function for a module built into the interpreter. Note that the inittab uses "``PyInit``" :ref:`initialization functions `; there is currently no way to include "``PyModExport_``" :ref:`export hooks `. .. c:function:: int PyImport_ExtendInittab(struct _inittab *newtab) Add a collection of modules to the table of built-in modules. The *newtab* array must end with a sentinel entry which contains ``NULL`` for the :c:member:`~_inittab.name` field; failure to provide the sentinel value can result in a memory fault. Returns ``0`` on success or ``-1`` if insufficient memory could be allocated to extend the internal table. In the event of failure, no modules are added to the internal table. This must be called before :c:func:`Py_Initialize`. If Python is initialized multiple times, :c:func:`PyImport_AppendInittab` or :c:func:`PyImport_ExtendInittab` must be called before each Python initialization. .. c:var:: struct _inittab *PyImport_Inittab The table of built-in modules used by Python initialization. Do not use this directly; use :c:func:`PyImport_AppendInittab` and :c:func:`PyImport_ExtendInittab` instead. .. c:function:: PyObject* PyImport_ImportModuleAttr(PyObject *mod_name, PyObject *attr_name) Import the module *mod_name* and get its attribute *attr_name*. Names must be Python :class:`str` objects. Helper function combining :c:func:`PyImport_Import` and :c:func:`PyObject_GetAttr`. For example, it can raise :exc:`ImportError` if the module is not found, and :exc:`AttributeError` if the attribute doesn't exist. .. versionadded:: 3.14 .. c:function:: PyObject* PyImport_ImportModuleAttrString(const char *mod_name, const char *attr_name) Similar to :c:func:`PyImport_ImportModuleAttr`, but names are UTF-8 encoded strings instead of Python :class:`str` objects. .. versionadded:: 3.14 .. c:function:: PyImport_LazyImportsMode PyImport_GetLazyImportsMode() Gets the current lazy imports mode. .. versionadded:: 3.15 .. c:function:: PyObject* PyImport_GetLazyImportsFilter() Return a :term:`strong reference` to the current lazy imports filter, or ``NULL`` if none exists. This function always succeeds. .. versionadded:: 3.15 .. c:function:: int PyImport_SetLazyImportsMode(PyImport_LazyImportsMode mode) Similar to :c:func:`PyImport_ImportModuleAttr`, but names are UTF-8 encoded strings instead of Python :class:`str` objects. This function always returns ``0``. .. versionadded:: 3.15 .. c:function:: int PyImport_SetLazyImportsFilter(PyObject *filter) Sets the current lazy imports filter. The *filter* should be a callable that will receive ``(importing_module_name, imported_module_name, [fromlist])`` when an import can potentially be lazy. The ``imported_module_name`` value is the resolved module name, so ``lazy from .spam import eggs`` passes ``package.spam``. The callable must return ``True`` if the import should be lazy and ``False`` otherwise. Return ``0`` on success and ``-1`` with an exception set otherwise. .. versionadded:: 3.15 .. c:type:: PyImport_LazyImportsMode Enumeration of possible lazy import modes. .. c:enumerator:: PyImport_LAZY_NORMAL Respect the ``lazy`` keyword in source code. This is the default mode. .. c:enumerator:: PyImport_LAZY_ALL Make all imports lazy by default. .. versionadded:: 3.15 .. c:function:: PyObject* PyImport_CreateModuleFromInitfunc(PyObject *spec, PyObject* (*initfunc)(void)) This function is a building block that enables embedders to implement the :py:meth:`~importlib.abc.Loader.create_module` step of custom static extension importers (e.g. of statically-linked extensions). *spec* must be a :class:`~importlib.machinery.ModuleSpec` object. *initfunc* must be an :ref:`initialization function `, the same as for :c:func:`PyImport_AppendInittab`. On success, create and return a module object. This module will not be initialized; call :c:func:`PyModule_Exec` to initialize it. (Custom importers should do this in their :py:meth:`~importlib.abc.Loader.exec_module` method.) If the spec's name is dotted and *initfunc* creates the module using only the last component of the name, the module's :attr:`~module.__name__` is set to the full name from the spec. On error, return NULL with an exception set. .. versionadded:: 3.15 --- ### Doc C Api Index (`Doc/c-api/index.rst`) .. _c-api-index: ################################## Python/C API reference manual ################################## This manual documents the API used by C and C++ programmers who want to write extension modules or embed Python. It is a companion to :ref:`extending-index`, which describes the general principles of extension writing but does not document the API functions in detail. .. toctree:: :maxdepth: 2 intro.rst stable.rst veryhigh.rst refcounting.rst exceptions.rst extension-modules.rst slots.rst utilities.rst abstract.rst concrete.rst interp-lifecycle.rst threads.rst synchronization.rst tls.rst subinterpreters.rst profiling.rst init_config.rst memory.rst objimpl.rst apiabiversion.rst monitoring.rst --- ### Doc C Api Init (`Doc/c-api/init.rst`) :orphan: Initialization, finalization, and threads ========================================= This page has been split up into the following: - :ref:`initialization` - :ref:`threads` - :ref:`synchronization` - :ref:`thread-local-storage` - :ref:`sub-interpreter-support` - :ref:`profiling` --- ### Doc C Api Init Config (`Doc/c-api/init_config.rst`) .. highlight:: c .. _init-config: *********************************** Python Initialization Configuration *********************************** .. _pyinitconfig_api: PyInitConfig C API ================== .. versionadded:: 3.14 Python can be initialized with :c:func:`Py_InitializeFromInitConfig`. The :c:func:`Py_RunMain` function can be used to write a customized Python program. See also :ref:`Initialization, Finalization, and Threads `. .. seealso:: :pep:`741` "Python Configuration C API". Example ------- Example of customized Python always running with the :ref:`Python Development Mode ` enabled; return ``-1`` on error: .. code-block:: c int init_python(void) { PyInitConfig *config = PyInitConfig_Create(); if (config == NULL) { printf("PYTHON INIT ERROR: memory allocation failed\n"); return -1; } // Enable the Python Development Mode if (PyInitConfig_SetInt(config, "dev_mode", 1) < 0) { goto error; } // Initialize Python with the configuration if (Py_InitializeFromInitConfig(config) < 0) { goto error; } PyInitConfig_Free(config); return 0; error: { // Display the error message. // // This uncommon braces style is used, because you cannot make // goto targets point to variable declarations. const char *err_msg; (void)PyInitConfig_GetError(config, &err_msg); printf("PYTHON INIT ERROR: %s\n", err_msg); PyInitConfig_Free(config); return -1; } } Create Config ------------- .. c:struct:: PyInitConfig Opaque structure to configure the Python initialization. .. c:function:: PyInitConfig* PyInitConfig_Create(void) Create a new initialization configuration using :ref:`Isolated Configuration ` default values. It must be freed by :c:func:`PyInitConfig_Free`. Return ``NULL`` on memory allocation failure. .. c:function:: void PyInitConfig_Free(PyInitConfig *config) Free memory of the initialization configuration *config*. If *config* is ``NULL``, no operation is performed. Error Handling -------------- .. c:function:: int PyInitConfig_GetError(PyInitConfig* config, const char **err_msg) Get the *config* error message. * Set *\*err_msg* and return ``1`` if an error is set. * Set *\*err_msg* to ``NULL`` and return ``0`` otherwise. An error message is a UTF-8 encoded string. If *config* has an exit code, format the exit code as an error message. The error message remains valid until another ``PyInitConfig`` function is called with *config*. The caller doesn't have to free the error message. .. c:function:: int PyInitConfig_GetExitCode(PyInitConfig* config, int *exitcode) Get the *config* exit code. Return ``0`` and leave *\*exitcode* unchanged. In Python 3.15, :c:func:`Py_InitializeFromInitConfig` sets an exit code if a command line option wants to exit Python. This is no longer the case in Python 3.16. Instead, the option is now processed in :c:func:`Py_RunMain`. This function became useless. .. deprecated:: 3.16 Get Options ----------- The configuration option *name* parameter must be a non-NULL null-terminated UTF-8 encoded string. See :ref:`Configuration Options `. .. c:function:: int PyInitConfig_HasOption(PyInitConfig *config, const char *name) Test if the configuration has an option called *name*. Return ``1`` if the option exists, or return ``0`` otherwise. .. c:function:: int PyInitConfig_GetInt(PyInitConfig *config, const char *name, int64_t *value) Get an integer configuration option. * Set *\*value*, and return ``0`` on success. * Set an error in *config* and return ``-1`` on error. .. c:function:: int PyInitConfig_GetStr(PyInitConfig *config, const char *name, char **value) Get a string configuration option as a null-terminated UTF-8 encoded string. * Set *\*value*, and return ``0`` on success. * Set an error in *config* and return ``-1`` on error. *\*value* can be set to ``NULL`` if the option is an optional string and the option is unset. On success, the string must be released with ``free(value)`` if it's not ``NULL``. .. c:function:: int PyInitConfig_GetStrList(PyInitConfig *config, const char *name, size_t *length, char ***items) Get a string list configuration option as an array of null-terminated UTF-8 encoded strings. * Set *\*length* and *\*value*, and return ``0`` on success. * Set an error in *config* and return ``-1`` on error. On success, the string list must be released with ``PyInitConfig_FreeStrList(length, items)``. .. c:function:: void PyInitConfig_FreeStrList(size_t length, char **items) Free memory of a string list created by ``PyInitConfig_GetStrList()``. Set Options ----------- The configuration option *name* parameter must be a non-NULL null-terminated UTF-8 encoded string. See :ref:`Configuration Options `. Some configuration options have side effects on other options. This logic is only implemented when ``Py_InitializeFromInitConfig()`` is called, not by the "Set" functions below. For example, setting ``dev_mode`` to ``1`` does not set ``faulthandler`` to ``1``. .. c:function:: int PyInitConfig_SetInt(PyInitConfig *config, const char *name, int64_t value) Set an integer configuration option. * Return ``0`` on success. * Set an error in *config* and return ``-1`` on error. .. c:function:: int PyInitConfig_SetStr(PyInitConfig *config, const char *name, const char *value) Set a string configuration option from a null-terminated UTF-8 encoded string. The string is copied. * Return ``0`` on success. * Set an error in *config* and return ``-1`` on error. .. c:function:: int PyInitConfig_SetStrList(PyInitConfig *config, const char *name, size_t length, char * const *items) Set a string list configuration option from an array of null-terminated UTF-8 encoded strings. The string list is copied. * Return ``0`` on success. * Set an error in *config* and return ``-1`` on error. Module ------ .. c:function:: int PyInitConfig_AddModule(PyInitConfig *config, const char *name, PyObject* (*initfunc)(void)) Add a built-in extension module to the table of built-in modules. The new module can be imported by the name *name*, and uses the function *initfunc* as the initialization function called on the first attempted import. * Return ``0`` on success. * Set an error in *config* and return ``-1`` on error. If Python is initialized multiple times, ``PyInitConfig_AddModule()`` must be called at each Python initialization. Similar to the :c:func:`PyImport_AppendInittab` function. Initialize Python ----------------- .. c:function:: int Py_InitializeFromInitConfig(PyInitConfig *config) Initialize Python from the initialization configuration. * Return ``0`` on success. * Set an error in *config* and return ``-1`` on error. .. versionchanged:: 3.16 The function no longer sets an exit code if a command line option wants to exit Python. Instead, the option is now processed in :c:func:`Py_RunMain`. .. _pyinitconfig-opts: Configuration Options ===================== .. list-table:: :header-rows: 1 * - Option - PyConfig/PyPreConfig member - Type - Visibility * - ``"allocator"`` - :c:member:`allocator ` - ``int`` - Read-only * - ``"argv"`` - :c:member:`argv ` - ``list[str]`` - Public * - ``"base_exec_prefix"`` - :c:member:`base_exec_prefix ` - ``str`` - Public * - ``"base_executable"`` - :c:member:`base_executable ` - ``str`` - Public * - ``"base_prefix"`` - :c:member:`base_prefix ` - ``str`` - Public * - ``"buffered_stdio"`` - :c:member:`buffered_stdio ` - ``bool`` - Read-only * - ``"bytes_warning"`` - :c:member:`bytes_warning ` - ``int`` - Public * - ``"check_hash_pycs_mode"`` - :c:member:`check_hash_pycs_mode ` - ``str`` - Read-only * - ``"code_debug_ranges"`` - :c:member:`code_debug_ranges ` - ``bool`` - Read-only * - ``"coerce_c_locale"`` - :c:member:`coerce_c_locale ` - ``bool`` - Read-only * - ``"coerce_c_locale_warn"`` - :c:member:`coerce_c_locale_warn ` - ``bool`` - Read-only * - ``"configure_c_stdio"`` - :c:member:`configure_c_stdio ` - ``bool`` - Read-only * - ``"configure_locale"`` - :c:member:`configure_locale ` - ``bool`` - Read-only * - ``"cpu_count"`` - :c:member:`cpu_count ` - ``int`` - Public * - ``"dev_mode"`` - :c:member:`dev_mode ` - ``bool`` - Read-only * - ``"dump_refs"`` - :c:member:`dump_refs ` - ``bool`` - Read-only * - ``"dump_refs_file"`` - :c:member:`dump_refs_file ` - ``str`` - Read-only * - ``"exec_prefix"`` - :c:member:`exec_prefix ` - ``str`` - Public * - ``"executable"`` - :c:member:`executable ` - ``str`` - Public * - ``"faulthandler"`` - :c:member:`faulthandler ` - ``bool`` - Read-only * - ``"filesystem_encoding"`` - :c:member:`filesystem_encoding ` - ``str`` - Read-only * - ``"filesystem_errors"`` - :c:member:`filesystem_errors ` - ``str`` - Read-only * - ``"hash_seed"`` - :c:member:`hash_seed ` - ``int`` - Read-only * - ``"home"`` - :c:member:`home ` - ``str`` - Read-only * - ``"import_time"`` - :c:member:`import_time ` - ``int`` - Read-only * - ``"inspect"`` - :c:member:`inspect ` - ``bool`` - Public * - ``"install_signal_handlers"`` - :c:member:`install_signal_handlers ` - ``bool`` - Read-only * - ``"int_max_str_digits"`` - :c:member:`int_max_str_digits ` - ``int`` - Public * - ``"interactive"`` - :c:member:`interactive ` - ``bool`` - Public * - ``"isolated"`` - :c:member:`isolated ` - ``bool`` - Read-only * - ``"legacy_windows_fs_encoding"`` - :c:member:`legacy_windows_fs_encoding ` - ``bool`` - Read-only * - ``"legacy_windows_stdio"`` - :c:member:`legacy_windows_stdio ` - ``bool`` - Read-only * - ``"malloc_stats"`` - :c:member:`malloc_stats ` - ``bool`` - Read-only * - ``"module_search_paths"`` - :c:member:`module_search_paths ` - ``list[str]`` - Public * - ``"optimization_level"`` - :c:member:`optimization_level ` - ``int`` - Public * - ``"orig_argv"`` - :c:member:`orig_argv ` - ``list[str]`` - Read-only * - ``"parse_argv"`` - :c:member:`parse_argv ` - ``bool`` - Read-only * - ``"parser_debug"`` - :c:member:`parser_debug ` - ``bool`` - Public * - ``"pathconfig_warnings"`` - :c:member:`pathconfig_warnings ` - ``bool`` - Read-only * - ``"perf_profiling"`` - :c:member:`perf_profiling ` - ``bool`` - Read-only * - ``"platlibdir"`` - :c:member:`platlibdir ` - ``str`` - Public * - ``"prefix"`` - :c:member:`prefix ` - ``str`` - Public * - ``"program_name"`` - :c:member:`program_name ` - ``str`` - Read-only * - ``"pycache_prefix"`` - :c:member:`pycache_prefix ` - ``str`` - Public * - ``"quiet"`` - :c:member:`quiet ` - ``bool`` - Public * - ``"run_command"`` - :c:member:`run_command ` - ``str`` - Read-only * - ``"run_filename"`` - :c:member:`run_filename ` - ``str`` - Read-only * - ``"run_module"`` - :c:member:`run_module ` - ``str`` - Read-only * - ``"run_presite"`` - :c:member:`run_presite ` - ``str`` - Read-only * - ``"safe_path"`` - :c:member:`safe_path ` - ``bool`` - Read-only * - ``"show_ref_count"`` - :c:member:`show_ref_count ` - ``bool`` - Read-only * - ``"site_import"`` - :c:member:`site_import ` - ``bool`` - Read-only * - ``"skip_source_first_line"`` - :c:member:`skip_source_first_line ` - ``bool`` - Read-only * - ``"stdio_encoding"`` - :c:member:`stdio_encoding ` - ``str`` - Read-only * - ``"stdio_errors"`` - :c:member:`stdio_errors ` - ``str`` - Read-only * - ``"stdlib_dir"`` - :c:member:`stdlib_dir ` - ``str`` - Public * - ``"tracemalloc"`` - :c:member:`tracemalloc ` - ``int`` - Read-only * - ``"use_environment"`` - :c:member:`use_environment ` - ``bool`` - Public * - ``"use_frozen_modules"`` - :c:member:`use_frozen_modules ` - ``bool`` - Read-only * - ``"use_hash_seed"`` - :c:member:`use_hash_seed ` - ``bool`` - Read-only * - ``"use_system_logger"`` - :c:member:`use_system_logger ` - ``bool`` - Read-only * - ``"user_site_directory"`` - :c:member:`user_site_directory ` - ``bool`` - Read-only * - ``"utf8_mode"`` - :c:member:`utf8_mode ` - ``bool`` - Read-only * - ``"verbose"`` - :c:member:`verbose ` - ``int`` - Public * - ``"warn_default_encoding"`` - :c:member:`warn_default_encoding ` - ``bool`` - Read-only * - ``"warnoptions"`` - :c:member:`warnoptions ` - ``list[str]`` - Public * - ``"write_bytecode"`` - :c:member:`write_bytecode ` - ``bool`` - Public * - ``"xoptions"`` - :c:member:`xoptions ` - ``dict[str, str]`` - Public * - ``"_pystats"`` - :c:member:`_pystats ` - ``bool`` - Read-only Visibility: * Public: Can be retrieved by :c:func:`PyConfig_Get` and set by :c:func:`PyConfig_Set`. * Read-only: Can be retrieved by :c:func:`PyConfig_Get`, but cannot be set by :c:func:`PyConfig_Set`. Runtime Python configuration API ================================ At runtime, it's possible to get and set configuration options using :c:func:`PyConfig_Get` and :c:func:`PyConfig_Set` functions. The configuration option *name* parameter must be a non-NULL null-terminated UTF-8 encoded string. See :ref:`Configuration Options `. Some options are read from the :mod:`sys` attributes. For example, the option ``"argv"`` is read from :data:`sys.argv`. .. c:function:: PyObject* PyConfig_Get(const char *name) Get the current runtime value of a configuration option as a Python object. * Return a new reference on success. * Set an exception and return ``NULL`` on error. The object type depends on the configuration option. It can be: * ``bool`` * ``int`` * ``str`` * ``list[str]`` * ``dict[str, str]`` The caller must have an :term:`attached thread state`. The function cannot be called before Python initialization nor after Python finalization. .. versionadded:: 3.14 .. c:function:: int PyConfig_GetInt(const char *name, int *value) Similar to :c:func:`PyConfig_Get`, but get the value as a C int. * Return ``0`` on success. * Set an exception and return ``-1`` on error. .. versionadded:: 3.14 .. c:function:: PyObject* PyConfig_Names(void) Get all configuration option names as a ``frozenset``. * Return a new reference on success. * Set an exception and return ``NULL`` on error. The caller must have an :term:`attached thread state`. The function cannot be called before Python initialization nor after Python finalization. .. versionadded:: 3.14 .. c:function:: int PyConfig_Set(const char *name, PyObject *value) Set the current runtime value of a configuration option. * Raise a :exc:`ValueError` if there is no option *name*. * Raise a :exc:`ValueError` if *value* is an invalid value. * Raise a :exc:`ValueError` if the option is read-only (cannot be set). * Raise a :exc:`TypeError` if *value* has not the proper type. The caller must have an :term:`attached thread state`. The function cannot be called before Python initialization nor after Python finalization. .. audit-event:: cpython.PyConfig_Set name,value c.PyConfig_Set .. versionadded:: 3.14 .. versionchanged:: 3.16 The function now replaces :data:`sys.flags` (create a new object), instead of modifying :data:`sys.flags` in-place. .. _pyconfig_api: PyConfig C API ============== .. versionadded:: 3.8 Python can be initialized with :c:func:`Py_InitializeFromConfig` and the :c:type:`PyConfig` structure. It can be preinitialized with :c:func:`Py_PreInitialize` and the :c:type:`PyPreConfig` structure. There are two kinds of configuration: * The :ref:`Python Configuration ` can be used to build a customized Python which behaves as the regular Python. For example, environment variables and command line arguments are used to configure Python. * The :ref:`Isolated Configuration ` can be used to embed Python into an application. It isolates Python from the system. For example, environment variables are ignored, the LC_CTYPE locale is left unchanged and no signal handler is registered. The :c:func:`Py_RunMain` function can be used to write a customized Python program. See also :ref:`Initialization, Finalization, and Threads `. .. seealso:: :pep:`587` "Python Initialization Configuration". Example ------- Example of customized Python always running in isolated mode:: int main(int argc, char **argv) { PyStatus status; PyConfig config; PyConfig_InitPythonConfig(&config); config.isolated = 1; /* Decode command line arguments. Implicitly preinitialize Python (in isolated mode). */ status = PyConfig_SetBytesArgv(&config, argc, argv); if (PyStatus_Exception(status)) { goto exception; } status = Py_InitializeFromConfig(&config); if (PyStatus_Exception(status)) { goto exception; } PyConfig_Clear(&config); return Py_RunMain(); exception: PyConfig_Clear(&config); /* Display the error message and exit the process with non-zero exit code */ Py_ExitStatusException(status); } PyWideStringList ---------------- .. c:type:: PyWideStringList List of ``wchar_t*`` strings. If *length* is non-zero, *items* must be non-``NULL`` and all strings must be non-``NULL``. .. c:namespace:: NULL Methods: .. c:function:: PyStatus PyWideStringList_Append(PyWideStringList *list, const wchar_t *item) Append *item* to *list*. Python must be preinitialized to call this function. .. c:function:: PyStatus PyWideStringList_Insert(PyWideStringList *list, Py_ssize_t index, const wchar_t *item) Insert *item* into *list* at *index*. If *index* is greater than or equal to *list* length, append *item* to *list*. *index* must be greater than or equal to ``0``. Python must be preinitialized to call this function. .. c:namespace:: PyWideStringList Structure fields: .. c:member:: Py_ssize_t length List length. .. c:member:: wchar_t** items List items. PyStatus -------- .. c:type:: PyStatus Structure to store an initialization function status: success, error or exit. For an error, it can store the C function name which created the error. Structure fields: .. c:member:: int exitcode Exit code. Argument passed to ``exit()``. .. deprecated:: 3.16 .. c:member:: const char *err_msg Error message. .. c:member:: const char *func Name of the function which created an error, can be ``NULL``. .. c:namespace:: NULL Functions to create a status: .. c:function:: PyStatus PyStatus_Ok(void) Success. .. c:function:: PyStatus PyStatus_Error(const char *err_msg) Initialization error with a message. *err_msg* must not be ``NULL``. .. c:function:: PyStatus PyStatus_NoMemory(void) Memory allocation failure (out of memory). .. c:function:: PyStatus PyStatus_Exit(int exitcode) Exit Python with the specified exit code. .. deprecated:: 3.16 :c:func:`Py_InitializeFromConfig` no longer sets an exit code if a command line option wants to exit Python. Instead, the option is now processed in :c:func:`Py_RunMain`. Functions to handle a status: .. c:function:: int PyStatus_Exception(PyStatus status) Is the status an error or an exit? If true, the exception must be handled; by calling :c:func:`Py_ExitStatusException` for example. .. c:function:: int PyStatus_IsError(PyStatus status) Is the result an error? .. c:function:: int PyStatus_IsExit(PyStatus status) Is the result an exit? .. deprecated:: 3.16 :c:func:`Py_InitializeFromConfig` no longer sets an exit code if a command line option wants to exit Python. Instead, the option is now processed in :c:func:`Py_RunMain`. .. c:function:: void Py_ExitStatusException(PyStatus status) Call ``exit(exitcode)`` if *status* is an exit. Print the error message and exit with a non-zero exit code if *status* is an error. Must only be called if ``PyStatus_Exception(status)`` is non-zero. .. note:: Internally, Python uses macros which set ``PyStatus.func``, whereas functions to create a status set ``func`` to ``NULL``. Example:: PyStatus alloc(void **ptr, size_t size) { *ptr = PyMem_RawMalloc(size); if (*ptr == NULL) { return PyStatus_NoMemory(); } return PyStatus_Ok(); } int main(int argc, char **argv) { void *ptr; PyStatus status = alloc(&ptr, 16); if (PyStatus_Exception(status)) { Py_ExitStatusException(status); } PyMem_Free(ptr); return 0; } PyPreConfig ----------- .. c:type:: PyPreConfig Structure used to preinitialize Python. .. c:namespace:: NULL Function to initialize a preconfiguration: .. c:function:: void PyPreConfig_InitPythonConfig(PyPreConfig *preconfig) Initialize the preconfiguration with :ref:`Python Configuration `. .. c:function:: void PyPreConfig_InitIsolatedConfig(PyPreConfig *preconfig) Initialize the preconfiguration with :ref:`Isolated Configuration `. .. c:namespace:: PyPreConfig Structure fields: .. c:member:: int allocator Name of the Python memory allocators: * ``PYMEM_ALLOCATOR_NOT_SET`` (``0``): don't change memory allocators (use defaults). * ``PYMEM_ALLOCATOR_DEFAULT`` (``1``): :ref:`default memory allocators `. * ``PYMEM_ALLOCATOR_DEBUG`` (``2``): :ref:`default memory allocators ` with :ref:`debug hooks `. * ``PYMEM_ALLOCATOR_MALLOC`` (``3``): use ``malloc()`` of the C library. * ``PYMEM_ALLOCATOR_MALLOC_DEBUG`` (``4``): force usage of ``malloc()`` with :ref:`debug hooks `. * ``PYMEM_ALLOCATOR_PYMALLOC`` (``5``): :ref:`Python pymalloc memory allocator `. * ``PYMEM_ALLOCATOR_PYMALLOC_DEBUG`` (``6``): :ref:`Python pymalloc memory allocator ` with :ref:`debug hooks `. * ``PYMEM_ALLOCATOR_MIMALLOC`` (``6``): use ``mimalloc``, a fast malloc replacement. * ``PYMEM_ALLOCATOR_MIMALLOC_DEBUG`` (``7``): use ``mimalloc``, a fast malloc replacement with :ref:`debug hooks `. ``PYMEM_ALLOCATOR_PYMALLOC`` and ``PYMEM_ALLOCATOR_PYMALLOC_DEBUG`` are not supported if Python is :option:`configured using --without-pymalloc <--without-pymalloc>`. ``PYMEM_ALLOCATOR_MIMALLOC`` and ``PYMEM_ALLOCATOR_MIMALLOC_DEBUG`` are not supported if Python is :option:`configured using --without-mimalloc <--without-mimalloc>` or if the underlying atomic support isn't available. See :ref:`Memory Management `. Default: ``PYMEM_ALLOCATOR_NOT_SET``. .. c:member:: int configure_locale Set the LC_CTYPE locale to the user preferred locale. If equals to ``0``, set :c:member:`~PyPreConfig.coerce_c_locale` and :c:member:`~PyPreConfig.coerce_c_locale_warn` members to ``0``. See the :term:`locale encoding`. Default: ``1`` in Python config, ``0`` in isolated config. .. c:member:: int coerce_c_locale If equals to ``2``, coerce the C locale. If equals to ``1``, read the LC_CTYPE locale to decide if it should be coerced. See the :term:`locale encoding`. Default: ``-1`` in Python config, ``0`` in isolated config. .. c:member:: int coerce_c_locale_warn If non-zero, emit a warning if the C locale is coerced. Default: ``-1`` in Python config, ``0`` in isolated config. .. c:member:: int dev_mode :ref:`Python Development Mode `: see :c:member:`PyConfig.dev_mode`. Default: ``-1`` in Python mode, ``0`` in isolated mode. .. c:member:: int isolated Isolated mode: see :c:member:`PyConfig.isolated`. Default: ``0`` in Python mode, ``1`` in isolated mode. .. c:member:: int legacy_windows_fs_encoding If non-zero: * Set :c:member:`PyPreConfig.utf8_mode` to ``0``, * Set :c:member:`PyConfig.filesystem_encoding` to ``"mbcs"``, * Set :c:member:`PyConfig.filesystem_errors` to ``"replace"``. Initialized from the :envvar:`PYTHONLEGACYWINDOWSFSENCODING` environment variable value. Only available on Windows. ``#ifdef MS_WINDOWS`` macro can be used for Windows specific code. Default: ``0``. .. c:member:: int parse_argv If non-zero, :c:func:`Py_PreInitializeFromArgs` and :c:func:`Py_PreInitializeFromBytesArgs` parse their ``argv`` argument the same way the regular Python parses command line arguments: see :ref:`Command Line Arguments `. Default: ``1`` in Python config, ``0`` in isolated config. .. c:member:: int use_environment Use :ref:`environment variables `? See :c:member:`PyConfig.use_environment`. Default: ``1`` in Python config and ``0`` in isolated config. .. c:member:: int utf8_mode If non-zero, enable the :ref:`Python UTF-8 Mode `. Set to ``0`` or ``1`` by the :option:`-X utf8 <-X>` command line option and the :envvar:`PYTHONUTF8` environment variable. Default: ``1``. .. _c-preinit: Preinitialize Python with PyPreConfig ------------------------------------- The preinitialization of Python: * Set the Python memory allocators (:c:member:`PyPreConfig.allocator`) * Configure the LC_CTYPE locale (:term:`locale encoding`) * Set the :ref:`Python UTF-8 Mode ` (:c:member:`PyPreConfig.utf8_mode`) The current preconfiguration (``PyPreConfig`` type) is stored in ``_PyRuntime.preconfig``. Functions to preinitialize Python: .. c:function:: PyStatus Py_PreInitialize(const PyPreConfig *preconfig) Preinitialize Python from *preconfig* preconfiguration. *preconfig* must not be ``NULL``. .. c:function:: PyStatus Py_PreInitializeFromBytesArgs(const PyPreConfig *preconfig, int argc, char * const *argv) Preinitialize Python from *preconfig* preconfiguration. Parse *argv* command line arguments (bytes strings) if :c:member:`~PyPreConfig.parse_argv` of *preconfig* is non-zero. *preconfig* must not be ``NULL``. .. c:function:: PyStatus Py_PreInitializeFromArgs(const PyPreConfig *preconfig, int argc, wchar_t * const * argv) Preinitialize Python from *preconfig* preconfiguration. Parse *argv* command line arguments (wide strings) if :c:member:`~PyPreConfig.parse_argv` of *preconfig* is non-zero. *preconfig* must not be ``NULL``. The caller is responsible to handle exceptions (error or exit) using :c:func:`PyStatus_Exception` and :c:func:`Py_ExitStatusException`. For :ref:`Python Configuration ` (:c:func:`PyPreConfig_InitPythonConfig`), if Python is initialized with command line arguments, the command line arguments must also be passed to preinitialize Python, since they have an effect on the pre-configuration like encodings. For example, the :option:`-X utf8 <-X>` command line option enables the :ref:`Python UTF-8 Mode `. ``PyMem_SetAllocator()`` can be called after :c:func:`Py_PreInitialize` and before :c:func:`Py_InitializeFromConfig` to install a custom memory allocator. It can be called before :c:func:`Py_PreInitialize` if :c:member:`PyPreConfig.allocator` is set to ``PYMEM_ALLOCATOR_NOT_SET``. Python memory allocation functions like :c:func:`PyMem_RawMalloc` must not be used before the Python preinitialization, whereas calling directly ``malloc()`` and ``free()`` is always safe. :c:func:`Py_DecodeLocale` must not be called before the Python preinitialization. Example using the preinitialization to enable the :ref:`Python UTF-8 Mode `:: PyStatus status; PyPreConfig preconfig; PyPreConfig_InitPythonConfig(&preconfig); preconfig.utf8_mode = 1; status = Py_PreInitialize(&preconfig); if (PyStatus_Exception(status)) { Py_ExitStatusException(status); } /* at this point, Python speaks UTF-8 */ Py_Initialize(); /* ... use Python API here ... */ Py_Finalize(); PyConfig -------- .. c:type:: PyConfig Structure containing most parameters to configure Python. When done, the :c:func:`PyConfig_Clear` function must be used to release the configuration memory. .. c:namespace:: NULL Structure methods: .. c:function:: void PyConfig_InitPythonConfig(PyConfig *config) Initialize configuration with the :ref:`Python Configuration `. .. c:function:: void PyConfig_InitIsolatedConfig(PyConfig *config) Initialize configuration with the :ref:`Isolated Configuration `. .. c:function:: PyStatus PyConfig_SetString(PyConfig *config, wchar_t * const *config_str, const wchar_t *str) Copy the wide character string *str* into ``*config_str``. :ref:`Preinitialize Python ` if needed. .. c:function:: PyStatus PyConfig_SetBytesString(PyConfig *config, wchar_t * const *config_str, const char *str) Decode *str* using :c:func:`Py_DecodeLocale` and set the result into ``*config_str``. :ref:`Preinitialize Python ` if needed. .. c:function:: PyStatus PyConfig_SetArgv(PyConfig *config, int argc, wchar_t * const *argv) Set command line arguments (:c:member:`~PyConfig.argv` member of *config*) from the *argv* list of wide character strings. :ref:`Preinitialize Python ` if needed. .. c:function:: PyStatus PyConfig_SetBytesArgv(PyConfig *config, int argc, char * const *argv) Set command line arguments (:c:member:`~PyConfig.argv` member of *config*) from the *argv* list of bytes strings. Decode bytes using :c:func:`Py_DecodeLocale`. :ref:`Preinitialize Python ` if needed. .. c:function:: PyStatus PyConfig_SetWideStringList(PyConfig *config, PyWideStringList *list, Py_ssize_t length, wchar_t **items) Set the list of wide strings *list* to *length* and *items*. :ref:`Preinitialize Python ` if needed. .. c:function:: PyStatus PyConfig_Read(PyConfig *config) Read all Python configuration. Fields which are already initialized are left unchanged. Fields for :ref:`path configuration ` are no longer calculated or modified when calling this function, as of Python 3.11. The :c:func:`PyConfig_Read` function only parses :c:member:`PyConfig.argv` arguments once: :c:member:`PyConfig.parse_argv` is set to ``2`` after arguments are parsed. Since Python arguments are stripped from :c:member:`PyConfig.argv`, parsing arguments twice would parse the application options as Python options. :ref:`Preinitialize Python ` if needed. .. versionchanged:: 3.10 The :c:member:`PyConfig.argv` arguments are now only parsed once, :c:member:`PyConfig.parse_argv` is set to ``2`` after arguments are parsed, and arguments are only parsed if :c:member:`PyConfig.parse_argv` equals ``1``. .. versionchanged:: 3.11 :c:func:`PyConfig_Read` no longer calculates all paths, and so fields listed under :ref:`Python Path Configuration ` may no longer be updated until :c:func:`Py_InitializeFromConfig` is called. .. c:function:: void PyConfig_Clear(PyConfig *config) Release configuration memory. Most ``PyConfig`` methods :ref:`preinitialize Python ` if needed. In that case, the Python preinitialization configuration (:c:type:`PyPreConfig`) is based on the :c:type:`PyConfig`. If configuration fields which are in common with :c:type:`PyPreConfig` are tuned, they must be set before calling a :c:type:`PyConfig` method: * :c:member:`PyConfig.dev_mode` * :c:member:`PyConfig.isolated` * :c:member:`PyConfig.parse_argv` * :c:member:`PyConfig.use_environment` Moreover, if :c:func:`PyConfig_SetArgv` or :c:func:`PyConfig_SetBytesArgv` is used, this method must be called before other methods, since the preinitialization configuration depends on command line arguments (if :c:member:`~PyConfig.parse_argv` is non-zero). The caller of these methods is responsible to handle exceptions (error or exit) using ``PyStatus_Exception()`` and ``Py_ExitStatusException()``. .. c:namespace:: PyConfig Structure fields: .. c:member:: PyWideStringList argv .. index:: single: main() single: argv (in module sys) Set :data:`sys.argv` command line arguments based on :c:member:`~PyConfig.argv`. These parameters are similar to those passed to the program's :c:func:`main` function with the difference that the first entry should refer to the script file to be executed rather than the executable hosting the Python interpreter. If there isn't a script that will be run, the first entry in :c:member:`~PyConfig.argv` can be an empty string. Set :c:member:`~PyConfig.parse_argv` to ``1`` to parse :c:member:`~PyConfig.argv` the same way the regular Python parses Python command line arguments and then to strip Python arguments from :c:member:`~PyConfig.argv`. If :c:member:`~PyConfig.argv` is empty, an empty string is added to ensure that :data:`sys.argv` always exists and is never empty. Default: ``NULL``. See also the :c:member:`~PyConfig.orig_argv` member. .. c:member:: int safe_path If equals to zero, ``Py_RunMain()`` prepends a potentially unsafe path to :data:`sys.path` at startup: * If :c:member:`argv[0] ` is equal to ``L"-m"`` (``python -m module``), prepend the current working directory. * If running a script (``python script.py``), prepend the script's directory. If it's a symbolic link, resolve symbolic links. * Otherwise (``python -c code`` and ``python``), prepend an empty string, which means the current working directory. Set to ``1`` by the :option:`-P` command line option and the :envvar:`PYTHONSAFEPATH` environment variable. Default: ``0`` in Python config, ``1`` in isolated config. .. versionadded:: 3.11 .. c:member:: wchar_t* base_exec_prefix :data:`sys.base_exec_prefix`. Default: ``NULL``. Part of the :ref:`Python Path Configuration ` output. See also :c:member:`PyConfig.exec_prefix`. .. c:member:: wchar_t* base_executable Python base executable: ``sys._base_executable``. Set by the ``__PYVENV_LAUNCHER__`` environment variable. Set from :c:member:`PyConfig.executable` if ``NULL``. Default: ``NULL``. Part of the :ref:`Python Path Configuration ` output. See also :c:member:`PyConfig.executable`. .. c:member:: wchar_t* base_prefix :data:`sys.base_prefix`. Default: ``NULL``. Part of the :ref:`Python Path Configuration ` output. See also :c:member:`PyConfig.prefix`. .. c:member:: int buffered_stdio If equals to ``0`` and :c:member:`~PyConfig.configure_c_stdio` is non-zero, disable buffering on the C streams stdout and stderr. Set to ``0`` by the :option:`-u` command line option and the :envvar:`PYTHONUNBUFFERED` environment variable. stdin is always opened in buffered mode. Default: ``1``. .. c:member:: int bytes_warning If equals to ``1``, issue a warning when comparing :class:`bytes` or :class:`bytearray` with :class:`str`, or comparing :class:`bytes` with :class:`int`. If equal or greater to ``2``, raise a :exc:`BytesWarning` exception in these cases. Incremented by the :option:`-b` command line option. Default: ``0``. .. deprecated-removed:: 3.15 3.17 The :option:`-b` and :option:`!-bb` options will become no-op in 3.17. :c:member:`~PyConfig.bytes_warning` member will be removed in 3.17. .. c:member:: int warn_default_encoding If non-zero, emit a :exc:`EncodingWarning` warning when :class:`io.TextIOWrapper` uses its default encoding. See :ref:`io-encoding-warning` for details. Default: ``0``. .. versionadded:: 3.10 .. c:member:: int code_debug_ranges If equals to ``0``, disables the inclusion of the end line and column mappings in code objects. Also disables traceback printing carets to specific error locations. Set to ``0`` by the :envvar:`PYTHONNODEBUGRANGES` environment variable and by the :option:`-X no_debug_ranges <-X>` command line option. Default: ``1``. .. versionadded:: 3.11 .. c:member:: wchar_t* check_hash_pycs_mode Control the validation behavior of hash-based ``.pyc`` files: value of the :option:`--check-hash-based-pycs` command line option. Valid values: - ``L"always"``: Hash the source file for invalidation regardless of value of the 'check_source' flag. - ``L"never"``: Assume that hash-based pycs always are valid. - ``L"default"``: The 'check_source' flag in hash-based pycs determines invalidation. Default: ``L"default"``. See also :pep:`552` "Deterministic pycs". .. c:member:: int configure_c_stdio If non-zero, configure C standard streams: * On Windows, set the binary mode (``O_BINARY``) on stdin, stdout and stderr. * If :c:member:`~PyConfig.buffered_stdio` equals zero, disable buffering of stdin, stdout and stderr streams. * If :c:member:`~PyConfig.interactive` is non-zero, enable stream buffering on stdin and stdout (only stdout on Windows). Default: ``1`` in Python config, ``0`` in isolated config. .. c:member:: int dev_mode If non-zero, enable the :ref:`Python Development Mode `. Set to ``1`` by the :option:`-X dev <-X>` option and the :envvar:`PYTHONDEVMODE` environment variable. Default: ``-1`` in Python mode, ``0`` in isolated mode. .. c:member:: int dump_refs Dump Python references? If non-zero, dump all objects which are still alive at exit. Set to ``1`` by the :envvar:`PYTHONDUMPREFS` environment variable. Needs a special build of Python with the ``Py_TRACE_REFS`` macro defined: see the :option:`configure --with-trace-refs option <--with-trace-refs>`. Default: ``0``. .. c:member:: wchar_t* dump_refs_file Filename where to dump Python references. Set by the :envvar:`PYTHONDUMPREFSFILE` environment variable. Default: ``NULL``. .. versionadded:: 3.11 .. c:member:: wchar_t* exec_prefix The site-specific directory prefix where the platform-dependent Python files are installed: :data:`sys.exec_prefix`. Default: ``NULL``. Part of the :ref:`Python Path Configuration ` output. See also :c:member:`PyConfig.base_exec_prefix`. .. c:member:: wchar_t* executable The absolute path of the executable binary for the Python interpreter: :data:`sys.executable`. Default: ``NULL``. Part of the :ref:`Python Path Configuration ` output. See also :c:member:`PyConfig.base_executable`. .. c:member:: int faulthandler Enable faulthandler? If non-zero, call :func:`faulthandler.enable` at startup. Set to ``1`` by :option:`-X faulthandler <-X>` and the :envvar:`PYTHONFAULTHANDLER` environment variable. Default: ``-1`` in Python mode, ``0`` in isolated mode. .. c:member:: wchar_t* filesystem_encoding :term:`Filesystem encoding `: :func:`sys.getfilesystemencoding`. On macOS, Android and VxWorks: use ``"utf-8"`` by default. On Windows: use ``"utf-8"`` by default, or ``"mbcs"`` if :c:member:`~PyPreConfig.legacy_windows_fs_encoding` of :c:type:`PyPreConfig` is non-zero. Default encoding on other platforms: * ``"utf-8"`` if :c:member:`PyPreConfig.utf8_mode` is non-zero. * ``"ascii"`` if Python detects that ``nl_langinfo(CODESET)`` announces the ASCII encoding, whereas the ``mbstowcs()`` function decodes from a different encoding (usually Latin1). * ``"utf-8"`` if ``nl_langinfo(CODESET)`` returns an empty string. * Otherwise, use the :term:`locale encoding`: ``nl_langinfo(CODESET)`` result. At Python startup, the encoding name is normalized to the Python codec name. For example, ``"ANSI_X3.4-1968"`` is replaced with ``"ascii"``. See also the :c:member:`~PyConfig.filesystem_errors` member. See also :c:func:`PyUnicode_EncodeFSDefault` and :ref:`related functions ` for encoding paths at runtime. .. c:member:: wchar_t* filesystem_errors :term:`Filesystem error handler `: :func:`sys.getfilesystemencodeerrors`. On Windows: use ``"surrogatepass"`` by default, or ``"replace"`` if :c:member:`~PyPreConfig.legacy_windows_fs_encoding` of :c:type:`PyPreConfig` is non-zero. On other platforms: use ``"surrogateescape"`` by default. Supported error handlers: * ``"strict"`` * ``"surrogateescape"`` * ``"surrogatepass"`` (only supported with the UTF-8 encoding) See also the :c:member:`~PyConfig.filesystem_encoding` member. .. c:member:: int use_frozen_modules If non-zero, use frozen modules. Set by the :envvar:`PYTHON_FROZEN_MODULES` environment variable. Default: ``1`` in a release build, or ``0`` in a :ref:`debug build `. .. c:member:: unsigned long hash_seed .. c:member:: int use_hash_seed Randomized hash function seed. If :c:member:`~PyConfig.use_hash_seed` is zero, a seed is chosen randomly at Python startup, and :c:member:`~PyConfig.hash_seed` is ignored. Set by the :envvar:`PYTHONHASHSEED` environment variable. Default *use_hash_seed* value: ``-1`` in Python mode, ``0`` in isolated mode. .. c:member:: wchar_t* home Set the default Python "home" directory, that is, the location of the standard Python libraries (see :envvar:`PYTHONHOME`). Set by the :envvar:`PYTHONHOME` environment variable. Default: ``NULL``. Part of the :ref:`Python Path Configuration ` input. .. c:member:: int import_time If ``1``, profile import time. If ``2``, include additional output that indicates when an imported module has already been loaded. Set by the :option:`-X importtime <-X>` option and the :envvar:`PYTHONPROFILEIMPORTTIME` environment variable. Default: ``0``. .. versionchanged:: 3.14 Added support for ``import_time = 2`` .. c:member:: int inspect Enter interactive mode after executing a script or a command. If greater than ``0``, enable inspect: when a script is passed as first argument or the -c option is used, enter interactive mode after executing the script or the command, even when :data:`sys.stdin` does not appear to be a terminal. Incremented by the :option:`-i` command line option. Set to ``1`` if the :envvar:`PYTHONINSPECT` environment variable is non-empty. Default: ``0``. .. c:member:: int install_signal_handlers Install Python signal handlers? Default: ``1`` in Python mode, ``0`` in isolated mode. .. c:member:: int interactive If greater than ``0``, enable the interactive mode (REPL). Incremented by the :option:`-i` command line option. Default: ``0``. .. c:member:: int int_max_str_digits Configures the :ref:`integer string conversion length limitation `. An initial value of ``-1`` means the value will be taken from the command line or environment or otherwise default to 4300 (:data:`sys.int_info.default_max_str_digits`). A value of ``0`` disables the limitation. Values greater than zero but less than 640 (:data:`sys.int_info.str_digits_check_threshold`) are unsupported and will produce an error. Configured by the :option:`-X int_max_str_digits <-X>` command line flag or the :envvar:`PYTHONINTMAXSTRDIGITS` environment variable. Default: ``-1`` in Python mode. 4300 (:data:`sys.int_info.default_max_str_digits`) in isolated mode. .. versionadded:: 3.12 .. c:member:: int cpu_count If the value of :c:member:`~PyConfig.cpu_count` is not ``-1`` then it will override the return values of :func:`os.cpu_count`, :func:`os.process_cpu_count`, and :func:`multiprocessing.cpu_count`. Configured by the :samp:`-X cpu_count={n|default}` command line flag or the :envvar:`PYTHON_CPU_COUNT` environment variable. Default: ``-1``. .. versionadded:: 3.13 .. c:member:: int isolated If greater than ``0``, enable isolated mode: * Set :c:member:`~PyConfig.safe_path` to ``1``: don't prepend a potentially unsafe path to :data:`sys.path` at Python startup, such as the current directory, the script's directory or an empty string. * Set :c:member:`~PyConfig.use_environment` to ``0``: ignore ``PYTHON`` environment variables. * Set :c:member:`~PyConfig.user_site_directory` to ``0``: don't add the user site directory to :data:`sys.path`. * Python REPL doesn't import :mod:`readline` nor enable default readline configuration on interactive prompts. Set to ``1`` by the :option:`-I` command line option. Default: ``0`` in Python mode, ``1`` in isolated mode. See also the :ref:`Isolated Configuration ` and :c:member:`PyPreConfig.isolated`. .. c:member:: int legacy_windows_stdio If non-zero, use :class:`io.FileIO` instead of :class:`!io._WindowsConsoleIO` for :data:`sys.stdin`, :data:`sys.stdout` and :data:`sys.stderr`. Set to ``1`` if the :envvar:`PYTHONLEGACYWINDOWSSTDIO` environment variable is set to a non-empty string. Only available on Windows. ``#ifdef MS_WINDOWS`` macro can be used for Windows specific code. Default: ``0``. See also the :pep:`528` (Change Windows console encoding to UTF-8). .. c:member:: int malloc_stats If non-zero, dump statistics on :ref:`Python pymalloc memory allocator ` at exit. Set to ``1`` by the :envvar:`PYTHONMALLOCSTATS` environment variable. The option is ignored if Python is :option:`configured using the --without-pymalloc option <--without-pymalloc>`. Default: ``0``. .. c:member:: wchar_t* platlibdir Platform library directory name: :data:`sys.platlibdir`. Set by the :envvar:`PYTHONPLATLIBDIR` environment variable. Default: value of the ``PLATLIBDIR`` macro which is set by the :option:`configure --with-platlibdir option <--with-platlibdir>` (default: ``"lib"``, or ``"DLLs"`` on Windows). Part of the :ref:`Python Path Configuration ` input. .. versionadded:: 3.9 .. versionchanged:: 3.11 This macro is now used on Windows to locate the standard library extension modules, typically under ``DLLs``. However, for compatibility, note that this value is ignored for any non-standard layouts, including in-tree builds and virtual environments. .. c:member:: wchar_t* pythonpath_env Module search paths (:data:`sys.path`) as a string separated by ``DELIM`` (:data:`os.pathsep`). Set by the :envvar:`PYTHONPATH` environment variable. Default: ``NULL``. Part of the :ref:`Python Path Configuration ` input. .. c:member:: PyWideStringList module_search_paths .. c:member:: int module_search_paths_set Module search paths: :data:`sys.path`. If :c:member:`~PyConfig.module_search_paths_set` is equal to ``0``, :c:func:`Py_InitializeFromConfig` will replace :c:member:`~PyConfig.module_search_paths` and sets :c:member:`~PyConfig.module_search_paths_set` to ``1``. Default: empty list (``module_search_paths``) and ``0`` (``module_search_paths_set``). Part of the :ref:`Python Path Configuration ` output. .. c:member:: int optimization_level Compilation optimization level: * ``0``: Peephole optimizer, set ``__debug__`` to ``True``. * ``1``: Level 0, remove assertions, set ``__debug__`` to ``False``. * ``2``: Level 1, strip docstrings. Incremented by the :option:`-O` command line option. Set to the :envvar:`PYTHONOPTIMIZE` environment variable value. Default: ``0``. .. c:member:: PyWideStringList orig_argv The list of the original command line arguments passed to the Python executable: :data:`sys.orig_argv`. If :c:member:`~PyConfig.orig_argv` list is empty and :c:member:`~PyConfig.argv` is not a list only containing an empty string, :c:func:`PyConfig_Read` copies :c:member:`~PyConfig.argv` into :c:member:`~PyConfig.orig_argv` before modifying :c:member:`~PyConfig.argv` (if :c:member:`~PyConfig.parse_argv` is non-zero). See also the :c:member:`~PyConfig.argv` member and the :c:func:`Py_GetArgcArgv` function. Default: empty list. .. versionadded:: 3.10 .. c:member:: int parse_argv Parse command line arguments? If equals to ``1``, parse :c:member:`~PyConfig.argv` the same way the regular Python parses :ref:`command line arguments `, and strip Python arguments from :c:member:`~PyConfig.argv`. The :c:func:`PyConfig_Read` function only parses :c:member:`PyConfig.argv` arguments once: :c:member:`PyConfig.parse_argv` is set to ``2`` after arguments are parsed. Since Python arguments are stripped from :c:member:`PyConfig.argv`, parsing arguments twice would parse the application options as Python options. Default: ``1`` in Python mode, ``0`` in isolated mode. .. versionchanged:: 3.10 The :c:member:`PyConfig.argv` arguments are now only parsed if :c:member:`PyConfig.parse_argv` equals to ``1``. .. c:member:: int parser_debug Parser debug mode. If greater than ``0``, turn on parser debugging output (for expert only, depending on compilation options). Incremented by the :option:`-d` command line option. Set to the :envvar:`PYTHONDEBUG` environment variable value. Needs a :ref:`debug build of Python ` (the ``Py_DEBUG`` macro must be defined). Default: ``0``. .. c:member:: int pathconfig_warnings If non-zero, calculation of path configuration is allowed to log warnings into ``stderr``. If equals to ``0``, suppress these warnings. Default: ``1`` in Python mode, ``0`` in isolated mode. Part of the :ref:`Python Path Configuration ` input. .. versionchanged:: 3.11 Now also applies on Windows. .. c:member:: wchar_t* prefix The site-specific directory prefix where the platform independent Python files are installed: :data:`sys.prefix`. Default: ``NULL``. Part of the :ref:`Python Path Configuration ` output. See also :c:member:`PyConfig.base_prefix`. .. c:member:: wchar_t* program_name Program name used to initialize :c:member:`~PyConfig.executable` and in early error messages during Python initialization. * On macOS, use :envvar:`PYTHONEXECUTABLE` environment variable if set. * If the ``WITH_NEXT_FRAMEWORK`` macro is defined, use ``__PYVENV_LAUNCHER__`` environment variable if set. * Use ``argv[0]`` of :c:member:`~PyConfig.argv` if available and non-empty. * Otherwise, use ``L"python"`` on Windows, or ``L"python3"`` on other platforms. Default: ``NULL``. Part of the :ref:`Python Path Configuration ` input. .. c:member:: wchar_t* pycache_prefix Directory where cached ``.pyc`` files are written: :data:`sys.pycache_prefix`. Set by the :option:`-X pycache_prefix=PATH <-X>` command line option and the :envvar:`PYTHONPYCACHEPREFIX` environment variable. The command-line option takes precedence. If ``NULL``, :data:`sys.pycache_prefix` is set to ``None``. Default: ``NULL``. .. c:member:: int quiet Quiet mode. If greater than ``0``, don't display the copyright and version at Python startup in interactive mode. Incremented by the :option:`-q` command line option. Default: ``0``. .. c:member:: wchar_t* run_command Value of the :option:`-c` command line option. Used by :c:func:`Py_RunMain`. Default: ``NULL``. .. c:member:: wchar_t* run_filename Filename passed on the command line: trailing command line argument without :option:`-c` or :option:`-m`. It is used by the :c:func:`Py_RunMain` function. For example, it is set to ``script.py`` by the ``python3 script.py arg`` command line. See also the :c:member:`PyConfig.skip_source_first_line` option. Default: ``NULL``. .. c:member:: wchar_t* run_module Value of the :option:`-m` command line option. Used by :c:func:`Py_RunMain`. Default: ``NULL``. .. c:member:: wchar_t* run_presite ``module`` or ``module:func`` entry point that should be executed before the :mod:`site` module is imported. Set by the :option:`-X presite=module:func <-X>` command-line option and the :envvar:`PYTHON_PRESITE` environment variable. The command-line option takes precedence. Needs a :ref:`debug build of Python ` (the ``Py_DEBUG`` macro must be defined). Default: ``NULL``. .. c:member:: int show_ref_count Show total reference count at exit (excluding :term:`immortal` objects)? Set to ``1`` by :option:`-X showrefcount <-X>` command line option. Needs a :ref:`debug build of Python ` (the ``Py_REF_DEBUG`` macro must be defined). Default: ``0``. .. c:member:: int site_import Import the :mod:`site` module at startup? If equal to zero, disable the import of the module site and the site-dependent manipulations of :data:`sys.path` that it entails. Also disable these manipulations if the :mod:`site` module is explicitly imported later (call :func:`site.main` if you want them to be triggered). Set to ``0`` by the :option:`-S` command line option. :data:`sys.flags.no_site ` is set to the inverted value of :c:member:`~PyConfig.site_import`. Default: ``1``. .. c:member:: int skip_source_first_line If non-zero, skip the first line of the :c:member:`PyConfig.run_filename` source. It allows the usage of non-Unix forms of ``#!cmd``. This is intended for a DOS specific hack only. Set to ``1`` by the :option:`-x` command line option. Default: ``0``. .. c:member:: wchar_t* stdio_encoding .. c:member:: wchar_t* stdio_errors Encoding and encoding errors of :data:`sys.stdin`, :data:`sys.stdout` and :data:`sys.stderr` (but :data:`sys.stderr` always uses ``"backslashreplace"`` error handler). Use the :envvar:`PYTHONIOENCODING` environment variable if it is non-empty. Default encoding: * ``"UTF-8"`` if :c:member:`PyPreConfig.utf8_mode` is non-zero. * Otherwise, use the :term:`locale encoding`. Default error handler: * On Windows: use ``"surrogateescape"``. * ``"surrogateescape"`` if :c:member:`PyPreConfig.utf8_mode` is non-zero, or if the LC_CTYPE locale is "C" or "POSIX". * ``"strict"`` otherwise. See also :c:member:`PyConfig.legacy_windows_stdio`. .. c:member:: int tracemalloc Enable tracemalloc? If non-zero, call :func:`tracemalloc.start` at startup. Set by :option:`-X tracemalloc=N <-X>` command line option and by the :envvar:`PYTHONTRACEMALLOC` environment variable. Default: ``-1`` in Python mode, ``0`` in isolated mode. .. c:member:: int perf_profiling Enable the Linux ``perf`` profiler support? If equals to ``1``, enable support for the Linux ``perf`` profiler. If equals to ``2``, enable support for the Linux ``perf`` profiler with DWARF JIT support. Set to ``1`` by :option:`-X perf <-X>` command-line option and the :envvar:`PYTHONPERFSUPPORT` environment variable. Set to ``2`` by the :option:`-X perf_jit <-X>` command-line option and the :envvar:`PYTHON_PERF_JIT_SUPPORT` environment variable. Default: ``-1``. .. seealso:: See :ref:`perf_profiling` for more information. .. versionadded:: 3.12 .. c:member:: wchar_t* stdlib_dir Directory of the Python standard library. Default: ``NULL``. .. versionadded:: 3.11 .. c:member:: int use_environment Use :ref:`environment variables `? If equals to zero, ignore the :ref:`environment variables `. Set to ``0`` by the :option:`-E` environment variable. Default: ``1`` in Python config and ``0`` in isolated config. .. c:member:: int use_system_logger If non-zero, ``stdout`` and ``stderr`` will be redirected to the system log. Only available on macOS 10.12 and later, and on iOS. Default: ``0`` (don't use the system log) on macOS; ``1`` on iOS (use the system log). .. versionadded:: 3.14 .. c:member:: int user_site_directory If non-zero, add the user site directory to :data:`sys.path`. Set to ``0`` by the :option:`-s` and :option:`-I` command line options. Set to ``0`` by the :envvar:`PYTHONNOUSERSITE` environment variable. Default: ``1`` in Python mode, ``0`` in isolated mode. .. c:member:: int verbose Verbose mode. If greater than ``0``, print a message each time a module is imported, showing the place (filename or built-in module) from which it is loaded. If greater than or equal to ``2``, print a message for each file that is checked for when searching for a module. Also provides information on module cleanup at exit. Incremented by the :option:`-v` command line option. Set by the :envvar:`PYTHONVERBOSE` environment variable value. Default: ``0``. .. c:member:: PyWideStringList warnoptions Options of the :mod:`warnings` module to build warnings filters, lowest to highest priority: :data:`sys.warnoptions`. The :mod:`warnings` module adds :data:`sys.warnoptions` in the reverse order: the last :c:member:`PyConfig.warnoptions` item becomes the first item of ``warnings.filters`` which is checked first (highest priority). The :option:`-W` command line options adds its value to :c:member:`~PyConfig.warnoptions`, it can be used multiple times. The :envvar:`PYTHONWARNINGS` environment variable can also be used to add warning options. Multiple options can be specified, separated by commas (``,``). Default: empty list. .. c:member:: int write_bytecode If equal to ``0``, Python won't try to write ``.pyc`` files on the import of source modules. Set to ``0`` by the :option:`-B` command line option and the :envvar:`PYTHONDONTWRITEBYTECODE` environment variable. :data:`sys.dont_write_bytecode` is initialized to the inverted value of :c:member:`~PyConfig.write_bytecode`. Default: ``1``. .. c:member:: PyWideStringList xoptions Values of the :option:`-X` command line options: :data:`sys._xoptions`. Default: empty list. .. c:member:: int _pystats If non-zero, write performance statistics at Python exit. Need a special build with the ``Py_STATS`` macro: see :option:`--enable-pystats`. Default: ``0``. If :c:member:`~PyConfig.parse_argv` is non-zero, :c:member:`~PyConfig.argv` arguments are parsed the same way the regular Python parses :ref:`command line arguments `, and Python arguments are stripped from :c:member:`~PyConfig.argv`. The :c:member:`~PyConfig.xoptions` options are parsed to set other options: see the :option:`-X` command line option. .. versionchanged:: 3.9 The ``show_alloc_count`` field has been removed. .. _init-from-config: Initialization with PyConfig ---------------------------- Initializing the interpreter from a populated configuration struct is handled by calling :c:func:`Py_InitializeFromConfig`. The caller is responsible to handle exceptions (error or exit) using :c:func:`PyStatus_Exception` and :c:func:`Py_ExitStatusException`. If :c:func:`PyImport_FrozenModules`, :c:func:`PyImport_AppendInittab` or :c:func:`PyImport_ExtendInittab` are used, they must be set or called after Python preinitialization and before the Python initialization. If Python is initialized multiple times, :c:func:`PyImport_AppendInittab` or :c:func:`PyImport_ExtendInittab` must be called before each Python initialization. The current configuration (``PyConfig`` type) is stored in ``PyInterpreterState.config``. Example setting the program name:: void init_python(void) { PyStatus status; PyConfig config; PyConfig_InitPythonConfig(&config); /* Set the program name. Implicitly preinitialize Python. */ status = PyConfig_SetString(&config, &config.program_name, L"/path/to/my_program"); if (PyStatus_Exception(status)) { goto exception; } status = Py_InitializeFromConfig(&config); if (PyStatus_Exception(status)) { goto exception; } PyConfig_Clear(&config); return; exception: PyConfig_Clear(&config); Py_ExitStatusException(status); } More complete example modifying the default configuration, read the configuration, and then override some parameters. Note that since 3.11, many parameters are not calculated until initialization, and so values cannot be read from the configuration structure. Any values set before initialize is called will be left unchanged by initialization:: PyStatus init_python(const char *program_name) { PyStatus status; PyConfig config; PyConfig_InitPythonConfig(&config); /* Set the program name before reading the configuration (decode byte string from the locale encoding). Implicitly preinitialize Python. */ status = PyConfig_SetBytesString(&config, &config.program_name, program_name); if (PyStatus_Exception(status)) { goto done; } /* Read all configuration at once */ status = PyConfig_Read(&config); if (PyStatus_Exception(status)) { goto done; } /* Specify sys.path explicitly */ /* If you want to modify the default set of paths, finish initialization first and then use PySys_GetAttrString("path") */ config.module_search_paths_set = 1; status = PyWideStringList_Append(&config.module_search_paths, L"/path/to/stdlib"); if (PyStatus_Exception(status)) { goto done; } status = PyWideStringList_Append(&config.module_search_paths, L"/path/to/more/modules"); if (PyStatus_Exception(status)) { goto done; } /* Override executable computed by PyConfig_Read() */ status = PyConfig_SetString(&config, &config.executable, L"/path/to/my_executable"); if (PyStatus_Exception(status)) { goto done; } status = Py_InitializeFromConfig(&config); done: PyConfig_Clear(&config); return status; } .. _init-isolated-conf: Isolated Configuration ---------------------- :c:func:`PyPreConfig_InitIsolatedConfig` and :c:func:`PyConfig_InitIsolatedConfig` functions create a configuration to isolate Python from the system. For example, to embed Python into an application. This configuration ignores global configuration variables, environment variables, command line arguments (:c:member:`PyConfig.argv` is not parsed) and user site directory. The C standard streams (ex: ``stdout``) and the LC_CTYPE locale are left unchanged. Signal handlers are not installed. Configuration files are still used with this configuration to determine paths that are unspecified. Ensure :c:member:`PyConfig.home` is specified to avoid computing the default path configuration. .. _init-python-config: Python Configuration -------------------- :c:func:`PyPreConfig_InitPythonConfig` and :c:func:`PyConfig_InitPythonConfig` functions create a configuration to build a customized Python which behaves as the regular Python. Environments variables and command line arguments are used to configure Python, whereas global configuration variables are ignored. This function enables C locale coercion (:pep:`538`) and :ref:`Python UTF-8 Mode ` (:pep:`540`) depending on the LC_CTYPE locale, :envvar:`PYTHONUTF8` and :envvar:`PYTHONCOERCECLOCALE` environment variables. .. _init-path-config: Python Path Configuration ------------------------- :c:type:`PyConfig` contains multiple fields for the path configuration: * Path configuration inputs: * :c:member:`PyConfig.home` * :c:member:`PyConfig.platlibdir` * :c:member:`PyConfig.pathconfig_warnings` * :c:member:`PyConfig.program_name` * :c:member:`PyConfig.pythonpath_env` * current working directory: to get absolute paths * ``PATH`` environment variable to get the program full path (from :c:member:`PyConfig.program_name`) * ``__PYVENV_LAUNCHER__`` environment variable * (Windows only) Application paths in the registry under "Software\Python\PythonCore\X.Y\PythonPath" of HKEY_CURRENT_USER and HKEY_LOCAL_MACHINE (where X.Y is the Python version). * Path configuration output fields: * :c:member:`PyConfig.base_exec_prefix` * :c:member:`PyConfig.base_executable` * :c:member:`PyConfig.base_prefix` * :c:member:`PyConfig.exec_prefix` * :c:member:`PyConfig.executable` * :c:member:`PyConfig.module_search_paths_set`, :c:member:`PyConfig.module_search_paths` * :c:member:`PyConfig.prefix` If at least one "output field" is not set, Python calculates the path configuration to fill unset fields. If :c:member:`~PyConfig.module_search_paths_set` is equal to ``0``, :c:member:`~PyConfig.module_search_paths` is overridden and :c:member:`~PyConfig.module_search_paths_set` is set to ``1``. It is possible to completely ignore the function calculating the default path configuration by setting explicitly all path configuration output fields listed above. A string is considered as set even if it is non-empty. ``module_search_paths`` is considered as set if ``module_search_paths_set`` is set to ``1``. In this case, ``module_search_paths`` will be used without modification. Set :c:member:`~PyConfig.pathconfig_warnings` to ``0`` to suppress warnings when calculating the path configuration (Unix only, Windows does not log any warning). If :c:member:`~PyConfig.base_prefix` or :c:member:`~PyConfig.base_exec_prefix` fields are not set, they inherit their value from :c:member:`~PyConfig.prefix` and :c:member:`~PyConfig.exec_prefix` respectively. :c:func:`Py_RunMain` and :c:func:`Py_Main` modify :data:`sys.path`: * If :c:member:`~PyConfig.run_filename` is set and is a directory which contains a ``__main__.py`` script, prepend :c:member:`~PyConfig.run_filename` to :data:`sys.path`. * If :c:member:`~PyConfig.isolated` is zero: * If :c:member:`~PyConfig.run_module` is set, prepend the current directory to :data:`sys.path`. Do nothing if the current directory cannot be read. * If :c:member:`~PyConfig.run_filename` is set, prepend the directory of the filename to :data:`sys.path`. * Otherwise, prepend an empty string to :data:`sys.path`. If :c:member:`~PyConfig.site_import` is non-zero, :data:`sys.path` can be modified by the :mod:`site` module. If :c:member:`~PyConfig.user_site_directory` is non-zero and the user's site-package directory exists, the :mod:`site` module appends the user's site-package directory to :data:`sys.path`. The following configuration files are used by the path configuration: * ``pyvenv.cfg`` * ``._pth`` file (ex: ``python._pth``) * ``pybuilddir.txt`` (Unix only) If a ``._pth`` file is present: * Set :c:member:`~PyConfig.isolated` to ``1``. * Set :c:member:`~PyConfig.use_environment` to ``0``. * Set :c:member:`~PyConfig.site_import` to ``0``. * Set :c:member:`~PyConfig.user_site_directory` to ``0`` (since 3.15). * Set :c:member:`~PyConfig.safe_path` to ``1``. If :c:member:`~PyConfig.home` is not set and a ``pyvenv.cfg`` file is present in the same directory as :c:member:`~PyConfig.executable`, or its parent, :c:member:`~PyConfig.prefix` and :c:member:`~PyConfig.exec_prefix` are set that location. When this happens, :c:member:`~PyConfig.base_prefix` and :c:member:`~PyConfig.base_exec_prefix` still keep their value, pointing to the base installation. See :ref:`sys-path-init-virtual-environments` for more information. The ``__PYVENV_LAUNCHER__`` environment variable is used to set :c:member:`PyConfig.base_executable`. .. versionchanged:: 3.14 :c:member:`~PyConfig.prefix`, and :c:member:`~PyConfig.exec_prefix`, are now set to the ``pyvenv.cfg`` directory. This was previously done by :mod:`site`, therefore affected by :option:`-S`. .. versionchanged:: 3.15 :c:member:`~PyConfig.user_site_directory` is now set to ``0`` when a ``._pth`` file is present. Py_GetArgcArgv() ================ .. c:function:: void Py_GetArgcArgv(int *argc, wchar_t ***argv) Get the original command line arguments, before Python modified them. See also :c:member:`PyConfig.orig_argv` member. Multi-Phase Initialization Private Provisional API ================================================== This section is a private provisional API introducing multi-phase initialization, the core feature of :pep:`432`: * "Core" initialization phase, "bare minimum Python": * Builtin types; * Builtin exceptions; * Builtin and frozen modules; * The :mod:`sys` module is only partially initialized (ex: :data:`sys.path` doesn't exist yet). * "Main" initialization phase, Python is fully initialized: * Install and configure :mod:`importlib`; * Apply the :ref:`Path Configuration `; * Install signal handlers; * Finish :mod:`sys` module initialization (ex: create :data:`sys.stdout` and :data:`sys.path`); * Enable optional features like :mod:`faulthandler` and :mod:`tracemalloc`; * Import the :mod:`site` module; * etc. Private provisional API: .. c:member:: int PyConfig._init_main If set to ``0``, :c:func:`Py_InitializeFromConfig` stops at the "Core" initialization phase. .. c:function:: PyStatus _Py_InitializeMain(void) Move to the "Main" initialization phase, finish the Python initialization. No module is imported during the "Core" phase and the ``importlib`` module is not configured: the :ref:`Path Configuration ` is only applied during the "Main" phase. It may allow to customize Python in Python to override or tune the :ref:`Path Configuration `, maybe install a custom :data:`sys.meta_path` importer or an import hook, etc. It may become possible to calculate the :ref:`Path Configuration ` in Python, after the Core phase and before the Main phase, which is one of the :pep:`432` motivation. The "Core" phase is not properly defined: what should be and what should not be available at this phase is not specified yet. The API is marked as private and provisional: the API can be modified or even be removed anytime until a proper public API is designed. Example running Python code between "Core" and "Main" initialization phases:: void init_python(void) { PyStatus status; PyConfig config; PyConfig_InitPythonConfig(&config); config._init_main = 0; /* ... customize 'config' configuration ... */ status = Py_InitializeFromConfig(&config); PyConfig_Clear(&config); if (PyStatus_Exception(status)) { Py_ExitStatusException(status); } /* Use sys.stderr because sys.stdout is only created by _Py_InitializeMain() */ int res = PyRun_SimpleString( "import sys; " "print('Run Python code before _Py_InitializeMain', " "file=sys.stderr)"); if (res < 0) { exit(1); } /* ... put more configuration code here ... */ status = _Py_InitializeMain(); if (PyStatus_Exception(status)) { Py_ExitStatusException(status); } } --- ### Doc C Api Interp Lifecycle (`Doc/c-api/interp-lifecycle.rst`) .. highlight:: c .. _initialization: Interpreter initialization and finalization =========================================== See :ref:`Python Initialization Configuration ` for details on how to configure the interpreter prior to initialization. .. _pre-init-safe: Before Python initialization ---------------------------- In an application embedding Python, the :c:func:`Py_Initialize` function must be called before using any other Python/C API functions; with the exception of a few functions. The following functions can be safely called before Python is initialized: * Functions that initialize the interpreter: * :c:func:`Py_Initialize` * :c:func:`Py_InitializeEx` * :c:func:`Py_InitializeFromConfig` * :c:func:`Py_BytesMain` * :c:func:`Py_Main` * the runtime pre-initialization functions covered in :ref:`init-config` * Configuration functions: * :c:func:`PyImport_AppendInittab` * :c:func:`PyImport_ExtendInittab` * :c:func:`!PyInitFrozenExtensions` * :c:func:`PyMem_SetAllocator` * :c:func:`PyMem_SetupDebugHooks` * :c:func:`PyObject_SetArenaAllocator` * the configuration functions covered in :ref:`init-config` * Informative functions: * :c:func:`Py_IsInitialized` * :c:func:`PyMem_GetAllocator` * :c:func:`PyObject_GetArenaAllocator` * :c:func:`Py_GetBuildInfo` * :c:func:`Py_GetCompiler` * :c:func:`Py_GetCopyright` * :c:func:`Py_GetPlatform` * :c:func:`Py_GetVersion` * :c:func:`Py_IsInitialized` * Utilities: * :c:func:`Py_DecodeLocale` * the status reporting and utility functions covered in :ref:`init-config` * Memory allocators: * :c:func:`PyMem_RawMalloc` * :c:func:`PyMem_RawRealloc` * :c:func:`PyMem_RawCalloc` * :c:func:`PyMem_RawFree` * Synchronization: * :c:func:`PyMutex_Lock` * :c:func:`PyMutex_Unlock` .. note:: Despite their apparent similarity to some of the functions listed above, the following functions **should not be called** before the interpreter has been initialized: :c:func:`Py_EncodeLocale`, and :c:func:`Py_RunMain`. Initializing and finalizing the interpreter ------------------------------------------- .. c:function:: void Py_Initialize() .. index:: single: modules (in module sys) single: path (in module sys) pair: module; builtins pair: module; __main__ pair: module; sys triple: module; search; path single: Py_FinalizeEx (C function) Initialize the Python interpreter. In an application embedding Python, this should be called before using any other Python/C API functions; see :ref:`Before Python Initialization ` for the few exceptions. This initializes the table of loaded modules (``sys.modules``), and creates the fundamental modules :mod:`builtins`, :mod:`__main__` and :mod:`sys`. It also initializes the module search path (``sys.path``). It does not set ``sys.argv``; use the :ref:`Python Initialization Configuration ` API for that. This is a no-op when called for a second time (without calling :c:func:`Py_FinalizeEx` first). There is no return value; it is a fatal error if the initialization fails. Use :c:func:`Py_InitializeFromConfig` to customize the :ref:`Python Initialization Configuration `. .. note:: On Windows, changes the console mode from ``O_TEXT`` to ``O_BINARY``, which will also affect non-Python uses of the console using the C Runtime. .. c:function:: void Py_InitializeEx(int initsigs) This function works like :c:func:`Py_Initialize` if *initsigs* is ``1``. If *initsigs* is ``0``, it skips initialization registration of signal handlers, which may be useful when CPython is embedded as part of a larger application. Use :c:func:`Py_InitializeFromConfig` to customize the :ref:`Python Initialization Configuration `. .. c:function:: PyStatus Py_InitializeFromConfig(const PyConfig *config) Initialize Python from *config* configuration, as described in :ref:`init-from-config`. See the :ref:`init-config` section for details on pre-initializing the interpreter, populating the runtime configuration structure, and querying the returned status structure. .. versionchanged:: 3.16 The function no longer returns an exit code if a command line option wants to exit Python. Instead, the option is processed in :c:func:`Py_RunMain`. .. c:function:: int Py_IsInitialized() Return true (nonzero) when the Python interpreter has been initialized, false (zero) if not. After :c:func:`Py_FinalizeEx` is called, this returns false until :c:func:`Py_Initialize` is called again. .. versionchanged:: 3.15 This function no longer returns true until initialization has fully completed, including import of the :mod:`site` module. Previously it could return true while :c:func:`Py_Initialize` was still running. .. c:function:: int Py_IsFinalizing() Return true (non-zero) if the main Python interpreter is :term:`shutting down `. Return false (zero) otherwise. .. versionadded:: 3.13 .. c:function:: int Py_FinalizeEx() Undo all initializations made by :c:func:`Py_Initialize` and subsequent use of Python/C API functions, and destroy all sub-interpreters (see :c:func:`Py_NewInterpreter` below) that were created and not yet destroyed since the last call to :c:func:`Py_Initialize`. This is a no-op when called for a second time (without calling :c:func:`Py_Initialize` again first). Since this is the reverse of :c:func:`Py_Initialize`, it should be called in the same thread with the same interpreter active. That means the main thread and the main interpreter. This should never be called while :c:func:`Py_RunMain` is running. Normally the return value is ``0``. If there were errors during finalization (flushing buffered data), ``-1`` is returned. Note that Python will do a best effort at freeing all memory allocated by the Python interpreter. Therefore, any C-Extension should make sure to correctly clean up all of the previously allocated PyObjects before using them in subsequent calls to :c:func:`Py_Initialize`. Otherwise it could introduce vulnerabilities and incorrect behavior. This function is provided for a number of reasons. An embedding application might want to restart Python without having to restart the application itself. An application that has loaded the Python interpreter from a dynamically loadable library (or DLL) might want to free all memory allocated by Python before unloading the DLL. During a hunt for memory leaks in an application a developer might want to free all memory allocated by Python before exiting from the application. **Bugs and caveats:** The destruction of modules and objects in modules is done in random order; this may cause destructors (:meth:`~object.__del__` methods) to fail when they depend on other objects (even functions) or modules. Dynamically loaded extension modules loaded by Python are not unloaded. Small amounts of memory allocated by the Python interpreter may not be freed (if you find a leak, please report it). Memory tied up in circular references between objects is not freed. Interned strings will all be deallocated regardless of their reference count. Some memory allocated by extension modules may not be freed. Some extensions may not work properly if their initialization routine is called more than once; this can happen if an application calls :c:func:`Py_Initialize` and :c:func:`Py_FinalizeEx` more than once. :c:func:`Py_FinalizeEx` must not be called recursively from within itself. Therefore, it must not be called by any code that may be run as part of the interpreter shutdown process, such as :py:mod:`atexit` handlers, object finalizers, or any code that may be run while flushing the stdout and stderr files. .. audit-event:: cpython._PySys_ClearAuditHooks "" c.Py_FinalizeEx .. versionadded:: 3.6 .. c:function:: void Py_Finalize() This is a backwards-compatible version of :c:func:`Py_FinalizeEx` that disregards the return value. .. c:function:: int Py_BytesMain(int argc, char **argv) Similar to :c:func:`Py_Main` but *argv* is an array of bytes strings, allowing the calling application to delegate the text decoding step to the CPython runtime. .. versionadded:: 3.8 .. c:function:: int Py_Main(int argc, wchar_t **argv) The main program for the standard interpreter, encapsulating a full initialization/finalization cycle, as well as additional behaviour to implement reading configurations settings from the environment and command line, and then executing ``__main__`` in accordance with :ref:`using-on-cmdline`. This is made available for programs which wish to support the full CPython command line interface, rather than just embedding a Python runtime in a larger application. The *argc* and *argv* parameters are similar to those which are passed to a C program's :c:func:`main` function, except that the *argv* entries are first converted to ``wchar_t`` using :c:func:`Py_DecodeLocale`. It is also important to note that the argument list entries may be modified to point to strings other than those passed in (however, the contents of the strings pointed to by the argument list are not modified). The return value is ``2`` if the argument list does not represent a valid Python command line, and otherwise the same as :c:func:`Py_RunMain`. In terms of the CPython runtime configuration APIs documented in the :ref:`runtime configuration ` section (and without accounting for error handling), ``Py_Main`` is approximately equivalent to:: PyConfig config; PyConfig_InitPythonConfig(&config); PyConfig_SetArgv(&config, argc, argv); Py_InitializeFromConfig(&config); PyConfig_Clear(&config); Py_RunMain(); In normal usage, an embedding application will call this function *instead* of calling :c:func:`Py_Initialize`, :c:func:`Py_InitializeEx` or :c:func:`Py_InitializeFromConfig` directly, and all settings will be applied as described elsewhere in this documentation. If this function is instead called *after* a preceding runtime initialization API call, then exactly which environmental and command line configuration settings will be updated is version dependent (as it depends on which settings correctly support being modified after they have already been set once when the runtime was first initialized). .. c:function:: int Py_RunMain(void) Executes the main module in a fully configured CPython runtime. Executes the command (:c:member:`PyConfig.run_command`), the script (:c:member:`PyConfig.run_filename`) or the module (:c:member:`PyConfig.run_module`) specified on the command line or in the configuration. If none of these values are set, runs the interactive Python prompt (REPL) using the ``__main__`` module's global namespace. If :c:member:`PyConfig.inspect` is not set (the default), the return value will be ``0`` if the interpreter exits normally (that is, without raising an exception), the exit status of an unhandled :exc:`SystemExit`, or ``1`` for any other unhandled exception. If :c:member:`PyConfig.inspect` is set (such as when the :option:`-i` option is used), rather than returning when the interpreter exits, execution will instead resume in an interactive Python prompt (REPL) using the ``__main__`` module's global namespace. If the interpreter exited with an exception, it is immediately raised in the REPL session. The function return value is then determined by the way the *REPL session* terminates: ``0``, ``1``, or the status of a :exc:`SystemExit`, as specified above. This function always finalizes the Python interpreter before it returns. See :ref:`Python Configuration ` for an example of a customized Python that always runs in isolated mode using :c:func:`Py_RunMain`. .. c:function:: int PyUnstable_AtExit(PyInterpreterState *interp, void (*func)(void *), void *data) Register an :mod:`atexit` callback for the target interpreter *interp*. This is similar to :c:func:`Py_AtExit`, but takes an explicit interpreter and data pointer for the callback. There must be an :term:`attached thread state` for *interp*. .. versionadded:: 3.13 .. _cautions-regarding-runtime-finalization: Cautions regarding interpreter finalization ------------------------------------------- In the late stage of :term:`interpreter shutdown`, after attempting to wait for non-daemon threads to exit (though this can be interrupted by :class:`KeyboardInterrupt`) and running the :mod:`atexit` functions, the runtime is marked as finalizing, meaning that :c:func:`Py_IsFinalizing` and :func:`sys.is_finalizing` return true. At this point, only the finalization thread (the thread that initiated finalization; this is typically the main thread) is allowed to :term:`attach ` a thread state. Other threads that attempt to attach during finalization, either explicitly (such as via :c:func:`PyThreadState_Ensure` or :c:macro:`Py_END_ALLOW_THREADS`) or implicitly (such as in-between bytecode instructions), will enter a **permanently blocked state**. Generally, this is harmless, but this can result in deadlocks. For example, a thread may be permanently blocked while holding a lock, meaning that the finalization thread can never acquire that lock. Prior to CPython 3.13, the thread would exit instead of hanging, which led to other issues (see the warning note at :c:func:`PyThread_exit_thread`). Gross? Yes. Starting in Python 3.15, there are a number of C APIs that make it possible to avoid these issues by temporarily preventing finalization: .. _interpreter-guards: .. seealso:: :pep:`788` explains the design, motivation and rationale for these APIs. .. c:type:: PyInterpreterGuard An opaque interpreter guard structure. By holding an interpreter guard, the caller can ensure that the interpreter will not finalize until the guard is closed (through :c:func:`PyInterpreterGuard_Close`). When a guard is held, a thread attempting to finalize the interpreter will block until the guard is closed before starting finalization. After finalization has started, threads are forever unable to acquire guards for that interpreter. This means that if you forget to close an interpreter guard, the process will **permanently hang** during finalization! Holding a guard for an interpreter is similar to holding a :term:`strong reference` to a Python object, except finalization does not happen automatically after all guards are released: it requires an explicit :c:func:`Py_EndInterpreter` call. .. versionadded:: 3.15 .. c:function:: PyInterpreterGuard *PyInterpreterGuard_FromCurrent(void) Create a finalization guard for the current interpreter. This will prevent finalization until the guard is closed. For example: .. code-block:: c // Temporarily prevent finalization. PyInterpreterGuard *guard = PyInterpreterGuard_FromCurrent(); if (guard == NULL) { // Finalization has already started or we're out of memory. return NULL; } Py_BEGIN_ALLOW_THREADS; // Do some critical processing here. For example, we can safely acquire // locks that might be acquired by the finalization thread. Py_END_ALLOW_THREADS; // Now that we're done with our critical processing, the interpreter is // allowed to finalize again. PyInterpreterGuard_Close(guard); On success, this function returns a guard for the current interpreter; on failure, it returns ``NULL`` with an exception set. This function will fail only if the current interpreter has already started finalizing, or if the process is out of memory. The guard pointer returned by this function must be eventually closed with :c:func:`PyInterpreterGuard_Close`; failing to do so will result in the Python process infinitely hanging. The caller must hold an :term:`attached thread state`. .. versionadded:: 3.15 .. c:function:: PyInterpreterGuard *PyInterpreterGuard_FromView(PyInterpreterView *view) Create a finalization guard for an interpreter through a view. On success, this function returns a guard to the interpreter represented by *view*. The view is still valid after calling this function. The guard must eventually be closed with :c:func:`PyInterpreterGuard_Close`. If the interpreter no longer exists, is already finalizing, or out of memory, then this function returns ``NULL`` without setting an exception. The caller does not need to hold an :term:`attached thread state`. .. versionadded:: 3.15 .. c:function:: void PyInterpreterGuard_Close(PyInterpreterGuard *guard) Close an interpreter guard, allowing the interpreter to start finalization if no other guards remain. If an interpreter guard is never closed, the interpreter will infinitely wait when trying to enter finalization! After an interpreter guard is closed, it may not be used in :c:func:`PyThreadState_Ensure`. Doing so will result in undefined behavior. This function cannot fail, and the caller doesn't need to hold an :term:`attached thread state`. .. versionadded:: 3.15 .. _interpreter-views: Interpreter views ----------------- In some cases, it may be necessary to access an interpreter that may have been deleted. This can be done using interpreter views. .. c:type:: PyInterpreterView An opaque view of an interpreter. This is a thread-safe way to access an interpreter that may have be finalizing or already destroyed. .. versionadded:: 3.15 .. c:function:: PyInterpreterView *PyInterpreterView_FromCurrent(void) Create a view to the current interpreter. This function is generally meant to be used alongside :c:func:`PyInterpreterGuard_FromView` or :c:func:`PyThreadState_EnsureFromView`. On success, this function returns a view to the current interpreter; on failure, it returns ``NULL`` with an exception set. The caller must hold an :term:`attached thread state`. .. versionadded:: 3.15 .. c:function:: void PyInterpreterView_Close(PyInterpreterView *view) Close an interpreter view. If an interpreter view is never closed, the view's memory will never be freed, but there are no other consequences. (In contrast, forgetting to close a guard will infinitely hang the main thread during finalization.) This function cannot fail, and the caller doesn't need to hold an :term:`attached thread state`. .. versionadded:: 3.15 .. c:function:: PyInterpreterView *PyInterpreterView_FromMain(void) Create a view for the main interpreter (the first and default interpreter in a Python process; see :c:func:`PyInterpreterState_Main`). On success, this function returns a view to the main interpreter; on failure, it returns ``NULL`` without an exception set. Failure indicates that the process is out of memory. Use this function when an interpreter pointer or view cannot be supplied by the caller, such as when a native threading library does not provide a ``void *arg`` parameter that could carry a :c:type:`PyInterpreterGuard` or :c:type:`PyInterpreterView`. In code that supports subinterpreters, prefer :c:func:`PyInterpreterView_FromCurrent` so the guard tracks the calling interpreter rather than the main one. The caller does not need to hold an :term:`attached thread state`. .. versionadded:: 3.15 Process-wide parameters ----------------------- .. c:function:: const char* Py_GetVersion() Return the version of this Python interpreter. This is a string that looks something like :: "3.0a5+ (py3k:63103M, May 12 2008, 00:53:55) \n[GCC 4.2.3]" .. index:: single: version (in module sys) The first word (up to the first space character) is the current Python version; the first characters are the major and minor version separated by a period. The returned string points into static storage; the caller should not modify its value. The value is available to Python code as :data:`sys.version`. See also the :c:var:`Py_Version` constant. .. c:function:: const char* Py_GetPlatform() .. index:: single: platform (in module sys) Return the platform identifier for the current platform. On Unix, this is formed from the "official" name of the operating system, converted to lower case, followed by the major revision number; e.g., for Solaris 2.x, which is also known as SunOS 5.x, the value is ``'sunos5'``. On macOS, it is ``'darwin'``. On Windows, it is ``'win'``. The returned string points into static storage; the caller should not modify its value. The value is available to Python code as ``sys.platform``. .. c:function:: const char* Py_GetCopyright() Return the official copyright string for the current Python version, for example ``'Copyright 1991-1995 Stichting Mathematisch Centrum, Amsterdam'`` .. index:: single: copyright (in module sys) The returned string points into static storage; the caller should not modify its value. The value is available to Python code as ``sys.copyright``. .. c:function:: const char* Py_GetCompiler() Return an indication of the compiler used to build the current Python version, in square brackets, for example:: "[GCC 2.7.2.2]" .. index:: single: version (in module sys) The returned string points into static storage; the caller should not modify its value. The value is available to Python code as part of the variable ``sys.version``. .. c:function:: const char* Py_GetBuildInfo() Return information about the sequence number and build date and time of the current Python interpreter instance, for example :: "#67, Aug 1 1997, 22:34:28" .. index:: single: version (in module sys) The returned string points into static storage; the caller should not modify its value. The value is available to Python code as part of the variable ``sys.version``. --- ### Doc C Api Intro (`Doc/c-api/intro.rst`) .. highlight:: c .. _api-intro: ************ Introduction ************ The Application Programmer's Interface to Python gives C and C++ programmers access to the Python interpreter at a variety of levels. The API is equally usable from C++, but for brevity it is generally referred to as the Python/C API. There are two fundamentally different reasons for using the Python/C API. The first reason is to write *extension modules* for specific purposes; these are C modules that extend the Python interpreter. This is probably the most common use. The second reason is to use Python as a component in a larger application; this technique is generally referred to as :dfn:`embedding` Python in an application. Writing an extension module is a relatively well-understood process, where a "cookbook" approach works well. There are several tools that automate the process to some extent. While people have embedded Python in other applications since its early existence, the process of embedding Python is less straightforward than writing an extension. Many API functions are useful independent of whether you're embedding or extending Python; moreover, most applications that embed Python will need to provide a custom extension as well, so it's probably a good idea to become familiar with writing an extension before attempting to embed Python in a real application. Language version compatibility ============================== Python's C API is compatible with C11 and C++11 versions of C and C++. This is a lower limit: the C API does not require features from later C/C++ versions. You do *not* need to enable your compiler's "c11 mode". Coding standards ================ If you're writing C code for inclusion in CPython, you **must** follow the guidelines and standards defined in :PEP:`7`. These guidelines apply regardless of the version of Python you are contributing to. Following these conventions is not necessary for your own third party extension modules, unless you eventually expect to contribute them to Python. .. _api-includes: Include Files ============= All function, type and macro definitions needed to use the Python/C API are included in your code by the following line:: #define PY_SSIZE_T_CLEAN #include This implies inclusion of the following standard headers: ````, ````, ````, ````, ```` and ```` (if available). .. note:: Since Python may define some pre-processor definitions which affect the standard headers on some systems, you *must* include :file:`Python.h` before any standard headers are included. It is recommended to always define ``PY_SSIZE_T_CLEAN`` before including ``Python.h``. See :ref:`arg-parsing` for a description of this macro. All user visible names defined by Python.h (except those defined by the included standard headers) have one of the prefixes ``Py`` or ``_Py``. Names beginning with ``_Py`` are for internal use by the Python implementation and should not be used by extension writers. Structure member names do not have a reserved prefix. .. note:: User code should never define names that begin with ``Py`` or ``_Py``. This confuses the reader, and jeopardizes the portability of the user code to future Python versions, which may define additional names beginning with one of these prefixes. The header files are typically installed with Python. On Unix, these are located in the directories :file:`{prefix}/include/pythonversion/` and :file:`{exec_prefix}/include/pythonversion/`, where :option:`prefix <--prefix>` and :option:`exec_prefix <--exec-prefix>` are defined by the corresponding parameters to Python's :program:`configure` script and *version* is ``'%d.%d' % sys.version_info[:2]``. On Windows, the headers are installed in :file:`{prefix}/include`, where ``prefix`` is the installation directory specified to the installer. To include the headers, place both directories (if different) on your compiler's search path for includes. Do *not* place the parent directories on the search path and then use ``#include ``; this will break on multi-platform builds since the platform independent headers under :option:`prefix <--prefix>` include the platform specific headers from :option:`exec_prefix <--exec-prefix>`. C++ users should note that although the API is defined entirely using C, the header files properly declare the entry points to be ``extern "C"``. As a result, there is no need to do anything special to use the API from C++. .. _capi-system-includes: System includes --------------- :file:`Python.h` includes several standard header files. C extensions should include the standard headers that they use, and should not rely on these implicit includes. The implicit includes are: * ```` * ```` (on Windows) * ```` * ```` * ```` * ```` * ```` * ```` * ```` (if present) The following are included for backwards compatibility, unless using :ref:`Limited API ` 3.13 or newer: * ```` * ```` (on POSIX) The following are included for backwards compatibility, unless using :ref:`Limited API ` 3.11 or newer: * ```` * ```` * ```` .. note:: Since Python may define some pre-processor definitions which affect the standard headers on some systems, you *must* include :file:`Python.h` before any standard headers are included. Useful macros ============= Several useful macros are defined in the Python header files. Many are defined closer to where they are useful (for example, :c:macro:`Py_RETURN_NONE`, :c:macro:`PyMODINIT_FUNC`). Others of a more general utility are defined here. This is not necessarily a complete listing. .. c:macro:: Py_CAN_START_THREADS If this macro is defined, then the current system is able to start threads. Currently, all systems supported by CPython (per :pep:`11`), with the exception of some WebAssembly platforms, support starting threads. .. versionadded:: 3.13 .. c:macro:: Py_GETENV(s) Like :samp:`getenv({s})`, but returns ``NULL`` if :option:`-E` was passed on the command line (see :c:member:`PyConfig.use_environment`). Docstring macros ---------------- .. c:macro:: PyDoc_STRVAR(name, str) Creates a variable with name *name* that can be used in docstrings. If Python is built without docstrings (:option:`--without-doc-strings`), the value will be an empty string. Example:: PyDoc_STRVAR(pop_doc, "Remove and return the rightmost element."); static PyMethodDef deque_methods[] = { // ... {"pop", (PyCFunction)deque_pop, METH_NOARGS, pop_doc}, // ... } Expands to :samp:`PyDoc_VAR({name}) = PyDoc_STR({str})`. .. c:macro:: PyDoc_STR(str) Expands to the given input string, or an empty string if docstrings are disabled (:option:`--without-doc-strings`). Example:: static PyMethodDef pysqlite_row_methods[] = { {"keys", (PyCFunction)pysqlite_row_keys, METH_NOARGS, PyDoc_STR("Returns the keys of the row.")}, {NULL, NULL} }; .. c:macro:: PyDoc_VAR(name) Declares a static character array variable with the given *name*. Expands to :samp:`static const char {name}[]` For example:: PyDoc_VAR(python_doc) = PyDoc_STR( "A genus of constricting snakes in the Pythonidae family native " "to the tropics and subtropics of the Eastern Hemisphere."); General utility macros ---------------------- The following macros are for common tasks not specific to Python. .. c:macro:: Py_UNUSED(arg) Use this for unused arguments in a function definition to silence compiler warnings. Example: ``int func(int a, int Py_UNUSED(b)) { return a; }``. .. versionadded:: 3.4 .. c:macro:: Py_GCC_ATTRIBUTE(name) Use a GCC attribute *name*, hiding it from compilers that don't support GCC attributes (such as MSVC). This expands to :samp:`__attribute__(({name)})` on a GCC compiler, and expands to nothing on compilers that don't support GCC attributes. Numeric utilities ^^^^^^^^^^^^^^^^^ .. c:macro:: Py_ABS(x) Return the absolute value of ``x``. The argument may be evaluated more than once. Consequently, do not pass an expression with side-effects directly to this macro. If the result cannot be represented (for example, if ``x`` has :c:macro:`!INT_MIN` value for :c:expr:`int` type), the behavior is undefined. Corresponds roughly to :samp:`(({x}) < 0 ? -({x}) : ({x}))` .. versionadded:: 3.3 .. c:macro:: Py_MAX(x, y) Py_MIN(x, y) Return the larger or smaller of the arguments, respectively. Any arguments may be evaluated more than once. Consequently, do not pass an expression with side-effects directly to this macro. :c:macro:`!Py_MAX` corresponds roughly to :samp:`((({x}) > ({y})) ? ({x}) : ({y}))`. .. versionadded:: 3.3 .. c:macro:: Py_ARITHMETIC_RIGHT_SHIFT(type, integer, positions) Similar to :samp:`{integer} >> {positions}`, but forces sign extension, as the C standard does not define whether a right-shift of a signed integer will perform sign extension or a zero-fill. *integer* should be any signed integer type. *positions* is the number of positions to shift to the right. Both *integer* and *positions* can be evaluated more than once; consequently, avoid directly passing a function call or some other operation with side-effects to this macro. Instead, store the result as a variable and then pass it. *type* is unused and only kept for backwards compatibility. Historically, *type* was used to cast *integer*. .. versionchanged:: 3.1 This macro is now valid for all signed integer types, not just those for which ``unsigned type`` is legal. As a result, *type* is no longer used. .. c:macro:: Py_CHARMASK(c) Argument must be a character or an integer in the range [-128, 127] or [0, 255]. This macro returns ``c`` cast to an ``unsigned char``. Assertion utilities ^^^^^^^^^^^^^^^^^^^ .. c:macro:: Py_UNREACHABLE() Use this when you have a code path that cannot be reached by design. For example, in the ``default:`` clause in a ``switch`` statement for which all possible values are covered in ``case`` statements. Use this in places where you might be tempted to put an ``assert(0)`` or ``abort()`` call. In release mode, the macro helps the compiler to optimize the code, and avoids a warning about unreachable code. For example, the macro is implemented with ``__builtin_unreachable()`` on GCC in release mode. In debug mode, and on unsupported compilers, the macro expands to a call to :c:func:`Py_FatalError`. A use for ``Py_UNREACHABLE()`` is following a call to a function that never returns but that is not declared ``_Noreturn``. If a code path is very unlikely code but can be reached under exceptional case, this macro must not be used. For example, under low memory condition or if a system call returns a value out of the expected range. In this case, it's better to report the error to the caller. If the error cannot be reported to caller, :c:func:`Py_FatalError` can be used. .. versionadded:: 3.7 .. c:macro:: Py_SAFE_DOWNCAST(value, larger, smaller) Cast *value* to type *smaller* from type *larger*, validating that no information was lost. On release builds of Python, this is roughly equivalent to :samp:`(({smaller}) {value})` (in C++, :samp:`static_cast<{smaller}>({value})` will be used instead). On debug builds (implying that :c:macro:`Py_DEBUG` is defined), this asserts that no information was lost with the cast from *larger* to *smaller*. *value*, *larger*, and *smaller* may all be evaluated more than once in the expression; consequently, do not pass an expression with side-effects directly to this macro. .. c:macro:: Py_BUILD_ASSERT(cond) Asserts a compile-time condition *cond*, as a statement. The build will fail if the condition is false or cannot be evaluated at compile time. Corresponds roughly to :samp:`static_assert({cond})` on C23 and above. For example:: Py_BUILD_ASSERT(sizeof(PyTime_t) == sizeof(int64_t)); .. versionadded:: 3.3 .. c:macro:: Py_BUILD_ASSERT_EXPR(cond) Asserts a compile-time condition *cond*, as an expression that evaluates to ``0``. The build will fail if the condition is false or cannot be evaluated at compile time. For example:: #define foo_to_char(foo) \ ((char *)(foo) + Py_BUILD_ASSERT_EXPR(offsetof(struct foo, string) == 0)) .. versionadded:: 3.3 Type size utilities ^^^^^^^^^^^^^^^^^^^ .. c:macro:: Py_ARRAY_LENGTH(array) Compute the length of a statically allocated C array at compile time. The *array* argument must be a C array with a size known at compile time. Passing an array with an unknown size, such as a heap-allocated array, will result in a compilation error on some compilers, or otherwise produce incorrect results. This is roughly equivalent to:: sizeof(array) / sizeof((array)[0]) .. c:macro:: Py_MEMBER_SIZE(type, member) Return the size of a structure (*type*) *member* in bytes. Corresponds roughly to :samp:`sizeof((({type} *)NULL)->{member})`. .. versionadded:: 3.6 Macro definition utilities ^^^^^^^^^^^^^^^^^^^^^^^^^^ .. c:macro:: Py_FORCE_EXPANSION(X) This is equivalent to :samp:`{X}`, which is useful for token-pasting in macros, as macro expansions in *X* are forcefully evaluated by the preprocessor. .. c:macro:: Py_STRINGIFY(x) Convert ``x`` to a C string. For example, ``Py_STRINGIFY(123)`` returns ``"123"``. .. versionadded:: 3.4 Declaration utilities --------------------- The following macros can be used in declarations. They are most useful for defining the C API itself, and have limited use for extension authors. Most of them expand to compiler-specific spellings of common extensions to the C language. .. c:macro:: Py_ALWAYS_INLINE Ask the compiler to always inline a static inline function. The compiler can ignore it and decide to not inline the function. Corresponds to ``always_inline`` attribute in GCC and ``__forceinline`` in MSVC. It can be used to inline performance critical static inline functions when building Python in debug mode with function inlining disabled. For example, MSC disables function inlining when building in debug mode. Marking blindly a static inline function with Py_ALWAYS_INLINE can result in worse performances (due to increased code size for example). The compiler is usually smarter than the developer for the cost/benefit analysis. If Python is :ref:`built in debug mode ` (if the :c:macro:`Py_DEBUG` macro is defined), the :c:macro:`Py_ALWAYS_INLINE` macro does nothing. It must be specified before the function return type. Usage:: static inline Py_ALWAYS_INLINE int random(void) { return 4; } .. versionadded:: 3.11 .. c:macro:: Py_NO_INLINE Disable inlining on a function. For example, it reduces the C stack consumption: useful on LTO+PGO builds which heavily inline code (see :issue:`33720`). Corresponds to the ``noinline`` attribute/specification on GCC and MSVC. Usage:: Py_NO_INLINE static int random(void) { return 4; } .. versionadded:: 3.11 .. c:macro:: Py_DEPRECATED(version) Use this to declare APIs that were deprecated in a specific CPython version. The macro must be placed before the symbol name. Example:: Py_DEPRECATED(3.8) PyAPI_FUNC(int) Py_OldFunction(void); .. versionchanged:: 3.8 MSVC support was added. .. c:macro:: Py_LOCAL(type) Declare a function returning the specified *type* using a fast-calling qualifier for functions that are local to the current file. Semantically, this is equivalent to :samp:`static {type}`. .. c:macro:: Py_LOCAL_INLINE(type) Equivalent to :c:macro:`Py_LOCAL` but additionally requests the function be inlined. .. c:macro:: Py_LOCAL_SYMBOL Macro used to declare a symbol as local to the shared library (hidden). On supported platforms, it ensures the symbol is not exported. On compatible versions of GCC/Clang, it expands to ``__attribute__((visibility("hidden")))``. .. c:macro:: Py_EXPORTED_SYMBOL Macro used to declare a symbol (function or data) as exported. On Windows, this expands to ``__declspec(dllexport)``. On compatible versions of GCC/Clang, it expands to ``__attribute__((visibility("default")))``. This macro is for defining the C API itself; extension modules should not use it. .. c:macro:: Py_IMPORTED_SYMBOL Macro used to declare a symbol as imported. On Windows, this expands to ``__declspec(dllimport)``. This macro is for defining the C API itself; extension modules should not use it. .. c:macro:: PyAPI_FUNC(type) Macro used by CPython to declare a function as part of the C API. Its expansion depends on the platform and build configuration. This macro is intended for defining CPython's C API itself; extension modules should not use it for their own symbols. .. c:macro:: PyAPI_DATA(type) Macro used by CPython to declare a public global variable as part of the C API. Its expansion depends on the platform and build configuration. This macro is intended for defining CPython's C API itself; extension modules should not use it for their own symbols. Outdated macros --------------- The following :term:`soft deprecated` macros have been used to features that have been standardized in C11 (or previous standards). .. c:macro:: Py_ALIGNED(num) On some GCC-like compilers, specify alignment to *num* bytes. This does nothing on other compilers. Use the standard ``alignas`` specifier rather than this macro. .. soft-deprecated:: 3.15 .. c:macro:: PY_FORMAT_SIZE_T The :c:func:`printf` formatting modifier for :c:type:`size_t`. Use ``"z"`` directly instead. .. soft-deprecated:: 3.15 .. c:macro:: Py_LL(number) Py_ULL(number) Use *number* as a ``long long`` or ``unsigned long long`` integer literal, respectively. Expands to *number* followed by ``LL`` or ``LLU``, respectively, but will expand to some compiler-specific suffixes on some older compilers. Consider using the C99 standard suffixes ``LL`` and ``LLU`` directly. .. soft-deprecated:: 3.15 .. c:macro:: PY_LONG_LONG PY_INT32_T PY_UINT32_T PY_INT64_T PY_UINT64_T Aliases for the types :c:type:`!long long`, :c:type:`!int32_t`, :c:type:`!uint32_t`. :c:type:`!int64_t` and :c:type:`!uint64_t`, respectively. Historically, these types needed compiler-specific extensions. .. soft-deprecated:: 3.15 .. c:macro:: PY_LLONG_MIN PY_LLONG_MAX PY_ULLONG_MAX PY_SIZE_MAX Aliases for the values :c:macro:`!LLONG_MIN`, :c:macro:`!LLONG_MAX`, :c:macro:`!ULLONG_MAX`, and :c:macro:`!SIZE_MAX`, respectively. Use these standard names instead. The required header, ````, :ref:`is included ` in ``Python.h``. .. soft-deprecated:: 3.15 .. c:macro:: Py_MEMCPY(dest, src, n) This is an alias to :c:func:`!memcpy`. .. soft-deprecated:: 3.14 Use :c:func:`!memcpy` directly instead. .. c:macro:: Py_UNICODE_SIZE Size of the :c:type:`!wchar_t` type. Use ``sizeof(wchar_t)`` or ``WCHAR_WIDTH/8`` instead. The required header for the latter, ````, :ref:`is included ` in ``Python.h``. .. soft-deprecated:: 3.15 .. c:macro:: Py_UNICODE_WIDE Defined if ``wchar_t`` can hold a Unicode character (UCS-4). Use ``sizeof(wchar_t) >= 4`` instead .. soft-deprecated:: 3.15 .. c:macro:: Py_VA_COPY This is an alias to the C99-standard ``va_copy`` function. Historically, this would use a compiler-specific method to copy a ``va_list``. .. versionchanged:: 3.6 This is now an alias to ``va_copy``. .. soft-deprecated:: 3.15 .. _api-objects: Objects, Types and Reference Counts =================================== .. index:: pair: object; type Most Python/C API functions have one or more arguments as well as a return value of type :c:expr:`PyObject*`. This type is a pointer to an opaque data type representing an arbitrary Python object. Since all Python object types are treated the same way by the Python language in most situations (e.g., assignments, scope rules, and argument passing), it is only fitting that they should be represented by a single C type. Almost all Python objects live on the heap: you never declare an automatic or static variable of type :c:type:`PyObject`, only pointer variables of type :c:expr:`PyObject*` can be declared. The sole exception are the type objects; since these must never be deallocated, they are typically static :c:type:`PyTypeObject` objects. All Python objects (even Python integers) have a :dfn:`type` and a :dfn:`reference count`. An object's type determines what kind of object it is (e.g., an integer, a list, or a user-defined function; there are many more as explained in :ref:`types`). For each of the well-known types there is a macro to check whether an object is of that type; for instance, ``PyList_Check(a)`` is true if (and only if) the object pointed to by *a* is a Python list. .. _api-refcounts: Reference Counts ---------------- The reference count is important because today's computers have a finite (and often severely limited) memory size; it counts how many different places there are that have a :term:`strong reference` to an object. Such a place could be another object, or a global (or static) C variable, or a local variable in some C function. When the last :term:`strong reference` to an object is released (i.e. its reference count becomes zero), the object is deallocated. If it contains references to other objects, those references are released. Those other objects may be deallocated in turn, if there are no more references to them, and so on. (There's an obvious problem with objects that reference each other here; for now, the solution is "don't do that.") .. index:: single: Py_INCREF (C function) single: Py_DECREF (C function) Reference counts are always manipulated explicitly. The normal way is to use the macro :c:func:`Py_INCREF` to take a new reference to an object (i.e. increment its reference count by one), and :c:func:`Py_DECREF` to release that reference (i.e. decrement the reference count by one). The :c:func:`Py_DECREF` macro is considerably more complex than the incref one, since it must check whether the reference count becomes zero and then cause the object's deallocator to be called. The deallocator is a function pointer contained in the object's type structure. The type-specific deallocator takes care of releasing references for other objects contained in the object if this is a compound object type, such as a list, as well as performing any additional finalization that's needed. There's no chance that the reference count can overflow; at least as many bits are used to hold the reference count as there are distinct memory locations in virtual memory (assuming ``sizeof(Py_ssize_t) >= sizeof(void*)``). Thus, the reference count increment is a simple operation. It is not necessary to hold a :term:`strong reference` (i.e. increment the reference count) for every local variable that contains a pointer to an object. In theory, the object's reference count goes up by one when the variable is made to point to it and it goes down by one when the variable goes out of scope. However, these two cancel each other out, so at the end the reference count hasn't changed. The only real reason to use the reference count is to prevent the object from being deallocated as long as our variable is pointing to it. If we know that there is at least one other reference to the object that lives at least as long as our variable, there is no need to take a new :term:`strong reference` (i.e. increment the reference count) temporarily. An important situation where this arises is in objects that are passed as arguments to C functions in an extension module that are called from Python; the call mechanism guarantees to hold a reference to every argument for the duration of the call. However, a common pitfall is to extract an object from a list and hold on to it for a while without taking a new reference. Some other operation might conceivably remove the object from the list, releasing that reference, and possibly deallocating it. The real danger is that innocent-looking operations may invoke arbitrary Python code which could do this; there is a code path which allows control to flow back to the user from a :c:func:`Py_DECREF`, so almost any operation is potentially dangerous. A safe approach is to always use the generic operations (functions whose name begins with ``PyObject_``, ``PyNumber_``, ``PySequence_`` or ``PyMapping_``). These operations always create a new :term:`strong reference` (i.e. increment the reference count) of the object they return. This leaves the caller with the responsibility to call :c:func:`Py_DECREF` when they are done with the result; this soon becomes second nature. .. _api-refcountdetails: Reference Count Details ^^^^^^^^^^^^^^^^^^^^^^^ The reference count behavior of functions in the Python/C API is best explained in terms of *ownership of references*. Ownership pertains to references, never to objects (objects are not owned: they are always shared). "Owning a reference" means being responsible for calling Py_DECREF on it when the reference is no longer needed. Ownership can also be transferred, meaning that the code that receives ownership of the reference then becomes responsible for eventually releasing it by calling :c:func:`Py_DECREF` or :c:func:`Py_XDECREF` when it's no longer needed---or passing on this responsibility (usually to its caller). When a function passes ownership of a reference on to its caller, the caller is said to receive a *new* reference. When no ownership is transferred, the caller is said to *borrow* the reference. Nothing needs to be done for a :term:`borrowed reference`. Conversely, when a calling function passes in a reference to an object, there are two possibilities: the function *steals* a reference to the object, or it does not. *Stealing a reference* means that when you pass a reference to a function, that function assumes that it now owns that reference. Since the new owner can use :c:func:`!Py_DECREF` at its discretion, you (the caller) must not use that reference after the call. .. index:: single: PyList_SetItem (C function) single: PyTuple_SetItem (C function) Few functions steal references; the two notable exceptions are :c:func:`PyList_SetItem` and :c:func:`PyTuple_SetItem`, which steal a reference to the item (but not to the tuple or list into which the item is put!). These functions were designed to steal a reference because of a common idiom for populating a tuple or list with newly created objects; for example, the code to create the tuple ``(1, 2, "three")`` could look like this (forgetting about error handling for the moment; a better way to code this is shown below):: PyObject *t; t = PyTuple_New(3); PyTuple_SetItem(t, 0, PyLong_FromLong(1L)); PyTuple_SetItem(t, 1, PyLong_FromLong(2L)); PyTuple_SetItem(t, 2, PyUnicode_FromString("three")); Here, :c:func:`PyLong_FromLong` returns a new reference which is immediately stolen by :c:func:`PyTuple_SetItem`. When you want to keep using an object although the reference to it will be stolen, use :c:func:`Py_INCREF` to grab another reference before calling the reference-stealing function. Incidentally, :c:func:`PyTuple_SetItem` is the *only* way to set tuple items; :c:func:`PySequence_SetItem` and :c:func:`PyObject_SetItem` refuse to do this since tuples are an immutable data type. You should only use :c:func:`PyTuple_SetItem` for tuples that you are creating yourself. Equivalent code for populating a list can be written using :c:func:`PyList_New` and :c:func:`PyList_SetItem`. However, in practice, you will rarely use these ways of creating and populating a tuple or list. There's a generic function, :c:func:`Py_BuildValue`, that can create most common objects from C values, directed by a :dfn:`format string`. For example, the above two blocks of code could be replaced by the following (which also takes care of the error checking):: PyObject *tuple, *list; tuple = Py_BuildValue("(iis)", 1, 2, "three"); list = Py_BuildValue("[iis]", 1, 2, "three"); It is much more common to use :c:func:`PyObject_SetItem` and friends with items whose references you are only borrowing, like arguments that were passed in to the function you are writing. In that case, their behaviour regarding references is much saner, since you don't have to take a new reference just so you can give that reference away ("have it be stolen"). For example, this function sets all items of a list (actually, any mutable sequence) to a given item:: int set_all(PyObject *target, PyObject *item) { Py_ssize_t i, n; n = PyObject_Length(target); if (n < 0) return -1; for (i = 0; i < n; i++) { PyObject *index = PyLong_FromSsize_t(i); if (!index) return -1; if (PyObject_SetItem(target, index, item) < 0) { Py_DECREF(index); return -1; } Py_DECREF(index); } return 0; } .. index:: single: set_all() The situation is slightly different for function return values. While passing a reference to most functions does not change your ownership responsibilities for that reference, many functions that return a reference to an object give you ownership of the reference. The reason is simple: in many cases, the returned object is created on the fly, and the reference you get is the only reference to the object. Therefore, the generic functions that return object references, like :c:func:`PyObject_GetItem` and :c:func:`PySequence_GetItem`, always return a new reference (the caller becomes the owner of the reference). It is important to realize that whether you own a reference returned by a function depends on which function you call only --- *the plumage* (the type of the object passed as an argument to the function) *doesn't enter into it!* Thus, if you extract an item from a list using :c:func:`PyList_GetItem`, you don't own the reference --- but if you obtain the same item from the same list using :c:func:`PySequence_GetItem` (which happens to take exactly the same arguments), you do own a reference to the returned object. .. index:: single: PyList_GetItem (C function) single: PySequence_GetItem (C function) Here is an example of how you could write a function that computes the sum of the items in a list of integers; once using :c:func:`PyList_GetItem`, and once using :c:func:`PySequence_GetItem`. :: long sum_list(PyObject *list) { Py_ssize_t i, n; long total = 0, value; PyObject *item; n = PyList_Size(list); if (n < 0) return -1; /* Not a list */ for (i = 0; i < n; i++) { item = PyList_GetItem(list, i); /* Can't fail */ if (!PyLong_Check(item)) continue; /* Skip non-integers */ value = PyLong_AsLong(item); if (value == -1 && PyErr_Occurred()) /* Integer too big to fit in a C long, bail out */ return -1; total += value; } return total; } .. index:: single: sum_list() :: long sum_sequence(PyObject *sequence) { Py_ssize_t i, n; long total = 0, value; PyObject *item; n = PySequence_Length(sequence); if (n < 0) return -1; /* Has no length */ for (i = 0; i < n; i++) { item = PySequence_GetItem(sequence, i); if (item == NULL) return -1; /* Not a sequence, or other failure */ if (PyLong_Check(item)) { value = PyLong_AsLong(item); Py_DECREF(item); if (value == -1 && PyErr_Occurred()) /* Integer too big to fit in a C long, bail out */ return -1; total += value; } else { Py_DECREF(item); /* Discard reference ownership */ } } return total; } .. index:: single: sum_sequence() .. _api-types: Types ----- There are few other data types that play a significant role in the Python/C API; most are simple C types such as :c:expr:`int`, :c:expr:`long`, :c:expr:`double` and :c:expr:`char*`. A few structure types are used to describe static tables used to list the functions exported by a module or the data attributes of a new object type, and another is used to describe the value of a complex number. These will be discussed together with the functions that use them. .. c:type:: Py_ssize_t A signed integral type such that ``sizeof(Py_ssize_t) == sizeof(size_t)``. C99 doesn't define such a thing directly (size_t is an unsigned integral type). See :pep:`353` for details. ``PY_SSIZE_T_MAX`` is the largest positive value of type :c:type:`Py_ssize_t`. .. _api-exceptions: Exceptions ========== The Python programmer only needs to deal with exceptions if specific error handling is required; unhandled exceptions are automatically propagated to the caller, then to the caller's caller, and so on, until they reach the top-level interpreter, where they are reported to the user accompanied by a stack traceback. .. index:: single: PyErr_Occurred (C function) For C programmers, however, error checking always has to be explicit. All functions in the Python/C API can raise exceptions, unless an explicit claim is made otherwise in a function's documentation. In general, when a function encounters an error, it sets an exception, discards any object references that it owns, and returns an error indicator. If not documented otherwise, this indicator is either ``NULL`` or ``-1``, depending on the function's return type. A few functions return a Boolean true/false result, with false indicating an error. Very few functions return no explicit error indicator or have an ambiguous return value, and require explicit testing for errors with :c:func:`PyErr_Occurred`. These exceptions are always explicitly documented. .. index:: single: PyErr_SetString (C function) single: PyErr_Clear (C function) Exception state is maintained in per-thread storage (this is equivalent to using global storage in an unthreaded application). A thread can be in one of two states: an exception has occurred, or not. The function :c:func:`PyErr_Occurred` can be used to check for this: it returns a borrowed reference to the exception type object when an exception has occurred, and ``NULL`` otherwise. There are a number of functions to set the exception state: :c:func:`PyErr_SetString` is the most common (though not the most general) function to set the exception state, and :c:func:`PyErr_Clear` clears the exception state. The full exception state consists of three objects (all of which can be ``NULL``): the exception type, the corresponding exception value, and the traceback. These have the same meanings as the Python result of ``sys.exc_info()``; however, they are not the same: the Python objects represent the last exception being handled by a Python :keyword:`try` ... :keyword:`except` statement, while the C level exception state only exists while an exception is being passed on between C functions until it reaches the Python bytecode interpreter's main loop, which takes care of transferring it to ``sys.exc_info()`` and friends. .. index:: single: exc_info (in module sys) Note that starting with Python 1.5, the preferred, thread-safe way to access the exception state from Python code is to call the function :func:`sys.exc_info`, which returns the per-thread exception state for Python code. Also, the semantics of both ways to access the exception state have changed so that a function which catches an exception will save and restore its thread's exception state so as to preserve the exception state of its caller. This prevents common bugs in exception handling code caused by an innocent-looking function overwriting the exception being handled; it also reduces the often unwanted lifetime extension for objects that are referenced by the stack frames in the traceback. As a general principle, a function that calls another function to perform some task should check whether the called function raised an exception, and if so, pass the exception state on to its caller. It should discard any object references that it owns, and return an error indicator, but it should *not* set another exception --- that would overwrite the exception that was just raised, and lose important information about the exact cause of the error. .. index:: single: sum_sequence() A simple example of detecting exceptions and passing them on is shown in the :c:func:`!sum_sequence` example above. It so happens that this example doesn't need to clean up any owned references when it detects an error. The following example function shows some error cleanup. First, to remind you why you like Python, we show the equivalent Python code:: def incr_item(dict, key): try: item = dict[key] except KeyError: item = 0 dict[key] = item + 1 .. index:: single: incr_item() Here is the corresponding C code, in all its glory:: int incr_item(PyObject *dict, PyObject *key) { /* Objects all initialized to NULL for Py_XDECREF */ PyObject *item = NULL, *const_one = NULL, *incremented_item = NULL; int rv = -1; /* Return value initialized to -1 (failure) */ item = PyObject_GetItem(dict, key); if (item == NULL) { /* Handle KeyError only: */ if (!PyErr_ExceptionMatches(PyExc_KeyError)) goto error; /* Clear the error and use zero: */ PyErr_Clear(); item = PyLong_FromLong(0L); if (item == NULL) goto error; } const_one = PyLong_FromLong(1L); if (const_one == NULL) goto error; incremented_item = PyNumber_Add(item, const_one); if (incremented_item == NULL) goto error; if (PyObject_SetItem(dict, key, incremented_item) < 0) goto error; rv = 0; /* Success */ /* Continue with cleanup code */ error: /* Cleanup code, shared by success and failure path */ /* Use Py_XDECREF() to ignore NULL references */ Py_XDECREF(item); Py_XDECREF(const_one); Py_XDECREF(incremented_item); return rv; /* -1 for error, 0 for success */ } .. index:: single: incr_item() .. index:: single: PyErr_ExceptionMatches (C function) single: PyErr_Clear (C function) single: Py_XDECREF (C function) This example represents an endorsed use of the ``goto`` statement in C! It illustrates the use of :c:func:`PyErr_ExceptionMatches` and :c:func:`PyErr_Clear` to handle specific exceptions, and the use of :c:func:`Py_XDECREF` to dispose of owned references that may be ``NULL`` (note the ``'X'`` in the name; :c:func:`Py_DECREF` would crash when confronted with a ``NULL`` reference). It is important that the variables used to hold owned references are initialized to ``NULL`` for this to work; likewise, the proposed return value is initialized to ``-1`` (failure) and only set to success after the final call made is successful. .. _api-embedding: Embedding Python ================ The one important task that only embedders (as opposed to extension writers) of the Python interpreter have to worry about is the initialization, and possibly the finalization, of the Python interpreter. Most functionality of the interpreter can only be used after the interpreter has been initialized. .. index:: single: Py_Initialize (C function) pair: module; builtins pair: module; __main__ pair: module; sys triple: module; search; path single: path (in module sys) The basic initialization function is :c:func:`Py_Initialize`. This initializes the table of loaded modules, and creates the fundamental modules :mod:`builtins`, :mod:`__main__`, and :mod:`sys`. It also initializes the module search path (``sys.path``). :c:func:`Py_Initialize` does not set the "script argument list" (``sys.argv``). If this variable is needed by Python code that will be executed later, setting :c:member:`PyConfig.argv` and :c:member:`PyConfig.parse_argv` must be set: see :ref:`Python Initialization Configuration `. On most systems (in particular, on Unix and Windows, although the details are slightly different), :c:func:`Py_Initialize` calculates the module search path based upon its best guess for the location of the standard Python interpreter executable, assuming that the Python library is found in a fixed location relative to the Python interpreter executable. In particular, it looks for a directory named :file:`lib/python{X.Y}` relative to the parent directory where the executable named :file:`python` is found on the shell command search path (the environment variable :envvar:`PATH`). For instance, if the Python executable is found in :file:`/usr/local/bin/python`, it will assume that the libraries are in :file:`/usr/local/lib/python{X.Y}`. (In fact, this particular path is also the "fallback" location, used when no executable file named :file:`python` is found along :envvar:`PATH`.) The user can override this behavior by setting the environment variable :envvar:`PYTHONHOME`, or insert additional directories in front of the standard path by setting :envvar:`PYTHONPATH`. The embedding application can steer the search by setting :c:member:`PyConfig.program_name` *before* calling :c:func:`Py_InitializeFromConfig`. Note that :envvar:`PYTHONHOME` still overrides this and :envvar:`PYTHONPATH` is still inserted in front of the standard path. .. index:: single: Py_IsInitialized (C function) Sometimes, it is desirable to "uninitialize" Python. For instance, the application may want to start over (make another call to :c:func:`Py_Initialize`) or the application is simply done with its use of Python and wants to free memory allocated by Python. This can be accomplished by calling :c:func:`Py_FinalizeEx`. The function :c:func:`Py_IsInitialized` returns true if Python is currently in the initialized state. More information about these functions is given in a later chapter. Notice that :c:func:`Py_FinalizeEx` does *not* free all memory allocated by the Python interpreter, e.g. memory allocated by extension modules currently cannot be released. .. _api-debugging: Debugging Builds ================ Python can be built with several macros to enable extra checks of the interpreter and extension modules. These checks tend to add a large amount of overhead to the runtime so they are not enabled by default. A full list of the various types of debugging builds is in the file :file:`Misc/SpecialBuilds.txt` in the Python source distribution. Builds are available that support tracing of reference counts, debugging the memory allocator, or low-level profiling of the main interpreter loop. Only the most frequently used builds will be described in the remainder of this section. .. c:macro:: Py_DEBUG Compiling the interpreter with the :c:macro:`!Py_DEBUG` macro defined produces what is generally meant by :ref:`a debug build of Python `. On Unix, :c:macro:`!Py_DEBUG` can be enabled by adding :option:`--with-pydebug` to the :file:`./configure` command. This will also disable compiler optimization. On Windows, selecting a debug build (e.g., by passing the :option:`-d` option to :file:`PCbuild/build.bat`) automatically enables :c:macro:`!Py_DEBUG`. Additionally, the presence of the not-Python-specific :c:macro:`!_DEBUG` macro, when defined by the compiler, will also implicitly enable :c:macro:`!Py_DEBUG`. In addition to the reference count debugging described below, extra checks are performed. See :ref:`Python Debug Build ` for more details. Defining ``Py_TRACE_REFS`` enables reference tracing (see the :option:`configure --with-trace-refs option <--with-trace-refs>`). When defined, a circular doubly linked list of active objects is maintained by adding two extra fields to every :c:type:`PyObject`. Total allocations are tracked as well. Upon exit, all existing references are printed. (In interactive mode this happens after every statement run by the interpreter.) Please refer to :file:`Misc/SpecialBuilds.txt` in the Python source distribution for more detailed information. .. _c-api-tools: Recommended third party tools ============================= The following third party tools offer both simpler and more sophisticated approaches to creating C, C++ and Rust extensions for Python: * `Cython `_ * `cffi `_ * `HPy `_ * `nanobind `_ (C++) * `Numba `_ * `pybind11 `_ (C++) * `PyO3 `_ (Rust) * `SWIG `_ Using tools such as these can help avoid writing code that is tightly bound to a particular version of CPython, avoid reference counting errors, and focus more on your own code than on using the CPython API. In general, new versions of Python can be supported by updating the tool, and your code will often use newer and more efficient APIs automatically. Some tools also support compiling for other implementations of Python from a single set of sources. These projects are not supported by the same people who maintain Python, and issues need to be raised with the projects directly. Remember to check that the project is still maintained and supported, as the list above may become outdated. .. seealso:: `Python Packaging User Guide: Binary Extensions `_ The Python Packaging User Guide not only covers several available tools that simplify the creation of binary extensions, but also discusses the various reasons why creating an extension module may be desirable in the first place. --- ### Doc C Api Iter (`Doc/c-api/iter.rst`) .. highlight:: c .. _iterator: Iterator Protocol ================= There are two functions specifically for working with iterators. .. c:function:: int PyIter_Check(PyObject *o) Return non-zero if the object *o* can be safely passed to :c:func:`PyIter_NextItem` and ``0`` otherwise. This function always succeeds. .. c:function:: int PyAIter_Check(PyObject *o) Return non-zero if the object *o* provides the :class:`AsyncIterator` protocol, and ``0`` otherwise. This function always succeeds. .. versionadded:: 3.10 .. c:function:: int PyIter_NextItem(PyObject *iter, PyObject **item) Return ``1`` and set *item* to a :term:`strong reference` of the next value of the iterator *iter* on success. Return ``0`` and set *item* to ``NULL`` if there are no remaining values. Return ``-1``, set *item* to ``NULL`` and set an exception on error. .. versionadded:: 3.14 .. c:function:: PyObject* PyIter_Next(PyObject *o) This is an older version of :c:func:`!PyIter_NextItem`, which is retained for backwards compatibility. Prefer :c:func:`PyIter_NextItem`. Return the next value from the iterator *o*. The object must be an iterator according to :c:func:`PyIter_Check` (it is up to the caller to check this). If there are no remaining values, returns ``NULL`` with no exception set. If an error occurs while retrieving the item, returns ``NULL`` and passes along the exception. .. c:type:: PySendResult The enum value used to represent different results of :c:func:`PyIter_Send`. .. versionadded:: 3.10 .. c:function:: PySendResult PyIter_Send(PyObject *iter, PyObject *arg, PyObject **presult) Sends the *arg* value into the iterator *iter*. Returns: - ``PYGEN_RETURN`` if iterator returns. Return value is returned via *presult*. - ``PYGEN_NEXT`` if iterator yields. Yielded value is returned via *presult*. - ``PYGEN_ERROR`` if iterator has raised an exception. *presult* is set to ``NULL``. .. versionadded:: 3.10 --- ### Doc C Api Iterator (`Doc/c-api/iterator.rst`) .. highlight:: c .. _iterator-objects: Iterator Objects ---------------- Python provides two general-purpose iterator objects. The first, a sequence iterator, works with an arbitrary sequence supporting the :meth:`~object.__getitem__` method. The second works with a callable object and a sentinel value, calling the callable for each item in the sequence, and ending the iteration when the sentinel value is returned. .. c:var:: PyTypeObject PySeqIter_Type Type object for iterator objects returned by :c:func:`PySeqIter_New` and the one-argument form of the :func:`iter` built-in function for built-in sequence types. .. c:function:: int PySeqIter_Check(PyObject *op) Return true if the type of *op* is :c:data:`PySeqIter_Type`. This function always succeeds. .. c:function:: PyObject* PySeqIter_New(PyObject *seq) Return an iterator that works with a general sequence object, *seq*. The iteration ends when the sequence raises :exc:`IndexError` for the subscripting operation. .. c:var:: PyTypeObject PyCallIter_Type Type object for iterator objects returned by :c:func:`PyCallIter_New` and the two-argument form of the :func:`iter` built-in function. .. c:function:: int PyCallIter_Check(PyObject *op) Return true if the type of *op* is :c:data:`PyCallIter_Type`. This function always succeeds. .. c:function:: PyObject* PyCallIter_New(PyObject *callable, PyObject *sentinel) Return a new iterator. The first parameter, *callable*, can be any Python callable object that can be called with no parameters; each call to it should return the next item in the iteration. When *callable* returns a value equal to *sentinel*, the iteration will be terminated. Range Objects ^^^^^^^^^^^^^ .. c:var:: PyTypeObject PyRange_Type The type object for :class:`range` objects. .. c:function:: int PyRange_Check(PyObject *o) Return true if the object *o* is an instance of a :class:`range` object. This function always succeeds. Builtin Iterator Types ^^^^^^^^^^^^^^^^^^^^^^ These are built-in iteration types that are included in Python's C API, but provide no additional functions. They are here for completeness. .. list-table:: :widths: auto :header-rows: 1 * * C type * Python type * * .. c:var:: PyTypeObject PyEnum_Type * :py:class:`enumerate` * * .. c:var:: PyTypeObject PyFilter_Type * :py:class:`filter` * * .. c:var:: PyTypeObject PyMap_Type * :py:class:`map` * * .. c:var:: PyTypeObject PyReversed_Type * :py:class:`reversed` * * .. c:var:: PyTypeObject PyZip_Type * :py:class:`zip` Other Iterator Objects ^^^^^^^^^^^^^^^^^^^^^^ .. c:var:: PyTypeObject PyByteArrayIter_Type .. c:var:: PyTypeObject PyBytesIter_Type .. c:var:: PyTypeObject PyListIter_Type .. c:var:: PyTypeObject PyListRevIter_Type .. c:var:: PyTypeObject PySetIter_Type .. c:var:: PyTypeObject PyTupleIter_Type .. c:var:: PyTypeObject PyRangeIter_Type .. c:var:: PyTypeObject PyLongRangeIter_Type .. c:var:: PyTypeObject PyDictIterKey_Type .. c:var:: PyTypeObject PyDictRevIterKey_Type .. c:var:: PyTypeObject PyDictIterValue_Type .. c:var:: PyTypeObject PyDictRevIterValue_Type .. c:var:: PyTypeObject PyDictIterItem_Type .. c:var:: PyTypeObject PyDictRevIterItem_Type .. c:var:: PyTypeObject PyODictIter_Type Type objects for iterators of various built-in objects. Do not create instances of these directly; prefer calling :c:func:`PyObject_GetIter` instead. Note that there is no guarantee that a given built-in type uses a given iterator type. For example, iterating over :class:`range` will use one of two iterator types depending on the size of the range. Other types may start using a similar scheme in the future, without warning. --- ### Doc C Api Lifecycle (`Doc/c-api/lifecycle.rst`) .. highlight:: c .. _life-cycle: Object Life Cycle ================= This section explains how a type's slots relate to each other throughout the life of an object. It is not intended to be a complete canonical reference for the slots; instead, refer to the slot-specific documentation in :ref:`type-structs` for details about a particular slot. Life Events ----------- The figure below illustrates the order of events that can occur throughout an object's life. An arrow from *A* to *B* indicates that event *B* can occur after event *A* has occurred, with the arrow's label indicating the condition that must be true for *B* to occur after *A*. .. only:: builder_html .. raw:: html .. raw:: html :file: lifecycle.dot.svg .. raw:: html .. only:: not builder_html .. image:: lifecycle.dot.svg :align: center :class: invert-in-dark-mode :alt: Diagram showing events in an object's life. Explained in detail below. .. container:: :name: life-events-graph-description Explanation: * When a new object is constructed by calling its type: #. :c:member:`~PyTypeObject.tp_new` is called to create a new object. #. :c:member:`~PyTypeObject.tp_alloc` is directly called by :c:member:`~PyTypeObject.tp_new` to allocate the memory for the new object. #. :c:member:`~PyTypeObject.tp_init` initializes the newly created object. :c:member:`!tp_init` can be called again to re-initialize an object, if desired. The :c:member:`!tp_init` call can also be skipped entirely, for example by Python code calling :py:meth:`~object.__new__`. * After :c:member:`!tp_init` completes, the object is ready to use. * Some time after the last reference to an object is removed: #. If an object is not marked as *finalized*, it might be finalized by marking it as *finalized* and calling its :c:member:`~PyTypeObject.tp_finalize` function. Python does *not* finalize an object when the last reference to it is deleted; use :c:func:`PyObject_CallFinalizerFromDealloc` to ensure that :c:member:`~PyTypeObject.tp_finalize` is always called. #. If the object is marked as finalized, :c:member:`~PyTypeObject.tp_clear` might be called by the garbage collector to clear references held by the object. It is *not* called when the object's reference count reaches zero. #. :c:member:`~PyTypeObject.tp_dealloc` is called to destroy the object. To avoid code duplication, :c:member:`~PyTypeObject.tp_dealloc` typically calls into :c:member:`~PyTypeObject.tp_clear` to free up the object's references. #. When :c:member:`~PyTypeObject.tp_dealloc` finishes object destruction, it directly calls :c:member:`~PyTypeObject.tp_free` (usually set to :c:func:`PyObject_Free` or :c:func:`PyObject_GC_Del` automatically as appropriate for the type) to deallocate the memory. * The :c:member:`~PyTypeObject.tp_finalize` function is permitted to add a reference to the object if desired. If it does, the object is *resurrected*, preventing its pending destruction. (Only :c:member:`!tp_finalize` is allowed to resurrect an object; :c:member:`~PyTypeObject.tp_clear` and :c:member:`~PyTypeObject.tp_dealloc` cannot without calling into :c:member:`!tp_finalize`.) Resurrecting an object may or may not cause the object's *finalized* mark to be removed. Currently, Python does not remove the *finalized* mark from a resurrected object if it supports garbage collection (i.e., the :c:macro:`Py_TPFLAGS_HAVE_GC` flag is set) but does remove the mark if the object does not support garbage collection; either or both of these behaviors may change in the future. * :c:member:`~PyTypeObject.tp_dealloc` can optionally call :c:member:`~PyTypeObject.tp_finalize` via :c:func:`PyObject_CallFinalizerFromDealloc` if it wishes to reuse that code to help with object destruction. This is recommended because it guarantees that :c:member:`!tp_finalize` is always called before destruction. See the :c:member:`~PyTypeObject.tp_dealloc` documentation for example code. * If the object is a member of a :term:`cyclic isolate` and either :c:member:`~PyTypeObject.tp_clear` fails to break the reference cycle or the cyclic isolate is not detected (perhaps :func:`gc.disable` was called, or the :c:macro:`Py_TPFLAGS_HAVE_GC` flag was erroneously omitted in one of the involved types), the objects remain indefinitely uncollectable (they "leak"). See :data:`gc.garbage`. If the object is marked as supporting garbage collection (the :c:macro:`Py_TPFLAGS_HAVE_GC` flag is set in :c:member:`~PyTypeObject.tp_flags`), the following events are also possible: * The garbage collector occasionally calls :c:member:`~PyTypeObject.tp_traverse` to identify :term:`cyclic isolates `. * When the garbage collector discovers a :term:`cyclic isolate`, it finalizes one of the objects in the group by marking it as *finalized* and calling its :c:member:`~PyTypeObject.tp_finalize` function, if it has one. This repeats until the cyclic isolate doesn't exist or all of the objects have been finalized. * :c:member:`~PyTypeObject.tp_finalize` is permitted to resurrect the object by adding a reference from outside the :term:`cyclic isolate`. The new reference causes the group of objects to no longer form a cyclic isolate (the reference cycle may still exist, but if it does the objects are no longer isolated). * When the garbage collector discovers a :term:`cyclic isolate` and all of the objects in the group have already been marked as *finalized*, the garbage collector clears one or more of the uncleared objects in the group (possibly concurrently) by calling each's :c:member:`~PyTypeObject.tp_clear` function. This repeats as long as the cyclic isolate still exists and not all of the objects have been cleared. Cyclic Isolate Destruction -------------------------- Listed below are the stages of life of a hypothetical :term:`cyclic isolate` that continues to exist after each member object is finalized or cleared. It is a memory leak if a cyclic isolate progresses through all of these stages; it should vanish once all objects are cleared, if not sooner. A cyclic isolate can vanish either because the reference cycle is broken or because the objects are no longer isolated due to finalizer resurrection (see :c:member:`~PyTypeObject.tp_finalize`). 0. **Reachable** (not yet a cyclic isolate): All objects are in their normal, reachable state. A reference cycle could exist, but an external reference means the objects are not yet isolated. #. **Unreachable but consistent:** The final reference from outside the cyclic group of objects has been removed, causing the objects to become isolated (thus a cyclic isolate is born). None of the group's objects have been finalized or cleared yet. The cyclic isolate remains at this stage until some future run of the garbage collector (not necessarily the next run because the next run might not scan every object). #. **Mix of finalized and not finalized:** Objects in a cyclic isolate are finalized one at a time, which means that there is a period of time when the cyclic isolate is composed of a mix of finalized and non-finalized objects. Finalization order is unspecified, so it can appear random. A finalized object must behave in a sane manner when non-finalized objects interact with it, and a non-finalized object must be able to tolerate the finalization of an arbitrary subset of its referents. #. **All finalized:** All objects in a cyclic isolate are finalized before any of them are cleared. #. **Mix of finalized and cleared:** The objects can be cleared serially or concurrently (but with the :term:`GIL` held); either way, some will finish before others. A finalized object must be able to tolerate the clearing of a subset of its referents. :pep:`442` calls this stage "cyclic trash". #. **Leaked:** If a cyclic isolate still exists after all objects in the group have been finalized and cleared, then the objects remain indefinitely uncollectable (see :data:`gc.garbage`). It is a bug if a cyclic isolate reaches this stage---it means the :c:member:`~PyTypeObject.tp_clear` methods of the participating objects have failed to break the reference cycle as required. If :c:member:`~PyTypeObject.tp_clear` did not exist, then Python would have no way to safely break a reference cycle. Simply destroying an object in a cyclic isolate would result in a dangling pointer, triggering undefined behavior when an object referencing the destroyed object is itself destroyed. The clearing step makes object destruction a two-phase process: first :c:member:`~PyTypeObject.tp_clear` is called to partially destroy the objects enough to detangle them from each other, then :c:member:`~PyTypeObject.tp_dealloc` is called to complete the destruction. Unlike clearing, finalization is not a phase of destruction. A finalized object must still behave properly by continuing to fulfill its design contracts. An object's finalizer is allowed to execute arbitrary Python code, and is even allowed to prevent the impending destruction by adding a reference. The finalizer is only related to destruction by call order---if it runs, it runs before destruction, which starts with :c:member:`~PyTypeObject.tp_clear` (if called) and concludes with :c:member:`~PyTypeObject.tp_dealloc`. The finalization step is not necessary to safely reclaim the objects in a cyclic isolate, but its existence makes it easier to design types that behave in a sane manner when objects are cleared. Clearing an object might necessarily leave it in a broken, partially destroyed state---it might be unsafe to call any of the cleared object's methods or access any of its attributes. With finalization, only finalized objects can possibly interact with cleared objects; non-finalized objects are guaranteed to interact with only non-cleared (but potentially finalized) objects. To summarize the possible interactions: * A non-finalized object might have references to or from non-finalized and finalized objects, but not to or from cleared objects. * A finalized object might have references to or from non-finalized, finalized, and cleared objects. * A cleared object might have references to or from finalized and cleared objects, but not to or from non-finalized objects. Without any reference cycles, an object can be simply destroyed once its last reference is deleted; the finalization and clearing steps are not necessary to safely reclaim unused objects. However, it can be useful to automatically call :c:member:`~PyTypeObject.tp_finalize` and :c:member:`~PyTypeObject.tp_clear` before destruction anyway because type design is simplified when all objects always experience the same series of events regardless of whether they participated in a cyclic isolate. Python currently only calls :c:member:`~PyTypeObject.tp_finalize` and :c:member:`~PyTypeObject.tp_clear` as needed to destroy a cyclic isolate; this may change in a future version. Functions --------- To allocate and free memory, see :ref:`allocating-objects`. .. c:function:: void PyObject_CallFinalizer(PyObject *op) Finalizes the object as described in :c:member:`~PyTypeObject.tp_finalize`. Call this function (or :c:func:`PyObject_CallFinalizerFromDealloc`) instead of calling :c:member:`~PyTypeObject.tp_finalize` directly because this function may deduplicate multiple calls to :c:member:`!tp_finalize`. Currently, calls are only deduplicated if the type supports garbage collection (i.e., the :c:macro:`Py_TPFLAGS_HAVE_GC` flag is set); this may change in the future. .. versionadded:: 3.4 .. c:function:: int PyObject_CallFinalizerFromDealloc(PyObject *op) Same as :c:func:`PyObject_CallFinalizer` but meant to be called at the beginning of the object's destructor (:c:member:`~PyTypeObject.tp_dealloc`). There must not be any references to the object. If the object's finalizer resurrects the object, this function returns -1; no further destruction should happen. Otherwise, this function returns 0 and destruction can continue normally. .. versionadded:: 3.4 .. seealso:: :c:member:`~PyTypeObject.tp_dealloc` for example code. --- ### Doc C Api List (`Doc/c-api/list.rst`) .. highlight:: c .. _listobjects: List Objects ------------ .. index:: pair: object; list .. c:type:: PyListObject This subtype of :c:type:`PyObject` represents a Python list object. .. c:var:: PyTypeObject PyList_Type This instance of :c:type:`PyTypeObject` represents the Python list type. This is the same object as :class:`list` in the Python layer. .. c:function:: int PyList_Check(PyObject *p) Return true if *p* is a list object or an instance of a subtype of the list type. This function always succeeds. .. c:function:: int PyList_CheckExact(PyObject *p) Return true if *p* is a list object, but not an instance of a subtype of the list type. This function always succeeds. .. c:function:: PyObject* PyList_New(Py_ssize_t len) Return a new list of length *len* on success, or ``NULL`` on failure. .. note:: If *len* is greater than zero, the returned list object's items are set to ``NULL``. Thus you cannot use abstract API functions such as :c:func:`PySequence_SetItem` or expose the object to Python code before setting all items to a real object with :c:func:`PyList_SetItem` or :c:func:`PyList_SET_ITEM()`. The following APIs are safe APIs before the list is fully initialized: :c:func:`PyList_SetItem()` and :c:func:`PyList_SET_ITEM()`. .. c:function:: Py_ssize_t PyList_Size(PyObject *list) .. index:: pair: built-in function; len Return the length of the list object in *list*; this is equivalent to ``len(list)`` on a list object. .. c:function:: Py_ssize_t PyList_GET_SIZE(PyObject *list) Similar to :c:func:`PyList_Size`, but without error checking. .. c:function:: PyObject* PyList_GetItemRef(PyObject *list, Py_ssize_t index) Return the object at position *index* in the list pointed to by *list*. The position must be non-negative; indexing from the end of the list is not supported. If *index* is out of bounds (:code:`<0 or >=len(list)`), return ``NULL`` and set an :exc:`IndexError` exception. .. versionadded:: 3.13 .. c:function:: PyObject* PyList_GetItem(PyObject *list, Py_ssize_t index) Like :c:func:`PyList_GetItemRef`, but returns a :term:`borrowed reference` instead of a :term:`strong reference`. .. note:: In the :term:`free-threaded build`, the returned :term:`borrowed reference` may become invalid if another thread modifies the list concurrently. Prefer :c:func:`PyList_GetItemRef`, which returns a :term:`strong reference`. .. c:function:: PyObject* PyList_GET_ITEM(PyObject *list, Py_ssize_t i) Similar to :c:func:`PyList_GetItem`, but without error checking. .. note:: In the :term:`free-threaded build`, the returned :term:`borrowed reference` may become invalid if another thread modifies the list concurrently. Prefer :c:func:`PyList_GetItemRef`, which returns a :term:`strong reference`. .. c:function:: int PyList_SetItem(PyObject *list, Py_ssize_t index, PyObject *item) Set the item at index *index* in list to *item*. Return ``0`` on success. If *index* is out of bounds, return ``-1`` and set an :exc:`IndexError` exception. .. note:: This function ":term:`steals `" a reference to *item*, even on error. On success, it discards a reference to an item already in the list at the affected position (unless it was ``NULL``). .. c:function:: void PyList_SET_ITEM(PyObject *list, Py_ssize_t i, PyObject *o) Macro form of :c:func:`PyList_SetItem` without error checking. This is normally only used to fill in new lists where there is no previous content. Bounds checking is performed as an assertion if Python is built in :ref:`debug mode ` or :option:`with assertions <--with-assertions>`. .. note:: This macro ":term:`steals `" a reference to *item*, and, unlike :c:func:`PyList_SetItem`, does *not* discard a reference to any item that is being replaced; any reference in *list* at position *i* will be leaked. .. note:: In the :term:`free-threaded build`, this macro has no internal synchronization. It is normally only used to fill in new lists where no other thread has a reference to the list. If the list may be shared, use :c:func:`PyList_SetItem` instead, which uses a :term:`per-object lock`. .. c:function:: int PyList_Insert(PyObject *list, Py_ssize_t index, PyObject *item) Insert the item *item* into list *list* in front of index *index*. Return ``0`` if successful; return ``-1`` and set an exception if unsuccessful. Analogous to ``list.insert(index, item)``. .. c:function:: int PyList_Append(PyObject *list, PyObject *item) Append the object *item* at the end of list *list*. Return ``0`` if successful; return ``-1`` and set an exception if unsuccessful. Analogous to ``list.append(item)``. .. c:function:: PyObject* PyList_GetSlice(PyObject *list, Py_ssize_t low, Py_ssize_t high) Return a list of the objects in *list* containing the objects *between* *low* and *high*. Return ``NULL`` and set an exception if unsuccessful. Analogous to ``list[low:high]``. Indexing from the end of the list is not supported. .. c:function:: int PyList_SetSlice(PyObject *list, Py_ssize_t low, Py_ssize_t high, PyObject *itemlist) Set the slice of *list* between *low* and *high* to the contents of *itemlist*. Analogous to ``list[low:high] = itemlist``. The *itemlist* may be ``NULL``, indicating the assignment of an empty list (slice deletion). Return ``0`` on success, ``-1`` on failure. Indexing from the end of the list is not supported. .. note:: In the :term:`free-threaded build`, when *itemlist* is a :class:`list`, both *list* and *itemlist* are locked for the duration of the operation. For other iterables (or ``NULL``), only *list* is locked. .. c:function:: int PyList_Extend(PyObject *list, PyObject *iterable) Extend *list* with the contents of *iterable*. This is the same as ``PyList_SetSlice(list, PY_SSIZE_T_MAX, PY_SSIZE_T_MAX, iterable)`` and analogous to ``list.extend(iterable)`` or ``list += iterable``. Raise an exception and return ``-1`` if *list* is not a :class:`list` object. Return 0 on success. .. versionadded:: 3.13 .. note:: In the :term:`free-threaded build`, when *iterable* is a :class:`list`, :class:`set`, :class:`dict`, or dict view, both *list* and *iterable* (or its underlying dict) are locked for the duration of the operation. For other iterables, only *list* is locked; *iterable* may be concurrently modified by another thread. .. c:function:: int PyList_Clear(PyObject *list) Remove all items from *list*. This is the same as ``PyList_SetSlice(list, 0, PY_SSIZE_T_MAX, NULL)`` and analogous to ``list.clear()`` or ``del list[:]``. Raise an exception and return ``-1`` if *list* is not a :class:`list` object. Return 0 on success. .. versionadded:: 3.13 .. c:function:: int PyList_Sort(PyObject *list) Sort the items of *list* in place. Return ``0`` on success, ``-1`` on failure. This is equivalent to ``list.sort()``. .. note:: In the :term:`free-threaded build`, element comparison via :meth:`~object.__lt__` can execute arbitrary Python code, during which the :term:`per-object lock` may be temporarily released. For built-in types (:class:`str`, :class:`int`, :class:`float`), the lock is not released during comparison. .. c:function:: int PyList_Reverse(PyObject *list) Reverse the items of *list* in place. Return ``0`` on success, ``-1`` on failure. This is the equivalent of ``list.reverse()``. .. c:function:: PyObject* PyList_AsTuple(PyObject *list) .. index:: pair: built-in function; tuple Return a new tuple object containing the contents of *list*; equivalent to ``tuple(list)``. --- ### Doc C Api Long (`Doc/c-api/long.rst`) .. highlight:: c .. _longobjects: Integer Objects --------------- .. index:: pair: object; long integer pair: object; integer All integers are implemented as "long" integer objects of arbitrary size. On error, most ``PyLong_As*`` APIs return ``(return type)-1`` which cannot be distinguished from a number. Use :c:func:`PyErr_Occurred` to disambiguate. .. c:type:: PyLongObject This subtype of :c:type:`PyObject` represents a Python integer object. .. c:var:: PyTypeObject PyLong_Type This instance of :c:type:`PyTypeObject` represents the Python integer type. This is the same object as :class:`int` in the Python layer. .. c:function:: int PyLong_Check(PyObject *p) Return true if its argument is a :c:type:`PyLongObject` or a subtype of :c:type:`PyLongObject`. This function always succeeds. .. c:function:: int PyLong_CheckExact(PyObject *p) Return true if its argument is a :c:type:`PyLongObject`, but not a subtype of :c:type:`PyLongObject`. This function always succeeds. .. c:function:: PyObject* PyLong_FromLong(long v) Return a new :c:type:`PyLongObject` object from *v*, or ``NULL`` on failure. .. impl-detail:: CPython keeps an array of integer objects for all integers between ``-5`` and ``1024``. When you create an int in that range you actually just get back a reference to the existing object. .. c:function:: PyObject* PyLong_FromUnsignedLong(unsigned long v) Return a new :c:type:`PyLongObject` object from a C :c:expr:`unsigned long`, or ``NULL`` on failure. .. c:function:: PyObject* PyLong_FromSsize_t(Py_ssize_t v) Return a new :c:type:`PyLongObject` object from a C :c:type:`Py_ssize_t`, or ``NULL`` on failure. .. c:function:: PyObject* PyLong_FromSize_t(size_t v) Return a new :c:type:`PyLongObject` object from a C :c:type:`size_t`, or ``NULL`` on failure. .. c:function:: PyObject* PyLong_FromLongLong(long long v) Return a new :c:type:`PyLongObject` object from a C :c:expr:`long long`, or ``NULL`` on failure. .. c:function:: PyObject* PyLong_FromUnsignedLongLong(unsigned long long v) Return a new :c:type:`PyLongObject` object from a C :c:expr:`unsigned long long`, or ``NULL`` on failure. .. c:function:: PyObject* PyLong_FromInt32(int32_t value) PyObject* PyLong_FromInt64(int64_t value) Return a new :c:type:`PyLongObject` object from a signed C :c:expr:`int32_t` or :c:expr:`int64_t`, or ``NULL`` with an exception set on failure. .. versionadded:: 3.14 .. c:function:: PyObject* PyLong_FromUInt32(uint32_t value) PyObject* PyLong_FromUInt64(uint64_t value) Return a new :c:type:`PyLongObject` object from an unsigned C :c:expr:`uint32_t` or :c:expr:`uint64_t`, or ``NULL`` with an exception set on failure. .. versionadded:: 3.14 .. c:function:: PyObject* PyLong_FromDouble(double v) Return a new :c:type:`PyLongObject` object from the integer part of *v*, or ``NULL`` on failure. .. c:function:: PyObject* PyLong_FromString(const char *str, char **pend, int base) Return a new :c:type:`PyLongObject` based on the string value in *str*, which is interpreted according to the radix in *base*, or ``NULL`` on failure. If *pend* is non-``NULL``, *\*pend* will point to the end of *str* on success or to the first character that could not be processed on error. If *base* is ``0``, *str* is interpreted using the :ref:`integers` definition; in this case, leading zeros in a non-zero decimal number raises a :exc:`ValueError`. If *base* is not ``0``, it must be between ``2`` and ``36``, inclusive. Leading and trailing whitespace and single underscores after a base specifier and between digits are ignored. If there are no digits or *str* is not NULL-terminated following the digits and trailing whitespace, :exc:`ValueError` will be raised. .. seealso:: :c:func:`PyLong_AsNativeBytes()` and :c:func:`PyLong_FromNativeBytes()` functions can be used to convert a :c:type:`PyLongObject` to/from an array of bytes in base ``256``. .. c:function:: PyObject* PyLong_FromUnicodeObject(PyObject *u, int base) Convert a sequence of Unicode digits in the string *u* to a Python integer value. .. versionadded:: 3.3 .. c:function:: PyObject* PyLong_FromVoidPtr(void *p) Create a Python integer from the pointer *p*. The pointer value can be retrieved from the resulting value using :c:func:`PyLong_AsVoidPtr`. .. c:function:: PyObject* PyLong_FromNativeBytes(const void* buffer, size_t n_bytes, int flags) Create a Python integer from the value contained in the first *n_bytes* of *buffer*, interpreted as a two's-complement signed number. *flags* are as for :c:func:`PyLong_AsNativeBytes`. Passing ``-1`` will select the native endian that CPython was compiled with and assume that the most-significant bit is a sign bit. Passing ``Py_ASNATIVEBYTES_UNSIGNED_BUFFER`` will produce the same result as calling :c:func:`PyLong_FromUnsignedNativeBytes`. Other flags are ignored. .. versionadded:: 3.13 .. c:function:: PyObject* PyLong_FromUnsignedNativeBytes(const void* buffer, size_t n_bytes, int flags) Create a Python integer from the value contained in the first *n_bytes* of *buffer*, interpreted as an unsigned number. *flags* are as for :c:func:`PyLong_AsNativeBytes`. Passing ``-1`` will select the native endian that CPython was compiled with and assume that the most-significant bit is not a sign bit. Flags other than endian are ignored. .. versionadded:: 3.13 .. c:macro:: PyLong_FromPid(pid) Macro for creating a Python integer from a process identifier. This can be defined as an alias to :c:func:`PyLong_FromLong` or :c:func:`PyLong_FromLongLong`, depending on the size of the system's PID type. .. versionadded:: 3.2 .. c:function:: long PyLong_AsLong(PyObject *obj) .. index:: single: LONG_MAX (C macro) single: OverflowError (built-in exception) Return a C :c:expr:`long` representation of *obj*. If *obj* is not an instance of :c:type:`PyLongObject`, first call its :meth:`~object.__index__` method (if present) to convert it to a :c:type:`PyLongObject`. Raise :exc:`OverflowError` if the value of *obj* is out of range for a :c:expr:`long`. Returns ``-1`` on error. Use :c:func:`PyErr_Occurred` to disambiguate. .. versionchanged:: 3.8 Use :meth:`~object.__index__` if available. .. versionchanged:: 3.10 This function will no longer use :meth:`~object.__int__`. .. c:namespace:: NULL .. c:function:: long PyLong_AS_LONG(PyObject *obj) Exactly equivalent to the preferred ``PyLong_AsLong``. In particular, it can fail with :exc:`OverflowError` or another exception. .. soft-deprecated:: 3.14 .. c:function:: int PyLong_AsInt(PyObject *obj) Similar to :c:func:`PyLong_AsLong`, but store the result in a C :c:expr:`int` instead of a C :c:expr:`long`. .. versionadded:: 3.13 .. c:function:: long PyLong_AsLongAndOverflow(PyObject *obj, int *overflow) Return a C :c:expr:`long` representation of *obj*. If *obj* is not an instance of :c:type:`PyLongObject`, first call its :meth:`~object.__index__` method (if present) to convert it to a :c:type:`PyLongObject`. If the value of *obj* is greater than :c:macro:`LONG_MAX` or less than :c:macro:`LONG_MIN`, set *\*overflow* to ``1`` or ``-1``, respectively, and return ``-1``; otherwise, set *\*overflow* to ``0``. If any other exception occurs set *\*overflow* to ``0`` and return ``-1`` as usual. Returns ``-1`` on error. Use :c:func:`PyErr_Occurred` to disambiguate. .. versionchanged:: 3.8 Use :meth:`~object.__index__` if available. .. versionchanged:: 3.10 This function will no longer use :meth:`~object.__int__`. .. c:function:: long long PyLong_AsLongLong(PyObject *obj) .. index:: single: OverflowError (built-in exception) Return a C :c:expr:`long long` representation of *obj*. If *obj* is not an instance of :c:type:`PyLongObject`, first call its :meth:`~object.__index__` method (if present) to convert it to a :c:type:`PyLongObject`. Raise :exc:`OverflowError` if the value of *obj* is out of range for a :c:expr:`long long`. Returns ``-1`` on error. Use :c:func:`PyErr_Occurred` to disambiguate. .. versionchanged:: 3.8 Use :meth:`~object.__index__` if available. .. versionchanged:: 3.10 This function will no longer use :meth:`~object.__int__`. .. c:function:: long long PyLong_AsLongLongAndOverflow(PyObject *obj, int *overflow) Return a C :c:expr:`long long` representation of *obj*. If *obj* is not an instance of :c:type:`PyLongObject`, first call its :meth:`~object.__index__` method (if present) to convert it to a :c:type:`PyLongObject`. If the value of *obj* is greater than :c:macro:`LLONG_MAX` or less than :c:macro:`LLONG_MIN`, set *\*overflow* to ``1`` or ``-1``, respectively, and return ``-1``; otherwise, set *\*overflow* to ``0``. If any other exception occurs set *\*overflow* to ``0`` and return ``-1`` as usual. Returns ``-1`` on error. Use :c:func:`PyErr_Occurred` to disambiguate. .. versionadded:: 3.2 .. versionchanged:: 3.8 Use :meth:`~object.__index__` if available. .. versionchanged:: 3.10 This function will no longer use :meth:`~object.__int__`. .. c:function:: Py_ssize_t PyLong_AsSsize_t(PyObject *pylong) .. index:: single: PY_SSIZE_T_MAX (C macro) single: OverflowError (built-in exception) Return a C :c:type:`Py_ssize_t` representation of *pylong*. *pylong* must be an instance of :c:type:`PyLongObject`. Raise :exc:`OverflowError` if the value of *pylong* is out of range for a :c:type:`Py_ssize_t`. Returns ``-1`` on error. Use :c:func:`PyErr_Occurred` to disambiguate. .. c:function:: unsigned long PyLong_AsUnsignedLong(PyObject *pylong) .. index:: single: ULONG_MAX (C macro) single: OverflowError (built-in exception) Return a C :c:expr:`unsigned long` representation of *pylong*. *pylong* must be an instance of :c:type:`PyLongObject`. Raise :exc:`OverflowError` if the value of *pylong* is out of range for a :c:expr:`unsigned long`. Returns ``(unsigned long)-1`` on error. Use :c:func:`PyErr_Occurred` to disambiguate. .. c:function:: size_t PyLong_AsSize_t(PyObject *pylong) .. index:: single: SIZE_MAX (C macro) single: OverflowError (built-in exception) Return a C :c:type:`size_t` representation of *pylong*. *pylong* must be an instance of :c:type:`PyLongObject`. Raise :exc:`OverflowError` if the value of *pylong* is out of range for a :c:type:`size_t`. Returns ``(size_t)-1`` on error. Use :c:func:`PyErr_Occurred` to disambiguate. .. c:function:: unsigned long long PyLong_AsUnsignedLongLong(PyObject *pylong) .. index:: single: OverflowError (built-in exception) Return a C :c:expr:`unsigned long long` representation of *pylong*. *pylong* must be an instance of :c:type:`PyLongObject`. Raise :exc:`OverflowError` if the value of *pylong* is out of range for an :c:expr:`unsigned long long`. Returns ``(unsigned long long)-1`` on error. Use :c:func:`PyErr_Occurred` to disambiguate. .. versionchanged:: 3.1 A negative *pylong* now raises :exc:`OverflowError`, not :exc:`TypeError`. .. c:function:: unsigned long PyLong_AsUnsignedLongMask(PyObject *obj) Return a C :c:expr:`unsigned long` representation of *obj*. If *obj* is not an instance of :c:type:`PyLongObject`, first call its :meth:`~object.__index__` method (if present) to convert it to a :c:type:`PyLongObject`. If the value of *obj* is out of range for an :c:expr:`unsigned long`, return the reduction of that value modulo ``ULONG_MAX + 1``. Returns ``(unsigned long)-1`` on error. Use :c:func:`PyErr_Occurred` to disambiguate. .. versionchanged:: 3.8 Use :meth:`~object.__index__` if available. .. versionchanged:: 3.10 This function will no longer use :meth:`~object.__int__`. .. c:function:: unsigned long long PyLong_AsUnsignedLongLongMask(PyObject *obj) Return a C :c:expr:`unsigned long long` representation of *obj*. If *obj* is not an instance of :c:type:`PyLongObject`, first call its :meth:`~object.__index__` method (if present) to convert it to a :c:type:`PyLongObject`. If the value of *obj* is out of range for an :c:expr:`unsigned long long`, return the reduction of that value modulo ``ULLONG_MAX + 1``. Returns ``(unsigned long long)-1`` on error. Use :c:func:`PyErr_Occurred` to disambiguate. .. versionchanged:: 3.8 Use :meth:`~object.__index__` if available. .. versionchanged:: 3.10 This function will no longer use :meth:`~object.__int__`. .. c:function:: int PyLong_AsInt32(PyObject *obj, int32_t *value) int PyLong_AsInt64(PyObject *obj, int64_t *value) Set *\*value* to a signed C :c:expr:`int32_t` or :c:expr:`int64_t` representation of *obj*. If *obj* is not an instance of :c:type:`PyLongObject`, first call its :meth:`~object.__index__` method (if present) to convert it to a :c:type:`PyLongObject`. If the *obj* value is out of range, raise an :exc:`OverflowError`. Set *\*value* and return ``0`` on success. Set an exception and return ``-1`` on error. *value* must not be ``NULL``. .. versionadded:: 3.14 .. c:function:: int PyLong_AsUInt32(PyObject *obj, uint32_t *value) int PyLong_AsUInt64(PyObject *obj, uint64_t *value) Set *\*value* to an unsigned C :c:expr:`uint32_t` or :c:expr:`uint64_t` representation of *obj*. If *obj* is not an instance of :c:type:`PyLongObject`, first call its :meth:`~object.__index__` method (if present) to convert it to a :c:type:`PyLongObject`. * If *obj* is negative, raise a :exc:`ValueError`. * If the *obj* value is out of range, raise an :exc:`OverflowError`. Set *\*value* and return ``0`` on success. Set an exception and return ``-1`` on error. *value* must not be ``NULL``. .. versionadded:: 3.14 .. c:function:: double PyLong_AsDouble(PyObject *pylong) Return a C :c:expr:`double` representation of *pylong*. *pylong* must be an instance of :c:type:`PyLongObject`. Raise :exc:`OverflowError` if the value of *pylong* is out of range for a :c:expr:`double`. Returns ``-1.0`` on error. Use :c:func:`PyErr_Occurred` to disambiguate. .. c:function:: void* PyLong_AsVoidPtr(PyObject *pylong) Convert a Python integer *pylong* to a C :c:expr:`void` pointer. If *pylong* cannot be converted, an :exc:`OverflowError` will be raised. This is only assured to produce a usable :c:expr:`void` pointer for values created with :c:func:`PyLong_FromVoidPtr`. Returns ``NULL`` on error. Use :c:func:`PyErr_Occurred` to disambiguate. .. c:function:: Py_ssize_t PyLong_AsNativeBytes(PyObject *pylong, void* buffer, Py_ssize_t n_bytes, int flags) Copy the Python integer value *pylong* to a native *buffer* of size *n_bytes*. The *flags* can be set to ``-1`` to behave similarly to a C cast, or to values documented below to control the behavior. Returns ``-1`` with an exception raised on error. This may happen if *pylong* cannot be interpreted as an integer, or if *pylong* was negative and the ``Py_ASNATIVEBYTES_REJECT_NEGATIVE`` flag was set. Otherwise, returns the number of bytes required to store the value. If this is equal to or less than *n_bytes*, the entire value was copied. All *n_bytes* of the buffer are written: remaining bytes filled by copies of the sign bit. If the returned value is greater than *n_bytes*, the value was truncated: as many of the lowest bits of the value as could fit are written, and the higher bits are ignored. This matches the typical behavior of a C-style downcast. .. note:: Overflow is not considered an error. If the returned value is larger than *n_bytes*, most significant bits were discarded. ``0`` will never be returned. Values are always copied as two's-complement. Usage example:: int32_t value; Py_ssize_t bytes = PyLong_AsNativeBytes(pylong, &value, sizeof(value), -1); if (bytes < 0) { // Failed. A Python exception was set with the reason. return NULL; } else if (bytes <= (Py_ssize_t)sizeof(value)) { // Success! } else { // Overflow occurred, but 'value' contains the truncated // lowest bits of pylong. } Passing zero to *n_bytes* will return the size of a buffer that would be large enough to hold the value. This may be larger than technically necessary, but not unreasonably so. If *n_bytes=0*, *buffer* may be ``NULL``. .. note:: Passing *n_bytes=0* to this function is not an accurate way to determine the bit length of the value. To get at the entire Python value of an unknown size, the function can be called twice: first to determine the buffer size, then to fill it:: // Ask how much space we need. Py_ssize_t expected = PyLong_AsNativeBytes(pylong, NULL, 0, -1); if (expected < 0) { // Failed. A Python exception was set with the reason. return NULL; } assert(expected != 0); // Impossible per the API definition. uint8_t *bignum = malloc(expected); if (!bignum) { PyErr_SetString(PyExc_MemoryError, "bignum malloc failed."); return NULL; } // Safely get the entire value. Py_ssize_t bytes = PyLong_AsNativeBytes(pylong, bignum, expected, -1); if (bytes < 0) { // Exception has been set. free(bignum); return NULL; } else if (bytes > expected) { // This should not be possible. PyErr_SetString(PyExc_RuntimeError, "Unexpected bignum truncation after a size check."); free(bignum); return NULL; } // The expected success given the above pre-check. // ... use bignum ... free(bignum); *flags* is either ``-1`` (``Py_ASNATIVEBYTES_DEFAULTS``) to select defaults that behave most like a C cast, or a combination of the other flags in the table below. Note that ``-1`` cannot be combined with other flags. Currently, ``-1`` corresponds to ``Py_ASNATIVEBYTES_NATIVE_ENDIAN | Py_ASNATIVEBYTES_UNSIGNED_BUFFER``. .. c:namespace:: NULL ============================================= ====== Flag Value ============================================= ====== .. c:macro:: Py_ASNATIVEBYTES_DEFAULTS ``-1`` .. c:macro:: Py_ASNATIVEBYTES_BIG_ENDIAN ``0`` .. c:macro:: Py_ASNATIVEBYTES_LITTLE_ENDIAN ``1`` .. c:macro:: Py_ASNATIVEBYTES_NATIVE_ENDIAN ``3`` .. c:macro:: Py_ASNATIVEBYTES_UNSIGNED_BUFFER ``4`` .. c:macro:: Py_ASNATIVEBYTES_REJECT_NEGATIVE ``8`` .. c:macro:: Py_ASNATIVEBYTES_ALLOW_INDEX ``16`` ============================================= ====== Specifying ``Py_ASNATIVEBYTES_NATIVE_ENDIAN`` will override any other endian flags. Passing ``2`` is reserved. By default, sufficient buffer will be requested to include a sign bit. For example, when converting 128 with *n_bytes=1*, the function will return 2 (or more) in order to store a zero sign bit. If ``Py_ASNATIVEBYTES_UNSIGNED_BUFFER`` is specified, a zero sign bit will be omitted from size calculations. This allows, for example, 128 to fit in a single-byte buffer. If the destination buffer is later treated as signed, a positive input value may become negative. Note that the flag does not affect handling of negative values: for those, space for a sign bit is always requested. Specifying ``Py_ASNATIVEBYTES_REJECT_NEGATIVE`` causes an exception to be set if *pylong* is negative. Without this flag, negative values will be copied provided there is enough space for at least one sign bit, regardless of whether ``Py_ASNATIVEBYTES_UNSIGNED_BUFFER`` was specified. If ``Py_ASNATIVEBYTES_ALLOW_INDEX`` is specified and a non-integer value is passed, its :meth:`~object.__index__` method will be called first. This may result in Python code executing and other threads being allowed to run, which could cause changes to other objects or values in use. When *flags* is ``-1``, this option is not set, and non-integer values will raise :exc:`TypeError`. .. note:: With the default *flags* (``-1``, or *UNSIGNED_BUFFER* without *REJECT_NEGATIVE*), multiple Python integers can map to a single value without overflow. For example, both ``255`` and ``-1`` fit a single-byte buffer and set all its bits. This matches typical C cast behavior. .. versionadded:: 3.13 .. c:macro:: PyLong_AsPid(pid) Macro for converting a Python integer into a process identifier. This can be defined as an alias to :c:func:`PyLong_AsLong`, :c:func:`PyLong_FromLongLong`, or :c:func:`PyLong_AsInt`, depending on the size of the system's PID type. .. versionadded:: 3.2 .. c:function:: int PyLong_GetSign(PyObject *obj, int *sign) Get the sign of the integer object *obj*. On success, set *\*sign* to the integer sign (0, -1 or +1 for zero, negative or positive integer, respectively) and return 0. On failure, return -1 with an exception set. This function always succeeds if *obj* is a :c:type:`PyLongObject` or its subtype. .. versionadded:: 3.14 .. c:function:: int PyLong_IsPositive(PyObject *obj) Check if the integer object *obj* is positive (``obj > 0``). If *obj* is an instance of :c:type:`PyLongObject` or its subtype, return ``1`` when it's positive and ``0`` otherwise. Else set an exception and return ``-1``. .. versionadded:: 3.14 .. c:function:: int PyLong_IsNegative(PyObject *obj) Check if the integer object *obj* is negative (``obj < 0``). If *obj* is an instance of :c:type:`PyLongObject` or its subtype, return ``1`` when it's negative and ``0`` otherwise. Else set an exception and return ``-1``. .. versionadded:: 3.14 .. c:function:: int PyLong_IsZero(PyObject *obj) Check if the integer object *obj* is zero. If *obj* is an instance of :c:type:`PyLongObject` or its subtype, return ``1`` when it's zero and ``0`` otherwise. Else set an exception and return ``-1``. .. versionadded:: 3.14 .. c:function:: PyObject* PyLong_GetInfo(void) On success, return a read only :term:`named tuple`, that holds information about Python's internal representation of integers. See :data:`sys.int_info` for description of individual fields. On failure, return ``NULL`` with an exception set. .. versionadded:: 3.1 .. c:function:: int PyUnstable_Long_IsCompact(const PyLongObject* op) Return 1 if *op* is compact, 0 otherwise. This function makes it possible for performance-critical code to implement a “fast path” for small integers. For compact values use :c:func:`PyUnstable_Long_CompactValue`; for others fall back to a :c:func:`PyLong_As* ` function or :c:func:`PyLong_AsNativeBytes`. The speedup is expected to be negligible for most users. Exactly what values are considered compact is an implementation detail and is subject to change. .. versionadded:: 3.12 .. c:function:: Py_ssize_t PyUnstable_Long_CompactValue(const PyLongObject* op) If *op* is compact, as determined by :c:func:`PyUnstable_Long_IsCompact`, return its value. Otherwise, the return value is undefined. .. versionadded:: 3.12 Export API ^^^^^^^^^^ .. versionadded:: 3.14 .. c:type:: PyLongLayout Layout of an array of "digits" ("limbs" in the GMP terminology), used to represent absolute value for arbitrary precision integers. Use :c:func:`PyLong_GetNativeLayout` to get the native layout of Python :class:`int` objects, used internally for integers with "big enough" absolute value. See also :data:`sys.int_info` which exposes similar information in Python. .. c:member:: uint8_t bits_per_digit Bits per digit. For example, a 15 bit digit means that bits 0-14 contain meaningful information. .. c:member:: uint8_t digit_size Digit size in bytes. For example, a 15 bit digit will require at least 2 bytes. .. c:member:: int8_t digits_order Digits order: - ``1`` for most significant digit first - ``-1`` for least significant digit first .. c:member:: int8_t digit_endianness Digit endianness: - ``1`` for most significant byte first (big endian) - ``-1`` for least significant byte first (little endian) .. c:function:: const PyLongLayout* PyLong_GetNativeLayout(void) Get the native layout of Python :class:`int` objects. See the :c:type:`PyLongLayout` structure. The function must not be called before Python initialization nor after Python finalization. The returned layout is valid until Python is finalized. The layout is the same for all Python sub-interpreters in a process, and so it can be cached. .. c:type:: PyLongExport Export of a Python :class:`int` object. There are two cases: * If :c:member:`digits` is ``NULL``, only use the :c:member:`value` member. * If :c:member:`digits` is not ``NULL``, use :c:member:`negative`, :c:member:`ndigits` and :c:member:`digits` members. .. c:member:: int64_t value The native integer value of the exported :class:`int` object. Only valid if :c:member:`digits` is ``NULL``. .. c:member:: uint8_t negative ``1`` if the number is negative, ``0`` otherwise. Only valid if :c:member:`digits` is not ``NULL``. .. c:member:: Py_ssize_t ndigits Number of digits in :c:member:`digits` array. Only valid if :c:member:`digits` is not ``NULL``. .. c:member:: const void *digits Read-only array of unsigned digits. Can be ``NULL``. .. c:function:: int PyLong_Export(PyObject *obj, PyLongExport *export_long) Export a Python :class:`int` object. *export_long* must point to a :c:type:`PyLongExport` structure allocated by the caller. It must not be ``NULL``. On success, fill in *\*export_long* and return ``0``. On error, set an exception and return ``-1``. :c:func:`PyLong_FreeExport` must be called when the export is no longer needed. .. impl-detail:: This function always succeeds if *obj* is a Python :class:`int` object or a subclass. .. c:function:: void PyLong_FreeExport(PyLongExport *export_long) Release the export *export_long* created by :c:func:`PyLong_Export`. .. impl-detail:: Calling :c:func:`PyLong_FreeExport` is optional if *export_long->digits* is ``NULL``. PyLongWriter API ^^^^^^^^^^^^^^^^ The :c:type:`PyLongWriter` API can be used to import an integer. .. versionadded:: 3.14 .. c:type:: PyLongWriter A Python :class:`int` writer instance. The instance must be destroyed by :c:func:`PyLongWriter_Finish` or :c:func:`PyLongWriter_Discard`. .. c:function:: PyLongWriter* PyLongWriter_Create(int negative, Py_ssize_t ndigits, void **digits) Create a :c:type:`PyLongWriter`. On success, allocate *\*digits* and return a writer. On error, set an exception and return ``NULL``. *negative* is ``1`` if the number is negative, or ``0`` otherwise. *ndigits* is the number of digits in the *digits* array. It must be greater than 0. *digits* must not be NULL. After a successful call to this function, the caller should fill in the array of digits *digits* and then call :c:func:`PyLongWriter_Finish` to get a Python :class:`int`. The layout of *digits* is described by :c:func:`PyLong_GetNativeLayout`. Digits must be in the range [``0``; ``(1 << bits_per_digit) - 1``] (where the :c:type:`~PyLongLayout.bits_per_digit` is the number of bits per digit). Any unused most significant digits must be set to ``0``. Alternately, call :c:func:`PyLongWriter_Discard` to destroy the writer instance without creating an :class:`~int` object. .. c:function:: PyObject* PyLongWriter_Finish(PyLongWriter *writer) Finish a :c:type:`PyLongWriter` created by :c:func:`PyLongWriter_Create`. On success, return a Python :class:`int` object. On error, set an exception and return ``NULL``. The function takes care of normalizing the digits and converts the object to a compact integer if needed. The writer instance and the *digits* array are invalid after the call. .. c:function:: void PyLongWriter_Discard(PyLongWriter *writer) Discard a :c:type:`PyLongWriter` created by :c:func:`PyLongWriter_Create`. If *writer* is ``NULL``, no operation is performed. The writer instance and the *digits* array are invalid after the call. Deprecated API ^^^^^^^^^^^^^^ These macros are :term:`soft deprecated`. They describe parameters of the internal representation of :c:type:`PyLongObject` instances. Use :c:func:`PyLong_GetNativeLayout` instead, along with :c:func:`PyLong_Export` to read integer data or :c:type:`PyLongWriter` to write it. These currently use the same layout, but are designed to continue working correctly even if CPython's internal integer representation changes. .. c:macro:: PyLong_SHIFT This is equivalent to :c:member:`~PyLongLayout.bits_per_digit` in the output of :c:func:`PyLong_GetNativeLayout`. .. c:macro:: PyLong_BASE This is currently equivalent to :c:expr:`1 << PyLong_SHIFT`. .. c:macro:: PyLong_MASK This is currently equivalent to :c:expr:`(1 << PyLong_SHIFT) - 1` --- ### Doc C Api Mapping (`Doc/c-api/mapping.rst`) .. highlight:: c .. _mapping: Mapping Protocol ================ See also :c:func:`PyObject_GetItem`, :c:func:`PyObject_SetItem` and :c:func:`PyObject_DelItem`. .. c:function:: int PyMapping_Check(PyObject *o) Return ``1`` if the object provides the mapping protocol or supports slicing, and ``0`` otherwise. Note that it returns ``1`` for Python classes with a :meth:`~object.__getitem__` method, since in general it is impossible to determine what type of keys the class supports. This function always succeeds. .. c:function:: Py_ssize_t PyMapping_Size(PyObject *o) Py_ssize_t PyMapping_Length(PyObject *o) .. index:: pair: built-in function; len Returns the number of keys in object *o* on success, and ``-1`` on failure. This is equivalent to the Python expression ``len(o)``. .. c:function:: PyObject* PyMapping_GetItemString(PyObject *o, const char *key) This is the same as :c:func:`PyObject_GetItem`, but *key* is specified as a :c:expr:`const char*` UTF-8 encoded bytes string, rather than a :c:expr:`PyObject*`. .. c:function:: int PyMapping_GetOptionalItem(PyObject *obj, PyObject *key, PyObject **result) Variant of :c:func:`PyObject_GetItem` which doesn't raise :exc:`KeyError` if the key is not found. If the key is found, return ``1`` and set *\*result* to a new :term:`strong reference` to the corresponding value. If the key is not found, return ``0`` and set *\*result* to ``NULL``; the :exc:`KeyError` is silenced. If an error other than :exc:`KeyError` is raised, return ``-1`` and set *\*result* to ``NULL``. .. versionadded:: 3.13 .. c:function:: int PyMapping_GetOptionalItemString(PyObject *obj, const char *key, PyObject **result) This is the same as :c:func:`PyMapping_GetOptionalItem`, but *key* is specified as a :c:expr:`const char*` UTF-8 encoded bytes string, rather than a :c:expr:`PyObject*`. .. versionadded:: 3.13 .. c:function:: int PyMapping_SetItemString(PyObject *o, const char *key, PyObject *v) This is the same as :c:func:`PyObject_SetItem`, but *key* is specified as a :c:expr:`const char*` UTF-8 encoded bytes string, rather than a :c:expr:`PyObject*`. .. c:function:: int PyMapping_DelItem(PyObject *o, PyObject *key) This is an alias of :c:func:`PyObject_DelItem`. .. c:function:: int PyMapping_DelItemString(PyObject *o, const char *key) This is the same as :c:func:`PyObject_DelItem`, but *key* is specified as a :c:expr:`const char*` UTF-8 encoded bytes string, rather than a :c:expr:`PyObject*`. .. c:function:: int PyMapping_HasKeyWithError(PyObject *o, PyObject *key) Return ``1`` if the mapping object has the key *key* and ``0`` otherwise. This is equivalent to the Python expression ``key in o``. On failure, return ``-1``. .. versionadded:: 3.13 .. c:function:: int PyMapping_HasKeyStringWithError(PyObject *o, const char *key) This is the same as :c:func:`PyMapping_HasKeyWithError`, but *key* is specified as a :c:expr:`const char*` UTF-8 encoded bytes string, rather than a :c:expr:`PyObject*`. .. versionadded:: 3.13 .. c:function:: int PyMapping_HasKey(PyObject *o, PyObject *key) Return ``1`` if the mapping object has the key *key* and ``0`` otherwise. This is equivalent to the Python expression ``key in o``. This function always succeeds. .. note:: Exceptions which occur when this calls the :meth:`~object.__getitem__` method are silently ignored. For proper error handling, use :c:func:`PyMapping_HasKeyWithError`, :c:func:`PyMapping_GetOptionalItem` or :c:func:`PyObject_GetItem()` instead. .. c:function:: int PyMapping_HasKeyString(PyObject *o, const char *key) This is the same as :c:func:`PyMapping_HasKey`, but *key* is specified as a :c:expr:`const char*` UTF-8 encoded bytes string, rather than a :c:expr:`PyObject*`. .. note:: Exceptions that occur when this calls the :meth:`~object.__getitem__` method or while creating the temporary :class:`str` object are silently ignored. For proper error handling, use :c:func:`PyMapping_HasKeyStringWithError`, :c:func:`PyMapping_GetOptionalItemString` or :c:func:`PyMapping_GetItemString` instead. .. c:function:: PyObject* PyMapping_Keys(PyObject *o) On success, return a list of the keys in object *o*. On failure, return ``NULL``. .. versionchanged:: 3.7 Previously, the function returned a list or a tuple. .. c:function:: PyObject* PyMapping_Values(PyObject *o) On success, return a list of the values in object *o*. On failure, return ``NULL``. .. versionchanged:: 3.7 Previously, the function returned a list or a tuple. .. c:function:: PyObject* PyMapping_Items(PyObject *o) On success, return a list of the items in object *o*, where each item is a tuple containing a key-value pair. On failure, return ``NULL``. .. versionchanged:: 3.7 Previously, the function returned a list or a tuple. --- ### Doc C Api Marshal (`Doc/c-api/marshal.rst`) .. highlight:: c .. _marshalling-utils: Data marshalling support ======================== These routines allow C code to work with serialized objects using the same data format as the :mod:`marshal` module. There are functions to write data into the serialization format, and additional functions that can be used to read the data back. Files used to store marshalled data must be opened in binary mode. Numeric values are stored with the least significant byte first. The module supports several versions of the data format; see the :py:mod:`Python module documentation ` for details. The following exceptions can be raised by these functions: :exc:`ValueError` if the value cannot be marshalled, :exc:`ValueError` or :exc:`TypeError` if the data is malformed, :exc:`EOFError` if the end of the data is reached before the value is complete, :exc:`OSError` if reading from or writing to a :c:expr:`FILE*` fails, :exc:`KeyboardInterrupt` if reading or writing is interrupted by a signal, and :exc:`MemoryError` if memory allocation fails. .. versionchanged:: 3.16 Previously, in functions taking a :c:expr:`FILE*`, the reading functions raised :exc:`EOFError` instead of :exc:`OSError` and :exc:`KeyboardInterrupt`, and the writing functions ignored I/O errors and interruptions. .. c:macro:: Py_MARSHAL_VERSION The current format version. See :py:data:`marshal.version`. .. c:function:: void PyMarshal_WriteLongToFile(long value, FILE *file, int version) Marshal a :c:expr:`long` integer, *value*, to *file*. This will only write the least-significant 32 bits of *value*; regardless of the size of the native :c:expr:`long` type. *version* indicates the file format. This function can fail, in which case it sets the error indicator. Use :c:func:`PyErr_Occurred` to check for that. .. note:: In Python 3.15 and earlier, this function does not set the error indicator if writing to *file* fails. The same caveat applies as for :c:func:`PyMarshal_WriteObjectToFile`. .. c:function:: void PyMarshal_WriteObjectToFile(PyObject *value, FILE *file, int version) Marshal a Python object, *value*, to *file*. *version* indicates the file format. The marshalled representation of *value* is built in memory and then written to *file*. Nothing is written if *value* cannot be marshalled. This function can fail, in which case it sets the error indicator. Use :c:func:`PyErr_Occurred` to check for that. .. note:: In Python 3.15 and earlier, this function can write only part of the data to *file* and return without setting the error indicator, for example if *value* contains an object which cannot be marshalled or if writing to *file* fails (see :gh:`155907`). Code which has to support these versions cannot rely on :c:func:`PyErr_Occurred` alone to know that all the data was written. It can use :c:func:`PyMarshal_WriteObjectToString` and write the result itself, or validate what was written. .. versionchanged:: 3.16 Previously, the data was written to *file* as it was produced, using a buffer of constant size, and incomplete data was written if *value* could not be marshalled. .. c:function:: PyObject* PyMarshal_WriteObjectToString(PyObject *value, int version) Return a bytes object containing the marshalled representation of *value*. *version* indicates the file format. On error, raises an exception and returns ``NULL``. The following functions allow marshalled values to be read back in. .. c:function:: long PyMarshal_ReadLongFromFile(FILE *file) Return a C :c:expr:`long` from the data stream in a :c:expr:`FILE*` opened for reading. Only a 32-bit value can be read in using this function, regardless of the native size of :c:expr:`long`. On error, raises an exception and returns ``-1``. .. c:function:: int PyMarshal_ReadShortFromFile(FILE *file) Return a C :c:expr:`short` from the data stream in a :c:expr:`FILE*` opened for reading. Only a 16-bit value can be read in using this function, regardless of the native size of :c:expr:`short`. On error, raises an exception and returns ``-1``. .. c:function:: PyObject* PyMarshal_ReadObjectFromFile(FILE *file) Return a Python object from the data stream in a :c:expr:`FILE*` opened for reading. On error, raises an exception and returns ``NULL``. .. c:function:: PyObject* PyMarshal_ReadLastObjectFromFile(FILE *file) Return a Python object from the data stream in a :c:expr:`FILE*` opened for reading. Unlike :c:func:`PyMarshal_ReadObjectFromFile`, this function assumes that no further objects will be read from the file, allowing it to aggressively load file data into memory so that the de-serialization can operate from data in memory rather than reading a byte at a time from the file. Only use this variant if you are certain that you won't be reading anything else from the file. On error, raises an exception and returns ``NULL``. .. c:function:: PyObject* PyMarshal_ReadObjectFromString(const char *data, Py_ssize_t len) Return a Python object from the data stream in a byte buffer containing *len* bytes pointed to by *data*. On error, raises an exception and returns ``NULL``. --- ### Doc C Api Memory (`Doc/c-api/memory.rst`) .. highlight:: c .. _memory: ***************** Memory Management ***************** .. _memoryoverview: Overview ======== Memory management in Python involves a private heap containing all Python objects and data structures. The management of this private heap is ensured internally by the *Python memory manager*. The Python memory manager has different components which deal with various dynamic storage management aspects, like sharing, segmentation, preallocation or caching. At the lowest level, a raw memory allocator ensures that there is enough room in the private heap for storing all Python-related data by interacting with the memory manager of the operating system. On top of the raw memory allocator, several object-specific allocators operate on the same heap and implement distinct memory management policies adapted to the peculiarities of every object type. For example, integer objects are managed differently within the heap than strings, tuples or dictionaries because integers imply different storage requirements and speed/space tradeoffs. The Python memory manager thus delegates some of the work to the object-specific allocators, but ensures that the latter operate within the bounds of the private heap. It is important to understand that the management of the Python heap is performed by the interpreter itself and that the user has no control over it, even if they regularly manipulate object pointers to memory blocks inside that heap. The allocation of heap space for Python objects and other internal buffers is performed on demand by the Python memory manager through the Python/C API functions listed in this document. .. index:: single: malloc (C function) single: calloc (C function) single: realloc (C function) single: free (C function) To avoid memory corruption, extension writers should never try to operate on Python objects with the functions exported by the C library: :c:func:`malloc`, :c:func:`calloc`, :c:func:`realloc` and :c:func:`free`. This will result in mixed calls between the C allocator and the Python memory manager with fatal consequences, because they implement different algorithms and operate on different heaps. However, one may safely allocate and release memory blocks with the C library allocator for individual purposes, as shown in the following example:: PyObject *res; char *buf = (char *) malloc(BUFSIZ); /* for I/O */ if (buf == NULL) return PyErr_NoMemory(); ...Do some I/O operation involving buf... res = PyBytes_FromString(buf); free(buf); /* malloc'ed */ return res; In this example, the memory request for the I/O buffer is handled by the C library allocator. The Python memory manager is involved only in the allocation of the bytes object returned as a result. In most situations, however, it is recommended to allocate memory from the Python heap specifically because the latter is under control of the Python memory manager. For example, this is required when the interpreter is extended with new object types written in C. Another reason for using the Python heap is the desire to *inform* the Python memory manager about the memory needs of the extension module. Even when the requested memory is used exclusively for internal, highly specific purposes, delegating all memory requests to the Python memory manager causes the interpreter to have a more accurate image of its memory footprint as a whole. Consequently, under certain circumstances, the Python memory manager may or may not trigger appropriate actions, like garbage collection, memory compaction or other preventive procedures. Note that by using the C library allocator as shown in the previous example, the allocated memory for the I/O buffer completely escapes the Python memory manager. .. seealso:: The :envvar:`PYTHONMALLOC` environment variable can be used to configure the memory allocators used by Python. The :envvar:`PYTHONMALLOCSTATS` environment variable can be used to print statistics of the :ref:`pymalloc memory allocator ` every time a new pymalloc object arena is created, and on shutdown. Allocator Domains ================= .. _allocator-domains: All allocating functions belong to one of three different "domains" (see also :c:type:`PyMemAllocatorDomain`). These domains represent different allocation strategies and are optimized for different purposes. The specific details on how every domain allocates memory or what internal functions each domain calls is considered an implementation detail, but for debugging purposes a simplified table can be found at :ref:`default-memory-allocators`. The APIs used to allocate and free a block of memory must be from the same domain. For example, :c:func:`PyMem_Free` must be used to free memory allocated using :c:func:`PyMem_Malloc`. The three allocation domains are: * Raw domain: intended for allocating memory for general-purpose memory buffers where the allocation *must* go to the system allocator or where the allocator can operate without an :term:`attached thread state`. The memory is requested directly from the system. See :ref:`Raw Memory Interface `. * "Mem" domain: intended for allocating memory for Python buffers and general-purpose memory buffers where the allocation must be performed with an :term:`attached thread state`. The memory is taken from the Python private heap. See :ref:`Memory Interface `. * Object domain: intended for allocating memory for Python objects. The memory is taken from the Python private heap. See :ref:`Object allocators `. .. note:: The :term:`free-threaded ` build requires that only Python objects are allocated using the "object" domain and that all Python objects are allocated using that domain. This differs from the prior Python versions, where this was only a best practice and not a hard requirement. For example, buffers (non-Python objects) should be allocated using :c:func:`PyMem_Malloc`, :c:func:`PyMem_RawMalloc`, or :c:func:`malloc`, but not :c:func:`PyObject_Malloc`. See :ref:`Memory Allocation APIs `. .. _raw-memoryinterface: Raw Memory Interface ==================== The following function sets are wrappers to the system allocator. These functions are thread-safe, so a :term:`thread state` does not need to be :term:`attached `. The :ref:`default raw memory allocator ` uses the following functions: :c:func:`malloc`, :c:func:`calloc`, :c:func:`realloc` and :c:func:`!free`; call ``malloc(1)`` (or ``calloc(1, 1)``) when requesting zero bytes. .. versionadded:: 3.4 .. c:function:: void* PyMem_RawMalloc(size_t n) Allocates *n* bytes and returns a pointer of type :c:expr:`void*` to the allocated memory, or ``NULL`` if the request fails. Requesting zero bytes returns a distinct non-``NULL`` pointer if possible, as if ``PyMem_RawMalloc(1)`` had been called instead. The memory will not have been initialized in any way. .. c:function:: void* PyMem_RawCalloc(size_t nelem, size_t elsize) Allocates *nelem* elements each of size *elsize* bytes and returns a pointer of type :c:expr:`void*` to the allocated memory, or ``NULL`` if the request fails. The memory is initialized to zeros. Requesting zero elements or elements of size zero bytes returns a distinct non-``NULL`` pointer if possible, as if ``PyMem_RawCalloc(1, 1)`` had been called instead. .. versionadded:: 3.5 .. c:function:: void* PyMem_RawRealloc(void *p, size_t n) Resizes the memory block pointed to by *p* to *n* bytes. The contents will be unchanged to the minimum of the old and the new sizes. If *p* is ``NULL``, the call is equivalent to ``PyMem_RawMalloc(n)``; else if *n* is equal to zero, the memory block is resized but is not freed, and the returned pointer is non-``NULL``. Unless *p* is ``NULL``, it must have been returned by a previous call to :c:func:`PyMem_RawMalloc`, :c:func:`PyMem_RawRealloc` or :c:func:`PyMem_RawCalloc`. If the request fails, :c:func:`PyMem_RawRealloc` returns ``NULL`` and *p* remains a valid pointer to the previous memory area. .. c:function:: void PyMem_RawFree(void *p) Frees the memory block pointed to by *p*, which must have been returned by a previous call to :c:func:`PyMem_RawMalloc`, :c:func:`PyMem_RawRealloc` or :c:func:`PyMem_RawCalloc`. Otherwise, or if ``PyMem_RawFree(p)`` has been called before, undefined behavior occurs. If *p* is ``NULL``, no operation is performed. .. _memoryinterface: Memory Interface ================ The following function sets, modeled after the ANSI C standard, but specifying behavior when requesting zero bytes, are available for allocating and releasing memory from the Python heap. In the GIL-enabled build (default build) the :ref:`default memory allocator ` uses the :ref:`pymalloc memory allocator `, whereas in the :term:`free-threaded build`, the default is the :ref:`mimalloc memory allocator ` instead. .. warning:: There must be an :term:`attached thread state` when using these functions. .. versionchanged:: 3.6 The default allocator is now pymalloc instead of system :c:func:`malloc`. .. versionchanged:: 3.13 In the :term:`free-threaded ` build, the default allocator is now :ref:`mimalloc `. .. c:function:: void* PyMem_Malloc(size_t n) Allocates *n* bytes and returns a pointer of type :c:expr:`void*` to the allocated memory, or ``NULL`` if the request fails. Requesting zero bytes returns a distinct non-``NULL`` pointer if possible, as if ``PyMem_Malloc(1)`` had been called instead. The memory will not have been initialized in any way. .. c:function:: void* PyMem_Calloc(size_t nelem, size_t elsize) Allocates *nelem* elements each of size *elsize* bytes and returns a pointer of type :c:expr:`void*` to the allocated memory, or ``NULL`` if the request fails. The memory is initialized to zeros. Requesting zero elements or elements of size zero bytes returns a distinct non-``NULL`` pointer if possible, as if ``PyMem_Calloc(1, 1)`` had been called instead. .. versionadded:: 3.5 .. c:function:: void* PyMem_Realloc(void *p, size_t n) Resizes the memory block pointed to by *p* to *n* bytes. The contents will be unchanged to the minimum of the old and the new sizes. If *p* is ``NULL``, the call is equivalent to ``PyMem_Malloc(n)``; else if *n* is equal to zero, the memory block is resized but is not freed, and the returned pointer is non-``NULL``. Unless *p* is ``NULL``, it must have been returned by a previous call to :c:func:`PyMem_Malloc`, :c:func:`PyMem_Realloc` or :c:func:`PyMem_Calloc`. If the request fails, :c:func:`PyMem_Realloc` returns ``NULL`` and *p* remains a valid pointer to the previous memory area. .. c:function:: void PyMem_Free(void *p) Frees the memory block pointed to by *p*, which must have been returned by a previous call to :c:func:`PyMem_Malloc`, :c:func:`PyMem_Realloc` or :c:func:`PyMem_Calloc`. Otherwise, or if ``PyMem_Free(p)`` has been called before, undefined behavior occurs. If *p* is ``NULL``, no operation is performed. The following type-oriented macros are provided for convenience. Note that *TYPE* refers to any C type. .. c:macro:: PyMem_New(TYPE, n) Same as :c:func:`PyMem_Malloc`, but allocates ``(n * sizeof(TYPE))`` bytes of memory. Returns a pointer cast to ``TYPE*``. The memory will not have been initialized in any way. .. c:macro:: PyMem_Resize(p, TYPE, n) Same as :c:func:`PyMem_Realloc`, but the memory block is resized to ``(n * sizeof(TYPE))`` bytes. Returns a pointer cast to ``TYPE*``. On return, *p* will be a pointer to the new memory area, or ``NULL`` in the event of failure. This is a C preprocessor macro; *p* is always reassigned. Save the original value of *p* to avoid losing memory when handling errors. .. c:function:: void PyMem_Del(void *p) Same as :c:func:`PyMem_Free`. Deprecated aliases ------------------ These are :term:`soft deprecated` aliases to existing functions and macros. They exist solely for backwards compatibility. .. list-table:: :widths: auto :header-rows: 1 * * Deprecated alias * Corresponding function or macro * * .. c:macro:: PyMem_MALLOC(size) * :c:func:`PyMem_Malloc` * * .. c:macro:: PyMem_NEW(type, size) * :c:macro:`PyMem_New` * * .. c:macro:: PyMem_REALLOC(ptr, size) * :c:func:`PyMem_Realloc` * * .. c:macro:: PyMem_RESIZE(ptr, type, size) * :c:macro:`PyMem_Resize` * * .. c:macro:: PyMem_FREE(ptr) * :c:func:`PyMem_Free` * * .. c:macro:: PyMem_DEL(ptr) * :c:func:`PyMem_Free` .. versionchanged:: 3.4 The macros are now aliases of the corresponding functions and macros. Previously, their behavior was the same, but their use did not necessarily preserve binary compatibility across Python versions. .. deprecated:: 2.0 .. _objectinterface: Object allocators ================= The following function sets, modeled after the ANSI C standard, but specifying behavior when requesting zero bytes, are available for allocating and releasing memory from the Python heap. .. note:: There is no guarantee that the memory returned by these allocators can be successfully cast to a Python object when intercepting the allocating functions in this domain by the methods described in the :ref:`Customize Memory Allocators ` section. The :ref:`default object allocator ` uses the :ref:`pymalloc memory allocator `. In the :term:`free-threaded ` build, the default is the :ref:`mimalloc memory allocator ` instead. .. warning:: There must be an :term:`attached thread state` when using these functions. .. c:function:: void* PyObject_Malloc(size_t n) Allocates *n* bytes and returns a pointer of type :c:expr:`void*` to the allocated memory, or ``NULL`` if the request fails. Requesting zero bytes returns a distinct non-``NULL`` pointer if possible, as if ``PyObject_Malloc(1)`` had been called instead. The memory will not have been initialized in any way. .. c:function:: void* PyObject_Calloc(size_t nelem, size_t elsize) Allocates *nelem* elements each of size *elsize* bytes and returns a pointer of type :c:expr:`void*` to the allocated memory, or ``NULL`` if the request fails. The memory is initialized to zeros. Requesting zero elements or elements of size zero bytes returns a distinct non-``NULL`` pointer if possible, as if ``PyObject_Calloc(1, 1)`` had been called instead. .. versionadded:: 3.5 .. c:function:: void* PyObject_Realloc(void *p, size_t n) Resizes the memory block pointed to by *p* to *n* bytes. The contents will be unchanged to the minimum of the old and the new sizes. If *p* is ``NULL``, the call is equivalent to ``PyObject_Malloc(n)``; else if *n* is equal to zero, the memory block is resized but is not freed, and the returned pointer is non-``NULL``. Unless *p* is ``NULL``, it must have been returned by a previous call to :c:func:`PyObject_Malloc`, :c:func:`PyObject_Realloc` or :c:func:`PyObject_Calloc`. If the request fails, :c:func:`PyObject_Realloc` returns ``NULL`` and *p* remains a valid pointer to the previous memory area. .. c:function:: void PyObject_Free(void *p) Frees the memory block pointed to by *p*, which must have been returned by a previous call to :c:func:`PyObject_Malloc`, :c:func:`PyObject_Realloc` or :c:func:`PyObject_Calloc`. Otherwise, or if ``PyObject_Free(p)`` has been called before, undefined behavior occurs. If *p* is ``NULL``, no operation is performed. Do not call this directly to free an object's memory; call the type's :c:member:`~PyTypeObject.tp_free` slot instead. Do not use this for memory allocated by :c:macro:`PyObject_GC_New` or :c:macro:`PyObject_GC_NewVar`; use :c:func:`PyObject_GC_Del` instead. .. seealso:: * :c:func:`PyObject_GC_Del` is the equivalent of this function for memory allocated by types that support garbage collection. * :c:func:`PyObject_Malloc` * :c:func:`PyObject_Realloc` * :c:func:`PyObject_Calloc` * :c:macro:`PyObject_New` * :c:macro:`PyObject_NewVar` * :c:func:`PyType_GenericAlloc` * :c:member:`~PyTypeObject.tp_free` .. _default-memory-allocators: Default Memory Allocators ========================= Default memory allocators: =================================== ======================= ==================== ====================== ====================== Configuration Name PyMem_RawMalloc PyMem_Malloc PyObject_Malloc =================================== ======================= ==================== ====================== ====================== Release build ``"pymalloc"`` ``malloc`` ``pymalloc`` ``pymalloc`` Debug build ``"pymalloc_debug"`` ``malloc`` + debug ``pymalloc`` + debug ``pymalloc`` + debug Release build, without pymalloc ``"malloc"`` ``malloc`` ``malloc`` ``malloc`` Debug build, without pymalloc ``"malloc_debug"`` ``malloc`` + debug ``malloc`` + debug ``malloc`` + debug Release build, with ASan or MSan ``"malloc"`` ``malloc`` ``malloc`` ``malloc`` Debug build, with ASan or MSan ``"malloc_debug"`` ``malloc`` + debug ``malloc`` + debug ``malloc`` + debug Free-threaded build ``"mimalloc"`` ``mimalloc`` ``mimalloc`` ``mimalloc`` Free-threaded debug build ``"mimalloc_debug"`` ``mimalloc`` + debug ``mimalloc`` + debug ``mimalloc`` + debug =================================== ======================= ==================== ====================== ====================== Legend: * Name: value for :envvar:`PYTHONMALLOC` environment variable. * ``malloc``: system allocators from the standard C library, C functions: :c:func:`malloc`, :c:func:`calloc`, :c:func:`realloc` and :c:func:`free`. * ``pymalloc``: :ref:`pymalloc memory allocator `. * ``mimalloc``: :ref:`mimalloc memory allocator `. * "+ debug": with :ref:`debug hooks on the Python memory allocators `. * "Debug build": :ref:`Python build in debug mode `. * "with ASan or MSan": sanitizer build as configured using the :option:`--with-address-sanitizer`, :option:`--with-hwaddress-sanitizer`, and/or :option:`--with-memory-sanitizer` option. .. _customize-memory-allocators: Customize Memory Allocators =========================== .. versionadded:: 3.4 .. c:type:: PyMemAllocatorEx Structure used to describe a memory block allocator. The structure has the following fields: +----------------------------------------------------------+---------------------------------------+ | Field | Meaning | +==========================================================+=======================================+ | ``void *ctx`` | user context passed as first argument | +----------------------------------------------------------+---------------------------------------+ | ``void* malloc(void *ctx, size_t size)`` | allocate a memory block | +----------------------------------------------------------+---------------------------------------+ | ``void* calloc(void *ctx, size_t nelem, size_t elsize)`` | allocate a memory block initialized | | | with zeros | +----------------------------------------------------------+---------------------------------------+ | ``void* realloc(void *ctx, void *ptr, size_t new_size)`` | allocate or resize a memory block | +----------------------------------------------------------+---------------------------------------+ | ``void free(void *ctx, void *ptr)`` | free a memory block | +----------------------------------------------------------+---------------------------------------+ .. versionchanged:: 3.5 The :c:type:`!PyMemAllocator` structure was renamed to :c:type:`PyMemAllocatorEx` and a new ``calloc`` field was added. .. c:type:: PyMemAllocatorDomain Enum used to identify an allocator domain. Domains: .. c:namespace:: NULL .. c:macro:: PYMEM_DOMAIN_RAW Functions: * :c:func:`PyMem_RawMalloc` * :c:func:`PyMem_RawRealloc` * :c:func:`PyMem_RawCalloc` * :c:func:`PyMem_RawFree` .. c:macro:: PYMEM_DOMAIN_MEM Functions: * :c:func:`PyMem_Malloc`, * :c:func:`PyMem_Realloc` * :c:func:`PyMem_Calloc` * :c:func:`PyMem_Free` .. c:macro:: PYMEM_DOMAIN_OBJ Functions: * :c:func:`PyObject_Malloc` * :c:func:`PyObject_Realloc` * :c:func:`PyObject_Calloc` * :c:func:`PyObject_Free` .. c:function:: void PyMem_GetAllocator(PyMemAllocatorDomain domain, PyMemAllocatorEx *allocator) Get the memory block allocator of the specified domain. .. c:function:: void PyMem_SetAllocator(PyMemAllocatorDomain domain, PyMemAllocatorEx *allocator) Set the memory block allocator of the specified domain. The new allocator must return a distinct non-``NULL`` pointer when requesting zero bytes. For the :c:macro:`PYMEM_DOMAIN_RAW` domain, the allocator must be thread-safe: a :term:`thread state` is not :term:`attached ` when the allocator is called. For the remaining domains, the allocator must also be thread-safe: the allocator may be called in different interpreters that do not share a :term:`GIL`. If the new allocator is not a hook (does not call the previous allocator), the :c:func:`PyMem_SetupDebugHooks` function must be called to reinstall the debug hooks on top on the new allocator. See also :c:member:`PyPreConfig.allocator` and :ref:`Preinitialize Python with PyPreConfig `. .. warning:: :c:func:`PyMem_SetAllocator` does have the following contract: * It can be called after :c:func:`Py_PreInitialize` and before :c:func:`Py_InitializeFromConfig` to install a custom memory allocator. There are no restrictions over the installed allocator other than the ones imposed by the domain (for instance, the Raw Domain allows the allocator to be called without an :term:`attached thread state`). See :ref:`the section on allocator domains ` for more information. * If called after Python has finish initializing (after :c:func:`Py_InitializeFromConfig` has been called) the allocator **must** wrap the existing allocator. Substituting the current allocator for some other arbitrary one is **not supported**. .. versionchanged:: 3.12 All allocators must be thread-safe. .. c:function:: void PyMem_SetupDebugHooks(void) Setup :ref:`debug hooks in the Python memory allocators ` to detect memory errors. .. _pymem-debug-hooks: Debug hooks on the Python memory allocators =========================================== When :ref:`Python is built in debug mode `, the :c:func:`PyMem_SetupDebugHooks` function is called at the :ref:`Python preinitialization ` to setup debug hooks on Python memory allocators to detect memory errors. The :envvar:`PYTHONMALLOC` environment variable can be used to install debug hooks on a Python compiled in release mode (ex: ``PYTHONMALLOC=debug``). The :c:func:`PyMem_SetupDebugHooks` function can be used to set debug hooks after calling :c:func:`PyMem_SetAllocator`. These debug hooks fill dynamically allocated memory blocks with special, recognizable bit patterns. Newly allocated memory is filled with the byte ``0xCD`` (``PYMEM_CLEANBYTE``), freed memory is filled with the byte ``0xDD`` (``PYMEM_DEADBYTE``). Memory blocks are surrounded by "forbidden bytes" filled with the byte ``0xFD`` (``PYMEM_FORBIDDENBYTE``). Strings of these bytes are unlikely to be valid addresses, floats, or ASCII strings. Runtime checks: - Detect API violations. For example, detect if :c:func:`PyObject_Free` is called on a memory block allocated by :c:func:`PyMem_Malloc`. - Detect write before the start of the buffer (buffer underflow). - Detect write after the end of the buffer (buffer overflow). - Check that there is an :term:`attached thread state` when allocator functions of :c:macro:`PYMEM_DOMAIN_OBJ` (ex: :c:func:`PyObject_Malloc`) and :c:macro:`PYMEM_DOMAIN_MEM` (ex: :c:func:`PyMem_Malloc`) domains are called. On error, the debug hooks use the :mod:`tracemalloc` module to get the traceback where a memory block was allocated. The traceback is only displayed if :mod:`tracemalloc` is tracing Python memory allocations and the memory block was traced. Let *S* = ``sizeof(size_t)``. ``2*S`` bytes are added at each end of each block of *N* bytes requested. The memory layout is like so, where p represents the address returned by a malloc-like or realloc-like function (``p[i:j]`` means the slice of bytes from ``*(p+i)`` inclusive up to ``*(p+j)`` exclusive; note that the treatment of negative indices differs from a Python slice): ``p[-2*S:-S]`` Number of bytes originally asked for. This is a size_t, big-endian (easier to read in a memory dump). ``p[-S]`` API identifier (ASCII character): * ``'r'`` for :c:macro:`PYMEM_DOMAIN_RAW`. * ``'m'`` for :c:macro:`PYMEM_DOMAIN_MEM`. * ``'o'`` for :c:macro:`PYMEM_DOMAIN_OBJ`. ``p[-S+1:0]`` Copies of PYMEM_FORBIDDENBYTE. Used to catch under- writes and reads. ``p[0:N]`` The requested memory, filled with copies of PYMEM_CLEANBYTE, used to catch reference to uninitialized memory. When a realloc-like function is called requesting a larger memory block, the new excess bytes are also filled with PYMEM_CLEANBYTE. When a free-like function is called, these are overwritten with PYMEM_DEADBYTE, to catch reference to freed memory. When a realloc- like function is called requesting a smaller memory block, the excess old bytes are also filled with PYMEM_DEADBYTE. ``p[N:N+S]`` Copies of PYMEM_FORBIDDENBYTE. Used to catch over- writes and reads. ``p[N+S:N+2*S]`` Only used if the ``PYMEM_DEBUG_SERIALNO`` macro is defined (not defined by default). A serial number, incremented by 1 on each call to a malloc-like or realloc-like function. Big-endian :c:type:`size_t`. If "bad memory" is detected later, the serial number gives an excellent way to set a breakpoint on the next run, to capture the instant at which this block was passed out. The static function bumpserialno() in obmalloc.c is the only place the serial number is incremented, and exists so you can set such a breakpoint easily. A realloc-like or free-like function first checks that the PYMEM_FORBIDDENBYTE bytes at each end are intact. If they've been altered, diagnostic output is written to stderr, and the program is aborted via Py_FatalError(). The other main failure mode is provoking a memory error when a program reads up one of the special bit patterns and tries to use it as an address. If you get in a debugger then and look at the object, you're likely to see that it's entirely filled with PYMEM_DEADBYTE (meaning freed memory is getting used) or PYMEM_CLEANBYTE (meaning uninitialized memory is getting used). .. versionchanged:: 3.6 The :c:func:`PyMem_SetupDebugHooks` function now also works on Python compiled in release mode. On error, the debug hooks now use :mod:`tracemalloc` to get the traceback where a memory block was allocated. The debug hooks now also check if there is an :term:`attached thread state` when functions of :c:macro:`PYMEM_DOMAIN_OBJ` and :c:macro:`PYMEM_DOMAIN_MEM` domains are called. .. versionchanged:: 3.8 Byte patterns ``0xCB`` (``PYMEM_CLEANBYTE``), ``0xDB`` (``PYMEM_DEADBYTE``) and ``0xFB`` (``PYMEM_FORBIDDENBYTE``) have been replaced with ``0xCD``, ``0xDD`` and ``0xFD`` to use the same values than Windows CRT debug ``malloc()`` and ``free()``. .. _pymalloc: The pymalloc allocator ====================== Python has a *pymalloc* allocator optimized for small objects (smaller or equal to 512 bytes) with a short lifetime. It uses memory mappings called "arenas" with a fixed size of either 256 KiB on 32-bit platforms or 1 MiB on 64-bit platforms. When Python is configured with :option:`--with-pymalloc-hugepages`, the arena size on 64-bit platforms is increased to 2 MiB to match the huge page size, and arena allocation will attempt to use huge pages (``MAP_HUGETLB`` on Linux, ``MEM_LARGE_PAGES`` on Windows) with automatic fallback to regular pages. It falls back to :c:func:`PyMem_RawMalloc` and :c:func:`PyMem_RawRealloc` for allocations larger than 512 bytes. *pymalloc* is the :ref:`default allocator ` of the :c:macro:`PYMEM_DOMAIN_MEM` (ex: :c:func:`PyMem_Malloc`) and :c:macro:`PYMEM_DOMAIN_OBJ` (ex: :c:func:`PyObject_Malloc`) domains. The arena allocator uses the following functions: * :c:func:`!VirtualAlloc` and :c:func:`!VirtualFree` on Windows, * :c:func:`!mmap` and :c:func:`!munmap` if available, * :c:func:`malloc` and :c:func:`free` otherwise. This allocator is disabled if Python is configured with the :option:`--without-pymalloc` option. It can also be disabled at runtime using the :envvar:`PYTHONMALLOC` environment variable (ex: ``PYTHONMALLOC=malloc``). The pymalloc allocator is disabled by default when Python is built with a sanitizer which does not track pymalloc allocations (:option:`--with-address-sanitizer`, :option:`--with-hwaddress-sanitizer`, :option:`--with-memory-sanitizer`). Use :envvar:`PYTHONMALLOC=pymalloc ` to enable pymalloc. Customize pymalloc Arena Allocator ---------------------------------- .. versionadded:: 3.4 .. c:type:: PyObjectArenaAllocator Structure used to describe an arena allocator. The structure has three fields: +--------------------------------------------------+---------------------------------------+ | Field | Meaning | +==================================================+=======================================+ | ``void *ctx`` | user context passed as first argument | +--------------------------------------------------+---------------------------------------+ | ``void* alloc(void *ctx, size_t size)`` | allocate an arena of size bytes | +--------------------------------------------------+---------------------------------------+ | ``void free(void *ctx, void *ptr, size_t size)`` | free an arena | +--------------------------------------------------+---------------------------------------+ .. c:function:: void PyObject_GetArenaAllocator(PyObjectArenaAllocator *allocator) Get the arena allocator. .. c:function:: void PyObject_SetArenaAllocator(PyObjectArenaAllocator *allocator) Set the arena allocator. .. _mimalloc: The mimalloc allocator ====================== .. versionadded:: 3.13 Python supports the `mimalloc `__ allocator when the underlying platform support is available. mimalloc is a general purpose allocator with excellent performance characteristics, initially developed by Daan Leijen for the runtime systems of the Koka and Lean languages. Unlike :ref:`pymalloc `, which is optimized for small objects (512 bytes or fewer), mimalloc handles allocations of any size. In the :term:`free-threaded ` build, mimalloc is the default and **required** allocator for the :c:macro:`PYMEM_DOMAIN_MEM` and :c:macro:`PYMEM_DOMAIN_OBJ` domains. It cannot be disabled in free-threaded builds. The free-threaded build uses per-thread mimalloc heaps, which allows allocation and deallocation to proceed without locking in most cases. In the default (non-free-threaded) build, mimalloc is available but not the default allocator. It can be selected at runtime using :envvar:`PYTHONMALLOC`\ ``=mimalloc`` (or ``mimalloc_debug`` to include :ref:`debug hooks `). It can be disabled at build time using the :option:`--without-mimalloc` configure option, but this option cannot be combined with :option:`--disable-gil`. tracemalloc C API ================= .. versionadded:: 3.7 .. c:function:: int PyTraceMalloc_Track(unsigned int domain, uintptr_t ptr, size_t size) Track an allocated memory block in the :mod:`tracemalloc` module. Return ``0`` on success, return ``-1`` on error (failed to allocate memory to store the trace). Return ``-2`` if tracemalloc is disabled. If memory block is already tracked, update the existing trace. .. c:function:: int PyTraceMalloc_Untrack(unsigned int domain, uintptr_t ptr) Untrack an allocated memory block in the :mod:`tracemalloc` module. Do nothing if the block was not tracked. Return ``-2`` if tracemalloc is disabled, otherwise return ``0``. .. _memoryexamples: Examples ======== Here is the example from section :ref:`memoryoverview`, rewritten so that the I/O buffer is allocated from the Python heap by using the first function set:: PyObject *res; char *buf = (char *) PyMem_Malloc(BUFSIZ); /* for I/O */ if (buf == NULL) return PyErr_NoMemory(); /* ...Do some I/O operation involving buf... */ res = PyBytes_FromString(buf); PyMem_Free(buf); /* allocated with PyMem_Malloc */ return res; The same code using the type-oriented function set:: PyObject *res; char *buf = PyMem_New(char, BUFSIZ); /* for I/O */ if (buf == NULL) return PyErr_NoMemory(); /* ...Do some I/O operation involving buf... */ res = PyBytes_FromString(buf); PyMem_Free(buf); /* allocated with PyMem_New */ return res; Note that in the two examples above, the buffer is always manipulated via functions belonging to the same set. Indeed, it is required to use the same memory API family for a given memory block, so that the risk of mixing different allocators is reduced to a minimum. The following code sequence contains two errors, one of which is labeled as *fatal* because it mixes two different allocators operating on different heaps. :: char *buf1 = PyMem_New(char, BUFSIZ); char *buf2 = (char *) malloc(BUFSIZ); char *buf3 = (char *) PyMem_Malloc(BUFSIZ); ... PyMem_Del(buf3); /* Wrong -- should be PyMem_Free() */ free(buf2); /* Right -- allocated via malloc() */ free(buf1); /* Fatal -- should be PyMem_Free() */ In addition to the functions aimed at handling raw memory blocks from the Python heap, objects in Python are allocated and released with :c:macro:`PyObject_New`, :c:macro:`PyObject_NewVar` and :c:func:`PyObject_Free`. These will be explained in the next chapter on defining and implementing new object types in C. --- ### Doc C Api Memoryview (`Doc/c-api/memoryview.rst`) .. highlight:: c .. _memoryview-objects: .. index:: pair: object; memoryview MemoryView objects ------------------ A :class:`memoryview` object exposes the C level :ref:`buffer interface ` as a Python object which can then be passed around like any other object. .. c:var:: PyTypeObject PyMemoryView_Type This instance of :c:type:`PyTypeObject` represents the Python memoryview type. This is the same object as :class:`memoryview` in the Python layer. .. c:function:: PyObject *PyMemoryView_FromObject(PyObject *obj) Create a memoryview object from an object that provides the buffer interface. If *obj* supports writable buffer exports, the memoryview object will be read/write, otherwise it may be either read-only or read/write at the discretion of the exporter. .. c:macro:: PyBUF_READ Flag to request a readonly buffer. .. c:macro:: PyBUF_WRITE Flag to request a writable buffer. .. c:function:: PyObject *PyMemoryView_FromMemory(char *mem, Py_ssize_t size, int flags) Create a memoryview object using *mem* as the underlying buffer. *flags* can be one of :c:macro:`PyBUF_READ` or :c:macro:`PyBUF_WRITE`. .. versionadded:: 3.3 .. c:function:: PyObject *PyMemoryView_FromBuffer(const Py_buffer *view) Create a memoryview object wrapping the given buffer structure *view*. For simple byte buffers, :c:func:`PyMemoryView_FromMemory` is the preferred function. .. c:function:: PyObject *PyMemoryView_GetContiguous(PyObject *obj, int buffertype, char order) Create a memoryview object to a :term:`contiguous` chunk of memory (in either 'C' or 'F'ortran *order*) from an object that defines the buffer interface. If memory is contiguous, the memoryview object points to the original memory. Otherwise, a copy is made and the memoryview points to a new bytes object. *buffertype* can be one of :c:macro:`PyBUF_READ` or :c:macro:`PyBUF_WRITE`. .. c:function:: int PyMemoryView_Check(PyObject *obj) Return true if the object *obj* is a memoryview object. It is not currently allowed to create subclasses of :class:`memoryview`. This function always succeeds. .. c:function:: Py_buffer *PyMemoryView_GET_BUFFER(PyObject *mview) Return a pointer to the memoryview's private copy of the exporter's buffer. *mview* **must** be a memoryview instance; this macro doesn't check its type, you must do it yourself or you will risk crashes. .. c:function:: PyObject *PyMemoryView_GET_BASE(PyObject *mview) Return either a pointer to the exporting object that the memoryview is based on or ``NULL`` if the memoryview has been created by one of the functions :c:func:`PyMemoryView_FromMemory` or :c:func:`PyMemoryView_FromBuffer`. *mview* **must** be a memoryview instance. --- ### Doc C Api Method (`Doc/c-api/method.rst`) .. highlight:: c .. _instancemethod-objects: Instance Method Objects ----------------------- .. index:: pair: object; instancemethod An instance method is a wrapper for a :c:type:`PyCFunction` and the new way to bind a :c:type:`PyCFunction` to a class object. It replaces the former call ``PyMethod_New(func, NULL, class)``. .. c:var:: PyTypeObject PyInstanceMethod_Type This instance of :c:type:`PyTypeObject` represents the Python instance method type. It is not exposed to Python programs. .. c:function:: int PyInstanceMethod_Check(PyObject *o) Return true if *o* is an instance method object (has type :c:data:`PyInstanceMethod_Type`). The parameter must not be ``NULL``. This function always succeeds. .. c:function:: PyObject* PyInstanceMethod_New(PyObject *func) Return a new instance method object, with *func* being any callable object. *func* is the function that will be called when the instance method is called. .. c:function:: PyObject* PyInstanceMethod_Function(PyObject *im) Return the function object associated with the instance method *im*. .. c:function:: PyObject* PyInstanceMethod_GET_FUNCTION(PyObject *im) Macro version of :c:func:`PyInstanceMethod_Function` which avoids error checking. .. _method-objects: Method Objects -------------- .. index:: pair: object; method Methods are bound function objects. Methods are always bound to an instance of a user-defined class. Unbound methods (methods bound to a class object) are no longer available. .. c:var:: PyTypeObject PyMethod_Type .. index:: single: MethodType (in module types) This instance of :c:type:`PyTypeObject` represents the Python method type. This is exposed to Python programs as ``types.MethodType``. .. c:function:: int PyMethod_Check(PyObject *o) Return true if *o* is a method object (has type :c:data:`PyMethod_Type`). The parameter must not be ``NULL``. This function always succeeds. .. c:function:: PyObject* PyMethod_New(PyObject *func, PyObject *self) Return a new method object, with *func* being any callable object and *self* the instance the method should be bound. *func* is the function that will be called when the method is called. *self* must not be ``NULL``. .. c:function:: PyObject* PyMethod_Function(PyObject *meth) Return the function object associated with the method *meth*. .. c:function:: PyObject* PyMethod_GET_FUNCTION(PyObject *meth) Macro version of :c:func:`PyMethod_Function` which avoids error checking. .. c:function:: PyObject* PyMethod_Self(PyObject *meth) Return the instance associated with the method *meth*. .. c:function:: PyObject* PyMethod_GET_SELF(PyObject *meth) Macro version of :c:func:`PyMethod_Self` which avoids error checking. --- ### Doc C Api Module (`Doc/c-api/module.rst`) .. highlight:: c .. _moduleobjects: Module Objects ============== .. index:: pair: object; module .. c:var:: PyTypeObject PyModule_Type .. index:: single: ModuleType (in module types) This instance of :c:type:`PyTypeObject` represents the Python module type. This is exposed to Python programs as :py:class:`types.ModuleType`. .. c:function:: int PyModule_Check(PyObject *p) Return true if *p* is a module object, or a subtype of a module object. This function always succeeds. .. c:function:: int PyModule_CheckExact(PyObject *p) Return true if *p* is a module object, but not a subtype of :c:data:`PyModule_Type`. This function always succeeds. .. c:function:: PyObject* PyModule_NewObject(PyObject *name) .. index:: single: __name__ (module attribute) single: __doc__ (module attribute) single: __file__ (module attribute) single: __package__ (module attribute) single: __loader__ (module attribute) Return a new module object with :attr:`module.__name__` set to *name*. The module's :attr:`!__name__`, :attr:`~module.__doc__`, :attr:`~module.__package__` and :attr:`~module.__loader__` attributes are filled in (all but :attr:`!__name__` are set to ``None``). The caller is responsible for setting a :attr:`~module.__file__` attribute. Return ``NULL`` with an exception set on error. .. versionadded:: 3.3 .. versionchanged:: 3.4 :attr:`~module.__package__` and :attr:`~module.__loader__` are now set to ``None``. .. c:function:: PyObject* PyModule_New(const char *name) Similar to :c:func:`PyModule_NewObject`, but the name is a UTF-8 encoded string instead of a Unicode object. .. c:function:: PyObject* PyModule_GetDict(PyObject *module) .. index:: single: __dict__ (module attribute) Return the dictionary object that implements *module*'s namespace; this object is the same as the :attr:`~object.__dict__` attribute of the module object. If *module* is not a module object (or a subtype of a module object), :exc:`SystemError` is raised and ``NULL`` is returned. It is recommended extensions use other ``PyModule_*`` and ``PyObject_*`` functions rather than directly manipulate a module's :attr:`~object.__dict__`. The returned reference is borrowed from the module; it is valid until the module is destroyed. .. c:function:: PyObject* PyModule_GetNameObject(PyObject *module) .. index:: single: __name__ (module attribute) single: SystemError (built-in exception) Return *module*'s :attr:`~module.__name__` value. If the module does not provide one, or if it is not a string, :exc:`SystemError` is raised and ``NULL`` is returned. .. versionadded:: 3.3 .. c:function:: const char* PyModule_GetName(PyObject *module) Similar to :c:func:`PyModule_GetNameObject` but return the name encoded to ``'utf-8'``. The returned buffer is only valid until the module is renamed or destroyed. Note that Python code may rename a module by setting its :py:attr:`~module.__name__` attribute. .. c:function:: PyModuleDef* PyModule_GetDef(PyObject *module) Return a pointer to the :c:type:`PyModuleDef` struct from which the module was created, or ``NULL`` if the module wasn't created from a definition. On error, return ``NULL`` with an exception set. Use :c:func:`PyErr_Occurred` to tell this case apart from a missing :c:type:`!PyModuleDef`. .. c:function:: PyObject* PyModule_GetFilenameObject(PyObject *module) .. index:: single: __file__ (module attribute) single: SystemError (built-in exception) Return the name of the file from which *module* was loaded using *module*'s :attr:`~module.__file__` attribute. If this is not defined, or if it is not a string, raise :exc:`SystemError` and return ``NULL``; otherwise return a reference to a Unicode object. .. versionadded:: 3.2 .. c:function:: const char* PyModule_GetFilename(PyObject *module) Similar to :c:func:`PyModule_GetFilenameObject` but return the filename encoded to 'utf-8'. The returned buffer is only valid until the module's :py:attr:`~module.__file__` attribute is reassigned or the module is destroyed. :c:func:`PyModule_GetFilename` raises :exc:`UnicodeEncodeError` on unencodable filenames, use :c:func:`PyModule_GetFilenameObject` instead. .. _c_module_slots: .. _pymoduledef_slot: Module definition ----------------- Modules created using the C API are typically defined using an array of :c:type:`PySlot` structs, which provides a "description" of how a module should be created. See :ref:`capi-slots` for more information on slots in general. .. versionchanged:: 3.15 Previously, a :c:type:`PyModuleDef` struct was necessary to define modules. The older way of defining modules is still available: consult either the :ref:`pymoduledef` section or earlier versions of this documentation if you plan to support earlier Python versions. The slots array is usually used to define an extension module's “main” module object (see :ref:`extension-modules` for details). It can also be used to :ref:`create extension modules dynamically `. Unless specified otherwise, the same slot ID may not be repeated in an array of slots. Metadata slots .............. .. c:macro:: Py_mod_name :c:member:`Slot ID ` for the name of the new module, as a NUL-terminated UTF8-encoded ``const char *``. Note that modules are typically created using a :py:class:`~importlib.machinery.ModuleSpec`, and when they are, the name from the spec will be used instead of :c:data:`!Py_mod_name`. However, it is still recommended to include this slot for introspection and debugging purposes. .. versionadded:: 3.15 Use :c:member:`PyModuleDef.m_name` instead to support previous versions. .. c:macro:: Py_mod_doc :c:type:`Slot ID ` for the docstring of the new module, as a NUL-terminated UTF8-encoded ``const char *``. Usually it is set to a variable created with :c:macro:`PyDoc_STRVAR`. .. versionadded:: 3.15 Use :c:member:`PyModuleDef.m_doc` instead to support previous versions. Feature slots ............. .. c:macro:: Py_mod_abi :c:member:`Slot ID ` whose value points to a :c:struct:`PyABIInfo` structure describing the ABI that the extension is using. A suitable :c:struct:`!PyABIInfo` variable can be defined using the :c:macro:`PyABIInfo_VAR` macro, as in: .. code-block:: c PyABIInfo_VAR(abi_info); static PySlot mymodule_slots[] = { PySlot_DATA(Py_mod_abi, &abi_info), ... }; When creating a module, Python checks the value of this slot using :c:func:`PyABIInfo_Check`. This slot is required, except for modules created from :c:struct:`PyModuleDef`. .. versionadded:: 3.15 .. c:macro:: Py_mod_multiple_interpreters :c:member:`Slot ID ` whose value is one of: .. c:namespace:: NULL .. c:macro:: Py_MOD_MULTIPLE_INTERPRETERS_NOT_SUPPORTED The module does not support being imported in subinterpreters. .. c:macro:: Py_MOD_MULTIPLE_INTERPRETERS_SUPPORTED The module supports being imported in subinterpreters, but only when they share the main interpreter's GIL. (See :ref:`isolating-extensions-howto`.) .. c:macro:: Py_MOD_PER_INTERPRETER_GIL_SUPPORTED The module supports being imported in subinterpreters, even when they have their own GIL. (See :ref:`isolating-extensions-howto`.) This slot determines whether or not importing this module in a subinterpreter will fail. If ``Py_mod_multiple_interpreters`` is not specified, the import machinery defaults to ``Py_MOD_MULTIPLE_INTERPRETERS_SUPPORTED``. For historical reasons, the values are declared as pointers (``void *``). When using :c:type:`PySlot` arrays, use :c:macro:`PySlot_DATA` for :c:macro:`!Py_mod_multiple_interpreters`: .. code-block:: c PySlot_DATA(Py_mod_multiple_interpreters, Py_MOD_PER_INTERPRETER_GIL_SUPPORTED) .. versionadded:: 3.12 .. c:macro:: Py_mod_gil :c:member:`Slot ID ` whose value is one of: .. c:namespace:: NULL .. c:macro:: Py_MOD_GIL_USED The module depends on the presence of the global interpreter lock (GIL), and may access global state without synchronization. .. c:macro:: Py_MOD_GIL_NOT_USED The module is safe to run without an active GIL. This slot is ignored by Python builds not configured with :option:`--disable-gil`. Otherwise, it determines whether or not importing this module will cause the GIL to be automatically enabled. See :ref:`whatsnew313-free-threaded-cpython` for more detail. If ``Py_mod_gil`` is not specified, the import machinery defaults to ``Py_MOD_GIL_USED``. For historical reasons, the values are declared as pointers (``void *``). When using :c:type:`PySlot` arrays, use :c:macro:`PySlot_DATA` for :c:macro:`!Py_mod_gil`: .. code-block:: c PySlot_DATA(Py_mod_gil, Py_MOD_GIL_NOT_USED) .. versionadded:: 3.13 Creation and initialization slots ................................. .. c:macro:: Py_mod_create :c:member:`Slot ID ` for a function that creates the module object itself. The function must have the signature: .. c:function:: PyObject* create_module(PyObject *spec, PyModuleDef *def) :no-index-entry: :no-contents-entry: The function will be called with: - *spec*: a ``ModuleSpec``-like object, meaning that any attributes defined for :py:class:`importlib.machinery.ModuleSpec` have matching semantics. However, any of the attributes may be missing. - *def*: ``NULL``, or the module definition if the module is created from one. The function should return a new module object, or set an error and return ``NULL``. This function should be kept minimal. In particular, it should not call arbitrary Python code, as trying to import the same module again may result in an infinite loop. If ``Py_mod_create`` is not specified, the import machinery will create a normal module object using :c:func:`PyModule_New`. The name is taken from *spec*, not the definition, to allow extension modules to dynamically adjust to their place in the module hierarchy and be imported under different names through symlinks, all while sharing a single module definition. There is no requirement for the returned object to be an instance of :c:type:`PyModule_Type`. However, some slots may only be used with :c:type:`!PyModule_Type` instances; in particular: - :c:macro:`Py_mod_exec`, - :ref:`module state slots ` (``Py_mod_state_*``), - :c:macro:`Py_mod_token`. .. versionadded:: 3.5 .. versionchanged:: 3.15 The *slots* argument may be a ``ModuleSpec``-like object, rather than a true :py:class:`~importlib.machinery.ModuleSpec` instance. Note that previous versions of CPython did not enforce this. The *def* argument may now be ``NULL``, since modules are not necessarily made from definitions. .. c:macro:: Py_mod_exec :c:member:`Slot ID ` for a function that will :dfn:`execute`, or initialize, the module. This function does the equivalent to executing the code of a Python module: typically, it adds classes and constants to the module. The signature of the function is: .. c:function:: int exec_module(PyObject* module) :no-index-entry: :no-contents-entry: See the :ref:`capi-module-support-functions` section for some useful functions to call. For backwards compatibility, the :c:type:`PyModuleDef.m_slots` array may contain multiple :c:macro:`!Py_mod_exec` slots; these are processed in the order they appear in the array. Elsewhere (that is, in arguments to :c:func:`PyModule_FromSlotsAndSpec` and in return values of :samp:`PyModExport_{}`), repeating the slot is not allowed. .. versionadded:: 3.5 .. versionchanged:: 3.15 Repeated ``Py_mod_exec`` slots are disallowed, except in :c:type:`PyModuleDef.m_slots`. .. c:macro:: Py_mod_methods :c:member:`Slot ID ` for a table of module-level functions, as an array of :c:type:`PyMethodDef` values suitable as the *functions* argument to :c:func:`PyModule_AddFunctions`. Like other slot IDs, a slots array may only contain one :c:macro:`!Py_mod_methods` entry. To add functions from multiple :c:type:`PyMethodDef` arrays, call :c:func:`PyModule_AddFunctions` in the :c:macro:`Py_mod_exec` function. The table must be statically allocated (or otherwise guaranteed to outlive the module object). .. versionadded:: 3.15 Use :c:member:`PyModuleDef.m_methods` instead to support previous versions. .. _ext-module-state: Module state ------------ Extension modules can have *module state* -- a piece of memory that is allocated on module creation, and freed when the module object is deallocated. The module state is specified using :ref:`dedicated slots `. A typical use of module state is storing an exception type -- or indeed *any* type object defined by the module -- Unlike the module's Python attributes, Python code cannot replace or delete data stored in module state. Keeping per-module information in attributes and module state, rather than in static globals, makes module objects *isolated* and safer for use in multiple sub-interpreters. It also helps Python do an orderly clean-up when it shuts down. Extensions that keep references to Python objects as part of module state must implement :c:macro:`Py_mod_state_traverse` and :c:macro:`Py_mod_state_clear` functions to avoid reference leaks. To retrieve the state from a given module, use the following functions: .. c:function:: void* PyModule_GetState(PyObject *module) Return the "state" of the module, that is, a pointer to the block of memory allocated at module creation time, or ``NULL``. See :c:macro:`Py_mod_state_size`. On error, return ``NULL`` with an exception set. Use :c:func:`PyErr_Occurred` to tell this case apart from missing module state. .. c:function:: int PyModule_GetStateSize(PyObject *module, Py_ssize_t *result) Set *\*result* to the size of *module*'s state, as specified using :c:macro:`Py_mod_state_size` (or :c:member:`PyModuleDef.m_size`), and return 0. On error, set *\*result* to -1, and return -1 with an exception set. .. versionadded:: 3.15 .. _ext-module-state-slots: Slots for defining module state ............................... The following :c:member:`slot IDs ` are available for defining the module state. .. c:macro:: Py_mod_state_size :c:member:`Slot ID ` for the size of the module state, in bytes. Setting the value to a non-negative value means that the module can be re-initialized and specifies the additional amount of memory it requires for its state. See :PEP:`3121` for more details. Use :c:func:`PyModule_GetStateSize` to retrieve the size of a given module. .. versionadded:: 3.15 Use :c:member:`PyModuleDef.m_size` instead to support previous versions. .. c:macro:: Py_mod_state_traverse :c:member:`Slot ID ` for a traversal function to call during GC traversal of the module object. The signature of the function, and meanings of the arguments, is similar as for :c:member:`PyTypeObject.tp_traverse`: .. c:function:: int traverse_module_state(PyObject *module, visitproc visit, void *arg) :no-index-entry: :no-contents-entry: This function is not called if the module state was requested but is not allocated yet. This is the case immediately after the module is created and before the module is executed (:c:data:`Py_mod_exec` function). More precisely, this function is not called if the state size (:c:data:`Py_mod_state_size`) is greater than 0 and the module state (as returned by :c:func:`PyModule_GetState`) is ``NULL``. .. versionadded:: 3.15 Use :c:member:`PyModuleDef.m_traverse` instead to support previous versions. .. c:macro:: Py_mod_state_clear :c:member:`Slot ID ` for a clear function to call during GC clearing of the module object. The signature of the function is: .. c:function:: int clear_module_state(PyObject* module) :no-index-entry: :no-contents-entry: This function is not called if the module state was requested but is not allocated yet. This is the case immediately after the module is created and before the module is executed (:c:data:`Py_mod_exec` function). More precisely, this function is not called if the state size (:c:data:`Py_mod_state_size`) is greater than 0 and the module state (as returned by :c:func:`PyModule_GetState`) is ``NULL``. Like :c:member:`PyTypeObject.tp_clear`, this function is not *always* called before a module is deallocated. For example, when reference counting is enough to determine that an object is no longer used, the cyclic garbage collector is not involved and the :c:macro:`Py_mod_state_free` function is called directly. .. versionadded:: 3.15 Use :c:member:`PyModuleDef.m_clear` instead to support previous versions. .. c:macro:: Py_mod_state_free :c:member:`Slot ID ` for a function to call during deallocation of the module object. The signature of the function is: .. c:function:: int free_module_state(PyObject* module) :no-index-entry: :no-contents-entry: This function is not called if the module state was requested but is not allocated yet. This is the case immediately after the module is created and before the module is executed (:c:data:`Py_mod_exec` function). More precisely, this function is not called if the state size (:c:data:`Py_mod_state_size`) is greater than 0 and the module state (as returned by :c:func:`PyModule_GetState`) is ``NULL``. .. versionadded:: 3.15 Use :c:member:`PyModuleDef.m_free` instead to support previous versions. .. _ext-module-token: Module token ............ Each module may have an associated *token*: a pointer-sized value intended to identify of the module state's memory layout. This means that if you have a module object, but you are not sure if it “belongs” to your extension, you can check using code like this: .. code-block:: c PyObject *module = void *module_token; if (PyModule_GetToken(module, &module_token) < 0) { return NULL; } if (module_token != your_token) { PyErr_SetString(PyExc_ValueError, "unexpected module") return NULL; } // This module's state has the expected memory layout; it's safe to cast struct my_state state = (struct my_state*)PyModule_GetState(module) A module's token -- and the *your_token* value to use in the above code -- is: - For modules created with :c:type:`PyModuleDef`: the address of that :c:type:`PyModuleDef`; - For modules defined with the :c:macro:`Py_mod_token` slot: the value of that slot; - For modules created from an ``PyModExport_*`` :ref:`export hook `: the slots array that the export hook returned (unless overridden with :c:macro:`Py_mod_token`). .. c:macro:: Py_mod_token :c:member:`Slot ID ` for the module token. If you use this slot to set the module token (rather than rely on the default), you must ensure that: * The pointer outlives the class, so it's not reused for something else while the class exists. * It "belongs" to the extension module where the class lives, so it will not clash with other extensions. * If the token points to a :c:type:`PyModuleDef` struct, the module should behave as if it was created from that :c:type:`PyModuleDef`. In particular, the module state must have matching layout and semantics. Modules created from :c:type:`PyModuleDef` always use the address of the :c:type:`PyModuleDef` as the token. This means that :c:macro:`!Py_mod_token` cannot be used in :c:member:`PyModuleDef.m_slots`. .. versionadded:: 3.15 .. c:function:: int PyModule_GetToken(PyObject *module, void** result) Set *\*result* to the module token for *module* and return 0. On error, set *\*result* to NULL, and return -1 with an exception set. .. versionadded:: 3.15 See also :c:func:`PyType_GetModuleByToken`. .. _module-from-slots: Creating extension modules dynamically -------------------------------------- The following functions may be used to create an extension module dynamically, rather than from an extension's :ref:`export hook `. .. c:function:: PyObject *PyModule_FromSlotsAndSpec(const PySlot *slots, PyObject *spec) Create a new module object, given an array of :ref:`slots ` and the :py:class:`~importlib.machinery.ModuleSpec` *spec*. The *slots* argument must point to an array of :c:type:`PySlot` structures, terminated by an entry with slot ID of 0 (typically written as :c:macro:`PySlot_END`). The array must include a :c:data:`Py_mod_abi` entry. The *spec* argument may be any ``ModuleSpec``-like object, as described in :c:macro:`Py_mod_create` documentation. Currently, the *spec* must have a ``name`` attribute. On success, return the new module. On error, return ``NULL`` with an exception set. Note that this does not process the module's execution slot (:c:data:`Py_mod_exec`). Both :c:func:`!PyModule_FromSlotsAndSpec` and :c:func:`PyModule_Exec` must be called to fully initialize a module. (See also :ref:`multi-phase-initialization`.) .. versionadded:: 3.15 .. c:function:: int PyModule_Exec(PyObject *module) Execute the :c:data:`Py_mod_exec` slot(s) of *module*. On success, return 0. On error, return -1 with an exception set. For clarity: If *module* has no slots, for example if it uses :ref:`legacy single-phase initialization `, this function does nothing and returns 0. .. versionadded:: 3.15 .. _pymoduledef: Module definition struct ------------------------ Traditionally, extension modules were defined using a *module definition* as the “description" of how a module should be created. Rather than using an array of :ref:`slots ` directly, the definition has dedicated members for most common functionality, and allows additional slots as an extension mechanism. This way of defining modules is still available and there are no plans to remove it. .. c:type:: PyModuleDef The module definition struct, which holds information needed to create a module object. This structure must be statically allocated (or be otherwise guaranteed to be valid while any modules created from it exist). Usually, there is only one variable of this type for each extension module defined this way. The struct, including all members, is part of the :ref:`Stable ABI ` for non-free-threaded builds (``abi3``). In the Stable ABI for free-threaded builds (``abi3t``), this struct is opaque, and unusable in practice; see :ref:`pymoduledef_slot` for a replacement. .. c:member:: PyModuleDef_Base m_base Always initialize this member to :c:macro:`PyModuleDef_HEAD_INIT`: .. c:namespace:: NULL .. c:type:: PyModuleDef_Base The type of :c:member:`!PyModuleDef.m_base`. The struct is part of the :ref:`Stable ABI ` for non-free-threaded builds (``abi3``). In the Stable ABI for Free-Threaded Builds (``abi3t``), this struct is opaque, and unusable in practice. .. c:macro:: PyModuleDef_HEAD_INIT The required initial value for :c:member:`!PyModuleDef.m_base`. .. c:member:: const char *m_name Corresponds to the :c:macro:`Py_mod_name` slot. .. c:member:: const char *m_doc These members correspond to the :c:macro:`Py_mod_doc` slot. Setting this to NULL is equivalent to omitting the slot. .. c:member:: Py_ssize_t m_size Corresponds to the :c:macro:`Py_mod_state_size` slot. Setting this to zero is equivalent to omitting the slot. When using :ref:`legacy single-phase initialization ` or when creating modules dynamically using :c:func:`PyModule_Create` or :c:func:`PyModule_Create2`, :c:member:`!m_size` may be set to -1. This indicates that the module does not support sub-interpreters, because it has global state. .. c:member:: PyMethodDef *m_methods Corresponds to the :c:macro:`Py_mod_methods` slot. Setting this to NULL is equivalent to omitting the slot. .. c:member:: PyModuleDef_Slot* m_slots An array of additional slots, terminated by a ``{0, NULL}`` entry. Note that the entries use the older :c:type:`PyModuleDef_Slot` structure, rather than :c:type:`PySlot`. If the array contains slots corresponding to :c:type:`PyModuleDef` members, the values must match. For example, if you use :c:macro:`Py_mod_name` in :c:member:`!m_slots`, :c:member:`PyModuleDef.m_name` must be set to the same pointer (not just an equal string). .. versionchanged:: 3.5 Prior to version 3.5, this member was always set to ``NULL``, and was defined as: .. c:member:: inquiry m_reload .. c:namespace:: NULL .. c:type:: PyModuleDef_Slot Older structure defining additional slots of a module. Note that a :c:type:`!PyModuleDef_Slot` array may be included in a :c:type:`!PySlot` array using :c:macro:`Py_mod_slots`, and vice versa using :c:macro:`Py_slot_subslots`. Each :c:type:`!PyModuleDef_Slot` structure ``modslot`` is interpreted as the following :c:type:`PySlot` structure:: (PySlot){ .sl_id=modslot.slot, .sl_flags=PySlot_INTPTR | sub_static, .sl_ptr=modslot.value } where ``sub_static`` is ``PySlot_STATIC`` if the slot requires the flag (such as for :c:macro:`Py_mod_methods`), or if this flag is present on the "parent" :c:macro:`!Py_mod_slots` slot (if any). .. c:member:: int slot Corresponds to :c:member:`PySlot.sl_id`. .. c:member:: void* value Corresponds to :c:member:`PySlot.sl_ptr`. .. versionadded:: 3.5 .. c:member:: traverseproc m_traverse inquiry m_clear freefunc m_free These members correspond to the :c:macro:`Py_mod_state_traverse`, :c:macro:`Py_mod_state_clear`, and :c:macro:`Py_mod_state_free` slots, respectively. Setting these members to NULL is equivalent to omitting the corresponding slots. .. versionchanged:: 3.9 :c:member:`m_traverse`, :c:member:`m_clear` and :c:member:`m_free` functions are no longer called before the module state is allocated. .. c:var:: PyTypeObject PyModuleDef_Type The type of ``PyModuleDef`` objects. .. c:macro:: Py_mod_slots :c:member:`Slot ID ` that works like :c:macro:`Py_slot_subslots`, except it specifies an array of :c:type:`PyModuleDef_Slot` structures. .. versionadded:: 3.15 .. _moduledef-dynamic: The following API can be used to create modules from a :c:type:`!PyModuleDef` struct: .. c:function:: PyObject* PyModule_Create(PyModuleDef *def) Create a new module object, given the definition in *def*. This is a macro that calls :c:func:`PyModule_Create2` with *module_api_version* set to :c:macro:`PYTHON_API_VERSION`, or to :c:macro:`PYTHON_ABI_VERSION` if using the :ref:`limited API `. .. c:function:: PyObject* PyModule_Create2(PyModuleDef *def, int module_api_version) Create a new module object, given the definition in *def*, assuming the API version *module_api_version*. If that version does not match the version of the running interpreter, a :exc:`RuntimeWarning` is emitted. Return ``NULL`` with an exception set on error. This function does not support slots. The :c:member:`~PyModuleDef.m_slots` member of *def* must be ``NULL``. .. note:: Most uses of this function should be using :c:func:`PyModule_Create` instead; only use this if you are sure you need it. .. c:function:: PyObject * PyModule_FromDefAndSpec(PyModuleDef *def, PyObject *spec) This macro calls :c:func:`PyModule_FromDefAndSpec2` with *module_api_version* set to :c:macro:`PYTHON_API_VERSION`, or to :c:macro:`PYTHON_ABI_VERSION` if using the :ref:`limited API `. .. versionadded:: 3.5 .. soft-deprecated:: 3.15 Prefer :c:func:`PyModule_FromSlotsAndSpec` in new code. .. c:function:: PyObject * PyModule_FromDefAndSpec2(PyModuleDef *def, PyObject *spec, int module_api_version) Create a new module object, given the definition in *def* and the ModuleSpec *spec*, assuming the API version *module_api_version*. If that version does not match the version of the running interpreter, a :exc:`RuntimeWarning` is emitted. Return ``NULL`` with an exception set on error. Note that this does not process execution slots (:c:data:`Py_mod_exec`). Both ``PyModule_FromDefAndSpec`` and ``PyModule_ExecDef`` must be called to fully initialize a module. .. note:: Most uses of this function should be using :c:func:`PyModule_FromDefAndSpec` instead; only use this if you are sure you need it. .. versionadded:: 3.5 .. soft-deprecated:: 3.15 Prefer :c:func:`PyModule_FromSlotsAndSpec` in new code. .. c:function:: int PyModule_ExecDef(PyObject *module, PyModuleDef *def) Process any execution slots (:c:data:`Py_mod_exec`) given in *def*. .. versionadded:: 3.5 .. soft-deprecated:: 3.15 To run a module's own execution slots, prefer :c:func:`PyModule_Exec`, which works on modules that were not created from a :c:type:`PyModuleDef` structure. .. c:macro:: PYTHON_API_VERSION PYTHON_API_STRING The C API version, as an integer (``1013``) and string (``"1013"``), respectively. Defined for backwards compatibility. Currently, this constant is not updated in new Python versions, and is not useful for versioning. This may change in the future. .. c:macro:: PYTHON_ABI_VERSION PYTHON_ABI_STRING Defined as ``3`` and ``"3"``, respectively, for backwards compatibility. Currently, this constant is not updated in new Python versions, and is not useful for versioning. This may change in the future. .. _capi-module-support-functions: Support functions ----------------- The following functions are provided to help initialize a module object. They are intended for a module's execution slot (:c:data:`Py_mod_exec`), the initialization function for legacy :ref:`single-phase initialization `, or code that creates modules dynamically. .. c:function:: int PyModule_AddObjectRef(PyObject *module, const char *name, PyObject *value) Add an object to *module* as *name*. This is a convenience function which can be used from the module's initialization function. On success, return ``0``. On error, raise an exception and return ``-1``. Example usage:: static int add_spam(PyObject *module, int value) { PyObject *obj = PyLong_FromLong(value); if (obj == NULL) { return -1; } int res = PyModule_AddObjectRef(module, "spam", obj); Py_DECREF(obj); return res; } To be convenient, the function accepts ``NULL`` *value* with an exception set. In this case, return ``-1`` and just leave the raised exception unchanged. The example can also be written without checking explicitly if *obj* is ``NULL``:: static int add_spam(PyObject *module, int value) { PyObject *obj = PyLong_FromLong(value); int res = PyModule_AddObjectRef(module, "spam", obj); Py_XDECREF(obj); return res; } Note that ``Py_XDECREF()`` should be used instead of ``Py_DECREF()`` in this case, since *obj* can be ``NULL``. The number of different *name* strings passed to this function should be kept small, usually by only using statically allocated strings as *name*. For names that aren't known at compile time, prefer calling :c:func:`PyUnicode_FromString` and :c:func:`PyObject_SetAttr` directly. For more details, see :c:func:`PyUnicode_InternFromString`, which may be used internally to create a key object. .. versionadded:: 3.10 .. c:function:: int PyModule_Add(PyObject *module, const char *name, PyObject *value) Similar to :c:func:`PyModule_AddObjectRef`, but ":term:`steals `" a reference to *value* (even on error). It can be called with a result of function that returns a new reference without bothering to check its result or even saving it to a variable. Example usage:: if (PyModule_Add(module, "spam", PyBytes_FromString(value)) < 0) { goto error; } .. versionadded:: 3.13 .. c:function:: int PyModule_AddObject(PyObject *module, const char *name, PyObject *value) Similar to :c:func:`PyModule_AddObjectRef`, but :term:`steals ` a reference to *value* on success (if it returns ``0``). The new :c:func:`PyModule_Add` or :c:func:`PyModule_AddObjectRef` functions are recommended, since it is easy to introduce reference leaks by misusing the :c:func:`PyModule_AddObject` function. .. note:: Unlike other functions that steal references, ``PyModule_AddObject()`` only releases the reference to *value* **on success**. This means that its return value must be checked, and calling code must :c:func:`Py_XDECREF` *value* manually on error. Example usage:: PyObject *obj = PyBytes_FromString(value); if (PyModule_AddObject(module, "spam", obj) < 0) { // If 'obj' is not NULL and PyModule_AddObject() failed, // 'obj' strong reference must be deleted with Py_XDECREF(). // If 'obj' is NULL, Py_XDECREF() does nothing. Py_XDECREF(obj); goto error; } // PyModule_AddObject() stole a reference to obj: // Py_XDECREF(obj) is not needed here. .. soft-deprecated:: 3.13 .. c:function:: int PyModule_AddIntConstant(PyObject *module, const char *name, long value) Add an integer constant to *module* as *name*. This convenience function can be used from the module's initialization function. Return ``-1`` with an exception set on error, ``0`` on success. This is a convenience function that calls :c:func:`PyLong_FromLong` and :c:func:`PyModule_AddObjectRef`; see their documentation for details. .. c:function:: int PyModule_AddStringConstant(PyObject *module, const char *name, const char *value) Add a string constant to *module* as *name*. This convenience function can be used from the module's initialization function. The string *value* must be ``NULL``-terminated. Return ``-1`` with an exception set on error, ``0`` on success. This is a convenience function that calls :c:func:`PyUnicode_InternFromString` and :c:func:`PyModule_AddObjectRef`; see their documentation for details. .. c:macro:: PyModule_AddIntMacro(module, macro) Add an int constant to *module*. The name and the value are taken from *macro*. For example ``PyModule_AddIntMacro(module, AF_INET)`` adds the int constant *AF_INET* with the value of *AF_INET* to *module*. Return ``-1`` with an exception set on error, ``0`` on success. .. c:macro:: PyModule_AddStringMacro(module, macro) Add a string constant to *module*. .. c:function:: int PyModule_AddType(PyObject *module, PyTypeObject *type) Add a type object to *module*. The type object is finalized by calling internally :c:func:`PyType_Ready`. The name of the type object is taken from the last component of :c:member:`~PyTypeObject.tp_name` after dot. Return ``-1`` with an exception set on error, ``0`` on success. .. versionadded:: 3.9 .. c:function:: int PyModule_AddFunctions(PyObject *module, PyMethodDef *functions) Add the functions from the ``NULL`` terminated *functions* array to *module*. Refer to the :c:type:`PyMethodDef` documentation for details on individual entries (due to the lack of a shared module namespace, module level "functions" implemented in C typically receive the module as their first parameter, making them similar to instance methods on Python classes). This function is called automatically when creating a module from ``PyModuleDef`` (such as when using :ref:`multi-phase-initialization`, ``PyModule_Create``, or ``PyModule_FromDefAndSpec``). Some module authors may prefer defining functions in multiple :c:type:`PyMethodDef` arrays; in that case they should call this function directly. The *functions* array must be statically allocated (or otherwise guaranteed to outlive the module object). .. versionadded:: 3.5 .. c:function:: int PyModule_SetDocString(PyObject *module, const char *docstring) Set the docstring for *module* to *docstring*. This function is called automatically when creating a module from ``PyModuleDef`` (such as when using :ref:`multi-phase-initialization`, ``PyModule_Create``, or ``PyModule_FromDefAndSpec``). Return ``0`` on success. Return ``-1`` with an exception set on error. .. versionadded:: 3.5 .. c:function:: int PyUnstable_Module_SetGIL(PyObject *module, void *gil) Indicate that *module* does or does not support running without the global interpreter lock (GIL), using one of the values from :c:macro:`Py_mod_gil`. It must be called during *module*'s initialization function when using :ref:`single-phase-initialization`. If this function is not called during module initialization, the import machinery assumes the module does not support running without the GIL. This function is only available in Python builds configured with :option:`--disable-gil`. Return ``-1`` with an exception set on error, ``0`` on success. .. versionadded:: 3.13 Module lookup (single-phase initialization) ........................................... The legacy :ref:`single-phase initialization ` initialization scheme creates singleton modules that can be looked up in the context of the current interpreter. This allows the module object to be retrieved later with only a reference to the module definition. These functions will not work on modules created using multi-phase initialization, since multiple such modules can be created from a single definition. .. c:function:: PyObject* PyState_FindModule(PyModuleDef *def) Returns the module object that was created from *def* for the current interpreter. This method requires that the module object has been attached to the interpreter state with :c:func:`PyState_AddModule` beforehand. In case the corresponding module object is not found or has not been attached to the interpreter state yet, it returns ``NULL``. .. c:function:: int PyState_AddModule(PyObject *module, PyModuleDef *def) Attaches the module object passed to the function to the interpreter state. This allows the module object to be accessible via :c:func:`PyState_FindModule`. Only effective on modules created using single-phase initialization. Python calls ``PyState_AddModule`` automatically after importing a module that uses :ref:`single-phase initialization `, so it is unnecessary (but harmless) to call it from module initialization code. An explicit call is needed only if the module's own init code subsequently calls ``PyState_FindModule``. The function is mainly intended for implementing alternative import mechanisms (either by calling it directly, or by referring to its implementation for details of the required state updates). If a module was attached previously using the same *def*, it is replaced by the new *module*. The caller must have an :term:`attached thread state`. Return ``-1`` with an exception set on error, ``0`` on success. .. versionadded:: 3.3 .. c:function:: int PyState_RemoveModule(PyModuleDef *def) Removes the module object created from *def* from the interpreter state. Return ``-1`` with an exception set on error, ``0`` on success. The caller must have an :term:`attached thread state`. .. versionadded:: 3.3 --- ### Doc C Api Monitoring (`Doc/c-api/monitoring.rst`) .. highlight:: c .. _c-api-monitoring: Monitoring C API ================ Added in version 3.13. An extension may need to interact with the event monitoring system. Subscribing to events and registering callbacks can be done via the Python API exposed in :mod:`sys.monitoring`. Generating Execution Events =========================== The functions below make it possible for an extension to fire monitoring events as it emulates the execution of Python code. Each of these functions accepts a ``PyMonitoringState`` struct which contains concise information about the activation state of events, as well as the event arguments, which include a ``PyObject*`` representing the code object, the instruction offset and sometimes additional, event-specific arguments (see :mod:`sys.monitoring` for details about the signatures of the different event callbacks). The ``codelike`` argument should be an instance of :class:`types.CodeType` or of a type that emulates it. The VM disables tracing when firing an event, so there is no need for user code to do that. Monitoring functions should not be called with an exception set, except those listed below as working with the current exception. .. c:type:: PyMonitoringState Representation of the state of an event type. It is allocated by the user while its contents are maintained by the monitoring API functions described below. All of the functions below return 0 on success and -1 (with an exception set) on error. See :mod:`sys.monitoring` for descriptions of the events. .. c:function:: int PyMonitoring_FirePyStartEvent(PyMonitoringState *state, PyObject *codelike, int32_t offset) Fire a ``PY_START`` event. .. c:function:: int PyMonitoring_FirePyResumeEvent(PyMonitoringState *state, PyObject *codelike, int32_t offset) Fire a ``PY_RESUME`` event. .. c:function:: int PyMonitoring_FirePyReturnEvent(PyMonitoringState *state, PyObject *codelike, int32_t offset, PyObject* retval) Fire a ``PY_RETURN`` event. .. c:function:: int PyMonitoring_FirePyYieldEvent(PyMonitoringState *state, PyObject *codelike, int32_t offset, PyObject* retval) Fire a ``PY_YIELD`` event. .. c:function:: int PyMonitoring_FireCallEvent(PyMonitoringState *state, PyObject *codelike, int32_t offset, PyObject* callable, PyObject *arg0) Fire a ``CALL`` event. .. c:function:: int PyMonitoring_FireLineEvent(PyMonitoringState *state, PyObject *codelike, int32_t offset, int lineno) Fire a ``LINE`` event. .. c:function:: int PyMonitoring_FireJumpEvent(PyMonitoringState *state, PyObject *codelike, int32_t offset, PyObject *target_offset) Fire a ``JUMP`` event. .. c:function:: int PyMonitoring_FireBranchLeftEvent(PyMonitoringState *state, PyObject *codelike, int32_t offset, PyObject *target_offset) Fire a ``BRANCH_LEFT`` event. .. c:function:: int PyMonitoring_FireBranchRightEvent(PyMonitoringState *state, PyObject *codelike, int32_t offset, PyObject *target_offset) Fire a ``BRANCH_RIGHT`` event. .. c:function:: int PyMonitoring_FireCReturnEvent(PyMonitoringState *state, PyObject *codelike, int32_t offset, PyObject *retval) Fire a ``C_RETURN`` event. .. c:function:: int PyMonitoring_FirePyThrowEvent(PyMonitoringState *state, PyObject *codelike, int32_t offset) Fire a ``PY_THROW`` event with the current exception (as returned by :c:func:`PyErr_GetRaisedException`). .. c:function:: int PyMonitoring_FireRaiseEvent(PyMonitoringState *state, PyObject *codelike, int32_t offset) Fire a ``RAISE`` event with the current exception (as returned by :c:func:`PyErr_GetRaisedException`). .. c:function:: int PyMonitoring_FireCRaiseEvent(PyMonitoringState *state, PyObject *codelike, int32_t offset) Fire a ``C_RAISE`` event with the current exception (as returned by :c:func:`PyErr_GetRaisedException`). .. c:function:: int PyMonitoring_FireReraiseEvent(PyMonitoringState *state, PyObject *codelike, int32_t offset) Fire a ``RERAISE`` event with the current exception (as returned by :c:func:`PyErr_GetRaisedException`). .. c:function:: int PyMonitoring_FireExceptionHandledEvent(PyMonitoringState *state, PyObject *codelike, int32_t offset) Fire an ``EXCEPTION_HANDLED`` event with the current exception (as returned by :c:func:`PyErr_GetRaisedException`). .. c:function:: int PyMonitoring_FirePyUnwindEvent(PyMonitoringState *state, PyObject *codelike, int32_t offset) Fire a ``PY_UNWIND`` event with the current exception (as returned by :c:func:`PyErr_GetRaisedException`). .. c:function:: int PyMonitoring_FireStopIterationEvent(PyMonitoringState *state, PyObject *codelike, int32_t offset, PyObject *value) Fire a ``STOP_ITERATION`` event. If ``value`` is an instance of :exc:`StopIteration`, it is used. Otherwise, a new :exc:`StopIteration` instance is created with ``value`` as its argument. Managing the Monitoring State ----------------------------- Monitoring states can be managed with the help of monitoring scopes. A scope would typically correspond to a Python function. .. c:function:: int PyMonitoring_EnterScope(PyMonitoringState *state_array, uint64_t *version, const uint8_t *event_types, Py_ssize_t length) Enter a monitored scope. ``event_types`` is an array of the event IDs for events that may be fired from the scope. For example, the ID of a ``PY_START`` event is the value ``PY_MONITORING_EVENT_PY_START``, which is numerically equal to the base-2 logarithm of ``sys.monitoring.events.PY_START``. ``state_array`` is an array with a monitoring state entry for each event in ``event_types``, it is allocated by the user but populated by :c:func:`!PyMonitoring_EnterScope` with information about the activation state of the event. The size of ``event_types`` (and hence also of ``state_array``) is given in ``length``. The ``version`` argument is a pointer to a value which should be allocated by the user together with ``state_array`` and initialized to 0, and then set only by :c:func:`!PyMonitoring_EnterScope` itself. It allows this function to determine whether event states have changed since the previous call, and to return quickly if they have not. The scopes referred to here are lexical scopes: a function, class or method. :c:func:`!PyMonitoring_EnterScope` should be called whenever the lexical scope is entered. Scopes can be reentered, reusing the same *state_array* and *version*, in situations like when emulating a recursive Python function. When a code-like's execution is paused, such as when emulating a generator, the scope needs to be exited and re-entered. The macros for *event_types* are: .. c:namespace:: NULL .. The table is here to make the docs searchable, and to allow automatic links to the identifiers. ================================================== ===================================== Macro Event ================================================== ===================================== .. c:macro:: PY_MONITORING_EVENT_BRANCH_LEFT :monitoring-event:`BRANCH_LEFT` .. c:macro:: PY_MONITORING_EVENT_BRANCH_RIGHT :monitoring-event:`BRANCH_RIGHT` .. c:macro:: PY_MONITORING_EVENT_CALL :monitoring-event:`CALL` .. c:macro:: PY_MONITORING_EVENT_C_RAISE :monitoring-event:`C_RAISE` .. c:macro:: PY_MONITORING_EVENT_C_RETURN :monitoring-event:`C_RETURN` .. c:macro:: PY_MONITORING_EVENT_EXCEPTION_HANDLED :monitoring-event:`EXCEPTION_HANDLED` .. c:macro:: PY_MONITORING_EVENT_INSTRUCTION :monitoring-event:`INSTRUCTION` .. c:macro:: PY_MONITORING_EVENT_JUMP :monitoring-event:`JUMP` .. c:macro:: PY_MONITORING_EVENT_LINE :monitoring-event:`LINE` .. c:macro:: PY_MONITORING_EVENT_PY_RESUME :monitoring-event:`PY_RESUME` .. c:macro:: PY_MONITORING_EVENT_PY_RETURN :monitoring-event:`PY_RETURN` .. c:macro:: PY_MONITORING_EVENT_PY_START :monitoring-event:`PY_START` .. c:macro:: PY_MONITORING_EVENT_PY_THROW :monitoring-event:`PY_THROW` .. c:macro:: PY_MONITORING_EVENT_PY_UNWIND :monitoring-event:`PY_UNWIND` .. c:macro:: PY_MONITORING_EVENT_PY_YIELD :monitoring-event:`PY_YIELD` .. c:macro:: PY_MONITORING_EVENT_RAISE :monitoring-event:`RAISE` .. c:macro:: PY_MONITORING_EVENT_RERAISE :monitoring-event:`RERAISE` .. c:macro:: PY_MONITORING_EVENT_STOP_ITERATION :monitoring-event:`STOP_ITERATION` ================================================== ===================================== .. c:function:: int PyMonitoring_ExitScope(void) Exit the last scope that was entered with :c:func:`!PyMonitoring_EnterScope`. .. c:function:: int PY_MONITORING_IS_INSTRUMENTED_EVENT(uint8_t ev) Return true if the event corresponding to the event ID *ev* is a :ref:`local event `. .. versionadded:: 3.13 .. soft-deprecated:: 3.14 --- ### Doc C Api None (`Doc/c-api/none.rst`) .. highlight:: c .. _noneobject: The ``None`` Object ------------------- .. index:: pair: object; None Note that the :c:type:`PyTypeObject` for ``None`` is not directly exposed in the Python/C API. Since ``None`` is a singleton, testing for object identity (using ``==`` in C) is sufficient. There is no :c:func:`!PyNone_Check` function for the same reason. .. c:var:: PyObject* Py_None The Python ``None`` object, denoting lack of value. This object has no methods and is :term:`immortal`. .. versionchanged:: 3.12 :c:data:`Py_None` is :term:`immortal`. .. c:macro:: Py_RETURN_NONE Return :c:data:`Py_None` from a function. --- ### Doc C Api Number (`Doc/c-api/number.rst`) .. highlight:: c .. _number: Number Protocol =============== .. c:function:: int PyNumber_Check(PyObject *o) Returns ``1`` if the object *o* provides numeric protocols, and false otherwise. This function always succeeds. .. versionchanged:: 3.8 Returns ``1`` if *o* is an index integer. .. c:function:: PyObject* PyNumber_Add(PyObject *o1, PyObject *o2) Returns the result of adding *o1* and *o2*, or ``NULL`` on failure. This is the equivalent of the Python expression ``o1 + o2``. .. c:function:: PyObject* PyNumber_Subtract(PyObject *o1, PyObject *o2) Returns the result of subtracting *o2* from *o1*, or ``NULL`` on failure. This is the equivalent of the Python expression ``o1 - o2``. .. c:function:: PyObject* PyNumber_Multiply(PyObject *o1, PyObject *o2) Returns the result of multiplying *o1* and *o2*, or ``NULL`` on failure. This is the equivalent of the Python expression ``o1 * o2``. .. c:function:: PyObject* PyNumber_MatrixMultiply(PyObject *o1, PyObject *o2) Returns the result of matrix multiplication on *o1* and *o2*, or ``NULL`` on failure. This is the equivalent of the Python expression ``o1 @ o2``. .. versionadded:: 3.5 .. c:function:: PyObject* PyNumber_FloorDivide(PyObject *o1, PyObject *o2) Return the floor of *o1* divided by *o2*, or ``NULL`` on failure. This is the equivalent of the Python expression ``o1 // o2``. .. c:function:: PyObject* PyNumber_TrueDivide(PyObject *o1, PyObject *o2) Return a reasonable approximation for the mathematical value of *o1* divided by *o2*, or ``NULL`` on failure. The return value is "approximate" because binary floating-point numbers are approximate; it is not possible to represent all real numbers in base two. This function can return a floating-point value when passed two integers. This is the equivalent of the Python expression ``o1 / o2``. .. c:function:: PyObject* PyNumber_Remainder(PyObject *o1, PyObject *o2) Returns the remainder of dividing *o1* by *o2*, or ``NULL`` on failure. This is the equivalent of the Python expression ``o1 % o2``. .. c:function:: PyObject* PyNumber_Divmod(PyObject *o1, PyObject *o2) .. index:: pair: built-in function; divmod See the built-in function :func:`divmod`. Returns ``NULL`` on failure. This is the equivalent of the Python expression ``divmod(o1, o2)``. .. c:function:: PyObject* PyNumber_Power(PyObject *o1, PyObject *o2, PyObject *o3) .. index:: pair: built-in function; pow See the built-in function :func:`pow`. Returns ``NULL`` on failure. This is the equivalent of the Python expression ``pow(o1, o2, o3)``, where *o3* is optional. If *o3* is to be ignored, pass :c:data:`Py_None` in its place (passing ``NULL`` for *o3* would cause an illegal memory access). .. c:function:: PyObject* PyNumber_Negative(PyObject *o) Returns the negation of *o* on success, or ``NULL`` on failure. This is the equivalent of the Python expression ``-o``. .. c:function:: PyObject* PyNumber_Positive(PyObject *o) Returns *o* on success, or ``NULL`` on failure. This is the equivalent of the Python expression ``+o``. .. c:function:: PyObject* PyNumber_Absolute(PyObject *o) .. index:: pair: built-in function; abs Returns the absolute value of *o*, or ``NULL`` on failure. This is the equivalent of the Python expression ``abs(o)``. .. c:function:: PyObject* PyNumber_Invert(PyObject *o) Returns the bitwise negation of *o* on success, or ``NULL`` on failure. This is the equivalent of the Python expression ``~o``. .. c:function:: PyObject* PyNumber_Lshift(PyObject *o1, PyObject *o2) Returns the result of left shifting *o1* by *o2* on success, or ``NULL`` on failure. This is the equivalent of the Python expression ``o1 << o2``. .. c:function:: PyObject* PyNumber_Rshift(PyObject *o1, PyObject *o2) Returns the result of right shifting *o1* by *o2* on success, or ``NULL`` on failure. This is the equivalent of the Python expression ``o1 >> o2``. .. c:function:: PyObject* PyNumber_And(PyObject *o1, PyObject *o2) Returns the "bitwise and" of *o1* and *o2* on success and ``NULL`` on failure. This is the equivalent of the Python expression ``o1 & o2``. .. c:function:: PyObject* PyNumber_Xor(PyObject *o1, PyObject *o2) Returns the "bitwise exclusive or" of *o1* by *o2* on success, or ``NULL`` on failure. This is the equivalent of the Python expression ``o1 ^ o2``. .. c:function:: PyObject* PyNumber_Or(PyObject *o1, PyObject *o2) Returns the "bitwise or" of *o1* and *o2* on success, or ``NULL`` on failure. This is the equivalent of the Python expression ``o1 | o2``. .. c:function:: PyObject* PyNumber_InPlaceAdd(PyObject *o1, PyObject *o2) Returns the result of adding *o1* and *o2*, or ``NULL`` on failure. The operation is done *in-place* when *o1* supports it. This is the equivalent of the Python statement ``o1 += o2``. .. c:function:: PyObject* PyNumber_InPlaceSubtract(PyObject *o1, PyObject *o2) Returns the result of subtracting *o2* from *o1*, or ``NULL`` on failure. The operation is done *in-place* when *o1* supports it. This is the equivalent of the Python statement ``o1 -= o2``. .. c:function:: PyObject* PyNumber_InPlaceMultiply(PyObject *o1, PyObject *o2) Returns the result of multiplying *o1* and *o2*, or ``NULL`` on failure. The operation is done *in-place* when *o1* supports it. This is the equivalent of the Python statement ``o1 *= o2``. .. c:function:: PyObject* PyNumber_InPlaceMatrixMultiply(PyObject *o1, PyObject *o2) Returns the result of matrix multiplication on *o1* and *o2*, or ``NULL`` on failure. The operation is done *in-place* when *o1* supports it. This is the equivalent of the Python statement ``o1 @= o2``. .. versionadded:: 3.5 .. c:function:: PyObject* PyNumber_InPlaceFloorDivide(PyObject *o1, PyObject *o2) Returns the mathematical floor of dividing *o1* by *o2*, or ``NULL`` on failure. The operation is done *in-place* when *o1* supports it. This is the equivalent of the Python statement ``o1 //= o2``. .. c:function:: PyObject* PyNumber_InPlaceTrueDivide(PyObject *o1, PyObject *o2) Return a reasonable approximation for the mathematical value of *o1* divided by *o2*, or ``NULL`` on failure. The return value is "approximate" because binary floating-point numbers are approximate; it is not possible to represent all real numbers in base two. This function can return a floating-point value when passed two integers. The operation is done *in-place* when *o1* supports it. This is the equivalent of the Python statement ``o1 /= o2``. .. c:function:: PyObject* PyNumber_InPlaceRemainder(PyObject *o1, PyObject *o2) Returns the remainder of dividing *o1* by *o2*, or ``NULL`` on failure. The operation is done *in-place* when *o1* supports it. This is the equivalent of the Python statement ``o1 %= o2``. .. c:function:: PyObject* PyNumber_InPlacePower(PyObject *o1, PyObject *o2, PyObject *o3) .. index:: pair: built-in function; pow See the built-in function :func:`pow`. Returns ``NULL`` on failure. The operation is done *in-place* when *o1* supports it. This is the equivalent of the Python statement ``o1 **= o2`` when o3 is :c:data:`Py_None`, or an in-place variant of ``pow(o1, o2, o3)`` otherwise. If *o3* is to be ignored, pass :c:data:`Py_None` in its place (passing ``NULL`` for *o3* would cause an illegal memory access). .. c:function:: PyObject* PyNumber_InPlaceLshift(PyObject *o1, PyObject *o2) Returns the result of left shifting *o1* by *o2* on success, or ``NULL`` on failure. The operation is done *in-place* when *o1* supports it. This is the equivalent of the Python statement ``o1 <<= o2``. .. c:function:: PyObject* PyNumber_InPlaceRshift(PyObject *o1, PyObject *o2) Returns the result of right shifting *o1* by *o2* on success, or ``NULL`` on failure. The operation is done *in-place* when *o1* supports it. This is the equivalent of the Python statement ``o1 >>= o2``. .. c:function:: PyObject* PyNumber_InPlaceAnd(PyObject *o1, PyObject *o2) Returns the "bitwise and" of *o1* and *o2* on success and ``NULL`` on failure. The operation is done *in-place* when *o1* supports it. This is the equivalent of the Python statement ``o1 &= o2``. .. c:function:: PyObject* PyNumber_InPlaceXor(PyObject *o1, PyObject *o2) Returns the "bitwise exclusive or" of *o1* by *o2* on success, or ``NULL`` on failure. The operation is done *in-place* when *o1* supports it. This is the equivalent of the Python statement ``o1 ^= o2``. .. c:function:: PyObject* PyNumber_InPlaceOr(PyObject *o1, PyObject *o2) Returns the "bitwise or" of *o1* and *o2* on success, or ``NULL`` on failure. The operation is done *in-place* when *o1* supports it. This is the equivalent of the Python statement ``o1 |= o2``. .. c:function:: PyObject* PyNumber_Long(PyObject *o) .. index:: pair: built-in function; int Returns the *o* converted to an integer object on success, or ``NULL`` on failure. This is the equivalent of the Python expression ``int(o)``. .. c:function:: PyObject* PyNumber_Float(PyObject *o) .. index:: pair: built-in function; float Returns the *o* converted to a float object on success, or ``NULL`` on failure. This is the equivalent of the Python expression ``float(o)``. .. c:function:: PyObject* PyNumber_Index(PyObject *o) Returns the *o* converted to a Python int on success or ``NULL`` with a :exc:`TypeError` exception raised on failure. .. versionchanged:: 3.10 The result always has exact type :class:`int`. Previously, the result could have been an instance of a subclass of ``int``. .. c:function:: PyObject* PyNumber_ToBase(PyObject *n, int base) Returns the integer *n* converted to base *base* as a string. The *base* argument must be one of 2, 8, 10, or 16. For base 2, 8, or 16, the returned string is prefixed with a base marker of ``'0b'``, ``'0o'``, or ``'0x'``, respectively. If *n* is not a Python int, it is converted with :c:func:`PyNumber_Index` first. .. c:function:: Py_ssize_t PyNumber_AsSsize_t(PyObject *o, PyObject *exc) Returns *o* converted to a :c:type:`Py_ssize_t` value if *o* can be interpreted as an integer. If the call fails, an exception is raised and ``-1`` is returned. If *o* can be converted to a Python int but the attempt to convert to a :c:type:`Py_ssize_t` value would raise an :exc:`OverflowError`, then the *exc* argument is the type of exception that will be raised (usually :exc:`IndexError` or :exc:`OverflowError`). If *exc* is ``NULL``, then the exception is cleared and the value is clipped to ``PY_SSIZE_T_MIN`` for a negative integer or ``PY_SSIZE_T_MAX`` for a positive integer. .. c:function:: int PyIndex_Check(PyObject *o) Returns ``1`` if *o* is an index integer (has the ``nb_index`` slot of the ``tp_as_number`` structure filled in), and ``0`` otherwise. This function always succeeds. --- ### Doc C Api Object (`Doc/c-api/object.rst`) .. highlight:: c .. _object: Object Protocol =============== .. c:function:: PyObject* Py_GetConstant(unsigned int constant_id) Get a :term:`strong reference` to a constant. Set an exception and return ``NULL`` if *constant_id* is invalid. *constant_id* must be one of these constant identifiers: .. c:namespace:: NULL ======================================== ===== ========================= Constant Identifier Value Returned object ======================================== ===== ========================= .. c:macro:: Py_CONSTANT_NONE ``0`` :py:data:`None` .. c:macro:: Py_CONSTANT_FALSE ``1`` :py:data:`False` .. c:macro:: Py_CONSTANT_TRUE ``2`` :py:data:`True` .. c:macro:: Py_CONSTANT_ELLIPSIS ``3`` :py:data:`Ellipsis` .. c:macro:: Py_CONSTANT_NOT_IMPLEMENTED ``4`` :py:data:`NotImplemented` .. c:macro:: Py_CONSTANT_ZERO ``5`` ``0`` .. c:macro:: Py_CONSTANT_ONE ``6`` ``1`` .. c:macro:: Py_CONSTANT_EMPTY_STR ``7`` ``''`` .. c:macro:: Py_CONSTANT_EMPTY_BYTES ``8`` ``b''`` .. c:macro:: Py_CONSTANT_EMPTY_TUPLE ``9`` ``()`` ======================================== ===== ========================= Numeric values are only given for projects which cannot use the constant identifiers. .. versionadded:: 3.13 .. impl-detail:: In CPython, all of these constants are :term:`immortal`. .. c:function:: PyObject* Py_GetConstantBorrowed(unsigned int constant_id) Similar to :c:func:`Py_GetConstant`, but return a :term:`borrowed reference`. This function is primarily intended for backwards compatibility: using :c:func:`Py_GetConstant` is recommended for new code. The reference is borrowed from the interpreter, and is valid until the interpreter finalization. .. versionadded:: 3.13 .. c:var:: PyObject* Py_NotImplemented The ``NotImplemented`` singleton, used to signal that an operation is not implemented for the given type combination. .. c:macro:: Py_RETURN_NOTIMPLEMENTED Properly handle returning :c:data:`Py_NotImplemented` from within a C function (that is, create a new :term:`strong reference` to :const:`NotImplemented` and return it). .. c:macro:: Py_PRINT_RAW Flag to be used with multiple functions that print the object (like :c:func:`PyObject_Print` and :c:func:`PyFile_WriteObject`). If passed, these functions use the :func:`str` of the object instead of the :func:`repr`. .. c:function:: int PyObject_Print(PyObject *o, FILE *fp, int flags) Print an object *o*, on file *fp*. Returns ``-1`` on error. The flags argument is used to enable certain printing options. The only option currently supported is :c:macro:`Py_PRINT_RAW`; if given, the :func:`str` of the object is written instead of the :func:`repr`. .. c:function:: void PyObject_Dump(PyObject *op) Dump an object *op* to ``stderr``. This should only be used for debugging. The output is intended to try dumping objects even after memory corruption: * Information is written starting with fields that are the least likely to crash when accessed. * This function can be called without an :term:`attached thread state`, but it's not recommended to do so: it can cause deadlocks. * An object that does not belong to the current interpreter may be dumped, but this may also cause crashes or unintended behavior. * Implement a heuristic to detect if the object memory has been freed. Don't display the object contents in this case, only its memory address. * The output format may change at any time. Example of output: .. code-block:: output object address : 0x7f80124702c0 object refcount : 2 object type : 0x9902e0 object type name: str object repr : 'abcdef' .. versionadded:: 3.15 .. c:function:: int PyObject_HasAttrWithError(PyObject *o, PyObject *attr_name) Returns ``1`` if *o* has the attribute *attr_name*, and ``0`` otherwise. This is equivalent to the Python expression ``hasattr(o, attr_name)``. On failure, return ``-1``. .. versionadded:: 3.13 .. c:function:: int PyObject_HasAttrStringWithError(PyObject *o, const char *attr_name) This is the same as :c:func:`PyObject_HasAttrWithError`, but *attr_name* is specified as a :c:expr:`const char*` UTF-8 encoded bytes string, rather than a :c:expr:`PyObject*`. .. versionadded:: 3.13 .. c:function:: int PyObject_HasAttr(PyObject *o, PyObject *attr_name) Returns ``1`` if *o* has the attribute *attr_name*, and ``0`` otherwise. This function always succeeds. .. note:: Exceptions that occur when this calls :meth:`~object.__getattr__` and :meth:`~object.__getattribute__` methods aren't propagated, but instead given to :func:`sys.unraisablehook`. For proper error handling, use :c:func:`PyObject_HasAttrWithError`, :c:func:`PyObject_GetOptionalAttr` or :c:func:`PyObject_GetAttr` instead. .. c:function:: int PyObject_HasAttrString(PyObject *o, const char *attr_name) This is the same as :c:func:`PyObject_HasAttr`, but *attr_name* is specified as a :c:expr:`const char*` UTF-8 encoded bytes string, rather than a :c:expr:`PyObject*`. .. note:: Exceptions that occur when this calls :meth:`~object.__getattr__` and :meth:`~object.__getattribute__` methods or while creating the temporary :class:`str` object are silently ignored. For proper error handling, use :c:func:`PyObject_HasAttrStringWithError`, :c:func:`PyObject_GetOptionalAttrString` or :c:func:`PyObject_GetAttrString` instead. .. c:function:: PyObject* PyObject_GetAttr(PyObject *o, PyObject *attr_name) Retrieve an attribute named *attr_name* from object *o*. Returns the attribute value on success, or ``NULL`` on failure. This is the equivalent of the Python expression ``o.attr_name``. If the missing attribute should not be treated as a failure, you can use :c:func:`PyObject_GetOptionalAttr` instead. .. c:function:: PyObject* PyObject_GetAttrString(PyObject *o, const char *attr_name) This is the same as :c:func:`PyObject_GetAttr`, but *attr_name* is specified as a :c:expr:`const char*` UTF-8 encoded bytes string, rather than a :c:expr:`PyObject*`. If the missing attribute should not be treated as a failure, you can use :c:func:`PyObject_GetOptionalAttrString` instead. .. c:function:: int PyObject_GetOptionalAttr(PyObject *obj, PyObject *attr_name, PyObject **result); Variant of :c:func:`PyObject_GetAttr` which doesn't raise :exc:`AttributeError` if the attribute is not found. If the attribute is found, return ``1`` and set *\*result* to a new :term:`strong reference` to the attribute. If the attribute is not found, return ``0`` and set *\*result* to ``NULL``; the :exc:`AttributeError` is silenced. If an error other than :exc:`AttributeError` is raised, return ``-1`` and set *\*result* to ``NULL``. .. versionadded:: 3.13 .. c:function:: int PyObject_GetOptionalAttrString(PyObject *obj, const char *attr_name, PyObject **result); This is the same as :c:func:`PyObject_GetOptionalAttr`, but *attr_name* is specified as a :c:expr:`const char*` UTF-8 encoded bytes string, rather than a :c:expr:`PyObject*`. .. versionadded:: 3.13 .. c:function:: PyObject* PyObject_GenericGetAttr(PyObject *o, PyObject *name) Generic attribute getter function that is meant to be put into a type object's ``tp_getattro`` slot. It looks for a descriptor in the dictionary of classes in the object's MRO as well as an attribute in the object's :attr:`~object.__dict__` (if present). As outlined in :ref:`descriptors`, data descriptors take preference over instance attributes, while non-data descriptors don't. Otherwise, an :exc:`AttributeError` is raised. .. c:function:: int PyObject_SetAttr(PyObject *o, PyObject *attr_name, PyObject *v) Set the value of the attribute named *attr_name*, for object *o*, to the value *v*. Raise an exception and return ``-1`` on failure; return ``0`` on success. This is the equivalent of the Python statement ``o.attr_name = v``. If *v* is ``NULL``, the attribute is deleted. This behaviour is deprecated in favour of using :c:func:`PyObject_DelAttr`, but there are currently no plans to remove it. The function must not be called with a ``NULL`` *v* and an exception set. This case can arise from forgetting ``NULL`` checks and would delete the attribute. .. versionchanged:: 3.15 Must not be called with NULL value if an exception is set. .. c:function:: int PyObject_SetAttrString(PyObject *o, const char *attr_name, PyObject *v) This is the same as :c:func:`PyObject_SetAttr`, but *attr_name* is specified as a :c:expr:`const char*` UTF-8 encoded bytes string, rather than a :c:expr:`PyObject*`. If *v* is ``NULL``, the attribute is deleted, but this feature is deprecated in favour of using :c:func:`PyObject_DelAttrString`. The function must not be called with a ``NULL`` *v* and an exception set. This case can arise from forgetting ``NULL`` checks and would delete the attribute. The number of different attribute names passed to this function should be kept small, usually by using a statically allocated string as *attr_name*. For attribute names that aren't known at compile time, prefer calling :c:func:`PyUnicode_FromString` and :c:func:`PyObject_SetAttr` directly. For more details, see :c:func:`PyUnicode_InternFromString`, which may be used internally to create a key object. .. versionchanged:: 3.15 Must not be called with NULL value if an exception is set. .. c:function:: int PyObject_GenericSetAttr(PyObject *o, PyObject *name, PyObject *value) Generic attribute setter and deleter function that is meant to be put into a type object's :c:member:`~PyTypeObject.tp_setattro` slot. It looks for a data descriptor in the dictionary of classes in the object's MRO, and if found it takes preference over setting or deleting the attribute in the instance dictionary. Otherwise, the attribute is set or deleted in the object's :attr:`~object.__dict__` (if present). On success, ``0`` is returned, otherwise an :exc:`AttributeError` is raised and ``-1`` is returned. .. c:function:: int PyObject_DelAttr(PyObject *o, PyObject *attr_name) Delete attribute named *attr_name*, for object *o*. Returns ``-1`` on failure. This is the equivalent of the Python statement ``del o.attr_name``. .. c:function:: int PyObject_DelAttrString(PyObject *o, const char *attr_name) This is the same as :c:func:`PyObject_DelAttr`, but *attr_name* is specified as a :c:expr:`const char*` UTF-8 encoded bytes string, rather than a :c:expr:`PyObject*`. The number of different attribute names passed to this function should be kept small, usually by using a statically allocated string as *attr_name*. For attribute names that aren't known at compile time, prefer calling :c:func:`PyUnicode_FromString` and :c:func:`PyObject_DelAttr` directly. For more details, see :c:func:`PyUnicode_InternFromString`, which may be used internally to create a key object for lookup. .. c:function:: PyObject* PyObject_GenericGetDict(PyObject *o, void *context) A generic implementation for the getter of a ``__dict__`` descriptor. It creates the dictionary if necessary. This function may also be called to get the :py:attr:`~object.__dict__` of the object *o*. Pass ``NULL`` for *context* when calling it. Since this function may need to allocate memory for the dictionary, it may be more efficient to call :c:func:`PyObject_GetAttr` when accessing an attribute on the object. On failure, returns ``NULL`` with an exception set. .. versionadded:: 3.3 .. c:function:: int PyObject_GenericSetDict(PyObject *o, PyObject *value, void *context) A generic implementation for the setter of a ``__dict__`` descriptor. This implementation does not allow the dictionary to be deleted. .. versionadded:: 3.3 .. c:function:: PyObject** _PyObject_GetDictPtr(PyObject *obj) Return a pointer to :py:attr:`~object.__dict__` of the object *obj*. If there is no ``__dict__``, return ``NULL`` without setting an exception. This function may need to allocate memory for the dictionary, so it may be more efficient to call :c:func:`PyObject_GetAttr` when accessing an attribute on the object. .. c:function:: PyObject* PyObject_RichCompare(PyObject *o1, PyObject *o2, int opid) Compare the values of *o1* and *o2* using the operation specified by *opid*, which must be one of :c:macro:`Py_LT`, :c:macro:`Py_LE`, :c:macro:`Py_EQ`, :c:macro:`Py_NE`, :c:macro:`Py_GT`, or :c:macro:`Py_GE`, corresponding to ``<``, ``<=``, ``==``, ``!=``, ``>``, or ``>=`` respectively. This is the equivalent of the Python expression ``o1 op o2``, where ``op`` is the operator corresponding to *opid*. Returns the value of the comparison on success, or ``NULL`` on failure. .. c:function:: int PyObject_RichCompareBool(PyObject *o1, PyObject *o2, int opid) Compare the values of *o1* and *o2* using the operation specified by *opid*, like :c:func:`PyObject_RichCompare`, but returns ``-1`` on error, ``0`` if the result is false, ``1`` otherwise. .. note:: If *o1* and *o2* are the same object, :c:func:`PyObject_RichCompareBool` will always return ``1`` for :c:macro:`Py_EQ` and ``0`` for :c:macro:`Py_NE`. .. c:function:: PyObject* PyObject_Format(PyObject *obj, PyObject *format_spec) Format *obj* using *format_spec*. This is equivalent to the Python expression ``format(obj, format_spec)``. *format_spec* may be ``NULL``. In this case the call is equivalent to ``format(obj)``. Returns the formatted string on success, ``NULL`` on failure. .. c:function:: PyObject* PyObject_Repr(PyObject *o) .. index:: pair: built-in function; repr Compute a string representation of object *o*. Returns the string representation on success, ``NULL`` on failure. This is the equivalent of the Python expression ``repr(o)``. Called by the :func:`repr` built-in function. If argument is ``NULL``, return the string ``''``. .. versionchanged:: 3.4 This function now includes a debug assertion to help ensure that it does not silently discard an active exception. .. c:function:: PyObject* PyObject_ASCII(PyObject *o) .. index:: pair: built-in function; ascii As :c:func:`PyObject_Repr`, compute a string representation of object *o*, but escape the non-ASCII characters in the string returned by :c:func:`PyObject_Repr` with ``\x``, ``\u`` or ``\U`` escapes. This generates a string similar to that returned by :c:func:`PyObject_Repr` in Python 2. Called by the :func:`ascii` built-in function. If argument is ``NULL``, return the string ``''``. .. index:: string; PyObject_Str (C function) .. c:function:: PyObject* PyObject_Str(PyObject *o) Compute a string representation of object *o*. Returns the string representation on success, ``NULL`` on failure. This is the equivalent of the Python expression ``str(o)``. Called by the :func:`str` built-in function and, therefore, by the :func:`print` function. If argument is ``NULL``, return the string ``''``. .. versionchanged:: 3.4 This function now includes a debug assertion to help ensure that it does not silently discard an active exception. .. c:function:: PyObject* PyObject_Bytes(PyObject *o) .. index:: pair: built-in function; bytes Compute a bytes representation of object *o*. ``NULL`` is returned on failure and a bytes object on success. This is equivalent to the Python expression ``bytes(o)``, when *o* is not an integer. Unlike ``bytes(o)``, a TypeError is raised when *o* is an integer instead of a zero-initialized bytes object. If argument is ``NULL``, return the :class:`bytes` object ``b''``. .. c:function:: int PyObject_IsSubclass(PyObject *derived, PyObject *cls) Return ``1`` if the class *derived* is identical to or derived from the class *cls*, otherwise return ``0``. In case of an error, return ``-1``. If *cls* is a tuple, the check will be done against every entry in *cls*. The result will be ``1`` when at least one of the checks returns ``1``, otherwise it will be ``0``. If *cls* has a :meth:`~type.__subclasscheck__` method, it will be called to determine the subclass status as described in :pep:`3119`. Otherwise, *derived* is a subclass of *cls* if it is a direct or indirect subclass, i.e. contained in :attr:`cls.__mro__ `. Normally only class objects, i.e. instances of :class:`type` or a derived class, are considered classes. However, objects can override this by having a :attr:`~type.__bases__` attribute (which must be a tuple of base classes). .. c:function:: int PyObject_IsInstance(PyObject *inst, PyObject *cls) Return ``1`` if *inst* is an instance of the class *cls* or a subclass of *cls*, or ``0`` if not. On error, returns ``-1`` and sets an exception. If *cls* is a tuple, the check will be done against every entry in *cls*. The result will be ``1`` when at least one of the checks returns ``1``, otherwise it will be ``0``. If *cls* has a :meth:`~type.__instancecheck__` method, it will be called to determine the subclass status as described in :pep:`3119`. Otherwise, *inst* is an instance of *cls* if its class is a subclass of *cls*. An instance *inst* can override what is considered its class by having a :attr:`~object.__class__` attribute. An object *cls* can override if it is considered a class, and what its base classes are, by having a :attr:`~type.__bases__` attribute (which must be a tuple of base classes). .. c:function:: Py_hash_t PyObject_Hash(PyObject *o) .. index:: pair: built-in function; hash Compute and return the hash value of an object *o*. On failure, return ``-1``. This is the equivalent of the Python expression ``hash(o)``. .. versionchanged:: 3.2 The return type is now Py_hash_t. This is a signed integer the same size as :c:type:`Py_ssize_t`. .. c:function:: Py_hash_t PyObject_HashNotImplemented(PyObject *o) Set a :exc:`TypeError` indicating that ``type(o)`` is not :term:`hashable` and return ``-1``. This function receives special treatment when stored in a ``tp_hash`` slot, allowing a type to explicitly indicate to the interpreter that it is not hashable. .. c:function:: int PyObject_IsTrue(PyObject *o) Returns ``1`` if the object *o* is considered to be true, and ``0`` otherwise. This is equivalent to the Python expression ``not not o``. On failure, return ``-1``. .. c:function:: int PyObject_Not(PyObject *o) Returns ``0`` if the object *o* is considered to be true, and ``1`` otherwise. This is equivalent to the Python expression ``not o``. On failure, return ``-1``. .. c:function:: PyObject* PyObject_Type(PyObject *o) .. index:: pair: built-in function; type When *o* is non-``NULL``, returns a type object corresponding to the object type of object *o*. On failure, raises :exc:`SystemError` and returns ``NULL``. This is equivalent to the Python expression ``type(o)``. This function creates a new :term:`strong reference` to the return value. There's really no reason to use this function instead of the :c:func:`Py_TYPE()` function, which returns a pointer of type :c:expr:`PyTypeObject*`, except when a new :term:`strong reference` is needed. .. c:function:: int PyObject_TypeCheck(PyObject *o, PyTypeObject *type) Return non-zero if the object *o* is of type *type* or a subtype of *type*, and ``0`` otherwise. Both parameters must be non-``NULL``. .. c:function:: Py_ssize_t PyObject_Size(PyObject *o) Py_ssize_t PyObject_Length(PyObject *o) .. index:: pair: built-in function; len Return the length of object *o*. If the object *o* provides either the sequence and mapping protocols, the sequence length is returned. On error, ``-1`` is returned. This is the equivalent to the Python expression ``len(o)``. .. c:function:: Py_ssize_t PyObject_LengthHint(PyObject *o, Py_ssize_t defaultvalue) Return an estimated length for the object *o*. First try to return its actual length, then an estimate using :meth:`~object.__length_hint__`, and finally return the default value. On error return ``-1``. This is the equivalent to the Python expression ``operator.length_hint(o, defaultvalue)``. .. versionadded:: 3.4 .. c:function:: PyObject* PyObject_GetItem(PyObject *o, PyObject *key) Return element of *o* corresponding to the object *key* or ``NULL`` on failure. This is the equivalent of the Python expression ``o[key]``. .. c:function:: int PyObject_SetItem(PyObject *o, PyObject *key, PyObject *v) Map the object *key* to the value *v*. Raise an exception and return ``-1`` on failure; return ``0`` on success. This is the equivalent of the Python statement ``o[key] = v``. This function *does not* steal a reference to *v*. .. c:function:: int PyObject_DelItem(PyObject *o, PyObject *key) Remove the mapping for the object *key* from the object *o*. Return ``-1`` on failure. This is equivalent to the Python statement ``del o[key]``. .. c:function:: int PyObject_DelItemString(PyObject *o, const char *key) This is the same as :c:func:`PyObject_DelItem`, but *key* is specified as a :c:expr:`const char*` UTF-8 encoded bytes string, rather than a :c:expr:`PyObject*`. .. c:function:: PyObject* PyObject_Dir(PyObject *o) This is equivalent to the Python expression ``dir(o)``, returning a (possibly empty) list of strings appropriate for the object argument, or ``NULL`` if there was an error. If the argument is ``NULL``, this is like the Python ``dir()``, returning the names of the current locals; in this case, if no execution frame is active then ``NULL`` is returned but :c:func:`PyErr_Occurred` will return false. .. c:function:: PyObject* PyObject_GetIter(PyObject *o) This is equivalent to the Python expression ``iter(o)``. It returns a new iterator for the object argument, or the object itself if the object is already an iterator. Raises :exc:`TypeError` and returns ``NULL`` if the object cannot be iterated. .. c:function:: PyObject* PyObject_SelfIter(PyObject *obj) This is equivalent to the Python ``__iter__(self): return self`` method. It is intended for :term:`iterator` types, to be used in the :c:member:`PyTypeObject.tp_iter` slot. .. c:function:: PyObject* PyObject_GetAIter(PyObject *o) This is the equivalent to the Python expression ``aiter(o)``. Takes an :class:`AsyncIterable` object and returns an :class:`AsyncIterator` for it. This is typically a new iterator but if the argument is an :class:`AsyncIterator`, this returns itself. Raises :exc:`TypeError` and returns ``NULL`` if the object cannot be iterated. .. versionadded:: 3.10 .. c:function:: void *PyObject_GetTypeData(PyObject *o, PyTypeObject *cls) Get a pointer to subclass-specific data reserved for *cls*. The object *o* must be an instance of *cls*, and *cls* must have been created using negative :c:member:`PyType_Spec.basicsize`. Python does not check this. On error, set an exception and return ``NULL``. .. versionadded:: 3.12 .. c:function:: Py_ssize_t PyType_GetTypeDataSize(PyTypeObject *cls) Return the size of the instance memory space reserved for *cls*, i.e. the size of the memory :c:func:`PyObject_GetTypeData` returns. This may be larger than requested using :c:member:`-PyType_Spec.basicsize `; it is safe to use this larger size (e.g. with :c:func:`!memset`). The type *cls* **must** have been created using negative :c:member:`PyType_Spec.basicsize`. Python does not check this. On error, set an exception and return a negative value. .. versionadded:: 3.12 .. c:function:: void *PyObject_GetItemData(PyObject *o) Get a pointer to per-item data for a class with :c:macro:`Py_TPFLAGS_ITEMS_AT_END`. On error, set an exception and return ``NULL``. :py:exc:`TypeError` is raised if *o* does not have :c:macro:`Py_TPFLAGS_ITEMS_AT_END` set. .. versionadded:: 3.12 .. c:function:: int PyObject_VisitManagedDict(PyObject *obj, visitproc visit, void *arg) Visit the managed dictionary of *obj*. This function must only be called in a traverse function of the type which has the :c:macro:`Py_TPFLAGS_MANAGED_DICT` flag set. .. versionadded:: 3.13 .. c:function:: void PyObject_ClearManagedDict(PyObject *obj) Clear the managed dictionary of *obj*. This function must only be called in a clear function of the type which has the :c:macro:`Py_TPFLAGS_MANAGED_DICT` flag set. .. versionadded:: 3.13 .. c:function:: int PyUnstable_Object_EnableDeferredRefcount(PyObject *obj) Enable `deferred reference counting `_ on *obj*, if supported by the runtime. In the :term:`free-threaded ` build, this allows the interpreter to avoid reference count adjustments to *obj*, which may improve multi-threaded performance. The tradeoff is that *obj* will only be deallocated by the tracing garbage collector, and not when the interpreter no longer has any references to it. This function returns ``1`` if deferred reference counting is enabled on *obj*, and ``0`` if deferred reference counting is not supported or if the hint was ignored by the interpreter, such as when deferred reference counting is already enabled on *obj*. This function is thread-safe, and cannot fail. This function does nothing on builds with the :term:`GIL` enabled, which do not support deferred reference counting. This also does nothing if *obj* is not an object tracked by the garbage collector (see :func:`gc.is_tracked` and :c:func:`PyObject_GC_IsTracked`). This function is intended to be used soon after *obj* is created, by the code that creates it, such as in the object's :c:member:`~PyTypeObject.tp_new` slot. .. versionadded:: 3.14 .. c:function:: int PyUnstable_Object_IsUniqueReferencedTemporary(PyObject *obj) Check if *obj* is a unique temporary object. Returns ``1`` if *obj* is known to be a unique temporary object, and ``0`` otherwise. This function cannot fail, but the check is conservative, and may return ``0`` in some cases even if *obj* is a unique temporary object. If an object is a unique temporary, it is guaranteed that the current code has the only reference to the object. For arguments to C functions, this should be used instead of checking if the reference count is ``1``. Starting with Python 3.14, the interpreter internally avoids some reference count modifications when loading objects onto the operands stack by :term:`borrowing ` references when possible, which means that a reference count of ``1`` by itself does not guarantee that a function argument uniquely referenced. In the example below, ``my_func`` is called with a unique temporary object as its argument:: my_func([1, 2, 3]) In the example below, ``my_func`` is **not** called with a unique temporary object as its argument, even if its refcount is ``1``:: my_list = [1, 2, 3] my_func(my_list) See also the function :c:func:`Py_REFCNT`. .. versionadded:: 3.14 .. c:function:: int PyUnstable_IsImmortal(PyObject *obj) This function returns non-zero if *obj* is :term:`immortal`, and zero otherwise. This function cannot fail. .. note:: Objects that are immortal in one CPython version are not guaranteed to be immortal in another. .. versionadded:: 3.14 .. c:function:: int PyUnstable_TryIncRef(PyObject *obj) Increments the reference count of *obj* if it is not zero. Returns ``1`` if the object's reference count was successfully incremented. Otherwise, this function returns ``0``. :c:func:`PyUnstable_EnableTryIncRef` must have been called earlier on *obj* or this function may spuriously return ``0`` in the :term:`free-threaded build`. This function is logically equivalent to the following C code, except that it behaves atomically in the :term:`free-threaded build`:: if (Py_REFCNT(op) > 0) { Py_INCREF(op); return 1; } return 0; This is intended as a building block for managing weak references without the overhead of a Python :ref:`weak reference object `. Typically, correct use of this function requires support from *obj*'s deallocator (:c:member:`~PyTypeObject.tp_dealloc`). For example, the following sketch could be adapted to implement a "weakmap" that works like a :py:class:`~weakref.WeakValueDictionary` for a specific type: .. code-block:: c PyMutex mutex; PyObject * add_entry(weakmap_key_type *key, PyObject *value) { PyUnstable_EnableTryIncRef(value); weakmap_type weakmap = ...; PyMutex_Lock(&mutex); weakmap_add_entry(weakmap, key, value); PyMutex_Unlock(&mutex); Py_RETURN_NONE; } PyObject * get_value(weakmap_key_type *key) { weakmap_type weakmap = ...; PyMutex_Lock(&mutex); PyObject *result = weakmap_find(weakmap, key); if (PyUnstable_TryIncRef(result)) { // `result` is safe to use PyMutex_Unlock(&mutex); return result; } // if we get here, `result` is starting to be garbage-collected, // but has not been removed from the weakmap yet PyMutex_Unlock(&mutex); return NULL; } // tp_dealloc function for weakmap values void value_dealloc(PyObject *value) { weakmap_type weakmap = ...; PyMutex_Lock(&mutex); weakmap_remove_value(weakmap, value); ... PyMutex_Unlock(&mutex); } .. versionadded:: 3.14 .. c:function:: void PyUnstable_EnableTryIncRef(PyObject *obj) Enables subsequent uses of :c:func:`PyUnstable_TryIncRef` on *obj*. The caller must hold a :term:`strong reference` to *obj* when calling this. .. versionadded:: 3.14 .. c:function:: int PyUnstable_Object_IsUniquelyReferenced(PyObject *op) Determine if *op* only has one reference. On GIL-enabled builds, this function is equivalent to :c:expr:`Py_REFCNT(op) == 1`. On a :term:`free-threaded build`, this checks if *op*'s :term:`reference count` is equal to one and additionally checks if *op* is only used by this thread. :c:expr:`Py_REFCNT(op) == 1` is **not** thread-safe on free-threaded builds; prefer this function. The caller must hold an :term:`attached thread state`, despite the fact that this function doesn't call into the Python interpreter. This function cannot fail. .. versionadded:: 3.14 .. c:function:: int PyUnstable_SetImmortal(PyObject *op) Marks the object *op* :term:`immortal`. The argument should be uniquely referenced by the calling thread. This is intended to be used for reducing reference counting contention in the :term:`free-threaded build` for objects which are shared across threads. This is a one-way process: objects can only be made immortal; they cannot be made mortal once again. Immortal objects do not participate in reference counting and will never be garbage collected. If the object is GC-tracked, it is untracked. This function is intended to be used soon after *op* is created, by the code that creates it, such as in the object's :c:member:`~PyTypeObject.tp_new` slot. Returns 1 if the object was made immortal and returns 0 if it was not. This function cannot fail. .. versionadded:: 3.15 --- ### Doc C Api Objimpl (`Doc/c-api/objimpl.rst`) .. highlight:: c .. _newtypes: ***************************** Object Implementation Support ***************************** This chapter describes the functions, types, and macros used when defining new object types. .. toctree:: allocation.rst lifecycle.rst structures.rst typeobj.rst gcsupport.rst --- ### Doc C Api Perfmaps (`Doc/c-api/perfmaps.rst`) .. highlight:: c .. _perfmaps: Support for Perf Maps ---------------------- On supported platforms (Linux and macOS), the runtime can take advantage of *perf map files* to make Python functions visible to an external profiling tool (such as `perf `_ or `samply `_). A running process may create a file in the ``/tmp`` directory, which contains entries that can map a section of executable code to a name. This interface is described in the `documentation of the Linux Perf tool `_. In Python, these helper APIs can be used by libraries and features that rely on generating machine code on the fly. Note that holding an :term:`attached thread state` is not required for these APIs. .. c:function:: int PyUnstable_PerfMapState_Init(void) Open the ``/tmp/perf-$pid.map`` file, unless it's already opened, and create a lock to ensure thread-safe writes to the file (provided the writes are done through :c:func:`PyUnstable_WritePerfMapEntry`). Normally, there's no need to call this explicitly; just use :c:func:`PyUnstable_WritePerfMapEntry` and it will initialize the state on first call. Returns ``0`` on success, ``-1`` on failure to create/open the perf map file, or ``-2`` on failure to create a lock. Check ``errno`` for more information about the cause of a failure. .. c:function:: int PyUnstable_WritePerfMapEntry(const void *code_addr, size_t code_size, const char *entry_name) Write one single entry to the ``/tmp/perf-$pid.map`` file. This function is thread safe. Here is what an example entry looks like:: # address size name 7f3529fcf759 b py::bar:/run/t.py Will call :c:func:`PyUnstable_PerfMapState_Init` before writing the entry, if the perf map file is not already opened. Returns ``0`` on success, or the same error codes as :c:func:`PyUnstable_PerfMapState_Init` on failure. .. c:function:: void PyUnstable_PerfMapState_Fini(void) Close the perf map file opened by :c:func:`PyUnstable_PerfMapState_Init`. This is called by the runtime itself during interpreter shut-down. In general, there shouldn't be a reason to explicitly call this, except to handle specific scenarios such as forking. .. c:function:: int PyUnstable_CopyPerfMapFile(const char *parent_filename) Open the ``/tmp/perf-$pid.map`` file and append the content of *parent_filename* to it. This function is available on all platforms but only generates output on platforms that support perf maps (currently only Linux). On other platforms, it does nothing. .. versionadded:: 3.13 .. c:function:: int PyUnstable_PerfTrampoline_CompileCode(PyCodeObject *code) Compile the given code object using the current perf trampoline. The "current" trampoline is the one set by the runtime or the most recent :c:func:`PyUnstable_PerfTrampoline_SetPersistAfterFork` call. If no trampoline is set, falls back to normal compilation (no perf map entry). :param code: The code object to compile. :return: 0 on success, -1 on failure. .. versionadded:: 3.13 .. c:function:: int PyUnstable_PerfTrampoline_SetPersistAfterFork(int enable) Set whether the perf trampoline should persist after a fork. * If ``enable`` is true (non-zero): perf map file remains open/valid post-fork. Child process inherits all existing perf map entries. * If ``enable`` is false (zero): perf map closes post-fork. Child process gets empty perf map. Default: false (clears on fork). :param enable: 1 to enable, 0 to disable. :return: 0 on success, -1 on failure. .. versionadded:: 3.13 --- ### Doc C Api Picklebuffer (`Doc/c-api/picklebuffer.rst`) .. highlight:: c .. _picklebuffer-objects: .. index:: pair: object; PickleBuffer Pickle buffer objects --------------------- .. versionadded:: 3.8 A :class:`pickle.PickleBuffer` object wraps a :ref:`buffer-providing object ` for out-of-band data transfer with the :mod:`pickle` module. .. c:var:: PyTypeObject PyPickleBuffer_Type This instance of :c:type:`PyTypeObject` represents the Python pickle buffer type. This is the same object as :class:`pickle.PickleBuffer` in the Python layer. .. c:function:: int PyPickleBuffer_Check(PyObject *op) Return true if *op* is a pickle buffer instance. This function always succeeds. .. c:function:: PyObject *PyPickleBuffer_FromObject(PyObject *obj) Create a pickle buffer from the object *obj*. This function will fail if *obj* doesn't support the :ref:`buffer protocol `. On success, return a new pickle buffer instance. On failure, set an exception and return ``NULL``. Analogous to calling :class:`pickle.PickleBuffer` with *obj* in Python. .. c:function:: const Py_buffer *PyPickleBuffer_GetBuffer(PyObject *picklebuf) Get a pointer to the underlying :c:type:`Py_buffer` that the pickle buffer wraps. The returned pointer is valid as long as *picklebuf* is alive and has not been released. The caller must not modify or free the returned :c:type:`Py_buffer`. If the pickle buffer has been released, raise :exc:`ValueError`. On success, return a pointer to the buffer view. On failure, set an exception and return ``NULL``. .. c:function:: int PyPickleBuffer_Release(PyObject *picklebuf) Release the underlying buffer held by the pickle buffer. Return ``0`` on success. On failure, set an exception and return ``-1``. Analogous to calling :meth:`pickle.PickleBuffer.release` in Python. --- ### Doc C Api Profiling (`Doc/c-api/profiling.rst`) .. highlight:: c .. _profiling: Profiling and tracing ===================== The Python interpreter provides some low-level support for attaching profiling and execution tracing facilities. These are used for profiling, debugging, and coverage analysis tools. This C interface allows the profiling or tracing code to avoid the overhead of calling through Python-level callable objects, making a direct C function call instead. The essential attributes of the facility have not changed; the interface allows trace functions to be installed per-thread, and the basic events reported to the trace function are the same as had been reported to the Python-level trace functions in previous versions. .. c:type:: int (*Py_tracefunc)(PyObject *obj, PyFrameObject *frame, int what, PyObject *arg) The type of the trace function registered using :c:func:`PyEval_SetProfile` and :c:func:`PyEval_SetTrace`. The first parameter is the object passed to the registration function as *obj*, *frame* is the frame object to which the event pertains, *what* is one of the constants :c:data:`PyTrace_CALL`, :c:data:`PyTrace_EXCEPTION`, :c:data:`PyTrace_LINE`, :c:data:`PyTrace_RETURN`, :c:data:`PyTrace_C_CALL`, :c:data:`PyTrace_C_EXCEPTION`, :c:data:`PyTrace_C_RETURN`, or :c:data:`PyTrace_OPCODE`, and *arg* depends on the value of *what*: +-------------------------------+----------------------------------------+ | Value of *what* | Meaning of *arg* | +===============================+========================================+ | :c:data:`PyTrace_CALL` | Always :c:data:`Py_None`. | +-------------------------------+----------------------------------------+ | :c:data:`PyTrace_EXCEPTION` | Exception information as returned by | | | :func:`sys.exc_info`. | +-------------------------------+----------------------------------------+ | :c:data:`PyTrace_LINE` | Always :c:data:`Py_None`. | +-------------------------------+----------------------------------------+ | :c:data:`PyTrace_RETURN` | Value being returned to the caller, | | | or ``NULL`` if caused by an exception. | +-------------------------------+----------------------------------------+ | :c:data:`PyTrace_C_CALL` | Function object being called. | +-------------------------------+----------------------------------------+ | :c:data:`PyTrace_C_EXCEPTION` | Function object being called. | +-------------------------------+----------------------------------------+ | :c:data:`PyTrace_C_RETURN` | Function object being called. | +-------------------------------+----------------------------------------+ | :c:data:`PyTrace_OPCODE` | Always :c:data:`Py_None`. | +-------------------------------+----------------------------------------+ .. c:var:: int PyTrace_CALL The value of the *what* parameter to a :c:type:`Py_tracefunc` function when a new call to a function or method is being reported, or a new entry into a generator. Note that the creation of the iterator for a generator function is not reported as there is no control transfer to the Python bytecode in the corresponding frame. .. c:var:: int PyTrace_EXCEPTION The value of the *what* parameter to a :c:type:`Py_tracefunc` function when an exception has been raised. The callback function is called with this value for *what* when after any bytecode is processed after which the exception becomes set within the frame being executed. The effect of this is that as exception propagation causes the Python stack to unwind, the callback is called upon return to each frame as the exception propagates. Only trace functions receive these events; they are not needed by the profiler. .. c:var:: int PyTrace_LINE The value passed as the *what* parameter to a :c:type:`Py_tracefunc` function (but not a profiling function) when a line-number event is being reported. It may be disabled for a frame by setting :attr:`~frame.f_trace_lines` to *0* on that frame. .. c:var:: int PyTrace_RETURN The value for the *what* parameter to :c:type:`Py_tracefunc` functions when a call is about to return. .. c:var:: int PyTrace_C_CALL The value for the *what* parameter to :c:type:`Py_tracefunc` functions when a C function is about to be called. .. c:var:: int PyTrace_C_EXCEPTION The value for the *what* parameter to :c:type:`Py_tracefunc` functions when a C function has raised an exception. .. c:var:: int PyTrace_C_RETURN The value for the *what* parameter to :c:type:`Py_tracefunc` functions when a C function has returned. .. c:var:: int PyTrace_OPCODE The value for the *what* parameter to :c:type:`Py_tracefunc` functions (but not profiling functions) when a new opcode is about to be executed. This event is not emitted by default: it must be explicitly requested by setting :attr:`~frame.f_trace_opcodes` to *1* on the frame. .. c:function:: void PyEval_SetProfile(Py_tracefunc func, PyObject *obj) Set the profiler function to *func*. The *obj* parameter is passed to the function as its first parameter, and may be any Python object, or ``NULL``. If the profile function needs to maintain state, using a different value for *obj* for each thread provides a convenient and thread-safe place to store it. The profile function is called for all monitored events except :c:data:`PyTrace_LINE` :c:data:`PyTrace_OPCODE` and :c:data:`PyTrace_EXCEPTION`. See also the :func:`sys.setprofile` function. The caller must have an :term:`attached thread state`. .. c:function:: void PyEval_SetProfileAllThreads(Py_tracefunc func, PyObject *obj) Like :c:func:`PyEval_SetProfile` but sets the profile function in all running threads belonging to the current interpreter instead of the setting it only on the current thread. The caller must have an :term:`attached thread state`. As :c:func:`PyEval_SetProfile`, this function ignores any exceptions raised while setting the profile functions in all threads. .. versionadded:: 3.12 .. c:function:: void PyEval_SetTrace(Py_tracefunc func, PyObject *obj) Set the tracing function to *func*. This is similar to :c:func:`PyEval_SetProfile`, except the tracing function does receive line-number events and per-opcode events, but does not receive any event related to C function objects being called. Any trace function registered using :c:func:`PyEval_SetTrace` will not receive :c:data:`PyTrace_C_CALL`, :c:data:`PyTrace_C_EXCEPTION` or :c:data:`PyTrace_C_RETURN` as a value for the *what* parameter. See also the :func:`sys.settrace` function. The caller must have an :term:`attached thread state`. .. c:function:: void PyEval_SetTraceAllThreads(Py_tracefunc func, PyObject *obj) Like :c:func:`PyEval_SetTrace` but sets the tracing function in all running threads belonging to the current interpreter instead of the setting it only on the current thread. The caller must have an :term:`attached thread state`. As :c:func:`PyEval_SetTrace`, this function ignores any exceptions raised while setting the trace functions in all threads. .. versionadded:: 3.12 Reference tracing ================= .. versionadded:: 3.13 .. c:type:: int (*PyRefTracer)(PyObject *, int event, void* data) The type of the trace function registered using :c:func:`PyRefTracer_SetTracer`. The first parameter is a Python object that has been just created (when **event** is set to :c:data:`PyRefTracer_CREATE`) or about to be destroyed (when **event** is set to :c:data:`PyRefTracer_DESTROY`). The **data** argument is the opaque pointer that was provided when :c:func:`PyRefTracer_SetTracer` was called. If a new tracing function is registered replacing the current one, a call to the trace function will be made with the object set to **NULL** and **event** set to :c:data:`PyRefTracer_TRACKER_REMOVED`. This will happen just before the new function is registered. .. versionadded:: 3.13 .. c:var:: int PyRefTracer_CREATE The value for the *event* parameter to :c:type:`PyRefTracer` functions when a Python object has been created. .. c:var:: int PyRefTracer_DESTROY The value for the *event* parameter to :c:type:`PyRefTracer` functions when a Python object has been destroyed. .. c:var:: int PyRefTracer_TRACKER_REMOVED The value for the *event* parameter to :c:type:`PyRefTracer` functions when the current tracer is about to be replaced by a new one. .. versionadded:: 3.14 .. c:function:: int PyRefTracer_SetTracer(PyRefTracer tracer, void *data) Register a reference tracer function. The function will be called when a new Python object has been created or when an object is going to be destroyed. If **data** is provided it must be an opaque pointer that will be provided when the tracer function is called. Return ``0`` on success. Set an exception and return ``-1`` on error. Note that tracer functions **must not** create Python objects inside or otherwise the call will be re-entrant. The tracer also **must not** clear any existing exception or set an exception. A :term:`thread state` will be active every time the tracer function is called. There must be an :term:`attached thread state` when calling this function. If another tracer function was already registered, the old function will be called with **event** set to :c:data:`PyRefTracer_TRACKER_REMOVED` just before the new function is registered. .. versionadded:: 3.13 .. c:function:: PyRefTracer PyRefTracer_GetTracer(void** data) Get the registered reference tracer function and the value of the opaque data pointer that was registered when :c:func:`PyRefTracer_SetTracer` was called. If no tracer was registered this function will return NULL and will set the **data** pointer to NULL. There must be an :term:`attached thread state` when calling this function. .. versionadded:: 3.13 --- ### Doc C Api Refcounting (`Doc/c-api/refcounting.rst`) .. highlight:: c .. _countingrefs: ****************** Reference Counting ****************** The functions and macros in this section are used for managing reference counts of Python objects. .. c:function:: Py_ssize_t Py_REFCNT(PyObject *o) Get the reference count of the Python object *o*. Note that the returned value may not actually reflect how many references to the object are actually held. For example, some objects are :term:`immortal` and have a very high refcount that does not reflect the actual number of references. Consequently, do not rely on the returned value to be accurate, other than a value of 0 or 1. Use the :c:func:`Py_SET_REFCNT()` function to set an object reference count. .. note:: On :term:`free-threaded builds ` of Python, returning 1 isn't sufficient to determine if it's safe to treat *o* as having no access by other threads. Use :c:func:`PyUnstable_Object_IsUniquelyReferenced` for that instead. See also the function :c:func:`PyUnstable_Object_IsUniqueReferencedTemporary()`. .. versionchanged:: 3.10 :c:func:`Py_REFCNT()` is changed to the inline static function. .. versionchanged:: 3.11 The parameter type is no longer :c:expr:`const PyObject*`. .. c:function:: void Py_SET_REFCNT(PyObject *o, Py_ssize_t refcnt) Set the object *o* reference counter to *refcnt*. On :ref:`Python build with Free Threading `, if *refcnt* is larger than ``UINT32_MAX``, the object is made :term:`immortal`. This function has no effect on :term:`immortal` objects. .. versionadded:: 3.9 .. versionchanged:: 3.12 Immortal objects are not modified. .. c:function:: void Py_INCREF(PyObject *o) Indicate taking a new :term:`strong reference` to object *o*, indicating it is in use and should not be destroyed. This function has no effect on :term:`immortal` objects. This function is usually used to convert a :term:`borrowed reference` to a :term:`strong reference` in-place. The :c:func:`Py_NewRef` function can be used to create a new :term:`strong reference`. When done using the object, release is by calling :c:func:`Py_DECREF`. The object must not be ``NULL``; if you aren't sure that it isn't ``NULL``, use :c:func:`Py_XINCREF`. Do not expect this function to actually modify *o* in any way. For at least :pep:`some objects <0683>`, this function has no effect. .. versionchanged:: 3.12 Immortal objects are not modified. .. c:function:: void Py_XINCREF(PyObject *o) Similar to :c:func:`Py_INCREF`, but the object *o* can be ``NULL``, in which case this has no effect. See also :c:func:`Py_XNewRef`. .. c:function:: PyObject* Py_NewRef(PyObject *o) Create a new :term:`strong reference` to an object: call :c:func:`Py_INCREF` on *o* and return the object *o*. When the :term:`strong reference` is no longer needed, :c:func:`Py_DECREF` should be called on it to release the reference. The object *o* must not be ``NULL``; use :c:func:`Py_XNewRef` if *o* can be ``NULL``. For example:: Py_INCREF(obj); self->attr = obj; can be written as:: self->attr = Py_NewRef(obj); See also :c:func:`Py_INCREF`. .. versionadded:: 3.10 .. c:function:: PyObject* Py_XNewRef(PyObject *o) Similar to :c:func:`Py_NewRef`, but the object *o* can be NULL. If the object *o* is ``NULL``, the function just returns ``NULL``. .. versionadded:: 3.10 .. c:function:: void Py_DECREF(PyObject *o) Release a :term:`strong reference` to object *o*, indicating the reference is no longer used. This function has no effect on :term:`immortal` objects. Once the last :term:`strong reference` is released (i.e. the object's reference count reaches 0), the object's type's deallocation function (which must not be ``NULL``) is invoked. This function is usually used to delete a :term:`strong reference` before exiting its scope. The object must not be ``NULL``; if you aren't sure that it isn't ``NULL``, use :c:func:`Py_XDECREF`. Do not expect this function to actually modify *o* in any way. For at least :pep:`some objects <683>`, this function has no effect. .. warning:: The deallocation function can cause arbitrary Python code to be invoked (e.g. when a class instance with a :meth:`~object.__del__` method is deallocated). While exceptions in such code are not propagated, the executed code has free access to all Python global variables. This means that any object that is reachable from a global variable should be in a consistent state before :c:func:`Py_DECREF` is invoked. For example, code to delete an object from a list should copy a reference to the deleted object in a temporary variable, update the list data structure, and then call :c:func:`Py_DECREF` for the temporary variable. .. versionchanged:: 3.12 Immortal objects are not modified. .. c:function:: void Py_XDECREF(PyObject *o) Similar to :c:func:`Py_DECREF`, but the object *o* can be ``NULL``, in which case this has no effect. The same warning from :c:func:`Py_DECREF` applies here as well. .. c:function:: void Py_CLEAR(PyObject *o) Release a :term:`strong reference` for object *o*. The object may be ``NULL``, in which case the macro has no effect; otherwise the effect is the same as for :c:func:`Py_DECREF`, except that the argument is also set to ``NULL``. The warning for :c:func:`Py_DECREF` does not apply with respect to the object passed because the macro carefully uses a temporary variable and sets the argument to ``NULL`` before releasing the reference. It is a good idea to use this macro whenever releasing a reference to an object that might be traversed during garbage collection. .. versionchanged:: 3.12 The macro argument is now only evaluated once. If the argument has side effects, these are no longer duplicated. .. c:function:: void Py_IncRef(PyObject *o) Indicate taking a new :term:`strong reference` to object *o*. A function version of :c:func:`Py_XINCREF`. It can be used for runtime dynamic embedding of Python. .. c:function:: void Py_DecRef(PyObject *o) Release a :term:`strong reference` to object *o*. A function version of :c:func:`Py_XDECREF`. It can be used for runtime dynamic embedding of Python. .. c:macro:: Py_SETREF(dst, src) Macro safely releasing a :term:`strong reference` to object *dst* and setting *dst* to *src*. As in case of :c:func:`Py_CLEAR`, "the obvious" code can be deadly:: Py_DECREF(dst); dst = src; The safe way is:: Py_SETREF(dst, src); That arranges to set *dst* to *src* *before* releasing the reference to the old value of *dst*, so that any code triggered as a side-effect of *dst* getting torn down no longer believes *dst* points to a valid object. .. versionadded:: 3.6 .. versionchanged:: 3.12 The macro arguments are now only evaluated once. If an argument has side effects, these are no longer duplicated. .. c:macro:: Py_XSETREF(dst, src) Variant of :c:macro:`Py_SETREF` macro that uses :c:func:`Py_XDECREF` instead of :c:func:`Py_DECREF`. .. versionadded:: 3.6 .. versionchanged:: 3.12 The macro arguments are now only evaluated once. If an argument has side effects, these are no longer duplicated. --- ### Doc C Api Reflection (`Doc/c-api/reflection.rst`) .. highlight:: c .. _reflection: Reflection ========== .. c:function:: PyObject* PyEval_GetBuiltins(void) .. deprecated:: 3.13 Use :c:func:`PyEval_GetFrameBuiltins` instead. Return a dictionary of the builtins in the current execution frame, or the interpreter of the thread state if no frame is currently executing. .. c:function:: PyObject* PyEval_GetLocals(void) .. deprecated:: 3.13 Use either :c:func:`PyEval_GetFrameLocals` to obtain the same behaviour as calling :func:`locals` in Python code, or else call :c:func:`PyFrame_GetLocals` on the result of :c:func:`PyEval_GetFrame` to access the :attr:`~frame.f_locals` attribute of the currently executing frame. Return a mapping providing access to the local variables in the current execution frame, or ``NULL`` if no frame is currently executing. Refer to :func:`locals` for details of the mapping returned at different scopes. As this function returns a :term:`borrowed reference`, the dictionary returned for :term:`optimized scopes ` is cached on the frame object and will remain alive as long as the frame object does. Unlike :c:func:`PyEval_GetFrameLocals` and :func:`locals`, subsequent calls to this function in the same frame will update the contents of the cached dictionary to reflect changes in the state of the local variables rather than returning a new snapshot. .. versionchanged:: 3.13 As part of :pep:`667`, :c:func:`PyFrame_GetLocals`, :func:`locals`, and :attr:`FrameType.f_locals ` no longer make use of the shared cache dictionary. Refer to the :ref:`What's New entry ` for additional details. .. c:function:: PyObject* PyEval_GetGlobals(void) .. deprecated:: 3.13 Use :c:func:`PyEval_GetFrameGlobals` instead. Return a dictionary of the global variables in the current execution frame, or ``NULL`` if no frame is currently executing. .. c:function:: PyFrameObject* PyEval_GetFrame(void) Return the :term:`attached thread state`'s frame, which is ``NULL`` if no frame is currently executing. See also :c:func:`PyThreadState_GetFrame`. .. c:function:: PyObject* PyEval_GetFrameBuiltins(void) Return a dictionary of the builtins in the current execution frame, or the interpreter of the thread state if no frame is currently executing. .. versionadded:: 3.13 .. c:function:: PyObject* PyEval_GetFrameLocals(void) Return a dictionary of the local variables in the current execution frame, or ``NULL`` if no frame is currently executing. Equivalent to calling :func:`locals` in Python code. To access :attr:`~frame.f_locals` on the current frame without making an independent snapshot in :term:`optimized scopes `, call :c:func:`PyFrame_GetLocals` on the result of :c:func:`PyEval_GetFrame`. .. versionadded:: 3.13 .. c:function:: PyObject* PyEval_GetFrameGlobals(void) Return a dictionary of the global variables in the current execution frame, or ``NULL`` if no frame is currently executing. Equivalent to calling :func:`globals` in Python code. .. versionadded:: 3.13 .. c:function:: const char* PyEval_GetFuncName(PyObject *func) Return the name of *func* if it is a function, class or instance object, else the name of *func*\s type. .. c:function:: const char* PyEval_GetFuncDesc(PyObject *func) Return a description string, depending on the type of *func*. Return values include "()" for functions and methods, " constructor", " instance", and " object". Concatenated with the result of :c:func:`PyEval_GetFuncName`, the result will be a description of *func*. --- ### Doc C Api Sentinel (`Doc/c-api/sentinel.rst`) .. highlight:: c .. _sentinelobjects: Sentinel objects ---------------- .. c:var:: PyTypeObject PySentinel_Type This instance of :c:type:`PyTypeObject` represents the Python :class:`sentinel` type. This is the same object as :class:`sentinel`. .. versionadded:: 3.15 .. c:function:: int PySentinel_Check(PyObject *o) Return true if *o* is a :class:`sentinel` object or a subtype. The :class:`sentinel` type does not currently allow subclasses, so this check is exact. Future Python versions may choose to allow subtyping. This function always succeeds. .. versionadded:: 3.15 .. c:function:: int PySentinel_CheckExact(PyObject *o) Return true if *o* is a :class:`sentinel` object, but not a subtype. The :class:`sentinel` type does not currently allow subclasses. Future Python versions may choose to allow subtyping. This function always succeeds. .. versionadded:: 3.15 .. c:function:: PyObject* PySentinel_New(const char *name, const char *module_name, const char *repr) Return a new :class:`sentinel` object with :attr:`~sentinel.__name__` set to *name* and :attr:`~sentinel.__module__` set to *module_name*. *name* must not be ``NULL``. If *module_name* is ``NULL``, :attr:`~sentinel.__module__` is set to ``None``. If *repr* is ``NULL``, ``repr()`` returns :attr:`~sentinel.__name__`. Return ``NULL`` with an exception set on failure. For pickling to work, *module_name* must be the name of an importable module, and the sentinel must be accessible from that module under a path matching *name*. Pickle treats *name* as a global variable name in *module_name* (see :meth:`object.__reduce__`). .. versionadded:: 3.15 --- ### Doc C Api Sequence (`Doc/c-api/sequence.rst`) .. highlight:: c .. _sequence: Sequence Protocol ================= .. c:function:: int PySequence_Check(PyObject *o) Return ``1`` if the object provides the sequence protocol, and ``0`` otherwise. Note that it returns ``1`` for Python classes with a :meth:`~object.__getitem__` method, unless they are :class:`dict` subclasses, since in general it is impossible to determine what type of keys the class supports. This function always succeeds. .. c:function:: Py_ssize_t PySequence_Size(PyObject *o) Py_ssize_t PySequence_Length(PyObject *o) .. index:: pair: built-in function; len Returns the number of objects in sequence *o* on success, and ``-1`` on failure. This is equivalent to the Python expression ``len(o)``. .. c:function:: PyObject* PySequence_Concat(PyObject *o1, PyObject *o2) Return the concatenation of *o1* and *o2* on success, and ``NULL`` on failure. This is the equivalent of the Python expression ``o1 + o2``. .. c:function:: PyObject* PySequence_Repeat(PyObject *o, Py_ssize_t count) Return the result of repeating sequence object *o* *count* times, or ``NULL`` on failure. This is the equivalent of the Python expression ``o * count``. .. c:function:: PyObject* PySequence_InPlaceConcat(PyObject *o1, PyObject *o2) Return the concatenation of *o1* and *o2* on success, and ``NULL`` on failure. The operation is done *in-place* when *o1* supports it. This is the equivalent of the Python expression ``o1 += o2``. .. c:function:: PyObject* PySequence_InPlaceRepeat(PyObject *o, Py_ssize_t count) Return the result of repeating sequence object *o* *count* times, or ``NULL`` on failure. The operation is done *in-place* when *o* supports it. This is the equivalent of the Python expression ``o *= count``. .. c:function:: PyObject* PySequence_GetItem(PyObject *o, Py_ssize_t i) Return the *i*\ th element of *o*, or ``NULL`` on failure. This is the equivalent of the Python expression ``o[i]``. .. c:function:: PyObject* PySequence_GetSlice(PyObject *o, Py_ssize_t i1, Py_ssize_t i2) Return the slice of sequence object *o* between *i1* and *i2*, or ``NULL`` on failure. This is the equivalent of the Python expression ``o[i1:i2]``. .. c:function:: int PySequence_SetItem(PyObject *o, Py_ssize_t i, PyObject *v) Assign object *v* to the *i*\ th element of *o*. Raise an exception and return ``-1`` on failure; return ``0`` on success. This is the equivalent of the Python statement ``o[i] = v``. This function *does not* ":term:`steal`" a reference to *v*. If *v* is ``NULL``, the element is deleted, but this feature is deprecated in favour of using :c:func:`PySequence_DelItem`. .. c:function:: int PySequence_DelItem(PyObject *o, Py_ssize_t i) Delete the *i*\ th element of object *o*. Returns ``-1`` on failure. This is the equivalent of the Python statement ``del o[i]``. .. c:function:: int PySequence_SetSlice(PyObject *o, Py_ssize_t i1, Py_ssize_t i2, PyObject *v) Assign the sequence object *v* to the slice in sequence object *o* from *i1* to *i2*. This is the equivalent of the Python statement ``o[i1:i2] = v``. .. c:function:: int PySequence_DelSlice(PyObject *o, Py_ssize_t i1, Py_ssize_t i2) Delete the slice in sequence object *o* from *i1* to *i2*. Returns ``-1`` on failure. This is the equivalent of the Python statement ``del o[i1:i2]``. .. c:function:: Py_ssize_t PySequence_Count(PyObject *o, PyObject *value) Return the number of occurrences of *value* in *o*, that is, return the number of keys for which ``o[key] == value``. On failure, return ``-1``. This is equivalent to the Python expression ``o.count(value)``. .. c:function:: int PySequence_Contains(PyObject *o, PyObject *value) Determine if *o* contains *value*. If an item in *o* is equal to *value*, return ``1``, otherwise return ``0``. On error, return ``-1``. This is equivalent to the Python expression ``value in o``. .. c:function:: int PySequence_In(PyObject *o, PyObject *value) Alias for :c:func:`PySequence_Contains`. .. soft-deprecated:: 3.14 The function should no longer be used to write new code. .. c:function:: Py_ssize_t PySequence_Index(PyObject *o, PyObject *value) Return the first index *i* for which ``o[i] == value``. On error, return ``-1``. This is equivalent to the Python expression ``o.index(value)``. .. c:function:: PyObject* PySequence_List(PyObject *o) Return a list object with the same contents as the sequence or iterable *o*, or ``NULL`` on failure. The returned list is guaranteed to be new. This is equivalent to the Python expression ``list(o)``. .. c:function:: PyObject* PySequence_Tuple(PyObject *o) .. index:: pair: built-in function; tuple Return a tuple object with the same contents as the sequence or iterable *o*, or ``NULL`` on failure. If *o* is a tuple, a new reference will be returned, otherwise a tuple will be constructed with the appropriate contents. This is equivalent to the Python expression ``tuple(o)``. .. c:function:: PyObject* PySequence_Fast(PyObject *o, const char *m) Return the sequence or iterable *o* as an object usable by the other ``PySequence_Fast*`` family of functions. If the object is not a sequence or iterable, raises :exc:`TypeError` with *m* as the message text. Returns ``NULL`` on failure. The ``PySequence_Fast*`` functions are thus named because they assume *o* is a :c:type:`PyTupleObject` or a :c:type:`PyListObject` and access the data fields of *o* directly. As a CPython implementation detail, if *o* is already a sequence or list, it will be returned. .. c:function:: Py_ssize_t PySequence_Fast_GET_SIZE(PyObject *o) Returns the length of *o*, assuming that *o* was returned by :c:func:`PySequence_Fast` and that *o* is not ``NULL``. The size can also be retrieved by calling :c:func:`PySequence_Size` on *o*, but :c:func:`PySequence_Fast_GET_SIZE` is faster because it can assume *o* is a list or tuple. .. c:function:: PyObject* PySequence_Fast_GET_ITEM(PyObject *o, Py_ssize_t i) Return the *i*\ th element of *o*, assuming that *o* was returned by :c:func:`PySequence_Fast`, *o* is not ``NULL``, and that *i* is within bounds. .. c:function:: PyObject** PySequence_Fast_ITEMS(PyObject *o) Return the underlying array of PyObject pointers. Assumes that *o* was returned by :c:func:`PySequence_Fast` and *o* is not ``NULL``. Note, if a list gets resized, the reallocation may relocate the items array. So, only use the underlying array pointer in contexts where the sequence cannot change. .. c:function:: PyObject* PySequence_ITEM(PyObject *o, Py_ssize_t i) Return the *i*\ th element of *o* or ``NULL`` on failure. Faster form of :c:func:`PySequence_GetItem` but without checking that :c:func:`PySequence_Check` on *o* is true and without adjustment for negative indices. --- ### Doc C Api Set (`Doc/c-api/set.rst`) .. highlight:: c .. _setobjects: Set Objects ----------- .. index:: pair: object; set pair: object; frozenset This section details the public API for :class:`set` and :class:`frozenset` objects. Any functionality not listed below is best accessed using either the abstract object protocol (including :c:func:`PyObject_CallMethod`, :c:func:`PyObject_RichCompareBool`, :c:func:`PyObject_Hash`, :c:func:`PyObject_Repr`, :c:func:`PyObject_IsTrue`, :c:func:`PyObject_Print`, and :c:func:`PyObject_GetIter`) or the abstract number protocol (including :c:func:`PyNumber_And`, :c:func:`PyNumber_Subtract`, :c:func:`PyNumber_Or`, :c:func:`PyNumber_Xor`, :c:func:`PyNumber_InPlaceAnd`, :c:func:`PyNumber_InPlaceSubtract`, :c:func:`PyNumber_InPlaceOr`, and :c:func:`PyNumber_InPlaceXor`). .. c:type:: PySetObject This subtype of :c:type:`PyObject` is used to hold the internal data for both :class:`set` and :class:`frozenset` objects. It is like a :c:type:`PyDictObject` in that it is a fixed size for small sets (much like tuple storage) and will point to a separate, variable sized block of memory for medium and large sized sets (much like list storage). None of the fields of this structure should be considered public and all are subject to change. All access should be done through the documented API rather than by manipulating the values in the structure. .. c:var:: PyTypeObject PySet_Type This is an instance of :c:type:`PyTypeObject` representing the Python :class:`set` type. .. c:var:: PyTypeObject PyFrozenSet_Type This is an instance of :c:type:`PyTypeObject` representing the Python :class:`frozenset` type. The following type check macros work on pointers to any Python object. Likewise, the constructor functions work with any iterable Python object. .. c:function:: int PySet_Check(PyObject *p) Return true if *p* is a :class:`set` object or an instance of a subtype. This function always succeeds. .. c:function:: int PyFrozenSet_Check(PyObject *p) Return true if *p* is a :class:`frozenset` object or an instance of a subtype. This function always succeeds. .. c:function:: int PyAnySet_Check(PyObject *p) Return true if *p* is a :class:`set` object, a :class:`frozenset` object, or an instance of a subtype. This function always succeeds. .. c:function:: int PySet_CheckExact(PyObject *p) Return true if *p* is a :class:`set` object but not an instance of a subtype. This function always succeeds. .. versionadded:: 3.10 .. c:function:: int PyAnySet_CheckExact(PyObject *p) Return true if *p* is a :class:`set` object or a :class:`frozenset` object but not an instance of a subtype. This function always succeeds. .. c:function:: int PyFrozenSet_CheckExact(PyObject *p) Return true if *p* is a :class:`frozenset` object but not an instance of a subtype. This function always succeeds. .. c:function:: PyObject* PySet_New(PyObject *iterable) Return a new :class:`set` containing objects returned by the *iterable*. The *iterable* may be ``NULL`` to create a new empty set. Return the new set on success or ``NULL`` on failure. Raise :exc:`TypeError` if *iterable* is not actually iterable. The constructor is also useful for copying a set (``c=set(s)``). .. note:: The operation is atomic on :term:`free threading ` when *iterable* is a :class:`set`, :class:`frozenset`, :class:`dict` or :class:`frozendict`. .. c:function:: PyObject* PyFrozenSet_New(PyObject *iterable) Return a new :class:`frozenset` containing objects returned by the *iterable*. The *iterable* may be ``NULL`` to create a new empty frozenset. Return the new set on success or ``NULL`` on failure. Raise :exc:`TypeError` if *iterable* is not actually iterable. .. note:: The operation is atomic on :term:`free threading ` when *iterable* is a :class:`set`, :class:`frozenset`, :class:`dict` or :class:`frozendict`. The following functions and macros are available for instances of :class:`set` or :class:`frozenset` or instances of their subtypes. .. c:function:: Py_ssize_t PySet_Size(PyObject *anyset) .. index:: pair: built-in function; len Return the length of a :class:`set` or :class:`frozenset` object. Equivalent to ``len(anyset)``. Raises a :exc:`SystemError` if *anyset* is not a :class:`set`, :class:`frozenset`, or an instance of a subtype. .. c:function:: Py_ssize_t PySet_GET_SIZE(PyObject *anyset) Macro form of :c:func:`PySet_Size` without error checking. .. c:function:: int PySet_Contains(PyObject *anyset, PyObject *key) Return ``1`` if found, ``0`` if not found, and ``-1`` if an error is encountered. Unlike the Python :meth:`~object.__contains__` method, this function does not automatically convert unhashable sets into temporary frozensets. Raise a :exc:`TypeError` if the *key* is unhashable. Raise :exc:`SystemError` if *anyset* is not a :class:`set`, :class:`frozenset`, or an instance of a subtype. .. note:: The operation is atomic on :term:`free threading ` when *key* is :class:`str`, :class:`int`, :class:`float`, :class:`bool` or :class:`bytes`. .. c:function:: int PySet_Add(PyObject *set, PyObject *key) Add *key* to a :class:`set` instance. Also works with :class:`frozenset` instances (like :c:func:`PyTuple_SetItem` it can be used to fill in the values of brand new frozensets before they are exposed to other code). Return ``0`` on success or ``-1`` on failure. Raise a :exc:`TypeError` if the *key* is unhashable. Raise a :exc:`MemoryError` if there is no room to grow. Raise a :exc:`SystemError` if *set* is not an instance of :class:`set` or its subtype. .. note:: The operation is atomic on :term:`free threading ` when *key* is :class:`str`, :class:`int`, :class:`float`, :class:`bool` or :class:`bytes`. The following functions are available for instances of :class:`set` or its subtypes but not for instances of :class:`frozenset` or its subtypes. .. c:function:: int PySet_Discard(PyObject *set, PyObject *key) Return ``1`` if found and removed, ``0`` if not found (no action taken), and ``-1`` if an error is encountered. Does not raise :exc:`KeyError` for missing keys. Raise a :exc:`TypeError` if the *key* is unhashable. Unlike the Python :meth:`~set.discard` method, this function does not automatically convert unhashable sets into temporary frozensets. Raise :exc:`SystemError` if *set* is not an instance of :class:`set` or its subtype. .. note:: The operation is atomic on :term:`free threading ` when *key* is :class:`str`, :class:`int`, :class:`float`, :class:`bool` or :class:`bytes`. .. c:function:: PyObject* PySet_Pop(PyObject *set) Return a new reference to an arbitrary object in the *set*, and removes the object from the *set*. Return ``NULL`` on failure. Raise :exc:`KeyError` if the set is empty. Raise a :exc:`SystemError` if *set* is not an instance of :class:`set` or its subtype. .. c:function:: int PySet_Clear(PyObject *set) Empty an existing set of all elements. Return ``0`` on success. Return ``-1`` and raise :exc:`SystemError` if *set* is not an instance of :class:`set` or its subtype. .. note:: In the :term:`free-threaded build`, the set is emptied before its entries are cleared, so other threads will observe an empty set rather than intermediate states. Deprecated API ^^^^^^^^^^^^^^ .. c:macro:: PySet_MINSIZE A constant representing the size of an internal preallocated table inside :c:type:`PySetObject` instances. This is documented solely for completeness, as there are no guarantees that a given version of CPython uses preallocated tables with a fixed size. In code that does not deal with unstable set internals, :c:macro:`!PySet_MINSIZE` can be replaced with a small constant like ``8``. If looking for the size of a set, use :c:func:`PySet_Size` instead. .. soft-deprecated:: 3.14 --- ### Doc C Api Slice (`Doc/c-api/slice.rst`) .. highlight:: c .. _slice-objects: Slice Objects ------------- .. c:var:: PyTypeObject PySlice_Type The type object for slice objects. This is the same as :class:`slice` in the Python layer. .. c:function:: int PySlice_Check(PyObject *ob) Return true if *ob* is a slice object; *ob* must not be ``NULL``. This function always succeeds. .. c:function:: PyObject* PySlice_New(PyObject *start, PyObject *stop, PyObject *step) Return a new slice object with the given values. The *start*, *stop*, and *step* parameters are used as the values of the slice object attributes of the same names. Any of the values may be ``NULL``, in which case the ``None`` will be used for the corresponding attribute. Return ``NULL`` with an exception set if the new object could not be allocated. .. c:function:: int PySlice_GetIndices(PyObject *slice, Py_ssize_t length, Py_ssize_t *start, Py_ssize_t *stop, Py_ssize_t *step) Retrieve the start, stop and step indices from the slice object *slice*, assuming a sequence of length *length*. Treats indices greater than *length* as errors. Returns ``0`` on success and ``-1`` on error with no exception set (unless one of the indices was not ``None`` and failed to be converted to an integer, in which case ``-1`` is returned with an exception set). You probably do not want to use this function. .. versionchanged:: 3.2 The parameter type for the *slice* parameter was ``PySliceObject*`` before. .. c:function:: int PySlice_GetIndicesEx(PyObject *slice, Py_ssize_t length, Py_ssize_t *start, Py_ssize_t *stop, Py_ssize_t *step, Py_ssize_t *slicelength) Usable replacement for :c:func:`PySlice_GetIndices`. Retrieve the start, stop, and step indices from the slice object *slice* assuming a sequence of length *length*, and store the length of the slice in *slicelength*. Out of bounds indices are clipped in a manner consistent with the handling of normal slices. *length* must not be negative. Return ``0`` on success and ``-1`` on error with an exception set. .. note:: This function is considered not safe for resizable sequences. Its invocation should be replaced by a combination of :c:func:`PySlice_Unpack` and :c:func:`PySlice_AdjustIndices` where :: if (PySlice_GetIndicesEx(slice, length, &start, &stop, &step, &slicelength) < 0) { // return error } is replaced by :: if (PySlice_Unpack(slice, &start, &stop, &step) < 0) { // return error } slicelength = PySlice_AdjustIndices(length, &start, &stop, step); .. versionchanged:: 3.2 The parameter type for the *slice* parameter was ``PySliceObject*`` before. .. versionchanged:: 3.6.1 If ``Py_LIMITED_API`` is not set or set to the value between ``0x03050400`` and ``0x03060000`` (not including) or ``0x03060100`` or higher :c:func:`!PySlice_GetIndicesEx` is implemented as a macro using :c:func:`!PySlice_Unpack` and :c:func:`!PySlice_AdjustIndices`. Arguments *start*, *stop* and *step* are evaluated more than once. .. deprecated:: 3.6.1 If ``Py_LIMITED_API`` is set to the value less than ``0x03050400`` or between ``0x03060000`` and ``0x03060100`` (not including) :c:func:`!PySlice_GetIndicesEx` is a deprecated function. .. c:function:: int PySlice_Unpack(PyObject *slice, Py_ssize_t *start, Py_ssize_t *stop, Py_ssize_t *step) Extract the start, stop and step data members from a slice object as C integers. Silently reduce values larger than ``PY_SSIZE_T_MAX`` to ``PY_SSIZE_T_MAX``, silently boost the start and stop values less than ``PY_SSIZE_T_MIN`` to ``PY_SSIZE_T_MIN``, and silently boost the step values less than ``-PY_SSIZE_T_MAX`` to ``-PY_SSIZE_T_MAX``. Return ``-1`` with an exception set on error, ``0`` on success. .. versionadded:: 3.6.1 .. c:function:: Py_ssize_t PySlice_AdjustIndices(Py_ssize_t length, Py_ssize_t *start, Py_ssize_t *stop, Py_ssize_t step) Adjust start/end slice indices assuming a sequence of the specified length. Out of bounds indices are clipped in a manner consistent with the handling of normal slices. *length* must not be negative. *step* must not be zero and must not be less than ``-PY_SSIZE_T_MAX``, as guaranteed by :c:func:`PySlice_Unpack`. Return the length of the slice. Always successful. Doesn't call Python code. .. versionadded:: 3.6.1 Ellipsis Object ^^^^^^^^^^^^^^^ .. c:var:: PyTypeObject PyEllipsis_Type The type of Python :const:`Ellipsis` object. Same as :class:`types.EllipsisType` in the Python layer. .. c:var:: PyObject *Py_Ellipsis The Python ``Ellipsis`` object. This object has no methods. Like :c:data:`Py_None`, it is an :term:`immortal` singleton object. .. versionchanged:: 3.12 :c:data:`Py_Ellipsis` is immortal. --- ### Doc C Api Slots (`Doc/c-api/slots.rst`) .. highlight:: c .. _capi-slots: Definition slots ================ To define :ref:`module objects ` and :ref:`classes ` using the C API, you may use an array of *slots* -- essentially, key-value pairs that describe features of the object to create. This decouples the data from the structures used at runtime, allowing CPython -- and other Python C API implementations -- to update the structures without breaking backwards compatibility. This section documents slots in general. For object-specific behavior and slot values, see documentation for functions that apply slots: - :c:func:`PyType_FromSlots` for types; - :c:func:`PyModule_FromSlotsAndSpec` and :ref:`extension-export-hook` for modules. When slots are passed to a function that applies them, the function will not modify the slot array, nor any data it points to (recursively). After the function is done, the caller is allowed to modify or deallocate the array and any data it points to (recursively), except data explicitly marked with :c:macro:`PySlot_STATIC`. Except when documented otherwise, multiple slots with the same ID (:c:member:`~PySlot.sl_id`) may not occur in a single slots array. .. versionadded:: 3.15 Slot arrays generalize an earlier way of defining objects: using :c:type:`PyType_Spec` with :c:type:`PyType_Slot` for types, and :c:type:`PyModuleDef` with :c:type:`PyModuleDef_Slot` for modules. The earlier API is :term:`soft deprecated`; there are no plans to remove it. Entries of the slots array use the following structure: .. c:type:: PySlot An entry in a slots array. Defined as: .. code-block:: c typedef struct { uint16_t sl_id; uint16_t sl_flags; uint32_t _reserved; // must be 0 union { void *sl_ptr; void (*sl_func)(void); Py_ssize_t sl_size; int64_t sl_int64; uint64_t sl_uint64; }; } PySlot; .. c:member:: uint16_t sl_id A slot ID, chosen from: - ``Py_slot_*`` values documented in :ref:`pyslot-common-ids` below; - ``Py_mod_*`` values for modules, as documented in :ref:`c_module_slots`; - Values for types, as documented in :ref:`pyslot_type_slot_ids`. A :c:member:`!sl_id` of zero (:c:macro:`Py_slot_end`) marks the end of a slots array. .. c:member:: void *sl_ptr void (*sl_func)(void) Py_ssize_t sl_size int64_t sl_int64 uint64_t sl_uint64 The data for the slot. These members are part of an anonymous union; the member to use depends on which data type is required by the slot ID: data pointer, function pointer, size, signed or unsigned integer, respectively. Except when documented otherwise for a specific slot ID, pointers (that is :c:member:`!sl_ptr` and :c:member:`!sl_func`) may not be NULL. .. c:member:: uint16_t sl_flags Zero or more of the following flags, OR-ed together: .. c:namespace:: NULL .. c:macro:: PySlot_STATIC All data the slot points to is statically allocated and constant. Thus, the interpreter does not need to copy the information. This flag is implied for function pointers. The flag applies even to data the slot points to “indirectly”, except for slots nested via :c:macro:`Py_slot_subslots` which may have their own :c:macro:`!PySlot_STATIC` flags. For example, if applied to a :c:macro:`Py_tp_members` slot that points to an array of :c:type:`PyMemberDef` structures, then the entire array, as well as the name and doc strings in its elements, must be static and constant. .. c:macro:: PySlot_INTPTR The data is stored in ``sl_ptr``; CPython will cast it to the appropriate type. This flag can simplify porting from the older :c:type:`PyType_Slot` and :c:type:`PyModuleDef_Slot` structures. .. c:macro:: PySlot_OPTIONAL If the slot ID is unknown, the interpreter should ignore the slot, rather than fail. For example, if Python 3.16 adds a new feature with a new slot ID,attr the corresponding slot may be marked :c:macro:`!PySlot_OPTIONAL` so that Python 3.15 ignores it. Note that the "optionality" only applies to unknown slot IDs. This flag does not make Python skip invalid values of known slots. .. versionadded:: 3.15 Convenience macros ------------------ .. c:macro:: PySlot_DATA(name, value) PySlot_FUNC(name, value) PySlot_SIZE(name, value) PySlot_INT64(name, value) PySlot_UINT64(name, value) PySlot_STATIC_DATA(name, value) Convenience macros to define :c:type:`!PySlot` structures with :c:member:`~PySlot.sl_id` and a particular union member set. :c:macro:`!PySlot_STATIC_DATA` sets the :c:macro:`PySlot_STATIC` flag; others set no flags. Note that these macros use *designated initializers*, a C language feature that C++ added in the 2020 version of the standard. If your code needs to be compatible with C++11 or older, use :c:macro:`PySlot_PTR` instead. Defined as:: #define PySlot_DATA(NAME, VALUE) \ {.sl_id=NAME, .sl_ptr=(void*)(VALUE)} #define PySlot_FUNC(NAME, VALUE) \ {.sl_id=NAME, .sl_func=(VALUE)} #define PySlot_SIZE(NAME, VALUE) \ {.sl_id=NAME, .sl_size=(VALUE)} #define PySlot_INT64(NAME, VALUE) \ {.sl_id=NAME, .sl_int64=(VALUE)} #define PySlot_UINT64(NAME, VALUE) \ {.sl_id=NAME, .sl_uint64=(VALUE)} #define PySlot_STATIC_DATA(NAME, VALUE) \ {.sl_id=NAME, .sl_flags=PySlot_STATIC, .sl_ptr=(VALUE)} .. versionadded:: 3.15 .. c:macro:: PySlot_END Convenience macro to mark the end of a :c:type:`!PySlot` array. Defined as:: #define PySlot_END {0} .. versionadded:: 3.15 .. c:macro:: PySlot_PTR(name, value) PySlot_PTR_STATIC(name, value) Convenience macros for use in C++11-compatible code. This version of C++ does not allow setting arbitrary union members in literals; instead, these macros set the :c:macro:`PySlot_INTPTR` flag and cast the value to ``(void*)``. Defined as:: #define PySlot_PTR(NAME, VALUE) \ {NAME, PySlot_INTPTR, {0}, {(void*)(VALUE)}} #define PySlot_PTR_STATIC(NAME, VALUE) \ {NAME, PySlot_INTPTR|Py_SLOT_STATIC, {0}, {(void*)(VALUE)}} .. versionadded:: 3.15 .. _pyslot-common-ids: Common slot IDs --------------- The following slot IDs may be used in both type and module definitions. .. c:macro:: Py_slot_end Marks the end of a slots array. Defined as zero. .. versionadded:: 3.15 .. c:macro:: Py_slot_subslots Nested slots array. The value (:c:member:`~PySlot.sl_ptr`) should point to an array of :c:type:`PySlot` structures. The slots in the array (up to but not including the zero-ID terminator) will be treated as if they were inserted if the current slot array, at the point :c:macro:`!Py_slot_subslots` appears. Slot nesting depth is limited to 5 levels. This restriction may be lifted in the future. .. versionadded:: 3.15 .. c:macro:: Py_slot_invalid Reserved; will always be treated as an unknown slot ID. Defined as ``UINT16_MAX`` (``0xFFFF``). When used with the :c:macro:`PySlot_OPTIONAL` flag, defines a slot with no effect. Without the flag, processing a slot with this ID will fail. .. versionadded:: 3.15 --- ### Doc C Api Stable (`Doc/c-api/stable.rst`) .. highlight:: c .. _stable: *********************** C API and ABI Stability *********************** Unless documented otherwise, Python's C API is covered by the Backwards Compatibility Policy, :pep:`387`. Most changes to it are source-compatible (typically by only adding new API). Changing existing API or removing API is only done after a deprecation period or to fix serious issues. CPython's Application Binary Interface (ABI) is forward- and backwards-compatible across a minor release (if these are compiled the same way; see :ref:`stable-abi-platform` below). So, code compiled for Python 3.10.0 will work on 3.10.8 and vice versa, but will need to be compiled separately for 3.9.x and 3.11.x. There are two tiers of C API with different stability expectations: - :ref:`Unstable API `, may change in minor versions without a deprecation period. It is marked by the ``PyUnstable`` prefix in names. - :ref:`Limited API `, is compatible across several minor releases. When :c:macro:`Py_LIMITED_API` is defined, only this subset is exposed from ``Python.h``. These are discussed in more detail below. Names prefixed by an underscore, such as ``_Py_InternalState``, are private API that can change without notice even in patch releases. If you need to use this API, consider reaching out to `CPython developers `_ to discuss adding public API for your use case. .. _unstable-c-api: Unstable C API ============== .. index:: single: PyUnstable Any API named with the ``PyUnstable`` prefix exposes CPython implementation details, and may change in every minor release (e.g. from 3.9 to 3.10) without any deprecation warnings. However, it will not change in a bugfix release (e.g. from 3.10.0 to 3.10.1). It is generally intended for specialized, low-level tools like debuggers. Projects that use this API are expected to follow CPython development and spend extra effort adjusting to changes. .. _stable-abi: .. _stable-application-binary-interface: Stable Application Binary Interfaces ==================================== Python's :dfn:`Stable ABI` allows extensions to be compatible with multiple versions of Python, without recompilation. .. note:: For simplicity, this document talks about *extensions*, but Stable ABI works the same way for all uses of the API – for example, embedding Python. There are two Stable ABIs: - ``abi3``, introduced in Python 3.2, is compatible with **non**-:term:`free-threaded ` builds of CPython. - ``abi3t``, introduced in Python 3.15, is compatible with :term:`free-threaded ` builds of CPython. It has stricter API limitations than ``abi3``. .. versionadded:: 3.15 ``abi3t`` was added in :pep:`803` It is possible for an extension to be compiled for *both* ``abi3`` and ``abi3t`` at the same time; the result will be compatible with both free-threaded and non-free-threaded builds of Python. Currently, this has no downsides compared to compiling for ``abi3t`` only. Each Stable ABI is versioned using the first two numbers of the Python version. For example, Stable ABI 3.14 corresponds to Python 3.14. An extension compiled for Stable ABI 3.x is ABI-compatible with Python 3.x and above. Extensions that target a stable ABI must only use a limited subset of the C API. This subset is known as the :dfn:`Limited API`; its contents are :ref:`listed below `. On Windows, extensions that use a Stable ABI should be linked against ``python3.dll`` rather than a version-specific library such as ``python39.dll``. This library only exposes the relevant symbols. On some platforms, Python will look for and load shared library files named with the ``abi3`` or ``abi3t`` tag (for example, ``mymodule.abi3.so``). :term:`Free-threaded ` interpreters only recognize the ``abi3t`` tag, while non-free-threaded ones will prefer ``abi3`` but fall back to ``abi3t``. Thus, extensions compatible with both ABIs should use the ``abi3t`` tag. Python does not necessarily check that extensions it loads have compatible ABI. Extension authors are encouraged to add a check using the :c:macro:`Py_mod_abi` slot or the :c:func:`PyABIInfo_Check` function, but the user (or their packaging tool) is ultimately responsible for ensuring that, for example, extensions built for Stable ABI 3.10 are not installed for lower versions of Python. All functions in Stable ABI are present as functions in Python's shared library, not solely as macros. This makes them usable in languages that don't use the C preprocessor, including Python's :py:mod:`ctypes`. .. _abi3-compiling: Compiling for Stable ABI ------------------------ .. note:: Build tools (such as, for example, meson-python, scikit-build-core, or Setuptools) often have a mechanism for setting macros and synchronizing them with extension filenames and other metadata. Prefer using such a mechanism, if it exists, over defining the macros manually. The rest of this section is mainly relevant for tool authors, and for people who compile extensions manually. .. seealso:: `list of recommended tools`_ in the Python Packaging User Guide .. _list of recommended tools: https://packaging.python.org/en/latest/guides/tool-recommendations/#build-backends-for-extension-modules To compile for a Stable ABI, define one or both of the following macros to the lowest Python version your extension should support, in :c:macro:`Py_PACK_VERSION` format. Typically, you should choose a specific value rather than the version of the Python headers you are compiling against. The macros must be defined before including ``Python.h``. Since :c:macro:`Py_PACK_VERSION` is not available at this point, you will need to use the numeric value directly. For reference, the values for a few recent Python versions are: .. version-hex-cheatsheet:: When one of the macros is defined, ``Python.h`` will only expose API that is compatible with the given Stable ABI -- that is, the :ref:`Limited API ` plus some definitions that need to be visible to the compiler but should not be used directly. When both are defined, ``Python.h`` will only expose API compatible with both Stable ABIs. .. c:macro:: Py_LIMITED_API Target ``abi3``, that is, non-:term:`free-threaded builds ` of CPython. See :ref:`above ` for common information. .. c:macro:: Py_TARGET_ABI3T Target ``abi3t``, that is, :term:`free-threaded builds ` of CPython. See :ref:`above ` for common information. .. versionadded:: 3.15 Both macros specify a target ABI; the different naming style is due to backwards compatibility. .. admonition:: Historical note You can also define ``Py_LIMITED_API`` as ``3``. This works the same as ``0x03020000`` (Python 3.2, the version that introduced Stable ABI). When both are defined, ``Python.h`` may, or may not, redefine :c:macro:`!Py_LIMITED_API` to match :c:macro:`!Py_TARGET_ABI3T`. On a :term:`free-threaded build` -- that is, when :c:macro:`Py_GIL_DISABLED` is defined -- :c:macro:`!Py_TARGET_ABI3T` defaults to the value of :c:macro:`!Py_LIMITED_API`. This means that there are two ways to build for both ``abi3`` and ``abi3t``: - define both :c:macro:`!Py_LIMITED_API` and :c:macro:`!Py_TARGET_ABI3T`, or - define only :c:macro:`!Py_LIMITED_API` and: - on Windows, define :c:macro:`!Py_GIL_DISABLED`; - on other systems, use the headers of free-threaded build of Python. .. _limited-api-scope-and-performance: Stable ABI Scope and Performance -------------------------------- The goal for Stable ABI is to allow everything that is possible with the full C API, but possibly with a performance penalty. Generally, compatibility with Stable ABI will require some changes to an extension's source code. For example, while :c:func:`PyList_GetItem` is available, its "unsafe" macro variant :c:func:`PyList_GET_ITEM` is not. The macro can be faster because it can rely on version-specific implementation details of the list object. For another example, when *not* compiling for Stable ABI, some C API functions are inlined or replaced by macros. Compiling for Stable ABI disables this inlining, allowing stability as Python's data structures are improved, but possibly reducing performance. By leaving out the :c:macro:`!Py_LIMITED_API` or :c:macro:`!Py_TARGET_ABI3T` definition, it is possible to compile Stable-ABI-compatible source for a version-specific ABI. A potentially faster version-specific extension can then be distributed alongside a version compiled for Stable ABI -- a slower but more compatible fallback. .. _limited-api-caveats: Stable ABI Caveats ------------------ Note that compiling for Stable ABI is *not* a complete guarantee that code will be compatible with the expected Python versions. Stable ABI prevents *ABI* issues, like linker errors due to missing symbols or data corruption due to changes in structure layouts or function signatures. However, other changes in Python can change the *behavior* of extensions. One issue that the :c:macro:`Py_TARGET_ABI3T` and :c:macro:`Py_LIMITED_API` macros do not guard against is calling a function with arguments that are invalid in a lower Python version. For example, consider a function that starts accepting ``NULL`` for an argument. In Python 3.9, ``NULL`` now selects a default behavior, but in Python 3.8, the argument will be used directly, causing a ``NULL`` dereference and crash. A similar argument works for fields of structs. For these reasons, we recommend testing an extension with *all* minor Python versions it supports. We also recommend reviewing documentation of all used API to check if it is explicitly part of the Limited API. Even with ``Py_LIMITED_API`` defined, a few private declarations are exposed for technical reasons (or even unintentionally, as bugs). Also note that while compiling with ``Py_LIMITED_API`` 3.8 means that the extension should *load* on Python 3.12, and *compile* with Python 3.12, the same source will not necessarily compile with ``Py_LIMITED_API`` set to 3.12. In general, parts of the Limited API may be deprecated and removed, provided that Stable ABI stays stable. .. _stable-abi-platform: Platform Considerations ======================= ABI stability depends not only on Python, but also on the compiler used, lower-level libraries and compiler options. For the purposes of the :ref:`Stable ABIs `, these details define a “platform”. They usually depend on the OS type and processor architecture It is the responsibility of each particular distributor of Python to ensure that all Python versions on a particular platform are built in a way that does not break the Stable ABIs, or the version-specific ABIs. This is the case with Windows and macOS releases from ``python.org`` and many third-party distributors. ABI Checking ============ .. versionadded:: 3.15 Python includes a rudimentary check for ABI compatibility. This check is not comprehensive. It only guards against common cases of incompatible modules being installed for the wrong interpreter. It also does not take :ref:`platform incompatibilities ` into account. It can only be done after an extension is successfully loaded. Despite these limitations, it is recommended that extension modules use this mechanism, so that detectable incompatibilities raise exceptions rather than crash. Most modules can use this check via the :c:data:`Py_mod_abi` slot and the :c:macro:`PyABIInfo_VAR` macro, for example like this: .. code-block:: c PyABIInfo_VAR(abi_info); static PyModuleDef_Slot mymodule_slots[] = { {Py_mod_abi, &abi_info}, ... }; The full API is described below for advanced use cases. .. c:function:: int PyABIInfo_Check(PyABIInfo *info, const char *module_name) Verify that the given *info* is compatible with the currently running interpreter. Return 0 on success. On failure, raise an exception and return -1. If the ABI is incompatible, the raised exception will be :py:exc:`ImportError`. The *module_name* argument can be ``NULL``, or point to a NUL-terminated UTF-8-encoded string used for error messages. Note that if *info* describes the ABI that the current code uses (as defined by :c:macro:`PyABIInfo_VAR`, for example), using any other Python C API may lead to crashes. In particular, it is not safe to examine the raised exception. .. versionadded:: 3.15 .. c:macro:: PyABIInfo_VAR(NAME) Define a static :c:struct:`PyABIInfo` variable with the given *NAME* that describes the ABI that the current code will use. This macro expands to: .. code-block:: c static PyABIInfo NAME = { 1, 0, PyABIInfo_DEFAULT_FLAGS, PY_VERSION_HEX, PyABIInfo_DEFAULT_ABI_VERSION } .. versionadded:: 3.15 .. c:type:: PyABIInfo .. c:member:: uint8_t abiinfo_major_version The major version of :c:struct:`PyABIInfo`. Can be set to: * ``0`` to skip all checking, or * ``1`` to specify this version of :c:struct:`!PyABIInfo`. .. c:member:: uint8_t abiinfo_minor_version The minor version of :c:struct:`PyABIInfo`. Must be set to ``0``; larger values are reserved for backwards-compatible future versions of :c:struct:`!PyABIInfo`. .. c:member:: uint16_t flags .. c:namespace:: NULL This field is usually set to the following macro: .. c:macro:: PyABIInfo_DEFAULT_FLAGS Default flags, based on current values of macros such as :c:macro:`Py_LIMITED_API` and :c:macro:`Py_GIL_DISABLED`. Alternately, the field can be set to the following flags, combined by bitwise OR. Unused bits must be set to zero. ABI variant -- one of: .. c:macro:: PyABIInfo_STABLE Specifies that Stable ABI is used. .. c:macro:: PyABIInfo_INTERNAL Specifies ABI specific to a particular build of CPython. Internal use only. Free-threading compatibility -- one of: .. c:macro:: PyABIInfo_FREETHREADED Specifies ABI compatible with :term:`free-threaded builds ` of CPython. (That is, ones compiled with :option:`--disable-gil`; with ``t`` in :py:data:`sys.abiflags`) .. c:macro:: PyABIInfo_GIL Specifies ABI compatible with non-free-threaded builds of CPython (ones compiled *without* :option:`--disable-gil`). .. c:macro:: PyABIInfo_FREETHREADING_AGNOSTIC Specifies ABI compatible with both free-threaded and non-free-threaded builds of CPython, that is, both ``abi3`` and ``abi3t``. .. c:member:: uint32_t build_version The version of the Python headers used to build the code, in the format used by :c:macro:`PY_VERSION_HEX`. This can be set to ``0`` to skip any checks related to this field. This option is meant mainly for projects that do not use the CPython headers directly, and do not emulate a specific version of them. .. c:member:: uint32_t abi_version The ABI version. For Stable ABI, this field should be the value of :c:macro:`Py_LIMITED_API` or :c:macro:`Py_TARGET_ABI3T`. If both are defined, use the smaller value. (If :c:macro:`Py_LIMITED_API` is ``3``; use :c:expr:`Py_PACK_VERSION(3, 2)` instead of ``3``.) Otherwise, it should be set to :c:macro:`PY_VERSION_HEX`. It can also be set to ``0`` to skip any checks related to this field. .. c:namespace:: NULL .. c:macro:: PyABIInfo_DEFAULT_ABI_VERSION The value that should be used for this field, based on current values of macros such as :c:macro:`Py_LIMITED_API`, :c:macro:`PY_VERSION_HEX` and :c:macro:`Py_GIL_DISABLED`. .. versionadded:: 3.15 .. _limited-c-api: .. _limited-api-list: Contents of Limited API ======================= This is the definitive list of :ref:`Limited API ` for Python |version|: .. limited-api-list:: --- ### Doc C Api Structures (`Doc/c-api/structures.rst`) .. highlight:: c .. _common-structs: Common Object Structures ======================== There are a large number of structures which are used in the definition of object types for Python. This section describes these structures and how they are used. Base object types and macros ---------------------------- All Python objects ultimately share a small number of fields at the beginning of the object's representation in memory. These are represented by the :c:type:`PyObject` and :c:type:`PyVarObject` types, which are defined, in turn, by the expansions of some macros also used, whether directly or indirectly, in the definition of all other Python objects. Additional macros can be found under :ref:`reference counting `. .. c:type:: PyObject All object types are extensions of this type. This is a type which contains the information Python needs to treat a pointer to an object as an object. In a normal "release" build, it contains only the object's reference count and a pointer to the corresponding type object. Nothing is actually declared to be a :c:type:`PyObject`, but every pointer to a Python object can be cast to a :c:expr:`PyObject*`. The members must not be accessed directly; instead use macros such as :c:macro:`Py_REFCNT` and :c:macro:`Py_TYPE`. In the :ref:`Stable ABI ` for Free-Threaded Builds (``abi3t``), this struct is opaque; its size and layout may change between Python versions. In Stable ABI for non-free-threaded builds (``abi3``), the :c:member:`!ob_refcnt` and :c:member:`!ob_type` fields are available, but using them directly is discouraged. .. c:member:: Py_ssize_t ob_refcnt The object's reference count, as returned by :c:macro:`Py_REFCNT`. Do not use this field directly; instead use functions and macros such as :c:macro:`!Py_REFCNT`, :c:func:`Py_INCREF` and :c:func:`Py_DecRef`. The field type may be different from ``Py_ssize_t``, depending on build configuration and platform. .. c:member:: PyTypeObject* ob_type The object's type. Do not use this field directly; use :c:macro:`Py_TYPE` and :c:func:`Py_SET_TYPE` instead. .. c:member:: PyMutex ob_mutex A :ref:`per-object lock `, present only in the :term:`free-threaded ` build (when :c:macro:`Py_GIL_DISABLED` is defined). This field is **reserved for use by the critical section API** (:c:macro:`Py_BEGIN_CRITICAL_SECTION` / :c:macro:`Py_END_CRITICAL_SECTION`). Do **not** lock it directly with ``PyMutex_Lock``; doing so can cause deadlocks. If you need your own lock, add a separate :c:type:`PyMutex` field to your object struct. .. versionadded:: 3.13 .. c:type:: PyVarObject An extension of :c:type:`PyObject` that adds the :c:member:`~PyVarObject.ob_size` field. This is intended for objects that have some notion of *length*. As with :c:type:`!PyObject`, the members must not be accessed directly; instead use macros such as :c:macro:`Py_SIZE`, :c:macro:`Py_REFCNT` and :c:macro:`Py_TYPE`. In the :ref:`Stable ABI ` for Free-Threaded Builds (``abi3t``), this struct is opaque; its size and layout may change between Python versions. In Stable ABI for non-free-threaded builds (``abi3``), the :c:member:`!ob_base` and :c:member:`!ob_size` fields are available, but using them directly is discouraged. .. c:member:: PyObject ob_base Common object header. Typically, this field is not accessed directly; instead :c:type:`!PyVarObject` can be cast to :c:type:`PyObject`. .. c:member:: Py_ssize_t ob_size A size field, whose contents should be considered an object's internal implementation detail. Do not use this field directly; use :c:macro:`Py_SIZE` instead. Object creation functions such as :c:func:`PyObject_NewVar` will generally set this field to the requested size (number of items). After creation, arbitrary values can be stored in :c:member:`!ob_size` using :c:macro:`Py_SET_SIZE`. To get an object's publicly exposed length, as returned by the Python function :py:func:`len`, use :c:func:`PyObject_Length` instead. .. c:macro:: PyObject_HEAD This is a macro used when declaring new types which represent objects without a varying length. The PyObject_HEAD macro expands to:: PyObject ob_base; See documentation of :c:type:`PyObject` above. .. c:macro:: PyObject_VAR_HEAD This is a macro used when declaring new types which represent objects with a length that varies from instance to instance. The PyObject_VAR_HEAD macro expands to:: PyVarObject ob_base; See documentation of :c:type:`PyVarObject` above. .. c:var:: PyTypeObject PyBaseObject_Type The base class of all other objects, the same as :class:`object` in Python. .. c:function:: int Py_Is(PyObject *x, PyObject *y) Test if the *x* object is the *y* object, the same as ``x is y`` in Python. .. versionadded:: 3.10 .. c:function:: int Py_IsNone(PyObject *x) Test if an object is the ``None`` singleton, the same as ``x is None`` in Python. .. versionadded:: 3.10 .. c:function:: int Py_IsTrue(PyObject *x) Test if an object is the ``True`` singleton, the same as ``x is True`` in Python. .. versionadded:: 3.10 .. c:function:: int Py_IsFalse(PyObject *x) Test if an object is the ``False`` singleton, the same as ``x is False`` in Python. .. versionadded:: 3.10 .. c:function:: PyTypeObject* Py_TYPE(PyObject *o) Get the type of the Python object *o*. The returned reference is :term:`borrowed ` from *o*. Do not release it with :c:func:`Py_DECREF` or similar. .. versionchanged:: 3.11 :c:func:`Py_TYPE()` is changed to an inline static function. The parameter type is no longer :c:expr:`const PyObject*`. .. c:function:: int Py_IS_TYPE(PyObject *o, PyTypeObject *type) Return non-zero if the object *o* type is *type*. Return zero otherwise. Equivalent to: ``Py_TYPE(o) == type``. .. versionadded:: 3.9 .. c:function:: void Py_SET_TYPE(PyObject *o, PyTypeObject *type) Set the type of object *o* to *type*, without any checking or reference counting. This is a very low-level operation. Consider instead setting the Python attribute :attr:`~object.__class__` using :c:func:`PyObject_SetAttrString` or similar. Note that assigning an incompatible type can lead to undefined behavior. If *type* is a :ref:`heap type `, the caller must create a new reference to it. Similarly, if the old type of *o* is a heap type, the caller must release a reference to that type. .. versionadded:: 3.9 .. c:function:: Py_ssize_t Py_SIZE(PyVarObject *o) Get the :c:member:`~PyVarObject.ob_size` field of *o*. .. versionchanged:: 3.11 :c:func:`Py_SIZE()` is changed to an inline static function. The parameter type is no longer :c:expr:`const PyVarObject*`. .. c:function:: void Py_SET_SIZE(PyVarObject *o, Py_ssize_t size) Set the :c:member:`~PyVarObject.ob_size` field of *o* to *size*. .. versionadded:: 3.9 .. c:macro:: PyObject_HEAD_INIT(type) This is a macro which expands to initialization values for a new :c:type:`PyObject` type. This macro expands to:: _PyObject_EXTRA_INIT 1, type, .. c:macro:: PyVarObject_HEAD_INIT(type, size) This is a macro which expands to initialization values for a new :c:type:`PyVarObject` type, including the :c:member:`~PyVarObject.ob_size` field. This macro expands to:: _PyObject_EXTRA_INIT 1, type, size, Implementing functions and methods ---------------------------------- .. c:type:: PyCFunction Type of the functions used to implement most Python callables in C. Functions of this type take two :c:expr:`PyObject*` parameters and return one such value. If the return value is ``NULL``, an exception shall have been set. If not ``NULL``, the return value is interpreted as the return value of the function as exposed in Python. The function must return a new reference. The function signature is:: PyObject *PyCFunction(PyObject *self, PyObject *args); .. c:type:: PyCFunctionWithKeywords Type of the functions used to implement Python callables in C with signature :ref:`METH_VARARGS | METH_KEYWORDS `. The function signature is:: PyObject *PyCFunctionWithKeywords(PyObject *self, PyObject *args, PyObject *kwargs); .. c:type:: PyCFunctionFast Type of the functions used to implement Python callables in C with signature :c:macro:`METH_FASTCALL`. The function signature is:: PyObject *PyCFunctionFast(PyObject *self, PyObject *const *args, Py_ssize_t nargs); .. c:type:: PyCFunctionFastWithKeywords Type of the functions used to implement Python callables in C with signature :ref:`METH_FASTCALL | METH_KEYWORDS `. The function signature is:: PyObject *PyCFunctionFastWithKeywords(PyObject *self, PyObject *const *args, Py_ssize_t nargs, PyObject *kwnames); .. c:type:: PyCMethod Type of the functions used to implement Python callables in C with signature :ref:`METH_METHOD | METH_FASTCALL | METH_KEYWORDS `. The function signature is:: PyObject *PyCMethod(PyObject *self, PyTypeObject *defining_class, PyObject *const *args, Py_ssize_t nargs, PyObject *kwnames) .. versionadded:: 3.9 .. c:type:: PyMethodDef Structure used to describe a method of an extension type. This structure has four fields: .. c:member:: const char *ml_name Name of the method. A ``NULL`` *ml_name* marks the end of a :c:type:`!PyMethodDef` array. .. c:member:: PyCFunction ml_meth Pointer to the C implementation. .. c:member:: int ml_flags Flags bits indicating how the call should be constructed. .. c:member:: const char *ml_doc Points to the contents of the docstring. The :c:member:`~PyMethodDef.ml_meth` is a C function pointer. The functions may be of different types, but they always return :c:expr:`PyObject*`. If the function is not of the :c:type:`PyCFunction`, the compiler will require a cast in the method table. Even though :c:type:`PyCFunction` defines the first parameter as :c:expr:`PyObject*`, it is common that the method implementation uses the specific C type of the *self* object. The :c:member:`~PyMethodDef.ml_flags` field is a bitfield which can include the following flags. The individual flags indicate either a calling convention or a binding convention. There are these calling conventions: .. c:macro:: METH_VARARGS This is the typical calling convention, where the methods have the type :c:type:`PyCFunction`. The function expects two :c:expr:`PyObject*` values. The first one is the *self* object for methods; for module functions, it is the module object. The second parameter (often called *args*) is a tuple object representing all arguments. This parameter is typically processed using :c:func:`PyArg_ParseTuple` or :c:func:`PyArg_UnpackTuple`. .. c:macro:: METH_KEYWORDS Can only be used in certain combinations with other flags: :ref:`METH_VARARGS | METH_KEYWORDS `, :ref:`METH_FASTCALL | METH_KEYWORDS ` and :ref:`METH_METHOD | METH_FASTCALL | METH_KEYWORDS `. .. _METH_VARARGS-METH_KEYWORDS: :c:expr:`METH_VARARGS | METH_KEYWORDS` Methods with these flags must be of type :c:type:`PyCFunctionWithKeywords`. The function expects three parameters: *self*, *args*, *kwargs* where *kwargs* is a dictionary of all the keyword arguments or possibly ``NULL`` if there are no keyword arguments. The parameters are typically processed using :c:func:`PyArg_ParseTupleAndKeywords`. .. c:macro:: METH_FASTCALL Fast calling convention supporting only positional arguments. The methods have the type :c:type:`PyCFunctionFast`. The first parameter is *self*, the second parameter is a C array of :c:expr:`PyObject*` values indicating the arguments and the third parameter is the number of arguments (the length of the array). .. versionadded:: 3.7 .. versionchanged:: 3.10 ``METH_FASTCALL`` is now part of the :ref:`stable ABI `. .. _METH_FASTCALL-METH_KEYWORDS: :c:expr:`METH_FASTCALL | METH_KEYWORDS` Extension of :c:macro:`METH_FASTCALL` supporting also keyword arguments, with methods of type :c:type:`PyCFunctionFastWithKeywords`. Keyword arguments are passed the same way as in the :ref:`vectorcall protocol `: there is an additional fourth :c:expr:`PyObject*` parameter which is a tuple representing the names of the keyword arguments (which are guaranteed to be strings) or possibly ``NULL`` if there are no keywords. The values of the keyword arguments are stored in the *args* array, after the positional arguments. .. versionadded:: 3.7 .. c:macro:: METH_METHOD Can only be used in the combination with other flags: :ref:`METH_METHOD | METH_FASTCALL | METH_KEYWORDS `. .. _METH_METHOD-METH_FASTCALL-METH_KEYWORDS: :c:expr:`METH_METHOD | METH_FASTCALL | METH_KEYWORDS` Extension of :ref:`METH_FASTCALL | METH_KEYWORDS ` supporting the *defining class*, that is, the class that contains the method in question. The defining class might be a superclass of ``Py_TYPE(self)``. The method needs to be of type :c:type:`PyCMethod`, the same as for ``METH_FASTCALL | METH_KEYWORDS`` with ``defining_class`` argument added after ``self``. .. versionadded:: 3.9 .. c:macro:: METH_NOARGS Methods without parameters don't need to check whether arguments are given if they are listed with the :c:macro:`METH_NOARGS` flag. They need to be of type :c:type:`PyCFunction`. The first parameter is typically named *self* and will hold a reference to the module or object instance. In all cases the second parameter will be ``NULL``. The function must have 2 parameters. Since the second parameter is unused, :c:macro:`Py_UNUSED` can be used to prevent a compiler warning. .. c:macro:: METH_O Methods with a single object argument can be listed with the :c:macro:`METH_O` flag, instead of invoking :c:func:`PyArg_ParseTuple` with a ``"O"`` argument. They have the type :c:type:`PyCFunction`, with the *self* parameter, and a :c:expr:`PyObject*` parameter representing the single argument. These two constants are not used to indicate the calling convention but the binding when used with methods of classes. These may not be used for functions defined for modules. At most one of these flags may be set for any given method. .. c:macro:: METH_CLASS .. index:: pair: built-in function; classmethod The method will be passed the type object as the first parameter rather than an instance of the type. This is used to create *class methods*, similar to what is created when using the :deco:`classmethod` built-in decorator. .. c:macro:: METH_STATIC .. index:: pair: built-in function; staticmethod The method will be passed ``NULL`` as the first parameter rather than an instance of the type. This is used to create *static methods*, similar to what is created when using the :deco:`staticmethod` built-in decorator. One other constant controls whether a method is loaded in place of another definition with the same method name. .. c:macro:: METH_COEXIST The method will be loaded in place of existing definitions. Without *METH_COEXIST*, the default is to skip repeated definitions. Since slot wrappers are loaded before the method table, the existence of a *sq_contains* slot, for example, would generate a wrapped method named :meth:`~object.__contains__` and preclude the loading of a corresponding PyCFunction with the same name. With the flag defined, the PyCFunction will be loaded in place of the wrapper object and will co-exist with the slot. This is helpful because calls to PyCFunctions are optimized more than wrapper object calls. .. c:var:: PyTypeObject PyCMethod_Type The type object corresponding to Python C method objects. This is available as :class:`types.BuiltinMethodType` in the Python layer. .. c:function:: int PyCMethod_Check(PyObject *op) Return true if *op* is an instance of the :c:type:`PyCMethod_Type` type or a subtype of it. This function always succeeds. .. c:function:: int PyCMethod_CheckExact(PyObject *op) This is the same as :c:func:`PyCMethod_Check`, but does not account for subtypes. .. c:function:: PyObject * PyCMethod_New(PyMethodDef *ml, PyObject *self, PyObject *module, PyTypeObject *cls) Turn *ml* into a Python :term:`callable` object. The caller must ensure that *ml* outlives the :term:`callable`. Typically, *ml* is defined as a static variable. The *self* parameter will be passed as the *self* argument to the C function in ``ml->ml_meth`` when invoked. *self* can be ``NULL``. The :term:`callable` object's ``__module__`` attribute can be set from the given *module* argument. *module* should be a Python string, which will be used as name of the module the function is defined in. If unavailable, it can be set to :const:`None` or ``NULL``. .. seealso:: :attr:`function.__module__` The *cls* parameter will be passed as the *defining_class* argument to the C function. Must be set if :c:macro:`METH_METHOD` is set on ``ml->ml_flags``. .. versionadded:: 3.9 .. c:var:: PyTypeObject PyCFunction_Type The type object corresponding to Python C function objects. This is available as :class:`types.BuiltinFunctionType` in the Python layer. .. c:function:: int PyCFunction_Check(PyObject *op) Return true if *op* is an instance of the :c:type:`PyCFunction_Type` type or a subtype of it. This function always succeeds. .. c:function:: int PyCFunction_CheckExact(PyObject *op) This is the same as :c:func:`PyCFunction_Check`, but does not account for subtypes. .. c:function:: PyObject * PyCFunction_NewEx(PyMethodDef *ml, PyObject *self, PyObject *module) Equivalent to ``PyCMethod_New(ml, self, module, NULL)``. .. c:function:: PyObject * PyCFunction_New(PyMethodDef *ml, PyObject *self) Equivalent to ``PyCMethod_New(ml, self, NULL, NULL)``. .. c:function:: int PyCFunction_GetFlags(PyObject *func) Get the function's flags on *func* as they were passed to :c:member:`~PyMethodDef.ml_flags`. If *func* is not a C function object, this fails with an exception. *func* must not be ``NULL``. This function returns the function's flags on success, and ``-1`` with an exception set on failure. .. c:function:: int PyCFunction_GET_FLAGS(PyObject *func) This is the same as :c:func:`PyCFunction_GetFlags`, but without error or type checking. .. c:function:: PyCFunction PyCFunction_GetFunction(PyObject *func) Get the function pointer on *func* as it was passed to :c:member:`~PyMethodDef.ml_meth`. If *func* is not a C function object, this fails with an exception. *func* must not be ``NULL``. This function returns the function pointer on success, and ``NULL`` with an exception set on failure. .. c:function:: int PyCFunction_GET_FUNCTION(PyObject *func) This is the same as :c:func:`PyCFunction_GetFunction`, but without error or type checking. .. c:function:: PyObject *PyCFunction_GetSelf(PyObject *func) Get the "self" object on *func*. This is the object that would be passed to the first argument of a :c:type:`PyCFunction`. For C function objects created through a :c:type:`PyMethodDef` on a :c:type:`PyModuleDef`, this is the resulting module object. If *func* is not a C function object, this fails with an exception. *func* must not be ``NULL``. This function returns a :term:`borrowed reference` to the "self" object on success, and ``NULL`` with an exception set on failure. .. c:function:: PyObject *PyCFunction_GET_SELF(PyObject *func) This is the same as :c:func:`PyCFunction_GetSelf`, but without error or type checking. Accessing attributes of extension types --------------------------------------- .. c:type:: PyMemberDef Structure which describes an attribute of a type which corresponds to a C struct member. When defining a class, put a NULL-terminated array of these structures in the :c:member:`~PyTypeObject.tp_members` slot. Its fields are, in order: .. c:member:: const char* name Name of the member. A NULL value marks the end of a ``PyMemberDef[]`` array. The string should be static, no copy is made of it. .. c:member:: int type The type of the member in the C struct. See :ref:`PyMemberDef-types` for the possible values. .. c:member:: Py_ssize_t offset The offset in bytes that the member is located on the type’s object struct. .. c:member:: int flags Zero or more of the :ref:`PyMemberDef-flags`, combined using bitwise OR. .. c:member:: const char* doc The docstring, or NULL. The string should be static, no copy is made of it. Typically, it is defined using :c:macro:`PyDoc_STR`. By default (when :c:member:`~PyMemberDef.flags` is ``0``), members allow both read and write access. Use the :c:macro:`Py_READONLY` flag for read-only access. Certain types, like :c:macro:`Py_T_STRING`, imply :c:macro:`Py_READONLY`. Only :c:macro:`Py_T_OBJECT_EX` (and legacy :c:macro:`T_OBJECT`) members can be deleted. .. _pymemberdef-offsets: For heap-allocated types (created using :c:func:`PyType_FromSpec` or similar), ``PyMemberDef`` may contain a definition for the special member ``"__vectorcalloffset__"``, corresponding to :c:member:`~PyTypeObject.tp_vectorcall_offset` in type objects. This member must be defined with ``Py_T_PYSSIZET``, and either ``Py_READONLY`` or ``Py_READONLY | Py_RELATIVE_OFFSET``. For example:: static PyMemberDef spam_type_members[] = { {"__vectorcalloffset__", Py_T_PYSSIZET, offsetof(Spam_object, vectorcall), Py_READONLY}, {NULL} /* Sentinel */ }; (You may need to ``#include `` for :c:func:`!offsetof`.) The legacy offsets :c:member:`~PyTypeObject.tp_dictoffset` and :c:member:`~PyTypeObject.tp_weaklistoffset` can be defined similarly using ``"__dictoffset__"`` and ``"__weaklistoffset__"`` members, but extensions are strongly encouraged to use :c:macro:`Py_TPFLAGS_MANAGED_DICT` and :c:macro:`Py_TPFLAGS_MANAGED_WEAKREF` instead. .. versionchanged:: 3.12 ``PyMemberDef`` is always available. Previously, it required including ``"structmember.h"``. .. versionchanged:: 3.14 :c:macro:`Py_RELATIVE_OFFSET` is now allowed for ``"__vectorcalloffset__"``, ``"__dictoffset__"`` and ``"__weaklistoffset__"``. .. c:function:: PyObject* PyMember_GetOne(const char *obj_addr, struct PyMemberDef *m) Get an attribute belonging to the object at address *obj_addr*. The attribute is described by ``PyMemberDef`` *m*. Returns ``NULL`` on error. .. versionchanged:: 3.12 ``PyMember_GetOne`` is always available. Previously, it required including ``"structmember.h"``. .. c:function:: int PyMember_SetOne(char *obj_addr, struct PyMemberDef *m, PyObject *o) Set an attribute belonging to the object at address *obj_addr* to object *o*. The attribute to set is described by ``PyMemberDef`` *m*. Returns ``0`` if successful and a negative value on failure. .. versionchanged:: 3.12 ``PyMember_SetOne`` is always available. Previously, it required including ``"structmember.h"``. .. _PyMemberDef-flags: Member flags ^^^^^^^^^^^^ The following flags can be used with :c:member:`PyMemberDef.flags`: .. c:macro:: Py_READONLY Not writable. .. c:macro:: Py_AUDIT_READ Emit an ``object.__getattr__`` :ref:`audit event ` before reading. .. c:macro:: Py_RELATIVE_OFFSET Indicates that the :c:member:`~PyMemberDef.offset` of this ``PyMemberDef`` entry indicates an offset from the subclass-specific data, rather than from ``PyObject``. Can only be used as part of the :c:data:`Py_tp_members` :c:type:`slot ` when creating a class using negative :c:member:`~PyType_Spec.basicsize`. It is mandatory in that case. When setting :c:member:`~PyTypeObject.tp_members` from the slot during class creation, Python clears the flag and sets :c:member:`PyMemberDef.offset` to the offset from the ``PyObject`` struct. .. index:: single: READ_RESTRICTED (C macro) single: WRITE_RESTRICTED (C macro) single: RESTRICTED (C macro) .. versionchanged:: 3.10 The :c:macro:`!RESTRICTED`, :c:macro:`!READ_RESTRICTED` and :c:macro:`!WRITE_RESTRICTED` macros available with ``#include "structmember.h"`` are deprecated. :c:macro:`!READ_RESTRICTED` and :c:macro:`!RESTRICTED` are equivalent to :c:macro:`Py_AUDIT_READ`; :c:macro:`!WRITE_RESTRICTED` does nothing. .. index:: single: READONLY (C macro) .. versionchanged:: 3.12 The :c:macro:`!READONLY` macro was renamed to :c:macro:`Py_READONLY`. The :c:macro:`!PY_AUDIT_READ` macro was renamed with the ``Py_`` prefix. The new names are now always available. Previously, these required ``#include "structmember.h"``. The header is still available and it provides the old names. .. _PyMemberDef-types: Member types ^^^^^^^^^^^^ :c:member:`PyMemberDef.type` can be one of the following macros corresponding to various C types. When the member is accessed in Python, it will be converted to the equivalent Python type. When it is set from Python, it will be converted back to the C type. If that is not possible, an exception such as :exc:`TypeError` or :exc:`ValueError` is raised. Unless marked (D), attributes defined this way cannot be deleted using e.g. :keyword:`del` or :py:func:`delattr`. ================================ ============================= ====================== Macro name C type Python type ================================ ============================= ====================== .. c:macro:: Py_T_BYTE :c:expr:`char` :py:class:`int` .. c:macro:: Py_T_SHORT :c:expr:`short` :py:class:`int` .. c:macro:: Py_T_INT :c:expr:`int` :py:class:`int` .. c:macro:: Py_T_LONG :c:expr:`long` :py:class:`int` .. c:macro:: Py_T_LONGLONG :c:expr:`long long` :py:class:`int` .. c:macro:: Py_T_UBYTE :c:expr:`unsigned char` :py:class:`int` .. c:macro:: Py_T_UINT :c:expr:`unsigned int` :py:class:`int` .. c:macro:: Py_T_USHORT :c:expr:`unsigned short` :py:class:`int` .. c:macro:: Py_T_ULONG :c:expr:`unsigned long` :py:class:`int` .. c:macro:: Py_T_ULONGLONG :c:expr:`unsigned long long` :py:class:`int` .. c:macro:: Py_T_PYSSIZET :c:expr:`Py_ssize_t` :py:class:`int` .. c:macro:: Py_T_FLOAT :c:expr:`float` :py:class:`float` .. c:macro:: Py_T_DOUBLE :c:expr:`double` :py:class:`float` .. c:macro:: Py_T_BOOL :c:expr:`char` :py:class:`bool` (written as 0 or 1) .. c:macro:: Py_T_STRING :c:expr:`const char *` (*) :py:class:`str` (RO) .. c:macro:: Py_T_STRING_INPLACE :c:expr:`const char[]` (*) :py:class:`str` (RO) .. c:macro:: Py_T_CHAR :c:expr:`char` (0-127) :py:class:`str` (**) .. c:macro:: Py_T_OBJECT_EX :c:expr:`PyObject *` :py:class:`object` (D) ================================ ============================= ====================== (*): Zero-terminated, UTF8-encoded C string. With :c:macro:`!Py_T_STRING` the C representation is a pointer; with :c:macro:`!Py_T_STRING_INPLACE` the string is stored directly in the structure. (**): String of length 1. Only ASCII is accepted. (RO): Implies :c:macro:`Py_READONLY`. (D): Can be deleted, in which case the pointer is set to ``NULL``. Reading a ``NULL`` pointer raises :py:exc:`AttributeError`. .. index:: single: T_BYTE (C macro) single: T_SHORT (C macro) single: T_INT (C macro) single: T_LONG (C macro) single: T_LONGLONG (C macro) single: T_UBYTE (C macro) single: T_USHORT (C macro) single: T_UINT (C macro) single: T_ULONG (C macro) single: T_ULONGULONG (C macro) single: T_PYSSIZET (C macro) single: T_FLOAT (C macro) single: T_DOUBLE (C macro) single: T_BOOL (C macro) single: T_CHAR (C macro) single: T_STRING (C macro) single: T_STRING_INPLACE (C macro) single: T_OBJECT_EX (C macro) single: structmember.h .. versionadded:: 3.12 In previous versions, the macros were only available with ``#include "structmember.h"`` and were named without the ``Py_`` prefix (e.g. as ``T_INT``). The header is still available and contains the old names, along with the following deprecated types: .. c:macro:: T_OBJECT Like ``Py_T_OBJECT_EX``, but ``NULL`` is converted to ``None``. This results in surprising behavior in Python: deleting the attribute effectively sets it to ``None``. .. c:macro:: T_NONE Always ``None``. Must be used with :c:macro:`Py_READONLY`. Defining Getters and Setters ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ .. c:type:: PyGetSetDef Structure to define property-like access for a type. See also description of the :c:member:`PyTypeObject.tp_getset` slot. .. c:member:: const char* name attribute name .. c:member:: getter get C function to get the attribute. .. c:member:: setter set Optional C function to set or delete the attribute. If ``NULL``, the attribute is read-only. .. c:member:: const char* doc optional docstring .. c:member:: void* closure Optional user data pointer, providing additional data for getter and setter. .. c:type:: PyObject *(*getter)(PyObject *, void *) The ``get`` function takes one :c:expr:`PyObject*` parameter (the instance) and a user data pointer (the associated ``closure``): It should return a new reference on success or ``NULL`` with a set exception on failure. .. c:type:: int (*setter)(PyObject *, PyObject *, void *) ``set`` functions take two :c:expr:`PyObject*` parameters (the instance and the value to be set) and a user data pointer (the associated ``closure``): In case the attribute should be deleted the second parameter is ``NULL``. Should return ``0`` on success or ``-1`` with a set exception on failure. --- ### Doc C Api Subinterpreters (`Doc/c-api/subinterpreters.rst`) .. highlight:: c .. _sub-interpreter-support: Multiple interpreters in a Python process ========================================= While in most uses, you will only embed a single Python interpreter, there are cases where you need to create several independent interpreters in the same process and perhaps even in the same thread. Sub-interpreters allow you to do that. The "main" interpreter is the first one created when the runtime initializes. It is usually the only Python interpreter in a process. Unlike sub-interpreters, the main interpreter has unique process-global responsibilities like signal handling. It is also responsible for execution during runtime initialization and is usually the active interpreter during runtime finalization. The :c:func:`PyInterpreterState_Main` function returns a pointer to its state. You can switch between sub-interpreters using the :c:func:`PyThreadState_Swap` function. You can create and destroy them using the following functions: .. c:type:: PyInterpreterConfig Structure containing most parameters to configure a sub-interpreter. Its values are used only in :c:func:`Py_NewInterpreterFromConfig` and never modified by the runtime. .. versionadded:: 3.12 Structure fields: .. c:member:: int use_main_obmalloc If this is ``0`` then the sub-interpreter will use its own "object" allocator state. Otherwise it will use (share) the main interpreter's. If this is ``0`` then :c:member:`~PyInterpreterConfig.check_multi_interp_extensions` must be ``1`` (non-zero). If this is ``1`` then :c:member:`~PyInterpreterConfig.gil` must not be :c:macro:`PyInterpreterConfig_OWN_GIL`. .. c:member:: int allow_fork If this is ``0`` then the runtime will not support forking the process in any thread where the sub-interpreter is currently active. Otherwise fork is unrestricted. Note that the :mod:`subprocess` module still works when fork is disallowed. .. c:member:: int allow_exec If this is ``0`` then the runtime will not support replacing the current process via exec (e.g. :func:`os.execv`) in any thread where the sub-interpreter is currently active. Otherwise exec is unrestricted. Note that the :mod:`subprocess` module still works when exec is disallowed. .. c:member:: int allow_threads If this is ``0`` then the sub-interpreter's :mod:`threading` module won't create threads. Otherwise threads are allowed. .. c:member:: int allow_daemon_threads If this is ``0`` then the sub-interpreter's :mod:`threading` module won't create daemon threads. Otherwise daemon threads are allowed (as long as :c:member:`~PyInterpreterConfig.allow_threads` is non-zero). .. c:member:: int check_multi_interp_extensions If this is ``0`` then all extension modules may be imported, including legacy (single-phase init) modules, in any thread where the sub-interpreter is currently active. Otherwise only multi-phase init extension modules (see :pep:`489`) may be imported. (Also see :c:macro:`Py_mod_multiple_interpreters`.) This must be ``1`` (non-zero) if :c:member:`~PyInterpreterConfig.use_main_obmalloc` is ``0``. .. c:member:: int gil This determines the operation of the GIL for the sub-interpreter. It may be one of the following: .. c:namespace:: NULL .. c:macro:: PyInterpreterConfig_DEFAULT_GIL Use the default selection (:c:macro:`PyInterpreterConfig_SHARED_GIL`). .. c:macro:: PyInterpreterConfig_SHARED_GIL Use (share) the main interpreter's GIL. .. c:macro:: PyInterpreterConfig_OWN_GIL Use the sub-interpreter's own GIL. If this is :c:macro:`PyInterpreterConfig_OWN_GIL` then :c:member:`PyInterpreterConfig.use_main_obmalloc` must be ``0``. .. c:function:: PyStatus Py_NewInterpreterFromConfig(PyThreadState **tstate_p, const PyInterpreterConfig *config) .. index:: pair: module; builtins pair: module; __main__ pair: module; sys single: stdout (in module sys) single: stderr (in module sys) single: stdin (in module sys) Create a new sub-interpreter. This is an (almost) totally separate environment for the execution of Python code. In particular, the new interpreter has separate, independent versions of all imported modules, including the fundamental modules :mod:`builtins`, :mod:`__main__` and :mod:`sys`. The table of loaded modules (``sys.modules``) and the module search path (``sys.path``) are also separate. The new environment has no ``sys.argv`` variable. It has new standard I/O stream file objects ``sys.stdin``, ``sys.stdout`` and ``sys.stderr`` (however these refer to the same underlying file descriptors). The given *config* controls the options with which the interpreter is initialized. Upon success, *tstate_p* will be set to the first :term:`thread state` created in the new sub-interpreter. This thread state is :term:`attached `. Note that no actual thread is created; see the discussion of thread states below. If creation of the new interpreter is unsuccessful, *tstate_p* is set to ``NULL``; no exception is set since the exception state is stored in the :term:`attached thread state`, which might not exist. Like all other Python/C API functions, an :term:`attached thread state` must be present before calling this function, but it might be detached upon returning. On success, the returned thread state will be :term:`attached `. If the sub-interpreter is created with its own :term:`GIL` then the :term:`attached thread state` of the calling interpreter will be detached. When the function returns, the new interpreter's :term:`thread state` will be :term:`attached ` to the current thread and the previous interpreter's :term:`attached thread state` will remain detached. .. versionadded:: 3.12 Sub-interpreters are most effective when isolated from each other, with certain functionality restricted:: PyInterpreterConfig config = { .use_main_obmalloc = 0, .allow_fork = 0, .allow_exec = 0, .allow_threads = 1, .allow_daemon_threads = 0, .check_multi_interp_extensions = 1, .gil = PyInterpreterConfig_OWN_GIL, }; PyThreadState *tstate = NULL; PyStatus status = Py_NewInterpreterFromConfig(&tstate, &config); if (PyStatus_Exception(status)) { Py_ExitStatusException(status); } Note that the config is used only briefly and does not get modified. During initialization the config's values are converted into various :c:type:`PyInterpreterState` values. A read-only copy of the config may be stored internally on the :c:type:`PyInterpreterState`. .. index:: single: Py_FinalizeEx (C function) single: Py_Initialize (C function) Extension modules are shared between (sub-)interpreters as follows: * For modules using multi-phase initialization, e.g. :c:func:`PyModule_FromDefAndSpec`, a separate module object is created and initialized for each interpreter. Only C-level static and global variables are shared between these module objects. * For modules using legacy :ref:`single-phase initialization `, e.g. :c:func:`PyModule_Create`, the first time a particular extension is imported, it is initialized normally, and a (shallow) copy of its module's dictionary is squirreled away. When the same extension is imported by another (sub-)interpreter, a new module is initialized and filled with the contents of this copy; the extension's ``init`` function is not called. Objects in the module's dictionary thus end up shared across (sub-)interpreters, which might cause unwanted behavior (see `Bugs and caveats`_ below). Note that this is different from what happens when an extension is imported after the interpreter has been completely re-initialized by calling :c:func:`Py_FinalizeEx` and :c:func:`Py_Initialize`; in that case, the extension's ``initmodule`` function *is* called again. As with multi-phase initialization, this means that only C-level static and global variables are shared between these modules. .. index:: single: close (in module os) .. c:function:: PyThreadState* Py_NewInterpreter(void) .. index:: pair: module; builtins pair: module; __main__ pair: module; sys single: stdout (in module sys) single: stderr (in module sys) single: stdin (in module sys) Create a new sub-interpreter. This is essentially just a wrapper around :c:func:`Py_NewInterpreterFromConfig` with a config that preserves the existing behavior. The result is an unisolated sub-interpreter that shares the main interpreter's GIL, allows fork/exec, allows daemon threads, and allows single-phase init modules. .. c:function:: void Py_EndInterpreter(PyThreadState *tstate) .. index:: single: Py_FinalizeEx (C function) Destroy the (sub-)interpreter represented by the given :term:`thread state`. The given thread state must be :term:`attached `. When the call returns, there will be no :term:`attached thread state`. All thread states associated with this interpreter are destroyed. :c:func:`Py_FinalizeEx` will destroy all sub-interpreters that haven't been explicitly destroyed at that point. .. _per-interpreter-gil: A per-interpreter GIL --------------------- .. versionadded:: 3.12 Using :c:func:`Py_NewInterpreterFromConfig` you can create a sub-interpreter that is completely isolated from other interpreters, including having its own GIL. The most important benefit of this isolation is that such an interpreter can execute Python code without being blocked by other interpreters or blocking any others. Thus a single Python process can truly take advantage of multiple CPU cores when running Python code. The isolation also encourages a different approach to concurrency than that of just using threads. (See :pep:`554` and :pep:`684`.) Using an isolated interpreter requires vigilance in preserving that isolation. That especially means not sharing any objects or mutable state without guarantees about thread-safety. Even objects that are otherwise immutable (e.g. ``None``, ``(1, 5)``) can't normally be shared because of the refcount. One simple but less-efficient approach around this is to use a global lock around all use of some state (or object). Alternately, effectively immutable objects (like integers or strings) can be made safe in spite of their refcounts by making them :term:`immortal`. In fact, this has been done for the builtin singletons, small integers, and a number of other builtin objects. If you preserve isolation then you will have access to proper multi-core computing without the complications that come with free-threading. Failure to preserve isolation will expose you to the full consequences of free-threading, including races and hard-to-debug crashes. Aside from that, one of the main challenges of using multiple isolated interpreters is how to communicate between them safely (not break isolation) and efficiently. The runtime and stdlib do not provide any standard approach to this yet. A future stdlib module would help mitigate the effort of preserving isolation and expose effective tools for communicating (and sharing) data between interpreters. Bugs and caveats ---------------- Because sub-interpreters (and the main interpreter) are part of the same process, the insulation between them isn't perfect --- for example, using low-level file operations like :func:`os.close` they can (accidentally or maliciously) affect each other's open files. Because of the way extensions are shared between (sub-)interpreters, some extensions may not work properly; this is especially likely when using single-phase initialization or (static) global variables. It is possible to insert objects created in one sub-interpreter into a namespace of another (sub-)interpreter; this should be avoided if possible. Special care should be taken to avoid sharing user-defined functions, methods, instances or classes between sub-interpreters, since import operations executed by such objects may affect the wrong (sub-)interpreter's dictionary of loaded modules. It is equally important to avoid sharing objects from which the above are reachable. Also note that combining this functionality with ``PyGILState_*`` APIs is delicate, because these APIs assume a bijection between Python thread states and OS-level threads, an assumption broken by the presence of sub-interpreters. It is highly recommended that you don't switch sub-interpreters between a pair of matching :c:func:`PyGILState_Ensure` and :c:func:`PyGILState_Release` calls. Furthermore, extensions (such as :mod:`ctypes`) using these APIs to allow calling of Python code from non-Python created threads will probably be broken when using sub-interpreters. High-level APIs --------------- .. c:type:: PyInterpreterState This data structure represents the state shared by a number of cooperating threads. Threads belonging to the same interpreter share their module administration and a few other internal items. There are no public members in this structure. Threads belonging to different interpreters initially share nothing, except process state like available memory, open file descriptors and such. The global interpreter lock is also shared by all threads, regardless of to which interpreter they belong. .. versionchanged:: 3.12 :pep:`684` introduced the possibility of a :ref:`per-interpreter GIL `. See :c:func:`Py_NewInterpreterFromConfig`. .. c:function:: PyInterpreterState* PyInterpreterState_Get(void) Get the current interpreter. Issue a fatal error if there is no :term:`attached thread state`. It cannot return NULL. .. versionadded:: 3.9 .. c:function:: int64_t PyInterpreterState_GetID(PyInterpreterState *interp) Return the interpreter's unique ID. If there was any error in doing so then ``-1`` is returned and an error is set. The caller must have an :term:`attached thread state`. .. versionadded:: 3.7 .. c:function:: PyObject* PyInterpreterState_GetDict(PyInterpreterState *interp) Return a dictionary in which interpreter-specific data may be stored. If this function returns ``NULL`` then no exception has been raised and the caller should assume no interpreter-specific dict is available. This is not a replacement for :c:func:`PyModule_GetState()`, which extensions should use to store interpreter-specific state information. The returned dictionary is borrowed from the interpreter and is valid until interpreter shutdown. .. versionadded:: 3.8 .. c:type:: PyObject* (*_PyFrameEvalFunction)(PyThreadState *tstate, _PyInterpreterFrame *frame, int throwflag) Type of a frame evaluation function. The *throwflag* parameter is used by the ``throw()`` method of generators: if non-zero, handle the current exception. .. versionchanged:: 3.9 The function now takes a *tstate* parameter. .. versionchanged:: 3.11 The *frame* parameter changed from ``PyFrameObject*`` to ``_PyInterpreterFrame*``. .. c:function:: _PyFrameEvalFunction _PyInterpreterState_GetEvalFrameFunc(PyInterpreterState *interp) Get the frame evaluation function. See the :pep:`523` "Adding a frame evaluation API to CPython". .. versionadded:: 3.9 .. c:function:: void _PyInterpreterState_SetEvalFrameFunc(PyInterpreterState *interp, _PyFrameEvalFunction eval_frame) Set the frame evaluation function. See the :pep:`523` "Adding a frame evaluation API to CPython". .. versionadded:: 3.9 .. c:function:: void _PyInterpreterState_SetEvalFrameAllowSpecialization(PyInterpreterState *interp, int allow_specialization) Enables or disables specialization why a custom frame evaluator is in place. If *allow_specialization* is non-zero, the adaptive specializer will continue to specialize bytecodes even though a custom eval frame function is set. When *allow_specialization* is zero, setting a custom eval frame disables specialization. The standard interpreter loop will continue to deopt while a frame evaluation API is in place - the frame evaluation function needs to handle the specialized opcodes to take advantage of this. .. versionadded:: 3.15 .. c:function:: int _PyInterpreterState_IsSpecializationEnabled(PyInterpreterState *interp) Return non-zero if adaptive specialization is enabled for the interpreter. Specialization is enabled when no custom eval frame function is set, or when one is set with *allow_specialization* enabled. .. versionadded:: 3.15 Low-level APIs -------------- All of the following functions must be called after :c:func:`Py_Initialize`. .. versionchanged:: 3.7 :c:func:`Py_Initialize()` now initializes the :term:`GIL` and sets an :term:`attached thread state`. .. c:function:: PyInterpreterState* PyInterpreterState_New() Create a new interpreter state object. An :term:`attached thread state` is not needed, but may optionally exist if it is necessary to serialize calls to this function. .. audit-event:: cpython.PyInterpreterState_New "" c.PyInterpreterState_New .. c:function:: void PyInterpreterState_Clear(PyInterpreterState *interp) Reset all information in an interpreter state object. There must be an :term:`attached thread state` for the interpreter. .. audit-event:: cpython.PyInterpreterState_Clear "" c.PyInterpreterState_Clear .. c:function:: void PyInterpreterState_Delete(PyInterpreterState *interp) Destroy an interpreter state object. There **should not** be an :term:`attached thread state` for the target interpreter. The interpreter state must have been reset with a previous call to :c:func:`PyInterpreterState_Clear`. .. _advanced-debugging: Advanced debugger support ------------------------- These functions are only intended to be used by advanced debugging tools. .. c:function:: PyInterpreterState* PyInterpreterState_Head() Return the interpreter state object at the head of the list of all such objects. .. c:function:: PyInterpreterState* PyInterpreterState_Main() Return the main interpreter state object. .. c:function:: PyInterpreterState* PyInterpreterState_Next(PyInterpreterState *interp) Return the next interpreter state object after *interp* from the list of all such objects. .. c:function:: PyThreadState * PyInterpreterState_ThreadHead(PyInterpreterState *interp) Return the pointer to the first :c:type:`PyThreadState` object in the list of threads associated with the interpreter *interp*. .. c:function:: PyThreadState* PyThreadState_Next(PyThreadState *tstate) Return the next thread state object after *tstate* from the list of all such objects belonging to the same :c:type:`PyInterpreterState` object. --- ### Doc C Api Synchronization (`Doc/c-api/synchronization.rst`) .. highlight:: c .. _synchronization: Synchronization primitives ========================== The C-API provides a basic mutual exclusion lock. .. c:type:: PyMutex A mutual exclusion lock. The :c:type:`!PyMutex` should be initialized to zero to represent the unlocked state. For example:: PyMutex mutex = {0}; Instances of :c:type:`!PyMutex` should not be copied or moved. Both the contents and address of a :c:type:`!PyMutex` are meaningful, and it must remain at a fixed, writable location in memory. .. note:: A :c:type:`!PyMutex` currently occupies one byte, but the size should be considered unstable. The size may change in future Python releases without a deprecation period. .. versionadded:: 3.13 .. c:function:: void PyMutex_Lock(PyMutex *m) Lock mutex *m*. If another thread has already locked it, the calling thread will block until the mutex is unlocked. While blocked, the thread will temporarily detach the :term:`thread state ` if one exists. .. versionadded:: 3.13 .. c:function:: void PyMutex_Unlock(PyMutex *m) Unlock mutex *m*. The mutex must be locked --- otherwise, the function will issue a fatal error. .. versionadded:: 3.13 .. c:function:: int PyMutex_IsLocked(PyMutex *m) Returns non-zero if the mutex *m* is currently locked, zero otherwise. .. note:: This function is intended for use in assertions and debugging only and should not be used to make concurrency control decisions, as the lock state may change immediately after the check. .. versionadded:: 3.14 .. _python-critical-section-api: Python critical section API --------------------------- The critical section API provides a deadlock avoidance layer on top of per-object locks for :term:`free-threaded ` CPython. They are intended to replace reliance on the :term:`global interpreter lock`, and are no-ops in versions of Python with the global interpreter lock. Critical sections are intended to be used for custom types implemented in C-API extensions. They should generally not be used with built-in types like :class:`list` and :class:`dict` because their public C-APIs already use critical sections internally, with the notable exception of :c:func:`PyDict_Next`, which requires critical section to be acquired externally. Critical sections avoid deadlocks by implicitly suspending active critical sections, hence, they do not provide exclusive access such as provided by traditional locks like :c:type:`PyMutex`. When a critical section is started, the per-object lock for the object is acquired. If the code executed inside the critical section calls C-API functions then it can suspend the critical section thereby releasing the per-object lock, so other threads can acquire the per-object lock for the same object. Variants that accept :c:type:`PyMutex` pointers rather than Python objects are also available. Use these variants to start a critical section in a situation where there is no :c:type:`PyObject` -- for example, when working with a C type that does not extend or wrap :c:type:`PyObject` but still needs to call into the C API in a manner that might lead to deadlocks. .. note:: Operations that need to lock two objects at once must use :c:macro:`Py_BEGIN_CRITICAL_SECTION2`. You *cannot* use nested critical sections to lock more than one object at once, because the inner critical section may suspend the outer critical sections. This API does not provide a way to lock more than two objects at once. Example usage:: static PyObject * set_field(MyObject *self, PyObject *value) { Py_BEGIN_CRITICAL_SECTION(self); Py_SETREF(self->field, Py_XNewRef(value)); Py_END_CRITICAL_SECTION(); Py_RETURN_NONE; } In the above example, :c:macro:`Py_SETREF` calls :c:macro:`Py_DECREF`, which can call arbitrary code through an object's deallocation function. The critical section API avoids potential deadlocks due to reentrancy and lock ordering by allowing the runtime to temporarily suspend the critical section if the code triggered by the finalizer blocks and calls :c:func:`PyEval_SaveThread`. .. _critical-section-macros: .. c:macro:: Py_BEGIN_CRITICAL_SECTION(op) Acquires the per-object lock for the object *op* and begins a critical section. In the free-threaded build, and when building for the :ref:`Stable ABI `, this macro expands to:: { PyCriticalSection _py_cs; PyCriticalSection_Begin(&_py_cs, (PyObject*)(op)) In the default build, this macro expands to ``{``. .. versionadded:: 3.13 .. c:macro:: Py_BEGIN_CRITICAL_SECTION_MUTEX(m) Locks the mutex *m* and begins a critical section. In the free-threaded build, this macro expands to:: { PyCriticalSection _py_cs; PyCriticalSection_BeginMutex(&_py_cs, m) Note that unlike :c:macro:`Py_BEGIN_CRITICAL_SECTION`, there is no cast for the argument of the macro - it must be a :c:type:`PyMutex` pointer. On the default build, this macro expands to ``{``. .. versionadded:: 3.14 .. c:macro:: Py_END_CRITICAL_SECTION() Ends the critical section and releases the per-object lock. In the free-threaded build, and when building for the :ref:`Stable ABI `, this macro expands to:: PyCriticalSection_End(&_py_cs); } In the default build, this macro expands to ``}``. .. versionadded:: 3.13 .. c:macro:: Py_BEGIN_CRITICAL_SECTION2(a, b) Acquires the per-object locks for the objects *a* and *b* and begins a critical section. The locks are acquired in a consistent order (lowest address first) to avoid lock ordering deadlocks. In the free-threaded build, this macro expands to:: { PyCriticalSection2 _py_cs2; PyCriticalSection2_Begin(&_py_cs2, (PyObject*)(a), (PyObject*)(b)) In the default build, this macro expands to ``{``. .. versionadded:: 3.13 .. c:macro:: Py_BEGIN_CRITICAL_SECTION2_MUTEX(m1, m2) Locks the mutexes *m1* and *m2* and begins a critical section. In the free-threaded build, and when building for the :ref:`Stable ABI `, this macro expands to:: { PyCriticalSection2 _py_cs2; PyCriticalSection2_BeginMutex(&_py_cs2, m1, m2) Note that unlike :c:macro:`Py_BEGIN_CRITICAL_SECTION2`, there is no cast for the arguments of the macro - they must be :c:type:`PyMutex` pointers. On the default build, this macro expands to ``{``. .. versionadded:: 3.14 .. c:macro:: Py_END_CRITICAL_SECTION2() Ends the critical section and releases the per-object locks. In the free-threaded build, and when building for the :ref:`Stable ABI `, this macro expands to:: PyCriticalSection2_End(&_py_cs2); } In the default build, this macro expands to ``}``. .. versionadded:: 3.13 Low-level critical section API ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ The following functions and structs are exposed for cases where C macros are not available. .. c:function:: void PyCriticalSection_Begin(PyCriticalSection *c, PyObject *op) void PyCriticalSection_End(PyCriticalSection *c) void PyCriticalSection2_Begin(PyCriticalSection2 *c, PyObject *a, PyObject *b) void PyCriticalSection2_End(PyCriticalSection2 *c); To be used only as in the macro expansions listed :ref:`earlier in this section `. In non-:term:`free-threaded ` builds of CPython, these functions do nothing. .. versionadded:: 3.13 .. c:type:: PyCriticalSection PyCriticalSection2 To be used only as in the macro expansions listed :ref:`earlier in this section `. Note that the contents of the structures are private and their meaning may change in future Python versions. .. versionadded:: 3.13 .. c:function:: void PyCriticalSection_BeginMutex(PyCriticalSection *c, PyMutex *m); void PyCriticalSection2_BeginMutex(PyCriticalSection2 *c, PyMutex *m1, PyMutex *m2); .. (These need to be in a separate section without a Stable ABI annotation.) To be used only as in the macro expansions listed :ref:`earlier in this section `. In non-:term:`free-threaded ` builds of CPython, these functions do nothing. .. versionadded:: 3.14 Legacy locking APIs ------------------- These APIs are obsolete since Python 3.13 with the introduction of :c:type:`PyMutex`. .. versionchanged:: 3.15 These APIs are now a simple wrapper around ``PyMutex``. .. c:type:: PyThread_type_lock A pointer to a mutual exclusion lock. .. c:type:: PyLockStatus The result of acquiring a lock with a timeout. .. c:namespace:: NULL .. c:enumerator:: PY_LOCK_FAILURE Failed to acquire the lock. .. c:enumerator:: PY_LOCK_ACQUIRED The lock was successfully acquired. .. c:enumerator:: PY_LOCK_INTR The lock was interrupted by a signal. .. c:function:: PyThread_type_lock PyThread_allocate_lock(void) Allocate a new lock. On success, this function returns a lock; on failure, this function returns ``0`` without an exception set. The caller does not need to hold an :term:`attached thread state`. .. versionchanged:: 3.15 This function now always uses :c:type:`PyMutex`. In prior versions, this would use a lock provided by the operating system. .. c:function:: void PyThread_free_lock(PyThread_type_lock lock) Destroy *lock*. The lock should not be held by any thread when calling this. The caller does not need to hold an :term:`attached thread state`. .. c:function:: PyLockStatus PyThread_acquire_lock_timed(PyThread_type_lock lock, long long microseconds, int intr_flag) Acquire *lock* with a timeout. This will wait for *microseconds* microseconds to acquire the lock. If the timeout expires, this function returns :c:enumerator:`PY_LOCK_FAILURE`. If *microseconds* is ``-1``, this will wait indefinitely until the lock has been released. If *intr_flag* is ``1``, acquiring the lock may be interrupted by a signal, in which case this function returns :c:enumerator:`PY_LOCK_INTR`. Upon interruption, it's generally expected that the caller makes a call to :c:func:`Py_MakePendingCalls` to propagate an exception to Python code. If the lock is successfully acquired, this function returns :c:enumerator:`PY_LOCK_ACQUIRED`. The caller does not need to hold an :term:`attached thread state`. .. c:function:: int PyThread_acquire_lock(PyThread_type_lock lock, int waitflag) Acquire *lock*. If *waitflag* is ``1`` and another thread currently holds the lock, this function will wait until the lock can be acquired and will always return ``1``. If *waitflag* is ``0`` and another thread holds the lock, this function will not wait and instead return ``0``. If the lock is not held by any other thread, then this function will acquire it and return ``1``. Unlike :c:func:`PyThread_acquire_lock_timed`, acquiring the lock cannot be interrupted by a signal. The caller does not need to hold an :term:`attached thread state`. .. c:function:: int PyThread_release_lock(PyThread_type_lock lock) Release *lock*. If *lock* is not held, then this function issues a fatal error. The caller does not need to hold an :term:`attached thread state`. --- ### Doc C Api Sys (`Doc/c-api/sys.rst`) .. highlight:: c .. _os: Operating System Utilities ========================== .. c:function:: PyObject* PyOS_FSPath(PyObject *path) Return the file system representation for *path*. If the object is a :class:`str` or :class:`bytes` object, then a new :term:`strong reference` is returned. If the object implements the :class:`os.PathLike` interface, then :meth:`~os.PathLike.__fspath__` is returned as long as it is a :class:`str` or :class:`bytes` object. Otherwise :exc:`TypeError` is raised and ``NULL`` is returned. .. versionadded:: 3.6 .. c:function:: int Py_FdIsInteractive(FILE *fp, const char *filename) Return true (nonzero) if the standard I/O file *fp* with name *filename* is deemed interactive. This is the case for files for which ``isatty(fileno(fp))`` is true. If the :c:member:`PyConfig.interactive` is non-zero, this function also returns true if the *filename* pointer is ``NULL`` or if the name is equal to one of the strings ``''`` or ``'???'``. This function must not be called before Python is initialized. .. c:function:: void PyOS_BeforeFork() Function to prepare some internal state before a process fork. This should be called before calling :c:func:`fork` or any similar function that clones the current process. Only available on systems where :c:func:`fork` is defined. .. warning:: The C :c:func:`fork` call should only be made from the :ref:`"main" thread ` (of the :ref:`"main" interpreter `). The same is true for ``PyOS_BeforeFork()``. .. versionadded:: 3.7 .. c:function:: void PyOS_AfterFork_Parent() Function to update some internal state after a process fork. This should be called from the parent process after calling :c:func:`fork` or any similar function that clones the current process, regardless of whether process cloning was successful. Only available on systems where :c:func:`fork` is defined. .. warning:: The C :c:func:`fork` call should only be made from the :ref:`"main" thread ` (of the :ref:`"main" interpreter `). The same is true for ``PyOS_AfterFork_Parent()``. .. versionadded:: 3.7 .. c:function:: void PyOS_AfterFork_Child() Function to update internal interpreter state after a process fork. This must be called from the child process after calling :c:func:`fork`, or any similar function that clones the current process, if there is any chance the process will call back into the Python interpreter. Only available on systems where :c:func:`fork` is defined. .. warning:: The C :c:func:`fork` call should only be made from the :ref:`"main" thread ` (of the :ref:`"main" interpreter `). The same is true for ``PyOS_AfterFork_Child()``. .. versionadded:: 3.7 .. seealso:: :func:`os.register_at_fork` allows registering custom Python functions to be called by :c:func:`PyOS_BeforeFork()`, :c:func:`PyOS_AfterFork_Parent` and :c:func:`PyOS_AfterFork_Child`. .. c:function:: void PyOS_AfterFork() Function to update some internal state after a process fork; this should be called in the new process if the Python interpreter will continue to be used. If a new executable is loaded into the new process, this function does not need to be called. .. deprecated:: 3.7 This function is superseded by :c:func:`PyOS_AfterFork_Child()`. .. c:function:: int PyOS_CheckStack() .. index:: single: USE_STACKCHECK (C macro) Return true when the interpreter runs out of stack space. This is a reliable check, but is only available when :c:macro:`!USE_STACKCHECK` is defined (currently on certain versions of Windows using the Microsoft Visual C++ compiler). :c:macro:`!USE_STACKCHECK` will be defined automatically; you should never change the definition in your own code. .. c:type:: void (*PyOS_sighandler_t)(int) .. c:function:: PyOS_sighandler_t PyOS_getsig(int i) Return the current signal handler for signal *i*. This is a thin wrapper around either :c:func:`!sigaction` or :c:func:`!signal`. Do not call those functions directly! .. c:function:: PyOS_sighandler_t PyOS_setsig(int i, PyOS_sighandler_t h) Set the signal handler for signal *i* to be *h*; return the old signal handler. This is a thin wrapper around either :c:func:`!sigaction` or :c:func:`!signal`. Do not call those functions directly! .. c:function:: int PyOS_InterruptOccurred(void) Check if a :c:macro:`!SIGINT` signal has been received. Returns ``1`` if a :c:macro:`!SIGINT` has occurred and clears the signal flag, or ``0`` otherwise. In most cases, you should prefer :c:func:`PyErr_CheckSignals` over this function. :c:func:`!PyErr_CheckSignals` invokes the appropriate signal handlers for all pending signals, allowing Python code to handle the signal properly. This function only detects :c:macro:`!SIGINT` and does not invoke any Python signal handlers. This function is async-signal-safe and this function cannot fail. The caller must hold an :term:`attached thread state`. .. c:function:: wchar_t* Py_DecodeLocale(const char* arg, size_t *size) .. warning:: This function should not be called directly: use the :c:type:`PyConfig` API with the :c:func:`PyConfig_SetBytesString` function which ensures that :ref:`Python is preinitialized `. This function must not be called before :ref:`Python is preinitialized ` and so that the LC_CTYPE locale is properly configured: see the :c:func:`Py_PreInitialize` function. Decode a byte string from the :term:`filesystem encoding ` with the :ref:`surrogateescape error handler `. Undecodable bytes are decoded as characters in range U+DC80..U+DCFF. If a byte sequence can be decoded as a surrogate character, escape the bytes using the surrogateescape error handler instead of decoding them. Return a pointer to a newly allocated wide character string, use :c:func:`PyMem_RawFree` to free the memory. If size is not ``NULL``, write the number of wide characters excluding the null character into ``*size`` On memory allocation failure, set *\*size* to ``(size_t)-1`` and return ``NULL``. On decode error, set *\*size* to ``(size_t)-2`` and return ``NULL``. Decoding errors should never happen, unless there is a bug in the C library. The :term:`filesystem encoding and error handler` are selected by :c:func:`PyConfig_Read`: see :c:member:`~PyConfig.filesystem_encoding` and :c:member:`~PyConfig.filesystem_errors` members of :c:type:`PyConfig`. Use the :c:func:`Py_EncodeLocale` function to encode the character string back to a byte string. .. seealso:: The :c:func:`PyUnicode_DecodeFSDefaultAndSize` and :c:func:`PyUnicode_DecodeLocaleAndSize` functions. .. versionadded:: 3.5 .. versionchanged:: 3.7 The function now uses the UTF-8 encoding in the :ref:`Python UTF-8 Mode `. .. versionchanged:: 3.8 The function now uses the UTF-8 encoding on Windows if :c:member:`PyPreConfig.legacy_windows_fs_encoding` is zero; .. c:function:: char* Py_EncodeLocale(const wchar_t *text, size_t *error_pos) Encode a wide character string to the :term:`filesystem encoding ` with the :ref:`surrogateescape error handler `. Surrogate characters in the range U+DC80..U+DCFF are encoded to bytes 0x80..0xFF. Return a pointer to a newly allocated byte string, use :c:func:`PyMem_Free` to free the memory. On memory allocation failure, set *\*error_pos* to ``(size_t)-1`` and return ``NULL``. On encoding error, set *\*error_pos* to the index of the first unencodable character and return ``NULL``. The :term:`filesystem encoding and error handler` are selected by :c:func:`PyConfig_Read`: see :c:member:`~PyConfig.filesystem_encoding` and :c:member:`~PyConfig.filesystem_errors` members of :c:type:`PyConfig`. Use the :c:func:`Py_DecodeLocale` function to decode the bytes string back to a wide character string. .. warning:: This function must not be called before :ref:`Python is preinitialized ` and so that the LC_CTYPE locale is properly configured: see the :c:func:`Py_PreInitialize` function. .. seealso:: The :c:func:`PyUnicode_EncodeFSDefault` and :c:func:`PyUnicode_EncodeLocale` functions. .. versionadded:: 3.5 .. versionchanged:: 3.7 The function now uses the UTF-8 encoding in the :ref:`Python UTF-8 Mode `. .. versionchanged:: 3.8 The function now uses the UTF-8 encoding on Windows if :c:member:`PyPreConfig.legacy_windows_fs_encoding` is zero. .. c:function:: FILE* Py_fopen(PyObject *path, const char *mode) Similar to :c:func:`!fopen`, but *path* is a Python object and an exception is set on error. *path* must be a :class:`str` object, a :class:`bytes` object, or a :term:`path-like object`. On success, return the new file pointer. On error, set an exception and return ``NULL``. The file must be closed by :c:func:`Py_fclose` rather than calling directly :c:func:`!fclose`. The file descriptor is created non-inheritable (:pep:`446`). The caller must have an :term:`attached thread state`. .. versionadded:: 3.14 .. c:function:: int Py_fclose(FILE *file) Close a file that was opened by :c:func:`Py_fopen`. On success, return ``0``. On error, return ``EOF`` and ``errno`` is set to indicate the error. In either case, any further access (including another call to :c:func:`Py_fclose`) to the stream results in undefined behavior. .. versionadded:: 3.14 .. _systemfunctions: System Functions ================ These are utility functions that make functionality from the :mod:`sys` module accessible to C code. They all work with the current interpreter thread's :mod:`sys` module's dict, which is contained in the internal thread state structure. .. c:function:: PyObject *PySys_GetAttr(PyObject *name) Get the attribute *name* of the :mod:`sys` module. Return a :term:`strong reference`. Raise :exc:`RuntimeError` and return ``NULL`` if it does not exist or if the :mod:`sys` module cannot be found. If the non-existing object should not be treated as a failure, you can use :c:func:`PySys_GetOptionalAttr` instead. .. versionadded:: 3.15 .. c:function:: PyObject *PySys_GetAttrString(const char *name) This is the same as :c:func:`PySys_GetAttr`, but *name* is specified as a :c:expr:`const char*` UTF-8 encoded bytes string, rather than a :c:expr:`PyObject*`. If the non-existing object should not be treated as a failure, you can use :c:func:`PySys_GetOptionalAttrString` instead. .. versionadded:: 3.15 .. c:function:: int PySys_GetOptionalAttr(PyObject *name, PyObject **result) Variant of :c:func:`PySys_GetAttr` which doesn't raise exception if the object does not exist. * Set *\*result* to a new :term:`strong reference` to the object and return ``1`` if the object exists. * Set *\*result* to ``NULL`` and return ``0`` without setting an exception if the object does not exist. * Set an exception, set *\*result* to ``NULL``, and return ``-1``, if an error occurred. .. versionadded:: 3.15 .. c:function:: int PySys_GetOptionalAttrString(const char *name, PyObject **result) This is the same as :c:func:`PySys_GetOptionalAttr`, but *name* is specified as a :c:expr:`const char*` UTF-8 encoded bytes string, rather than a :c:expr:`PyObject*`. .. versionadded:: 3.15 .. c:function:: PyObject *PySys_GetObject(const char *name) Similar to :c:func:`PySys_GetAttrString`, but return a :term:`borrowed reference` and return ``NULL`` *without* setting exception on failure. Preserves exception that was set before the call. .. c:function:: int PySys_SetObject(const char *name, PyObject *v) Set *name* in the :mod:`sys` module to *v* unless *v* is ``NULL``, in which case *name* is deleted from the sys module. Returns ``0`` on success, ``-1`` on error. .. c:function:: void PySys_WriteStdout(const char *format, ...) Write the output string described by *format* to :data:`sys.stdout`. No exceptions are raised, even if truncation occurs (see below). *format* should limit the total size of the formatted output string to 1000 bytes or less -- after 1000 bytes, the output string is truncated. In particular, this means that no unrestricted "%s" formats should occur; these should be limited using "%.s" where is a decimal number calculated so that plus the maximum size of other formatted text does not exceed 1000 bytes. Also watch out for "%f", which can print hundreds of digits for very large numbers. If a problem occurs, or :data:`sys.stdout` is unset, the formatted message is written to the real (C level) *stdout*. .. c:function:: void PySys_WriteStderr(const char *format, ...) As :c:func:`PySys_WriteStdout`, but write to :data:`sys.stderr` or *stderr* instead. .. c:function:: void PySys_FormatStdout(const char *format, ...) Function similar to PySys_WriteStdout() but format the message using :c:func:`PyUnicode_FromFormatV` and don't truncate the message to an arbitrary length. .. versionadded:: 3.2 .. c:function:: void PySys_FormatStderr(const char *format, ...) As :c:func:`PySys_FormatStdout`, but write to :data:`sys.stderr` or *stderr* instead. .. versionadded:: 3.2 .. c:function:: PyObject *PySys_GetXOptions() Return the current dictionary of :option:`-X` options, similarly to :data:`sys._xoptions`. On error, ``NULL`` is returned and an exception is set. .. versionadded:: 3.2 .. c:function:: int PySys_Audit(const char *event, const char *format, ...) Raise an auditing event with any active hooks. Return zero for success and non-zero with an exception set on failure. The *event* string argument must not be *NULL*. If any hooks have been added, *format* and other arguments will be used to construct a tuple to pass. Apart from ``N``, the same format characters as used in :c:func:`Py_BuildValue` are available. If the built value is not a tuple, it will be added into a single-element tuple. The ``N`` format option must not be used. It consumes a reference, but since there is no way to know whether arguments to this function will be consumed, using it may cause reference leaks. Note that ``#`` format characters should always be treated as :c:type:`Py_ssize_t`, regardless of whether ``PY_SSIZE_T_CLEAN`` was defined. :func:`sys.audit` performs the same function from Python code. See also :c:func:`PySys_AuditTuple`. .. versionadded:: 3.8 .. versionchanged:: 3.8.2 Require :c:type:`Py_ssize_t` for ``#`` format characters. Previously, an unavoidable deprecation warning was raised. .. c:function:: int PySys_AuditTuple(const char *event, PyObject *args) Similar to :c:func:`PySys_Audit`, but pass arguments as a Python object. *args* must be a :class:`tuple`. To pass no arguments, *args* can be *NULL*. .. versionadded:: 3.13 .. c:function:: int PySys_AddAuditHook(Py_AuditHookFunction hook, void *userData) Append the callable *hook* to the list of active auditing hooks. Return zero on success and non-zero on failure. If the runtime has been initialized, also set an error on failure. Hooks added through this API are called for all interpreters created by the runtime. The *userData* pointer is passed into the hook function. Since hook functions may be called from different runtimes, this pointer should not refer directly to Python state. This function is safe to call before :c:func:`Py_Initialize`. When called after runtime initialization, existing audit hooks are notified and may silently abort the operation by raising an error subclassed from :class:`RuntimeError` (other errors will not be silenced). The hook function is always called with an :term:`attached thread state` by the Python interpreter that raised the event. See :pep:`578` for a detailed description of auditing. Functions in the runtime and standard library that raise events are listed in the :ref:`audit events table `. Details are in each function's documentation. .. audit-event:: sys.addaudithook "" c.PySys_AddAuditHook If the interpreter is initialized, this function raises an auditing event ``sys.addaudithook`` with no arguments. If any existing hooks raise an exception derived from :class:`RuntimeError`, the new hook will not be added and the exception is cleared. As a result, callers cannot assume that their hook has been added unless they control all existing hooks. .. c:namespace:: NULL .. c:type:: int (*Py_AuditHookFunction) (const char *event, PyObject *args, void *userData) The type of the hook function. *event* is the C string event argument passed to :c:func:`PySys_Audit` or :c:func:`PySys_AuditTuple`. *args* is guaranteed to be a :c:type:`PyTupleObject`. *userData* is the argument passed to PySys_AddAuditHook(). .. versionadded:: 3.8 .. versionchanged:: 3.8.1 Exceptions derived from :class:`Exception` but not :class:`RuntimeError` are no longer suppressed. .. _processcontrol: Process Control =============== .. c:function:: void Py_FatalError(const char *message) .. index:: single: abort (C function) Print a fatal error message and kill the process. No cleanup is performed. This function should only be invoked when a condition is detected that would make it dangerous to continue using the Python interpreter; e.g., when the object administration appears to be corrupted. On Unix, the standard C library function :c:func:`!abort` is called which will attempt to produce a :file:`core` file. The ``Py_FatalError()`` function is replaced with a macro which logs automatically the name of the current function, unless the ``Py_LIMITED_API`` macro is defined. .. versionchanged:: 3.9 Log the function name automatically. .. c:function:: void Py_Exit(int status) .. index:: single: Py_FinalizeEx (C function) single: exit (C function) Exit the current process. This calls :c:func:`Py_FinalizeEx` and then calls the standard C library function ``exit(status)``. If :c:func:`Py_FinalizeEx` indicates an error, the exit status is set to 120. .. versionchanged:: 3.6 Errors from finalization no longer ignored. .. c:function:: int Py_AtExit(void (*func) ()) .. index:: single: Py_FinalizeEx (C function) single: cleanup functions Register a cleanup function to be called by :c:func:`Py_FinalizeEx`. The cleanup function will be called with no arguments and should return no value. At most 32 cleanup functions can be registered. When the registration is successful, :c:func:`Py_AtExit` returns ``0``; on failure, it returns ``-1``. The cleanup function registered last is called first. Each cleanup function will be called at most once. Since Python's internal finalization will have completed before the cleanup function, no Python APIs should be called by *func*. .. seealso:: :c:func:`PyUnstable_AtExit` for passing a ``void *data`` argument. --- ### Doc C Api Threads (`Doc/c-api/threads.rst`) .. highlight:: c .. _threads: Thread states and the global interpreter lock ============================================= .. index:: single: global interpreter lock single: interpreter lock single: lock, interpreter Unless on a :term:`free-threaded build` of :term:`CPython`, the Python interpreter is generally not thread-safe. In order to support multi-threaded Python programs, there's a global lock, called the :term:`global interpreter lock` or :term:`GIL`, that must be held by a thread before accessing Python objects. Without the lock, even the simplest operations could cause problems in a multi-threaded program: for example, when two threads simultaneously increment the reference count of the same object, the reference count could end up being incremented only once instead of twice. As such, only a thread that holds the GIL may operate on Python objects or invoke Python's C API. .. index:: single: setswitchinterval (in module sys) In order to emulate concurrency, the interpreter regularly tries to switch threads between bytecode instructions (see :func:`sys.setswitchinterval`). This is why locks are also necessary for thread-safety in pure-Python code. Additionally, the global interpreter lock is released around blocking I/O operations, such as reading or writing to a file. From the C API, this is done by :ref:`detaching the thread state `. .. index:: single: PyThreadState (C type) The Python interpreter keeps some thread-local information inside a data structure called :c:type:`PyThreadState`, known as a :term:`thread state`. Each thread has a thread-local pointer to a :c:type:`PyThreadState`; a thread state referenced by this pointer is considered to be :term:`attached `. A thread can only have one :term:`attached thread state` at a time. An attached thread state is typically analogous with holding the GIL, except on free-threaded builds. On builds with the GIL enabled, attaching a thread state will block until the GIL can be acquired. However, even on builds with the GIL disabled, it is still required to have an attached thread state, as the interpreter needs to keep track of which threads may access Python objects. .. note:: Even on the free-threaded build, attaching a thread state may block, as the GIL can be re-enabled or threads might be temporarily suspended (such as during a garbage collection). Generally, there will always be an attached thread state when using Python's C API, including during embedding and when implementing methods, so it's uncommon to need to set up a thread state on your own. Only in some specific cases, such as in a :c:macro:`Py_BEGIN_ALLOW_THREADS` block or in a fresh thread, will the thread not have an attached thread state. If uncertain, check if :c:func:`PyThreadState_GetUnchecked` returns ``NULL``. If it turns out that you do need to create a thread state, it is recommended to use :c:func:`PyThreadState_Ensure` or :c:func:`PyThreadState_EnsureFromView`, which will manage the thread state for you. .. _detaching-thread-state: Detaching the thread state from extension code ---------------------------------------------- Most extension code manipulating the :term:`thread state` has the following simple structure:: Save the thread state in a local variable. ... Do some blocking I/O operation ... Restore the thread state from the local variable. This is so common that a pair of macros exists to simplify it:: Py_BEGIN_ALLOW_THREADS ... Do some blocking I/O operation ... Py_END_ALLOW_THREADS .. index:: single: Py_BEGIN_ALLOW_THREADS (C macro) single: Py_END_ALLOW_THREADS (C macro) The :c:macro:`Py_BEGIN_ALLOW_THREADS` macro opens a new block and declares a hidden local variable; the :c:macro:`Py_END_ALLOW_THREADS` macro closes the block. The block above expands to the following code:: PyThreadState *_save; _save = PyEval_SaveThread(); ... Do some blocking I/O operation ... PyEval_RestoreThread(_save); .. index:: single: PyEval_RestoreThread (C function) single: PyEval_SaveThread (C function) Here is how these functions work: The attached thread state implies that the GIL is held for the interpreter. To detach it, :c:func:`PyEval_SaveThread` is called and the result is stored in a local variable. By detaching the thread state, the GIL is released, which allows other threads to attach to the interpreter and execute while the current thread performs blocking I/O. When the I/O operation is complete, the old thread state is reattached by calling :c:func:`PyEval_RestoreThread`, which will wait until the GIL can be acquired. .. note:: Performing blocking I/O is the most common use case for detaching the thread state, but it is also useful to call it over long-running native code that doesn't need access to Python objects or Python's C API. For example, the standard :mod:`zlib` and :mod:`hashlib` modules detach the :term:`thread state ` when compressing or hashing data. On a :term:`free-threaded build`, the :term:`GIL` is usually out of the question, but **detaching the thread state is still required**, because the interpreter periodically needs to block all threads to get a consistent view of Python objects without the risk of race conditions. For example, CPython currently suspends all threads for a short period of time while running the garbage collector. .. warning:: Detaching the thread state can lead to unexpected behavior during interpreter finalization. See :ref:`cautions-regarding-runtime-finalization` for more details. APIs ^^^^ The following macros are normally used without a trailing semicolon; look for example usage in the Python source distribution. .. note:: These macros are still necessary on the :term:`free-threaded build` to prevent deadlocks. .. c:macro:: Py_BEGIN_ALLOW_THREADS This macro expands to ``{ PyThreadState *_save; _save = PyEval_SaveThread();``. Note that it contains an opening brace; it must be matched with a following :c:macro:`Py_END_ALLOW_THREADS` macro. See above for further discussion of this macro. .. c:macro:: Py_END_ALLOW_THREADS This macro expands to ``PyEval_RestoreThread(_save); }``. Note that it contains a closing brace; it must be matched with an earlier :c:macro:`Py_BEGIN_ALLOW_THREADS` macro. See above for further discussion of this macro. .. c:macro:: Py_BLOCK_THREADS This macro expands to ``PyEval_RestoreThread(_save);``: it is equivalent to :c:macro:`Py_END_ALLOW_THREADS` without the closing brace. .. c:macro:: Py_UNBLOCK_THREADS This macro expands to ``_save = PyEval_SaveThread();``: it is equivalent to :c:macro:`Py_BEGIN_ALLOW_THREADS` without the opening brace and variable declaration. .. _non-python-created-threads: .. _c-api-foreign-threads: Using the C API from foreign threads ------------------------------------ When threads are created using the dedicated Python APIs (such as the :mod:`threading` module), a thread state is automatically associated with them, However, when a thread is created from native code (for example, by a third-party library with its own thread management), it doesn't hold an attached thread state. If you need to call Python code from these threads (often this will be part of a callback API provided by the aforementioned third-party library), you must first register these threads with the interpreter by creating a new thread state and attaching it. The easiest way to do this is through :c:func:`PyThreadState_Ensure` or :c:func:`PyThreadState_EnsureFromView`. .. note:: These functions require an argument pointing to the desired interpreter; such a pointer can be acquired via a call to :c:func:`PyInterpreterGuard_FromCurrent` (for ``PyThreadState_Ensure``) or :c:func:`PyInterpreterView_FromCurrent` (for ``PyThreadState_EnsureFromView``) from the function that creates the thread. If no pointer is available (such as when the given native thread library doesn't provide a data argument), :c:func:`PyInterpreterView_FromMain` can be used to get a view for the main interpreter, but note that this will make the code incompatible with subinterpreters. For example:: // The return value of PyInterpreterGuard_FromCurrent() from the // function that created this thread. PyInterpreterGuard *guard = thread_data->guard; // Create a new thread state for the interpreter. PyThreadStateToken *token = PyThreadState_Ensure(guard); if (token == NULL) { PyInterpreterGuard_Close(guard); return; } // We have a valid thread state -- perform Python actions here. result = CallSomeFunction(); // Evaluate result or handle exceptions. // Release the thread state. No calls to the C API are allowed beyond this // point. PyThreadState_Release(token); PyInterpreterGuard_Close(guard); Keep in mind that calling ``PyThreadState_Ensure`` might not always create a new thread state, and calling ``PyThreadState_Release`` might not always detach it. These functions may reuse an existing attached thread state, or may re-attach a thread state that was previously attached for the current thread. .. seealso:: :pep:`788` .. _c-api-reuse-thread-state: Reusing a thread state across repeated calls ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ Creating and destroying a :c:type:`PyThreadState` is not free, and is more expensive on a :term:`free-threaded build`. A foreign thread that calls into the interpreter many times -- for example, a worker thread in a native thread pool -- should avoid creating a fresh thread state on every entry and destroying it on every exit. Instead, set up one thread state when the thread starts (or lazily on its first call into Python), attach and detach it around each call, and tear it down once when the thread exits. Manage the thread state explicitly with :c:func:`PyThreadState_New`, attaching and detaching it with :c:func:`PyEval_RestoreThread` and :c:func:`PyEval_SaveThread`. This happens in three distinct phases, at different points in the thread's life. When the thread starts, create one thread state for it. ``interp`` is the target interpreter, captured by the code that created this thread while it held an attached thread state (for example via :c:func:`PyInterpreterState_Get`):: PyThreadState *tstate = PyThreadState_New(interp); Then, on each call into Python -- which may happen many times over the thread's life -- attach the thread state, make the Python C API calls that require it, and detach again so the thread does not hold the GIL while off doing non-Python work:: PyEval_RestoreThread(tstate); result = CallSomeFunction(); /* your Python C API calls go here */ PyEval_SaveThread(); When the thread is finished calling into Python, destroy the thread state once:: PyEval_RestoreThread(tstate); PyThreadState_Clear(tstate); PyThreadState_DeleteCurrent(); The general-purpose entry points for calling in from a foreign thread -- :c:func:`PyThreadState_Ensure` and the older :c:func:`PyGILState_Ensure` -- do *not* guarantee a persistent thread state: their thread-state lifetime is deliberately implementation-defined, so a matched acquire/release pair may create and destroy a thread state each time. Use :c:func:`PyThreadState_New`, as shown here, whenever you specifically want to reuse one thread state across calls. The code that created the foreign thread must arrange for the shutdown sequence to run before the thread exits, and before :c:func:`Py_FinalizeEx` is called. If interpreter finalization begins first, the shutdown :c:func:`PyEval_RestoreThread` call will hang the thread rather than return (see :ref:`cautions-regarding-runtime-finalization`). If the thread exits without running the shutdown sequence, the thread state is leaked for the remainder of the process. .. _c-api-attach-detach: Attaching/detaching thread states ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ .. c:function:: PyThreadStateToken *PyThreadState_Ensure(PyInterpreterGuard *guard) Ensure that the thread has an attached thread state for the interpreter protected by *guard*, and thus can safely invoke that interpreter. It is OK to call this function if the thread already has an attached thread state, as long as there is a subsequent call to :c:func:`PyThreadState_Release` that matches this one (meaning that "nested" calls to this function are permitted). The function's effect (if any) will be reversed by the matching call to :c:func:`PyThreadState_Release`. On error, this function returns ``NULL`` *without* an exception set. Do not call :c:func:`!PyThreadState_Release` in this case. On success, this function returns a pointer value that must be passed to the matching call to :c:func:`!PyThreadState_Release`. The conditions in which this function creates a new :term:`thread state` are considered unstable and implementation-dependent. If you need to control the exact lifetime of a thread state, consider using :c:func:`PyThreadState_New`. However, do not avoid this function solely on the basis that the lifetime of the thread state may be inconsistent across versions; changes to this function will be done with caution and in a backwards-compatible manner. In particular, the saving of thread-local variables and similar state will be retained across Python versions. .. impl-detail:: The exact behavior of whether this function creates a new thread state is described below, but be aware that this may change in the future. First, this function checks if an attached thread state is present. If there is, this function then checks if the interpreter of that thread state matches the interpreter guarded by *guard*. If that is the case, this function simply marks the thread state as being used by a ``PyThreadState_Ensure`` call and returns. If there is no attached thread state, then this function checks if any thread state has been used by the current OS thread. (This is returned by :c:func:`PyGILState_GetThisThreadState`.) If there was, then this function checks if that thread state's interpreter matches *guard*. If it does, it is re-attached and marked as used. Otherwise, if both of the above cases fail, a new thread state is created for *guard*. It is then attached and marked as owned by ``PyThreadState_Ensure``. .. versionadded:: 3.15 .. c:function:: PyThreadStateToken *PyThreadState_EnsureFromView(PyInterpreterView *view) Get an attached thread state for the interpreter referenced by *view*. The behavior and return value are the same as for :c:func:`PyThreadState_Ensure`; additionally, if the function succeeds, the interpreter referenced by *view* will be implicitly guarded. The guard will be released upon the corresponding :c:func:`PyThreadState_Release` call. .. versionadded:: 3.15 .. c:function:: void PyThreadState_Release(PyThreadStateToken *token) Undo a :c:func:`PyThreadState_Ensure` or :c:func:`PyThreadState_EnsureFromView` call. This must be called exactly once for each successful *Ensure* call, with *token* set to that call's return value. The state that was attached before the corresponding *Ensure* call (if any) will be attached when :c:func:`PyThreadState_Release` returns. The exact behavior of whether this function deletes a thread state is considered unstable and implementation-dependent. .. impl-detail:: Currently, this function will decrement an internal counter on the attached thread state. If this counter ever reaches below zero, this function emits a fatal error (via :c:func:`Py_FatalError`). If the attached thread state is owned by ``PyThreadState_Ensure``, then the attached thread state will be deallocated and deleted upon the internal counter reaching zero. Otherwise, nothing happens when the counter reaches zero. .. versionadded:: 3.15 .. c:type:: PyThreadStateToken An opaque token retrieved from a :c:func:`PyThreadState_Ensure` call and passed to a corresponding :c:func:`PyThreadState_Release` call. .. _legacy-api: .. _gilstate: GIL-state APIs -------------- The following APIs are generally not compatible with subinterpreters and will hang the process during interpreter finalization (see :ref:`cautions-regarding-runtime-finalization`). As such, these APIs were :term:`soft deprecated` in Python 3.15 in favor of the :ref:`new APIs `. .. c:type:: PyGILState_STATE The type of the value returned by :c:func:`PyGILState_Ensure` and passed to :c:func:`PyGILState_Release`. .. c:enumerator:: PyGILState_LOCKED The GIL was already held when :c:func:`PyGILState_Ensure` was called. .. c:enumerator:: PyGILState_UNLOCKED The GIL was not held when :c:func:`PyGILState_Ensure` was called. .. c:function:: PyGILState_STATE PyGILState_Ensure() Ensure that the current thread is ready to call the Python C API regardless of the current state of Python, or of the :term:`attached thread state`. This may be called as many times as desired by a thread as long as each call is matched with a call to :c:func:`PyGILState_Release`. In general, other thread-related APIs may be used between :c:func:`PyGILState_Ensure` and :c:func:`PyGILState_Release` calls as long as the thread state is restored to its previous state before the Release(). For example, normal usage of the :c:macro:`Py_BEGIN_ALLOW_THREADS` and :c:macro:`Py_END_ALLOW_THREADS` macros is acceptable. The return value is an opaque "handle" to the :term:`attached thread state` when :c:func:`PyGILState_Ensure` was called, and must be passed to :c:func:`PyGILState_Release` to ensure Python is left in the same state. Even though recursive calls are allowed, these handles *cannot* be shared - each unique call to :c:func:`PyGILState_Ensure` must save the handle for its call to :c:func:`PyGILState_Release`. When the function returns, there will be an :term:`attached thread state` and the thread will be able to call arbitrary Python code. This function has no way to return an error. As such, errors are either fatal (that is, they send ``SIGABRT`` and crash the process; see :c:func:`Py_FatalError`), or the thread will be permanently blocked (such as during interpreter finalization). .. warning:: Calling this function when the interpreter is finalizing will infinitely hang the thread, which may cause deadlocks. :ref:`cautions-regarding-runtime-finalization` for more details. In addition, this function generally does not work with subinterpreters when used from foreign threads, because this function has no way of knowing which interpreter created the thread (and as such, will implicitly pick the main interpreter). .. versionchanged:: 3.14 Hangs the current thread, rather than terminating it, if called while the interpreter is finalizing. .. soft-deprecated:: 3.15 Use :c:func:`PyThreadState_Ensure` or :c:func:`PyThreadState_EnsureFromView` instead. .. c:function:: void PyGILState_Release(PyGILState_STATE) Release any resources previously acquired. After this call, Python's state will be the same as it was prior to the corresponding :c:func:`PyGILState_Ensure` call (but generally this state will be unknown to the caller, hence the use of the GIL-state API). Every call to :c:func:`PyGILState_Ensure` must be matched by a call to :c:func:`PyGILState_Release` on the same thread. .. soft-deprecated:: 3.15 Use :c:func:`PyThreadState_Release` instead. .. c:function:: PyThreadState* PyGILState_GetThisThreadState() Get the :term:`thread state` that was most recently :term:`attached ` for this thread. (If the most recent thread state has been deleted, this returns ``NULL``.) If the caller has an attached thread state, it is returned. In other terms, this function returns the thread state that will be used by :c:func:`PyGILState_Ensure`. If this returns ``NULL``, then ``PyGILState_Ensure`` will create a new thread state. This function cannot fail. .. soft-deprecated:: 3.15 Use :c:func:`PyThreadState_Get` or :c:func:`PyThreadState_GetUnchecked` instead. .. c:function:: int PyGILState_Check() Return ``1`` if the current thread has an :term:`attached thread state` that matches the thread state returned by :c:func:`PyGILState_GetThisThreadState`. If the caller has no attached thread state or it otherwise doesn't match, then this returns ``0``. If the current Python process has ever created a subinterpreter, this function will *always* return ``1``. This is mainly a helper/diagnostic function. .. versionadded:: 3.4 .. soft-deprecated:: 3.15 Use ``PyThreadState_GetUnchecked() != NULL`` instead. .. _fork-and-threads: Cautions about fork() --------------------- Another important thing to note about threads is their behaviour in the face of the C :c:func:`fork` call. On most systems with :c:func:`fork`, after a process forks only the thread that issued the fork will exist. This has a concrete impact both on how locks must be handled and on all stored state in CPython's runtime. The fact that only the "current" thread remains means any locks held by other threads will never be released. Python solves this for :func:`os.fork` by acquiring the locks it uses internally before the fork, and releasing them afterwards. In addition, it resets any :ref:`lock-objects` in the child. When extending or embedding Python, there is no way to inform Python of additional (non-Python) locks that need to be acquired before or reset after a fork. OS facilities such as :c:func:`!pthread_atfork` would need to be used to accomplish the same thing. Additionally, when extending or embedding Python, calling :c:func:`fork` directly rather than through :func:`os.fork` (and returning to or calling into Python) may result in a deadlock by one of Python's internal locks being held by a thread that is defunct after the fork. :c:func:`PyOS_AfterFork_Child` tries to reset the necessary locks, but is not always able to. The fact that all other threads go away also means that CPython's runtime state there must be cleaned up properly, which :func:`os.fork` does. This means finalizing all other :c:type:`PyThreadState` objects belonging to the current interpreter and all other :c:type:`PyInterpreterState` objects. Due to this and the special nature of the :ref:`"main" interpreter `, :c:func:`fork` should only be called in that interpreter's "main" thread, where the CPython global runtime was originally initialized. The only exception is if :c:func:`exec` will be called immediately after. High-level APIs --------------- These are the most commonly used types and functions when writing multi-threaded C extensions. .. c:type:: PyThreadState This data structure represents the state of a single thread. The only public data member is: .. c:member:: PyInterpreterState *interp This thread's interpreter state. .. c:function:: PyThreadState* PyEval_SaveThread() Detach the :term:`attached thread state` and return it. The thread will have no :term:`thread state` upon returning. .. c:function:: void PyEval_RestoreThread(PyThreadState *tstate) Set the :term:`attached thread state` to *tstate*. The passed :term:`thread state` **should not** be :term:`attached `, otherwise deadlock ensues. *tstate* will be attached upon returning. .. note:: Calling this function from a thread when the runtime is finalizing will hang the thread until the program exits, even if the thread was not created by Python. Refer to :ref:`cautions-regarding-runtime-finalization` for more details. .. versionchanged:: 3.14 Hangs the current thread, rather than terminating it, if called while the interpreter is finalizing. .. c:function:: PyThreadState* PyThreadState_Get() Return the :term:`attached thread state`. If the thread has no attached thread state, (such as when inside of :c:macro:`Py_BEGIN_ALLOW_THREADS` block), then this issues a fatal error (so that the caller needn't check for ``NULL``). See also :c:func:`PyThreadState_GetUnchecked`. .. c:function:: PyThreadState* PyThreadState_GetUnchecked() Similar to :c:func:`PyThreadState_Get`, but don't kill the process with a fatal error if it is NULL. The caller is responsible to check if the result is NULL. .. versionadded:: 3.13 In Python 3.5 to 3.12, the function was private and known as ``_PyThreadState_UncheckedGet()``. .. c:function:: PyThreadState* PyThreadState_Swap(PyThreadState *tstate) Set the :term:`attached thread state` to *tstate*, and return the :term:`thread state` that was attached prior to calling. This function is safe to call without an :term:`attached thread state`; it will simply return ``NULL`` indicating that there was no prior thread state. .. seealso:: :c:func:`PyEval_ReleaseThread` .. note:: Similar to :c:func:`PyGILState_Ensure`, this function will hang the thread if the runtime is finalizing. Low-level APIs -------------- .. c:function:: PyThreadState* PyThreadState_New(PyInterpreterState *interp) Create a new thread state object belonging to the given interpreter object. An :term:`attached thread state` is not needed. .. c:function:: void PyThreadState_Clear(PyThreadState *tstate) Reset all information in a :term:`thread state` object. *tstate* must be :term:`attached ` .. versionchanged:: 3.9 This function now calls the :c:member:`!PyThreadState.on_delete` callback. Previously, that happened in :c:func:`PyThreadState_Delete`. .. versionchanged:: 3.13 The :c:member:`!PyThreadState.on_delete` callback was removed. .. c:function:: void PyThreadState_Delete(PyThreadState *tstate) Destroy a :term:`thread state` object. *tstate* should not be :term:`attached ` to any thread. *tstate* must have been reset with a previous call to :c:func:`PyThreadState_Clear`. .. c:function:: void PyThreadState_DeleteCurrent(void) Detach the :term:`attached thread state` (which must have been reset with a previous call to :c:func:`PyThreadState_Clear`) and then destroy it. No :term:`thread state` will be :term:`attached ` upon returning. .. c:function:: PyFrameObject* PyThreadState_GetFrame(PyThreadState *tstate) Get the current frame of the Python thread state *tstate*. Return a :term:`strong reference`. Return ``NULL`` if no frame is currently executing. See also :c:func:`PyEval_GetFrame`. *tstate* must not be ``NULL``, and must be :term:`attached `. .. versionadded:: 3.9 .. c:function:: uint64_t PyThreadState_GetID(PyThreadState *tstate) Get the unique :term:`thread state` identifier of the Python thread state *tstate*. *tstate* must not be ``NULL``, and must be :term:`attached `. .. versionadded:: 3.9 .. c:function:: PyInterpreterState* PyThreadState_GetInterpreter(PyThreadState *tstate) Get the interpreter of the Python thread state *tstate*. *tstate* must not be ``NULL``, and must be :term:`attached `. .. versionadded:: 3.9 .. c:function:: void PyThreadState_EnterTracing(PyThreadState *tstate) Suspend tracing and profiling in the Python thread state *tstate*. Resume them using the :c:func:`PyThreadState_LeaveTracing` function. .. versionadded:: 3.11 .. c:function:: void PyThreadState_LeaveTracing(PyThreadState *tstate) Resume tracing and profiling in the Python thread state *tstate* suspended by the :c:func:`PyThreadState_EnterTracing` function. See also :c:func:`PyEval_SetTrace` and :c:func:`PyEval_SetProfile` functions. .. versionadded:: 3.11 .. c:function:: int PyUnstable_ThreadState_SetStackProtection(PyThreadState *tstate, void *stack_start_addr, size_t stack_size) Set the stack protection start address and stack protection size of a Python thread state. On success, return ``0``. On failure, set an exception and return ``-1``. CPython implements :ref:`recursion control ` for C code by raising :py:exc:`RecursionError` when it notices that the machine execution stack is close to overflow. See for example the :c:func:`Py_EnterRecursiveCall` function. For this, it needs to know the location of the current thread's stack, which it normally gets from the operating system. When the stack is changed, for example using context switching techniques like the Boost library's ``boost::context``, you must call :c:func:`~PyUnstable_ThreadState_SetStackProtection` to inform CPython of the change. Call :c:func:`~PyUnstable_ThreadState_SetStackProtection` either before or after changing the stack. Do not call any other Python C API between the call and the stack change. See :c:func:`PyUnstable_ThreadState_ResetStackProtection` for undoing this operation. .. versionadded:: 3.15 .. c:function:: void PyUnstable_ThreadState_ResetStackProtection(PyThreadState *tstate) Reset the stack protection start address and stack protection size of a Python thread state to the operating system defaults. See :c:func:`PyUnstable_ThreadState_SetStackProtection` for an explanation. .. versionadded:: 3.15 .. c:function:: PyObject* PyThreadState_GetDict() Return a dictionary in which extensions can store thread-specific state information. Each extension should use a unique key to store a state in the dictionary. It is okay to call this function when no :term:`thread state` is :term:`attached `. If this function returns ``NULL`` and no exception has been raised, then the caller should assume no thread state is attached. .. c:function:: void PyEval_AcquireThread(PyThreadState *tstate) :term:`Attach ` *tstate* to the current thread, which must not be ``NULL`` or already :term:`attached `. The calling thread must not already have an :term:`attached thread state`. .. note:: Calling this function from a thread when the runtime is finalizing will hang the thread until the program exits, even if the thread was not created by Python. Refer to :ref:`cautions-regarding-runtime-finalization` for more details. .. versionchanged:: 3.8 Updated to be consistent with :c:func:`PyEval_RestoreThread`, :c:func:`Py_END_ALLOW_THREADS`, and :c:func:`PyGILState_Ensure`, and terminate the current thread if called while the interpreter is finalizing. .. versionchanged:: 3.14 Hangs the current thread, rather than terminating it, if called while the interpreter is finalizing. :c:func:`PyEval_RestoreThread` is a higher-level function which is always available (even when threads have not been initialized). .. c:function:: void PyEval_ReleaseThread(PyThreadState *tstate) Detach the :term:`attached thread state`. The *tstate* argument, which must not be ``NULL``, is only used to check that it represents the :term:`attached thread state` --- if it isn't, a fatal error is reported. :c:func:`PyEval_SaveThread` is a higher-level function which is always available (even when threads have not been initialized). Asynchronous notifications ========================== A mechanism is provided to make asynchronous notifications to the main interpreter thread. These notifications take the form of a function pointer and a void pointer argument. .. c:function:: int Py_AddPendingCall(int (*func)(void *), void *arg) Schedule a function to be called from the main interpreter thread. On success, ``0`` is returned and *func* is queued for being called in the main thread. On failure, ``-1`` is returned without setting any exception. When successfully queued, *func* will be *eventually* called from the main interpreter thread with the argument *arg*. It will be called asynchronously with respect to normally running Python code, but with both these conditions met: * on a :term:`bytecode` boundary; * with the main thread holding an :term:`attached thread state` (*func* can therefore use the full C API). *func* must return ``0`` on success, or ``-1`` on failure with an exception set. *func* won't be interrupted to perform another asynchronous notification recursively, but it can still be interrupted to switch threads if the :term:`thread state ` is detached. This function doesn't need an :term:`attached thread state`. However, to call this function in a subinterpreter, the caller must have an :term:`attached thread state`. Otherwise, the function *func* can be scheduled to be called from the wrong interpreter. .. warning:: This is a low-level function, only useful for very special cases. There is no guarantee that *func* will be called as quick as possible. If the main thread is busy executing a system call, *func* won't be called before the system call returns. This function is generally **not** suitable for calling Python code from arbitrary C threads. Instead, use :c:func:`PyThreadState_EnsureFromView`. .. versionadded:: 3.1 .. versionchanged:: 3.9 If this function is called in a subinterpreter, the function *func* is now scheduled to be called from the subinterpreter, rather than being called from the main interpreter. Each subinterpreter now has its own list of scheduled calls. .. versionchanged:: 3.12 This function now always schedules *func* to be run in the main interpreter. .. c:function:: int Py_MakePendingCalls(void) Execute all pending calls. This is usually executed automatically by the interpreter. This function returns ``0`` on success, and returns ``-1`` with an exception set on failure. If this is not called in the main thread of the main interpreter, this function does nothing and returns ``0``. The caller must hold an :term:`attached thread state`. .. versionadded:: 3.1 .. versionchanged:: 3.12 This function only runs pending calls in the main interpreter. .. c:function:: int PyThreadState_SetAsyncExc(unsigned long id, PyObject *exc) Schedule an exception to be raised asynchronously in a thread. If the thread has a previously scheduled exception, it is overwritten. The *id* argument is the thread id of the target thread, as returned by :c:func:`PyThread_get_thread_ident`. *exc* is the class of the exception to be raised, or ``NULL`` to clear the pending exception (if any). Return the number of affected thread states. This is normally ``1`` if *id* is found, even when no change was made (the given *exc* was already pending, or *exc* is ``NULL`` but no exception is pending). If the thread id isn't found, return ``0``. This raises no exceptions. To prevent naive misuse, you must write your own C extension to call this. This function must be called with an :term:`attached thread state`. This function does not :term:`steal` any references to *exc*. This function does not necessarily interrupt system calls such as :py:func:`~time.sleep`. .. versionchanged:: 3.7 The type of the *id* parameter changed from :c:expr:`long` to :c:expr:`unsigned long`. Operating system thread APIs ============================ .. c:macro:: PYTHREAD_INVALID_THREAD_ID Sentinel value for an invalid thread ID. This is currently equivalent to ``(unsigned long)-1``. .. c:function:: unsigned long PyThread_start_new_thread(void (*func)(void *), void *arg) Start function *func* in a new thread with argument *arg*. The resulting thread is not intended to be joined. *func* must not be ``NULL``, but *arg* may be ``NULL``. On success, this function returns the identifier of the new thread; on failure, this returns :c:macro:`PYTHREAD_INVALID_THREAD_ID`. The caller does not need to hold an :term:`attached thread state`. .. c:function:: unsigned long PyThread_get_thread_ident(void) Return the identifier of the current thread, which will never be zero. This function cannot fail, and the caller does not need to hold an :term:`attached thread state`. .. seealso:: :py:func:`threading.get_ident` and :py:attr:`threading.Thread.ident` expose this identifier to Python. .. c:function:: PyObject *PyThread_GetInfo(void) Get general information about the current thread in the form of a :ref:`struct sequence ` object. This information is accessible as :py:attr:`sys.thread_info` in Python. On success, this returns a new :term:`strong reference` to the thread information; on failure, this returns ``NULL`` with an exception set. The caller must hold an :term:`attached thread state`. .. c:macro:: PY_HAVE_THREAD_NATIVE_ID This macro is defined when the system supports native thread IDs. .. c:function:: unsigned long PyThread_get_thread_native_id(void) Get the native identifier of the current thread as it was assigned by the operating system's kernel, which will never be less than zero. This function is only available when :c:macro:`PY_HAVE_THREAD_NATIVE_ID` is defined. This function cannot fail, and the caller does not need to hold an :term:`attached thread state`. .. seealso:: :py:func:`threading.get_native_id` .. c:function:: void PyThread_exit_thread(void) Terminate the current thread. This function is generally considered unsafe and should be avoided. It is kept solely for backwards compatibility. This function is only safe to call if all functions in the full call stack are written to safely allow it. .. warning:: If the current system uses POSIX threads (also known as "pthreads"), this calls :manpage:`pthread_exit(3)`, which attempts to unwind the stack and call C++ destructors on some libc implementations. However, if a ``noexcept`` function is reached, it may terminate the process. Other systems, such as macOS, do unwinding. On Windows, this function calls ``_endthreadex()``, which kills the thread without calling C++ destructors. In any case, there is a risk of corruption on the thread's stack. .. deprecated:: 3.14 .. c:function:: void PyThread_init_thread(void) Initialize ``PyThread*`` APIs. Python executes this function automatically, so there's little need to call it from an extension module. .. c:function:: int PyThread_set_stacksize(size_t size) Set the stack size of the current thread to *size* bytes. This function returns ``0`` on success, ``-1`` if *size* is invalid, or ``-2`` if the system does not support changing the stack size. This function does not set exceptions. The caller does not need to hold an :term:`attached thread state`. .. c:function:: size_t PyThread_get_stacksize(void) Return the stack size of the current thread in bytes, or ``0`` if the system's default stack size is in use. The caller does not need to hold an :term:`attached thread state`. --- ### Doc C Api Time (`Doc/c-api/time.rst`) .. highlight:: c .. _c-api-time: PyTime C API ============ .. versionadded:: 3.13 The clock C API provides access to system clocks. It is similar to the Python :mod:`time` module. For C API related to the :mod:`datetime` module, see :ref:`datetimeobjects`. Types ----- .. c:type:: PyTime_t A timestamp or duration in nanoseconds, represented as a signed 64-bit integer. The reference point for timestamps depends on the clock used. For example, :c:func:`PyTime_Time` returns timestamps relative to the UNIX epoch. The supported range is around [-292.3 years; +292.3 years]. Using the Unix epoch (January 1st, 1970) as reference, the supported date range is around [1677-09-21; 2262-04-11]. The exact limits are exposed as constants: .. c:var:: PyTime_t PyTime_MIN Minimum value of :c:type:`PyTime_t`. .. c:var:: PyTime_t PyTime_MAX Maximum value of :c:type:`PyTime_t`. Clock Functions --------------- The following functions take a pointer to a :c:expr:`PyTime_t` that they set to the value of a particular clock. Details of each clock are given in the documentation of the corresponding Python function. The functions return ``0`` on success, or ``-1`` (with an exception set) on failure. On integer overflow, they set the :c:data:`PyExc_OverflowError` exception and set ``*result`` to the value clamped to the ``[PyTime_MIN; PyTime_MAX]`` range. (On current systems, integer overflows are likely caused by misconfigured system time.) As any other C API (unless otherwise specified), the functions must be called with an :term:`attached thread state`. .. c:function:: int PyTime_Monotonic(PyTime_t *result) Read the monotonic clock. See :func:`time.monotonic` for important details on this clock. .. c:function:: int PyTime_PerfCounter(PyTime_t *result) Read the performance counter. See :func:`time.perf_counter` for important details on this clock. .. c:function:: int PyTime_Time(PyTime_t *result) Read the “wall clock” time. See :func:`time.time` for details important on this clock. Raw Clock Functions ------------------- Similar to clock functions, but don't set an exception on error and don't require the caller to have an :term:`attached thread state`. On success, the functions return ``0``. On failure, they set ``*result`` to ``0`` and return ``-1``, *without* setting an exception. To get the cause of the error, :term:`attach ` a :term:`thread state`, and call the regular (non-``Raw``) function. Note that the regular function may succeed after the ``Raw`` one failed. .. c:function:: int PyTime_MonotonicRaw(PyTime_t *result) Similar to :c:func:`PyTime_Monotonic`, but don't set an exception on error and don't require an :term:`attached thread state`. .. c:function:: int PyTime_PerfCounterRaw(PyTime_t *result) Similar to :c:func:`PyTime_PerfCounter`, but don't set an exception on error and don't require an :term:`attached thread state`. .. c:function:: int PyTime_TimeRaw(PyTime_t *result) Similar to :c:func:`PyTime_Time`, but don't set an exception on error and don't require an :term:`attached thread state`. Conversion functions -------------------- .. c:function:: double PyTime_AsSecondsDouble(PyTime_t t) Convert a timestamp to a number of seconds as a C :c:expr:`double`. The function cannot fail, but note that :c:expr:`double` has limited accuracy for large values. --- ### Doc C Api Tls (`Doc/c-api/tls.rst`) .. highlight:: c .. _thread-local-storage: Thread-local storage support ============================ The Python interpreter provides low-level support for thread-local storage (TLS) which wraps the underlying native TLS implementation to support the Python-level thread-local storage API (:class:`threading.local`). The CPython C level APIs are similar to those offered by pthreads and Windows: use a thread key and functions to associate a :c:expr:`void*` value per thread. A :term:`thread state` does *not* need to be :term:`attached ` when calling these functions; they supply their own locking. Note that :file:`Python.h` does not include the declaration of the TLS APIs, you need to include :file:`pythread.h` to use thread-local storage. .. note:: None of these API functions handle memory management on behalf of the :c:expr:`void*` values. You need to allocate and deallocate them yourself. If the :c:expr:`void*` values happen to be :c:expr:`PyObject*`, these functions don't do refcount operations on them either. .. _thread-specific-storage-api: Thread-specific storage API --------------------------- The thread-specific storage (TSS) API was introduced to supersede the use of the existing TLS API within the CPython interpreter. This API uses a new type :c:type:`Py_tss_t` instead of :c:expr:`int` to represent thread keys. .. versionadded:: 3.7 .. seealso:: "A New C-API for Thread-Local Storage in CPython" (:pep:`539`) .. c:type:: Py_tss_t This data structure represents the state of a thread key, the definition of which may depend on the underlying TLS implementation, and it has an internal field representing the key's initialization state. There are no public members in this structure. When :ref:`Py_LIMITED_API ` is not defined, static allocation of this type by :c:macro:`Py_tss_NEEDS_INIT` is allowed. .. c:macro:: Py_tss_NEEDS_INIT This macro expands to the initializer for :c:type:`Py_tss_t` variables. Note that this macro won't be defined with :ref:`Py_LIMITED_API `. Dynamic allocation ------------------ Dynamic allocation of the :c:type:`Py_tss_t`, required in extension modules built with :ref:`Py_LIMITED_API `, where static allocation of this type is not possible due to its implementation being opaque at build time. .. c:function:: Py_tss_t* PyThread_tss_alloc() Return a value which is the same state as a value initialized with :c:macro:`Py_tss_NEEDS_INIT`, or ``NULL`` in the case of dynamic allocation failure. .. c:function:: void PyThread_tss_free(Py_tss_t *key) Free the given *key* allocated by :c:func:`PyThread_tss_alloc`, after first calling :c:func:`PyThread_tss_delete` to ensure any associated thread locals have been unassigned. This is a no-op if the *key* argument is ``NULL``. .. note:: A freed key becomes a dangling pointer. You should reset the key to ``NULL``. Methods ------- The parameter *key* of these functions must not be ``NULL``. Moreover, the behaviors of :c:func:`PyThread_tss_set` and :c:func:`PyThread_tss_get` are undefined if the given :c:type:`Py_tss_t` has not been initialized by :c:func:`PyThread_tss_create`. .. c:function:: int PyThread_tss_is_created(Py_tss_t *key) Return a non-zero value if the given :c:type:`Py_tss_t` has been initialized by :c:func:`PyThread_tss_create`. .. c:function:: int PyThread_tss_create(Py_tss_t *key) Return a zero value on successful initialization of a TSS key. The behavior is undefined if the value pointed to by the *key* argument is not initialized by :c:macro:`Py_tss_NEEDS_INIT`. This function can be called repeatedly on the same key -- calling it on an already initialized key is a no-op and immediately returns success. .. c:function:: void PyThread_tss_delete(Py_tss_t *key) Destroy a TSS key to forget the values associated with the key across all threads, and change the key's initialization state to uninitialized. A destroyed key is able to be initialized again by :c:func:`PyThread_tss_create`. This function can be called repeatedly on the same key -- calling it on an already destroyed key is a no-op. .. c:function:: int PyThread_tss_set(Py_tss_t *key, void *value) Return a zero value to indicate successfully associating a :c:expr:`void*` value with a TSS key in the current thread. Each thread has a distinct mapping of the key to a :c:expr:`void*` value. .. c:function:: void* PyThread_tss_get(Py_tss_t *key) Return the :c:expr:`void*` value associated with a TSS key in the current thread. This returns ``NULL`` if no value is associated with the key in the current thread. .. _thread-local-storage-api: Legacy APIs ----------- .. deprecated:: 3.7 This API is superseded by the :ref:`thread-specific storage (TSS) API `. .. note:: This version of the API does not support platforms where the native TLS key is defined in a way that cannot be safely cast to ``int``. On such platforms, :c:func:`PyThread_create_key` will return immediately with a failure status, and the other TLS functions will all be no-ops on such platforms. Due to the compatibility problem noted above, this version of the API should not be used in new code. .. c:function:: int PyThread_create_key() .. c:function:: void PyThread_delete_key(int key) .. c:function:: int PyThread_set_key_value(int key, void *value) .. c:function:: void* PyThread_get_key_value(int key) .. c:function:: void PyThread_delete_key_value(int key) .. c:function:: void PyThread_ReInitTLS() --- ### Doc C Api Tuple (`Doc/c-api/tuple.rst`) .. highlight:: c .. _tupleobjects: Tuple Objects ------------- .. index:: pair: object; tuple .. c:type:: PyTupleObject This subtype of :c:type:`PyObject` represents a Python tuple object. .. c:var:: PyTypeObject PyTuple_Type This instance of :c:type:`PyTypeObject` represents the Python tuple type; it is the same object as :class:`tuple` in the Python layer. .. c:function:: int PyTuple_Check(PyObject *p) Return true if *p* is a tuple object or an instance of a subtype of the tuple type. This function always succeeds. .. c:function:: int PyTuple_CheckExact(PyObject *p) Return true if *p* is a tuple object, but not an instance of a subtype of the tuple type. This function always succeeds. .. c:function:: PyObject* PyTuple_New(Py_ssize_t len) Return a new tuple object of size *len*, or ``NULL`` with an exception set on failure. .. c:function:: PyObject* PyTuple_FromArray(PyObject *const *array, Py_ssize_t size) Create a tuple of *size* items and copy references from *array* to the new tuple. *array* can be NULL if *size* is ``0``. On success, return a new reference. On error, set an exception and return ``NULL``. .. versionadded:: 3.15 .. c:function:: PyObject* PyTuple_Pack(Py_ssize_t n, ...) Return a new tuple object of size *n*, or ``NULL`` with an exception set on failure. The tuple values are initialized to the subsequent *n* C arguments pointing to Python objects. ``PyTuple_Pack(2, a, b)`` is equivalent to ``Py_BuildValue("(OO)", a, b)``. .. c:function:: Py_ssize_t PyTuple_Size(PyObject *p) Take a pointer to a tuple object, and return the size of that tuple. On error, return ``-1`` with an exception set. .. c:function:: Py_ssize_t PyTuple_GET_SIZE(PyObject *p) Like :c:func:`PyTuple_Size`, but without error checking. .. c:function:: PyObject* PyTuple_GetItem(PyObject *p, Py_ssize_t pos) Return the object at position *pos* in the tuple pointed to by *p*. If *pos* is negative or out of bounds, return ``NULL`` and set an :exc:`IndexError` exception. The returned reference is borrowed from the tuple *p* (that is: it is only valid as long as you hold a reference to *p*). To get a :term:`strong reference`, use :c:func:`Py_NewRef(PyTuple_GetItem(...)) ` or :c:func:`PySequence_GetItem`. .. c:function:: PyObject* PyTuple_GET_ITEM(PyObject *p, Py_ssize_t pos) Like :c:func:`PyTuple_GetItem`, but does no checking of its arguments. .. c:function:: PyObject* PyTuple_GetSlice(PyObject *p, Py_ssize_t low, Py_ssize_t high) Return the slice of the tuple pointed to by *p* between *low* and *high*, or ``NULL`` with an exception set on failure. This is the equivalent of the Python expression ``p[low:high]``. Indexing from the end of the tuple is not supported. .. c:function:: int PyTuple_SetItem(PyObject *p, Py_ssize_t pos, PyObject *o) Insert a reference to object *o* at position *pos* of the tuple pointed to by *p*. Return ``0`` on success. If *pos* is out of bounds, return ``-1`` and set an :exc:`IndexError` exception. This function should only be used to fill in brand new tuples; using it on an existing tuple is thread-unsafe. .. note:: This function ":term:`steals `" a reference to *o* and discards a reference to an item already in the tuple at the affected position (unless it was NULL). .. c:function:: void PyTuple_SET_ITEM(PyObject *p, Py_ssize_t pos, PyObject *o) Like :c:func:`PyTuple_SetItem`, but does no error checking, and should *only* be used to fill in brand new tuples, using it on an existing tuple is thread-unsafe. Bounds checking is performed as an assertion if Python is built in :ref:`debug mode ` or :option:`with assertions <--with-assertions>`. .. note:: This function ":term:`steals `" a reference to *o*, and, unlike :c:func:`PyTuple_SetItem`, does *not* discard a reference to any item that is being replaced; any reference in the tuple at position *pos* will be leaked. .. warning:: This macro should *only* be used on tuples that are newly created. Using this macro on a tuple that is already in use (or in other words, has a refcount > 1) could lead to undefined behavior. .. c:function:: int _PyTuple_Resize(PyObject **p, Py_ssize_t newsize) Can be used to resize a tuple. *newsize* will be the new length of the tuple. Because tuples are *supposed* to be immutable, this should only be used if there is only one reference to the object. Do *not* use this if the tuple may already be known to some other part of the code. The tuple will always grow or shrink at the end. Think of this as destroying the old tuple and creating a new one, only more efficiently. Returns ``0`` on success. Client code should never assume that the resulting value of ``*p`` will be the same as before calling this function. If the object referenced by ``*p`` is replaced, the original ``*p`` is destroyed. On failure, returns ``-1`` and sets ``*p`` to ``NULL``, and raises :exc:`MemoryError` or :exc:`SystemError`. .. _struct-sequence-objects: Struct Sequence Objects ----------------------- A struct sequence object is a :term:`named tuple`, that is, a sequence whose items can also be accessed through attributes. It is similar to :func:`collections.namedtuple`, but provides a slightly different interface. To create a struct sequence, you first have to create a specific struct sequence type. .. c:function:: PyTypeObject* PyStructSequence_NewType(PyStructSequence_Desc *desc) Create a new struct sequence type from the data in *desc*, described below. Instances of the resulting type can be created with :c:func:`PyStructSequence_New`. Return ``NULL`` with an exception set on failure. .. c:function:: void PyStructSequence_InitType(PyTypeObject *type, PyStructSequence_Desc *desc) Initializes a struct sequence type *type* from *desc* in place. .. c:function:: int PyStructSequence_InitType2(PyTypeObject *type, PyStructSequence_Desc *desc) Like :c:func:`PyStructSequence_InitType`, but returns ``0`` on success and ``-1`` with an exception set on failure. .. versionadded:: 3.4 .. c:type:: PyStructSequence_Desc Contains the meta information of a struct sequence type to create. .. c:member:: const char *name Fully qualified name of the type; null-terminated UTF-8 encoded. The name must contain the module name. .. c:member:: const char *doc Pointer to docstring for the type or ``NULL`` to omit. .. c:member:: PyStructSequence_Field *fields Pointer to ``NULL``-terminated array with field names of the new type. .. c:member:: int n_in_sequence Number of fields visible to the Python side (if used as tuple). .. c:type:: PyStructSequence_Field Describes a field of a struct sequence. As a struct sequence is modeled as a tuple, all fields are typed as :c:expr:`PyObject*`. The index in the :c:member:`~PyStructSequence_Desc.fields` array of the :c:type:`PyStructSequence_Desc` determines which field of the struct sequence is described. .. c:member:: const char *name Name for the field or ``NULL`` to end the list of named fields, set to :c:data:`PyStructSequence_UnnamedField` to leave unnamed. .. c:member:: const char *doc Field docstring or ``NULL`` to omit. .. c:var:: const char * const PyStructSequence_UnnamedField Special value for a field name to leave it unnamed. .. versionchanged:: 3.9 The type was changed from ``char *``. .. c:function:: PyObject* PyStructSequence_New(PyTypeObject *type) Creates an instance of *type*, which must have been created with :c:func:`PyStructSequence_NewType`. Return ``NULL`` with an exception set on failure. .. c:function:: PyObject* PyStructSequence_GetItem(PyObject *p, Py_ssize_t pos) Return the object at position *pos* in the struct sequence pointed to by *p*. The returned reference is borrowed from the struct sequence *p* (that is: it is only valid as long as you hold a reference to *p*). Bounds checking is performed as an assertion if Python is built in :ref:`debug mode ` or :option:`with assertions <--with-assertions>`. .. c:function:: PyObject* PyStructSequence_GET_ITEM(PyObject *p, Py_ssize_t pos) Alias to :c:func:`PyStructSequence_GetItem`. .. versionchanged:: 3.13 Now implemented as an alias to :c:func:`PyStructSequence_GetItem`. .. c:function:: void PyStructSequence_SetItem(PyObject *p, Py_ssize_t pos, PyObject *o) Sets the field at index *pos* of the struct sequence *p* to value *o*. Like :c:func:`PyTuple_SET_ITEM`, this should only be used to fill in brand new instances. Bounds checking is performed as an assertion if Python is built in :ref:`debug mode ` or :option:`with assertions <--with-assertions>`. .. note:: This function ":term:`steals `" a reference to *o*. .. c:function:: void PyStructSequence_SET_ITEM(PyObject *p, Py_ssize_t *pos, PyObject *o) Alias to :c:func:`PyStructSequence_SetItem`. .. versionchanged:: 3.13 Now implemented as an alias to :c:func:`PyStructSequence_SetItem`. --- ### Doc C Api Type (`Doc/c-api/type.rst`) .. highlight:: c .. _typeobjects: Type Objects ============ .. index:: pair: object; type .. c:type:: PyTypeObject The C structure of the objects used to describe built-in types. .. c:var:: PyTypeObject PyType_Type This is the type object for type objects; it is the same object as :class:`type` in the Python layer. .. c:function:: int PyType_Check(PyObject *o) Return non-zero if the object *o* is a type object, including instances of types derived from the standard type object. Return 0 in all other cases. This function always succeeds. .. c:function:: int PyType_CheckExact(PyObject *o) Return non-zero if the object *o* is a type object, but not a subtype of the standard type object. Return 0 in all other cases. This function always succeeds. .. c:function:: unsigned int PyType_ClearCache() Clear the internal lookup cache. Return the current version tag. .. versionchanged:: 3.16 This function is now a no-op as the type cache is now implemented per-type. It still returns the current version tag. .. c:function:: unsigned long PyType_GetFlags(PyTypeObject* type) Return the :c:member:`~PyTypeObject.tp_flags` member of *type*. This function is primarily meant for use with ``Py_LIMITED_API``; the individual flag bits are guaranteed to be stable across Python releases, but access to :c:member:`~PyTypeObject.tp_flags` itself is not part of the :ref:`limited API `. .. versionadded:: 3.2 .. versionchanged:: 3.4 The return type is now ``unsigned long`` rather than ``long``. .. c:function:: PyObject* PyType_GetDict(PyTypeObject* type) Return the type object's internal namespace, which is otherwise only exposed via a read-only proxy (:attr:`cls.__dict__ `). This is a replacement for accessing :c:member:`~PyTypeObject.tp_dict` directly. The returned dictionary must be treated as read-only. This function is meant for specific embedding and language-binding cases, where direct access to the dict is necessary and indirect access (e.g. via the proxy or :c:func:`PyObject_GetAttr`) isn't adequate. Extension modules should continue to use ``tp_dict``, directly or indirectly, when setting up their own types. .. versionadded:: 3.12 .. c:function:: void PyType_Modified(PyTypeObject *type) Invalidate the internal lookup cache for the type and all of its subtypes. This function must be called after any manual modification of the attributes or base classes of the type. .. c:function:: int PyType_AddWatcher(PyType_WatchCallback callback) Register *callback* as a type watcher. Return a non-negative integer ID which must be passed to future calls to :c:func:`PyType_Watch`. In case of error (e.g. no more watcher IDs available), return ``-1`` and set an exception. In free-threaded builds, :c:func:`PyType_AddWatcher` is not thread-safe, so it must be called at start up (before spawning the first thread). .. versionadded:: 3.12 .. c:function:: int PyType_ClearWatcher(int watcher_id) Clear watcher identified by *watcher_id* (previously returned from :c:func:`PyType_AddWatcher`). Return ``0`` on success, ``-1`` on error (e.g. if *watcher_id* was never registered.) An extension should never call ``PyType_ClearWatcher`` with a *watcher_id* that was not returned to it by a previous call to :c:func:`PyType_AddWatcher`. .. versionadded:: 3.12 .. c:function:: int PyType_Watch(int watcher_id, PyObject *type) Mark *type* as watched. The callback granted *watcher_id* by :c:func:`PyType_AddWatcher` will be called whenever :c:func:`PyType_Modified` reports a change to *type*. (The callback may be called only once for a series of consecutive modifications to *type*, if :c:func:`!_PyType_Lookup` is not called on *type* between the modifications; this is an implementation detail and subject to change.) The callback is also invoked when a watched heap type is deallocated. An extension should never call ``PyType_Watch`` with a *watcher_id* that was not returned to it by a previous call to :c:func:`PyType_AddWatcher`. .. versionadded:: 3.12 .. versionchanged:: 3.15 The callback is now also invoked when a watched heap type is deallocated. .. c:function:: int PyType_Unwatch(int watcher_id, PyObject *type) Mark *type* as not watched. This undoes a previous call to :c:func:`PyType_Watch`. *type* must not be ``NULL``. An extension should never call this function with a *watcher_id* that was not returned to it by a previous call to :c:func:`PyType_AddWatcher`. On success, this function returns ``0``. On failure, this function returns ``-1`` with an exception set. .. versionadded:: 3.12 .. c:type:: int (*PyType_WatchCallback)(PyObject *type) Type of a type-watcher callback function. The callback must not modify *type* or cause :c:func:`PyType_Modified` to be called on *type* or any type in its MRO; violating this rule could cause infinite recursion. The callback may be called during type deallocation. In this case, the type object is temporarily resurrected (its reference count is at least 1) and all its attributes are still valid. However, the callback should not store new strong references to the type, as this would resurrect the object and prevent its deallocation. .. versionadded:: 3.12 .. versionchanged:: 3.15 The callback may now be called during deallocation of a watched heap type. .. c:function:: int PyType_HasFeature(PyTypeObject *o, int feature) Return non-zero if the type object *o* sets the feature *feature*. Type features are denoted by single bit flags. .. c:function:: int PyType_FastSubclass(PyTypeObject *type, int flag) Return non-zero if the type object *type* sets the subclass flag *flag*. Subclass flags are denoted by :c:macro:`Py_TPFLAGS_*_SUBCLASS `. This function is used by many ``_Check`` functions for common types. .. seealso:: :c:func:`PyObject_TypeCheck`, which is used as a slower alternative in ``_Check`` functions for types that don't come with subclass flags. .. c:function:: int PyType_IS_GC(PyTypeObject *o) Return true if the type object includes support for the cycle detector; this tests the type flag :c:macro:`Py_TPFLAGS_HAVE_GC`. .. c:function:: int PyType_IsSubtype(PyTypeObject *a, PyTypeObject *b) Return true if *a* is a subtype of *b*. This function only checks for actual subtypes, which means that :meth:`~type.__subclasscheck__` is not called on *b*. Call :c:func:`PyObject_IsSubclass` to do the same check that :func:`issubclass` would do. .. c:function:: PyObject* PyType_GenericAlloc(PyTypeObject *type, Py_ssize_t nitems) Generic handler for the :c:member:`~PyTypeObject.tp_alloc` slot of a type object. Uses Python's default memory allocation mechanism to allocate memory for a new instance, zeros the memory, then initializes the memory as if by calling :c:func:`PyObject_Init` or :c:func:`PyObject_InitVar`. Do not call this directly to allocate memory for an object; call the type's :c:member:`~PyTypeObject.tp_alloc` slot instead. For types that support garbage collection (i.e., the :c:macro:`Py_TPFLAGS_HAVE_GC` flag is set), this function behaves like :c:macro:`PyObject_GC_New` or :c:macro:`PyObject_GC_NewVar` (except the memory is guaranteed to be zeroed before initialization), and should be paired with :c:func:`PyObject_GC_Del` in :c:member:`~PyTypeObject.tp_free`. Otherwise, it behaves like :c:macro:`PyObject_New` or :c:macro:`PyObject_NewVar` (except the memory is guaranteed to be zeroed before initialization) and should be paired with :c:func:`PyObject_Free` in :c:member:`~PyTypeObject.tp_free`. .. c:function:: PyObject* PyType_GenericNew(PyTypeObject *type, PyObject *args, PyObject *kwds) Generic handler for the :c:member:`~PyTypeObject.tp_new` slot of a type object. Creates a new instance using the type's :c:member:`~PyTypeObject.tp_alloc` slot and returns the resulting object. .. c:function:: int PyType_Ready(PyTypeObject *type) Finalize a type object. This should be called on all type objects to finish their initialization. This function is responsible for adding inherited slots from a type's base class. Return ``0`` on success, or return ``-1`` and sets an exception on error. .. note:: If some of the base classes implements the GC protocol and the provided type does not include the :c:macro:`Py_TPFLAGS_HAVE_GC` in its flags, then the GC protocol will be automatically implemented from its parents. On the contrary, if the type being created does include :c:macro:`Py_TPFLAGS_HAVE_GC` in its flags then it **must** implement the GC protocol itself by at least implementing the :c:member:`~PyTypeObject.tp_traverse` handle. .. c:function:: PyObject* PyType_GetName(PyTypeObject *type) Return the type's name. Equivalent to getting the type's :attr:`~type.__name__` attribute. .. versionadded:: 3.11 .. c:function:: PyObject* PyType_GetQualName(PyTypeObject *type) Return the type's qualified name. Equivalent to getting the type's :attr:`~type.__qualname__` attribute. .. versionadded:: 3.11 .. c:function:: PyObject* PyType_GetFullyQualifiedName(PyTypeObject *type) Return the type's fully qualified name. Equivalent to ``f"{type.__module__}.{type.__qualname__}"``, or :attr:`type.__qualname__` if :attr:`type.__module__` is not a string or is equal to ``"builtins"``. .. versionadded:: 3.13 .. c:function:: PyObject* PyType_GetModuleName(PyTypeObject *type) Return the type's module name. Equivalent to getting the :attr:`type.__module__` attribute. .. versionadded:: 3.13 .. c:function:: void* PyType_GetSlot(PyTypeObject *type, int slot) Return the function pointer stored in the given slot. If the result is ``NULL``, this indicates that either the slot is ``NULL``, or that the function was called with invalid parameters. Callers will typically cast the result pointer into the appropriate function type. See :c:member:`PyType_Slot.slot` for possible values of the *slot* argument. .. versionadded:: 3.4 .. versionchanged:: 3.10 :c:func:`PyType_GetSlot` can now accept all types. Previously, it was limited to :ref:`heap types `. .. c:function:: PyObject* PyType_GetModule(PyTypeObject *type) Return the module object associated with the given type when the type was created using :c:func:`PyType_FromModuleAndSpec`. The returned reference is :term:`borrowed ` from *type*, and will be valid as long as you hold a reference to *type*. Do not release it with :c:func:`Py_DECREF` or similar. If no module is associated with the given type, sets :py:class:`TypeError` and returns ``NULL``. This function is usually used to get the module in which a method is defined. Note that in such a method, ``PyType_GetModule(Py_TYPE(self))`` may not return the intended result. ``Py_TYPE(self)`` may be a *subclass* of the intended class, and subclasses are not necessarily defined in the same module as their superclass. See :c:type:`PyCMethod` to get the class that defines the method. See :c:func:`PyType_GetModuleByToken` for cases when :c:type:`!PyCMethod` cannot be used. .. versionadded:: 3.9 .. c:function:: void* PyType_GetModuleState(PyTypeObject *type) Return the state of the module object associated with the given type. This is a shortcut for calling :c:func:`PyModule_GetState()` on the result of :c:func:`PyType_GetModule`. If no module is associated with the given type, sets :py:class:`TypeError` and returns ``NULL``. If the *type* has an associated module but its state is ``NULL``, returns ``NULL`` without setting an exception. .. versionadded:: 3.9 .. c:function:: PyObject* PyType_GetModuleByToken(PyTypeObject *type, const void *mod_token) Find the first superclass whose module has the given :ref:`module token `, and return that module. If no module is found, raises a :py:class:`TypeError` and returns ``NULL``. This function is intended to be used together with :c:func:`PyModule_GetState()` to get module state from slot methods (such as :c:member:`~PyTypeObject.tp_init` or :c:member:`~PyNumberMethods.nb_add`) and other places where a method's defining class cannot be passed using the :c:type:`PyCMethod` calling convention. .. versionadded:: 3.15 .. c:function:: PyObject* PyType_GetModuleByDef(PyTypeObject *type, struct PyModuleDef *def) Find the first superclass whose module was created from the given :c:type:`PyModuleDef` *def*, or whose :ref:`module token ` is equal to *def*, and return that module. Note that modules created from a :c:type:`PyModuleDef` always have their token set to the :c:type:`PyModuleDef`'s address. In other words, this function is equivalent to :c:func:`PyType_GetModuleByToken`, except that it: - returns a borrowed reference, and - has a non-``void*`` argument type (which is a cosmetic difference in C). The returned reference is :term:`borrowed ` from *type*, and will be valid as long as you hold a reference to *type*. Do not release it with :c:func:`Py_DECREF` or similar. .. versionadded:: 3.11 .. c:function:: int PyType_GetBaseByToken(PyTypeObject *type, void *tp_token, PyTypeObject **result) Find the first superclass in *type*'s :term:`method resolution order` whose :c:macro:`Py_tp_token` token is equal to *tp_token*. * If found, set *\*result* to a new :term:`strong reference` to it and return ``1``. * If not found, set *\*result* to ``NULL`` and return ``0``. * On error, set *\*result* to ``NULL`` and return ``-1`` with an exception set. The *result* argument may be ``NULL``, in which case *\*result* is not set. Use this if you need only the return value. The *tp_token* argument may not be ``NULL``. .. versionadded:: 3.14 .. c:function:: int PyUnstable_Type_AssignVersionTag(PyTypeObject *type) Attempt to assign a version tag to the given type. Returns 1 if the type already had a valid version tag or a new one was assigned, or 0 if a new tag could not be assigned. .. versionadded:: 3.12 .. c:function:: int PyType_SUPPORTS_WEAKREFS(PyTypeObject *type) Return true if instances of *type* support creating weak references, false otherwise. This function always succeeds. *type* must not be ``NULL``. .. seealso:: * :ref:`weakrefobjects` * :py:mod:`weakref` .. _creating-heap-types: Creating Heap-Allocated Types ----------------------------- The following function is used to create :ref:`heap types `: .. c:function:: PyObject *PyType_FromSlots(const PySlot *slots) Create and return a :ref:`heap type ` from a :c:type:`!PySlot` array. See :ref:`capi-slots` for general information on slots, and :ref:`pyslot_type_slot_ids` for slots specific to type creation. This function calls :c:func:`PyType_Ready` on the new type. Note that this function does *not* fully match the behavior of calling :py:class:`type() ` or using the :keyword:`class` statement. With user-provided base types or metaclasses, prefer :ref:`calling ` :py:class:`type` (or the metaclass) over ``PyType_From*`` functions. Specifically: * :py:meth:`~object.__new__` is not called on the new class (and it must be set to ``type.__new__``). * :py:meth:`~object.__init__` is not called on the new class. * :py:meth:`~object.__init_subclass__` is not called on any bases. * :py:meth:`~object.__set_name__` is not called on new descriptors. Slots are typically defined as a global static constant arrays. However, sometimes slot values are not statically known at compile time. For example, slots like :c:data:`Py_tp_bases`, :c:data:`Py_tp_metaclass` and :c:data:`Py_tp_module` require live Python objects. In this case, it is recommended to put such slots on the stack, and use :c:macro:`Py_slot_subslots` to refer to an array of static slots. For example:: static const PySlot my_slots[] = { PySlot_STATIC_DATA(Py_tp_name, "MyClass"), PySlot_FUNC(Py_tp_repr, my_repr_func), ... PySlot_END }; PyObject *make_my_class(PyObject *module) { PySlot all_slots[] = { PySlot_STATIC_DATA(Py_slot_subslots, my_slots), PySlot_DATA(Py_tp_module, module), PySlot_END }; return PyType_FromSlots(all_slots); } Heap types created without the :c:macro:`Py_TPFLAGS_IMMUTABLETYPE` flag may be modified, for example by setting attributes on them, as with classes defined in Python code. Sometimes, such modifications are necessary to fully initialize a type, but you may wish to prevent users from changing the type after the initialization is done: .. c:function:: int PyType_Freeze(PyTypeObject *type) Make a type immutable: set the :c:macro:`Py_TPFLAGS_IMMUTABLETYPE` flag. All base classes of *type* must be immutable. On success, return ``0``. On error, set an exception and return ``-1``. The type must not be used before it's made immutable. For example, type instances must not be created before the type is made immutable. .. versionadded:: 3.14 .. _pyslot_type_slot_ids: Type slot IDs ............. Most type slot IDs are named like the field names of the structures :c:type:`PyTypeObject`, :c:type:`PyNumberMethods`, :c:type:`PySequenceMethods`, :c:type:`PyMappingMethods` and :c:type:`PyAsyncMethods` with an added ``Py_`` prefix. For example, use: * :c:data:`Py_tp_dealloc` to set :c:member:`PyTypeObject.tp_dealloc` * :c:data:`Py_nb_add` to set :c:member:`PyNumberMethods.nb_add` * :c:data:`Py_sq_length` to set :c:member:`PySequenceMethods.sq_length` The following slots need additional considerations when specified as slots: * :c:data:`Py_tp_name` * :c:data:`Py_tp_basicsize` and :c:data:`Py_tp_extra_basicsize` * :c:data:`Py_tp_itemsize` * :c:data:`Py_tp_flags` Additional slots do not directly correspond to a :c:type:`!PyTypeObject` struct field: * :c:data:`Py_tp_token` * :c:data:`Py_tp_metaclass` * :c:data:`Py_tp_module` The following “offset” fields cannot be set using :c:type:`PyType_Slot`: * :c:member:`~PyTypeObject.tp_weaklistoffset` (use :c:macro:`Py_TPFLAGS_MANAGED_WEAKREF` instead if possible) * :c:member:`~PyTypeObject.tp_dictoffset` (use :c:macro:`Py_TPFLAGS_MANAGED_DICT` instead if possible) * :c:member:`~PyTypeObject.tp_vectorcall_offset` (use ``"__vectorcalloffset__"`` in :ref:`PyMemberDef `) If it is not possible to switch to a ``MANAGED`` flag (for example, for vectorcall or to support Python older than 3.12), specify the offset in :c:data:`Py_tp_members`. See :ref:`PyMemberDef documentation ` for details. The following internal fields cannot be set at all when creating a heap type: * :c:member:`~PyTypeObject.tp_dict`, :c:member:`~PyTypeObject.tp_mro`, :c:member:`~PyTypeObject.tp_cache`, :c:member:`~PyTypeObject.tp_subclasses`, and :c:member:`~PyTypeObject.tp_weaklist`. The :c:data:`Py_tp_base` slot is equivalent to :c:data:`Py_tp_bases`; both may be set either to a type or a tuple of types. If both are specified, the value of :c:data:`Py_tp_bases` is used. Slot values may not be ``NULL``, except for the following: * :c:data:`Py_tp_doc` * :c:data:`Py_tp_token` (for clarity, prefer :c:data:`Py_TP_USE_SPEC` rather than ``NULL``) .. versionchanged:: 3.9 Slots in :c:type:`PyBufferProcs` may be set in the unlimited API. .. versionchanged:: 3.11 :c:member:`~PyBufferProcs.bf_getbuffer` and :c:member:`~PyBufferProcs.bf_releasebuffer` are now available under the :ref:`limited API `. .. versionchanged:: 3.14 The field :c:member:`~PyTypeObject.tp_vectorcall` can now be set using :c:data:`Py_tp_vectorcall`. See the field's documentation for details. .. versionchanged:: 3.15 The :c:data:`Py_tp_bases` slot may be set to a single type object, making it equivalent to the :c:data:`Py_tp_base` slot. Previously, a tuple of types was required. The following slots correspond to fields in the underlying type structure, but need extra remarks for use as slots: .. c:macro:: Py_tp_name :c:member:`Slot ID ` for the name of the type, used to set :c:member:`PyTypeObject.tp_name`. This slot (or :c:member:`PyType_Spec.name`) is required to create a type. This may not be used in :c:member:`PyType_Spec.slots`. Use :c:member:`PyType_Spec.name` instead. .. impl-detail:: CPython processes slots in order. It is recommended to put ``Py_tp_name`` at the beginning of the slots array, so that if processing of a later slots fails, error messages can include the name. .. versionadded:: 3.15 .. c:macro:: Py_tp_basicsize :c:member:`Slot ID ` for the size of the instance in bytes. It is used to set :c:member:`PyTypeObject.tp_basicsize`. The value must be positive. This may not be used in :c:member:`PyType_Spec.slots`. Use :c:member:`PyType_Spec.basicsize` instead. This slot may not be used with :c:func:`PyType_GetSlot`. Use :c:member:`PyTypeObject.tp_basicsize` instead if needed, but be aware that a type's size is often considered an implementation detail. .. versionadded:: 3.15 .. c:macro:: Py_tp_extra_basicsize :c:member:`Slot ID ` for type data size in bytes, that is, how much space instances of the class need *in addition* to space needed for superclasses. The value is used, together with the size of superclasses, to set :c:member:`PyTypeObject.tp_basicsize`. Python will insert padding as needed to meet :c:member:`!tp_basicsize`'s alignment requirements. Use :c:func:`PyObject_GetTypeData` to get a pointer to subclass-specific memory reserved this way. The value must be positive. To specify that instances need no additional size (that is, size should be inherited), omit the :c:macro:`!Py_tp_extra_basicsize` slot rather than set it to zero. Specifying both :c:macro:`Py_tp_basicsize` and :c:macro:`!Py_tp_extra_basicsize` is an error. This may not be used in :c:member:`PyType_Spec.slots`. Use negative :c:member:`PyType_Spec.basicsize` instead. This slot may not be used with :c:func:`PyType_GetSlot`. .. versionadded:: 3.15 .. c:macro:: Py_tp_itemsize :c:member:`Slot ID ` for the size of one element of a variable-size type, in bytes. Used to set :c:member:`PyTypeObject.tp_itemsize`. See :c:member:`!tp_itemsize` documentation for caveats. The value must be positive. If this slot is missing, :c:member:`~PyTypeObject.tp_itemsize` is inherited. Extending arbitrary variable-sized classes is dangerous, since some types use a fixed offset for variable-sized memory, which can then overlap fixed-sized memory used by a subclass. To help prevent mistakes, inheriting ``itemsize`` is only possible in the following situations: - The base is not variable-sized (its :c:member:`~PyTypeObject.tp_itemsize` is zero). - The requested :c:member:`PyType_Spec.basicsize` is positive, suggesting that the memory layout of the base class is known. - The requested :c:member:`PyType_Spec.basicsize` is zero, suggesting that the subclass does not access the instance's memory directly. - With the :c:macro:`Py_TPFLAGS_ITEMS_AT_END` flag. This may not be used in :c:member:`PyType_Spec.slots`. Use :c:member:`PyType_Spec.itemsize` instead. This slot may not be used with :c:func:`PyType_GetSlot`. .. versionadded:: 3.15 .. c:macro:: Py_tp_flags :c:member:`Slot ID ` for type flags, used to set :c:member:`PyTypeObject.tp_flags`. The ``Py_TPFLAGS_HEAPTYPE`` flag is not set, :c:func:`PyType_FromSpecWithBases` sets it automatically. This may not be used in :c:member:`PyType_Spec.slots`. Use negative :c:member:`PyType_Spec.basicsize` instead. This slot may not be used with :c:func:`PyType_GetSlot`. Use :c:func:`PyType_GetFlags` instead. .. versionadded:: 3.15 .. c:macro:: Py_tp_bases :c:member:`Slot ID ` for type flags, used to set :c:member:`PyTypeObject.tp_bases`. The slot can be set to a tuple of type objects which the newly created type should inherit from, like the "positional arguments" of a Python :ref:`class definition `. Alternately, the slot can be set to a single type object to specify a single base. The effect is the same as specifying a one-element tuple. .. versionchanged:: 3.15 Previously, :c:macro:`!Py_tp_bases` required a tuple of types. .. c:macro:: Py_tp_base Equivalent to :c:macro:`Py_tp_bases` (with ``s`` at the end). If both are specified, :c:macro:`!Py_tp_bases` takes priority and this slot is ignored. .. versionchanged:: 3.15 Previously, :c:macro:`!Py_tp_base` required a single type, not a tuple. .. soft-deprecated:: 3.15 When not targeting older Python versions, prefer :c:macro:`!Py_tp_bases`. The following slots do not correspond to public fields in the underlying structures: .. c:macro:: Py_tp_metaclass :c:member:`Slot ID ` for the metaclass used to construct the resulting type object. When omitted the metaclass is derived from bases (:c:macro:`Py_tp_bases` or the *bases* argument of :c:func:`PyType_FromMetaclass`). Metaclasses that override :c:member:`~PyTypeObject.tp_new` are not supported, except if ``tp_new`` is ``NULL``. This may not be used in :c:member:`PyType_Spec.slots`. Use :c:func:`PyType_FromMetaclass` to specify a metaclass with :c:type:`!PyType_Spec`. This slot may not be used with :c:func:`PyType_GetSlot`. Use :c:func:`Py_TYPE` on the type object instead. .. versionadded:: 3.15 .. c:macro:: Py_tp_module :c:member:`Slot ID ` for recording the module in which the new class is defined. The value must be a module object. The module is associated with the new type and can later be retrieved with :c:func:`PyType_GetModule`. The associated module is not inherited by subclasses; it must be specified for each class individually. This may not be used in :c:member:`PyType_Spec.slots`. Use :c:func:`PyType_FromMetaclass` to specify a module with :c:type:`!PyType_Spec`. This slot may not be used with :c:func:`PyType_GetSlot`. Use :c:func:`PyType_GetModule` instead. .. versionadded:: 3.15 .. c:macro:: Py_tp_token :c:member:`Slot ID ` for recording a static memory layout ID for a class. If the class is defined using a :c:type:`PyType_Spec`, and that spec is statically allocated, the token can be set to the spec using the special value :c:data:`Py_TP_USE_SPEC`: .. code-block:: c static PyType_Slot foo_slots[] = { {Py_tp_token, Py_TP_USE_SPEC}, It can also be set to an arbitrary pointer, but you must ensure that: * The pointer outlives the class, so it's not reused for something else while the class exists. * It "belongs" to the extension module where the class lives, so it will not clash with other extensions. Use :c:func:`PyType_GetBaseByToken` to check if a class's superclass has a given token -- that is, check whether the memory layout is compatible. To get the token for a given class (without considering superclasses), use :c:func:`PyType_GetSlot` with ``Py_tp_token``. .. versionadded:: 3.14 .. c:namespace:: NULL .. c:macro:: Py_TP_USE_SPEC Used as a value with :c:data:`Py_tp_token` to set the token to the class's :c:type:`PyType_Spec`. May only be used for classes defined using :c:type:`!PyType_Spec`. Expands to ``NULL``. .. versionadded:: 3.14 .. c:macro:: Py_tp_slots :c:member:`Slot ID ` that works like :c:macro:`Py_slot_subslots`, except it specifies an array of :c:type:`PyType_Slot` structures. .. versionadded:: 3.15 Soft-deprecated API ------------------- The following functions are :term:`soft deprecated`. They will continue to work, but new features will be added as slots for :c:func:`PyType_FromSlots`, not as arguments to new ``PyType_From*`` functions. .. c:function:: PyObject* PyType_FromMetaclass(PyTypeObject *metaclass, PyObject *module, PyType_Spec *spec, PyObject *bases) Create and return a :ref:`heap type ` from the *spec* (see :c:macro:`Py_TPFLAGS_HEAPTYPE`). A non-``NULL`` *metaclass* argument corresponds to the :c:macro:`Py_tp_metaclass` slot. A non-``NULL`` *bases* argument corresponds to the :c:data:`Py_tp_bases` slot, and takes precedence over :c:data:`Py_tp_bases` and :c:data:`Py_tp_bases` slots. A non-``NULL`` *module* argument corresponds to the :c:macro:`Py_tp_module` slot. This function calls :c:func:`PyType_Ready` on the new type. Note that this function does *not* fully match the behavior of calling :py:class:`type() ` or using the :keyword:`class` statement. See the note in :c:func:`PyType_FromSlots` documentation for details. .. versionadded:: 3.12 .. soft-deprecated:: 3.15 Prefer :c:func:`PyType_FromSlots` in new code. .. c:function:: PyObject* PyType_FromModuleAndSpec(PyObject *module, PyType_Spec *spec, PyObject *bases) Equivalent to ``PyType_FromMetaclass(NULL, module, spec, bases)``. .. versionadded:: 3.9 .. versionchanged:: 3.10 The function now accepts a single class as the *bases* argument and ``NULL`` as the ``tp_doc`` slot. .. versionchanged:: 3.12 The function now finds and uses a metaclass corresponding to the provided base classes. Previously, only :class:`type` instances were returned. The :c:member:`~PyTypeObject.tp_new` of the metaclass is *ignored*. which may result in incomplete initialization. Creating classes whose metaclass overrides :c:member:`~PyTypeObject.tp_new` is deprecated. .. versionchanged:: 3.14 Creating classes whose metaclass overrides :c:member:`~PyTypeObject.tp_new` is no longer allowed. .. soft-deprecated:: 3.15 Prefer :c:func:`PyType_FromSlots` in new code. .. c:function:: PyObject* PyType_FromSpecWithBases(PyType_Spec *spec, PyObject *bases) Equivalent to ``PyType_FromMetaclass(NULL, NULL, spec, bases)``. .. versionadded:: 3.3 .. versionchanged:: 3.12 The function now finds and uses a metaclass corresponding to the provided base classes. Previously, only :class:`type` instances were returned. The :c:member:`~PyTypeObject.tp_new` of the metaclass is *ignored*. which may result in incomplete initialization. Creating classes whose metaclass overrides :c:member:`~PyTypeObject.tp_new` is deprecated. .. versionchanged:: 3.14 Creating classes whose metaclass overrides :c:member:`~PyTypeObject.tp_new` is no longer allowed. .. soft-deprecated:: 3.15 Prefer :c:func:`PyType_FromSlots` in new code. .. c:function:: PyObject* PyType_FromSpec(PyType_Spec *spec) Equivalent to ``PyType_FromMetaclass(NULL, NULL, spec, NULL)``. .. versionchanged:: 3.12 The function now finds and uses a metaclass corresponding to the base classes provided in *Py_tp_base[s]* slots. Previously, only :class:`type` instances were returned. The :c:member:`~PyTypeObject.tp_new` of the metaclass is *ignored*. which may result in incomplete initialization. Creating classes whose metaclass overrides :c:member:`~PyTypeObject.tp_new` is deprecated. .. versionchanged:: 3.14 Creating classes whose metaclass overrides :c:member:`~PyTypeObject.tp_new` is no longer allowed. .. soft-deprecated:: 3.15 Prefer :c:func:`PyType_FromSlots` in new code. .. raw:: html .. c:type:: PyType_Spec Structure defining a type's behavior, used for soft-deprecated functions like :c:func:`PyType_FromMetaclass`. This structure contains several members that can instead be specified as :ref:`slots ` for :c:func:`PyType_FromSlots`, and an array of slot entries with a simpler structure. .. c:member:: const char* name Corresponds to :c:macro:`Py_tp_name`. .. c:member:: int basicsize If positive, corresponds to :c:macro:`Py_tp_basicsize`. If negative, corresponds to :c:macro:`Py_tp_extra_basicsize` set to the absolute value. .. versionchanged:: 3.12 Previously, this field could not be negative. .. c:member:: int itemsize Corresponds to :c:macro:`Py_tp_itemsize`. .. c:member:: unsigned int flags Corresponds to :c:macro:`Py_tp_flags`. .. c:member:: PyType_Slot *slots Array of :c:type:`PyType_Slot` (not :c:type:`PySlot`) structures. Terminated by the special slot value ``{0, NULL}``. Each slot ID should be specified at most once. .. c:namespace:: NULL .. raw:: html .. c:type:: PyType_Slot Structure defining optional functionality of a type, used for soft-deprecated functions like :c:func:`PyType_FromMetaclass`. Note that a :c:type:`!PyType_Slot` array may be included in a :c:type:`!PySlot` array using :c:macro:`Py_tp_slots`, and vice versa using :c:macro:`Py_slot_subslots`. Each :c:type:`!PyType_Slot` structure ``tpslot`` is interpreted as the following :c:type:`PySlot` structure:: (PySlot){ .sl_id=tpslot.slot, .sl_flags=PySlot_INTPTR | sub_static, .sl_ptr=tpslot.func } where ``sub_static`` is ``PySlot_STATIC`` if the slot requires the flag (such as for :c:macro:`Py_tp_methods`), or if this flag is present on the "parent" :c:macro:`!Py_tp_slots` slot (if any). .. c:member:: int slot Corresponds to :c:member:`PySlot.sl_id`. .. c:member:: void *pfunc Corresponds to :c:member:`PySlot.sl_ptr`. --- ### Doc C Api Typehints (`Doc/c-api/typehints.rst`) .. highlight:: c .. _typehintobjects: Objects for Type Hinting ------------------------ Various built-in types for type hinting are provided. Currently, two types exist -- :ref:`GenericAlias ` and :ref:`Union `. Only ``GenericAlias`` is exposed to C. .. c:function:: PyObject* Py_GenericAlias(PyObject *origin, PyObject *args) Create a :ref:`GenericAlias ` object. Equivalent to calling the Python class :class:`types.GenericAlias`. The *origin* and *args* arguments set the ``GenericAlias``\ 's ``__origin__`` and ``__args__`` attributes respectively. *origin* should be a :c:expr:`PyTypeObject*`, and *args* can be a :c:expr:`PyTupleObject*` or any ``PyObject*``. If *args* passed is not a tuple, a 1-tuple is automatically constructed and ``__args__`` is set to ``(args,)``. Minimal checking is done for the arguments, so the function will succeed even if *origin* is not a type. The ``GenericAlias``\ 's ``__parameters__`` attribute is constructed lazily from ``__args__``. On failure, an exception is raised and ``NULL`` is returned. Here's an example of how to make an extension type generic:: ... static PyMethodDef my_obj_methods[] = { // Other methods. ... {"__class_getitem__", Py_GenericAlias, METH_O|METH_CLASS, "my_obj is generic over its contained type"} ... } .. seealso:: The data model method :meth:`~object.__class_getitem__`. .. versionadded:: 3.9 .. c:var:: PyTypeObject Py_GenericAliasType The C type of the object returned by :c:func:`Py_GenericAlias`. Equivalent to :class:`types.GenericAlias` in Python. .. versionadded:: 3.9 --- ### Doc C Api Typeobj (`Doc/c-api/typeobj.rst`) .. highlight:: c .. _type-structs: Type Object Structures ====================== Perhaps one of the most important structures of the Python object system is the structure that defines a new type: the :c:type:`PyTypeObject` structure. Type objects can be handled using any of the ``PyObject_*`` or ``PyType_*`` functions, but do not offer much that's interesting to most Python applications. These objects are fundamental to how objects behave, so they are very important to the interpreter itself and to any extension module that implements new types. Type objects are fairly large compared to most of the standard types. The reason for the size is that each type object stores a large number of values, mostly C function pointers, each of which implements a small part of the type's functionality. The fields of the type object are examined in detail in this section. The fields will be described in the order in which they occur in the structure. In addition to the following quick reference, the :ref:`typedef-examples` section provides at-a-glance insight into the meaning and use of :c:type:`PyTypeObject`. Quick Reference --------------- .. _tp-slots-table: "tp slots" ^^^^^^^^^^ .. table:: :widths: 18,18,18,1,1,1,1 +------------------------------------------------+-----------------------------------+-------------------+---------------+ | PyTypeObject Slot [#slots]_ | :ref:`Type ` | special | Info [#cols]_ | | | | methods/attrs +---+---+---+---+ | | | | O | T | D | I | +================================================+===================================+===================+===+===+===+===+ | :c:member:`~PyTypeObject.tp_name` | const char * | __name__ | X | X | | | +------------------------------------------------+-----------------------------------+-------------------+---+---+---+---+ | :c:member:`~PyTypeObject.tp_basicsize` | :c:type:`Py_ssize_t` | | X | X | | X | +------------------------------------------------+-----------------------------------+-------------------+---+---+---+---+ | :c:member:`~PyTypeObject.tp_itemsize` | :c:type:`Py_ssize_t` | | | X | | X | +------------------------------------------------+-----------------------------------+-------------------+---+---+---+---+ | :c:member:`~PyTypeObject.tp_dealloc` | :c:type:`destructor` | | X | X | | X | +------------------------------------------------+-----------------------------------+-------------------+---+---+---+---+ | :c:member:`~PyTypeObject.tp_vectorcall_offset` | :c:type:`Py_ssize_t` | | | X | | X | +------------------------------------------------+-----------------------------------+-------------------+---+---+---+---+ | (:c:member:`~PyTypeObject.tp_getattr`) | :c:type:`getattrfunc` | __getattribute__, | | | | G | | | | __getattr__ | | | | | +------------------------------------------------+-----------------------------------+-------------------+---+---+---+---+ | (:c:member:`~PyTypeObject.tp_setattr`) | :c:type:`setattrfunc` | __setattr__, | | | | G | | | | __delattr__ | | | | | +------------------------------------------------+-----------------------------------+-------------------+---+---+---+---+ | :c:member:`~PyTypeObject.tp_as_async` | :c:type:`PyAsyncMethods` * | :ref:`sub-slots` | | | | % | +------------------------------------------------+-----------------------------------+-------------------+---+---+---+---+ | :c:member:`~PyTypeObject.tp_repr` | :c:type:`reprfunc` | __repr__ | X | X | | X | +------------------------------------------------+-----------------------------------+-------------------+---+---+---+---+ | :c:member:`~PyTypeObject.tp_as_number` | :c:type:`PyNumberMethods` * | :ref:`sub-slots` | | | | % | +------------------------------------------------+-----------------------------------+-------------------+---+---+---+---+ | :c:member:`~PyTypeObject.tp_as_sequence` | :c:type:`PySequenceMethods` * | :ref:`sub-slots` | | | | % | +------------------------------------------------+-----------------------------------+-------------------+---+---+---+---+ | :c:member:`~PyTypeObject.tp_as_mapping` | :c:type:`PyMappingMethods` * | :ref:`sub-slots` | | | | % | +------------------------------------------------+-----------------------------------+-------------------+---+---+---+---+ | :c:member:`~PyTypeObject.tp_hash` | :c:type:`hashfunc` | __hash__ | X | | | G | +------------------------------------------------+-----------------------------------+-------------------+---+---+---+---+ | :c:member:`~PyTypeObject.tp_call` | :c:type:`ternaryfunc` | __call__ | | X | | X | +------------------------------------------------+-----------------------------------+-------------------+---+---+---+---+ | :c:member:`~PyTypeObject.tp_str` | :c:type:`reprfunc` | __str__ | X | | | X | +------------------------------------------------+-----------------------------------+-------------------+---+---+---+---+ | :c:member:`~PyTypeObject.tp_getattro` | :c:type:`getattrofunc` | __getattribute__, | X | X | | G | | | | __getattr__ | | | | | +------------------------------------------------+-----------------------------------+-------------------+---+---+---+---+ | :c:member:`~PyTypeObject.tp_setattro` | :c:type:`setattrofunc` | __setattr__, | X | X | | G | | | | __delattr__ | | | | | +------------------------------------------------+-----------------------------------+-------------------+---+---+---+---+ | :c:member:`~PyTypeObject.tp_as_buffer` | :c:type:`PyBufferProcs` * | :ref:`sub-slots` | | | | % | +------------------------------------------------+-----------------------------------+-------------------+---+---+---+---+ | :c:member:`~PyTypeObject.tp_flags` | unsigned long | | X | X | | ? | +------------------------------------------------+-----------------------------------+-------------------+---+---+---+---+ | :c:member:`~PyTypeObject.tp_doc` | const char * | __doc__ | X | X | | | +------------------------------------------------+-----------------------------------+-------------------+---+---+---+---+ | :c:member:`~PyTypeObject.tp_traverse` | :c:type:`traverseproc` | | | X | | G | +------------------------------------------------+-----------------------------------+-------------------+---+---+---+---+ | :c:member:`~PyTypeObject.tp_clear` | :c:type:`inquiry` | | | X | | G | +------------------------------------------------+-----------------------------------+-------------------+---+---+---+---+ | :c:member:`~PyTypeObject.tp_richcompare` | :c:type:`richcmpfunc` | __lt__, | X | | | G | | | | __le__, | | | | | | | | __eq__, | | | | | | | | __ne__, | | | | | | | | __gt__, | | | | | | | | __ge__ | | | | | +------------------------------------------------+-----------------------------------+-------------------+---+---+---+---+ | (:c:member:`~PyTypeObject.tp_weaklistoffset`) | :c:type:`Py_ssize_t` | | | X | | ? | +------------------------------------------------+-----------------------------------+-------------------+---+---+---+---+ | :c:member:`~PyTypeObject.tp_iter` | :c:type:`getiterfunc` | __iter__ | | | | X | +------------------------------------------------+-----------------------------------+-------------------+---+---+---+---+ | :c:member:`~PyTypeObject.tp_iternext` | :c:type:`iternextfunc` | __next__ | | | | X | +------------------------------------------------+-----------------------------------+-------------------+---+---+---+---+ | :c:member:`~PyTypeObject.tp_methods` | :c:type:`PyMethodDef` [] | | X | X | | | +------------------------------------------------+-----------------------------------+-------------------+---+---+---+---+ | :c:member:`~PyTypeObject.tp_members` | :c:type:`PyMemberDef` [] | | | X | | | +------------------------------------------------+-----------------------------------+-------------------+---+---+---+---+ | :c:member:`~PyTypeObject.tp_getset` | :c:type:`PyGetSetDef` [] | | X | X | | | +------------------------------------------------+-----------------------------------+-------------------+---+---+---+---+ | :c:member:`~PyTypeObject.tp_base` | :c:type:`PyTypeObject` * | __base__ | | | X | | +------------------------------------------------+-----------------------------------+-------------------+---+---+---+---+ | :c:member:`~PyTypeObject.tp_dict` | :c:type:`PyObject` * | __dict__ | | | ? | | +------------------------------------------------+-----------------------------------+-------------------+---+---+---+---+ | :c:member:`~PyTypeObject.tp_descr_get` | :c:type:`descrgetfunc` | __get__ | | | | X | +------------------------------------------------+-----------------------------------+-------------------+---+---+---+---+ | :c:member:`~PyTypeObject.tp_descr_set` | :c:type:`descrsetfunc` | __set__, | | | | X | | | | __delete__ | | | | | +------------------------------------------------+-----------------------------------+-------------------+---+---+---+---+ | (:c:member:`~PyTypeObject.tp_dictoffset`) | :c:type:`Py_ssize_t` | | | X | | ? | +------------------------------------------------+-----------------------------------+-------------------+---+---+---+---+ | :c:member:`~PyTypeObject.tp_init` | :c:type:`initproc` | __init__ | X | X | | X | +------------------------------------------------+-----------------------------------+-------------------+---+---+---+---+ | :c:member:`~PyTypeObject.tp_alloc` | :c:type:`allocfunc` | | X | | ? | ? | +------------------------------------------------+-----------------------------------+-------------------+---+---+---+---+ | :c:member:`~PyTypeObject.tp_new` | :c:type:`newfunc` | __new__ | X | X | ? | ? | +------------------------------------------------+-----------------------------------+-------------------+---+---+---+---+ | :c:member:`~PyTypeObject.tp_free` | :c:type:`freefunc` | | X | X | ? | ? | +------------------------------------------------+-----------------------------------+-------------------+---+---+---+---+ | :c:member:`~PyTypeObject.tp_is_gc` | :c:type:`inquiry` | | | X | | X | +------------------------------------------------+-----------------------------------+-------------------+---+---+---+---+ | <:c:member:`~PyTypeObject.tp_bases`> | :c:type:`PyObject` * | __bases__ | | | ~ | | +------------------------------------------------+-----------------------------------+-------------------+---+---+---+---+ | <:c:member:`~PyTypeObject.tp_mro`> | :c:type:`PyObject` * | __mro__ | | | ~ | | +------------------------------------------------+-----------------------------------+-------------------+---+---+---+---+ | [:c:member:`~PyTypeObject.tp_cache`] | :c:type:`PyObject` * | | | | | +------------------------------------------------+-----------------------------------+-------------------+---+---+---+---+ | [:c:member:`~PyTypeObject.tp_subclasses`] | void * | __subclasses__ | | | | +------------------------------------------------+-----------------------------------+-------------------+---+---+---+---+ | [:c:member:`~PyTypeObject.tp_weaklist`] | :c:type:`PyObject` * | | | | | +------------------------------------------------+-----------------------------------+-------------------+---+---+---+---+ | (:c:member:`~PyTypeObject.tp_del`) | :c:type:`destructor` | | | | | | +------------------------------------------------+-----------------------------------+-------------------+---+---+---+---+ | [:c:member:`~PyTypeObject.tp_version_tag`] | unsigned int | | | | | +------------------------------------------------+-----------------------------------+-------------------+---+---+---+---+ | :c:member:`~PyTypeObject.tp_finalize` | :c:type:`destructor` | __del__ | | | | X | +------------------------------------------------+-----------------------------------+-------------------+---+---+---+---+ | :c:member:`~PyTypeObject.tp_vectorcall` | :c:type:`vectorcallfunc` | | | | | | +------------------------------------------------+-----------------------------------+-------------------+---+---+---+---+ | [:c:member:`~PyTypeObject.tp_watched`] | unsigned char | | | | | | +------------------------------------------------+-----------------------------------+-------------------+---+---+---+---+ .. [#slots] **()**: A slot name in parentheses indicates it is (effectively) deprecated. **<>**: Names in angle brackets should be initially set to ``NULL`` and treated as read-only. **[]**: Names in square brackets are for internal use only. **** (as a prefix) means the field is required (must be non-``NULL``). .. [#cols] Columns: **"O"**: set on :c:data:`PyBaseObject_Type` **"T"**: set on :c:data:`PyType_Type` **"D"**: default (if slot is set to ``NULL``) .. code-block:: none X - PyType_Ready sets this value if it is NULL ~ - PyType_Ready always sets this value (it should be NULL) ? - PyType_Ready may set this value depending on other slots Also see the inheritance column ("I"). **"I"**: inheritance .. code-block:: none X - type slot is inherited via *PyType_Ready* if defined with a *NULL* value % - the slots of the sub-struct are inherited individually G - inherited, but only in combination with other slots; see the slot's description ? - it's complicated; see the slot's description Note that some slots are effectively inherited through the normal attribute lookup chain. .. _sub-slots: sub-slots ^^^^^^^^^ .. table:: :widths: 26,17,12 +---------------------------------------------------------+-----------------------------------+---------------+ | Slot | :ref:`Type ` | special | | | | methods | +=========================================================+===================================+===============+ | :c:member:`~PyAsyncMethods.am_await` | :c:type:`unaryfunc` | __await__ | +---------------------------------------------------------+-----------------------------------+---------------+ | :c:member:`~PyAsyncMethods.am_aiter` | :c:type:`unaryfunc` | __aiter__ | +---------------------------------------------------------+-----------------------------------+---------------+ | :c:member:`~PyAsyncMethods.am_anext` | :c:type:`unaryfunc` | __anext__ | +---------------------------------------------------------+-----------------------------------+---------------+ | :c:member:`~PyAsyncMethods.am_send` | :c:type:`sendfunc` | | +---------------------------------------------------------+-----------------------------------+---------------+ | | +---------------------------------------------------------+-----------------------------------+---------------+ | :c:member:`~PyNumberMethods.nb_add` | :c:type:`binaryfunc` | __add__ | | | | __radd__ | +---------------------------------------------------------+-----------------------------------+---------------+ | :c:member:`~PyNumberMethods.nb_inplace_add` | :c:type:`binaryfunc` | __iadd__ | +---------------------------------------------------------+-----------------------------------+---------------+ | :c:member:`~PyNumberMethods.nb_subtract` | :c:type:`binaryfunc` | __sub__ | | | | __rsub__ | +---------------------------------------------------------+-----------------------------------+---------------+ | :c:member:`~PyNumberMethods.nb_inplace_subtract` | :c:type:`binaryfunc` | __isub__ | +---------------------------------------------------------+-----------------------------------+---------------+ | :c:member:`~PyNumberMethods.nb_multiply` | :c:type:`binaryfunc` | __mul__ | | | | __rmul__ | +---------------------------------------------------------+-----------------------------------+---------------+ | :c:member:`~PyNumberMethods.nb_inplace_multiply` | :c:type:`binaryfunc` | __imul__ | +---------------------------------------------------------+-----------------------------------+---------------+ | :c:member:`~PyNumberMethods.nb_remainder` | :c:type:`binaryfunc` | __mod__ | | | | __rmod__ | +---------------------------------------------------------+-----------------------------------+---------------+ | :c:member:`~PyNumberMethods.nb_inplace_remainder` | :c:type:`binaryfunc` | __imod__ | +---------------------------------------------------------+-----------------------------------+---------------+ | :c:member:`~PyNumberMethods.nb_divmod` | :c:type:`binaryfunc` | __divmod__ | | | | __rdivmod__ | +---------------------------------------------------------+-----------------------------------+---------------+ | :c:member:`~PyNumberMethods.nb_power` | :c:type:`ternaryfunc` | __pow__ | | | | __rpow__ | +---------------------------------------------------------+-----------------------------------+---------------+ | :c:member:`~PyNumberMethods.nb_inplace_power` | :c:type:`ternaryfunc` | __ipow__ | +---------------------------------------------------------+-----------------------------------+---------------+ | :c:member:`~PyNumberMethods.nb_negative` | :c:type:`unaryfunc` | __neg__ | +---------------------------------------------------------+-----------------------------------+---------------+ | :c:member:`~PyNumberMethods.nb_positive` | :c:type:`unaryfunc` | __pos__ | +---------------------------------------------------------+-----------------------------------+---------------+ | :c:member:`~PyNumberMethods.nb_absolute` | :c:type:`unaryfunc` | __abs__ | +---------------------------------------------------------+-----------------------------------+---------------+ | :c:member:`~PyNumberMethods.nb_bool` | :c:type:`inquiry` | __bool__ | +---------------------------------------------------------+-----------------------------------+---------------+ | :c:member:`~PyNumberMethods.nb_invert` | :c:type:`unaryfunc` | __invert__ | +---------------------------------------------------------+-----------------------------------+---------------+ | :c:member:`~PyNumberMethods.nb_lshift` | :c:type:`binaryfunc` | __lshift__ | | | | __rlshift__ | +---------------------------------------------------------+-----------------------------------+---------------+ | :c:member:`~PyNumberMethods.nb_inplace_lshift` | :c:type:`binaryfunc` | __ilshift__ | +---------------------------------------------------------+-----------------------------------+---------------+ | :c:member:`~PyNumberMethods.nb_rshift` | :c:type:`binaryfunc` | __rshift__ | | | | __rrshift__ | +---------------------------------------------------------+-----------------------------------+---------------+ | :c:member:`~PyNumberMethods.nb_inplace_rshift` | :c:type:`binaryfunc` | __irshift__ | +---------------------------------------------------------+-----------------------------------+---------------+ | :c:member:`~PyNumberMethods.nb_and` | :c:type:`binaryfunc` | __and__ | | | | __rand__ | +---------------------------------------------------------+-----------------------------------+---------------+ | :c:member:`~PyNumberMethods.nb_inplace_and` | :c:type:`binaryfunc` | __iand__ | +---------------------------------------------------------+-----------------------------------+---------------+ | :c:member:`~PyNumberMethods.nb_xor` | :c:type:`binaryfunc` | __xor__ | | | | __rxor__ | +---------------------------------------------------------+-----------------------------------+---------------+ | :c:member:`~PyNumberMethods.nb_inplace_xor` | :c:type:`binaryfunc` | __ixor__ | +---------------------------------------------------------+-----------------------------------+---------------+ | :c:member:`~PyNumberMethods.nb_or` | :c:type:`binaryfunc` | __or__ | | | | __ror__ | +---------------------------------------------------------+-----------------------------------+---------------+ | :c:member:`~PyNumberMethods.nb_inplace_or` | :c:type:`binaryfunc` | __ior__ | +---------------------------------------------------------+-----------------------------------+---------------+ | :c:member:`~PyNumberMethods.nb_int` | :c:type:`unaryfunc` | __int__ | +---------------------------------------------------------+-----------------------------------+---------------+ | :c:member:`~PyNumberMethods.nb_reserved` | void * | | +---------------------------------------------------------+-----------------------------------+---------------+ | :c:member:`~PyNumberMethods.nb_float` | :c:type:`unaryfunc` | __float__ | +---------------------------------------------------------+-----------------------------------+---------------+ | :c:member:`~PyNumberMethods.nb_floor_divide` | :c:type:`binaryfunc` | __floordiv__ | +---------------------------------------------------------+-----------------------------------+---------------+ | :c:member:`~PyNumberMethods.nb_inplace_floor_divide` | :c:type:`binaryfunc` | __ifloordiv__ | +---------------------------------------------------------+-----------------------------------+---------------+ | :c:member:`~PyNumberMethods.nb_true_divide` | :c:type:`binaryfunc` | __truediv__ | +---------------------------------------------------------+-----------------------------------+---------------+ | :c:member:`~PyNumberMethods.nb_inplace_true_divide` | :c:type:`binaryfunc` | __itruediv__ | +---------------------------------------------------------+-----------------------------------+---------------+ | :c:member:`~PyNumberMethods.nb_index` | :c:type:`unaryfunc` | __index__ | +---------------------------------------------------------+-----------------------------------+---------------+ | :c:member:`~PyNumberMethods.nb_matrix_multiply` | :c:type:`binaryfunc` | __matmul__ | | | | __rmatmul__ | +---------------------------------------------------------+-----------------------------------+---------------+ | :c:member:`~PyNumberMethods.nb_inplace_matrix_multiply` | :c:type:`binaryfunc` | __imatmul__ | +---------------------------------------------------------+-----------------------------------+---------------+ | | +---------------------------------------------------------+-----------------------------------+---------------+ | :c:member:`~PyMappingMethods.mp_length` | :c:type:`lenfunc` | __len__ | +---------------------------------------------------------+-----------------------------------+---------------+ | :c:member:`~PyMappingMethods.mp_subscript` | :c:type:`binaryfunc` | __getitem__ | +---------------------------------------------------------+-----------------------------------+---------------+ | :c:member:`~PyMappingMethods.mp_ass_subscript` | :c:type:`objobjargproc` | __setitem__, | | | | __delitem__ | +---------------------------------------------------------+-----------------------------------+---------------+ | | +---------------------------------------------------------+-----------------------------------+---------------+ | :c:member:`~PySequenceMethods.sq_length` | :c:type:`lenfunc` | __len__ | +---------------------------------------------------------+-----------------------------------+---------------+ | :c:member:`~PySequenceMethods.sq_concat` | :c:type:`binaryfunc` | __add__ | +---------------------------------------------------------+-----------------------------------+---------------+ | :c:member:`~PySequenceMethods.sq_repeat` | :c:type:`ssizeargfunc` | __mul__ | +---------------------------------------------------------+-----------------------------------+---------------+ | :c:member:`~PySequenceMethods.sq_item` | :c:type:`ssizeargfunc` | __getitem__ | +---------------------------------------------------------+-----------------------------------+---------------+ | :c:member:`~PySequenceMethods.sq_ass_item` | :c:type:`ssizeobjargproc` | __setitem__ | | | | __delitem__ | +---------------------------------------------------------+-----------------------------------+---------------+ | :c:member:`~PySequenceMethods.sq_contains` | :c:type:`objobjproc` | __contains__ | +---------------------------------------------------------+-----------------------------------+---------------+ | :c:member:`~PySequenceMethods.sq_inplace_concat` | :c:type:`binaryfunc` | __iadd__ | +---------------------------------------------------------+-----------------------------------+---------------+ | :c:member:`~PySequenceMethods.sq_inplace_repeat` | :c:type:`ssizeargfunc` | __imul__ | +---------------------------------------------------------+-----------------------------------+---------------+ | | +---------------------------------------------------------+-----------------------------------+---------------+ | :c:member:`~PyBufferProcs.bf_getbuffer` | :c:func:`getbufferproc` | __buffer__ | +---------------------------------------------------------+-----------------------------------+---------------+ | :c:member:`~PyBufferProcs.bf_releasebuffer` | :c:func:`releasebufferproc` | __release_\ | | | | buffer\__ | +---------------------------------------------------------+-----------------------------------+---------------+ .. _slot-typedefs-table: slot typedefs ^^^^^^^^^^^^^ +-----------------------------+-----------------------------+----------------------+ | typedef | Parameter Types | Return Type | +=============================+=============================+======================+ | :c:type:`allocfunc` | .. line-block:: | :c:type:`PyObject` * | | | | | | | :c:type:`PyTypeObject` * | | | | :c:type:`Py_ssize_t` | | +-----------------------------+-----------------------------+----------------------+ | :c:type:`destructor` | :c:type:`PyObject` * | void | +-----------------------------+-----------------------------+----------------------+ | :c:type:`freefunc` | void * | void | +-----------------------------+-----------------------------+----------------------+ | :c:type:`traverseproc` | .. line-block:: | int | | | | | | | :c:type:`PyObject` * | | | | :c:type:`visitproc` | | | | void * | | +-----------------------------+-----------------------------+----------------------+ | :c:type:`newfunc` | .. line-block:: | :c:type:`PyObject` * | | | | | | | :c:type:`PyTypeObject` * | | | | :c:type:`PyObject` * | | | | :c:type:`PyObject` * | | +-----------------------------+-----------------------------+----------------------+ | :c:type:`initproc` | .. line-block:: | int | | | | | | | :c:type:`PyObject` * | | | | :c:type:`PyObject` * | | | | :c:type:`PyObject` * | | +-----------------------------+-----------------------------+----------------------+ | :c:type:`reprfunc` | :c:type:`PyObject` * | :c:type:`PyObject` * | +-----------------------------+-----------------------------+----------------------+ | :c:type:`getattrfunc` | .. line-block:: | :c:type:`PyObject` * | | | | | | | :c:type:`PyObject` * | | | | const char * | | +-----------------------------+-----------------------------+----------------------+ | :c:type:`setattrfunc` | .. line-block:: | int | | | | | | | :c:type:`PyObject` * | | | | const char * | | | | :c:type:`PyObject` * | | +-----------------------------+-----------------------------+----------------------+ | :c:type:`getattrofunc` | .. line-block:: | :c:type:`PyObject` * | | | | | | | :c:type:`PyObject` * | | | | :c:type:`PyObject` * | | +-----------------------------+-----------------------------+----------------------+ | :c:type:`setattrofunc` | .. line-block:: | int | | | | | | | :c:type:`PyObject` * | | | | :c:type:`PyObject` * | | | | :c:type:`PyObject` * | | +-----------------------------+-----------------------------+----------------------+ | :c:type:`descrgetfunc` | .. line-block:: | :c:type:`PyObject` * | | | | | | | :c:type:`PyObject` * | | | | :c:type:`PyObject` * | | | | :c:type:`PyObject` * | | +-----------------------------+-----------------------------+----------------------+ | :c:type:`descrsetfunc` | .. line-block:: | int | | | | | | | :c:type:`PyObject` * | | | | :c:type:`PyObject` * | | | | :c:type:`PyObject` * | | +-----------------------------+-----------------------------+----------------------+ | :c:type:`hashfunc` | :c:type:`PyObject` * | Py_hash_t | +-----------------------------+-----------------------------+----------------------+ | :c:type:`richcmpfunc` | .. line-block:: | :c:type:`PyObject` * | | | | | | | :c:type:`PyObject` * | | | | :c:type:`PyObject` * | | | | int | | +-----------------------------+-----------------------------+----------------------+ | :c:type:`getiterfunc` | :c:type:`PyObject` * | :c:type:`PyObject` * | +-----------------------------+-----------------------------+----------------------+ | :c:type:`iternextfunc` | :c:type:`PyObject` * | :c:type:`PyObject` * | +-----------------------------+-----------------------------+----------------------+ | :c:type:`lenfunc` | :c:type:`PyObject` * | :c:type:`Py_ssize_t` | +-----------------------------+-----------------------------+----------------------+ | :c:type:`getbufferproc` | .. line-block:: | int | | | | | | | :c:type:`PyObject` * | | | | :c:type:`Py_buffer` * | | | | int | | +-----------------------------+-----------------------------+----------------------+ | :c:type:`releasebufferproc` | .. line-block:: | void | | | | | | | :c:type:`PyObject` * | | | | :c:type:`Py_buffer` * | | +-----------------------------+-----------------------------+----------------------+ | :c:type:`inquiry` | :c:type:`PyObject` * | int | +-----------------------------+-----------------------------+----------------------+ | :c:type:`unaryfunc` | .. line-block:: | :c:type:`PyObject` * | | | | | | | :c:type:`PyObject` * | | +-----------------------------+-----------------------------+----------------------+ | :c:type:`binaryfunc` | .. line-block:: | :c:type:`PyObject` * | | | | | | | :c:type:`PyObject` * | | | | :c:type:`PyObject` * | | +-----------------------------+-----------------------------+----------------------+ | :c:type:`ternaryfunc` | .. line-block:: | :c:type:`PyObject` * | | | | | | | :c:type:`PyObject` * | | | | :c:type:`PyObject` * | | | | :c:type:`PyObject` * | | +-----------------------------+-----------------------------+----------------------+ | :c:type:`ssizeargfunc` | .. line-block:: | :c:type:`PyObject` * | | | | | | | :c:type:`PyObject` * | | | | :c:type:`Py_ssize_t` | | +-----------------------------+-----------------------------+----------------------+ | :c:type:`ssizeobjargproc` | .. line-block:: | int | | | | | | | :c:type:`PyObject` * | | | | :c:type:`Py_ssize_t` | | | | :c:type:`PyObject` * | | +-----------------------------+-----------------------------+----------------------+ | :c:type:`objobjproc` | .. line-block:: | int | | | | | | | :c:type:`PyObject` * | | | | :c:type:`PyObject` * | | +-----------------------------+-----------------------------+----------------------+ | :c:type:`objobjargproc` | .. line-block:: | int | | | | | | | :c:type:`PyObject` * | | | | :c:type:`PyObject` * | | | | :c:type:`PyObject` * | | +-----------------------------+-----------------------------+----------------------+ See :ref:`slot-typedefs` below for more detail. PyTypeObject Definition ----------------------- The structure definition for :c:type:`PyTypeObject` can be found in :file:`Include/cpython/object.h`. For convenience of reference, this repeats the definition found there: .. XXX Drop this? .. literalinclude:: ../includes/typestruct.h PyObject Slots -------------- The type object structure extends the :c:type:`PyVarObject` structure. The :c:member:`~PyVarObject.ob_size` field is used for dynamic types (created by :c:func:`!type_new`, usually called from a class statement). Note that :c:data:`PyType_Type` (the metatype) initializes :c:member:`~PyTypeObject.tp_itemsize`, which means that its instances (i.e. type objects) *must* have the :c:member:`~PyVarObject.ob_size` field. :c:member:`PyObject.ob_refcnt` The type object's reference count is initialized to ``1`` by the ``PyObject_HEAD_INIT`` macro. Note that for :ref:`statically allocated type objects `, the type's instances (objects whose :c:member:`~PyObject.ob_type` points back to the type) do *not* count as references. But for :ref:`dynamically allocated type objects `, the instances *do* count as references. **Inheritance:** This field is not inherited by subtypes. :c:member:`PyObject.ob_type` This is the type's type, in other words its metatype. It is initialized by the argument to the ``PyObject_HEAD_INIT`` macro, and its value should normally be ``&PyType_Type``. However, for dynamically loadable extension modules that must be usable on Windows (at least), the compiler complains that this is not a valid initializer. Therefore, the convention is to pass ``NULL`` to the ``PyObject_HEAD_INIT`` macro and to initialize this field explicitly at the start of the module's initialization function, before doing anything else. This is typically done like this:: Foo_Type.ob_type = &PyType_Type; This should be done before any instances of the type are created. :c:func:`PyType_Ready` checks if :c:member:`~PyObject.ob_type` is ``NULL``, and if so, initializes it to the :c:member:`~PyObject.ob_type` field of the base class. :c:func:`PyType_Ready` will not change this field if it is non-zero. **Inheritance:** This field is inherited by subtypes. PyVarObject Slots ----------------- :c:member:`PyVarObject.ob_size` For :ref:`statically allocated type objects `, this should be initialized to zero. For :ref:`dynamically allocated type objects `, this field has a special internal meaning. This field should be accessed using the :c:func:`Py_SIZE()` macro. **Inheritance:** This field is not inherited by subtypes. PyTypeObject Slots ------------------ Each slot has a section describing inheritance. If :c:func:`PyType_Ready` may set a value when the field is set to ``NULL`` then there will also be a "Default" section. (Note that many fields set on :c:data:`PyBaseObject_Type` and :c:data:`PyType_Type` effectively act as defaults.) .. c:member:: const char* PyTypeObject.tp_name See :c:macro:`Py_tp_name` for the corresponding :c:member:`Slot ID `. Pointer to a NUL-terminated string containing the name of the type. For types that are accessible as module globals, the string should be the full module name, followed by a dot, followed by the type name; for built-in types, it should be just the type name. If the module is a submodule of a package, the full package name is part of the full module name. For example, a type named :class:`!T` defined in module :mod:`!M` in subpackage :mod:`!Q` in package :mod:`!P` should have the :c:member:`~PyTypeObject.tp_name` initializer ``"P.Q.M.T"``. For :ref:`dynamically allocated type objects `, this should just be the type name, and the module name explicitly stored in the type dict as the value for key ``'__module__'``. For :ref:`statically allocated type objects `, the *tp_name* field should contain a dot. Everything before the last dot is made accessible as the :attr:`~type.__module__` attribute, and everything after the last dot is made accessible as the :attr:`~type.__name__` attribute. If no dot is present, the entire :c:member:`~PyTypeObject.tp_name` field is made accessible as the :attr:`~type.__name__` attribute, and the :attr:`~type.__module__` attribute is undefined (unless explicitly set in the dictionary, as explained above). This means your type will be impossible to pickle. Additionally, it will not be listed in module documentations created with pydoc. This field must not be ``NULL``. It is the only required field in :c:func:`PyTypeObject` (other than potentially :c:member:`~PyTypeObject.tp_itemsize`). **Inheritance:** This field is not inherited by subtypes. .. c:member:: Py_ssize_t PyTypeObject.tp_basicsize Py_ssize_t PyTypeObject.tp_itemsize These fields allow calculating the size in bytes of instances of the type. See :c:macro:`Py_tp_basicsize`, :c:macro:`Py_tp_extra_basicsize` and :c:macro:`Py_tp_itemsize` for the corresponding :c:member:`Slot IDs `. There are two kinds of types: types with fixed-length instances have a zero :c:member:`!tp_itemsize` field, types with variable-length instances have a non-zero :c:member:`!tp_itemsize` field. For a type with fixed-length instances, all instances have the same size, given in :c:member:`!tp_basicsize`. (Exceptions to this rule can be made using :c:func:`PyUnstable_Object_GC_NewWithExtraData`.) For a type with variable-length instances, the instances must have an :c:member:`~PyVarObject.ob_size` field, and the instance size is :c:member:`!tp_basicsize` plus N times :c:member:`!tp_itemsize`, where N is the "length" of the object. Functions like :c:func:`PyObject_NewVar` will take the value of N as an argument, and store in the instance's :c:member:`~PyVarObject.ob_size` field. Note that the :c:member:`~PyVarObject.ob_size` field may later be used for other purposes. For example, :py:type:`int` instances use the bits of :c:member:`~PyVarObject.ob_size` in an implementation-defined way; the underlying storage and its size should be accessed using :c:func:`PyLong_Export`. .. note:: The :c:member:`~PyVarObject.ob_size` field should be accessed using the :c:func:`Py_SIZE()` and :c:func:`Py_SET_SIZE()` macros. Also, the presence of an :c:member:`~PyVarObject.ob_size` field in the instance layout doesn't mean that the instance structure is variable-length. For example, the :py:type:`list` type has fixed-length instances, yet those instances have a :c:member:`~PyVarObject.ob_size` field. (As with :py:type:`int`, avoid reading lists' :c:member:`!ob_size` directly. Call :c:func:`PyList_Size` instead.) The :c:member:`!tp_basicsize` includes size needed for data of the type's :c:member:`~PyTypeObject.tp_base`, plus any extra data needed by each instance. The correct way to set :c:member:`!tp_basicsize` is to use the ``sizeof`` operator on the struct used to declare the instance layout. This struct must include the struct used to declare the base type. In other words, :c:member:`!tp_basicsize` must be greater than or equal to the base's :c:member:`!tp_basicsize`. Since every type is a subtype of :py:type:`object`, this struct must include :c:type:`PyObject` or :c:type:`PyVarObject` (depending on whether :c:member:`~PyVarObject.ob_size` should be included). These are usually defined by the macro :c:macro:`PyObject_HEAD` or :c:macro:`PyObject_VAR_HEAD`, respectively. The basic size does not include the GC header size, as that header is not part of :c:macro:`PyObject_HEAD`. For cases where struct used to declare the base type is unknown, see :c:member:`PyType_Spec.basicsize` and :c:func:`PyType_FromMetaclass`. Notes about alignment: - :c:member:`!tp_basicsize` must be a multiple of ``_Alignof(PyObject)``. When using ``sizeof`` on a ``struct`` that includes :c:macro:`PyObject_HEAD`, as recommended, the compiler ensures this. When not using a C ``struct``, or when using compiler extensions like ``__attribute__((packed))``, it is up to you. - If the variable items require a particular alignment, :c:member:`!tp_basicsize` and :c:member:`!tp_itemsize` must each be a multiple of that alignment. For example, if a type's variable part stores a ``double``, it is your responsibility that both fields are a multiple of ``_Alignof(double)``. **Inheritance:** These fields are inherited separately by subtypes. (That is, if the field is set to zero, :c:func:`PyType_Ready` will copy the value from the base type, indicating that the instances do not need additional storage.) If the base type has a non-zero :c:member:`~PyTypeObject.tp_itemsize`, it is generally not safe to set :c:member:`~PyTypeObject.tp_itemsize` to a different non-zero value in a subtype (though this depends on the implementation of the base type). .. c:member:: destructor PyTypeObject.tp_dealloc .. corresponding-type-slot:: Py_tp_dealloc A pointer to the instance destructor function. The function signature is:: void tp_dealloc(PyObject *self); The destructor function should remove all references which the instance owns (e.g., call :c:func:`Py_CLEAR`), free all memory buffers owned by the instance, and call the type's :c:member:`~PyTypeObject.tp_free` function to free the object itself. If you may call functions that may set the error indicator, you must use :c:func:`PyErr_GetRaisedException` and :c:func:`PyErr_SetRaisedException` to ensure you don't clobber a preexisting error indicator (the deallocation could have occurred while processing a different error): .. code-block:: c static void foo_dealloc(foo_object *self) { PyObject *et, *ev, *etb; PyObject *exc = PyErr_GetRaisedException(); ... PyErr_SetRaisedException(exc); } The dealloc handler itself must not raise an exception; if it hits an error case it should call :c:func:`PyErr_FormatUnraisable` to log (and clear) an unraisable exception. No guarantees are made about when an object is destroyed, except: * Python will destroy an object immediately or some time after the final reference to the object is deleted, unless its finalizer (:c:member:`~PyTypeObject.tp_finalize`) subsequently resurrects the object. * An object will not be destroyed while it is being automatically finalized (:c:member:`~PyTypeObject.tp_finalize`) or automatically cleared (:c:member:`~PyTypeObject.tp_clear`). CPython currently destroys an object immediately from :c:func:`Py_DECREF` when the new reference count is zero, but this may change in a future version. It is recommended to call :c:func:`PyObject_CallFinalizerFromDealloc` at the beginning of :c:member:`!tp_dealloc` to guarantee that the object is always finalized before destruction. If the type supports garbage collection (the :c:macro:`Py_TPFLAGS_HAVE_GC` flag is set), the destructor should call :c:func:`PyObject_GC_UnTrack` before clearing any member fields. It is permissible to call :c:member:`~PyTypeObject.tp_clear` from :c:member:`!tp_dealloc` to reduce code duplication and to guarantee that the object is always cleared before destruction. Beware that :c:member:`!tp_clear` might have already been called. If the type is heap allocated (:c:macro:`Py_TPFLAGS_HEAPTYPE`), the deallocator should release the owned reference to its type object (via :c:func:`Py_DECREF`) after calling the type deallocator. See the example code below.:: static void foo_dealloc(PyObject *op) { foo_object *self = (foo_object *) op; PyObject_GC_UnTrack(self); Py_CLEAR(self->ref); Py_TYPE(self)->tp_free(self); } :c:member:`!tp_dealloc` must leave the exception status unchanged. If it needs to call something that might raise an exception, the exception state must be backed up first and restored later (after logging any exceptions with :c:func:`PyErr_WriteUnraisable`). Example:: static void foo_dealloc(PyObject *self) { PyObject *exc = PyErr_GetRaisedException(); if (PyObject_CallFinalizerFromDealloc(self) < 0) { // self was resurrected. goto done; } PyTypeObject *tp = Py_TYPE(self); if (tp->tp_flags & Py_TPFLAGS_HAVE_GC) { PyObject_GC_UnTrack(self); } // Optional, but convenient to avoid code duplication. if (tp->tp_clear && tp->tp_clear(self) < 0) { PyErr_WriteUnraisable(self); } // Any additional destruction goes here. tp->tp_free(self); self = NULL; // In case PyErr_WriteUnraisable() is called below. if (tp->tp_flags & Py_TPFLAGS_HEAPTYPE) { Py_CLEAR(tp); } done: // Optional, if something was called that might have raised an // exception. if (PyErr_Occurred()) { PyErr_WriteUnraisable(self); } PyErr_SetRaisedException(exc); } :c:member:`!tp_dealloc` may be called from any Python thread, not just the thread which created the object (if the object becomes part of a refcount cycle, that cycle might be collected by a garbage collection on any thread). This is not a problem for Python API calls, since the thread on which :c:member:`!tp_dealloc` is called with an :term:`attached thread state`. However, if the object being destroyed in turn destroys objects from some other C library, care should be taken to ensure that destroying those objects on the thread which called :c:member:`!tp_dealloc` will not violate any assumptions of the library. **Inheritance:** This field is inherited by subtypes. .. seealso:: :ref:`life-cycle` for details about how this slot relates to other slots. .. c:member:: Py_ssize_t PyTypeObject.tp_vectorcall_offset An optional offset to a per-instance function that implements calling the object using the :ref:`vectorcall protocol `, a more efficient alternative of the simpler :c:member:`~PyTypeObject.tp_call`. This field is only used if the flag :c:macro:`Py_TPFLAGS_HAVE_VECTORCALL` is set. If so, this must be a positive integer containing the offset in the instance of a :c:type:`vectorcallfunc` pointer. The *vectorcallfunc* pointer may be ``NULL``, in which case the instance behaves as if :c:macro:`Py_TPFLAGS_HAVE_VECTORCALL` was not set: calling the instance falls back to :c:member:`~PyTypeObject.tp_call`. Any class that sets ``Py_TPFLAGS_HAVE_VECTORCALL`` must also set :c:member:`~PyTypeObject.tp_call` and make sure its behaviour is consistent with the *vectorcallfunc* function. This can be done by setting *tp_call* to :c:func:`PyVectorcall_Call`. .. versionchanged:: 3.8 Before version 3.8, this slot was named ``tp_print``. In Python 2.x, it was used for printing to a file. In Python 3.0 to 3.7, it was unused. .. versionchanged:: 3.12 Before version 3.12, it was not recommended for :ref:`mutable heap types ` to implement the vectorcall protocol. When a user sets :attr:`~object.__call__` in Python code, only *tp_call* is updated, likely making it inconsistent with the vectorcall function. Since 3.12, setting ``__call__`` will disable vectorcall optimization by clearing the :c:macro:`Py_TPFLAGS_HAVE_VECTORCALL` flag. **Inheritance:** This field is always inherited. However, the :c:macro:`Py_TPFLAGS_HAVE_VECTORCALL` flag is not always inherited. If it's not set, then the subclass won't use :ref:`vectorcall `, except when :c:func:`PyVectorcall_Call` is explicitly called. .. c:member:: getattrfunc PyTypeObject.tp_getattr .. corresponding-type-slot:: Py_tp_getattr An optional pointer to the get-attribute-string function. This field is deprecated. When it is defined, it should point to a function that acts the same as the :c:member:`~PyTypeObject.tp_getattro` function, but taking a C string instead of a Python string object to give the attribute name. **Inheritance:** Group: :c:member:`~PyTypeObject.tp_getattr`, :c:member:`~PyTypeObject.tp_getattro` This field is inherited by subtypes together with :c:member:`~PyTypeObject.tp_getattro`: a subtype inherits both :c:member:`~PyTypeObject.tp_getattr` and :c:member:`~PyTypeObject.tp_getattro` from its base type when the subtype's :c:member:`~PyTypeObject.tp_getattr` and :c:member:`~PyTypeObject.tp_getattro` are both ``NULL``. .. c:member:: setattrfunc PyTypeObject.tp_setattr .. corresponding-type-slot:: Py_tp_setattr An optional pointer to the function for setting and deleting attributes. This field is deprecated. When it is defined, it should point to a function that acts the same as the :c:member:`~PyTypeObject.tp_setattro` function, but taking a C string instead of a Python string object to give the attribute name. **Inheritance:** Group: :c:member:`~PyTypeObject.tp_setattr`, :c:member:`~PyTypeObject.tp_setattro` This field is inherited by subtypes together with :c:member:`~PyTypeObject.tp_setattro`: a subtype inherits both :c:member:`~PyTypeObject.tp_setattr` and :c:member:`~PyTypeObject.tp_setattro` from its base type when the subtype's :c:member:`~PyTypeObject.tp_setattr` and :c:member:`~PyTypeObject.tp_setattro` are both ``NULL``. .. c:member:: PyAsyncMethods* PyTypeObject.tp_as_async Pointer to an additional structure that contains fields relevant only to objects which implement :term:`awaitable` and :term:`asynchronous iterator` protocols at the C-level. See :ref:`async-structs` for details. .. versionadded:: 3.5 Formerly known as ``tp_compare`` and ``tp_reserved``. **Inheritance:** The :c:member:`~PyTypeObject.tp_as_async` field is not inherited, but the contained fields are inherited individually. .. c:member:: reprfunc PyTypeObject.tp_repr .. corresponding-type-slot:: Py_tp_repr .. index:: pair: built-in function; repr An optional pointer to a function that implements the built-in function :func:`repr`. The signature is the same as for :c:func:`PyObject_Repr`:: PyObject *tp_repr(PyObject *self); The function must return a string or a Unicode object. Ideally, this function should return a string that, when passed to :func:`eval`, given a suitable environment, returns an object with the same value. If this is not feasible, it should return a string starting with ``'<'`` and ending with ``'>'`` from which both the type and the value of the object can be deduced. **Inheritance:** This field is inherited by subtypes. **Default:** When this field is not set, a string of the form ``<%s object at %p>`` is returned, where ``%s`` is replaced by the type name, and ``%p`` by the object's memory address. .. c:member:: PyNumberMethods* PyTypeObject.tp_as_number Pointer to an additional structure that contains fields relevant only to objects which implement the number protocol. These fields are documented in :ref:`number-structs`. **Inheritance:** The :c:member:`~PyTypeObject.tp_as_number` field is not inherited, but the contained fields are inherited individually. .. c:member:: PySequenceMethods* PyTypeObject.tp_as_sequence Pointer to an additional structure that contains fields relevant only to objects which implement the sequence protocol. These fields are documented in :ref:`sequence-structs`. **Inheritance:** The :c:member:`~PyTypeObject.tp_as_sequence` field is not inherited, but the contained fields are inherited individually. .. c:member:: PyMappingMethods* PyTypeObject.tp_as_mapping Pointer to an additional structure that contains fields relevant only to objects which implement the mapping protocol. These fields are documented in :ref:`mapping-structs`. **Inheritance:** The :c:member:`~PyTypeObject.tp_as_mapping` field is not inherited, but the contained fields are inherited individually. .. c:member:: hashfunc PyTypeObject.tp_hash .. corresponding-type-slot:: Py_tp_hash .. index:: pair: built-in function; hash An optional pointer to a function that implements the built-in function :func:`hash`. The signature is the same as for :c:func:`PyObject_Hash`:: Py_hash_t tp_hash(PyObject *); The value ``-1`` should not be returned as a normal return value; when an error occurs during the computation of the hash value, the function should set an exception and return ``-1``. When this field is not set (*and* :c:member:`~PyTypeObject.tp_richcompare` is not set), an attempt to take the hash of the object raises :exc:`TypeError`. This is the same as setting it to :c:func:`PyObject_HashNotImplemented`. This field can be set explicitly to :c:func:`PyObject_HashNotImplemented` to block inheritance of the hash method from a parent type. This is interpreted as the equivalent of ``__hash__ = None`` at the Python level, causing ``isinstance(o, collections.Hashable)`` to correctly return ``False``. Note that the converse is also true - setting ``__hash__ = None`` on a class at the Python level will result in the ``tp_hash`` slot being set to :c:func:`PyObject_HashNotImplemented`. **Inheritance:** Group: :c:member:`~PyTypeObject.tp_hash`, :c:member:`~PyTypeObject.tp_richcompare` This field is inherited by subtypes together with :c:member:`~PyTypeObject.tp_richcompare`: a subtype inherits both of :c:member:`~PyTypeObject.tp_richcompare` and :c:member:`~PyTypeObject.tp_hash`, when the subtype's :c:member:`~PyTypeObject.tp_richcompare` and :c:member:`~PyTypeObject.tp_hash` are both ``NULL``. **Default:** :c:data:`PyBaseObject_Type` uses :c:func:`PyObject_GenericHash`. .. c:member:: ternaryfunc PyTypeObject.tp_call .. corresponding-type-slot:: Py_tp_call An optional pointer to a function that implements calling the object. This should be ``NULL`` if the object is not callable. The signature is the same as for :c:func:`PyObject_Call`:: PyObject *tp_call(PyObject *self, PyObject *args, PyObject *kwargs); **Inheritance:** This field is inherited by subtypes. .. c:member:: reprfunc PyTypeObject.tp_str .. corresponding-type-slot:: Py_tp_str An optional pointer to a function that implements the built-in operation :func:`str`. (Note that :class:`str` is a type now, and :func:`str` calls the constructor for that type. This constructor calls :c:func:`PyObject_Str` to do the actual work, and :c:func:`PyObject_Str` will call this handler.) The signature is the same as for :c:func:`PyObject_Str`:: PyObject *tp_str(PyObject *self); The function must return a string or a Unicode object. It should be a "friendly" string representation of the object, as this is the representation that will be used, among other things, by the :func:`print` function. **Inheritance:** This field is inherited by subtypes. **Default:** When this field is not set, :c:func:`PyObject_Repr` is called to return a string representation. .. c:member:: getattrofunc PyTypeObject.tp_getattro .. corresponding-type-slot:: Py_tp_getattro An optional pointer to the get-attribute function. The signature is the same as for :c:func:`PyObject_GetAttr`:: PyObject *tp_getattro(PyObject *self, PyObject *attr); It is usually convenient to set this field to :c:func:`PyObject_GenericGetAttr`, which implements the normal way of looking for object attributes. **Inheritance:** Group: :c:member:`~PyTypeObject.tp_getattr`, :c:member:`~PyTypeObject.tp_getattro` This field is inherited by subtypes together with :c:member:`~PyTypeObject.tp_getattr`: a subtype inherits both :c:member:`~PyTypeObject.tp_getattr` and :c:member:`~PyTypeObject.tp_getattro` from its base type when the subtype's :c:member:`~PyTypeObject.tp_getattr` and :c:member:`~PyTypeObject.tp_getattro` are both ``NULL``. **Default:** :c:data:`PyBaseObject_Type` uses :c:func:`PyObject_GenericGetAttr`. .. c:member:: setattrofunc PyTypeObject.tp_setattro .. corresponding-type-slot:: Py_tp_setattro An optional pointer to the function for setting and deleting attributes. The signature is the same as for :c:func:`PyObject_SetAttr`:: int tp_setattro(PyObject *self, PyObject *attr, PyObject *value); In addition, setting *value* to ``NULL`` to delete an attribute must be supported. It is usually convenient to set this field to :c:func:`PyObject_GenericSetAttr`, which implements the normal way of setting object attributes. **Inheritance:** Group: :c:member:`~PyTypeObject.tp_setattr`, :c:member:`~PyTypeObject.tp_setattro` This field is inherited by subtypes together with :c:member:`~PyTypeObject.tp_setattr`: a subtype inherits both :c:member:`~PyTypeObject.tp_setattr` and :c:member:`~PyTypeObject.tp_setattro` from its base type when the subtype's :c:member:`~PyTypeObject.tp_setattr` and :c:member:`~PyTypeObject.tp_setattro` are both ``NULL``. **Default:** :c:data:`PyBaseObject_Type` uses :c:func:`PyObject_GenericSetAttr`. .. c:member:: PyBufferProcs* PyTypeObject.tp_as_buffer Pointer to an additional structure that contains fields relevant only to objects which implement the buffer interface. These fields are documented in :ref:`buffer-structs`. **Inheritance:** The :c:member:`~PyTypeObject.tp_as_buffer` field is not inherited, but the contained fields are inherited individually. .. c:member:: unsigned long PyTypeObject.tp_flags See :c:macro:`Py_tp_flags` for the corresponding :c:member:`Slot ID `. This field is a bit mask of various flags. Some flags indicate variant semantics for certain situations; others are used to indicate that certain fields in the type object (or in the extension structures referenced via :c:member:`~PyTypeObject.tp_as_number`, :c:member:`~PyTypeObject.tp_as_sequence`, :c:member:`~PyTypeObject.tp_as_mapping`, and :c:member:`~PyTypeObject.tp_as_buffer`) that were historically not always present are valid; if such a flag bit is clear, the type fields it guards must not be accessed and must be considered to have a zero or ``NULL`` value instead. **Inheritance:** Inheritance of this field is complicated. Most flag bits are inherited individually, i.e. if the base type has a flag bit set, the subtype inherits this flag bit. The flag bits that pertain to extension structures are strictly inherited if the extension structure is inherited, i.e. the base type's value of the flag bit is copied into the subtype together with a pointer to the extension structure. The :c:macro:`Py_TPFLAGS_HAVE_GC` flag bit is inherited together with the :c:member:`~PyTypeObject.tp_traverse` and :c:member:`~PyTypeObject.tp_clear` fields, i.e. if the :c:macro:`Py_TPFLAGS_HAVE_GC` flag bit is clear in the subtype and the :c:member:`~PyTypeObject.tp_traverse` and :c:member:`~PyTypeObject.tp_clear` fields in the subtype exist and have ``NULL`` values. .. XXX are most flag bits *really* inherited individually? **Default:** :c:data:`PyBaseObject_Type` uses ``Py_TPFLAGS_DEFAULT | Py_TPFLAGS_BASETYPE``. **Bit Masks:** .. c:namespace:: NULL The following bit masks are currently defined; these can be ORed together using the ``|`` operator to form the value of the :c:member:`~PyTypeObject.tp_flags` field. The macro :c:func:`PyType_HasFeature` takes a type and a flags value, *tp* and *f*, and checks whether ``tp->tp_flags & f`` is non-zero. .. c:macro:: Py_TPFLAGS_HEAPTYPE This bit is set when the type object itself is allocated on the heap, for example, types created dynamically using :c:func:`PyType_FromSpec`. In this case, the :c:member:`~PyObject.ob_type` field of its instances is considered a reference to the type, and the type object is INCREF'ed when a new instance is created, and DECREF'ed when an instance is destroyed (this does not apply to instances of subtypes; only the type referenced by the instance's ob_type gets INCREF'ed or DECREF'ed). Heap types should also :ref:`support garbage collection ` as they can form a reference cycle with their own module object. **Inheritance:** ??? .. c:macro:: Py_TPFLAGS_BASETYPE This bit is set when the type can be used as the base type of another type. If this bit is clear, the type cannot be subtyped (similar to a "final" class in Java). **Inheritance:** ??? .. c:macro:: Py_TPFLAGS_READY This bit is set when the type object has been fully initialized by :c:func:`PyType_Ready`. **Inheritance:** ??? .. c:macro:: Py_TPFLAGS_READYING This bit is set while :c:func:`PyType_Ready` is in the process of initializing the type object. **Inheritance:** ??? .. c:macro:: Py_TPFLAGS_HAVE_GC This bit is set when the object supports garbage collection. If this bit is set, memory for new instances (see :c:member:`~PyTypeObject.tp_alloc`) must be allocated using :c:macro:`PyObject_GC_New` or :c:func:`PyType_GenericAlloc` and deallocated (see :c:member:`~PyTypeObject.tp_free`) using :c:func:`PyObject_GC_Del`. More information in section :ref:`supporting-cycle-detection`. **Inheritance:** Group: :c:macro:`Py_TPFLAGS_HAVE_GC`, :c:member:`~PyTypeObject.tp_traverse`, :c:member:`~PyTypeObject.tp_clear` The :c:macro:`Py_TPFLAGS_HAVE_GC` flag bit is inherited together with the :c:member:`~PyTypeObject.tp_traverse` and :c:member:`~PyTypeObject.tp_clear` fields, i.e. if the :c:macro:`Py_TPFLAGS_HAVE_GC` flag bit is clear in the subtype and the :c:member:`~PyTypeObject.tp_traverse` and :c:member:`~PyTypeObject.tp_clear` fields in the subtype exist and have ``NULL`` values. .. c:macro:: Py_TPFLAGS_DEFAULT This is a bitmask of all the bits that pertain to the existence of certain fields in the type object and its extension structures. Currently, it includes the following bits: :c:macro:`Py_TPFLAGS_HAVE_STACKLESS_EXTENSION`. **Inheritance:** ??? .. c:macro:: Py_TPFLAGS_METHOD_DESCRIPTOR This bit indicates that objects behave like unbound methods. If this flag is set for ``type(meth)``, then: - ``meth.__get__(obj, cls)(*args, **kwds)`` (with ``obj`` not None) must be equivalent to ``meth(obj, *args, **kwds)``. - ``meth.__get__(None, cls)(*args, **kwds)`` must be equivalent to ``meth(*args, **kwds)``. This flag enables an optimization for typical method calls like ``obj.meth()``: it avoids creating a temporary "bound method" object for ``obj.meth``. .. versionadded:: 3.8 **Inheritance:** This flag is never inherited by types without the :c:macro:`Py_TPFLAGS_IMMUTABLETYPE` flag set. For extension types, it is inherited whenever :c:member:`~PyTypeObject.tp_descr_get` is inherited. .. c:macro:: Py_TPFLAGS_MANAGED_DICT This bit indicates that instances of the class have a :attr:`~object.__dict__` attribute, and that the space for the dictionary is managed by the VM. If this flag is set, :c:macro:`Py_TPFLAGS_HAVE_GC` must also be set. The type traverse function must call :c:func:`PyObject_VisitManagedDict` and its clear function must call :c:func:`PyObject_ClearManagedDict`. .. versionadded:: 3.12 **Inheritance:** This flag is inherited unless the :c:member:`~PyTypeObject.tp_dictoffset` field is set in a superclass. .. c:macro:: Py_TPFLAGS_MANAGED_WEAKREF This bit indicates that instances of the class should be weakly referenceable. If this flag is set, :c:macro:`Py_TPFLAGS_HAVE_GC` must also be set. .. versionadded:: 3.12 **Inheritance:** This flag is inherited unless the :c:member:`~PyTypeObject.tp_weaklistoffset` field is set in a superclass. .. c:macro:: Py_TPFLAGS_PREHEADER These bits indicate that the VM will manage some fields by storing them before the object. Currently, this macro is equivalent to :c:expr:`Py_TPFLAGS_MANAGED_DICT | Py_TPFLAGS_MANAGED_WEAKREF`. This macro value relies on the implementation of the VM, so its value is not stable and may change in a future version. Prefer using individual flags instead. .. versionadded:: 3.12 .. c:macro:: Py_TPFLAGS_ITEMS_AT_END Only usable with variable-size types, i.e. ones with non-zero :c:member:`~PyTypeObject.tp_itemsize`. Indicates that the variable-sized portion of an instance of this type is at the end of the instance's memory area, at an offset of ``Py_TYPE(obj)->tp_basicsize`` (which may be different in each subclass). When setting this flag, be sure that all superclasses either use this memory layout, or are not variable-sized. Python does not check this. .. versionadded:: 3.12 **Inheritance:** This flag is inherited. .. XXX Document more flags here? .. c:macro:: Py_TPFLAGS_LONG_SUBCLASS .. c:macro:: Py_TPFLAGS_LIST_SUBCLASS .. c:macro:: Py_TPFLAGS_TUPLE_SUBCLASS .. c:macro:: Py_TPFLAGS_BYTES_SUBCLASS .. c:macro:: Py_TPFLAGS_UNICODE_SUBCLASS .. c:macro:: Py_TPFLAGS_DICT_SUBCLASS .. c:macro:: Py_TPFLAGS_BASE_EXC_SUBCLASS .. c:macro:: Py_TPFLAGS_TYPE_SUBCLASS Functions such as :c:func:`PyLong_Check` will call :c:func:`PyType_FastSubclass` with one of these flags to quickly determine if a type is a subclass of a built-in type; such specific checks are faster than a generic check, like :c:func:`PyObject_IsInstance`. Custom types that inherit from built-ins should have their :c:member:`~PyTypeObject.tp_flags` set appropriately, or the code that interacts with such types will behave differently depending on what kind of check is used. .. c:macro:: Py_TPFLAGS_HAVE_FINALIZE This bit is set when the :c:member:`~PyTypeObject.tp_finalize` slot is present in the type structure. .. versionadded:: 3.4 .. deprecated:: 3.8 This flag isn't necessary anymore, as the interpreter assumes the :c:member:`~PyTypeObject.tp_finalize` slot is always present in the type structure. .. c:macro:: _Py_TPFLAGS_HAVE_VECTORCALL :no-typesetting: .. c:macro:: Py_TPFLAGS_HAVE_VECTORCALL This bit is set when the class implements the :ref:`vectorcall protocol `. See :c:member:`~PyTypeObject.tp_vectorcall_offset` for details. **Inheritance:** This bit is inherited if :c:member:`~PyTypeObject.tp_call` is also inherited. .. versionadded:: 3.8 as ``_Py_TPFLAGS_HAVE_VECTORCALL`` .. versionchanged:: 3.9 Renamed to the current name, without the leading underscore. The old provisional name is :term:`soft deprecated`. .. versionchanged:: 3.12 This flag is now removed from a class when the class's :py:meth:`~object.__call__` method is reassigned. This flag can now be inherited by mutable classes. .. c:macro:: Py_TPFLAGS_IMMUTABLETYPE This bit is set for type objects that are immutable: type attributes cannot be set nor deleted. :c:func:`PyType_Ready` automatically applies this flag to :ref:`static types `. **Inheritance:** This flag is not inherited. .. versionadded:: 3.10 .. c:macro:: Py_TPFLAGS_DISALLOW_INSTANTIATION Disallow creating instances of the type: set :c:member:`~PyTypeObject.tp_new` to NULL and don't create the ``__new__`` key in the type dictionary. The flag must be set before creating the type, not after. For example, it must be set before :c:func:`PyType_Ready` is called on the type. The flag is set automatically on :ref:`static types ` if :c:member:`~PyTypeObject.tp_base` is NULL or ``&PyBaseObject_Type`` and :c:member:`~PyTypeObject.tp_new` is NULL. **Inheritance:** This flag is not inherited. However, subclasses will not be instantiable unless they provide a non-NULL :c:member:`~PyTypeObject.tp_new` (which is only possible via the C API). .. note:: To disallow instantiating a class directly but allow instantiating its subclasses (e.g. for an :term:`abstract base class`), do not use this flag. Instead, make :c:member:`~PyTypeObject.tp_new` only succeed for subclasses. .. versionadded:: 3.10 .. c:macro:: Py_TPFLAGS_MAPPING This bit indicates that instances of the class may match mapping patterns when used as the subject of a :keyword:`match` block. It is automatically set when registering or subclassing :class:`collections.abc.Mapping`, and unset when registering :class:`collections.abc.Sequence`. .. note:: :c:macro:`Py_TPFLAGS_MAPPING` and :c:macro:`Py_TPFLAGS_SEQUENCE` are mutually exclusive; it is an error to enable both flags simultaneously. **Inheritance:** This flag is inherited by types that do not already set :c:macro:`Py_TPFLAGS_SEQUENCE`. .. seealso:: :pep:`634` -- Structural Pattern Matching: Specification .. versionadded:: 3.10 .. c:macro:: Py_TPFLAGS_SEQUENCE This bit indicates that instances of the class may match sequence patterns when used as the subject of a :keyword:`match` block. It is automatically set when registering or subclassing :class:`collections.abc.Sequence`, and unset when registering :class:`collections.abc.Mapping`. .. note:: :c:macro:`Py_TPFLAGS_MAPPING` and :c:macro:`Py_TPFLAGS_SEQUENCE` are mutually exclusive; it is an error to enable both flags simultaneously. **Inheritance:** This flag is inherited by types that do not already set :c:macro:`Py_TPFLAGS_MAPPING`. .. seealso:: :pep:`634` -- Structural Pattern Matching: Specification .. versionadded:: 3.10 .. c:macro:: Py_TPFLAGS_VALID_VERSION_TAG Internal. Do not set or unset this flag. To indicate that a class has changed call :c:func:`PyType_Modified` .. warning:: This flag is present in header files, but is not be used. It will be removed in a future version of CPython .. c:macro:: Py_TPFLAGS_HAVE_VERSION_TAG This macro does nothing. Historically, this would indicate that the :c:member:`~PyTypeObject.tp_version_tag` field was available and initialized. .. soft-deprecated:: 3.13 .. c:macro:: Py_TPFLAGS_INLINE_VALUES This bit indicates that instances of this type will have an "inline values" array (containing the object's attributes) placed directly after the end of the object. This requires that :c:macro:`Py_TPFLAGS_HAVE_GC` is set. **Inheritance:** This flag is not inherited. .. versionadded:: 3.13 .. c:macro:: Py_TPFLAGS_IS_ABSTRACT This bit indicates that this is an abstract type and therefore cannot be instantiated. **Inheritance:** This flag is not inherited. .. seealso:: :mod:`abc` .. c:macro:: Py_TPFLAGS_HAVE_STACKLESS_EXTENSION Internal. Do not set or unset this flag. Historically, this was a reserved flag for use in Stackless Python. .. warning:: This flag is present in header files, but is not be used. This may be removed in a future version of CPython. .. c:member:: const char* PyTypeObject.tp_doc .. corresponding-type-slot:: Py_tp_doc An optional pointer to a NUL-terminated C string giving the docstring for this type object. This is exposed as the :attr:`~type.__doc__` attribute on the type and instances of the type. **Inheritance:** This field is *not* inherited by subtypes. .. c:member:: traverseproc PyTypeObject.tp_traverse .. corresponding-type-slot:: Py_tp_traverse An optional pointer to a traversal function for the garbage collector. This is only used if the :c:macro:`Py_TPFLAGS_HAVE_GC` flag bit is set. See :ref:`gc-traversal` for documentation. **Inheritance:** Group: :c:macro:`Py_TPFLAGS_HAVE_GC`, :c:member:`~PyTypeObject.tp_traverse`, :c:member:`~PyTypeObject.tp_clear` This field is inherited by subtypes together with :c:member:`~PyTypeObject.tp_clear` and the :c:macro:`Py_TPFLAGS_HAVE_GC` flag bit: the flag bit, :c:member:`~PyTypeObject.tp_traverse`, and :c:member:`~PyTypeObject.tp_clear` are all inherited from the base type if they are all zero in the subtype. .. c:member:: inquiry PyTypeObject.tp_clear .. corresponding-type-slot:: Py_tp_clear An optional pointer to a clear function. The signature is:: int tp_clear(PyObject *); The purpose of this function is to break reference cycles that are causing a :term:`cyclic isolate` so that the objects can be safely destroyed. A cleared object is a partially destroyed object; the object is not obligated to satisfy design invariants held during normal use. :c:member:`!tp_clear` does not need to delete references to objects that can't participate in reference cycles, such as Python strings or Python integers. However, it may be convenient to clear all references, and write the type's :c:member:`~PyTypeObject.tp_dealloc` function to invoke :c:member:`!tp_clear` to avoid code duplication. (Beware that :c:member:`!tp_clear` might have already been called. Prefer calling idempotent functions like :c:func:`Py_CLEAR`.) Any non-trivial cleanup should be performed in :c:member:`~PyTypeObject.tp_finalize` instead of :c:member:`!tp_clear`. .. note:: If :c:member:`!tp_clear` fails to break a reference cycle then the objects in the :term:`cyclic isolate` may remain indefinitely uncollectable ("leak"). See :data:`gc.garbage`. .. note:: Referents (direct and indirect) might have already been cleared; they are not guaranteed to be in a consistent state. .. note:: The :c:member:`~PyTypeObject.tp_clear` function can be called from any thread. .. note:: An object is not guaranteed to be automatically cleared before its destructor (:c:member:`~PyTypeObject.tp_dealloc`) is called. This function differs from the destructor (:c:member:`~PyTypeObject.tp_dealloc`) in the following ways: * The purpose of clearing an object is to remove references to other objects that might participate in a reference cycle. The purpose of the destructor, on the other hand, is a superset: it must release *all* resources it owns, including references to objects that cannot participate in a reference cycle (e.g., integers) as well as the object's own memory (by calling :c:member:`~PyTypeObject.tp_free`). * When :c:member:`!tp_clear` is called, other objects might still hold references to the object being cleared. Because of this, :c:member:`!tp_clear` must not deallocate the object's own memory (:c:member:`~PyTypeObject.tp_free`). The destructor, on the other hand, is only called when no (strong) references exist, and as such, must safely destroy the object itself by deallocating it. * :c:member:`!tp_clear` might never be automatically called. An object's destructor, on the other hand, will be automatically called some time after the object becomes unreachable (i.e., either there are no references to the object or the object is a member of a :term:`cyclic isolate`). No guarantees are made about when, if, or how often Python automatically clears an object, except: * Python will not automatically clear an object if it is reachable, i.e., there is a reference to it and it is not a member of a :term:`cyclic isolate`. * Python will not automatically clear an object if it has not been automatically finalized (see :c:member:`~PyTypeObject.tp_finalize`). (If the finalizer resurrected the object, the object may or may not be automatically finalized again before it is cleared.) * If an object is a member of a :term:`cyclic isolate`, Python will not automatically clear it if any member of the cyclic isolate has not yet been automatically finalized (:c:member:`~PyTypeObject.tp_finalize`). * Python will not destroy an object until after any automatic calls to its :c:member:`!tp_clear` function have returned. This ensures that the act of breaking a reference cycle does not invalidate the ``self`` pointer while :c:member:`!tp_clear` is still executing. * Python will not automatically call :c:member:`!tp_clear` multiple times concurrently. CPython currently only automatically clears objects as needed to break reference cycles in a :term:`cyclic isolate`, but future versions might clear objects regularly before their destruction. Taken together, all :c:member:`~PyTypeObject.tp_clear` functions in the system must combine to break all reference cycles. This is subtle, and if in any doubt supply a :c:member:`~PyTypeObject.tp_clear` function. For example, the tuple type does not implement a :c:member:`~PyTypeObject.tp_clear` function, because it's possible to prove that no reference cycle can be composed entirely of tuples. Therefore the :c:member:`~PyTypeObject.tp_clear` functions of other types are responsible for breaking any cycle containing a tuple. This isn't immediately obvious, and there's rarely a good reason to avoid implementing :c:member:`~PyTypeObject.tp_clear`. Implementations of :c:member:`~PyTypeObject.tp_clear` should drop the instance's references to those of its members that may be Python objects, and set its pointers to those members to ``NULL``, as in the following example:: static int local_clear(PyObject *op) { localobject *self = (localobject *) op; Py_CLEAR(self->key); Py_CLEAR(self->args); Py_CLEAR(self->kw); Py_CLEAR(self->dict); return 0; } The :c:func:`Py_CLEAR` macro should be used, because clearing references is delicate: the reference to the contained object must not be released (via :c:func:`Py_DECREF`) until after the pointer to the contained object is set to ``NULL``. This is because releasing the reference may cause the contained object to become trash, triggering a chain of reclamation activity that may include invoking arbitrary Python code (due to finalizers, or weakref callbacks, associated with the contained object). If it's possible for such code to reference *self* again, it's important that the pointer to the contained object be ``NULL`` at that time, so that *self* knows the contained object can no longer be used. The :c:func:`Py_CLEAR` macro performs the operations in a safe order. If the :c:macro:`Py_TPFLAGS_MANAGED_DICT` bit is set in the :c:member:`~PyTypeObject.tp_flags` field, the clear function must call :c:func:`PyObject_ClearManagedDict` like this:: PyObject_ClearManagedDict((PyObject*)self); More information about Python's garbage collection scheme can be found in section :ref:`supporting-cycle-detection`. **Inheritance:** Group: :c:macro:`Py_TPFLAGS_HAVE_GC`, :c:member:`~PyTypeObject.tp_traverse`, :c:member:`~PyTypeObject.tp_clear` This field is inherited by subtypes together with :c:member:`~PyTypeObject.tp_traverse` and the :c:macro:`Py_TPFLAGS_HAVE_GC` flag bit: the flag bit, :c:member:`~PyTypeObject.tp_traverse`, and :c:member:`~PyTypeObject.tp_clear` are all inherited from the base type if they are all zero in the subtype. .. seealso:: :ref:`life-cycle` for details about how this slot relates to other slots. .. c:member:: richcmpfunc PyTypeObject.tp_richcompare .. corresponding-type-slot:: Py_tp_richcompare An optional pointer to the rich comparison function, whose signature is:: PyObject *tp_richcompare(PyObject *self, PyObject *other, int op); The first parameter is guaranteed to be an instance of the type that is defined by :c:type:`PyTypeObject`. The function should return the result of the comparison (usually ``Py_True`` or ``Py_False``). If the comparison is undefined, it must return ``Py_NotImplemented``, if another error occurred it must return ``NULL`` and set an exception condition. The following constants are defined to be used as the third argument for :c:member:`~PyTypeObject.tp_richcompare` and for :c:func:`PyObject_RichCompare`: .. c:namespace:: NULL +--------------------+------------+ | Constant | Comparison | +====================+============+ | .. c:macro:: Py_LT | ``<`` | +--------------------+------------+ | .. c:macro:: Py_LE | ``<=`` | +--------------------+------------+ | .. c:macro:: Py_EQ | ``==`` | +--------------------+------------+ | .. c:macro:: Py_NE | ``!=`` | +--------------------+------------+ | .. c:macro:: Py_GT | ``>`` | +--------------------+------------+ | .. c:macro:: Py_GE | ``>=`` | +--------------------+------------+ The following macro is defined to ease writing rich comparison functions: .. c:macro:: Py_RETURN_RICHCOMPARE(VAL_A, VAL_B, op) Return ``Py_True`` or ``Py_False`` from the function, depending on the result of a comparison. VAL_A and VAL_B must be orderable by C comparison operators (for example, they may be C ints or floats). The third argument specifies the requested operation, as for :c:func:`PyObject_RichCompare`. The returned value is a new :term:`strong reference`. On error, sets an exception and returns ``NULL`` from the function. .. versionadded:: 3.7 **Inheritance:** Group: :c:member:`~PyTypeObject.tp_hash`, :c:member:`~PyTypeObject.tp_richcompare` This field is inherited by subtypes together with :c:member:`~PyTypeObject.tp_hash`: a subtype inherits :c:member:`~PyTypeObject.tp_richcompare` and :c:member:`~PyTypeObject.tp_hash` when the subtype's :c:member:`~PyTypeObject.tp_richcompare` and :c:member:`~PyTypeObject.tp_hash` are both ``NULL``. **Default:** :c:data:`PyBaseObject_Type` provides a :c:member:`~PyTypeObject.tp_richcompare` implementation, which may be inherited. However, if only :c:member:`~PyTypeObject.tp_hash` is defined, not even the inherited function is used and instances of the type will not be able to participate in any comparisons. .. c:member:: Py_ssize_t PyTypeObject.tp_weaklistoffset While this field is still supported, :c:macro:`Py_TPFLAGS_MANAGED_WEAKREF` should be used instead, if at all possible. If the instances of this type are weakly referenceable, this field is greater than zero and contains the offset in the instance structure of the weak reference list head (ignoring the GC header, if present); this offset is used by :c:func:`PyObject_ClearWeakRefs` and the ``PyWeakref_*`` functions. The instance structure needs to include a field of type :c:expr:`PyObject*` which is initialized to ``NULL``. Do not confuse this field with :c:member:`~PyTypeObject.tp_weaklist`; that is the list head for weak references to the type object itself. It is an error to set both the :c:macro:`Py_TPFLAGS_MANAGED_WEAKREF` bit and :c:member:`~PyTypeObject.tp_weaklistoffset`. **Inheritance:** This field is inherited by subtypes, but see the rules listed below. A subtype may override this offset; this means that the subtype uses a different weak reference list head than the base type. Since the list head is always found via :c:member:`~PyTypeObject.tp_weaklistoffset`, this should not be a problem. **Default:** If the :c:macro:`Py_TPFLAGS_MANAGED_WEAKREF` bit is set in the :c:member:`~PyTypeObject.tp_flags` field, then :c:member:`~PyTypeObject.tp_weaklistoffset` will be set to a negative value, to indicate that it is unsafe to use this field. .. c:member:: getiterfunc PyTypeObject.tp_iter .. corresponding-type-slot:: Py_tp_iter An optional pointer to a function that returns an :term:`iterator` for the object. Its presence normally signals that the instances of this type are :term:`iterable` (although sequences may be iterable without this function). This function has the same signature as :c:func:`PyObject_GetIter`:: PyObject *tp_iter(PyObject *self); **Inheritance:** This field is inherited by subtypes. .. c:member:: iternextfunc PyTypeObject.tp_iternext .. corresponding-type-slot:: Py_tp_iternext An optional pointer to a function that returns the next item in an :term:`iterator`. The signature is:: PyObject *tp_iternext(PyObject *self); When the iterator is :term:`exhausted`, the ``tp_iternext`` function must return ``NULL``; a :exc:`StopIteration` exception may or may not be set. When another error occurs, it must return ``NULL`` too. The presence of ``tp_iternext`` signals that the instances of this type are iterators. Iterator types should also define the :c:member:`~PyTypeObject.tp_iter` function, and that function should return the iterator instance itself (not a new iterator instance). This function has the same signature as :c:func:`PyIter_Next`. **Inheritance:** This field is inherited by subtypes. .. c:member:: struct PyMethodDef* PyTypeObject.tp_methods .. corresponding-type-slot:: Py_tp_methods An optional pointer to a static ``NULL``-terminated array of :c:type:`PyMethodDef` structures, declaring regular methods of this type. For each entry in the array, an entry is added to the type's dictionary (see :c:member:`~PyTypeObject.tp_dict` below) containing a method descriptor. **Inheritance:** This field is not inherited by subtypes (methods are inherited through a different mechanism). .. c:member:: struct PyMemberDef* PyTypeObject.tp_members .. corresponding-type-slot:: Py_tp_members An optional pointer to a static ``NULL``-terminated array of :c:type:`PyMemberDef` structures, declaring regular data members (fields or slots) of instances of this type. For each entry in the array, an entry is added to the type's dictionary (see :c:member:`~PyTypeObject.tp_dict` below) containing a member descriptor. **Inheritance:** This field is not inherited by subtypes (members are inherited through a different mechanism). .. c:member:: struct PyGetSetDef* PyTypeObject.tp_getset .. corresponding-type-slot:: Py_tp_getset An optional pointer to a static ``NULL``-terminated array of :c:type:`PyGetSetDef` structures, declaring computed attributes of instances of this type. For each entry in the array, an entry is added to the type's dictionary (see :c:member:`~PyTypeObject.tp_dict` below) containing a getset descriptor. **Inheritance:** This field is not inherited by subtypes (computed attributes are inherited through a different mechanism). .. c:member:: PyTypeObject* PyTypeObject.tp_base An optional pointer to a base type from which type properties are inherited. At this level, only single inheritance is supported; multiple inheritance require dynamically creating a type object by calling the metatype. For the corresponding slot ID, see :c:macro:`Py_tp_base`. .. note:: .. from Modules/xxmodule.c Slot initialization is subject to the rules of initializing globals. C99 requires the initializers to be "address constants". Function designators like :c:func:`PyType_GenericNew`, with implicit conversion to a pointer, are valid C99 address constants. However, the unary '&' operator applied to a non-static variable like :c:data:`PyBaseObject_Type` is not required to produce an address constant. Compilers may support this (gcc does), MSVC does not. Both compilers are strictly standard conforming in this particular behavior. Consequently, :c:member:`~PyTypeObject.tp_base` should be set in the extension module's init function. **Inheritance:** This field is not inherited by subtypes (obviously). **Default:** This field defaults to ``&PyBaseObject_Type`` (which to Python programmers is known as the type :class:`object`). .. c:member:: PyObject* PyTypeObject.tp_dict The type's dictionary is stored here by :c:func:`PyType_Ready`. This field should normally be initialized to ``NULL`` before PyType_Ready is called; it may also be initialized to a dictionary containing initial attributes for the type. Once :c:func:`PyType_Ready` has initialized the type, extra attributes for the type may be added to this dictionary only if they don't correspond to overloaded operations (like :meth:`~object.__add__`). Once initialization for the type has finished, this field should be treated as read-only. Some types may not store their dictionary in this slot. Use :c:func:`PyType_GetDict` to retrieve the dictionary for an arbitrary type. .. versionchanged:: 3.12 Internals detail: For static builtin types, this is always ``NULL``. Instead, the dict for such types is stored on ``PyInterpreterState``. Use :c:func:`PyType_GetDict` to get the dict for an arbitrary type. **Inheritance:** This field is not inherited by subtypes (though the attributes defined in here are inherited through a different mechanism). **Default:** If this field is ``NULL``, :c:func:`PyType_Ready` will assign a new dictionary to it. .. warning:: It is not safe to use :c:func:`PyDict_SetItem` on or otherwise modify :c:member:`~PyTypeObject.tp_dict` with the dictionary C-API. .. c:member:: descrgetfunc PyTypeObject.tp_descr_get .. corresponding-type-slot:: Py_tp_descr_get An optional pointer to a "descriptor get" function. The function signature is:: PyObject * tp_descr_get(PyObject *self, PyObject *obj, PyObject *type); .. XXX explain more? **Inheritance:** This field is inherited by subtypes. .. c:member:: descrsetfunc PyTypeObject.tp_descr_set .. corresponding-type-slot:: Py_tp_descr_set An optional pointer to a function for setting and deleting a descriptor's value. The function signature is:: int tp_descr_set(PyObject *self, PyObject *obj, PyObject *value); The *value* argument is set to ``NULL`` to delete the value. .. XXX explain more? **Inheritance:** This field is inherited by subtypes. .. c:member:: Py_ssize_t PyTypeObject.tp_dictoffset While this field is still supported, :c:macro:`Py_TPFLAGS_MANAGED_DICT` should be used instead, if at all possible. If the instances of this type have a dictionary containing instance variables, this field is non-zero and contains the offset in the instances of the type of the instance variable dictionary; this offset is used by :c:func:`PyObject_GenericGetAttr`. Do not confuse this field with :c:member:`~PyTypeObject.tp_dict`; that is the dictionary for attributes of the type object itself. The value specifies the offset of the dictionary from the start of the instance structure. The :c:member:`~PyTypeObject.tp_dictoffset` should be regarded as write-only. To get the pointer to the dictionary call :c:func:`PyObject_GenericGetDict`. Calling :c:func:`PyObject_GenericGetDict` may need to allocate memory for the dictionary, so it is may be more efficient to call :c:func:`PyObject_GetAttr` when accessing an attribute on the object. It is an error to set both the :c:macro:`Py_TPFLAGS_MANAGED_DICT` bit and :c:member:`~PyTypeObject.tp_dictoffset`. **Inheritance:** This field is inherited by subtypes. A subtype should not override this offset; doing so could be unsafe, if C code tries to access the dictionary at the previous offset. To properly support inheritance, use :c:macro:`Py_TPFLAGS_MANAGED_DICT`. **Default:** This slot has no default. For :ref:`static types `, if the field is ``NULL`` then no :attr:`~object.__dict__` gets created for instances. If the :c:macro:`Py_TPFLAGS_MANAGED_DICT` bit is set in the :c:member:`~PyTypeObject.tp_flags` field, then :c:member:`~PyTypeObject.tp_dictoffset` will be set to ``-1``, to indicate that it is unsafe to use this field. .. c:member:: initproc PyTypeObject.tp_init .. corresponding-type-slot:: Py_tp_init An optional pointer to an instance initialization function. This function corresponds to the :meth:`~object.__init__` method of classes. Like :meth:`!__init__`, it is possible to create an instance without calling :meth:`!__init__`, and it is possible to reinitialize an instance by calling its :meth:`!__init__` method again. The function signature is:: int tp_init(PyObject *self, PyObject *args, PyObject *kwds); The self argument is the instance to be initialized; the *args* and *kwds* arguments represent positional and keyword arguments of the call to :meth:`~object.__init__`. The :c:member:`~PyTypeObject.tp_init` function, if not ``NULL``, is called when an instance is created normally by calling its type, after the type's :c:member:`~PyTypeObject.tp_new` function has returned an instance of the type. If the :c:member:`~PyTypeObject.tp_new` function returns an instance of some other type that is not a subtype of the original type, no :c:member:`~PyTypeObject.tp_init` function is called; if :c:member:`~PyTypeObject.tp_new` returns an instance of a subtype of the original type, the subtype's :c:member:`~PyTypeObject.tp_init` is called. Returns ``0`` on success, ``-1`` and sets an exception on error. **Inheritance:** This field is inherited by subtypes. **Default:** For :ref:`static types ` this field does not have a default. .. c:member:: allocfunc PyTypeObject.tp_alloc .. corresponding-type-slot:: Py_tp_alloc An optional pointer to an instance allocation function. The function signature is:: PyObject *tp_alloc(PyTypeObject *self, Py_ssize_t nitems); **Inheritance:** Static subtypes inherit this slot, which will be :c:func:`PyType_GenericAlloc` if inherited from :class:`object`. :ref:`Heap subtypes ` do not inherit this slot. **Default:** For heap subtypes, this field is always set to :c:func:`PyType_GenericAlloc`. For static subtypes, this slot is inherited (see above). .. c:member:: newfunc PyTypeObject.tp_new .. corresponding-type-slot:: Py_tp_new An optional pointer to an instance creation function. The function signature is:: PyObject *tp_new(PyTypeObject *subtype, PyObject *args, PyObject *kwds); The *subtype* argument is the type of the object being created; the *args* and *kwds* arguments represent positional and keyword arguments of the call to the type. Note that *subtype* doesn't have to equal the type whose :c:member:`~PyTypeObject.tp_new` function is called; it may be a subtype of that type (but not an unrelated type). The :c:member:`~PyTypeObject.tp_new` function should call ``subtype->tp_alloc(subtype, nitems)`` to allocate space for the object, and then do only as much further initialization as is absolutely necessary. Initialization that can safely be ignored or repeated should be placed in the :c:member:`~PyTypeObject.tp_init` handler. A good rule of thumb is that for immutable types, all initialization should take place in :c:member:`~PyTypeObject.tp_new`, while for mutable types, most initialization should be deferred to :c:member:`~PyTypeObject.tp_init`. Set the :c:macro:`Py_TPFLAGS_DISALLOW_INSTANTIATION` flag to disallow creating instances of the type in Python. **Inheritance:** This field is inherited by subtypes, except it is not inherited by :ref:`static types ` whose :c:member:`~PyTypeObject.tp_base` is ``NULL`` or ``&PyBaseObject_Type``. **Default:** For :ref:`static types ` this field has no default. This means if the slot is defined as ``NULL``, the type cannot be called to create new instances; presumably there is some other way to create instances, like a factory function. .. c:member:: freefunc PyTypeObject.tp_free .. corresponding-type-slot:: Py_tp_free An optional pointer to an instance deallocation function. Its signature is:: void tp_free(void *self); This function must free the memory allocated by :c:member:`~PyTypeObject.tp_alloc`. **Inheritance:** Static subtypes inherit this slot, which will be :c:func:`PyObject_Free` if inherited from :class:`object`. Exception: If the type supports garbage collection (i.e., the :c:macro:`Py_TPFLAGS_HAVE_GC` flag is set in :c:member:`~PyTypeObject.tp_flags`) and it would inherit :c:func:`PyObject_Free`, then this slot is not inherited but instead defaults to :c:func:`PyObject_GC_Del`. :ref:`Heap subtypes ` do not inherit this slot. **Default:** For :ref:`heap subtypes `, this slot defaults to a deallocator suitable to match :c:func:`PyType_GenericAlloc` and the value of the :c:macro:`Py_TPFLAGS_HAVE_GC` flag. For static subtypes, this slot is inherited (see above). .. c:member:: inquiry PyTypeObject.tp_is_gc .. corresponding-type-slot:: Py_tp_is_gc An optional pointer to a function called by the garbage collector. The garbage collector needs to know whether a particular object is collectible or not. Normally, it is sufficient to look at the object's type's :c:member:`~PyTypeObject.tp_flags` field, and check the :c:macro:`Py_TPFLAGS_HAVE_GC` flag bit. But some types have a mixture of statically and dynamically allocated instances, and the statically allocated instances are not collectible. Such types should define this function; it should return ``1`` for a collectible instance, and ``0`` for a non-collectible instance. The signature is:: int tp_is_gc(PyObject *self); (The only example of this are types themselves. The metatype, :c:data:`PyType_Type`, defines this function to distinguish between statically and :ref:`dynamically allocated types `.) **Inheritance:** This field is inherited by subtypes. **Default:** This slot has no default. If this field is ``NULL``, :c:macro:`Py_TPFLAGS_HAVE_GC` is used as the functional equivalent. .. c:member:: PyObject* PyTypeObject.tp_bases Tuple of base types. This field should be set to ``NULL`` and treated as read-only. Python will fill it in when the type is :c:func:`initialized `. For the corresponding slot ID, see :c:macro:`Py_tp_bases`. .. warning:: Multiple inheritance does not work well for statically defined types. If you set ``tp_bases`` to a tuple, Python will not raise an error, but some slots will only be inherited from the first base. **Inheritance:** This field is not inherited. .. c:member:: PyObject* PyTypeObject.tp_mro Tuple containing the expanded set of base types, starting with the type itself and ending with :class:`object`, in Method Resolution Order. This field should be set to ``NULL`` and treated as read-only. Python will fill it in when the type is :c:func:`initialized `. **Inheritance:** This field is not inherited; it is calculated fresh by :c:func:`PyType_Ready`. .. c:member:: PyObject* PyTypeObject.tp_cache Unused. Internal use only. **Inheritance:** This field is not inherited. .. c:member:: void* PyTypeObject.tp_subclasses A collection of subclasses. Internal use only. May be an invalid pointer. To get a list of subclasses, call the Python method :py:meth:`~type.__subclasses__`. .. versionchanged:: 3.12 For some types, this field does not hold a valid :c:expr:`PyObject*`. The type was changed to :c:expr:`void*` to indicate this. **Inheritance:** This field is not inherited. .. c:member:: PyObject* PyTypeObject.tp_weaklist Weak reference list head, for weak references to this type object. Not inherited. Internal use only. .. versionchanged:: 3.12 Internals detail: For the static builtin types this is always ``NULL``, even if weakrefs are added. Instead, the weakrefs for each are stored on ``PyInterpreterState``. Use the public C-API or the internal ``_PyObject_GET_WEAKREFS_LISTPTR()`` macro to avoid the distinction. **Inheritance:** This field is not inherited. .. c:member:: destructor PyTypeObject.tp_del .. corresponding-type-slot:: Py_tp_del This field is deprecated. Use :c:member:`~PyTypeObject.tp_finalize` instead. .. c:member:: unsigned int PyTypeObject.tp_version_tag Used to index into the method cache. Internal use only. **Inheritance:** This field is not inherited. .. c:member:: destructor PyTypeObject.tp_finalize .. corresponding-type-slot:: Py_tp_finalize An optional pointer to an instance finalization function. This is the C implementation of the :meth:`~object.__del__` special method. Its signature is:: void tp_finalize(PyObject *self); The primary purpose of finalization is to perform any non-trivial cleanup that must be performed before the object is destroyed, while the object and any other objects it directly or indirectly references are still in a consistent state. The finalizer is allowed to execute arbitrary Python code. Before Python automatically finalizes an object, some of the object's direct or indirect referents might have themselves been automatically finalized. However, none of the referents will have been automatically cleared (:c:member:`~PyTypeObject.tp_clear`) yet. Other non-finalized objects might still be using a finalized object, so the finalizer must leave the object in a sane state (e.g., invariants are still met). .. note:: After Python automatically finalizes an object, Python might start automatically clearing (:c:member:`~PyTypeObject.tp_clear`) the object and its referents (direct and indirect). Cleared objects are not guaranteed to be in a consistent state; a finalized object must be able to tolerate cleared referents. .. note:: An object is not guaranteed to be automatically finalized before its destructor (:c:member:`~PyTypeObject.tp_dealloc`) is called. It is recommended to call :c:func:`PyObject_CallFinalizerFromDealloc` at the beginning of :c:member:`!tp_dealloc` to guarantee that the object is always finalized before destruction. .. note:: The :c:member:`~PyTypeObject.tp_finalize` function can be called from any thread, although the :term:`GIL` will be held. .. note:: The :c:member:`!tp_finalize` function can be called during shutdown, after some global variables have been deleted. See the documentation of the :meth:`~object.__del__` method for details. When Python finalizes an object, it behaves like the following algorithm: #. Python might mark the object as *finalized*. Currently, Python always marks objects whose type supports garbage collection (i.e., the :c:macro:`Py_TPFLAGS_HAVE_GC` flag is set in :c:member:`~PyTypeObject.tp_flags`) and never marks other types of objects; this might change in a future version. #. If the object is not marked as *finalized* and its :c:member:`!tp_finalize` finalizer function is non-``NULL``, the finalizer function is called. #. If the finalizer function was called and the finalizer made the object reachable (i.e., there is a reference to the object and it is not a member of a :term:`cyclic isolate`), then the finalizer is said to have *resurrected* the object. It is unspecified whether the finalizer can also resurrect the object by adding a new reference to the object that does not make it reachable, i.e., the object is (still) a member of a cyclic isolate. #. If the finalizer resurrected the object, the object's pending destruction is canceled and the object's *finalized* mark might be removed if present. Currently, Python never removes the *finalized* mark; this might change in a future version. *Automatic finalization* refers to any finalization performed by Python except via calls to :c:func:`PyObject_CallFinalizer` or :c:func:`PyObject_CallFinalizerFromDealloc`. No guarantees are made about when, if, or how often an object is automatically finalized, except: * Python will not automatically finalize an object if it is reachable, i.e., there is a reference to it and it is not a member of a :term:`cyclic isolate`. * Python will not automatically finalize an object if finalizing it would not mark the object as *finalized*. Currently, this applies to objects whose type does not support garbage collection, i.e., the :c:macro:`Py_TPFLAGS_HAVE_GC` flag is not set. Such objects can still be manually finalized by calling :c:func:`PyObject_CallFinalizer` or :c:func:`PyObject_CallFinalizerFromDealloc`. * Python will not automatically finalize any two members of a :term:`cyclic isolate` concurrently. * Python will not automatically finalize an object after it has automatically cleared (:c:member:`~PyTypeObject.tp_clear`) the object. * If an object is a member of a :term:`cyclic isolate`, Python will not automatically finalize it after automatically clearing (see :c:member:`~PyTypeObject.tp_clear`) any other member. * Python will automatically finalize every member of a :term:`cyclic isolate` before it automatically clears (see :c:member:`~PyTypeObject.tp_clear`) any of them. * If Python is going to automatically clear an object (:c:member:`~PyTypeObject.tp_clear`), it will automatically finalize the object first. Python currently only automatically finalizes objects that are members of a :term:`cyclic isolate`, but future versions might finalize objects regularly before their destruction. To manually finalize an object, do not call this function directly; call :c:func:`PyObject_CallFinalizer` or :c:func:`PyObject_CallFinalizerFromDealloc` instead. :c:member:`~PyTypeObject.tp_finalize` should leave the current exception status unchanged. The recommended way to write a non-trivial finalizer is to back up the exception at the beginning by calling :c:func:`PyErr_GetRaisedException` and restore the exception at the end by calling :c:func:`PyErr_SetRaisedException`. If an exception is encountered in the middle of the finalizer, log and clear it with :c:func:`PyErr_WriteUnraisable` or :c:func:`PyErr_FormatUnraisable`. For example:: static void foo_finalize(PyObject *self) { // Save the current exception, if any. PyObject *exc = PyErr_GetRaisedException(); // ... if (do_something_that_might_raise() != success_indicator) { PyErr_WriteUnraisable(self); goto done; } done: // Restore the saved exception. This silently discards any exception // raised above, so be sure to call PyErr_WriteUnraisable first if // necessary. PyErr_SetRaisedException(exc); } **Inheritance:** This field is inherited by subtypes. .. versionadded:: 3.4 .. versionchanged:: 3.8 Before version 3.8 it was necessary to set the :c:macro:`Py_TPFLAGS_HAVE_FINALIZE` flags bit in order for this field to be used. This is no longer required. .. seealso:: * :pep:`442`: "Safe object finalization" * :ref:`life-cycle` for details about how this slot relates to other slots. * :c:func:`PyObject_CallFinalizer` * :c:func:`PyObject_CallFinalizerFromDealloc` .. c:member:: vectorcallfunc PyTypeObject.tp_vectorcall .. corresponding-type-slot:: Py_tp_vectorcall A :ref:`vectorcall function ` to use for calls of this type object (rather than instances). In other words, ``tp_vectorcall`` can be used to optimize ``type.__call__``, which typically returns a new instance of *type*. As with any vectorcall function, if ``tp_vectorcall`` is ``NULL``, the *tp_call* protocol (``Py_TYPE(type)->tp_call``) is used instead. .. note:: The :ref:`vectorcall protocol ` requires that the vectorcall function has the same behavior as the corresponding ``tp_call``. This means that ``type->tp_vectorcall`` must match the behavior of ``Py_TYPE(type)->tp_call``. Specifically, if *type* uses the default metaclass, ``type->tp_vectorcall`` must behave the same as :c:expr:`PyType_Type->tp_call`, which: - calls ``type->tp_new``, - if the result is a subclass of *type*, calls ``type->tp_init`` on the result of ``tp_new``, and - returns the result of ``tp_new``. Typically, ``tp_vectorcall`` is overridden to optimize this process for specific :c:member:`~PyTypeObject.tp_new` and :c:member:`~PyTypeObject.tp_init`. When doing this for user-subclassable types, note that both can be overridden (using :py:func:`~object.__new__` and :py:func:`~object.__init__`, respectively). **Inheritance:** This field is never inherited. .. versionadded:: 3.9 (the field exists since 3.8 but it's only used since 3.9) .. c:member:: unsigned char PyTypeObject.tp_watched Internal. Do not use. .. versionadded:: 3.12 .. _static-types: Static Types ------------ Traditionally, types defined in C code are *static*, that is, a static :c:type:`PyTypeObject` structure is defined directly in code and initialized using :c:func:`PyType_Ready`. This results in types that are limited relative to types defined in Python: * Static types are limited to one base, i.e. they cannot use multiple inheritance. * Static type objects (but not necessarily their instances) are immutable. It is not possible to add or modify the type object's attributes from Python. * Static type objects are shared across :ref:`sub-interpreters `, so they should not include any subinterpreter-specific state. Also, since :c:type:`PyTypeObject` is only part of the :ref:`Limited API ` as an opaque struct, any extension modules using static types must be compiled for a specific Python minor version. .. _heap-types: Heap Types ---------- An alternative to :ref:`static types ` is *heap-allocated types*, or *heap types* for short, which correspond closely to classes created by Python's ``class`` statement. Heap types have the :c:macro:`Py_TPFLAGS_HEAPTYPE` flag set. This is done by filling a :c:type:`PyType_Spec` structure and calling :c:func:`PyType_FromSpec`, :c:func:`PyType_FromSpecWithBases`, :c:func:`PyType_FromModuleAndSpec`, or :c:func:`PyType_FromMetaclass`. .. _number-structs: Number Object Structures ------------------------ .. c:type:: PyNumberMethods This structure holds pointers to the functions which an object uses to implement the number protocol. Each function is used by the function of similar name documented in the :ref:`number` section. .. XXX Drop the definition? Here is the structure definition:: typedef struct { binaryfunc nb_add; binaryfunc nb_subtract; binaryfunc nb_multiply; binaryfunc nb_remainder; binaryfunc nb_divmod; ternaryfunc nb_power; unaryfunc nb_negative; unaryfunc nb_positive; unaryfunc nb_absolute; inquiry nb_bool; unaryfunc nb_invert; binaryfunc nb_lshift; binaryfunc nb_rshift; binaryfunc nb_and; binaryfunc nb_xor; binaryfunc nb_or; unaryfunc nb_int; void *nb_reserved; unaryfunc nb_float; binaryfunc nb_inplace_add; binaryfunc nb_inplace_subtract; binaryfunc nb_inplace_multiply; binaryfunc nb_inplace_remainder; ternaryfunc nb_inplace_power; binaryfunc nb_inplace_lshift; binaryfunc nb_inplace_rshift; binaryfunc nb_inplace_and; binaryfunc nb_inplace_xor; binaryfunc nb_inplace_or; binaryfunc nb_floor_divide; binaryfunc nb_true_divide; binaryfunc nb_inplace_floor_divide; binaryfunc nb_inplace_true_divide; unaryfunc nb_index; binaryfunc nb_matrix_multiply; binaryfunc nb_inplace_matrix_multiply; } PyNumberMethods; .. note:: Binary and ternary functions must check the type of all their operands, and implement the necessary conversions (at least one of the operands is an instance of the defined type). If the operation is not defined for the given operands, binary and ternary functions must return ``Py_NotImplemented``, if another error occurred they must return ``NULL`` and set an exception. .. note:: The :c:member:`~PyNumberMethods.nb_reserved` field should always be ``NULL``. It was previously called :c:member:`!nb_long`, and was renamed in Python 3.0.1. .. c:member:: binaryfunc PyNumberMethods.nb_add .. corresponding-type-slot:: Py_nb_add .. c:member:: binaryfunc PyNumberMethods.nb_subtract .. corresponding-type-slot:: Py_nb_subtract .. c:member:: binaryfunc PyNumberMethods.nb_multiply .. corresponding-type-slot:: Py_nb_multiply .. c:member:: binaryfunc PyNumberMethods.nb_remainder .. corresponding-type-slot:: Py_nb_remainder .. c:member:: binaryfunc PyNumberMethods.nb_divmod .. corresponding-type-slot:: Py_nb_divmod .. c:member:: ternaryfunc PyNumberMethods.nb_power .. corresponding-type-slot:: Py_nb_power .. c:member:: unaryfunc PyNumberMethods.nb_negative .. corresponding-type-slot:: Py_nb_negative .. c:member:: unaryfunc PyNumberMethods.nb_positive .. corresponding-type-slot:: Py_nb_positive .. c:member:: unaryfunc PyNumberMethods.nb_absolute .. corresponding-type-slot:: Py_nb_absolute .. c:member:: inquiry PyNumberMethods.nb_bool .. corresponding-type-slot:: Py_nb_bool .. c:member:: unaryfunc PyNumberMethods.nb_invert .. corresponding-type-slot:: Py_nb_invert .. c:member:: binaryfunc PyNumberMethods.nb_lshift .. corresponding-type-slot:: Py_nb_lshift .. c:member:: binaryfunc PyNumberMethods.nb_rshift .. corresponding-type-slot:: Py_nb_rshift .. c:member:: binaryfunc PyNumberMethods.nb_and .. corresponding-type-slot:: Py_nb_and .. c:member:: binaryfunc PyNumberMethods.nb_xor .. corresponding-type-slot:: Py_nb_xor .. c:member:: binaryfunc PyNumberMethods.nb_or .. corresponding-type-slot:: Py_nb_or .. c:member:: unaryfunc PyNumberMethods.nb_int .. corresponding-type-slot:: Py_nb_int .. c:member:: void *PyNumberMethods.nb_reserved .. c:member:: unaryfunc PyNumberMethods.nb_float .. corresponding-type-slot:: Py_nb_float .. c:member:: binaryfunc PyNumberMethods.nb_inplace_add .. corresponding-type-slot:: Py_nb_inplace_add .. c:member:: binaryfunc PyNumberMethods.nb_inplace_subtract .. corresponding-type-slot:: Py_nb_inplace_subtract .. c:member:: binaryfunc PyNumberMethods.nb_inplace_multiply .. corresponding-type-slot:: Py_nb_inplace_multiply .. c:member:: binaryfunc PyNumberMethods.nb_inplace_remainder .. corresponding-type-slot:: Py_nb_inplace_remainder .. c:member:: ternaryfunc PyNumberMethods.nb_inplace_power .. corresponding-type-slot:: Py_nb_inplace_power .. c:member:: binaryfunc PyNumberMethods.nb_inplace_lshift .. corresponding-type-slot:: Py_nb_inplace_lshift .. c:member:: binaryfunc PyNumberMethods.nb_inplace_rshift .. corresponding-type-slot:: Py_nb_inplace_rshift .. c:member:: binaryfunc PyNumberMethods.nb_inplace_and .. corresponding-type-slot:: Py_nb_inplace_and .. c:member:: binaryfunc PyNumberMethods.nb_inplace_xor .. corresponding-type-slot:: Py_nb_inplace_xor .. c:member:: binaryfunc PyNumberMethods.nb_inplace_or .. corresponding-type-slot:: Py_nb_inplace_or .. c:member:: binaryfunc PyNumberMethods.nb_floor_divide .. corresponding-type-slot:: Py_nb_floor_divide .. c:member:: binaryfunc PyNumberMethods.nb_true_divide .. corresponding-type-slot:: Py_nb_true_divide .. c:member:: binaryfunc PyNumberMethods.nb_inplace_floor_divide .. corresponding-type-slot:: Py_nb_inplace_floor_divide .. c:member:: binaryfunc PyNumberMethods.nb_inplace_true_divide .. corresponding-type-slot:: Py_nb_inplace_true_divide .. c:member:: unaryfunc PyNumberMethods.nb_index .. corresponding-type-slot:: Py_nb_index .. c:member:: binaryfunc PyNumberMethods.nb_matrix_multiply .. corresponding-type-slot:: Py_nb_matrix_multiply .. c:member:: binaryfunc PyNumberMethods.nb_inplace_matrix_multiply .. corresponding-type-slot:: Py_nb_inplace_matrix_multiply .. _mapping-structs: Mapping Object Structures ------------------------- .. c:type:: PyMappingMethods This structure holds pointers to the functions which an object uses to implement the mapping protocol. It has three members: .. c:member:: lenfunc PyMappingMethods.mp_length .. corresponding-type-slot:: Py_mp_length This function is used by :c:func:`PyMapping_Size` and :c:func:`PyObject_Size`, and has the same signature. This slot may be set to ``NULL`` if the object has no defined length. .. c:member:: binaryfunc PyMappingMethods.mp_subscript .. corresponding-type-slot:: Py_mp_subscript This function is used by :c:func:`PyObject_GetItem` and :c:func:`PySequence_GetSlice`, and has the same signature as :c:func:`!PyObject_GetItem`. This slot must be filled for the :c:func:`PyMapping_Check` function to return ``1``, it can be ``NULL`` otherwise. .. c:member:: objobjargproc PyMappingMethods.mp_ass_subscript .. corresponding-type-slot:: Py_mp_ass_subscript This function is used by :c:func:`PyObject_SetItem`, :c:func:`PyObject_DelItem`, :c:func:`PySequence_SetSlice` and :c:func:`PySequence_DelSlice`. It has the same signature as :c:func:`!PyObject_SetItem`, but *v* can also be set to ``NULL`` to delete an item. If this slot is ``NULL``, the object does not support item assignment and deletion. .. _sequence-structs: Sequence Object Structures -------------------------- .. c:type:: PySequenceMethods This structure holds pointers to the functions which an object uses to implement the sequence protocol. .. c:member:: lenfunc PySequenceMethods.sq_length .. corresponding-type-slot:: Py_sq_length This function is used by :c:func:`PySequence_Size` and :c:func:`PyObject_Size`, and has the same signature. It is also used for handling negative indices via the :c:member:`~PySequenceMethods.sq_item` and the :c:member:`~PySequenceMethods.sq_ass_item` slots. .. c:member:: binaryfunc PySequenceMethods.sq_concat .. corresponding-type-slot:: Py_sq_concat This function is used by :c:func:`PySequence_Concat` and has the same signature. It is also used by the ``+`` operator, after trying the numeric addition via the :c:member:`~PyNumberMethods.nb_add` slot. .. c:member:: ssizeargfunc PySequenceMethods.sq_repeat .. corresponding-type-slot:: Py_sq_repeat This function is used by :c:func:`PySequence_Repeat` and has the same signature. It is also used by the ``*`` operator, after trying numeric multiplication via the :c:member:`~PyNumberMethods.nb_multiply` slot. .. c:member:: ssizeargfunc PySequenceMethods.sq_item .. corresponding-type-slot:: Py_sq_item This function is used by :c:func:`PySequence_GetItem` and has the same signature. It is also used by :c:func:`PyObject_GetItem`, after trying the subscription via the :c:member:`~PyMappingMethods.mp_subscript` slot. This slot must be filled for the :c:func:`PySequence_Check` function to return ``1``, it can be ``NULL`` otherwise. Negative indexes are handled as follows: if the :c:member:`~PySequenceMethods.sq_length` slot is filled, it is called and the sequence length is used to compute a positive index which is passed to :c:member:`~PySequenceMethods.sq_item`. If :c:member:`!sq_length` is ``NULL``, the index is passed as is to the function. .. c:member:: ssizeobjargproc PySequenceMethods.sq_ass_item .. corresponding-type-slot:: Py_sq_ass_item This function is used by :c:func:`PySequence_SetItem` and has the same signature. It is also used by :c:func:`PyObject_SetItem` and :c:func:`PyObject_DelItem`, after trying the item assignment and deletion via the :c:member:`~PyMappingMethods.mp_ass_subscript` slot. This slot may be left to ``NULL`` if the object does not support item assignment and deletion. .. c:member:: objobjproc PySequenceMethods.sq_contains .. corresponding-type-slot:: Py_sq_contains This function may be used by :c:func:`PySequence_Contains` and has the same signature. This slot may be left to ``NULL``, in this case :c:func:`!PySequence_Contains` simply traverses the sequence until it finds a match. .. c:member:: binaryfunc PySequenceMethods.sq_inplace_concat .. corresponding-type-slot:: Py_sq_inplace_concat This function is used by :c:func:`PySequence_InPlaceConcat` and has the same signature. It should modify its first operand, and return it. This slot may be left to ``NULL``, in this case :c:func:`!PySequence_InPlaceConcat` will fall back to :c:func:`PySequence_Concat`. It is also used by the augmented assignment ``+=``, after trying numeric in-place addition via the :c:member:`~PyNumberMethods.nb_inplace_add` slot. .. c:member:: ssizeargfunc PySequenceMethods.sq_inplace_repeat .. corresponding-type-slot:: Py_sq_inplace_repeat This function is used by :c:func:`PySequence_InPlaceRepeat` and has the same signature. It should modify its first operand, and return it. This slot may be left to ``NULL``, in this case :c:func:`!PySequence_InPlaceRepeat` will fall back to :c:func:`PySequence_Repeat`. It is also used by the augmented assignment ``*=``, after trying numeric in-place multiplication via the :c:member:`~PyNumberMethods.nb_inplace_multiply` slot. .. _buffer-structs: Buffer Object Structures ------------------------ .. c:type:: PyBufferProcs This structure holds pointers to the functions required by the :ref:`Buffer protocol `. The protocol defines how an exporter object can expose its internal data to consumer objects. .. c:member:: getbufferproc PyBufferProcs.bf_getbuffer .. corresponding-type-slot:: Py_bf_getbuffer The signature of this function is:: int (PyObject *exporter, Py_buffer *view, int flags); Handle a request to *exporter* to fill in *view* as specified by *flags*. Except for point (3), an implementation of this function MUST take these steps: (1) Check if the request can be met. If not, raise :exc:`BufferError`, set ``view->obj`` to ``NULL`` and return ``-1``. (2) Fill in the requested fields. (3) Increment an internal counter for the number of exports. (4) Set ``view->obj`` to *exporter* and increment ``view->obj``. (5) Return ``0``. **Thread safety:** In the :term:`free-threaded build`, implementations must ensure: * The export counter increment in step (3) is atomic. * The underlying buffer data remains valid and at a stable memory location for the lifetime of all exports. * For objects that support resizing or reallocation (such as :class:`bytearray`), the export counter is checked atomically before such operations, and :exc:`BufferError` is raised if exports exist. * The function is safe to call concurrently from multiple threads. See also :ref:`thread-safety-memoryview` for the Python-level thread safety guarantees of :class:`memoryview` objects. If *exporter* is part of a chain or tree of buffer providers, two main schemes can be used: * Re-export: Each member of the tree acts as the exporting object and sets ``view->obj`` to a new reference to itself. * Redirect: The buffer request is redirected to the root object of the tree. Here, ``view->obj`` will be a new reference to the root object. The individual fields of *view* are described in section :ref:`Buffer structure `, the rules how an exporter must react to specific requests are in section :ref:`Buffer request types `. All memory pointed to in the :c:type:`Py_buffer` structure belongs to the exporter and must remain valid until there are no consumers left. :c:member:`~Py_buffer.format`, :c:member:`~Py_buffer.shape`, :c:member:`~Py_buffer.strides`, :c:member:`~Py_buffer.suboffsets` and :c:member:`~Py_buffer.internal` are read-only for the consumer. :c:func:`PyBuffer_FillInfo` provides an easy way of exposing a simple bytes buffer while dealing correctly with all request types. :c:func:`PyObject_GetBuffer` is the interface for the consumer that wraps this function. .. c:member:: releasebufferproc PyBufferProcs.bf_releasebuffer .. corresponding-type-slot:: Py_bf_releasebuffer The signature of this function is:: void (PyObject *exporter, Py_buffer *view); Handle a request to release the resources of the buffer. If no resources need to be released, :c:member:`PyBufferProcs.bf_releasebuffer` may be ``NULL``. Otherwise, a standard implementation of this function will take these optional steps: (1) Decrement an internal counter for the number of exports. (2) If the counter is ``0``, free all memory associated with *view*. **Thread safety:** In the :term:`free-threaded build`: * The export counter decrement in step (1) must be atomic. * Resource cleanup when the counter reaches zero must be done atomically, as the final release may race with concurrent releases from other threads and deallocation must only happen once. The exporter MUST use the :c:member:`~Py_buffer.internal` field to keep track of buffer-specific resources. This field is guaranteed to remain constant, while a consumer MAY pass a copy of the original buffer as the *view* argument. This function MUST NOT decrement ``view->obj``, since that is done automatically in :c:func:`PyBuffer_Release` (this scheme is useful for breaking reference cycles). :c:func:`PyBuffer_Release` is the interface for the consumer that wraps this function. .. _async-structs: Async Object Structures ----------------------- .. versionadded:: 3.5 .. c:type:: PyAsyncMethods This structure holds pointers to the functions required to implement :term:`awaitable` and :term:`asynchronous iterator` objects. Here is the structure definition:: typedef struct { unaryfunc am_await; unaryfunc am_aiter; unaryfunc am_anext; sendfunc am_send; } PyAsyncMethods; .. c:member:: unaryfunc PyAsyncMethods.am_await .. corresponding-type-slot:: Py_am_await The signature of this function is:: PyObject *am_await(PyObject *self); The returned object must be an :term:`iterator`, i.e. :c:func:`PyIter_Check` must return ``1`` for it. This slot may be set to ``NULL`` if an object is not an :term:`awaitable`. .. c:member:: unaryfunc PyAsyncMethods.am_aiter .. corresponding-type-slot:: Py_am_aiter The signature of this function is:: PyObject *am_aiter(PyObject *self); Must return an :term:`asynchronous iterator` object. See :meth:`~object.__anext__` for details. This slot may be set to ``NULL`` if an object does not implement asynchronous iteration protocol. .. c:member:: unaryfunc PyAsyncMethods.am_anext .. corresponding-type-slot:: Py_am_anext The signature of this function is:: PyObject *am_anext(PyObject *self); Must return an :term:`awaitable` object. See :meth:`~object.__anext__` for details. This slot may be set to ``NULL``. .. c:member:: sendfunc PyAsyncMethods.am_send .. corresponding-type-slot:: Py_am_send The signature of this function is:: PySendResult am_send(PyObject *self, PyObject *arg, PyObject **result); See :c:func:`PyIter_Send` for details. This slot may be set to ``NULL``. .. versionadded:: 3.10 .. _slot-typedefs: Slot Type typedefs ------------------ .. c:type:: PyObject *(*allocfunc)(PyTypeObject *cls, Py_ssize_t nitems) The purpose of this function is to separate memory allocation from memory initialization. It should return a pointer to a block of memory of adequate length for the instance, suitably aligned, and initialized to zeros, but with :c:member:`~PyObject.ob_refcnt` set to ``1`` and :c:member:`~PyObject.ob_type` set to the type argument. If the type's :c:member:`~PyTypeObject.tp_itemsize` is non-zero, the object's :c:member:`~PyVarObject.ob_size` field should be initialized to *nitems* and the length of the allocated memory block should be ``tp_basicsize + nitems*tp_itemsize``, rounded up to a multiple of ``sizeof(void*)``; otherwise, *nitems* is not used and the length of the block should be :c:member:`~PyTypeObject.tp_basicsize`. This function should not do any other instance initialization, not even to allocate additional memory; that should be done by :c:member:`~PyTypeObject.tp_new`. .. c:type:: void (*destructor)(PyObject *) .. c:type:: void (*freefunc)(void *) See :c:member:`~PyTypeObject.tp_free`. .. c:type:: PyObject *(*newfunc)(PyTypeObject *, PyObject *, PyObject *) See :c:member:`~PyTypeObject.tp_new`. .. c:type:: int (*initproc)(PyObject *, PyObject *, PyObject *) See :c:member:`~PyTypeObject.tp_init`. .. c:type:: PyObject *(*reprfunc)(PyObject *) See :c:member:`~PyTypeObject.tp_repr`. .. c:type:: PyObject *(*getattrfunc)(PyObject *self, char *attr) Return the value of the named attribute for the object. .. c:type:: int (*setattrfunc)(PyObject *self, char *attr, PyObject *value) Set the value of the named attribute for the object. The value argument is set to ``NULL`` to delete the attribute. .. c:type:: PyObject *(*getattrofunc)(PyObject *self, PyObject *attr) Return the value of the named attribute for the object. See :c:member:`~PyTypeObject.tp_getattro`. .. c:type:: int (*setattrofunc)(PyObject *self, PyObject *attr, PyObject *value) Set the value of the named attribute for the object. The value argument is set to ``NULL`` to delete the attribute. See :c:member:`~PyTypeObject.tp_setattro`. .. c:type:: PyObject *(*descrgetfunc)(PyObject *, PyObject *, PyObject *) See :c:member:`~PyTypeObject.tp_descr_get`. .. c:type:: int (*descrsetfunc)(PyObject *, PyObject *, PyObject *) See :c:member:`~PyTypeObject.tp_descr_set`. .. c:type:: Py_hash_t (*hashfunc)(PyObject *) See :c:member:`~PyTypeObject.tp_hash`. .. c:type:: PyObject *(*richcmpfunc)(PyObject *, PyObject *, int) See :c:member:`~PyTypeObject.tp_richcompare`. .. c:type:: PyObject *(*getiterfunc)(PyObject *) See :c:member:`~PyTypeObject.tp_iter`. .. c:type:: PyObject *(*iternextfunc)(PyObject *) See :c:member:`~PyTypeObject.tp_iternext`. .. c:type:: Py_ssize_t (*lenfunc)(PyObject *) .. c:type:: int (*getbufferproc)(PyObject *, Py_buffer *, int) .. c:type:: void (*releasebufferproc)(PyObject *, Py_buffer *) .. c:type:: PyObject *(*unaryfunc)(PyObject *) .. c:type:: PyObject *(*binaryfunc)(PyObject *, PyObject *) .. c:type:: PySendResult (*sendfunc)(PyObject *, PyObject *, PyObject **) See :c:member:`~PyAsyncMethods.am_send`. .. c:type:: PyObject *(*ternaryfunc)(PyObject *, PyObject *, PyObject *) .. c:type:: PyObject *(*ssizeargfunc)(PyObject *, Py_ssize_t) .. c:type:: int (*ssizeobjargproc)(PyObject *, Py_ssize_t, PyObject *) .. c:type:: int (*objobjproc)(PyObject *, PyObject *) .. c:type:: int (*objobjargproc)(PyObject *, PyObject *, PyObject *) .. _typedef-examples: Examples -------- The following are simple examples of Python type definitions. They include common usage you may encounter. Some demonstrate tricky corner cases. For more examples, practical info, and a tutorial, see :ref:`defining-new-types` and :ref:`new-types-topics`. A basic :ref:`static type `:: typedef struct { PyObject_HEAD const char *data; } MyObject; static PyTypeObject MyObject_Type = { PyVarObject_HEAD_INIT(NULL, 0) .tp_name = "mymod.MyObject", .tp_basicsize = sizeof(MyObject), .tp_doc = PyDoc_STR("My objects"), .tp_new = myobj_new, .tp_dealloc = (destructor)myobj_dealloc, .tp_repr = (reprfunc)myobj_repr, }; You may also find older code (especially in the CPython code base) with a more verbose initializer:: static PyTypeObject MyObject_Type = { PyVarObject_HEAD_INIT(NULL, 0) "mymod.MyObject", /* tp_name */ sizeof(MyObject), /* tp_basicsize */ 0, /* tp_itemsize */ (destructor)myobj_dealloc, /* tp_dealloc */ 0, /* tp_vectorcall_offset */ 0, /* tp_getattr */ 0, /* tp_setattr */ 0, /* tp_as_async */ (reprfunc)myobj_repr, /* tp_repr */ 0, /* tp_as_number */ 0, /* tp_as_sequence */ 0, /* tp_as_mapping */ 0, /* tp_hash */ 0, /* tp_call */ 0, /* tp_str */ 0, /* tp_getattro */ 0, /* tp_setattro */ 0, /* tp_as_buffer */ 0, /* tp_flags */ PyDoc_STR("My objects"), /* tp_doc */ 0, /* tp_traverse */ 0, /* tp_clear */ 0, /* tp_richcompare */ 0, /* tp_weaklistoffset */ 0, /* tp_iter */ 0, /* tp_iternext */ 0, /* tp_methods */ 0, /* tp_members */ 0, /* tp_getset */ 0, /* tp_base */ 0, /* tp_dict */ 0, /* tp_descr_get */ 0, /* tp_descr_set */ 0, /* tp_dictoffset */ 0, /* tp_init */ 0, /* tp_alloc */ myobj_new, /* tp_new */ }; A type that supports weakrefs, instance dicts, and hashing:: typedef struct { PyObject_HEAD const char *data; } MyObject; static PyTypeObject MyObject_Type = { PyVarObject_HEAD_INIT(NULL, 0) .tp_name = "mymod.MyObject", .tp_basicsize = sizeof(MyObject), .tp_doc = PyDoc_STR("My objects"), .tp_flags = Py_TPFLAGS_DEFAULT | Py_TPFLAGS_BASETYPE | Py_TPFLAGS_HAVE_GC | Py_TPFLAGS_MANAGED_DICT | Py_TPFLAGS_MANAGED_WEAKREF, .tp_new = myobj_new, .tp_traverse = (traverseproc)myobj_traverse, .tp_clear = (inquiry)myobj_clear, .tp_alloc = PyType_GenericNew, .tp_dealloc = (destructor)myobj_dealloc, .tp_repr = (reprfunc)myobj_repr, .tp_hash = (hashfunc)myobj_hash, .tp_richcompare = PyBaseObject_Type.tp_richcompare, }; A str subclass that cannot be subclassed and cannot be called to create instances (e.g. uses a separate factory func) using :c:macro:`Py_TPFLAGS_DISALLOW_INSTANTIATION` flag:: typedef struct { PyUnicodeObject raw; char *extra; } MyStr; static PyTypeObject MyStr_Type = { PyVarObject_HEAD_INIT(NULL, 0) .tp_name = "mymod.MyStr", .tp_basicsize = sizeof(MyStr), .tp_base = NULL, // set to &PyUnicode_Type in module init .tp_doc = PyDoc_STR("my custom str"), .tp_flags = Py_TPFLAGS_DEFAULT | Py_TPFLAGS_DISALLOW_INSTANTIATION, .tp_repr = (reprfunc)myobj_repr, }; The simplest :ref:`static type ` with fixed-length instances:: typedef struct { PyObject_HEAD } MyObject; static PyTypeObject MyObject_Type = { PyVarObject_HEAD_INIT(NULL, 0) .tp_name = "mymod.MyObject", }; The simplest :ref:`static type ` with variable-length instances:: typedef struct { PyObject_VAR_HEAD const char *data[1]; } MyObject; static PyTypeObject MyObject_Type = { PyVarObject_HEAD_INIT(NULL, 0) .tp_name = "mymod.MyObject", .tp_basicsize = sizeof(MyObject) - sizeof(char *), .tp_itemsize = sizeof(char *), }; --- ### Doc C Api Unicode (`Doc/c-api/unicode.rst`) .. highlight:: c .. _unicodeobjects: Unicode Objects and Codecs -------------------------- Unicode Objects ^^^^^^^^^^^^^^^ Since the implementation of :pep:`393` in Python 3.3, Unicode objects internally use a variety of representations, in order to allow handling the complete range of Unicode characters while staying memory efficient. There are special cases for strings where all code points are below 128, 256, or 65536; otherwise, code points must be below 1114112 (which is the full Unicode range). UTF-8 representation is created on demand and cached in the Unicode object. .. impl-detail:: The internal buffer always includes an extra trailing null character for compatibility with null terminated C strings. This extra character is not counted in :c:func:`PyUnicode_GetLength` nor in the various *size* arguments of the functions below. .. note:: The :c:type:`Py_UNICODE` representation has been removed since Python 3.12 with deprecated APIs. See :pep:`623` for more information. Unicode Type """""""""""" These are the basic Unicode object types used for the Unicode implementation in Python: .. c:var:: PyTypeObject PyUnicode_Type This instance of :c:type:`PyTypeObject` represents the Python Unicode type. It is exposed to Python code as :py:class:`str`. .. c:var:: PyTypeObject PyUnicodeIter_Type This instance of :c:type:`PyTypeObject` represents the Python Unicode iterator type. It is used to iterate over Unicode string objects. .. c:type:: Py_UCS4 Py_UCS2 Py_UCS1 These types are typedefs for unsigned integer types wide enough to contain characters of 32 bits, 16 bits and 8 bits, respectively. When dealing with single Unicode characters, use :c:type:`Py_UCS4`. .. versionadded:: 3.3 .. c:type:: PyASCIIObject PyCompactUnicodeObject PyUnicodeObject These subtypes of :c:type:`PyObject` represent a Python Unicode object. In almost all cases, they shouldn't be used directly, since all API functions that deal with Unicode objects take and return :c:type:`PyObject` pointers. .. versionadded:: 3.3 The structure of a particular object can be determined using the following macros. The macros cannot fail; their behavior is undefined if their argument is not a Python Unicode object. .. c:namespace:: NULL .. c:macro:: PyUnicode_IS_COMPACT(o) True if *o* uses the :c:struct:`PyCompactUnicodeObject` structure. .. versionadded:: 3.3 .. c:macro:: PyUnicode_IS_COMPACT_ASCII(o) True if *o* uses the :c:struct:`PyASCIIObject` structure. .. versionadded:: 3.3 The following APIs are C macros and static inlined functions for fast checks and access to internal read-only data of Unicode objects: .. c:function:: int PyUnicode_Check(PyObject *obj) Return true if the object *obj* is a Unicode object or an instance of a Unicode subtype. This function always succeeds. .. c:function:: int PyUnicode_CheckExact(PyObject *obj) Return true if the object *obj* is a Unicode object, but not an instance of a subtype. This function always succeeds. .. c:function:: Py_ssize_t PyUnicode_GET_LENGTH(PyObject *unicode) Return the length of the Unicode string, in code points. *unicode* has to be a Unicode object in the "canonical" representation (not checked). .. versionadded:: 3.3 .. c:function:: Py_UCS1* PyUnicode_1BYTE_DATA(PyObject *unicode) Py_UCS2* PyUnicode_2BYTE_DATA(PyObject *unicode) Py_UCS4* PyUnicode_4BYTE_DATA(PyObject *unicode) Return a pointer to the canonical representation cast to UCS1, UCS2 or UCS4 integer types for direct character access. No checks are performed if the canonical representation has the correct character size; use :c:func:`PyUnicode_KIND` to select the right function. .. versionadded:: 3.3 .. c:macro:: PyUnicode_1BYTE_KIND PyUnicode_2BYTE_KIND PyUnicode_4BYTE_KIND Return values of the :c:func:`PyUnicode_KIND` macro. .. versionadded:: 3.3 .. versionchanged:: 3.12 ``PyUnicode_WCHAR_KIND`` has been removed. .. c:function:: int PyUnicode_KIND(PyObject *unicode) Return one of the PyUnicode kind constants (see above) that indicate how many bytes per character this Unicode object uses to store its data. *unicode* has to be a Unicode object in the "canonical" representation (not checked). .. versionadded:: 3.3 .. c:function:: void* PyUnicode_DATA(PyObject *unicode) Return a void pointer to the raw Unicode buffer. *unicode* has to be a Unicode object in the "canonical" representation (not checked). .. versionadded:: 3.3 .. c:function:: Py_UCS4 PyUnicode_READ(int kind, void *data, Py_ssize_t index) Read a code point from a canonical representation *data* (as obtained with :c:func:`PyUnicode_DATA`). No checks are performed. .. impl-detail:: Accept reading the trailing null character at index :c:func:`PyUnicode_GetLength`. .. versionadded:: 3.3 .. c:function:: Py_UCS4 PyUnicode_READ_CHAR(PyObject *unicode, Py_ssize_t index) Read a character from a Unicode object *unicode*, which must be in the "canonical" representation. This is less efficient than :c:func:`PyUnicode_READ` if you do multiple consecutive reads. .. impl-detail:: Accept reading the trailing null character at index :c:func:`PyUnicode_GetLength`. .. versionadded:: 3.3 .. c:function:: Py_UCS4 PyUnicode_MAX_CHAR_VALUE(PyObject *unicode) Return the maximum code point that is suitable for creating another string based on *unicode*, which must be in the "canonical" representation. This is always an approximation but more efficient than iterating over the string. .. versionadded:: 3.3 .. c:function:: int PyUnicode_IsIdentifier(PyObject *unicode) Return ``1`` if the string is a valid identifier according to the language definition, section :ref:`identifiers`. Return ``0`` otherwise. .. versionchanged:: 3.9 The function does not call :c:func:`Py_FatalError` anymore if the string is not ready. .. c:function:: unsigned int PyUnicode_IS_ASCII(PyObject *unicode) Return true if the string only contains ASCII characters. Equivalent to :py:meth:`str.isascii`. .. versionadded:: 3.2 .. c:function:: Py_hash_t PyUnstable_Unicode_GET_CACHED_HASH(PyObject *str) If the hash of *str*, as returned by :c:func:`PyObject_Hash`, has been cached and is immediately available, return it. Otherwise, return ``-1`` *without* setting an exception. If *str* is not a string (that is, if ``PyUnicode_Check(obj)`` is false), the behavior is undefined. This function never fails with an exception. Note that there are no guarantees on when an object's hash is cached, and the (non-)existence of a cached hash does not imply that the string has any other properties. Unicode Character Properties """""""""""""""""""""""""""" Unicode provides many different character properties. The most often needed ones are available through these macros which are mapped to C functions depending on the Python configuration. .. c:function:: int Py_UNICODE_ISSPACE(Py_UCS4 ch) Return ``1`` or ``0`` depending on whether *ch* is a whitespace character. .. c:function:: int Py_UNICODE_ISLOWER(Py_UCS4 ch) Return ``1`` or ``0`` depending on whether *ch* is a lowercase character. .. c:function:: int Py_UNICODE_ISUPPER(Py_UCS4 ch) Return ``1`` or ``0`` depending on whether *ch* is an uppercase character. .. c:function:: int Py_UNICODE_ISTITLE(Py_UCS4 ch) Return ``1`` or ``0`` depending on whether *ch* is a titlecase character. .. c:function:: int Py_UNICODE_ISLINEBREAK(Py_UCS4 ch) Return ``1`` or ``0`` depending on whether *ch* is a linebreak character. .. c:function:: int Py_UNICODE_ISDECIMAL(Py_UCS4 ch) Return ``1`` or ``0`` depending on whether *ch* is a decimal character. .. c:function:: int Py_UNICODE_ISDIGIT(Py_UCS4 ch) Return ``1`` or ``0`` depending on whether *ch* is a digit character. .. c:function:: int Py_UNICODE_ISNUMERIC(Py_UCS4 ch) Return ``1`` or ``0`` depending on whether *ch* is a numeric character. .. c:function:: int Py_UNICODE_ISALPHA(Py_UCS4 ch) Return ``1`` or ``0`` depending on whether *ch* is an alphabetic character. .. c:function:: int Py_UNICODE_ISALNUM(Py_UCS4 ch) Return ``1`` or ``0`` depending on whether *ch* is an alphanumeric character. .. c:function:: int Py_UNICODE_ISPRINTABLE(Py_UCS4 ch) Return ``1`` or ``0`` depending on whether *ch* is a printable character, in the sense of :meth:`str.isprintable`. These APIs can be used for fast direct character conversions: .. c:function:: Py_UCS4 Py_UNICODE_TOLOWER(Py_UCS4 ch) Return the character *ch* converted to lower case. .. c:function:: Py_UCS4 Py_UNICODE_TOUPPER(Py_UCS4 ch) Return the character *ch* converted to upper case. .. c:function:: Py_UCS4 Py_UNICODE_TOTITLE(Py_UCS4 ch) Return the character *ch* converted to title case. .. c:function:: int Py_UNICODE_TODECIMAL(Py_UCS4 ch) Return the character *ch* converted to a decimal positive integer. Return ``-1`` if this is not possible. This function does not raise exceptions. .. c:function:: int Py_UNICODE_TODIGIT(Py_UCS4 ch) Return the character *ch* converted to a single digit integer. Return ``-1`` if this is not possible. This function does not raise exceptions. .. c:function:: double Py_UNICODE_TONUMERIC(Py_UCS4 ch) Return the character *ch* converted to a double. Return ``-1.0`` if this is not possible. This function does not raise exceptions. These APIs can be used to work with surrogates: .. c:function:: int Py_UNICODE_IS_SURROGATE(Py_UCS4 ch) Check if *ch* is a surrogate (``0xD800 <= ch <= 0xDFFF``). .. c:function:: int Py_UNICODE_IS_HIGH_SURROGATE(Py_UCS4 ch) Check if *ch* is a high surrogate (``0xD800 <= ch <= 0xDBFF``). .. c:function:: int Py_UNICODE_IS_LOW_SURROGATE(Py_UCS4 ch) Check if *ch* is a low surrogate (``0xDC00 <= ch <= 0xDFFF``). .. c:function:: Py_UCS4 Py_UNICODE_HIGH_SURROGATE(Py_UCS4 ch) Return the high UTF-16 surrogate (``0xD800`` to ``0xDBFF``) for a Unicode code point in the range ``[0x10000; 0x10FFFF]``. .. c:function:: Py_UCS4 Py_UNICODE_LOW_SURROGATE(Py_UCS4 ch) Return the low UTF-16 surrogate (``0xDC00`` to ``0xDFFF``) for a Unicode code point in the range ``[0x10000; 0x10FFFF]``. .. c:function:: Py_UCS4 Py_UNICODE_JOIN_SURROGATES(Py_UCS4 high, Py_UCS4 low) Join two surrogate code points and return a single :c:type:`Py_UCS4` value. *high* and *low* are respectively the leading and trailing surrogates in a surrogate pair. *high* must be in the range ``[0xD800; 0xDBFF]`` and *low* must be in the range ``[0xDC00; 0xDFFF]``. Creating and accessing Unicode strings """""""""""""""""""""""""""""""""""""" To create Unicode objects and access their basic sequence properties, use these APIs: .. c:function:: PyObject* PyUnicode_FromKindAndData(int kind, const void *buffer, \ Py_ssize_t size) Create a new Unicode object with the given *kind* (possible values are :c:macro:`PyUnicode_1BYTE_KIND` etc., as returned by :c:func:`PyUnicode_KIND`). The *buffer* must point to an array of *size* units of 1, 2 or 4 bytes per character, as given by the kind. If necessary, the input *buffer* is copied and transformed into the canonical representation. For example, if the *buffer* is a UCS4 string (:c:macro:`PyUnicode_4BYTE_KIND`) and it consists only of codepoints in the UCS1 range, it will be transformed into UCS1 (:c:macro:`PyUnicode_1BYTE_KIND`). All characters must be in range [U+0000; U+10ffff]. If *kind* is :c:macro:`PyUnicode_4BYTE_KIND` and the string contains invalid characters, the behavior is undefined. .. versionadded:: 3.3 .. c:function:: PyObject* PyUnicode_FromStringAndSize(const char *str, Py_ssize_t size) Create a Unicode object from the char buffer *str*. The bytes will be interpreted as being UTF-8 encoded. The buffer is copied into the new object. The return value might be a shared object, i.e. modification of the data is not allowed. This function raises :exc:`SystemError` when: * *size* < 0, * *str* is ``NULL`` and *size* > 0 .. versionchanged:: 3.12 *str* == ``NULL`` with *size* > 0 is not allowed anymore. .. c:function:: PyObject *PyUnicode_FromString(const char *str) Create a Unicode object from a UTF-8 encoded null-terminated char buffer *str*. .. c:function:: PyObject* PyUnicode_FromFormat(const char *format, ...) Take a C :c:func:`printf`\ -style *format* string and a variable number of arguments, calculate the size of the resulting Python Unicode string and return a string with the values formatted into it. The variable arguments must be C types and must correspond exactly to the format characters in the *format* ASCII-encoded string. A conversion specifier contains two or more characters and has the following components, which must occur in this order: #. The ``'%'`` character, which marks the start of the specifier. #. Conversion flags (optional), which affect the result of some conversion types. #. Minimum field width (optional). If specified as an ``'*'`` (asterisk), the actual width is given in the next argument, which must be of type :c:expr:`int`, and the object to convert comes after the minimum field width and optional precision. #. Precision (optional), given as a ``'.'`` (dot) followed by the precision. If specified as ``'*'`` (an asterisk), the actual precision is given in the next argument, which must be of type :c:expr:`int`, and the value to convert comes after the precision. #. Length modifier (optional). #. Conversion type. The conversion flag characters are: .. tabularcolumns:: |l|L| +-------+-------------------------------------------------------------+ | Flag | Meaning | +=======+=============================================================+ | ``0`` | The conversion will be zero padded for numeric values. | +-------+-------------------------------------------------------------+ | ``-`` | The converted value is left adjusted (overrides the ``0`` | | | flag if both are given). | +-------+-------------------------------------------------------------+ The length modifiers for following integer conversions (``d``, ``i``, ``o``, ``u``, ``x``, or ``X``) specify the type of the argument (:c:expr:`int` by default): .. tabularcolumns:: |l|L| +----------+-----------------------------------------------------+ | Modifier | Types | +==========+=====================================================+ | ``l`` | :c:expr:`long` or :c:expr:`unsigned long` | +----------+-----------------------------------------------------+ | ``ll`` | :c:expr:`long long` or :c:expr:`unsigned long long` | +----------+-----------------------------------------------------+ | ``j`` | :c:type:`intmax_t` or :c:type:`uintmax_t` | +----------+-----------------------------------------------------+ | ``z`` | :c:type:`size_t` or :c:type:`ssize_t` | +----------+-----------------------------------------------------+ | ``t`` | :c:type:`ptrdiff_t` | +----------+-----------------------------------------------------+ The length modifier ``l`` for following conversions ``s`` or ``V`` specify that the type of the argument is :c:expr:`const wchar_t*`. The conversion specifiers are: .. list-table:: :widths: auto :header-rows: 1 * - Conversion Specifier - Type - Comment * - ``%`` - *n/a* - The literal ``%`` character. * - ``d``, ``i`` - Specified by the length modifier - The decimal representation of a signed C integer. * - ``u`` - Specified by the length modifier - The decimal representation of an unsigned C integer. * - ``o`` - Specified by the length modifier - The octal representation of an unsigned C integer. * - ``x`` - Specified by the length modifier - The hexadecimal representation of an unsigned C integer (lowercase). * - ``X`` - Specified by the length modifier - The hexadecimal representation of an unsigned C integer (uppercase). * - ``c`` - :c:expr:`int` - A single character. * - ``s`` - :c:expr:`const char*` or :c:expr:`const wchar_t*` - A null-terminated C character array. * - ``p`` - :c:expr:`const void*` - The hex representation of a C pointer. Mostly equivalent to ``printf("%p")`` except that it is guaranteed to start with the literal ``0x`` regardless of what the platform's ``printf`` yields. * - ``A`` - :c:expr:`PyObject*` - The result of calling :func:`ascii`. * - ``U`` - :c:expr:`PyObject*` - A Unicode object. * - ``V`` - :c:expr:`PyObject*`, :c:expr:`const char*` or :c:expr:`const wchar_t*` - A Unicode object (which may be ``NULL``) and a null-terminated C character array as a second parameter (which will be used, if the first parameter is ``NULL``). * - ``S`` - :c:expr:`PyObject*` - The result of calling :c:func:`PyObject_Str`. * - ``R`` - :c:expr:`PyObject*` - The result of calling :c:func:`PyObject_Repr`. * - ``T`` - :c:expr:`PyObject*` - Get the fully qualified name of an object type; call :c:func:`PyType_GetFullyQualifiedName`. * - ``#T`` - :c:expr:`PyObject*` - Similar to ``T`` format, but use a colon (``:``) as separator between the module name and the qualified name. * - ``N`` - :c:expr:`PyTypeObject*` - Get the fully qualified name of a type; call :c:func:`PyType_GetFullyQualifiedName`. * - ``#N`` - :c:expr:`PyTypeObject*` - Similar to ``N`` format, but use a colon (``:``) as separator between the module name and the qualified name. .. note:: The width formatter unit is number of characters rather than bytes. The precision formatter unit is number of bytes or :c:type:`wchar_t` items (if the length modifier ``l`` is used) for ``"%s"`` and ``"%V"`` (if the ``PyObject*`` argument is ``NULL``), and a number of characters for ``"%A"``, ``"%U"``, ``"%S"``, ``"%R"`` and ``"%V"`` (if the ``PyObject*`` argument is not ``NULL``). .. note:: Unlike to C :c:func:`printf` the ``0`` flag has effect even when a precision is given for integer conversions (``d``, ``i``, ``u``, ``o``, ``x``, or ``X``). .. versionchanged:: 3.2 Support for ``"%lld"`` and ``"%llu"`` added. .. versionchanged:: 3.3 Support for ``"%li"``, ``"%lli"`` and ``"%zi"`` added. .. versionchanged:: 3.4 Support width and precision formatter for ``"%s"``, ``"%A"``, ``"%U"``, ``"%V"``, ``"%S"``, ``"%R"`` added. .. versionchanged:: 3.12 Support for conversion specifiers ``o`` and ``X``. Support for length modifiers ``j`` and ``t``. Length modifiers are now applied to all integer conversions. Length modifier ``l`` is now applied to conversion specifiers ``s`` and ``V``. Support for variable width and precision ``*``. Support for flag ``-``. An unrecognized format character now sets a :exc:`SystemError`. In previous versions it caused all the rest of the format string to be copied as-is to the result string, and any extra arguments discarded. .. versionchanged:: 3.13 Support for ``%T``, ``%#T``, ``%N`` and ``%#N`` formats added. .. c:function:: PyObject* PyUnicode_FromFormatV(const char *format, va_list vargs) Identical to :c:func:`PyUnicode_FromFormat` except that it takes exactly two arguments. .. c:function:: PyObject* PyUnicode_FromObject(PyObject *obj) Copy an instance of a Unicode subtype to a new true Unicode object if necessary. If *obj* is already a true Unicode object (not a subtype), return a new :term:`strong reference` to the object. Objects other than Unicode or its subtypes will cause a :exc:`TypeError`. .. c:function:: PyObject* PyUnicode_FromOrdinal(int ordinal) Create a Unicode Object from the given Unicode code point *ordinal*. The ordinal must be in ``range(0x110000)``. A :exc:`ValueError` is raised in the case it is not. .. c:function:: PyObject* PyUnicode_FromEncodedObject(PyObject *obj, \ const char *encoding, const char *errors) Decode an encoded object *obj* to a Unicode object. :class:`bytes`, :class:`bytearray` and other :term:`bytes-like objects ` are decoded according to the given *encoding* and using the error handling defined by *errors*. Both can be ``NULL`` to have the interface use the default values (see :ref:`builtincodecs` for details). All other objects, including Unicode objects, cause a :exc:`TypeError` to be set. The API returns ``NULL`` if there was an error. The caller is responsible for decref'ing the returned objects. .. c:function:: void PyUnicode_Append(PyObject **p_left, PyObject *right) Append the string *right* to the end of *p_left*. *p_left* must point to a :term:`strong reference` to a Unicode object; :c:func:`!PyUnicode_Append` releases (":term:`steals `") this reference. On error, set *\*p_left* to ``NULL`` and set an exception. On success, set *\*p_left* to a new strong reference to the result. .. c:function:: void PyUnicode_AppendAndDel(PyObject **p_left, PyObject *right) The function is similar to :c:func:`PyUnicode_Append`, with the only difference being that it decrements the reference count of *right* by one. .. c:function:: PyObject* PyUnicode_BuildEncodingMap(PyObject* string) Return a mapping suitable for decoding a custom single-byte encoding. Given a Unicode string *string* of up to 256 characters representing an encoding table, returns either a compact internal mapping object or a dictionary mapping character ordinals to byte values. Raises a :exc:`TypeError` and return ``NULL`` on invalid input. .. versionadded:: 3.2 .. c:function:: const char* PyUnicode_GetDefaultEncoding(void) Return the name of the default string encoding, ``"utf-8"``. See :func:`sys.getdefaultencoding`. The returned string does not need to be freed, and is valid until interpreter shutdown. .. c:function:: Py_ssize_t PyUnicode_GetLength(PyObject *unicode) Return the length of the Unicode object, in code points. On error, set an exception and return ``-1``. .. impl-detail:: The length does not count the trailing null character. .. versionadded:: 3.3 .. c:function:: Py_UCS4 PyUnicode_ReadChar(PyObject *unicode, Py_ssize_t index) Read a character from a string. This function checks that *unicode* is a Unicode object and the index is not out of bounds, in contrast to :c:func:`PyUnicode_READ_CHAR`, which performs no error checking. Return character on success, ``-1`` on error with an exception set. .. impl-detail:: Do not accept reading the trailing null character at index :c:func:`PyUnicode_GetLength`. .. versionadded:: 3.3 .. c:function:: PyObject* PyUnicode_Substring(PyObject *unicode, Py_ssize_t start, \ Py_ssize_t end) Return a substring of *unicode*, from character index *start* (included) to character index *end* (excluded). Negative indices are not supported. On error, set an exception and return ``NULL``. .. versionadded:: 3.3 .. c:function:: Py_UCS4* PyUnicode_AsUCS4(PyObject *unicode, Py_UCS4 *buffer, \ Py_ssize_t buflen, int copy_null) Copy the string *unicode* into a UCS4 buffer, including a null character, if *copy_null* is set. Returns ``NULL`` and sets an exception on error (in particular, a :exc:`SystemError` if *buflen* is smaller than the length of *unicode*). *buffer* is returned on success. .. versionadded:: 3.3 .. c:function:: Py_UCS4* PyUnicode_AsUCS4Copy(PyObject *unicode) Copy the string *unicode* into a new UCS4 buffer that is allocated using :c:func:`PyMem_Malloc`. If this fails, ``NULL`` is returned with a :exc:`MemoryError` set. The returned buffer always has an extra null code point appended. .. versionadded:: 3.3 Locale Encoding """"""""""""""" The current locale encoding can be used to decode text from the operating system. .. c:function:: PyObject* PyUnicode_DecodeLocaleAndSize(const char *str, \ Py_ssize_t length, \ const char *errors) Decode a string from UTF-8 on Android and VxWorks, or from the current locale encoding on other platforms. The supported error handlers are ``"strict"`` and ``"surrogateescape"`` (:pep:`383`). The decoder uses ``"strict"`` error handler if *errors* is ``NULL``. *str* must end with a null character but cannot contain embedded null characters. Use :c:func:`PyUnicode_DecodeFSDefaultAndSize` to decode a string from the :term:`filesystem encoding and error handler`. This function ignores the :ref:`Python UTF-8 Mode `. .. seealso:: The :c:func:`Py_DecodeLocale` function. .. versionadded:: 3.3 .. versionchanged:: 3.7 The function now also uses the current locale encoding for the ``surrogateescape`` error handler, except on Android. Previously, :c:func:`Py_DecodeLocale` was used for the ``surrogateescape``, and the current locale encoding was used for ``strict``. .. c:function:: PyObject* PyUnicode_DecodeLocale(const char *str, const char *errors) Similar to :c:func:`PyUnicode_DecodeLocaleAndSize`, but compute the string length using :c:func:`!strlen`. .. versionadded:: 3.3 .. c:function:: PyObject* PyUnicode_EncodeLocale(PyObject *unicode, const char *errors) Encode a Unicode object to UTF-8 on Android and VxWorks, or to the current locale encoding on other platforms. The supported error handlers are ``"strict"`` and ``"surrogateescape"`` (:pep:`383`). The encoder uses ``"strict"`` error handler if *errors* is ``NULL``. Return a :class:`bytes` object. *unicode* cannot contain embedded null characters. Use :c:func:`PyUnicode_EncodeFSDefault` to encode a string to the :term:`filesystem encoding and error handler`. This function ignores the :ref:`Python UTF-8 Mode `. .. seealso:: The :c:func:`Py_EncodeLocale` function. .. versionadded:: 3.3 .. versionchanged:: 3.7 The function now also uses the current locale encoding for the ``surrogateescape`` error handler, except on Android. Previously, :c:func:`Py_EncodeLocale` was used for the ``surrogateescape``, and the current locale encoding was used for ``strict``. .. _capi-filesystem-encoding: File System Encoding """""""""""""""""""" Functions encoding to and decoding from the :term:`filesystem encoding and error handler` (:pep:`383` and :pep:`529`). To encode file names to :class:`bytes` during argument parsing, the ``"O&"`` converter should be used, passing :c:func:`!PyUnicode_FSConverter` as the conversion function: .. c:function:: int PyUnicode_FSConverter(PyObject* obj, void* result) :ref:`PyArg_Parse\* converter `: encode :class:`str` objects -- obtained directly or through the :class:`os.PathLike` interface -- to :class:`bytes` using :c:func:`PyUnicode_EncodeFSDefault`; :class:`bytes` objects are output as-is. *result* must be an address of a C variable of type :c:expr:`PyObject*` (or :c:expr:`PyBytesObject*`). On success, set the variable to a new :term:`strong reference` to a :ref:`bytes object ` which must be released when it is no longer used and return a non-zero value (:c:macro:`Py_CLEANUP_SUPPORTED`). Embedded null bytes are not allowed in the result. On failure, return ``0`` with an exception set. If *obj* is ``NULL``, the function releases a strong reference stored in the variable referred by *result* and returns ``1``. .. versionadded:: 3.1 .. versionchanged:: 3.6 Accepts a :term:`path-like object`. To decode file names to :class:`str` during argument parsing, the ``"O&"`` converter should be used, passing :c:func:`!PyUnicode_FSDecoder` as the conversion function: .. c:function:: int PyUnicode_FSDecoder(PyObject* obj, void* result) :ref:`PyArg_Parse\* converter `: decode :class:`bytes` objects -- obtained either directly or indirectly through the :class:`os.PathLike` interface -- to :class:`str` using :c:func:`PyUnicode_DecodeFSDefaultAndSize`; :class:`str` objects are output as-is. *result* must be an address of a C variable of type :c:expr:`PyObject*` (or :c:expr:`PyUnicodeObject*`). On success, set the variable to a new :term:`strong reference` to a :ref:`Unicode object ` which must be released when it is no longer used and return a non-zero value (:c:macro:`Py_CLEANUP_SUPPORTED`). Embedded null characters are not allowed in the result. On failure, return ``0`` with an exception set. If *obj* is ``NULL``, release the strong reference to the object referred to by *result* and return ``1``. .. versionadded:: 3.2 .. versionchanged:: 3.6 Accepts a :term:`path-like object`. .. c:function:: PyObject* PyUnicode_DecodeFSDefaultAndSize(const char *str, Py_ssize_t size) Decode a string from the :term:`filesystem encoding and error handler`. If you need to decode a string from the current locale encoding, use :c:func:`PyUnicode_DecodeLocaleAndSize`. .. seealso:: The :c:func:`Py_DecodeLocale` function. .. versionchanged:: 3.6 The :term:`filesystem error handler ` is now used. .. c:function:: PyObject* PyUnicode_DecodeFSDefault(const char *str) Decode a null-terminated string from the :term:`filesystem encoding and error handler`. If the string length is known, use :c:func:`PyUnicode_DecodeFSDefaultAndSize`. .. versionchanged:: 3.6 The :term:`filesystem error handler ` is now used. .. c:function:: PyObject* PyUnicode_EncodeFSDefault(PyObject *unicode) Encode a Unicode object to the :term:`filesystem encoding and error handler`, and return :class:`bytes`. Note that the resulting :class:`bytes` object can contain null bytes. If you need to encode a string to the current locale encoding, use :c:func:`PyUnicode_EncodeLocale`. .. seealso:: The :c:func:`Py_EncodeLocale` function. .. versionadded:: 3.2 .. versionchanged:: 3.6 The :term:`filesystem error handler ` is now used. wchar_t Support """"""""""""""" :c:type:`wchar_t` support for platforms which support it: .. c:function:: PyObject* PyUnicode_FromWideChar(const wchar_t *wstr, Py_ssize_t size) Create a Unicode object from the :c:type:`wchar_t` buffer *wstr* of the given *size*. Passing ``-1`` as the *size* indicates that the function must itself compute the length, using :c:func:`!wcslen`. Return ``NULL`` on failure. .. c:function:: Py_ssize_t PyUnicode_AsWideChar(PyObject *unicode, wchar_t *wstr, Py_ssize_t size) Copy the Unicode object contents into the :c:type:`wchar_t` buffer *wstr*. At most *size* :c:type:`wchar_t` characters are copied (excluding a possibly trailing null termination character). Return the number of :c:type:`wchar_t` characters copied or ``-1`` in case of an error. When *wstr* is ``NULL``, instead return the *size* that would be required to store all of *unicode* including a terminating null. Note that the resulting :c:expr:`wchar_t*` string may or may not be null-terminated. It is the responsibility of the caller to make sure that the :c:expr:`wchar_t*` string is null-terminated in case this is required by the application. Also, note that the :c:expr:`wchar_t*` string might contain null characters, which would cause the string to be truncated when used with most C functions. .. c:function:: wchar_t* PyUnicode_AsWideCharString(PyObject *unicode, Py_ssize_t *size) Convert the Unicode object to a wide character string. The output string always ends with a null character. If *size* is not ``NULL``, write the number of wide characters (excluding the trailing null termination character) into *\*size*. Note that the resulting :c:type:`wchar_t` string might contain null characters, which would cause the string to be truncated when used with most C functions. If *size* is ``NULL`` and the :c:expr:`wchar_t*` string contains null characters a :exc:`ValueError` is raised. Returns a buffer allocated by :c:macro:`PyMem_New` (use :c:func:`PyMem_Free` to free it) on success. On error, returns ``NULL`` and *\*size* is undefined. Raises a :exc:`MemoryError` if memory allocation is failed. .. versionadded:: 3.2 .. versionchanged:: 3.7 Raises a :exc:`ValueError` if *size* is ``NULL`` and the :c:expr:`wchar_t*` string contains null characters. .. _builtincodecs: Built-in Codecs ^^^^^^^^^^^^^^^ Python provides a set of built-in codecs which are written in C for speed. All of these codecs are directly usable via the following functions. Many of the following APIs take two arguments encoding and errors, and they have the same semantics as the ones of the built-in :func:`str` string object constructor. Setting encoding to ``NULL`` causes the default encoding to be used which is UTF-8. The file system calls should use :c:func:`PyUnicode_FSConverter` for encoding file names. This uses the :term:`filesystem encoding and error handler` internally. Error handling is set by errors which may also be set to ``NULL`` meaning to use the default handling defined for the codec. Default error handling for all built-in codecs is "strict" (:exc:`ValueError` is raised). The codecs all use a similar interface. Only deviations from the following generic ones are documented for simplicity. Generic Codecs """""""""""""" The following macro is provided: .. c:macro:: Py_UNICODE_REPLACEMENT_CHARACTER The Unicode code point ``U+FFFD`` (replacement character). This Unicode character is used as the replacement character during decoding if the *errors* argument is set to "replace". These are the generic codec APIs: .. c:function:: PyObject* PyUnicode_Decode(const char *str, Py_ssize_t size, \ const char *encoding, const char *errors) Create a Unicode object by decoding *size* bytes of the encoded string *str*. *encoding* and *errors* have the same meaning as the parameters of the same name in the :func:`str` built-in function. The codec to be used is looked up using the Python codec registry. Return ``NULL`` if an exception was raised by the codec. .. c:function:: PyObject* PyUnicode_AsEncodedString(PyObject *unicode, \ const char *encoding, const char *errors) Encode a Unicode object and return the result as Python bytes object. *encoding* and *errors* have the same meaning as the parameters of the same name in the Unicode :meth:`~str.encode` method. The codec to be used is looked up using the Python codec registry. Return ``NULL`` if an exception was raised by the codec. UTF-8 Codecs """""""""""" These are the UTF-8 codec APIs: .. c:function:: PyObject* PyUnicode_DecodeUTF8(const char *str, Py_ssize_t size, const char *errors) Create a Unicode object by decoding *size* bytes of the UTF-8 encoded string *str*. Return ``NULL`` if an exception was raised by the codec. .. c:function:: PyObject* PyUnicode_DecodeUTF8Stateful(const char *str, Py_ssize_t size, \ const char *errors, Py_ssize_t *consumed) If *consumed* is ``NULL``, behave like :c:func:`PyUnicode_DecodeUTF8`. If *consumed* is not ``NULL``, trailing incomplete UTF-8 byte sequences will not be treated as an error. Those bytes will not be decoded and the number of bytes that have been decoded will be stored in *consumed*. .. c:function:: PyObject* PyUnicode_AsUTF8String(PyObject *unicode) Encode a Unicode object using UTF-8 and return the result as Python bytes object. Error handling is "strict". Return ``NULL`` if an exception was raised by the codec. The function fails if the string contains surrogate code points (``U+D800`` - ``U+DFFF``). .. c:function:: const char* PyUnicode_AsUTF8AndSize(PyObject *unicode, Py_ssize_t *size) Return a pointer to the UTF-8 encoding of the Unicode object, and store the size of the encoded representation (in bytes) in *size*. The *size* argument can be ``NULL``; in this case no size will be stored. The returned buffer always has an extra null byte appended (not included in *size*), regardless of whether there are any other null code points. On error, set an exception, set *size* to ``-1`` (if it's not NULL) and return ``NULL``. The function fails if the string contains surrogate code points (``U+D800`` - ``U+DFFF``). This caches the UTF-8 representation of the string in the Unicode object, and subsequent calls will return a pointer to the same buffer. The caller is not responsible for deallocating the buffer. The buffer is deallocated and pointers to it become invalid when the Unicode object is garbage collected. .. versionadded:: 3.3 .. versionchanged:: 3.7 The return type is now ``const char *`` rather than ``char *``. .. versionchanged:: 3.10 This function is a part of the :ref:`limited API `. .. c:function:: const char* PyUnicode_AsUTF8(PyObject *unicode) As :c:func:`PyUnicode_AsUTF8AndSize`, but does not store the size. .. warning:: This function does not have any special behavior for `null characters `_ embedded within *unicode*. As a result, strings containing null characters will remain in the returned string, which some C functions might interpret as the end of the string, leading to truncation. If truncation is an issue, it is recommended to use :c:func:`PyUnicode_AsUTF8AndSize` instead. .. versionadded:: 3.3 .. versionchanged:: 3.7 The return type is now ``const char *`` rather than ``char *``. UTF-32 Codecs """"""""""""" These are the UTF-32 codec APIs: .. c:function:: PyObject* PyUnicode_DecodeUTF32(const char *str, Py_ssize_t size, \ const char *errors, int *byteorder) Decode *size* bytes from a UTF-32 encoded buffer string and return the corresponding Unicode object. *errors* (if non-``NULL``) defines the error handling. It defaults to "strict". If *byteorder* is non-``NULL``, the decoder starts decoding using the given byte order:: *byteorder == -1: little endian *byteorder == 0: native order *byteorder == 1: big endian If ``*byteorder`` is zero, and the first four bytes of the input data are a byte order mark (BOM), the decoder switches to this byte order and the BOM is not copied into the resulting Unicode string. If ``*byteorder`` is ``-1`` or ``1``, any byte order mark is copied to the output. After completion, *\*byteorder* is set to the current byte order at the end of input data. If *byteorder* is ``NULL``, the codec starts in native order mode. Return ``NULL`` if an exception was raised by the codec. .. c:function:: PyObject* PyUnicode_DecodeUTF32Stateful(const char *str, Py_ssize_t size, \ const char *errors, int *byteorder, Py_ssize_t *consumed) If *consumed* is ``NULL``, behave like :c:func:`PyUnicode_DecodeUTF32`. If *consumed* is not ``NULL``, :c:func:`PyUnicode_DecodeUTF32Stateful` will not treat trailing incomplete UTF-32 byte sequences (such as a number of bytes not divisible by four) as an error. Those bytes will not be decoded and the number of bytes that have been decoded will be stored in *consumed*. .. c:function:: PyObject* PyUnicode_AsUTF32String(PyObject *unicode) Return a Python byte string using the UTF-32 encoding in native byte order. The string always starts with a BOM mark. Error handling is "strict". Return ``NULL`` if an exception was raised by the codec. UTF-16 Codecs """"""""""""" These are the UTF-16 codec APIs: .. c:function:: PyObject* PyUnicode_DecodeUTF16(const char *str, Py_ssize_t size, \ const char *errors, int *byteorder) Decode *size* bytes from a UTF-16 encoded buffer string and return the corresponding Unicode object. *errors* (if non-``NULL``) defines the error handling. It defaults to "strict". If *byteorder* is non-``NULL``, the decoder starts decoding using the given byte order:: *byteorder == -1: little endian *byteorder == 0: native order *byteorder == 1: big endian If ``*byteorder`` is zero, and the first two bytes of the input data are a byte order mark (BOM), the decoder switches to this byte order and the BOM is not copied into the resulting Unicode string. If ``*byteorder`` is ``-1`` or ``1``, any byte order mark is copied to the output (where it will result in either a ``\ufeff`` or a ``\ufffe`` character). After completion, ``*byteorder`` is set to the current byte order at the end of input data. If *byteorder* is ``NULL``, the codec starts in native order mode. Return ``NULL`` if an exception was raised by the codec. .. c:function:: PyObject* PyUnicode_DecodeUTF16Stateful(const char *str, Py_ssize_t size, \ const char *errors, int *byteorder, Py_ssize_t *consumed) If *consumed* is ``NULL``, behave like :c:func:`PyUnicode_DecodeUTF16`. If *consumed* is not ``NULL``, :c:func:`PyUnicode_DecodeUTF16Stateful` will not treat trailing incomplete UTF-16 byte sequences (such as an odd number of bytes or a split surrogate pair) as an error. Those bytes will not be decoded and the number of bytes that have been decoded will be stored in *consumed*. .. c:function:: PyObject* PyUnicode_AsUTF16String(PyObject *unicode) Return a Python byte string using the UTF-16 encoding in native byte order. The string always starts with a BOM mark. Error handling is "strict". Return ``NULL`` if an exception was raised by the codec. UTF-7 Codecs """""""""""" These are the UTF-7 codec APIs: .. c:function:: PyObject* PyUnicode_DecodeUTF7(const char *str, Py_ssize_t size, const char *errors) Create a Unicode object by decoding *size* bytes of the UTF-7 encoded string *str*. Return ``NULL`` if an exception was raised by the codec. .. c:function:: PyObject* PyUnicode_DecodeUTF7Stateful(const char *str, Py_ssize_t size, \ const char *errors, Py_ssize_t *consumed) If *consumed* is ``NULL``, behave like :c:func:`PyUnicode_DecodeUTF7`. If *consumed* is not ``NULL``, trailing incomplete UTF-7 base-64 sections will not be treated as an error. Those bytes will not be decoded and the number of bytes that have been decoded will be stored in *consumed*. Unicode-Escape Codecs """"""""""""""""""""" These are the "Unicode Escape" codec APIs: .. c:function:: PyObject* PyUnicode_DecodeUnicodeEscape(const char *str, \ Py_ssize_t size, const char *errors) Create a Unicode object by decoding *size* bytes of the Unicode-Escape encoded string *str*. Return ``NULL`` if an exception was raised by the codec. .. c:function:: PyObject* PyUnicode_AsUnicodeEscapeString(PyObject *unicode) Encode a Unicode object using Unicode-Escape and return the result as a bytes object. Error handling is "strict". Return ``NULL`` if an exception was raised by the codec. Raw-Unicode-Escape Codecs """"""""""""""""""""""""" These are the "Raw Unicode Escape" codec APIs: .. c:function:: PyObject* PyUnicode_DecodeRawUnicodeEscape(const char *str, \ Py_ssize_t size, const char *errors) Create a Unicode object by decoding *size* bytes of the Raw-Unicode-Escape encoded string *str*. Return ``NULL`` if an exception was raised by the codec. .. c:function:: PyObject* PyUnicode_AsRawUnicodeEscapeString(PyObject *unicode) Encode a Unicode object using Raw-Unicode-Escape and return the result as a bytes object. Error handling is "strict". Return ``NULL`` if an exception was raised by the codec. Latin-1 Codecs """""""""""""" These are the Latin-1 codec APIs: Latin-1 corresponds to the first 256 Unicode ordinals and only these are accepted by the codecs during encoding. .. c:function:: PyObject* PyUnicode_DecodeLatin1(const char *str, Py_ssize_t size, const char *errors) Create a Unicode object by decoding *size* bytes of the Latin-1 encoded string *str*. Return ``NULL`` if an exception was raised by the codec. .. c:function:: PyObject* PyUnicode_AsLatin1String(PyObject *unicode) Encode a Unicode object using Latin-1 and return the result as Python bytes object. Error handling is "strict". Return ``NULL`` if an exception was raised by the codec. ASCII Codecs """""""""""" These are the ASCII codec APIs. Only 7-bit ASCII data is accepted. All other codes generate errors. .. c:function:: PyObject* PyUnicode_DecodeASCII(const char *str, Py_ssize_t size, const char *errors) Create a Unicode object by decoding *size* bytes of the ASCII encoded string *str*. Return ``NULL`` if an exception was raised by the codec. .. c:function:: PyObject* PyUnicode_AsASCIIString(PyObject *unicode) Encode a Unicode object using ASCII and return the result as Python bytes object. Error handling is "strict". Return ``NULL`` if an exception was raised by the codec. Character Map Codecs """""""""""""""""""" This codec is special in that it can be used to implement many different codecs (and this is in fact what was done to obtain most of the standard codecs included in the :mod:`!encodings` package). The codec uses mappings to encode and decode characters. The mapping objects provided must support the :meth:`~object.__getitem__` mapping interface; dictionaries and sequences work well. These are the mapping codec APIs: .. c:function:: PyObject* PyUnicode_DecodeCharmap(const char *str, Py_ssize_t length, \ PyObject *mapping, const char *errors) Create a Unicode object by decoding *size* bytes of the encoded string *str* using the given *mapping* object. Return ``NULL`` if an exception was raised by the codec. If *mapping* is ``NULL``, Latin-1 decoding will be applied. Else *mapping* must map bytes ordinals (integers in the range from 0 to 255) to Unicode strings, integers (which are then interpreted as Unicode ordinals) or ``None``. Unmapped data bytes -- ones which cause a :exc:`LookupError`, as well as ones which get mapped to ``None``, ``0xFFFE`` or ``'\ufffe'``, are treated as undefined mappings and cause an error. .. c:function:: PyObject* PyUnicode_AsCharmapString(PyObject *unicode, PyObject *mapping) Encode a Unicode object using the given *mapping* object and return the result as a bytes object. Error handling is "strict". Return ``NULL`` if an exception was raised by the codec. The *mapping* object must map Unicode ordinal integers to bytes objects, integers in the range from 0 to 255 or ``None``. Unmapped character ordinals (ones which cause a :exc:`LookupError`) as well as mapped to ``None`` are treated as "undefined mapping" and cause an error. The following codec API is special in that maps Unicode to Unicode. .. c:function:: PyObject* PyUnicode_Translate(PyObject *unicode, PyObject *table, const char *errors) Translate a string by applying a character mapping table to it and return the resulting Unicode object. Return ``NULL`` if an exception was raised by the codec. The mapping table must map Unicode ordinal integers to Unicode ordinal integers or ``None`` (causing deletion of the character). Mapping tables need only provide the :meth:`~object.__getitem__` interface; dictionaries and sequences work well. Unmapped character ordinals (ones which cause a :exc:`LookupError`) are left untouched and are copied as-is. *errors* has the usual meaning for codecs. It may be ``NULL`` which indicates to use the default error handling. MBCS codecs for Windows """"""""""""""""""""""" These are the MBCS codec APIs. They are currently only available on Windows and use the Win32 MBCS converters to implement the conversions. Note that MBCS (or DBCS) is a class of encodings, not just one. The target encoding is defined by the user settings on the machine running the codec. .. c:function:: PyObject* PyUnicode_DecodeMBCS(const char *str, Py_ssize_t size, const char *errors) Create a Unicode object by decoding *size* bytes of the MBCS encoded string *str*. Return ``NULL`` if an exception was raised by the codec. .. c:function:: PyObject* PyUnicode_DecodeMBCSStateful(const char *str, Py_ssize_t size, \ const char *errors, Py_ssize_t *consumed) If *consumed* is ``NULL``, behave like :c:func:`PyUnicode_DecodeMBCS`. If *consumed* is not ``NULL``, :c:func:`PyUnicode_DecodeMBCSStateful` will not decode trailing lead byte and the number of bytes that have been decoded will be stored in *consumed*. .. c:function:: PyObject* PyUnicode_DecodeCodePageStateful(int code_page, const char *str, \ Py_ssize_t size, const char *errors, Py_ssize_t *consumed) Similar to :c:func:`PyUnicode_DecodeMBCSStateful`, except uses the code page specified by *code_page*. .. c:function:: PyObject* PyUnicode_AsMBCSString(PyObject *unicode) Encode a Unicode object using MBCS and return the result as Python bytes object. Error handling is "strict". Return ``NULL`` if an exception was raised by the codec. .. c:function:: PyObject* PyUnicode_EncodeCodePage(int code_page, PyObject *unicode, const char *errors) Encode the Unicode object using the specified code page and return a Python bytes object. Return ``NULL`` if an exception was raised by the codec. Use :c:macro:`!CP_ACP` code page to get the MBCS encoder. .. versionadded:: 3.3 .. _unicodemethodsandslots: Methods and Slot Functions ^^^^^^^^^^^^^^^^^^^^^^^^^^ The following APIs are capable of handling Unicode objects and strings on input (we refer to them as strings in the descriptions) and return Unicode objects or integers as appropriate. They all return ``NULL`` or ``-1`` if an exception occurs. .. c:function:: PyObject* PyUnicode_Concat(PyObject *left, PyObject *right) Concat two strings giving a new Unicode string. .. c:function:: PyObject* PyUnicode_Split(PyObject *unicode, PyObject *sep, Py_ssize_t maxsplit) Split a string giving a list of Unicode strings. If *sep* is ``NULL``, splitting will be done at all whitespace substrings. Otherwise, splits occur at the given separator. At most *maxsplit* splits will be done. If negative, no limit is set. Separators are not included in the resulting list. On error, return ``NULL`` with an exception set. Equivalent to :py:meth:`str.split`. .. c:function:: PyObject* PyUnicode_RSplit(PyObject *unicode, PyObject *sep, Py_ssize_t maxsplit) Similar to :c:func:`PyUnicode_Split`, but splitting will be done beginning at the end of the string. On error, return ``NULL`` with an exception set. Equivalent to :py:meth:`str.rsplit`. .. c:function:: PyObject* PyUnicode_Splitlines(PyObject *unicode, int keepends) Split a Unicode string at line breaks, returning a list of Unicode strings. CRLF is considered to be one line break. If *keepends* is ``0``, the Line break characters are not included in the resulting strings. .. c:function:: PyObject* PyUnicode_Partition(PyObject *unicode, PyObject *sep) Split a Unicode string at the first occurrence of *sep*, and return a 3-tuple containing the part before the separator, the separator itself, and the part after the separator. If the separator is not found, return a 3-tuple containing the string itself, followed by two empty strings. *sep* must not be empty. On error, return ``NULL`` with an exception set. Equivalent to :py:meth:`str.partition`. .. c:function:: PyObject* PyUnicode_RPartition(PyObject *unicode, PyObject *sep) Similar to :c:func:`PyUnicode_Partition`, but split a Unicode string at the last occurrence of *sep*. If the separator is not found, return a 3-tuple containing two empty strings, followed by the string itself. *sep* must not be empty. On error, return ``NULL`` with an exception set. Equivalent to :py:meth:`str.rpartition`. .. c:function:: PyObject* PyUnicode_Join(PyObject *separator, PyObject *seq) Join a sequence of strings using the given *separator* and return the resulting Unicode string. .. c:function:: Py_ssize_t PyUnicode_Tailmatch(PyObject *unicode, PyObject *substr, \ Py_ssize_t start, Py_ssize_t end, int direction) Return ``1`` if *substr* matches ``unicode[start:end]`` at the given tail end (*direction* == ``-1`` means to do a prefix match, *direction* == ``1`` a suffix match), ``0`` otherwise. Return ``-1`` if an error occurred. .. c:function:: Py_ssize_t PyUnicode_Find(PyObject *unicode, PyObject *substr, \ Py_ssize_t start, Py_ssize_t end, int direction) Return the first position of *substr* in ``unicode[start:end]`` using the given *direction* (*direction* == ``1`` means to do a forward search, *direction* == ``-1`` a backward search). The return value is the index of the first match; a value of ``-1`` indicates that no match was found, and ``-2`` indicates that an error occurred and an exception has been set. .. c:function:: Py_ssize_t PyUnicode_FindChar(PyObject *unicode, Py_UCS4 ch, \ Py_ssize_t start, Py_ssize_t end, int direction) Return the first position of the character *ch* in ``unicode[start:end]`` using the given *direction* (*direction* == ``1`` means to do a forward search, *direction* == ``-1`` a backward search). The return value is the index of the first match; a value of ``-1`` indicates that no match was found, and ``-2`` indicates that an error occurred and an exception has been set. .. versionadded:: 3.3 .. versionchanged:: 3.7 *start* and *end* are now adjusted to behave like ``unicode[start:end]``. .. c:function:: Py_ssize_t PyUnicode_Count(PyObject *unicode, PyObject *substr, \ Py_ssize_t start, Py_ssize_t end) Return the number of non-overlapping occurrences of *substr* in ``unicode[start:end]``. Return ``-1`` if an error occurred. .. c:function:: PyObject* PyUnicode_Replace(PyObject *unicode, PyObject *substr, \ PyObject *replstr, Py_ssize_t maxcount) Replace at most *maxcount* occurrences of *substr* in *unicode* with *replstr* and return the resulting Unicode object. *maxcount* == ``-1`` means replace all occurrences. .. c:function:: int PyUnicode_Compare(PyObject *left, PyObject *right) Compare two strings and return ``-1``, ``0``, ``1`` for less than, equal, and greater than, respectively. This function returns ``-1`` upon failure, so one should call :c:func:`PyErr_Occurred` to check for errors. .. seealso:: The :c:func:`PyUnicode_Equal` function. .. c:function:: int PyUnicode_Equal(PyObject *a, PyObject *b) Test if two strings are equal: * Return ``1`` if *a* is equal to *b*. * Return ``0`` if *a* is not equal to *b*. * Set a :exc:`TypeError` exception and return ``-1`` if *a* or *b* is not a :class:`str` object. The function always succeeds if *a* and *b* are :class:`str` objects. The function works for :class:`str` subclasses, but does not honor custom ``__eq__()`` method. .. seealso:: The :c:func:`PyUnicode_Compare` function. .. versionadded:: 3.14 .. c:function:: int PyUnicode_EqualToUTF8AndSize(PyObject *unicode, const char *string, Py_ssize_t size) Compare a Unicode object with a char buffer which is interpreted as being UTF-8 or ASCII encoded and return true (``1``) if they are equal, or false (``0``) otherwise. If the Unicode object contains surrogate code points (``U+D800`` - ``U+DFFF``) or the C string is not valid UTF-8, false (``0``) is returned. This function does not raise exceptions. .. versionadded:: 3.13 .. c:function:: int PyUnicode_EqualToUTF8(PyObject *unicode, const char *string) Similar to :c:func:`PyUnicode_EqualToUTF8AndSize`, but compute *string* length using :c:func:`!strlen`. If the Unicode object contains null characters, false (``0``) is returned. .. versionadded:: 3.13 .. c:function:: int PyUnicode_CompareWithASCIIString(PyObject *unicode, const char *string) Compare a Unicode object, *unicode*, with *string* and return ``-1``, ``0``, ``1`` for less than, equal, and greater than, respectively. It is best to pass only ASCII-encoded strings, but the function interprets the input string as ISO-8859-1 if it contains non-ASCII characters. This function does not raise exceptions. .. c:function:: PyObject* PyUnicode_RichCompare(PyObject *left, PyObject *right, int op) Rich compare two Unicode strings and return one of the following: * ``NULL`` in case an exception was raised * :c:data:`Py_True` or :c:data:`Py_False` for successful comparisons * :c:data:`Py_NotImplemented` in case the type combination is unknown Possible values for *op* are :c:macro:`Py_GT`, :c:macro:`Py_GE`, :c:macro:`Py_EQ`, :c:macro:`Py_NE`, :c:macro:`Py_LT`, and :c:macro:`Py_LE`. .. c:function:: PyObject* PyUnicode_Format(PyObject *format, PyObject *args) Return a new string object from *format* and *args*; this is analogous to ``format % args``. .. c:function:: int PyUnicode_Contains(PyObject *unicode, PyObject *substr) Check whether *substr* is contained in *unicode* and return true or false accordingly. *substr* has to coerce to a one element Unicode string. ``-1`` is returned if there was an error. .. c:function:: void PyUnicode_InternInPlace(PyObject **p_unicode) Intern the argument :c:expr:`*p_unicode` in place. The argument must be the address of a pointer variable pointing to a Python Unicode string object. If there is an existing interned string that is the same as :c:expr:`*p_unicode`, it sets :c:expr:`*p_unicode` to it (releasing the reference to the old string object and creating a new :term:`strong reference` to the interned string object), otherwise it leaves :c:expr:`*p_unicode` alone and interns it. (Clarification: even though there is a lot of talk about references, think of this function as reference-neutral. You must own the object you pass in; after the call you no longer own the passed-in reference, but you newly own the result.) This function never raises an exception. On error, it leaves its argument unchanged without interning it. Instances of subclasses of :py:class:`str` may not be interned, that is, :c:expr:`PyUnicode_CheckExact(*p_unicode)` must be true. If it is not, then -- as with any other error -- the argument is left unchanged. Note that interned strings are not “immortal”. You must keep a reference to the result to benefit from interning. .. c:function:: PyObject* PyUnicode_InternFromString(const char *str) A combination of :c:func:`PyUnicode_FromString` and :c:func:`PyUnicode_InternInPlace`, meant for statically allocated strings. Return a new ("owned") reference to either a new Unicode string object that has been interned, or an earlier interned string object with the same value. Python may keep a reference to the result, or make it :term:`immortal`, preventing it from being garbage-collected promptly. For interning an unbounded number of different strings, such as ones coming from user input, prefer calling :c:func:`PyUnicode_FromString` and :c:func:`PyUnicode_InternInPlace` directly. .. c:function:: unsigned int PyUnicode_CHECK_INTERNED(PyObject *str) Return a non-zero value if *str* is interned, zero if not. The *str* argument must be a string; this is not checked. This function always succeeds. .. impl-detail:: A non-zero return value may carry additional information about *how* the string is interned. The meaning of such non-zero values, as well as each specific string's intern-related details, may change between CPython versions. PyUnicodeWriter ^^^^^^^^^^^^^^^ The :c:type:`PyUnicodeWriter` API can be used to create a Python :class:`str` object. .. versionadded:: 3.14 .. c:type:: PyUnicodeWriter A Unicode writer instance. The instance must be destroyed by :c:func:`PyUnicodeWriter_Finish` on success, or :c:func:`PyUnicodeWriter_Discard` on error. The API is **not thread safe**. To share a writer with multiple threads, a critical section or a lock is needed. .. c:function:: PyUnicodeWriter* PyUnicodeWriter_Create(Py_ssize_t length) Create a Unicode writer instance. *length* must be greater than or equal to ``0``. If *length* is greater than ``0``, preallocate an internal buffer of *length* characters. Set an exception and return ``NULL`` on error. .. c:function:: PyObject* PyUnicodeWriter_Finish(PyUnicodeWriter *writer) Return the final Python :class:`str` object and destroy the writer instance. Set an exception and return ``NULL`` on error. The writer instance is invalid after this call. .. c:function:: void PyUnicodeWriter_Discard(PyUnicodeWriter *writer) Discard the internal Unicode buffer and destroy the writer instance. If *writer* is ``NULL``, no operation is performed. The writer instance is invalid after this call. .. c:function:: int PyUnicodeWriter_WriteChar(PyUnicodeWriter *writer, Py_UCS4 ch) Write the single Unicode character *ch* into *writer*. On success, return ``0``. On error, set an exception, leave the writer unchanged, and return ``-1``. .. c:function:: int PyUnicodeWriter_WriteUTF8(PyUnicodeWriter *writer, const char *str, Py_ssize_t size) Decode the string *str* from UTF-8 in strict mode and write the output into *writer*. *size* is the string length in bytes. If *size* is equal to ``-1``, call ``strlen(str)`` to get the string length. On success, return ``0``. On error, set an exception, leave the writer unchanged, and return ``-1``. See also :c:func:`PyUnicodeWriter_DecodeUTF8Stateful`. .. c:function:: int PyUnicodeWriter_WriteASCII(PyUnicodeWriter *writer, const char *str, Py_ssize_t size) Write the ASCII string *str* into *writer*. *size* is the string length in bytes. If *size* is equal to ``-1``, call ``strlen(str)`` to get the string length. *str* must only contain ASCII characters. The behavior is undefined if *str* contains non-ASCII characters. On success, return ``0``. On error, set an exception, leave the writer unchanged, and return ``-1``. .. c:function:: int PyUnicodeWriter_WriteWideChar(PyUnicodeWriter *writer, const wchar_t *str, Py_ssize_t size) Write the wide string *str* into *writer*. *size* is a number of wide characters. If *size* is equal to ``-1``, call ``wcslen(str)`` to get the string length. On success, return ``0``. On error, set an exception, leave the writer unchanged, and return ``-1``. .. c:function:: int PyUnicodeWriter_WriteUCS4(PyUnicodeWriter *writer, const Py_UCS4 *str, Py_ssize_t size) Write the UCS4 string *str* into *writer*. *size* is a number of UCS4 characters. All characters must be in range [U+0000; U+10ffff]. If the string contains invalid characters, the behavior is undefined. On success, return ``0``. On error, set an exception, leave the writer unchanged, and return ``-1``. .. c:function:: int PyUnicodeWriter_WriteStr(PyUnicodeWriter *writer, PyObject *obj) Call :c:func:`PyObject_Str` on *obj* and write the output into *writer*. On success, return ``0``. On error, set an exception, leave the writer unchanged, and return ``-1``. To write a :class:`str` subclass which overrides the :meth:`~object.__str__` method, :c:func:`PyUnicode_FromObject` can be used to get the original string. .. c:function:: int PyUnicodeWriter_WriteRepr(PyUnicodeWriter *writer, PyObject *obj) Call :c:func:`PyObject_Repr` on *obj* and write the output into *writer*. If *obj* is ``NULL``, write the string ``""`` into *writer*. On success, return ``0``. On error, set an exception, leave the writer unchanged, and return ``-1``. .. versionchanged:: 3.14.4 Added support for ``NULL``. .. c:function:: int PyUnicodeWriter_WriteSubstring(PyUnicodeWriter *writer, PyObject *str, Py_ssize_t start, Py_ssize_t end) Write the substring ``str[start:end]`` into *writer*. *str* must be Python :class:`str` object. *start* must be greater than or equal to 0, and less than or equal to *end*. *end* must be less than or equal to *str* length. On success, return ``0``. On error, set an exception, leave the writer unchanged, and return ``-1``. .. c:function:: int PyUnicodeWriter_Format(PyUnicodeWriter *writer, const char *format, ...) Similar to :c:func:`PyUnicode_FromFormat`, but write the output directly into *writer*. On success, return ``0``. On error, set an exception, leave the writer unchanged, and return ``-1``. .. c:function:: int PyUnicodeWriter_DecodeUTF8Stateful(PyUnicodeWriter *writer, const char *str, Py_ssize_t size, const char *errors, Py_ssize_t *consumed) Decode the string *str* from UTF-8 with *errors* error handler and write the output into *writer*. *size* is the string length in bytes. If *size* is equal to ``-1``, call ``strlen(str)`` to get the string length. *errors* is an :ref:`error handler ` name, such as ``"replace"``. If *errors* is ``NULL``, use the strict error handler. If *consumed* is not ``NULL``, set *\*consumed* to the number of decoded bytes on success. If *consumed* is ``NULL``, treat trailing incomplete UTF-8 byte sequences as an error. On success, return ``0``. On error, set an exception, leave the writer unchanged, and return ``-1``. See also :c:func:`PyUnicodeWriter_WriteUTF8`. Deprecated API ^^^^^^^^^^^^^^ The following API is deprecated. .. c:type:: Py_UNICODE This is a typedef of :c:type:`wchar_t`, which is a 16-bit type or 32-bit type depending on the platform. Please use :c:type:`wchar_t` directly instead. .. versionchanged:: 3.3 In previous versions, this was a 16-bit type or a 32-bit type depending on whether you selected a "narrow" or "wide" Unicode version of Python at build time. .. deprecated-removed:: 3.13 3.16 .. c:function:: int PyUnicode_READY(PyObject *unicode) Do nothing and return ``0``. This API is kept only for backward compatibility, but there are no plans to remove it. .. versionadded:: 3.3 .. deprecated:: 3.10 This API does nothing since Python 3.12. Previously, this needed to be called for each string created using the old API (:c:func:`!PyUnicode_FromUnicode` or similar). .. c:function:: unsigned int PyUnicode_IS_READY(PyObject *unicode) Do nothing and return ``1``. This API is kept only for backward compatibility, but there are no plans to remove it. .. versionadded:: 3.3 .. deprecated:: 3.14 This API does nothing since Python 3.12. Previously, this could be called to check if :c:func:`PyUnicode_READY` is necessary. .. _pyunicode-new-mutating: Mutating string objects """"""""""""""""""""""" The following functions allow creating a blank string object of a given size, then filling in its contents. They are :term:`soft deprecated`: they break the assumption that strings are immutable, making them hard to use correctly. Prefer using the :c:type:`PyUnicodeWriter` API, or one of the ``PyUnicode_From*`` functions such as :c:func:`PyUnicode_FromStringAndSize`. If you do use the functions below, take care to not "use" such a string while it is being modified. In particular, before it's filled with its final contents, a string: - must not be hashed, - must not be :c:func:`converted to UTF-8 `, or another non-"canonical" representation, - must not have its reference count changed, - must not be accessed from another thread, - must not be shared with code that might do one of the above. This list is not exhaustive. Avoiding these uses is your responsibility; Python does not always check these requirements. .. c:function:: PyObject* PyUnicode_New(Py_ssize_t size, Py_UCS4 maxchar) Create a new Unicode object. *maxchar* should be the true maximum code point to be placed in the string. As an approximation, it can be rounded up to the nearest value in the sequence 127, 255, 65535, 1114111. On error, set an exception and return ``NULL``. See :ref:`pyunicode-new-mutating` for important warnings and caveats. .. versionadded:: 3.3 .. soft-deprecated:: 3.16 See :ref:`pyunicode-new-mutating`. .. c:function:: void PyUnicode_WRITE(int kind, void *data, \ Py_ssize_t index, Py_UCS4 value) Write the code point *value* to the given zero-based *index* in a string. The *kind* value and *data* pointer must have been obtained from a string using :c:func:`PyUnicode_KIND` and :c:func:`PyUnicode_DATA` respectively. You must hold a reference to that string while calling :c:func:`!PyUnicode_WRITE`. All requirements of :c:func:`PyUnicode_WriteChar` also apply. The function performs no checks for any of its requirements, and is intended for usage in loops. The owning string must not be "used" yet. See :ref:`pyunicode-new-mutating` for details. .. versionadded:: 3.3 .. soft-deprecated:: 3.16 Use the :c:type:`PyUnicodeWriter` API instead. .. c:function:: Py_ssize_t PyUnicode_CopyCharacters(PyObject *to, \ Py_ssize_t to_start, \ PyObject *from, \ Py_ssize_t from_start, \ Py_ssize_t how_many) Copy characters from one Unicode object into another. This function performs character conversion when necessary and falls back to :c:func:`!memcpy` if possible. Returns ``-1`` and sets an exception on error, otherwise returns the number of copied characters. The destination string must not be "used" yet. See :ref:`pyunicode-new-mutating` for details. .. versionadded:: 3.3 .. soft-deprecated:: 3.16 Use the :c:type:`PyUnicodeWriter` API instead. .. c:function:: int PyUnicode_Resize(PyObject **unicode, Py_ssize_t length); Resize a Unicode object *\*unicode* to the new *length* in code points. Try to resize the string in place (which is usually faster than allocating a new string and copying characters), or create a new string. *\*unicode* is modified to point to the new (resized) object and ``0`` is returned on success. Otherwise, ``-1`` is returned and an exception is set, and *\*unicode* is left untouched. The function doesn't check string content, the result may not be a string in canonical representation. *\*unicode* must not be "used" yet. See :ref:`pyunicode-new-mutating` for details. .. soft-deprecated:: 3.16 Use the :c:type:`PyUnicodeWriter` API instead. .. c:function:: Py_ssize_t PyUnicode_Fill(PyObject *unicode, Py_ssize_t start, \ Py_ssize_t length, Py_UCS4 fill_char) Fill a string with a character: write *fill_char* into ``unicode[start:start+length]``. Fail if *fill_char* is bigger than the string maximum character, or if the string has more than 1 reference. Return the number of written characters, or return ``-1`` and raise an exception on error. *unicode* must not be "used" yet. See :ref:`pyunicode-new-mutating` for details. .. versionadded:: 3.3 .. soft-deprecated:: 3.16 Use the :c:type:`PyUnicodeWriter` API instead. .. c:function:: int PyUnicode_WriteChar(PyObject *unicode, Py_ssize_t index, \ Py_UCS4 character) Write a *character* to the string *unicode* at the zero-based *index*. Return ``0`` on success, ``-1`` on error with an exception set. This function checks that *unicode* is a Unicode object, that the index is not out of bounds, and that the object's reference count is one. See :c:func:`PyUnicode_WRITE` for a version that skips these checks, making them your responsibility. *unicode* must not be "used" yet. See :ref:`pyunicode-new-mutating` for details. .. versionadded:: 3.3 .. soft-deprecated:: 3.16 Use the :c:type:`PyUnicodeWriter` API instead. --- ### Doc C Api Utilities (`Doc/c-api/utilities.rst`) .. highlight:: c .. _utilities: ********* Utilities ********* The functions in this chapter perform various utility tasks, ranging from helping C code be more portable across platforms, using Python modules from C, and parsing function arguments and constructing Python values from C values. .. toctree:: sys.rst import.rst marshal.rst arg.rst conversion.rst hash.rst reflection.rst codec.rst time.rst perfmaps.rst --- ### Doc C Api Veryhigh (`Doc/c-api/veryhigh.rst`) .. highlight:: c .. _veryhigh: ************************* The Very High Level Layer ************************* The functions in this chapter will let you execute Python source code given in a file or a buffer, but they will not let you interact in a more detailed way with the interpreter. Several of these functions accept a start symbol from the grammar as a parameter. The available start symbols are :c:data:`Py_eval_input`, :c:data:`Py_file_input`, :c:data:`Py_single_input`, and :c:data:`Py_func_type_input`. These are described following the functions which accept them as parameters. Note also that several of these functions take :c:expr:`FILE*` parameters. One particular issue which needs to be handled carefully is that the :c:type:`FILE` structure for different C libraries can be different and incompatible. Under Windows (at least), it is possible for dynamically linked extensions to actually use different libraries, so care should be taken that :c:expr:`FILE*` parameters are only passed to these functions if it is certain that they were created by the same library that the Python runtime is using. .. c:function:: int PyRun_AnyFile(FILE *fp, const char *filename) This is a simplified interface to :c:func:`PyRun_AnyFileExFlags` below, leaving *closeit* set to ``0`` and *flags* set to ``NULL``. .. c:function:: int PyRun_AnyFileFlags(FILE *fp, const char *filename, PyCompilerFlags *flags) This is a simplified interface to :c:func:`PyRun_AnyFileExFlags` below, leaving the *closeit* argument set to ``0``. .. c:function:: int PyRun_AnyFileEx(FILE *fp, const char *filename, int closeit) This is a simplified interface to :c:func:`PyRun_AnyFileExFlags` below, leaving the *flags* argument set to ``NULL``. .. c:function:: int PyRun_AnyFileExFlags(FILE *fp, const char *filename, int closeit, PyCompilerFlags *flags) If *fp* refers to a file associated with an interactive device (console or terminal input or Unix pseudo-terminal), return the value of :c:func:`PyRun_InteractiveLoop`, otherwise return the result of :c:func:`PyRun_SimpleFile`. *filename* is decoded from the filesystem encoding (:func:`sys.getfilesystemencoding`). If *filename* is ``NULL``, this function uses ``"???"`` as the filename. If *closeit* is true, the file is closed before ``PyRun_SimpleFileExFlags()`` returns. .. c:function:: int PyRun_SimpleString(const char *command) This is a simplified interface to :c:func:`PyRun_SimpleStringFlags` below, leaving the :c:struct:`PyCompilerFlags`\* argument set to ``NULL``. .. c:function:: int PyRun_SimpleStringFlags(const char *command, PyCompilerFlags *flags) Executes the Python source code from *command* in the :mod:`__main__` module according to the *flags* argument. If :mod:`__main__` does not already exist, it is created. Returns ``0`` on success or ``-1`` if an exception was raised. If there was an error, there is no way to get the exception information. For the meaning of *flags*, see below. Note that if an otherwise unhandled :exc:`SystemExit` is raised, this function will not return ``-1``, but exit the process, as long as :c:member:`PyConfig.inspect` is zero. .. c:function:: int PyRun_SimpleFile(FILE *fp, const char *filename) This is a simplified interface to :c:func:`PyRun_SimpleFileExFlags` below, leaving *closeit* set to ``0`` and *flags* set to ``NULL``. .. c:function:: int PyRun_SimpleFileEx(FILE *fp, const char *filename, int closeit) This is a simplified interface to :c:func:`PyRun_SimpleFileExFlags` below, leaving *flags* set to ``NULL``. .. c:function:: int PyRun_SimpleFileExFlags(FILE *fp, const char *filename, int closeit, PyCompilerFlags *flags) Similar to :c:func:`PyRun_SimpleStringFlags`, but the Python source code is read from *fp* instead of an in-memory string. *filename* should be the name of the file, it is decoded from :term:`filesystem encoding and error handler`. If *closeit* is true, the file is closed before ``PyRun_SimpleFileExFlags()`` returns. .. note:: On Windows, *fp* should be opened as binary mode (e.g. ``fopen(filename, "rb")``). Otherwise, Python may not handle script file with LF line ending correctly. .. c:function:: int PyRun_InteractiveOneObject(FILE *fp, PyObject *filename, PyCompilerFlags *flags) Read and execute a single statement from a file associated with an interactive device according to the *flags* argument. The user will be prompted using ``sys.ps1`` and ``sys.ps2``. *filename* must be a Python :class:`str` object. Returns ``0`` when the input was executed successfully, ``-1`` if there was an exception, or an error code from the :file:`errcode.h` include file distributed as part of Python if there was a parse error. (Note that :file:`errcode.h` is not included by :file:`Python.h`, so must be included specifically if needed.) .. c:function:: int PyRun_InteractiveOne(FILE *fp, const char *filename) This is a simplified interface to :c:func:`PyRun_InteractiveOneFlags` below, leaving *flags* set to ``NULL``. .. c:function:: int PyRun_InteractiveOneFlags(FILE *fp, const char *filename, PyCompilerFlags *flags) Similar to :c:func:`PyRun_InteractiveOneObject`, but *filename* is a :c:expr:`const char*`, which is decoded from the :term:`filesystem encoding and error handler`. .. c:function:: int PyRun_InteractiveLoop(FILE *fp, const char *filename) This is a simplified interface to :c:func:`PyRun_InteractiveLoopFlags` below, leaving *flags* set to ``NULL``. .. c:function:: int PyRun_InteractiveLoopFlags(FILE *fp, const char *filename, PyCompilerFlags *flags) Read and execute statements from a file associated with an interactive device until EOF is reached. The user will be prompted using ``sys.ps1`` and ``sys.ps2``. *filename* is decoded from the :term:`filesystem encoding and error handler`. Returns ``0`` at EOF or a negative number upon failure. .. c:var:: int (*PyOS_InputHook)(void) Can be set to point to a function with the prototype ``int func(void)``. The function will be called when Python's interpreter prompt is about to become idle and wait for user input from the terminal. The return value is ignored. Overriding this hook can be used to integrate the interpreter's prompt with other event loops, as done in :file:`Modules/_tkinter.c` in the Python source code. .. versionchanged:: 3.12 This function is only called from the :ref:`main interpreter `. .. c:var:: char* (*PyOS_ReadlineFunctionPointer)(FILE *, FILE *, const char *) Can be set to point to a function with the prototype ``char *func(FILE *stdin, FILE *stdout, char *prompt)``, overriding the default function used to read a single line of input at the interpreter's prompt. The function is expected to output the string *prompt* if it's not ``NULL``, and then read a line of input from the provided standard input file, returning the resulting string. For example, The :mod:`readline` module sets this hook to provide line-editing and tab-completion features. The result must be a string allocated by :c:func:`PyMem_RawMalloc` or :c:func:`PyMem_RawRealloc`, or ``NULL`` if an error occurred. .. versionchanged:: 3.4 The result must be allocated by :c:func:`PyMem_RawMalloc` or :c:func:`PyMem_RawRealloc`, instead of being allocated by :c:func:`PyMem_Malloc` or :c:func:`PyMem_Realloc`. .. versionchanged:: 3.12 This function is only called from the :ref:`main interpreter `. .. c:function:: PyObject* PyRun_String(const char *str, int start, PyObject *globals, PyObject *locals) This is a simplified interface to :c:func:`PyRun_StringFlags` below, leaving *flags* set to ``NULL``. .. c:function:: PyObject* PyRun_StringFlags(const char *str, int start, PyObject *globals, PyObject *locals, PyCompilerFlags *flags) Execute Python source code from *str* in the context specified by the objects *globals* and *locals* with the compiler flags specified by *flags*. *globals* must be a dictionary; *locals* can be any object that implements the mapping protocol. The parameter *start* specifies the start symbol and must be one of the :ref:`available start symbols `. Returns the result of executing the code as a Python object, or ``NULL`` if an exception was raised. .. c:function:: PyObject* PyRun_File(FILE *fp, const char *filename, int start, PyObject *globals, PyObject *locals) This is a simplified interface to :c:func:`PyRun_FileExFlags` below, leaving *closeit* set to ``0`` and *flags* set to ``NULL``. .. c:function:: PyObject* PyRun_FileEx(FILE *fp, const char *filename, int start, PyObject *globals, PyObject *locals, int closeit) This is a simplified interface to :c:func:`PyRun_FileExFlags` below, leaving *flags* set to ``NULL``. .. c:function:: PyObject* PyRun_FileFlags(FILE *fp, const char *filename, int start, PyObject *globals, PyObject *locals, PyCompilerFlags *flags) This is a simplified interface to :c:func:`PyRun_FileExFlags` below, leaving *closeit* set to ``0``. .. c:function:: PyObject* PyRun_FileExFlags(FILE *fp, const char *filename, int start, PyObject *globals, PyObject *locals, int closeit, PyCompilerFlags *flags) Similar to :c:func:`PyRun_StringFlags`, but the Python source code is read from *fp* instead of an in-memory string. *filename* should be the name of the file, it is decoded from the :term:`filesystem encoding and error handler`. If *closeit* is true, the file is closed before :c:func:`PyRun_FileExFlags` returns. .. c:function:: PyObject* Py_CompileString(const char *str, const char *filename, int start) This is a simplified interface to :c:func:`Py_CompileStringFlags` below, leaving *flags* set to ``NULL``. .. c:function:: PyObject* Py_CompileStringFlags(const char *str, const char *filename, int start, PyCompilerFlags *flags) This is a simplified interface to :c:func:`Py_CompileStringExFlags` below, with *optimize* set to ``-1``. .. c:function:: PyObject* Py_CompileStringObject(const char *str, PyObject *filename, int start, PyCompilerFlags *flags, int optimize) Parse and compile the Python source code in *str*, returning the resulting code object. The start symbol is given by *start*; this can be used to constrain the code which can be compiled and should be :ref:`available start symbols `. The filename specified by *filename* is used to construct the code object and may appear in tracebacks or :exc:`SyntaxError` exception messages. This returns ``NULL`` if the code cannot be parsed or compiled. The integer *optimize* specifies the optimization level of the compiler; a value of ``-1`` selects the optimization level of the interpreter as given by :option:`-O` options. Explicit levels are ``0`` (no optimization; ``__debug__`` is true), ``1`` (asserts are removed, ``__debug__`` is false) or ``2`` (docstrings are removed too). .. versionadded:: 3.4 .. c:function:: PyObject* Py_CompileStringExFlags(const char *str, const char *filename, int start, PyCompilerFlags *flags, int optimize) Like :c:func:`Py_CompileStringObject`, but *filename* is a byte string decoded from the :term:`filesystem encoding and error handler`. .. versionadded:: 3.2 .. c:function:: PyObject* PyEval_EvalCode(PyObject *co, PyObject *globals, PyObject *locals) This is a simplified interface to :c:func:`PyEval_EvalCodeEx`, with just the code object, and global and local variables. The other arguments are set to ``NULL``. .. c:function:: PyObject* PyEval_EvalCodeEx(PyObject *co, PyObject *globals, PyObject *locals, PyObject *const *args, int argcount, PyObject *const *kws, int kwcount, PyObject *const *defs, int defcount, PyObject *kwdefs, PyObject *closure) Evaluate a precompiled code object, given a particular environment for its evaluation. This environment consists of a dictionary of global variables, a mapping object of local variables, arrays of arguments, keywords and defaults, a dictionary of default values for :ref:`keyword-only ` arguments and a closure tuple of cells. .. c:function:: PyObject* PyEval_EvalFrame(PyFrameObject *f) Evaluate an execution frame. This is a simplified interface to :c:func:`PyEval_EvalFrameEx`, for backward compatibility. .. c:function:: PyObject* PyEval_EvalFrameEx(PyFrameObject *f, int throwflag) This is the main, unvarnished function of Python interpretation. The code object associated with the execution frame *f* is executed, interpreting bytecode and executing calls as needed. The additional *throwflag* parameter can mostly be ignored - if true, then it causes an exception to immediately be thrown; this is used for the :meth:`~generator.throw` methods of generator objects. .. versionchanged:: 3.4 This function now includes a debug assertion to help ensure that it does not silently discard an active exception. .. c:function:: int PyEval_MergeCompilerFlags(PyCompilerFlags *cf) This function changes the flags of the current evaluation frame, and returns true on success, false on failure. .. c:struct:: PyCompilerFlags This is the structure used to hold compiler flags. In cases where code is only being compiled, it is passed as ``int flags``, and in cases where code is being executed, it is passed as ``PyCompilerFlags *flags``. In this case, ``from __future__ import`` can modify *flags*. Whenever ``PyCompilerFlags *flags`` is ``NULL``, :c:member:`~PyCompilerFlags.cf_flags` is treated as equal to ``0``, and any modification due to ``from __future__ import`` is discarded. .. c:member:: int cf_flags Compiler flags. .. c:member:: int cf_feature_version *cf_feature_version* is the minor Python version. It should be initialized to ``PY_MINOR_VERSION``. The field is ignored by default, it is used if and only if ``PyCF_ONLY_AST`` flag is set in :c:member:`~PyCompilerFlags.cf_flags`. .. versionchanged:: 3.8 Added *cf_feature_version* field. The available compiler flags are accessible as macros: .. c:namespace:: NULL .. c:macro:: PyCF_ALLOW_TOP_LEVEL_AWAIT PyCF_ONLY_AST PyCF_OPTIMIZED_AST PyCF_TYPE_COMMENTS See :ref:`compiler flags ` in documentation of the :py:mod:`!ast` Python module, which exports these constants under the same names. .. rubric:: Low-level flags The following flags and masks serve narrow needs of the standard library and interactive interpreters. Code outside the standard library rarely has a reason to use them. They are considered implementation details and may change at any time. .. c:macro:: PyCF_ALLOW_INCOMPLETE_INPUT This flag is a private interface between the compiler and the :mod:`codeop` module. Do not use it; its behavior is unsupported and may change without warning. With this flag set, when compilation fails because the source text ends where more input is expected, for example in the middle of an indented block or an unterminated string literal, the error raised is the undocumented ``_IncompleteInputError``, a subclass of :exc:`SyntaxError`. The :mod:`codeop` module sets this flag, together with :c:macro:`PyCF_DONT_IMPLY_DEDENT`, to tell input that is incomplete apart from input with a real syntax error, so that interactive interpreters know when to prompt for another line instead of reporting an error. .. versionadded:: 3.11 .. c:macro:: PyCF_DONT_IMPLY_DEDENT By default, when compiling with the :c:var:`Py_single_input` start symbol, reaching the end of the source text implicitly closes any open indented blocks. With this flag set, open blocks are only closed if the last line of the source ends with a newline; otherwise, compilation fails with a :exc:`SyntaxError`: .. code-block:: c PyCompilerFlags flags = { .cf_flags = 0, .cf_feature_version = PY_MINOR_VERSION, }; const char *source = "if a:\n pass"; /* The "if" block is closed implicitly; this returns a code object: */ Py_CompileStringFlags(source, "", Py_single_input, &flags); /* With the flag, this fails with a SyntaxError, because the last line does not end with a newline: */ flags.cf_flags = PyCF_DONT_IMPLY_DEDENT; Py_CompileStringFlags(source, "", Py_single_input, &flags); The :mod:`codeop` module uses this flag to detect incomplete interactive input. While the user is still typing inside an indented block, the source does not yet end with a newline, so it fails to compile and the user is prompted for another line. .. c:macro:: PyCF_IGNORE_COOKIE Read the source text as UTF-8, ignoring its :pep:`263` encoding declaration ("coding cookie"), if any: .. code-block:: c PyCompilerFlags flags = { .cf_flags = 0, .cf_feature_version = PY_MINOR_VERSION, }; const char *source = "# coding: latin-1\ns = '\xe9'\n"; /* The coding cookie is honored: byte 0xE9 is decoded as Latin-1, and this returns a code object that sets s to "é": */ Py_CompileStringFlags(source, "", Py_file_input, &flags); /* With the flag, the cookie is ignored and compilation fails with a SyntaxError, because 0xE9 is not valid UTF-8: */ flags.cf_flags = PyCF_IGNORE_COOKIE; Py_CompileStringFlags(source, "", Py_file_input, &flags); The :func:`compile`, :func:`eval` and :func:`exec` built-in functions set this flag when the source is a :class:`str` object, because they pass the text to the parser encoded as UTF-8. .. c:macro:: PyCF_SOURCE_IS_UTF8 Mark the source text as known to be UTF-8 encoded. The :func:`compile`, :func:`eval` and :func:`exec` built-in functions set this flag, but it currently has no effect. The "``PyCF``" flags above can be combined with "``CO_FUTURE``" flags such as :c:macro:`CO_FUTURE_ANNOTATIONS` to enable features normally selectable using :ref:`future statements `. See :ref:`c_codeobject_flags` for a complete list. The following masks combine several flags: .. c:macro:: PyCF_MASK Bitmask of all ``CO_FUTURE`` flags (see :ref:`c_codeobject_flags`), which select features normally enabled by :ref:`future statements `. When code compiled with a ``PyCompilerFlags *flags`` argument contains a ``from __future__ import`` statement, the flag for the imported feature is added to *flags*, so that code executed later in the same context inherits it. .. c:macro:: PyCF_MASK_OBSOLETE Do not use this mask in new code. It is kept only so that old code passing its flags to :func:`compile` keeps working. Bitmask of flags for obsolete future features that no longer have any effect. .. c:macro:: PyCF_COMPILE_MASK Bitmask of all ``PyCF`` flags that change how the source is compiled, such as :c:macro:`PyCF_ONLY_AST`. The :func:`compile` built-in function uses this mask to validate its *flags* argument. .. _start-symbols: Available start symbols ^^^^^^^^^^^^^^^^^^^^^^^ .. c:var:: int Py_eval_input .. index:: single: Py_CompileString (C function) The start symbol from the Python grammar for isolated expressions; for use with :c:func:`Py_CompileString`. .. c:var:: int Py_file_input .. index:: single: Py_CompileString (C function) The start symbol from the Python grammar for sequences of statements as read from a file or other source; for use with :c:func:`Py_CompileString`. This is the symbol to use when compiling arbitrarily long Python source code. .. c:var:: int Py_single_input .. index:: single: Py_CompileString (C function) The start symbol from the Python grammar for a single statement; for use with :c:func:`Py_CompileString`. This is the symbol used for the interactive interpreter loop. .. c:var:: int Py_func_type_input .. index:: single: Py_CompileString (C function) The start symbol from the Python grammar for a function type; for use with :c:func:`Py_CompileString`. This is used to parse "signature type comments" from :pep:`484`. This requires the :c:macro:`PyCF_ONLY_AST` flag to be set. .. seealso:: * :py:class:`ast.FunctionType` * :pep:`484` .. versionadded:: 3.8 Stack Effects ^^^^^^^^^^^^^ .. seealso:: :py:func:`dis.stack_effect` .. c:macro:: PY_INVALID_STACK_EFFECT Sentinel value representing an invalid stack effect. This is currently equivalent to ``INT_MAX``. .. versionadded:: 3.8 .. c:function:: int PyCompile_OpcodeStackEffect(int opcode, int oparg) Compute the stack effect of *opcode* with argument *oparg*. On success, this function returns the stack effect; on failure, this returns :c:macro:`PY_INVALID_STACK_EFFECT`. .. versionadded:: 3.4 .. c:function:: int PyCompile_OpcodeStackEffectWithJump(int opcode, int oparg, int jump) Similar to :c:func:`PyCompile_OpcodeStackEffect`, but don't include the stack effect of jumping if *jump* is zero. If *jump* is ``0``, this will not include the stack effect of jumping, but if *jump* is ``1`` or ``-1``, this will include it. On success, this function returns the stack effect; on failure, this returns :c:macro:`PY_INVALID_STACK_EFFECT`. .. versionadded:: 3.8 --- ### Doc C Api Weakref (`Doc/c-api/weakref.rst`) .. highlight:: c .. _weakrefobjects: Weak Reference Objects ---------------------- Python supports *weak references* as first-class objects. There are two specific object types which directly implement weak references. The first is a simple reference object, and the second acts as a proxy for the original object as much as it can. .. c:function:: int PyWeakref_Check(PyObject *ob) Return non-zero if *ob* is either a reference or proxy object. This function always succeeds. .. c:function:: int PyWeakref_CheckRef(PyObject *ob) Return non-zero if *ob* is a reference object or a subclass of the reference type. This function always succeeds. .. c:function:: int PyWeakref_CheckRefExact(PyObject *ob) Return non-zero if *ob* is a reference object, but not a subclass of the reference type. This function always succeeds. .. c:function:: int PyWeakref_CheckProxy(PyObject *ob) Return non-zero if *ob* is a proxy object. This function always succeeds. .. c:function:: PyObject* PyWeakref_NewRef(PyObject *ob, PyObject *callback) Return a weak reference object for the object *ob*. This will always return a new reference, but is not guaranteed to create a new object; an existing reference object may be returned. The second parameter, *callback*, can be a callable object that receives notification when *ob* is garbage collected; it should accept a single parameter, which will be the weak reference object itself. *callback* may also be ``None`` or ``NULL``. If *ob* is not a weakly referenceable object, or if *callback* is not callable, ``None``, or ``NULL``, this will raise :exc:`TypeError` and return ``NULL``. .. versionchanged:: 3.16 Raise :exc:`!TypeError` if *callback* is not callable, ``None``, or ``NULL``. .. seealso:: :c:func:`PyType_SUPPORTS_WEAKREFS` for checking if *ob* is weakly referenceable. .. c:function:: PyObject* PyWeakref_NewProxy(PyObject *ob, PyObject *callback) Return a weak reference proxy object for the object *ob*. This will always return a new reference, but is not guaranteed to create a new object; an existing proxy object may be returned. The second parameter, *callback*, can be a callable object that receives notification when *ob* is garbage collected; it should accept a single parameter, which will be the weak reference object itself. *callback* may also be ``None`` or ``NULL``. If *ob* is not a weakly referenceable object, or if *callback* is not callable, ``NULL``, this will raise :exc:`TypeError` and return ``NULL``. .. versionchanged:: 3.16 Raise :exc:`!TypeError` if *callback* is not callable, ``None``, or ``NULL``. .. seealso:: :c:func:`PyType_SUPPORTS_WEAKREFS` for checking if *ob* is weakly referenceable. .. c:function:: int PyWeakref_GetRef(PyObject *ref, PyObject **pobj) Get a :term:`strong reference` to the referenced object from a weak reference, *ref*, into *\*pobj*. * On success, set *\*pobj* to a new :term:`strong reference` to the referenced object and return 1. * If the reference is dead, set *\*pobj* to ``NULL`` and return 0. * On error, raise an exception and return -1. .. versionadded:: 3.13 .. c:function:: int PyWeakref_IsDead(PyObject *ref) Test if the weak reference *ref* is dead. Returns 1 if the reference is dead, 0 if it is alive, and -1 with an error set if *ref* is not a weak reference object. .. versionadded:: 3.14 .. c:function:: void PyObject_ClearWeakRefs(PyObject *object) This function is called by the :c:member:`~PyTypeObject.tp_dealloc` handler to clear weak references. This iterates through the weak references for *object* and calls callbacks for those references which have one. It returns when all callbacks have been attempted. .. c:function:: void PyUnstable_Object_ClearWeakRefsNoCallbacks(PyObject *object) Clears the weakrefs for *object* without calling the callbacks. This function is called by the :c:member:`~PyTypeObject.tp_dealloc` handler for types with finalizers (i.e., :meth:`~object.__del__`). The handler for those objects first calls :c:func:`PyObject_ClearWeakRefs` to clear weakrefs and call their callbacks, then the finalizer, and finally this function to clear any weakrefs that may have been created by the finalizer. In most circumstances, it's more appropriate to use :c:func:`PyObject_ClearWeakRefs` to clear weakrefs instead of this function. .. versionadded:: 3.13 --- ### Doc Deprecations C Api Pending Removal In 3.14 (`Doc/deprecations/c-api-pending-removal-in-3.14.rst`) Pending removal in Python 3.14 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ * The ``ma_version_tag`` field in :c:type:`PyDictObject` for extension modules (:pep:`699`; :gh:`101193`). * Creating :c:data:`immutable types ` with mutable bases (:gh:`95388`). --- ### Doc Deprecations C Api Pending Removal In 3.15 (`Doc/deprecations/c-api-pending-removal-in-3.15.rst`) Pending removal in Python 3.15 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ * The :c:func:`!PyImport_ImportModuleNoBlock`: Use :c:func:`PyImport_ImportModule` instead. * :c:func:`!PyWeakref_GetObject` and :c:func:`!PyWeakref_GET_OBJECT`: Use :c:func:`PyWeakref_GetRef` instead. The `pythoncapi-compat project `__ can be used to get :c:func:`PyWeakref_GetRef` on Python 3.12 and older. * :c:func:`!PyUnicode_AsDecodedObject`: Use :c:func:`PyCodec_Decode` instead. * :c:func:`!PyUnicode_AsDecodedUnicode`: Use :c:func:`PyCodec_Decode` instead; Note that some codecs (for example, "base64") may return a type other than :class:`str`, such as :class:`bytes`. * :c:func:`!PyUnicode_AsEncodedObject`: Use :c:func:`PyCodec_Encode` instead. * :c:func:`!PyUnicode_AsEncodedUnicode`: Use :c:func:`PyCodec_Encode` instead; Note that some codecs (for example, "base64") may return a type other than :class:`bytes`, such as :class:`str`. * Python initialization functions, deprecated in Python 3.13: * :c:func:`!Py_GetPath`: Use :c:func:`PyConfig_Get("module_search_paths") ` (:data:`sys.path`) instead. * :c:func:`!Py_GetPrefix`: Use :c:func:`PyConfig_Get("base_prefix") ` (:data:`sys.base_prefix`) instead. Use :c:func:`PyConfig_Get("prefix") ` (:data:`sys.prefix`) if :ref:`virtual environments ` need to be handled. * :c:func:`!Py_GetExecPrefix`: Use :c:func:`PyConfig_Get("base_exec_prefix") ` (:data:`sys.base_exec_prefix`) instead. Use :c:func:`PyConfig_Get("exec_prefix") ` (:data:`sys.exec_prefix`) if :ref:`virtual environments ` need to be handled. * :c:func:`!Py_GetProgramFullPath`: Use :c:func:`PyConfig_Get("executable") ` (:data:`sys.executable`) instead. * :c:func:`!Py_GetProgramName`: Use :c:func:`PyConfig_Get("executable") ` (:data:`sys.executable`) instead. * :c:func:`!Py_GetPythonHome`: Use :c:func:`PyConfig_Get("home") ` or the :envvar:`PYTHONHOME` environment variable instead. The `pythoncapi-compat project `__ can be used to get :c:func:`PyConfig_Get` on Python 3.13 and older. --- ### Doc Deprecations C Api Pending Removal In 3.16 (`Doc/deprecations/c-api-pending-removal-in-3.16.rst`) Pending removal in Python 3.16 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ * Functions to configure Python's initialization, deprecated in Python 3.11: * :c:func:`!PySys_SetArgvEx()`: Set :c:member:`PyConfig.argv` instead. * :c:func:`!PySys_SetArgv()`: Set :c:member:`PyConfig.argv` instead. * :c:func:`!Py_SetProgramName()`: Set :c:member:`PyConfig.program_name` instead. * :c:func:`!Py_SetPythonHome()`: Set :c:member:`PyConfig.home` instead. The :c:func:`Py_InitializeFromConfig` API should be used with :c:type:`PyConfig` instead. * Global configuration variables: * :c:var:`!Py_DebugFlag`: Use :c:member:`PyConfig.parser_debug` or :c:func:`PyConfig_Get("parser_debug") ` instead. * :c:var:`!Py_VerboseFlag`: Use :c:member:`PyConfig.verbose` or :c:func:`PyConfig_Get("verbose") ` instead. * :c:var:`!Py_QuietFlag`: Use :c:member:`PyConfig.quiet` or :c:func:`PyConfig_Get("quiet") ` instead. * :c:var:`!Py_InteractiveFlag`: Use :c:member:`PyConfig.interactive` or :c:func:`PyConfig_Get("interactive") ` instead. * :c:var:`!Py_InspectFlag`: Use :c:member:`PyConfig.inspect` or :c:func:`PyConfig_Get("inspect") ` instead. * :c:var:`!Py_OptimizeFlag`: Use :c:member:`PyConfig.optimization_level` or :c:func:`PyConfig_Get("optimization_level") ` instead. * :c:var:`!Py_NoSiteFlag`: Use :c:member:`PyConfig.site_import` or :c:func:`PyConfig_Get("site_import") ` instead. * :c:var:`!Py_BytesWarningFlag`: Use :c:member:`PyConfig.bytes_warning` or :c:func:`PyConfig_Get("bytes_warning") ` instead. * :c:var:`!Py_FrozenFlag`: Use :c:member:`PyConfig.pathconfig_warnings` or :c:func:`PyConfig_Get("pathconfig_warnings") ` instead. * :c:var:`!Py_IgnoreEnvironmentFlag`: Use :c:member:`PyConfig.use_environment` or :c:func:`PyConfig_Get("use_environment") ` instead. * :c:var:`!Py_DontWriteBytecodeFlag`: Use :c:member:`PyConfig.write_bytecode` or :c:func:`PyConfig_Get("write_bytecode") ` instead. * :c:var:`!Py_NoUserSiteDirectory`: Use :c:member:`PyConfig.user_site_directory` or :c:func:`PyConfig_Get("user_site_directory") ` instead. * :c:var:`!Py_UnbufferedStdioFlag`: Use :c:member:`PyConfig.buffered_stdio` or :c:func:`PyConfig_Get("buffered_stdio") ` instead. * :c:var:`!Py_HashRandomizationFlag`: Use :c:member:`PyConfig.use_hash_seed` and :c:member:`PyConfig.hash_seed` or :c:func:`PyConfig_Get("hash_seed") ` instead. * :c:var:`!Py_IsolatedFlag`: Use :c:member:`PyConfig.isolated` or :c:func:`PyConfig_Get("isolated") ` instead. * :c:var:`!Py_LegacyWindowsFSEncodingFlag`: Use :c:member:`PyPreConfig.legacy_windows_fs_encoding` or :c:func:`PyConfig_Get("legacy_windows_fs_encoding") ` instead. * :c:var:`!Py_LegacyWindowsStdioFlag`: Use :c:member:`PyConfig.legacy_windows_stdio` or :c:func:`PyConfig_Get("legacy_windows_stdio") ` instead. * :c:var:`!Py_FileSystemDefaultEncoding`, :c:var:`!Py_HasFileSystemDefaultEncoding`: Use :c:member:`PyConfig.filesystem_encoding` or :c:func:`PyConfig_Get("filesystem_encoding") ` instead. * :c:var:`!Py_FileSystemDefaultEncodeErrors`: Use :c:member:`PyConfig.filesystem_errors` or :c:func:`PyConfig_Get("filesystem_errors") ` instead. * :c:var:`!Py_UTF8Mode`: Use :c:member:`PyPreConfig.utf8_mode` or :c:func:`PyConfig_Get("utf8_mode") ` instead. (see :c:func:`Py_PreInitialize`) The :c:func:`Py_InitializeFromConfig` API should be used with :c:type:`PyConfig` to set these options. Or :c:func:`PyConfig_Get` can be used to get these options at runtime. * :c:type:`Py_UNICODE` which was deprecated by :pep:`393`. --- ### Doc Deprecations C Api Pending Removal In 3.18 (`Doc/deprecations/c-api-pending-removal-in-3.18.rst`) Pending removal in Python 3.18 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ * The following private functions are deprecated and planned for removal in Python 3.18: * :c:func:`!_PyBytes_Join`: use :c:func:`PyBytes_Join`. * :c:func:`!_PyDict_GetItemStringWithError`: use :c:func:`PyDict_GetItemStringRef`. * :c:func:`!_PyDict_Pop()`: use :c:func:`PyDict_Pop`. * :c:func:`!_PyLong_Sign()`: use :c:func:`PyLong_GetSign`. * :c:func:`!_PyLong_FromDigits` and :c:func:`!_PyLong_New`: use :c:func:`PyLongWriter_Create`. * :c:func:`!_PyThreadState_UncheckedGet`: use :c:func:`PyThreadState_GetUnchecked`. * :c:func:`!_PyUnicode_AsString`: use :c:func:`PyUnicode_AsUTF8`. * :c:func:`!_PyUnicodeWriter_Init`: replace ``_PyUnicodeWriter_Init(&writer)`` with :c:func:`writer = PyUnicodeWriter_Create(0) `. * :c:func:`!_PyUnicodeWriter_Finish`: replace ``_PyUnicodeWriter_Finish(&writer)`` with :c:func:`PyUnicodeWriter_Finish(writer) `. * :c:func:`!_PyUnicodeWriter_Dealloc`: replace ``_PyUnicodeWriter_Dealloc(&writer)`` with :c:func:`PyUnicodeWriter_Discard(writer) `. * :c:func:`!_PyUnicodeWriter_WriteChar`: replace ``_PyUnicodeWriter_WriteChar(&writer, ch)`` with :c:func:`PyUnicodeWriter_WriteChar(writer, ch) `. * :c:func:`!_PyUnicodeWriter_WriteStr`: replace ``_PyUnicodeWriter_WriteStr(&writer, str)`` with :c:func:`PyUnicodeWriter_WriteStr(writer, str) `. * :c:func:`!_PyUnicodeWriter_WriteSubstring`: replace ``_PyUnicodeWriter_WriteSubstring(&writer, str, start, end)`` with :c:func:`PyUnicodeWriter_WriteSubstring(writer, str, start, end) `. * :c:func:`!_PyUnicodeWriter_WriteASCIIString`: replace ``_PyUnicodeWriter_WriteASCIIString(&writer, str)`` with :c:func:`PyUnicodeWriter_WriteASCII(writer, str) `. * :c:func:`!_PyUnicodeWriter_WriteLatin1String`: replace ``_PyUnicodeWriter_WriteLatin1String(&writer, str)`` with :c:func:`PyUnicodeWriter_WriteUTF8(writer, str) `. * :c:func:`!_PyUnicodeWriter_Prepare`: (no replacement). * :c:func:`!_PyUnicodeWriter_PrepareKind`: (no replacement). * :c:func:`!_Py_HashPointer`: use :c:func:`Py_HashPointer`. * :c:func:`!_Py_fopen_obj`: use :c:func:`Py_fopen`. * :c:func:`PyGen_New`: (no replacement). * :c:func:`PyGen_NewWithQualName`: (no replacement). * :c:func:`PyCoro_New`: (no replacement). * :c:func:`PyAsyncGen_New`: (no replacement). The `pythoncapi-compat project `__ can be used to get these new public functions on Python 3.13 and older. (Contributed by Victor Stinner in :gh:`128863`.) --- --- METRICS --- - Files Extracted: 100 - Estimated Token Budget: ~442096 tokens - Canonical Reference: https://codewiki.google/github.com/python/cpython