pytest Documentation

File: doc/en/announce/index.rst Copy Chapter

Release announcements

.. toctree::
:maxdepth: 2

release-9.1.1
release-9.1.0
release-9.0.3
release-9.0.2
release-9.0.1
release-9.0.0
release-8.4.2
release-8.4.1
release-8.4.0
release-8.3.5
release-8.3.4
release-8.3.3
release-8.3.2
release-8.3.1
release-8.3.0
release-8.2.2
release-8.2.1
release-8.2.0
release-8.1.2
release-8.1.1
release-8.1.0
release-8.0.2
release-8.0.1
release-8.0.0
release-8.0.0rc2
release-8.0.0rc1
release-7.4.4
release-7.4.3
release-7.4.2
release-7.4.1
release-7.4.0
release-7.3.2
release-7.3.1
release-7.3.0
release-7.2.2
release-7.2.1
release-7.2.0
release-7.1.3
release-7.1.2
release-7.1.1
release-7.1.0
release-7.0.1
release-7.0.0
release-7.0.0rc1
release-6.2.5
release-6.2.4
release-6.2.3
release-6.2.2
release-6.2.1
release-6.2.0
release-6.1.2
release-6.1.1
release-6.1.0
release-6.0.2
release-6.0.1
release-6.0.0
release-6.0.0rc1
release-5.4.3
release-5.4.2
release-5.4.1
release-5.4.0
release-5.3.5
release-5.3.4
release-5.3.3
release-5.3.2
release-5.3.1
release-5.3.0
release-5.2.4
release-5.2.3
release-5.2.2
release-5.2.1
release-5.2.0
release-5.1.3
release-5.1.2
release-5.1.1
release-5.1.0
release-5.0.1
release-5.0.0
release-4.6.9
release-4.6.8
release-4.6.7
release-4.6.6
release-4.6.5
release-4.6.4
release-4.6.3
release-4.6.2
release-4.6.1
release-4.6.0
release-4.5.0
release-4.4.2
release-4.4.1
release-4.4.0
release-4.3.1
release-4.3.0
release-4.2.1
release-4.2.0
release-4.1.1
release-4.1.0
release-4.0.2
release-4.0.1
release-4.0.0
release-3.10.1
release-3.10.0
release-3.9.3
release-3.9.2
release-3.9.1
release-3.9.0
release-3.8.2
release-3.8.1
release-3.8.0
release-3.7.4
release-3.7.3
release-3.7.2
release-3.7.1
release-3.7.0
release-3.6.4
release-3.6.3
release-3.6.2
release-3.6.1
release-3.6.0
release-3.5.1
release-3.5.0
release-3.4.2
release-3.4.1
release-3.4.0
release-3.3.2
release-3.3.1
release-3.3.0
release-3.2.5
release-3.2.4
release-3.2.3
release-3.2.2
release-3.2.1
release-3.2.0
release-3.1.3
release-3.1.2
release-3.1.1
release-3.1.0
release-3.0.7
release-3.0.6
release-3.0.5
release-3.0.4
release-3.0.3
release-3.0.2
release-3.0.1
release-3.0.0
sprint2016
release-2.9.2
release-2.9.1
release-2.9.0
release-2.8.7
release-2.8.6
release-2.8.5
release-2.8.4
release-2.8.3
release-2.8.2
release-2.7.2
release-2.7.1
release-2.7.0
release-2.6.3
release-2.6.2
release-2.6.1
release-2.6.0
release-2.5.2
release-2.5.1
release-2.5.0
release-2.4.2
release-2.4.1
release-2.4.0
release-2.3.5
release-2.3.4
release-2.3.3
release-2.3.2
release-2.3.1
release-2.3.0
release-2.2.4
release-2.2.2
release-2.2.1
release-2.2.0
release-2.1.3
release-2.1.2
release-2.1.1
release-2.1.0
release-2.0.3
release-2.0.2
release-2.0.1
release-2.0.0

File: doc/en/announce/release-2.0.0.rst Copy Chapter

py.test 2.0.0: asserts++, unittest++, reporting++, config++, docs++

Welcome to pytest-2.0.0, a major new release of "py.test", the rapid
easy Python testing tool. There are many new features and enhancements,
see below for summary and detailed lists. A lot of long-deprecated code
has been removed, resulting in a much smaller and cleaner
implementation. See the new docs with examples here:

http://pytest.org/en/stable/index.html

A note on packaging: pytest used to part of the "py" distribution up
until version py-1.3.4 but this has changed now: pytest-2.0.0 only
contains py.test related code and is expected to be backward-compatible
to existing test code. If you want to install pytest, just type one of::

pip install -U pytest
easy_install -U pytest

Many thanks to all issue reporters and people asking questions or
complaining. Particular thanks to Floris Bruynooghe and Ronny Pfannschmidt
for their great coding contributions and many others for feedback and help.

best,
holger krekel

New Features Copy Chapter

• new invocations through Python interpreter and from Python::

  python -m pytest # on all pythons >= 2.5

  or from a python program::

  import pytest ; pytest.main(arglist, pluginlist)

  see http://pytest.org/en/stable/how-to/usage.html for details.

• new and better reporting information in assert expressions
  if comparing lists, sequences or strings.

  see http://pytest.org/en/stable/how-to/assert.html#newreport

• new configuration through ini-files (setup.cfg or tox.ini recognized),
  for example::

  [pytest]
  norecursedirs = .hg data* # don't ever recurse in such dirs
  addopts = -x --pyargs # add these command line options by default

  see http://pytest.org/en/stable/reference/customize.html

• improved standard unittest support. In general py.test should now
  better be able to run custom unittest.TestCases like twisted trial
  or Django based TestCases. Also you can now run the tests of an
  installed 'unittest' package with py.test::

  py.test --pyargs unittest

• new "-q" option which decreases verbosity and prints a more
  nose/unittest-style "dot" output.

• many, many, more detailed improvements details

Fixes Copy Chapter

• fix issue126 - introduce py.test.set_trace() to trace execution via
  PDB during the running of tests even if capturing is ongoing.
• fix issue124 - make reporting more resilient against tests opening
  files on filedescriptor 1 (stdout).
• fix issue109 - sibling conftest.py files will not be loaded.
  (and Directory collectors cannot be customized anymore from a Directory's
  conftest.py - this needs to happen at least one level up).
• fix issue88 (finding custom test nodes from command line arg)
• fix issue93 stdout/stderr is captured while importing conftest.py
• fix bug: unittest collected functions now also can have "pytestmark"
  applied at class/module level

Important Notes Copy Chapter

• The usual way in pre-2.0 times to use py.test in python code was
  to import "py" and then e.g. use "py.test.raises" for the helper.
  This remains valid and is not planned to be deprecated. However,
  in most examples and internal code you'll find "import pytest"
  and "pytest.raises" used as the recommended default way.

• pytest now first performs collection of the complete test suite
  before running any test. This changes for example the semantics of when
  pytest_collectstart/pytest_collectreport are called. Some plugins may
  need upgrading.

• The pytest package consists of a 400 LOC core.py and about 20 builtin plugins,
  summing up to roughly 5000 LOCs, including docstrings. To be fair, it also
  uses generic code from the "pylib", and the new "py" package to help
  with filesystem and introspection/code manipulation.

(Incompatible) Removals Copy Chapter

• py.test.config is now only available if you are in a test run.

• the following (mostly already deprecated) functionality was removed:

  • removed support for Module/Class/... collection node definitions
    in conftest.py files. They will cause nothing special.
  • removed support for calling the pre-1.0 collection API of "run()" and "join"
  • removed reading option values from conftest.py files or env variables.
    This can now be done much, much, better and easier through the ini-file
    mechanism and the "addopts" entry in particular.
  • removed the "disabled" attribute in test classes. Use the skipping
    and pytestmark mechanism to skip or xfail a test class.

• py.test.collect.Directory does not exist anymore and it
  is not possible to provide an own "Directory" object.
  If you have used this and don't know what to do, get
  in contact. We'll figure something out.

  Note that pytest_collect_directory() is still called but
  any return value will be ignored. This allows to keep
  old code working that performed for example "py.test.skip()"
  in collect() to prevent recursion into directory trees
  if a certain dependency or command line option is missing.

see :ref:changelog for more detailed changes.

File: doc/en/announce/release-2.0.1.rst Copy Chapter

py.test 2.0.1: bug fixes

Welcome to pytest-2.0.1, a maintenance and bug fix release of pytest,
a mature testing tool for Python, supporting CPython 2.4-3.2, Jython
and latest PyPy interpreters. See extensive docs with tested examples here:

http://pytest.org/

If you want to install or upgrade pytest, just type one of::

pip install -U pytest # or
easy_install -U pytest

Many thanks to all issue reporters and people asking questions or
complaining. Particular thanks to Floris Bruynooghe and Ronny Pfannschmidt
for their great coding contributions and many others for feedback and help.

best,
holger krekel

Changes between 2.0.0 and 2.0.1 Copy Chapter

• refine and unify initial capturing so that it works nicely
  even if the logging module is used on an early-loaded conftest.py
  file or plugin.
• fix issue12 - show plugin versions with "--version" and
  "--traceconfig" and also document how to add extra information
  to reporting test header
• fix issue17 (import-* reporting issue on python3) by
  requiring py>1.4.0 (1.4.1 is going to include it)
• fix issue10 (numpy arrays truth checking) by refining
  assertion interpretation in py lib
• fix issue15: make nose compatibility tests compatible
  with python3 (now that nose-1.0 supports python3)
• remove somewhat surprising "same-conftest" detection because
  it ignores conftest.py when they appear in several subdirs.
• improve assertions ("not in"), thanks Floris Bruynooghe
• improve behaviour/warnings when running on top of "python -OO"
  (assertions and docstrings are turned off, leading to potential
  false positives)
• introduce a pytest_cmdline_processargs(args) hook
  to allow dynamic computation of command line arguments.
  This fixes a regression because py.test prior to 2.0
  allowed to set command line options from conftest.py
  files which so far pytest-2.0 only allowed from ini-files now.
• fix issue7: assert failures in doctest modules.
  unexpected failures in doctests will not generally
  show nicer, i.e. within the doctest failing context.
• fix issue9: setup/teardown functions for an xfail-marked
  test will report as xfail if they fail but report as normally
  passing (not xpassing) if they succeed. This only is true
  for "direct" setup/teardown invocations because teardown_class/
  teardown_module cannot closely relate to a single test.
• fix issue14: no logging errors at process exit
• refinements to "collecting" output on non-ttys
• refine internal plugin registration and --traceconfig output
• introduce a mechanism to prevent/unregister plugins from the
  command line, see http://pytest.org/en/stable/how-to/plugins.html#cmdunregister
• activate resultlog plugin by default
• fix regression wrt yielded tests which due to the
  collection-before-running semantics were not
  setup as with pytest 1.3.4. Note, however, that
  the recommended and much cleaner way to do test
  parametrization remains the "pytest_generate_tests"
  mechanism, see the docs.

File: doc/en/announce/release-2.0.2.rst Copy Chapter

py.test 2.0.2: bug fixes, improved xfail/skip expressions, speed ups

Welcome to pytest-2.0.2, a maintenance and bug fix release of pytest,
a mature testing tool for Python, supporting CPython 2.4-3.2, Jython
and latest PyPy interpreters. See the extensive docs with tested examples here:

http://pytest.org/

If you want to install or upgrade pytest, just type one of::

pip install -U pytest # or
easy_install -U pytest

Many thanks to all issue reporters and people asking questions
or complaining, particularly Jurko for his insistence,
Laura, Victor and Brianna for helping with improving
and Ronny for his general advise.

best,
holger krekel

Changes between 2.0.1 and 2.0.2 Copy Chapter

• tackle issue32 - speed up test runs of very quick test functions
  by reducing the relative overhead

• fix issue30 - extended xfail/skipif handling and improved reporting.
  If you have a syntax error in your skip/xfail
  expressions you now get nice error reports.

  Also you can now access module globals from xfail/skipif
  expressions so that this for example works now::

  import pytest
  import mymodule
  @pytest.mark.skipif("mymodule.version[0] == "1")
  def test_function():
  pass

  This will not run the test function if the module's version string
  does not start with a "1". Note that specifying a string instead
  of a boolean expressions allows py.test to report meaningful information
  when summarizing a test run as to what conditions lead to skipping
  (or xfail-ing) tests.

• fix issue28 - setup_method and pytest_generate_tests work together
  The setup_method fixture method now gets called also for
  test function invocations generated from the pytest_generate_tests
  hook.

• fix issue27 - collectonly and keyword-selection (-k) now work together
  Also, if you do "py.test --collectonly -q" you now get a flat list
  of test ids that you can use to paste to the py.test commandline
  in order to execute a particular test.

• fix issue25 avoid reported problems with --pdb and python3.2/encodings output

• fix issue23 - tmpdir argument now works on Python3.2 and WindowsXP
  Starting with Python3.2 os.symlink may be supported. By requiring
  a newer py lib version the py.path.local() implementation acknowledges
  this.

• fixed typos in the docs (thanks Victor Garcia, Brianna Laugher) and particular
  thanks to Laura Creighton who also reviewed parts of the documentation.

• fix slightly wrong output of verbose progress reporting for classes
  (thanks Amaury)

• more precise (avoiding of) deprecation warnings for node.Class|Function accesses

• avoid std unittest assertion helper code in tracebacks (thanks Ronny)

File: doc/en/announce/release-2.0.3.rst Copy Chapter

py.test 2.0.3: bug fixes and speed ups

Welcome to pytest-2.0.3, a maintenance and bug fix release of pytest,
a mature testing tool for Python, supporting CPython 2.4-3.2, Jython
and latest PyPy interpreters. See the extensive docs with tested examples here:

http://pytest.org/

If you want to install or upgrade pytest, just type one of::

pip install -U pytest # or
easy_install -U pytest

There also is a bugfix release 1.6 of pytest-xdist, the plugin
that enables seamless distributed and "looponfail" testing for Python.

best,
holger krekel

Changes between 2.0.2 and 2.0.3 Copy Chapter

• fix issue38: nicer tracebacks on calls to hooks, particularly early
  configure/sessionstart ones

• fix missing skip reason/meta information in junitxml files, reported
  via http://lists.idyll.org/pipermail/testing-in-python/2011-March/003928.html

• fix issue34: avoid collection failure with "test" prefixed classes
  deriving from object.

• don't require zlib (and other libs) for genscript plugin without
  --genscript actually being used.

• speed up skips (by not doing a full traceback representation
  internally)

• fix issue37: avoid invalid characters in junitxml's output

File: doc/en/announce/release-2.1.0.rst Copy Chapter

py.test 2.1.0: perfected assertions and bug fixes

Welcome to the release of pytest-2.1, a mature testing tool for Python,
supporting CPython 2.4-3.2, Jython and latest PyPy interpreters. See
the improved extensive docs (now also as PDF!) with tested examples here:

http://pytest.org/

The single biggest news about this release are perfected assertions
courtesy of Benjamin Peterson. You can now safely use assert
statements in test modules without having to worry about side effects
or python optimization ("-OO") options. This is achieved by rewriting
assert statements in test modules upon import, using a PEP302 hook.
See https://docs.pytest.org/en/stable/how-to/assert.html for
detailed information. The work has been partly sponsored by my company,
merlinux GmbH.

For further details on bug fixes and smaller enhancements see below.

If you want to install or upgrade pytest, just type one of::

pip install -U pytest # or
easy_install -U pytest

best,
holger krekel / https://merlinux.eu/

Changes between 2.0.3 and 2.1.0 Copy Chapter

• fix issue53 call nosestyle setup functions with correct ordering
• fix issue58 and issue59: new assertion code fixes
• merge Benjamin's assertionrewrite branch: now assertions
  for test modules on python 2.6 and above are done by rewriting
  the AST and saving the pyc file before the test module is imported.
  see doc/assert.txt for more info.
• fix issue43: improve doctests with better traceback reporting on
  unexpected exceptions
• fix issue47: timing output in junitxml for test cases is now correct
• fix issue48: typo in MarkInfo repr leading to exception
• fix issue49: avoid confusing error when initialization partially fails
• fix issue44: env/username expansion for junitxml file path
• show releaselevel information in test runs for pypy
• reworked doc pages for better navigation and PDF generation
• report KeyboardInterrupt even if interrupted during session startup
• fix issue 35 - provide PDF doc version and download link from index page

File: doc/en/announce/release-2.1.1.rst Copy Chapter

py.test 2.1.1: assertion fixes and improved junitxml output

pytest-2.1.1 is a backward compatible maintenance release of the
popular py.test testing tool. See extensive docs with examples here:

http://pytest.org/

Most bug fixes address remaining issues with the perfected assertions
introduced with 2.1.0 - many thanks to the bug reporters and to Benjamin
Peterson for helping to fix them. Also, junitxml output now produces
system-out/err tags which lead to better displays of tracebacks with Jenkins.

Also a quick note to package maintainers and others interested: there now
is a "pytest" man page which can be generated with "make man" in doc/.

If you want to install or upgrade pytest, just type one of::

pip install -U pytest # or
easy_install -U pytest

best,
holger krekel / https://merlinux.eu/

Changes between 2.1.0 and 2.1.1 Copy Chapter

• fix issue64 / pytest.set_trace now works within pytest_generate_tests hooks
• fix issue60 / fix error conditions involving the creation of pycache
• fix issue63 / assertion rewriting on inserts involving strings containing '%'
• fix assertion rewriting on calls with a ** arg
• don't cache rewritten modules if bytecode generation is disabled
• fix assertion rewriting in read-only directories
• fix issue59: provide system-out/err tags for junitxml output
• fix issue61: assertion rewriting on boolean operations with 3 or more operands
• you can now build a man page with "cd doc ; make man"

File: doc/en/announce/release-2.1.2.rst Copy Chapter

py.test 2.1.2: bug fixes and fixes for jython

pytest-2.1.2 is a minor backward compatible maintenance release of the
popular py.test testing tool. pytest is commonly used for unit,
functional- and integration testing. See extensive docs with examples
here:

http://pytest.org/

Most bug fixes address remaining issues with the perfected assertions
introduced in the 2.1 series - many thanks to the bug reporters and to Benjamin
Peterson for helping to fix them. pytest should also work better with
Jython-2.5.1 (and Jython trunk).

If you want to install or upgrade pytest, just type one of::

pip install -U pytest # or
easy_install -U pytest

best,
holger krekel / https://merlinux.eu/

Changes between 2.1.1 and 2.1.2 Copy Chapter

• fix assertion rewriting on files with windows newlines on some Python versions
• refine test discovery by package/module name (--pyargs), thanks Florian Mayer
• fix issue69 / assertion rewriting fixed on some boolean operations
• fix issue68 / packages now work with assertion rewriting
• fix issue66: use different assertion rewriting caches when the -O option is passed
• don't try assertion rewriting on Jython, use reinterp

File: doc/en/announce/release-2.1.3.rst Copy Chapter

py.test 2.1.3: just some more fixes

pytest-2.1.3 is a minor backward compatible maintenance release of the
popular py.test testing tool. It is commonly used for unit, functional-
and integration testing. See extensive docs with examples here:

http://pytest.org/

The release contains another fix to the perfected assertions introduced
with the 2.1 series as well as the new possibility to customize reporting
for assertion expressions on a per-directory level.

If you want to install or upgrade pytest, just type one of::

pip install -U pytest # or
easy_install -U pytest

Thanks to the bug reporters and to Ronny Pfannschmidt, Benjamin Peterson
and Floris Bruynooghe who implemented the fixes.

best,
holger krekel

Changes between 2.1.2 and 2.1.3 Copy Chapter

• fix issue79: assertion rewriting failed on some comparisons in boolops,
• correctly handle zero length arguments (a la pytest '')
• fix issue67 / junitxml now contains correct test durations
• fix issue75 / skipping test failure on jython
• fix issue77 / Allow assertrepr_compare hook to apply to a subset of tests