kyfx
/
pytest
mirror of https://github.com/pytest-dev/pytest.git
4
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity

Merge remote-tracking branch 'origin/main' into raisesgroup

This commit is contained in:
jakkdl 2025-03-03 12:28:16 +01:00
parent 4e9765203d e18b245f0a
commit 5186f3677e
No known key found for this signature in database
GPG Key ID: 442D17F1B28AF751
55 changed files with 903 additions and 199 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

2
.pre-commit-config.yaml
View File

@ -1,6 +1,6 @@
repos:
- repo: https://github.com/astral-sh/ruff-pre-commit
rev: "v0.9.6"
rev: "v0.9.7"
hooks:
- id: ruff
args: ["--fix"]

3
AUTHORS
View File

@ -32,6 +32,7 @@ Andras Tim
Andrea Cimatoribus
Andreas Motl
Andreas Zeidler
Andrew Pikul
Andrew Shapton
Andrey Paramonov
Andrzej Klajnert
@ -43,6 +44,7 @@ Anthony Shaw
Anthony Sottile
Anton Grinevich
Anton Lodder
Anton Zhilin
Antony Lee
Arel Cordero
Arias Emmanuel
@ -263,6 +265,7 @@ lovetheguitar
Lukas Bednar
Luke Murphy
Maciek Fijalkowski
Maggie Chung
Maho
Maik Figura
Mandeep Bhutani

1
changelog/10558.doc.rst
View File

@ -1 +0,0 @@
Fix ambiguous docstring of :func:`pytest.Config.getoption`.

1
changelog/10829.doc.rst
View File

@ -1 +0,0 @@
Improve documentation on the current handling of the ``--basetemp`` option and its lack of retention functionality (:ref:`temporary directory location and retention`).

1
changelog/11777.bugfix.rst
View File

@ -1 +0,0 @@
Fixed issue where sequences were still being shortened even with ``-vv`` verbosity.

1
changelog/12081.feature.rst Normal file
View File

@ -0,0 +1 @@
Added :fixture:`capteesys` to capture AND pass output to next handler set by ``--capture=``.

1
changelog/12497.contrib.rst
View File

@ -1 +0,0 @@
Fixed two failing pdb-related tests on Python 3.13.

1
changelog/12592.bugfix.rst
View File

@ -1 +0,0 @@
Fixed :class:`KeyError` crash when using ``--import-mode=importlib`` in a directory layout where a directory contains a child directory with the same name.

1
changelog/12818.bugfix.rst
View File

@ -1 +0,0 @@
Assertion rewriting now preserves the source ranges of the original instructions, making it play well with tools that deal with the ``AST``, like `executing <https://github.com/alexmojaki/executing>`__.

1
changelog/12849.bugfix.rst
View File

@ -1 +0,0 @@
ANSI escape codes for colored output now handled correctly in :func:`pytest.fail` with `pytrace=False`.

1
changelog/12866.doc.rst
View File

@ -1 +0,0 @@
Improved cross-references concerning the :fixture:`recwarn` fixture.

1
changelog/12888.bugfix.rst
View File

@ -1 +0,0 @@
Fixed broken input when using Python 3.13+ and a ``libedit`` build of Python, such as on macOS or with uv-managed Python binaries from the ``python-build-standalone`` project. This could manifest e.g. by a broken prompt when using ``Pdb``, or seeing empty inputs with manual usage of ``input()`` and suspended capturing.

1
changelog/12946.bugfix.rst Normal file
View File

@ -0,0 +1 @@
Fixed missing help for :mod:`pdb` commands wrapped by pytest -- by :user:`adamchainz`.

1
changelog/12966.doc.rst
View File

@ -1 +0,0 @@
Clarify :ref:`filterwarnings` docs on filter precedence/order when using multiple :ref:`@pytest.mark.filterwarnings <pytest.mark.filterwarnings ref>` marks.

1
changelog/13026.bugfix.rst
View File

@ -1 +0,0 @@
Fixed :class:`AttributeError` crash when using ``--import-mode=importlib`` when top-level directory same name as another module of the standard library.

1
changelog/13053.bugfix.rst
View File

@ -1 +0,0 @@
Fixed a regression in pytest 8.3.4 where, when using ``--import-mode=importlib``, a directory containing py file with the same name would cause an ``ImportError``

1
changelog/13083.bugfix.rst
View File

@ -1 +0,0 @@
Fixed issue where pytest could crash if one of the collected directories got removed during collection.

1
changelog/13112.contrib.rst
View File

@ -1 +0,0 @@
Fixed selftest failures in ``test_terminal.py`` with Pygments >= 2.19.0

3
changelog/13228.feature.rst Normal file
View File

@ -0,0 +1,3 @@
:ref:`hidden-param` can now be used in ``id`` of :func:`pytest.param` or in
``ids`` of :py:func:`Metafunc.parametrize <pytest.Metafunc.parametrize>`.
It hides the parameter set from the test name.

1
changelog/13253.feature.rst Normal file
View File

@ -0,0 +1 @@
New flag: :ref:`--disable-plugin-autoload <disable_plugin_autoload>` which works as an alternative to :envvar:`PYTEST_DISABLE_PLUGIN_AUTOLOAD` when setting environment variables is inconvenient; and allows setting it in config files with :confval:`addopts`.

1
changelog/9353.bugfix.rst
View File

@ -1 +0,0 @@
:func:`pytest.approx` now uses strict equality when given booleans.

2
doc/en/announce/index.rst
View File

@ -6,6 +6,8 @@ Release announcements
:maxdepth: 2
release-8.3.5
release-8.3.4
release-8.3.3
release-8.3.2
release-8.3.1

30
doc/en/announce/release-8.3.4.rst Normal file
View File

@ -0,0 +1,30 @@
pytest-8.3.4
=======================================
pytest 8.3.4 has just been released to PyPI.
This is a bug-fix release, being a drop-in replacement. To upgrade::
pip install --upgrade pytest
The full changelog is available at https://docs.pytest.org/en/stable/changelog.html.
Thanks to all of the contributors to this release:
* Bruno Oliveira
* Florian Bruhin
* Frank Hoffmann
* Jakob van Santen
* Leonardus Chen
* Pierre Sassoulas
* Pradeep Kumar
* Ran Benita
* Serge Smertin
* Stefaan Lippens
* Sviatoslav Sydorenko (Святослав Сидоренко)
* dongfangtianyu
* suspe
Happy testing,
The pytest Development Team

26
doc/en/announce/release-8.3.5.rst Normal file
View File

@ -0,0 +1,26 @@
pytest-8.3.5
=======================================
pytest 8.3.5 has just been released to PyPI.
This is a bug-fix release, being a drop-in replacement.
The full changelog is available at https://docs.pytest.org/en/stable/changelog.html.
Thanks to all of the contributors to this release:
* Bruno Oliveira
* Florian Bruhin
* John Litborn
* Kenny Y
* Ran Benita
* Sadra Barikbin
* Vincent (Wen Yu) Ge
* delta87
* dongfangtianyu
* mwychung
* 🇺🇦 Sviatoslav Sydorenko (Святослав Сидоренко)
Happy testing,
The pytest Development Team

4
doc/en/broken-dep-constraints.txt
View File

@ -1,6 +1,2 @@
# This file contains transitive dependencies that need to be pinned for some reason.
# Eventually this file will be empty, but in this case keep it around for future use.
# Pin towncrier temporarily due to incompatibility with sphinxcontrib-towncrier:
# https://github.com/sphinx-contrib/sphinxcontrib-towncrier/issues/92
towncrier<24.7

44
doc/en/builtin.rst
View File

@ -33,7 +33,7 @@ For information about fixtures, see :ref:`fixtures`. To see a complete list of a
Values can be any object handled by the json stdlib module.
capsysbinary -- .../_pytest/capture.py:1006
capsysbinary -- .../_pytest/capture.py:1024
Enable bytes capturing of writes to ``sys.stdout`` and ``sys.stderr``.
The captured output is made available via ``capsysbinary.readouterr()``
@ -51,7 +51,7 @@ For information about fixtures, see :ref:`fixtures`. To see a complete list of a
captured = capsysbinary.readouterr()
assert captured.out == b"hello\n"
capfd -- .../_pytest/capture.py:1034
capfd -- .../_pytest/capture.py:1052
Enable text capturing of writes to file descriptors ``1`` and ``2``.
The captured output is made available via ``capfd.readouterr()`` method
@ -69,7 +69,7 @@ For information about fixtures, see :ref:`fixtures`. To see a complete list of a
captured = capfd.readouterr()
assert captured.out == "hello\n"
capfdbinary -- .../_pytest/capture.py:1062
capfdbinary -- .../_pytest/capture.py:1080
Enable bytes capturing of writes to file descriptors ``1`` and ``2``.
The captured output is made available via ``capfd.readouterr()`` method
@ -87,7 +87,7 @@ For information about fixtures, see :ref:`fixtures`. To see a complete list of a
captured = capfdbinary.readouterr()
assert captured.out == b"hello\n"
capsys -- .../_pytest/capture.py:978
capsys -- .../_pytest/capture.py:996
Enable text capturing of writes to ``sys.stdout`` and ``sys.stderr``.
The captured output is made available via ``capsys.readouterr()`` method
@ -178,16 +178,11 @@ For information about fixtures, see :ref:`fixtures`. To see a complete list of a
Return a :class:`pytest.TempdirFactory` instance for the test session.
tmpdir -- .../_pytest/legacypath.py:305
Return a temporary directory path object which is unique to each test
function invocation, created as a sub directory of the base temporary
directory.
By default, a new base temporary directory is created each test session,
and old bases are removed after 3 sessions, to aid in debugging. If
``--basetemp`` is used then it is cleared each session. See
:ref:`temporary directory location and retention`.
The returned object is a `legacy_path`_ object.
Return a temporary directory (as `legacy_path`_ object)
which is unique to each test function invocation.
The temporary directory is created as a subdirectory
of the base temporary directory, with configurable retention,
as discussed in :ref:`temporary directory location and retention`.
.. note::
These days, it is preferred to use ``tmp_path``.
@ -236,22 +231,15 @@ For information about fixtures, see :ref:`fixtures`. To see a complete list of a
See :ref:`warnings` for information on warning categories.
tmp_path_factory [session scope] -- .../_pytest/tmpdir.py:242
tmp_path_factory [session scope] -- .../_pytest/tmpdir.py:241
Return a :class:`pytest.TempPathFactory` instance for the test session.
tmp_path -- .../_pytest/tmpdir.py:257
Return a temporary directory path object which is unique to each test
function invocation, created as a sub directory of the base temporary
directory.
By default, a new base temporary directory is created each test session,
and old bases are removed after 3 sessions, to aid in debugging.
This behavior can be configured with :confval:`tmp_path_retention_count` and
:confval:`tmp_path_retention_policy`.
If ``--basetemp`` is used then it is cleared each session. See
:ref:`temporary directory location and retention`.
The returned object is a :class:`pathlib.Path` object.
tmp_path -- .../_pytest/tmpdir.py:256
Return a temporary directory (as :class:`pathlib.Path` object)
which is unique to each test function invocation.
The temporary directory is created as a subdirectory
of the base temporary directory, with configurable retention,
as discussed in :ref:`temporary directory location and retention`.
========================== no tests ran in 0.12s ===========================

82
doc/en/changelog.rst
View File

@ -31,6 +31,88 @@ with advance notice in the **Deprecations** section of releases.
.. towncrier release notes start
pytest 8.3.5 (2025-03-02)
=========================
Bug fixes
---------
- `#11777 <https://github.com/pytest-dev/pytest/issues/11777>`_: Fixed issue where sequences were still being shortened even with ``-vv`` verbosity.
- `#12888 <https://github.com/pytest-dev/pytest/issues/12888>`_: Fixed broken input when using Python 3.13+ and a ``libedit`` build of Python, such as on macOS or with uv-managed Python binaries from the ``python-build-standalone`` project. This could manifest e.g. by a broken prompt when using ``Pdb``, or seeing empty inputs with manual usage of ``input()`` and suspended capturing.
- `#13026 <https://github.com/pytest-dev/pytest/issues/13026>`_: Fixed :class:`AttributeError` crash when using ``--import-mode=importlib`` when top-level directory same name as another module of the standard library.
- `#13053 <https://github.com/pytest-dev/pytest/issues/13053>`_: Fixed a regression in pytest 8.3.4 where, when using ``--import-mode=importlib``, a directory containing py file with the same name would cause an ``ImportError``
- `#13083 <https://github.com/pytest-dev/pytest/issues/13083>`_: Fixed issue where pytest could crash if one of the collected directories got removed during collection.
Improved documentation
----------------------
- `#12842 <https://github.com/pytest-dev/pytest/issues/12842>`_: Added dedicated page about using types with pytest.
See :ref:`types` for detailed usage.
Contributor-facing changes
--------------------------
- `#13112 <https://github.com/pytest-dev/pytest/issues/13112>`_: Fixed selftest failures in ``test_terminal.py`` with Pygments >= 2.19.0
- `#13256 <https://github.com/pytest-dev/pytest/issues/13256>`_: Support for Towncrier versions released in 2024 has been re-enabled
when building Sphinx docs -- by :user:`webknjaz`.
pytest 8.3.4 (2024-12-01)
=========================
Bug fixes
---------
- `#12592 <https://github.com/pytest-dev/pytest/issues/12592>`_: Fixed :class:`KeyError` crash when using ``--import-mode=importlib`` in a directory layout where a directory contains a child directory with the same name.
- `#12818 <https://github.com/pytest-dev/pytest/issues/12818>`_: Assertion rewriting now preserves the source ranges of the original instructions, making it play well with tools that deal with the ``AST``, like `executing <https://github.com/alexmojaki/executing>`__.
- `#12849 <https://github.com/pytest-dev/pytest/issues/12849>`_: ANSI escape codes for colored output now handled correctly in :func:`pytest.fail` with `pytrace=False`.
- `#9353 <https://github.com/pytest-dev/pytest/issues/9353>`_: :func:`pytest.approx` now uses strict equality when given booleans.
Improved documentation
----------------------
- `#10558 <https://github.com/pytest-dev/pytest/issues/10558>`_: Fix ambiguous docstring of :func:`pytest.Config.getoption`.
- `#10829 <https://github.com/pytest-dev/pytest/issues/10829>`_: Improve documentation on the current handling of the ``--basetemp`` option and its lack of retention functionality (:ref:`temporary directory location and retention`).
- `#12866 <https://github.com/pytest-dev/pytest/issues/12866>`_: Improved cross-references concerning the :fixture:`recwarn` fixture.
- `#12966 <https://github.com/pytest-dev/pytest/issues/12966>`_: Clarify :ref:`filterwarnings` docs on filter precedence/order when using multiple :ref:`@pytest.mark.filterwarnings <pytest.mark.filterwarnings ref>` marks.
Contributor-facing changes
--------------------------
- `#12497 <https://github.com/pytest-dev/pytest/issues/12497>`_: Fixed two failing pdb-related tests on Python 3.13.
pytest 8.3.3 (2024-09-09)
=========================

15
doc/en/example/parametrize.rst
View File

@ -162,7 +162,7 @@ objects, they are still using the default pytest representation:
rootdir: /home/sweet/project
collected 8 items
<Dir parametrize.rst-204>
<Dir parametrize.rst-206>
<Module test_time.py>
<Function test_timedistance_v0[a0-b0-expected0]>
<Function test_timedistance_v0[a1-b1-expected1]>
@ -239,7 +239,7 @@ If you just collect tests you'll also nicely see 'advanced' and 'basic' as varia
rootdir: /home/sweet/project
collected 4 items
<Dir parametrize.rst-204>
<Dir parametrize.rst-206>
<Module test_scenarios.py>
<Class TestSampleWithScenarios>
<Function test_demo1[basic]>
@ -318,7 +318,7 @@ Let's first see how it looks like at collection time:
rootdir: /home/sweet/project
collected 2 items
<Dir parametrize.rst-204>
<Dir parametrize.rst-206>
<Module test_backends.py>
<Function test_db_initialized[d1]>
<Function test_db_initialized[d2]>
@ -503,11 +503,12 @@ Running it results in some skips if we don't have all the python interpreters in
.. code-block:: pytest
. $ pytest -rs -q multipython.py
ssssssssssss...ssssssssssss [100%]
sssssssssssssssssssssssssss [100%]
========================= short test summary info ==========================
SKIPPED [12] multipython.py:67: 'python3.9' not found
SKIPPED [12] multipython.py:67: 'python3.11' not found
3 passed, 24 skipped in 0.12s
SKIPPED [9] multipython.py:67: 'python3.9' not found
SKIPPED [9] multipython.py:67: 'python3.10' not found
SKIPPED [9] multipython.py:67: 'python3.11' not found
27 skipped in 0.12s
Parametrization of optional implementations/imports
---------------------------------------------------

4
doc/en/example/pythoncollection.rst
View File

@ -152,7 +152,7 @@ The test collection would look like this:
configfile: pytest.ini
collected 2 items
<Dir pythoncollection.rst-205>
<Dir pythoncollection.rst-207>
<Module check_myapp.py>
<Class CheckMyApp>
<Function simple_check>
@ -215,7 +215,7 @@ You can always peek at the collection tree without running tests like this:
configfile: pytest.ini
collected 3 items
<Dir pythoncollection.rst-205>
<Dir pythoncollection.rst-207>
<Dir CWD>
<Module pythoncollection.py>
<Function test_function>

18
doc/en/example/reportingdemo.rst
View File

@ -568,12 +568,12 @@ Here is a nice run of several failures and how ``pytest`` presents things:
E + where False = <built-in method startswith of str object at 0xdeadbeef0027>('456')
E + where <built-in method startswith of str object at 0xdeadbeef0027> = '123'.startswith
E + where '123' = <function TestMoreErrors.test_startswith_nested.<locals>.f at 0xdeadbeef0029>()
E + and '456' = <function TestMoreErrors.test_startswith_nested.<locals>.g at 0xdeadbeef002a>()
E + and '456' = <function TestMoreErrors.test_startswith_nested.<locals>.g at 0xdeadbeef0003>()
failure_demo.py:237: AssertionError
_____________________ TestMoreErrors.test_global_func ______________________
self = <failure_demo.TestMoreErrors object at 0xdeadbeef002b>
self = <failure_demo.TestMoreErrors object at 0xdeadbeef002a>
def test_global_func(self):
> assert isinstance(globf(42), float)
@ -584,18 +584,18 @@ Here is a nice run of several failures and how ``pytest`` presents things:
failure_demo.py:240: AssertionError
_______________________ TestMoreErrors.test_instance _______________________
self = <failure_demo.TestMoreErrors object at 0xdeadbeef002c>
self = <failure_demo.TestMoreErrors object at 0xdeadbeef002b>
def test_instance(self):
self.x = 6 * 7
> assert self.x != 42
E assert 42 != 42
E + where 42 = <failure_demo.TestMoreErrors object at 0xdeadbeef002c>.x
E + where 42 = <failure_demo.TestMoreErrors object at 0xdeadbeef002b>.x
failure_demo.py:244: AssertionError
_______________________ TestMoreErrors.test_compare ________________________
self = <failure_demo.TestMoreErrors object at 0xdeadbeef002d>
self = <failure_demo.TestMoreErrors object at 0xdeadbeef002c>
def test_compare(self):
> assert globf(10) < 5
@ -605,7 +605,7 @@ Here is a nice run of several failures and how ``pytest`` presents things:
failure_demo.py:247: AssertionError
_____________________ TestMoreErrors.test_try_finally ______________________
self = <failure_demo.TestMoreErrors object at 0xdeadbeef002e>
self = <failure_demo.TestMoreErrors object at 0xdeadbeef002d>
def test_try_finally(self):
x = 1
@ -616,7 +616,7 @@ Here is a nice run of several failures and how ``pytest`` presents things:
failure_demo.py:252: AssertionError
___________________ TestCustomAssertMsg.test_single_line ___________________
self = <failure_demo.TestCustomAssertMsg object at 0xdeadbeef002f>
self = <failure_demo.TestCustomAssertMsg object at 0xdeadbeef002e>
def test_single_line(self):
class A:
@ -631,7 +631,7 @@ Here is a nice run of several failures and how ``pytest`` presents things:
failure_demo.py:263: AssertionError
____________________ TestCustomAssertMsg.test_multiline ____________________
self = <failure_demo.TestCustomAssertMsg object at 0xdeadbeef0030>
self = <failure_demo.TestCustomAssertMsg object at 0xdeadbeef002f>
def test_multiline(self):
class A:
@ -650,7 +650,7 @@ Here is a nice run of several failures and how ``pytest`` presents things:
failure_demo.py:270: AssertionError
___________________ TestCustomAssertMsg.test_custom_repr ___________________
self = <failure_demo.TestCustomAssertMsg object at 0xdeadbeef0031>
self = <failure_demo.TestCustomAssertMsg object at 0xdeadbeef0030>
def test_custom_repr(self):
class JSON:

2
doc/en/example/simple.rst
View File

@ -164,7 +164,7 @@ Now we'll get feedback on a bad argument:
$ pytest -q --cmdopt=type3
ERROR: usage: pytest [options] [file_or_dir] [file_or_dir] [...]
pytest: error: argument --cmdopt: invalid choice: 'type3' (choose from 'type1', 'type2')
pytest: error: argument --cmdopt: invalid choice: 'type3' (choose from type1, type2)
If you need to provide more detailed error messages, you can use the

14
doc/en/explanation/flaky.rst
View File

@ -117,8 +117,11 @@ This is a limited list, please submit an issue or pull request to expand it!
* Gao, Zebao, Yalan Liang, Myra B. Cohen, Atif M. Memon, and Zhen Wang. "Making system user interactive tests repeatable: When and what should we control?." In *Software Engineering (ICSE), 2015 IEEE/ACM 37th IEEE International Conference on*, vol. 1, pp. 55-65. IEEE, 2015. `PDF <http://www.cs.umd.edu/~atif/pubs/gao-icse15.pdf>`__
* Palomba, Fabio, and Andy Zaidman. "Does refactoring of test smells induce fixing flaky tests?." In *Software Maintenance and Evolution (ICSME), 2017 IEEE International Conference on*, pp. 1-12. IEEE, 2017. `PDF in Google Drive <https://drive.google.com/file/d/10HdcCQiuQVgW3yYUJD-TSTq1NbYEprl0/view>`__
* Bell, Jonathan, Owolabi Legunsen, Michael Hilton, Lamyaa Eloussi, Tifany Yung, and Darko Marinov. "DeFlaker: Automatically detecting flaky tests." In *Proceedings of the 2018 International Conference on Software Engineering*. 2018. `PDF <https://www.jonbell.net/icse18-deflaker.pdf>`__
* Dutta, Saikat and Shi, August and Choudhary, Rutvik and Zhang, Zhekun and Jain, Aryaman and Misailovic, Sasa. "Detecting flaky tests in probabilistic and machine learning applications." In *Proceedings of the 29th ACM SIGSOFT International Symposium on Software Testing and Analysis (ISSTA)*, pp. 211-224. ACM, 2020. `PDF <https://www.cs.cornell.edu/~saikatd/papers/flash-issta20.pdf>`__
* Bell, Jonathan, Owolabi Legunsen, Michael Hilton, Lamyaa Eloussi, Tifany Yung, and Darko Marinov. "DeFlaker: Automatically detecting flaky tests." In *Proceedings of the 2018 International Conference on Software Engineering*. 2018. `PDF <https://www.jonbell.net/icse18-deflaker.pdf#section-Research>`__
* Dutta, Saikat and Shi, August and Choudhary, Rutvik and Zhang, Zhekun and Jain, Aryaman and Misailovic, Sasa. "Detecting flaky tests in probabilistic and machine learning applications." In *Proceedings of the 29th ACM SIGSOFT International Symposium on Software Testing and Analysis (ISSTA)*, pp. 211-224. ACM, 2020. `PDF <https://www.cs.cornell.edu/~saikatd/papers/flash-issta20.pdf>`__
* Habchi, Sarra and Haben, Guillaume and Sohn, Jeongju and Franci, Adriano and Papadakis, Mike and Cordy, Maxime and Le Traon, Yves. "What Made This Test Flake? Pinpointing Classes Responsible for Test Flakiness." In Proceedings of the 38th IEEE International Conference on Software Maintenance and Evolution (ICSME), IEEE, 2022. `PDF <https://arxiv.org/abs/2207.10143>`__
* Lamprou, Sokrates. "Non-deterministic tests and where to find them: Empirically investigating the relationship between flaky tests and test smells by examining test order dependency." Bachelor thesis, Department of Computer and Information Science, Linköping University, 2022. LIU-IDA/LITH-EX-G–19/056–SE. `PDF <https://www.diva-portal.org/smash/get/diva2:1713691/FULLTEXT01.pdf>`__
* Leinen, Fabian and Elsner, Daniel and Pretschner, Alexander and Stahlbauer, Andreas and Sailer, Michael and Jürgens, Elmar. "Cost of Flaky Tests in Continuous Integration: An Industrial Case Study." Technical University of Munich and CQSE GmbH, Munich, Germany, 2023. `PDF <https://mediatum.ub.tum.de/doc/1730194/1730194.pdf>`__
Resources
^^^^^^^^^
@ -137,5 +140,12 @@ Resources
* `Flaky Tests at Google and How We Mitigate Them <https://testing.googleblog.com/2016/05/flaky-tests-at-google-and-how-we.html>`_ by John Micco, 2016
* `Where do Google's flaky tests come from? <https://testing.googleblog.com/2017/04/where-do-our-flaky-tests-come-from.html>`_ by Jeff Listfield, 2017
* Dropbox:
* `Athena: Our automated build health management system <https://dropbox.tech/infrastructure/athena-our-automated-build-health-management-system>`_ by Utsav Shah, 2019
* `How To Manage Flaky Tests in your CI Workflows <https://mill-build.org/blog/4-flaky-tests.html>`_ by Li Haoyi, 2025
* Uber:
* `Handling Flaky Unit Tests in Java <https://www.uber.com/blog/handling-flaky-tests-java/>`_ by Uber Engineering, 2021
* `Flaky Tests Overhaul at Uber <https://www.uber.com/blog/flaky-tests-overhaul/>`_ by Uber Engineering, 2024
.. _pytest-xdist: https://github.com/pytest-dev/pytest-xdist

1
doc/en/explanation/index.rst
View File

@ -12,5 +12,6 @@ Explanation
fixtures
goodpractices
pythonpath
types
ci
flaky

89
doc/en/explanation/types.rst Normal file
View File

@ -0,0 +1,89 @@
.. _types:
Typing in pytest
================
.. note::
This page assumes the reader is familiar with Python's typing system and its advantages.
For more information, refer to `Python's Typing Documentation <https://docs.python.org/3/library/typing.html>`_.
Why type tests?
---------------
Typing tests provides significant advantages:
- **Readability:** Clearly defines expected inputs and outputs, improving readability, especially in complex or parameterized tests.
- **Refactoring:** This is the main benefit in typing tests, as it will greatly help with refactoring, letting the type checker point out the necessary changes in both production and tests, without needing to run the full test suite.
For production code, typing also helps catching some bugs that might not be caught by tests at all (regardless of coverage), for example:
.. code-block:: python
def get_caption(target: int, items: list[tuple[int, str]]) -> str:
for value, caption in items:
if value == target:
return caption
The type checker will correctly error out that the function might return `None`, however even a full coverage test suite might miss that case:
.. code-block:: python
def test_get_caption() -> None:
assert get_caption(10, [(1, "foo"), (10, "bar")]) == "bar"
Note the code above has 100% coverage, but the bug is not caught (of course the example is "obvious", but serves to illustrate the point).
Using typing in test suites
---------------------------
To type fixtures in pytest, just add normal types to the fixture functions -- there is nothing special that needs to be done just because of the `fixture` decorator.
.. code-block:: python
import pytest
@pytest.fixture
def sample_fixture() -> int:
return 38
In the same manner, the fixtures passed to test functions need be annotated with the fixture's return type:
.. code-block:: python
def test_sample_fixture(sample_fixture: int) -> None:
assert sample_fixture == 38
From the POV of the type checker, it does not matter that `sample_fixture` is actually a fixture managed by pytest, all it matters to it is that `sample_fixture` is a parameter of type `int`.
The same logic applies to :ref:`@pytest.mark.parametrize <@pytest.mark.parametrize>`:
.. code-block:: python
@pytest.mark.parametrize("input_value, expected_output", [(1, 2), (5, 6), (10, 11)])
def test_increment(input_value: int, expected_output: int) -> None:
assert input_value + 1 == expected_output
The same logic applies when typing fixture functions which receive other fixtures:
.. code-block:: python
@pytest.fixture
def mock_env_user(monkeypatch: pytest.MonkeyPatch) -> None:
monkeypatch.setenv("USER", "TestingUser")
Conclusion
----------
Incorporating typing into pytest tests enhances **clarity**, improves **debugging** and **maintenance**, and ensures **type safety**.
These practices lead to a **robust**, **readable**, and **easily maintainable** test suite that is better equipped to handle future changes with minimal risk of errors.

2
doc/en/getting-started.rst
View File

@ -22,7 +22,7 @@ Install ``pytest``
.. code-block:: bash
$ pytest --version
pytest 8.3.3
pytest 8.3.5
.. _`simpletest`:

10
doc/en/how-to/capture-stdout-stderr.rst
View File

@ -4,6 +4,12 @@
How to capture stdout/stderr output
=========================================================
Pytest intercepts stdout and stderr as configured by the ``--capture=``
command-line argument or by using fixtures. The ``--capture=`` flag configures
reporting, whereas the fixtures offer more granular control and allows
inspection of output during testing. The reports can be customized with the
`-r flag <../reference/reference.html#command-line-flags>`_.
Default stdout/stderr/stdin capturing behaviour
---------------------------------------------------------
@ -106,8 +112,8 @@ of the failing function and hide the other one:
Accessing captured output from a test function
---------------------------------------------------
The :fixture:`capsys`, :fixture:`capsysbinary`, :fixture:`capfd`, and :fixture:`capfdbinary` fixtures
allow access to ``stdout``/``stderr`` output created during test execution.
The :fixture:`capsys`, :fixture:`capteesys`, :fixture:`capsysbinary`, :fixture:`capfd`, and :fixture:`capfdbinary`
fixtures allow access to ``stdout``/``stderr`` output created during test execution.
Here is an example test function that performs some output related checks:

2
doc/en/how-to/fixtures.rst
View File

@ -1418,7 +1418,7 @@ Running the above tests results in the following test IDs being used:
rootdir: /home/sweet/project
collected 12 items
<Dir fixtures.rst-224>
<Dir fixtures.rst-227>
<Module test_anothersmtp.py>
<Function test_showhelo[smtp.gmail.com]>
<Function test_showhelo[mail.python.org]>

27
doc/en/how-to/plugins.rst
View File

@ -133,4 +133,29 @@ CI server), you can set ``PYTEST_ADDOPTS`` environment variable to
See :ref:`findpluginname` for how to obtain the name of a plugin.
.. _`builtin plugins`:
.. _`disable_plugin_autoload`:
Disabling plugins from autoloading
----------------------------------
If you want to disable plugins from loading automatically, instead of requiring you to
manually specify each plugin with ``-p`` or :envvar:`PYTEST_PLUGINS`, you can use ``--disable-plugin-autoload`` or :envvar:`PYTEST_DISABLE_PLUGIN_AUTOLOAD`.
.. code-block:: bash
export PYTEST_DISABLE_PLUGIN_AUTOLOAD=1
export PYTEST_PLUGINS=NAME,NAME2
pytest
.. code-block:: bash
pytest --disable-plugin-autoload -p NAME,NAME2
.. code-block:: ini
[pytest]
addopts = --disable-plugin-autoload -p NAME,NAME2
.. versionadded:: 8.4
The ``--disable-plugin-autoload`` command-line flag.

4
doc/en/reference/fixtures.rst
View File

@ -32,6 +32,10 @@ Built-in fixtures
:fixture:`capsys`
Capture, as text, output to ``sys.stdout`` and ``sys.stderr``.
:fixture:`capteesys`
Capture in the same manner as :fixture:`capsys`, but also pass text
through according to ``--capture=``.
:fixture:`capsysbinary`
Capture, as bytes, output to ``sys.stdout`` and ``sys.stderr``.

128
doc/en/reference/plugin_list.rst
View File

@ -27,14 +27,14 @@ please refer to `the update script <https://github.com/pytest-dev/pytest/blob/ma
creating a PDF, because otherwise the table gets far too wide for the
page.
This list contains 1587 plugins.
This list contains 1588 plugins.
.. only:: not latex
=============================================== ====================================================================================================================================================================================================================================================================================================================================================================================== ============== ===================== ================================================
name summary last_release status requires
=============================================== ====================================================================================================================================================================================================================================================================================================================================================================================== ============== ===================== ================================================
:pypi:`databricks-labs-pytester` Python Testing for Databricks Feb 03, 2025 4 - Beta pytest>=8.3
:pypi:`databricks-labs-pytester` Python Testing for Databricks Feb 27, 2025 4 - Beta pytest>=8.3
:pypi:`logassert` Simple but powerful assertion and verification of logged lines Jan 29, 2025 5 - Production/Stable pytest; extra == "dev"
:pypi:`logot` Test whether your code is logging correctly 🪵 Mar 23, 2024 5 - Production/Stable pytest<9,>=7; extra == "pytest"
:pypi:`nuts` Network Unit Testing System Jul 19, 2024 N/A pytest<8,>=7
@ -161,7 +161,7 @@ This list contains 1587 plugins.
:pypi:`pytest-bdd-wrappers` Feb 11, 2020 2 - Pre-Alpha N/A
:pypi:`pytest-beakerlib` A pytest plugin that reports test results to the BeakerLib framework Mar 17, 2017 5 - Production/Stable pytest
:pypi:`pytest-beartype` Pytest plugin to run your tests with beartype checking enabled. Oct 31, 2024 N/A pytest
:pypi:`pytest-bec-e2e` BEC pytest plugin for end-to-end tests Feb 17, 2025 3 - Alpha pytest
:pypi:`pytest-bec-e2e` BEC pytest plugin for end-to-end tests Feb 28, 2025 3 - Alpha pytest
:pypi:`pytest-beds` Fixtures for testing Google Appengine (GAE) apps Jun 07, 2016 4 - Beta N/A
:pypi:`pytest-beeprint` use icdiff for better error messages in pytest assertions Jul 04, 2023 4 - Beta N/A
:pypi:`pytest-bench` Benchmark utility that plugs into pytest. Jul 21, 2014 3 - Alpha N/A
@ -255,6 +255,7 @@ This list contains 1587 plugins.
:pypi:`pytest_cleanup` Automated, comprehensive and well-organised pytest test cases. Jan 28, 2020 N/A N/A
:pypi:`pytest-cleanuptotal` A cleanup plugin for pytest Nov 08, 2024 5 - Production/Stable N/A
:pypi:`pytest-clerk` A set of pytest fixtures to help with integration testing with Clerk. Jan 30, 2025 N/A pytest<9.0.0,>=8.0.0
:pypi:`pytest-cli2-ansible` Feb 28, 2025 N/A N/A
:pypi:`pytest-click` Pytest plugin for Click Feb 11, 2022 5 - Production/Stable pytest (>=5.0)
:pypi:`pytest-cli-fixtures` Automatically register fixtures for custom CLI arguments Jul 28, 2022 N/A pytest (~=7.0)
:pypi:`pytest-clld` Oct 23, 2024 N/A pytest>=3.9
@ -466,7 +467,7 @@ This list contains 1587 plugins.
:pypi:`pytest-easy-server` Pytest plugin for easy testing against servers May 01, 2021 4 - Beta pytest (<5.0.0,>=4.3.1) ; python_version < "3.5"
:pypi:`pytest-ebics-sandbox` A pytest plugin for testing against an EBICS sandbox server. Requires docker. Aug 15, 2022 N/A N/A
:pypi:`pytest-ec2` Pytest execution on EC2 instance Oct 22, 2019 3 - Alpha N/A
:pypi:`pytest-echo` pytest plugin with mechanisms for echoing environment variables, package version and generic attributes Feb 01, 2025 5 - Production/Stable pytest>=6.0
:pypi:`pytest-echo` pytest plugin with mechanisms for echoing environment variables, package version and generic attributes Feb 22, 2025 5 - Production/Stable pytest>=6.0
:pypi:`pytest-edit` Edit the source code of a failed test with \`pytest --edit\`. Nov 17, 2024 N/A pytest
:pypi:`pytest-ekstazi` Pytest plugin to select test using Ekstazi algorithm Sep 10, 2022 N/A pytest
:pypi:`pytest-elasticsearch` Elasticsearch fixtures and fixture factories for Pytest. Dec 03, 2024 5 - Production/Stable pytest>=7.0
@ -527,7 +528,7 @@ This list contains 1587 plugins.
:pypi:`pytest-exit-status` Enhance. Jan 25, 2025 N/A pytest>=8.0.0
:pypi:`pytest-expect` py.test plugin to store test expectations and mark tests based on them Apr 21, 2016 4 - Beta N/A
:pypi:`pytest-expectdir` A pytest plugin to provide initial/expected directories, and check a test transforms the initial directory to the expected one Mar 19, 2023 5 - Production/Stable pytest (>=5.0)
:pypi:`pytest-expected` Record and play back your expectations Jan 14, 2025 N/A pytest
:pypi:`pytest-expected` Record and play back your expectations Feb 26, 2025 N/A pytest
:pypi:`pytest-expecter` Better testing with expecter and pytest. Sep 18, 2022 5 - Production/Stable N/A
:pypi:`pytest-expectr` This plugin is used to expect multiple assert using pytest framework. Oct 05, 2018 N/A pytest (>=2.4.2)
:pypi:`pytest-expect-test` A fixture to support expect tests in pytest Apr 10, 2023 4 - Beta pytest (>=3.5.0)
@ -574,7 +575,7 @@ This list contains 1587 plugins.
:pypi:`pytest-filter-subpackage` Pytest plugin for filtering based on sub-packages Mar 04, 2024 5 - Production/Stable pytest >=4.6
:pypi:`pytest-find-dependencies` A pytest plugin to find dependencies between tests Mar 16, 2024 4 - Beta pytest >=4.3.0
:pypi:`pytest-finer-verdicts` A pytest plugin to treat non-assertion failures as test errors. Jun 18, 2020 N/A pytest (>=5.4.3)
:pypi:`pytest-firefox` pytest plugin to manipulate firefox Aug 08, 2017 3 - Alpha pytest (>=3.0.2)
:pypi:`pytest-firefox` Feb 28, 2025 N/A N/A
:pypi:`pytest-fixture-classes` Fixtures as classes that work well with dependency injection, autocompletetion, type checkers, and language servers Sep 02, 2023 5 - Production/Stable pytest
:pypi:`pytest-fixturecollection` A pytest plugin to collect tests based on fixtures being used by tests Feb 22, 2024 4 - Beta pytest >=3.5.0
:pypi:`pytest-fixture-config` Fixture configuration utils for py.test Oct 17, 2024 5 - Production/Stable pytest
@ -602,12 +603,11 @@ This list contains 1587 plugins.
:pypi:`pytest-flexreport` Apr 15, 2023 4 - Beta pytest
:pypi:`pytest-fluent` A pytest plugin in order to provide logs via fluentd Aug 14, 2024 4 - Beta pytest>=7.0.0
:pypi:`pytest-fluentbit` A pytest plugin in order to provide logs via fluentbit Jun 16, 2023 4 - Beta pytest (>=7.0.0)
:pypi:`pytest-fly` pytest observer Dec 03, 2024 3 - Alpha pytest
:pypi:`pytest-fly` pytest observer Feb 28, 2025 3 - Alpha pytest
:pypi:`pytest-flyte` Pytest fixtures for simplifying Flyte integration testing May 03, 2021 N/A pytest
:pypi:`pytest-focus` A pytest plugin that alerts user of failed test cases with screen notifications May 04, 2019 4 - Beta pytest
:pypi:`pytest-forbid` Mar 07, 2023 N/A pytest (>=7.2.2,<8.0.0)
:pypi:`pytest-forcefail` py.test plugin to make the test failing regardless of pytest.mark.xfail May 15, 2018 4 - Beta N/A
:pypi:`pytest-forks` Fork helper for pytest Mar 05, 2024 N/A N/A
:pypi:`pytest-forward-compatability` A name to avoid typosquating pytest-foward-compatibility Sep 06, 2020 N/A N/A
:pypi:`pytest-forward-compatibility` A pytest plugin to shim pytest commandline options for fowards compatibility Sep 29, 2020 N/A N/A
:pypi:`pytest-frappe` Pytest Frappe Plugin - A set of pytest fixtures to test Frappe applications Jul 30, 2024 4 - Beta pytest>=7.0.0
@ -619,7 +619,7 @@ This list contains 1587 plugins.
:pypi:`pytest-frozen-uuids` Deterministically frozen UUID's for your tests Apr 17, 2022 N/A pytest (>=3.0)
:pypi:`pytest-func-cov` Pytest plugin for measuring function coverage Apr 15, 2021 3 - Alpha pytest (>=5)
:pypi:`pytest-funparam` An alternative way to parametrize test cases. Dec 02, 2021 4 - Beta pytest >=4.6.0
:pypi:`pytest-fv` pytest extensions to support running functional-verification jobs Jan 31, 2025 N/A pytest
:pypi:`pytest-fv` pytest extensions to support running functional-verification jobs Feb 27, 2025 N/A pytest
:pypi:`pytest-fxa` pytest plugin for Firefox Accounts Aug 28, 2018 5 - Production/Stable N/A
:pypi:`pytest-fxa-mte` pytest plugin for Firefox Accounts Oct 02, 2024 5 - Production/Stable N/A
:pypi:`pytest-fxtest` Oct 27, 2020 N/A N/A
@ -678,7 +678,7 @@ This list contains 1587 plugins.
:pypi:`pytest-history` Pytest plugin to keep a history of your pytest runs Jan 14, 2024 N/A pytest (>=7.4.3,<8.0.0)
:pypi:`pytest-home` Home directory fixtures Jul 28, 2024 5 - Production/Stable pytest
:pypi:`pytest-homeassistant` A pytest plugin for use with homeassistant custom components. Aug 12, 2020 4 - Beta N/A
:pypi:`pytest-homeassistant-custom-component` Experimental package to automatically extract test plugins for Home Assistant custom components Feb 22, 2025 3 - Alpha pytest==8.3.4
:pypi:`pytest-homeassistant-custom-component` Experimental package to automatically extract test plugins for Home Assistant custom components Mar 01, 2025 3 - Alpha pytest==8.3.4
:pypi:`pytest-honey` A simple plugin to use with pytest Jan 07, 2022 4 - Beta pytest (>=3.5.0)
:pypi:`pytest-honors` Report on tests that honor constraints, and guard against regressions Mar 06, 2020 4 - Beta N/A
:pypi:`pytest-hot-reloading` Sep 23, 2024 N/A N/A
@ -701,7 +701,7 @@ This list contains 1587 plugins.
:pypi:`pytest-httpdbg` A pytest plugin to record HTTP(S) requests with stack trace. Feb 11, 2025 4 - Beta pytest>=7.0.0
:pypi:`pytest-http-mocker` Pytest plugin for http mocking (via https://github.com/vilus/mocker) Oct 20, 2019 N/A N/A
:pypi:`pytest-httpretty` A thin wrapper of HTTPretty for pytest Feb 16, 2014 3 - Alpha N/A
:pypi:`pytest_httpserver` pytest-httpserver is a httpserver for pytest Jan 21, 2025 3 - Alpha N/A
:pypi:`pytest_httpserver` pytest-httpserver is a httpserver for pytest Feb 24, 2025 3 - Alpha N/A
:pypi:`pytest-httptesting` http_testing framework on top of pytest Dec 19, 2024 N/A pytest>=8.2.0
:pypi:`pytest-httpx` Send responses to httpx. Nov 28, 2024 5 - Production/Stable pytest==8.*
:pypi:`pytest-httpx-blockage` Disable httpx requests during a test run Feb 16, 2023 N/A pytest (>=7.2.1)
@ -755,7 +755,7 @@ This list contains 1587 plugins.
:pypi:`pytest-ipynb2` Pytest plugin to run tests in Jupyter Notebooks Feb 19, 2025 N/A pytest
:pypi:`pytest-ipywidgets` Feb 18, 2025 N/A pytest
:pypi:`pytest-isolate` Run pytest tests in isolated subprocesses Jan 16, 2025 4 - Beta pytest
:pypi:`pytest-isolate-mpi` pytest-isolate-mpi allows for MPI-parallel tests being executed in a segfault and MPI_Abort safe manner Dec 05, 2024 4 - Beta pytest>=5
:pypi:`pytest-isolate-mpi` pytest-isolate-mpi allows for MPI-parallel tests being executed in a segfault and MPI_Abort safe manner Feb 24, 2025 4 - Beta pytest>=5
:pypi:`pytest-isort` py.test plugin to check import ordering using isort Mar 05, 2024 5 - Production/Stable pytest (>=5.0)
:pypi:`pytest-it` Pytest plugin to display test reports as a plaintext spec, inspired by Rspec: https://github.com/mattduck/pytest-it. Jan 29, 2024 4 - Beta N/A
:pypi:`pytest-item-dict` Get a hierarchical dict of session.items Nov 14, 2024 4 - Beta pytest>=8.3.0
@ -805,6 +805,7 @@ This list contains 1587 plugins.
:pypi:`pytest-lamp` Jan 06, 2017 3 - Alpha N/A
:pypi:`pytest-langchain` Pytest-style test runner for langchain agents Feb 26, 2023 N/A pytest
:pypi:`pytest-lark` Create fancy and clear HTML test reports. Nov 05, 2023 N/A N/A
:pypi:`pytest-latin-hypercube` Implementation of Latin Hypercube Sampling for pytest. Feb 27, 2025 N/A pytest
:pypi:`pytest-launchable` Launchable Pytest Plugin Apr 05, 2023 N/A pytest (>=4.2.0)
:pypi:`pytest-layab` Pytest fixtures for layab. Oct 05, 2020 5 - Production/Stable N/A
:pypi:`pytest-lazy-fixture` It helps to use fixtures in pytest.mark.parametrize Feb 01, 2020 4 - Beta pytest (>=3.2.5)
@ -841,7 +842,7 @@ This list contains 1587 plugins.
:pypi:`pytest-logging` Configures logging and allows tweaking the log level with a py.test flag Nov 04, 2015 4 - Beta N/A
:pypi:`pytest-logging-end-to-end-test-tool` Sep 23, 2022 N/A pytest (>=7.1.2,<8.0.0)
:pypi:`pytest-logging-strict` pytest fixture logging configured from packaged YAML Jan 19, 2025 3 - Alpha pytest
:pypi:`pytest-logikal` Common testing environment Jan 31, 2025 5 - Production/Stable pytest==8.3.4
:pypi:`pytest-logikal` Common testing environment Mar 02, 2025 5 - Production/Stable pytest==8.3.4
:pypi:`pytest-log-report` Package for creating a pytest test run reprot Dec 26, 2019 N/A N/A
:pypi:`pytest-logscanner` Pytest plugin for logscanner (A logger for python logging outputting to easily viewable (and filterable) html files. Good for people not grep savey, and color higlighting and quickly changing filters might even bye useful for commandline wizards.) Sep 30, 2024 4 - Beta pytest>=8.2.2
:pypi:`pytest-loguru` Pytest Loguru Mar 20, 2024 5 - Production/Stable pytest; extra == "test"
@ -908,7 +909,7 @@ This list contains 1587 plugins.
:pypi:`pytest-modifyjunit` Utility for adding additional properties to junit xml for IDM QE Jan 10, 2019 N/A N/A
:pypi:`pytest-molecule` PyTest Molecule Plugin :: discover and run molecule tests Mar 29, 2022 5 - Production/Stable pytest (>=7.0.0)
:pypi:`pytest-molecule-JC` PyTest Molecule Plugin :: discover and run molecule tests Jul 18, 2023 5 - Production/Stable pytest (>=7.0.0)
:pypi:`pytest-mongo` MongoDB process and client fixtures plugin for Pytest. Mar 13, 2024 5 - Production/Stable pytest >=6.2
:pypi:`pytest-mongo` MongoDB process and client fixtures plugin for Pytest. Feb 28, 2025 5 - Production/Stable pytest>=6.2
:pypi:`pytest-mongodb` pytest plugin for MongoDB fixtures May 16, 2023 5 - Production/Stable N/A
:pypi:`pytest-mongodb-nono` pytest plugin for MongoDB Jan 07, 2025 N/A N/A
:pypi:`pytest-mongodb-ry` pytest plugin for MongoDB Jan 21, 2025 N/A N/A
@ -958,7 +959,7 @@ This list contains 1587 plugins.
:pypi:`pytest-nocustom` Run all tests without custom markers Aug 05, 2024 5 - Production/Stable N/A
:pypi:`pytest-node-dependency` pytest plugin for controlling execution flow Apr 10, 2024 5 - Production/Stable N/A
:pypi:`pytest-nodev` Test-driven source code search for Python. Jul 21, 2016 4 - Beta pytest (>=2.8.1)
:pypi:`pytest-nogarbage` Ensure a test produces no garbage Aug 29, 2021 5 - Production/Stable pytest (>=4.6.0)
:pypi:`pytest-nogarbage` Ensure a test produces no garbage Feb 24, 2025 5 - Production/Stable pytest>=4.6.0
:pypi:`pytest-nose-attrib` pytest plugin to use nose @attrib marks decorators and pick tests based on attributes and partially uses nose-attrib plugin approach Aug 13, 2023 N/A N/A
:pypi:`pytest_notebook` A pytest plugin for testing Jupyter Notebooks. Nov 28, 2023 4 - Beta pytest>=3.5.0
:pypi:`pytest-notice` Send pytest execution result email Nov 05, 2020 N/A N/A
@ -1076,14 +1077,14 @@ This list contains 1587 plugins.
:pypi:`pytest-porcochu` Show surprise when tests are passing Nov 28, 2024 5 - Production/Stable N/A
:pypi:`pytest-portion` Select a portion of the collected tests Jan 28, 2021 4 - Beta pytest (>=3.5.0)
:pypi:`pytest-postgres` Run PostgreSQL in Docker container in Pytest. Mar 22, 2020 N/A pytest
:pypi:`pytest-postgresql` Postgresql fixtures and fixture factories for Pytest. Sep 05, 2024 5 - Production/Stable pytest>=6.2
:pypi:`pytest-postgresql` Postgresql fixtures and fixture factories for Pytest. Feb 23, 2025 5 - Production/Stable pytest>=6.2
:pypi:`pytest-power` pytest plugin with powerful fixtures Dec 31, 2020 N/A pytest (>=5.4)
:pypi:`pytest-powerpack` A plugin containing extra batteries for pytest Jan 04, 2025 N/A pytest<9.0.0,>=8.1.1
:pypi:`pytest-prefer-nested-dup-tests` A Pytest plugin to drop duplicated tests during collection, but will prefer keeping nested packages. Apr 27, 2022 4 - Beta pytest (>=7.1.1,<8.0.0)
:pypi:`pytest-pretty` pytest plugin for printing summary data as I want it Apr 05, 2023 5 - Production/Stable pytest>=7
:pypi:`pytest-pretty-terminal` pytest plugin for generating prettier terminal output Jan 31, 2022 N/A pytest (>=3.4.1)
:pypi:`pytest-pride` Minitest-style test colors Apr 02, 2016 3 - Alpha N/A
:pypi:`pytest-print` pytest-print adds the printer fixture you can use to print messages to the user (directly to the pytest runner, not stdout) Sep 17, 2024 5 - Production/Stable pytest>=8.3.2
:pypi:`pytest-print` pytest-print adds the printer fixture you can use to print messages to the user (directly to the pytest runner, not stdout) Feb 25, 2025 5 - Production/Stable pytest>=8.3.2
:pypi:`pytest-priority` pytest plugin for add priority for tests Aug 19, 2024 N/A pytest
:pypi:`pytest-proceed` Oct 01, 2024 N/A pytest
:pypi:`pytest-profiles` pytest plugin for configuration profiles Dec 09, 2021 4 - Beta pytest (>=3.7.0)
@ -1192,7 +1193,7 @@ This list contains 1587 plugins.
:pypi:`pytest-reportlog` Replacement for the --resultlog option, focused in simplicity and extensibility May 22, 2023 3 - Alpha pytest
:pypi:`pytest-report-me` A pytest plugin to generate report. Dec 31, 2020 N/A pytest
:pypi:`pytest-report-parameters` pytest plugin for adding tests' parameters to junit report Jun 18, 2020 3 - Alpha pytest (>=2.4.2)
:pypi:`pytest-reportportal` Agent for Reporting results of tests to the Report Portal Dec 04, 2024 N/A pytest>=3.8.0
:pypi:`pytest-reportportal` Agent for Reporting results of tests to the Report Portal Feb 28, 2025 N/A pytest>=4.6.10
:pypi:`pytest-report-stream` A pytest plugin which allows to stream test reports at runtime Oct 22, 2023 4 - Beta N/A
:pypi:`pytest-repo-structure` Pytest Repo Structure Mar 18, 2024 1 - Planning N/A
:pypi:`pytest-req` pytest requests plugin Aug 31, 2024 5 - Production/Stable pytest<9.0.0,>=8.3.2
@ -1200,7 +1201,7 @@ This list contains 1587 plugins.
:pypi:`pytest-requests` A simple plugin to use with pytest Jun 24, 2019 4 - Beta pytest (>=3.5.0)
:pypi:`pytest-requestselapsed` collect and show http requests elapsed time Aug 14, 2022 N/A N/A
:pypi:`pytest-requests-futures` Pytest Plugin to Mock Requests Futures Jul 06, 2022 5 - Production/Stable pytest
:pypi:`pytest-requirements` pytest plugin for using custom markers to relate tests to requirements and usecases Nov 29, 2024 N/A pytest
:pypi:`pytest-requirements` pytest plugin for using custom markers to relate tests to requirements and usecases Feb 28, 2025 N/A pytest
:pypi:`pytest-requires` A pytest plugin to elegantly skip tests with optional requirements Dec 21, 2021 4 - Beta pytest (>=3.5.0)
:pypi:`pytest-reraise` Make multi-threaded pytest test cases fail when they should Sep 20, 2022 5 - Production/Stable pytest (>=4.6)
:pypi:`pytest-rerun` Re-run only changed files in specified branch Jul 08, 2019 N/A pytest (>=3.6)
@ -1209,7 +1210,7 @@ This list contains 1587 plugins.
:pypi:`pytest-rerunfailures` pytest plugin to re-run tests to eliminate flaky failures Nov 20, 2024 5 - Production/Stable pytest!=8.2.2,>=7.4
:pypi:`pytest-rerunfailures-all-logs` pytest plugin to re-run tests to eliminate flaky failures Mar 07, 2022 5 - Production/Stable N/A
:pypi:`pytest-reserial` Pytest fixture for recording and replaying serial port traffic. Dec 22, 2024 4 - Beta pytest
:pypi:`pytest-resilient-circuits` Resilient Circuits fixtures for PyTest Dec 16, 2024 N/A pytest~=7.0
:pypi:`pytest-resilient-circuits` Resilient Circuits fixtures for PyTest Feb 28, 2025 N/A pytest~=7.0
:pypi:`pytest-resource` Load resource fixture plugin to use with pytest Nov 14, 2018 4 - Beta N/A
:pypi:`pytest-resource-path` Provides path for uniform access to test resources in isolated directory May 01, 2021 5 - Production/Stable pytest (>=3.5.0)
:pypi:`pytest-resource-usage` Pytest plugin for reporting running time and peak memory usage Nov 06, 2022 5 - Production/Stable pytest>=7.0.0
@ -1225,7 +1226,7 @@ This list contains 1587 plugins.
:pypi:`pytest-retry` Adds the ability to retry flaky tests in CI environments Jan 19, 2025 N/A pytest>=7.0.0
:pypi:`pytest-retry-class` A pytest plugin to rerun entire class on failure Nov 24, 2024 N/A pytest>=5.3
:pypi:`pytest-reusable-testcases` Apr 28, 2023 N/A N/A
:pypi:`pytest-revealtype-injector` Pytest plugin for replacing reveal_type() calls inside test functions with static and runtime type checking result comparison, for confirming type annotation validity. Dec 19, 2024 4 - Beta pytest>=7.0
:pypi:`pytest-revealtype-injector` Pytest plugin for replacing reveal_type() calls inside test functions with static and runtime type checking result comparison, for confirming type annotation validity. Feb 27, 2025 4 - Beta pytest<9,>=7.0
:pypi:`pytest-reverse` Pytest plugin to reverse test order. Oct 25, 2024 5 - Production/Stable pytest
:pypi:`pytest-rich` Leverage rich for richer test session output Dec 12, 2024 4 - Beta pytest>=7.0
:pypi:`pytest-richer` Pytest plugin providing a Rich based reporter. Oct 27, 2023 3 - Alpha pytest
@ -1261,7 +1262,7 @@ This list contains 1587 plugins.
:pypi:`pytest-sanity` Dec 07, 2020 N/A N/A
:pypi:`pytest-sa-pg` May 14, 2019 N/A N/A
:pypi:`pytest_sauce` pytest_sauce provides sane and helpful methods worked out in clearcode to run py.test tests with selenium/saucelabs Jul 14, 2014 3 - Alpha N/A
:pypi:`pytest-sbase` A complete web automation framework for end-to-end testing. Feb 20, 2025 5 - Production/Stable N/A
:pypi:`pytest-sbase` A complete web automation framework for end-to-end testing. Feb 26, 2025 5 - Production/Stable N/A
:pypi:`pytest-scenario` pytest plugin for test scenarios Feb 06, 2017 3 - Alpha N/A
:pypi:`pytest-scenario-files` A pytest plugin that generates unit test scenarios from data files. Nov 21, 2024 5 - Production/Stable pytest>=7.0
:pypi:`pytest-schedule` Automate and customize test scheduling effortlessly on local machines. Oct 31, 2024 N/A N/A
@ -1273,12 +1274,12 @@ This list contains 1587 plugins.
:pypi:`pytest-select` A pytest plugin which allows to (de-)select tests from a file. Jan 18, 2019 3 - Alpha pytest (>=3.0)
:pypi:`pytest-selenium` pytest plugin for Selenium Feb 01, 2024 5 - Production/Stable pytest>=6.0.0
:pypi:`pytest-selenium-auto` pytest plugin to automatically capture screenshots upon selenium webdriver events Nov 07, 2023 N/A pytest >= 7.0.0
:pypi:`pytest-seleniumbase` A complete web automation framework for end-to-end testing. Feb 20, 2025 5 - Production/Stable N/A
:pypi:`pytest-seleniumbase` A complete web automation framework for end-to-end testing. Feb 26, 2025 5 - Production/Stable N/A
:pypi:`pytest-selenium-enhancer` pytest plugin for Selenium Apr 29, 2022 5 - Production/Stable N/A
:pypi:`pytest-selenium-pdiff` A pytest package implementing perceptualdiff for Selenium tests. Apr 06, 2017 2 - Pre-Alpha N/A
:pypi:`pytest-selfie` A pytest plugin for selfie snapshot testing. Dec 16, 2024 N/A pytest>=8.0.0
:pypi:`pytest-send-email` Send pytest execution result email Sep 02, 2024 N/A pytest
:pypi:`pytest-sentry` A pytest plugin to send testrun information to Sentry.io Nov 25, 2024 N/A pytest
:pypi:`pytest-sentry` A pytest plugin to send testrun information to Sentry.io Feb 24, 2025 N/A pytest
:pypi:`pytest-sequence-markers` Pytest plugin for sequencing markers for execution of tests May 23, 2023 5 - Production/Stable N/A
:pypi:`pytest-server` test server exec cmd Sep 09, 2024 N/A N/A
:pypi:`pytest-server-fixtures` Extensible server fixtures for py.test Nov 29, 2024 5 - Production/Stable pytest
@ -1627,7 +1628,7 @@ This list contains 1587 plugins.
:pypi:`databricks-labs-pytester`
*last release*: Feb 03, 2025,
*last release*: Feb 27, 2025,
*status*: 4 - Beta,
*requires*: pytest>=8.3
@ -2516,7 +2517,7 @@ This list contains 1587 plugins.
Pytest plugin to run your tests with beartype checking enabled.
:pypi:`pytest-bec-e2e`
*last release*: Feb 17, 2025,
*last release*: Feb 28, 2025,
*status*: 3 - Alpha,
*requires*: pytest
@ -3173,6 +3174,13 @@ This list contains 1587 plugins.
A set of pytest fixtures to help with integration testing with Clerk.
:pypi:`pytest-cli2-ansible`
*last release*: Feb 28, 2025,
*status*: N/A,
*requires*: N/A
:pypi:`pytest-click`
*last release*: Feb 11, 2022,
*status*: 5 - Production/Stable,
@ -4651,7 +4659,7 @@ This list contains 1587 plugins.
Pytest execution on EC2 instance
:pypi:`pytest-echo`
*last release*: Feb 01, 2025,
*last release*: Feb 22, 2025,
*status*: 5 - Production/Stable,
*requires*: pytest>=6.0
@ -5078,7 +5086,7 @@ This list contains 1587 plugins.
A pytest plugin to provide initial/expected directories, and check a test transforms the initial directory to the expected one
:pypi:`pytest-expected`
*last release*: Jan 14, 2025,
*last release*: Feb 26, 2025,
*status*: N/A,
*requires*: pytest
@ -5407,11 +5415,11 @@ This list contains 1587 plugins.
A pytest plugin to treat non-assertion failures as test errors.
:pypi:`pytest-firefox`
*last release*: Aug 08, 2017,
*status*: 3 - Alpha,
*requires*: pytest (>=3.0.2)
*last release*: Feb 28, 2025,
*status*: N/A,
*requires*: N/A
pytest plugin to manipulate firefox
:pypi:`pytest-fixture-classes`
*last release*: Sep 02, 2023,
@ -5603,7 +5611,7 @@ This list contains 1587 plugins.
A pytest plugin in order to provide logs via fluentbit
:pypi:`pytest-fly`
*last release*: Dec 03, 2024,
*last release*: Feb 28, 2025,
*status*: 3 - Alpha,
*requires*: pytest
@ -5637,13 +5645,6 @@ This list contains 1587 plugins.
py.test plugin to make the test failing regardless of pytest.mark.xfail
:pypi:`pytest-forks`
*last release*: Mar 05, 2024,
*status*: N/A,
*requires*: N/A
Fork helper for pytest
:pypi:`pytest-forward-compatability`
*last release*: Sep 06, 2020,
*status*: N/A,
@ -5722,7 +5723,7 @@ This list contains 1587 plugins.
An alternative way to parametrize test cases.
:pypi:`pytest-fv`
*last release*: Jan 31, 2025,
*last release*: Feb 27, 2025,
*status*: N/A,
*requires*: pytest
@ -6135,7 +6136,7 @@ This list contains 1587 plugins.
A pytest plugin for use with homeassistant custom components.
:pypi:`pytest-homeassistant-custom-component`
*last release*: Feb 22, 2025,
*last release*: Mar 01, 2025,
*status*: 3 - Alpha,
*requires*: pytest==8.3.4
@ -6296,7 +6297,7 @@ This list contains 1587 plugins.
A thin wrapper of HTTPretty for pytest
:pypi:`pytest_httpserver`
*last release*: Jan 21, 2025,
*last release*: Feb 24, 2025,
*status*: 3 - Alpha,
*requires*: N/A
@ -6674,7 +6675,7 @@ This list contains 1587 plugins.
Run pytest tests in isolated subprocesses
:pypi:`pytest-isolate-mpi`
*last release*: Dec 05, 2024,
*last release*: Feb 24, 2025,
*status*: 4 - Beta,
*requires*: pytest>=5
@ -7023,6 +7024,13 @@ This list contains 1587 plugins.
Create fancy and clear HTML test reports.
:pypi:`pytest-latin-hypercube`
*last release*: Feb 27, 2025,
*status*: N/A,
*requires*: pytest
Implementation of Latin Hypercube Sampling for pytest.
:pypi:`pytest-launchable`
*last release*: Apr 05, 2023,
*status*: N/A,
@ -7276,7 +7284,7 @@ This list contains 1587 plugins.
pytest fixture logging configured from packaged YAML
:pypi:`pytest-logikal`
*last release*: Jan 31, 2025,
*last release*: Mar 02, 2025,
*status*: 5 - Production/Stable,
*requires*: pytest==8.3.4
@ -7745,9 +7753,9 @@ This list contains 1587 plugins.
PyTest Molecule Plugin :: discover and run molecule tests
:pypi:`pytest-mongo`
*last release*: Mar 13, 2024,
*last release*: Feb 28, 2025,
*status*: 5 - Production/Stable,
*requires*: pytest >=6.2
*requires*: pytest>=6.2
MongoDB process and client fixtures plugin for Pytest.
@ -8095,9 +8103,9 @@ This list contains 1587 plugins.
Test-driven source code search for Python.
:pypi:`pytest-nogarbage`
*last release*: Aug 29, 2021,
*last release*: Feb 24, 2025,
*status*: 5 - Production/Stable,
*requires*: pytest (>=4.6.0)
*requires*: pytest>=4.6.0
Ensure a test produces no garbage
@ -8921,7 +8929,7 @@ This list contains 1587 plugins.
Run PostgreSQL in Docker container in Pytest.
:pypi:`pytest-postgresql`
*last release*: Sep 05, 2024,
*last release*: Feb 23, 2025,
*status*: 5 - Production/Stable,
*requires*: pytest>=6.2
@ -8970,7 +8978,7 @@ This list contains 1587 plugins.
Minitest-style test colors
:pypi:`pytest-print`
*last release*: Sep 17, 2024,
*last release*: Feb 25, 2025,
*status*: 5 - Production/Stable,
*requires*: pytest>=8.3.2
@ -9733,9 +9741,9 @@ This list contains 1587 plugins.
pytest plugin for adding tests' parameters to junit report
:pypi:`pytest-reportportal`
*last release*: Dec 04, 2024,
*last release*: Feb 28, 2025,
*status*: N/A,
*requires*: pytest>=3.8.0
*requires*: pytest>=4.6.10
Agent for Reporting results of tests to the Report Portal
@ -9789,7 +9797,7 @@ This list contains 1587 plugins.
Pytest Plugin to Mock Requests Futures
:pypi:`pytest-requirements`
*last release*: Nov 29, 2024,
*last release*: Feb 28, 2025,
*status*: N/A,
*requires*: pytest
@ -9852,7 +9860,7 @@ This list contains 1587 plugins.
Pytest fixture for recording and replaying serial port traffic.
:pypi:`pytest-resilient-circuits`
*last release*: Dec 16, 2024,
*last release*: Feb 28, 2025,
*status*: N/A,
*requires*: pytest~=7.0
@ -9964,9 +9972,9 @@ This list contains 1587 plugins.
:pypi:`pytest-revealtype-injector`
*last release*: Dec 19, 2024,
*last release*: Feb 27, 2025,
*status*: 4 - Beta,
*requires*: pytest>=7.0
*requires*: pytest<9,>=7.0
Pytest plugin for replacing reveal_type() calls inside test functions with static and runtime type checking result comparison, for confirming type annotation validity.
@ -10216,7 +10224,7 @@ This list contains 1587 plugins.
pytest_sauce provides sane and helpful methods worked out in clearcode to run py.test tests with selenium/saucelabs
:pypi:`pytest-sbase`
*last release*: Feb 20, 2025,
*last release*: Feb 26, 2025,
*status*: 5 - Production/Stable,
*requires*: N/A
@ -10300,7 +10308,7 @@ This list contains 1587 plugins.
pytest plugin to automatically capture screenshots upon selenium webdriver events
:pypi:`pytest-seleniumbase`
*last release*: Feb 20, 2025,
*last release*: Feb 26, 2025,
*status*: 5 - Production/Stable,
*requires*: N/A
@ -10335,7 +10343,7 @@ This list contains 1587 plugins.
Send pytest execution result email
:pypi:`pytest-sentry`
*last release*: Nov 25, 2024,
*last release*: Feb 24, 2025,
*status*: N/A,
*requires*: pytest

41
doc/en/reference/reference.rst
View File

@ -20,6 +20,16 @@ The current pytest version, as a string::
>>> pytest.__version__
'7.0.0'
.. _`hidden-param`:
pytest.HIDDEN_PARAM
~~~~~~~~~~~~~~~~~~~
.. versionadded:: 8.4
Can be passed to ``ids`` of :py:func:`Metafunc.parametrize <pytest.Metafunc.parametrize>`
or to ``id`` of :func:`pytest.param` to hide a parameter set from the test name.
Can only be used at most 1 time, as test names need to be unique.
.. _`version-tuple`:
@ -402,6 +412,16 @@ capsys
.. autoclass:: pytest.CaptureFixture()
:members:
.. fixture:: capteesys
capteesys
~~~~~~~~~
**Tutorial**: :ref:`captures`
.. autofunction:: _pytest.capture.capteesys()
:no-auto-options:
.. fixture:: capsysbinary
capsysbinary
@ -1178,8 +1198,9 @@ as discussed in :ref:`temporary directory location and retention`.
.. envvar:: PYTEST_DISABLE_PLUGIN_AUTOLOAD
When set, disables plugin auto-loading through :std:doc:`entry point packaging
metadata <packaging:guides/creating-and-discovering-plugins>`. Only explicitly
specified plugins will be loaded.
metadata <packaging:guides/creating-and-discovering-plugins>`. Only plugins
explicitly specified in :envvar:`PYTEST_PLUGINS` or with ``-p`` will be loaded.
See also :ref:`--disable-plugin-autoload <disable_plugin_autoload>`.
.. envvar:: PYTEST_PLUGINS
@ -1189,6 +1210,8 @@ Contains comma-separated list of modules that should be loaded as plugins:
export PYTEST_PLUGINS=mymodule.plugin,xdist
See also ``-p``.
.. envvar:: PYTEST_THEME
Sets a `pygment style <https://pygments.org/docs/styles/>`_ to use for the code output.
@ -2096,7 +2119,7 @@ All the command-line flags can be obtained by running ``pytest --help``::
Show cache contents, don't perform collection or
tests. Optional argument: glob (default: '*').
--cache-clear Remove all cache contents at start of test run
--lfnf={all,none}, --last-failed-no-failures={all,none}
--lfnf, --last-failed-no-failures={all,none}
With ``--lf``, determines whether to execute tests
when there are no previously (known) failures or
when no cached ``lastfailed`` data was found.
@ -2142,11 +2165,13 @@ All the command-line flags can be obtained by running ``pytest --help``::
Whether code should be highlighted (only if --color
is also enabled). Default: yes.
--pastebin=mode Send failed|all info to bpaste.net pastebin service
--junit-xml=path Create junit-xml style report file at given path
--junit-prefix=str Prepend prefix to classnames in junit-xml output
--junitxml, --junit-xml=path
Create junit-xml style report file at given path
--junitprefix, --junit-prefix=str
Prepend prefix to classnames in junit-xml output
pytest-warnings:
-W PYTHONWARNINGS, --pythonwarnings=PYTHONWARNINGS
-W, --pythonwarnings PYTHONWARNINGS
Set which warnings to report, see -W option of
Python itself
--maxfail=num Exit after first num failures or errors
@ -2155,7 +2180,7 @@ All the command-line flags can be obtained by running ``pytest --help``::
--strict-markers Markers not registered in the `markers` section of
the configuration file raise errors
--strict (Deprecated) alias to --strict-markers
-c FILE, --config-file=FILE
-c, --config-file FILE
Load configuration from `FILE` instead of trying to
locate one of the implicit configuration files.
--continue-on-collection-errors
@ -2209,7 +2234,7 @@ All the command-line flags can be obtained by running ``pytest --help``::
Store internal tracing debug information in this log
file. This file is opened with 'w' and truncated as
a result, care advised. Default: pytestdebug.log.
-o OVERRIDE_INI, --override-ini=OVERRIDE_INI
-o, --override-ini OVERRIDE_INI
Override ini option with "option=value" style, e.g.
`-o xfail_strict=True -o cache_dir=cache`.
--assert=MODE Control assertion debugging tools.

41
src/_pytest/capture.py
View File

@ -922,11 +922,13 @@ class CaptureFixture(Generic[AnyStr]):
captureclass: type[CaptureBase[AnyStr]],
request: SubRequest,
*,
config: dict[str, Any] | None = None,
_ispytest: bool = False,
) -> None:
check_ispytest(_ispytest)
self.captureclass: type[CaptureBase[AnyStr]] = captureclass
self.request = request
self._config = config if config else {}
self._capture: MultiCapture[AnyStr] | None = None
self._captured_out: AnyStr = self.captureclass.EMPTY_BUFFER
self._captured_err: AnyStr = self.captureclass.EMPTY_BUFFER
@ -935,8 +937,8 @@ class CaptureFixture(Generic[AnyStr]):
if self._capture is None:
self._capture = MultiCapture(
in_=None,
out=self.captureclass(1),
err=self.captureclass(2),
out=self.captureclass(1, **self._config),
err=self.captureclass(2, **self._config),
)
self._capture.start_capturing()
@ -1022,6 +1024,41 @@ def capsys(request: SubRequest) -> Generator[CaptureFixture[str]]:
capman.unset_fixture()
@fixture
def capteesys(request: SubRequest) -> Generator[CaptureFixture[str]]:
r"""Enable simultaneous text capturing and pass-through of writes
to ``sys.stdout`` and ``sys.stderr`` as defined by ``--capture=``.
The captured output is made available via ``capteesys.readouterr()`` method
calls, which return a ``(out, err)`` namedtuple.
``out`` and ``err`` will be ``text`` objects.
The output is also passed-through, allowing it to be "live-printed",
reported, or both as defined by ``--capture=``.
Returns an instance of :class:`CaptureFixture[str] <pytest.CaptureFixture>`.
Example:
.. code-block:: python
def test_output(capsys):
print("hello")
captured = capteesys.readouterr()
assert captured.out == "hello\n"
"""
capman: CaptureManager = request.config.pluginmanager.getplugin("capturemanager")
capture_fixture = CaptureFixture(
SysCapture, request, config=dict(tee=True), _ispytest=True
)
capman.set_fixture(capture_fixture)
capture_fixture._start()
yield capture_fixture
capture_fixture.close()
capman.unset_fixture()
@fixture
def capsysbinary(request: SubRequest) -> Generator[CaptureFixture[bytes]]:
r"""Enable bytes capturing of writes to ``sys.stdout`` and ``sys.stderr``.

26
src/_pytest/config/__init__.py
View File

@ -70,6 +70,7 @@ from _pytest.warning_types import warn_explicit_for
if TYPE_CHECKING:
from _pytest.assertions.rewrite import AssertionRewritingHook
from _pytest.cacheprovider import Cache
from _pytest.terminal import TerminalReporter
@ -1271,6 +1272,10 @@ class Config:
"""
ns, unknown_args = self._parser.parse_known_and_unknown_args(args)
mode = getattr(ns, "assertmode", "plain")
disable_autoload = getattr(ns, "disable_plugin_autoload", False) or bool(
os.environ.get("PYTEST_DISABLE_PLUGIN_AUTOLOAD")
)
if mode == "rewrite":
import _pytest.assertion
@ -1279,16 +1284,18 @@ class Config:
except SystemError:
mode = "plain"
else:
self._mark_plugins_for_rewrite(hook)
self._mark_plugins_for_rewrite(hook, disable_autoload)
self._warn_about_missing_assertion(mode)
def _mark_plugins_for_rewrite(self, hook) -> None:
def _mark_plugins_for_rewrite(
self, hook: AssertionRewritingHook, disable_autoload: bool
) -> None:
"""Given an importhook, mark for rewrite any top-level
modules or packages in the distribution package for
all pytest plugins."""
self.pluginmanager.rewrite_hook = hook
if os.environ.get("PYTEST_DISABLE_PLUGIN_AUTOLOAD"):
if disable_autoload:
# We don't autoload from distribution package entry points,
# no need to continue.
return
@ -1393,10 +1400,15 @@ class Config:
self._consider_importhook(args)
self._configure_python_path()
self.pluginmanager.consider_preparse(args, exclude_only=False)
if not os.environ.get("PYTEST_DISABLE_PLUGIN_AUTOLOAD"):
# Don't autoload from distribution package entry point. Only
# explicitly specified plugins are going to be loaded.
if (
not os.environ.get("PYTEST_DISABLE_PLUGIN_AUTOLOAD")
and not self.known_args_namespace.disable_plugin_autoload
):
# Autoloading from distribution package entry point has
# not been disabled.
self.pluginmanager.load_setuptools_entrypoints("pytest11")
# Otherwise only plugins explicitly specified in PYTEST_PLUGINS
# are going to be loaded.
self.pluginmanager.consider_env()
self.known_args_namespace = self._parser.parse_known_args(
@ -1419,7 +1431,7 @@ class Config:
except ConftestImportFailure as e:
if self.known_args_namespace.help or self.known_args_namespace.version:
# we don't want to prevent --help/--version to work
# so just let is pass and print a warning at the end
# so just let it pass and print a warning at the end
self.issue_config_time_warning(
PytestConfigWarning(f"could not load initial conftests: {e.path}"),
stacklevel=2,

20
src/_pytest/debugging.py
View File

@ -159,6 +159,9 @@ class pytestPDB:
cls._recursive_debug -= 1
return ret
if hasattr(pdb_cls, "do_debug"):
do_debug.__doc__ = pdb_cls.do_debug.__doc__
def do_continue(self, arg):
ret = super().do_continue(arg)
if cls._recursive_debug == 0:
@ -185,15 +188,17 @@ class pytestPDB:
self._continued = True
return ret
if hasattr(pdb_cls, "do_continue"):
do_continue.__doc__ = pdb_cls.do_continue.__doc__
do_c = do_cont = do_continue
def do_quit(self, arg):
"""Raise Exit outcome when quit command is used in pdb.
This is a bit of a hack - it would be better if BdbQuit
could be handled, but this would require to wrap the
whole pytest run, and adjust the report etc.
"""
# Raise Exit outcome when quit command is used in pdb.
#
# This is a bit of a hack - it would be better if BdbQuit
# could be handled, but this would require to wrap the
# whole pytest run, and adjust the report etc.
ret = super().do_quit(arg)
if cls._recursive_debug == 0:
@ -201,6 +206,9 @@ class pytestPDB:
return ret
if hasattr(pdb_cls, "do_quit"):
do_quit.__doc__ = pdb_cls.do_quit.__doc__
do_q = do_quit
do_exit = do_quit

9
src/_pytest/helpconfig.py
View File

@ -70,7 +70,14 @@ def pytest_addoption(parser: Parser) -> None:
metavar="name",
help="Early-load given plugin module name or entry point (multi-allowed). "
"To avoid loading of plugins, use the `no:` prefix, e.g. "
"`no:doctest`.",
"`no:doctest`. See also --disable-plugin-autoload.",
)
group.addoption(
"--disable-plugin-autoload",
action="store_true",
default=False,
help="Disable plugin auto-loading through entry point packaging metadata. "
"Only plugins explicitly specified in -p or env var PYTEST_PLUGINS will be loaded.",
)
group.addoption(
"--traceconfig",

14
src/_pytest/mark/__init__.py
View File

@ -12,8 +12,10 @@ from typing import TYPE_CHECKING
from .expression import Expression
from .expression import ParseError
from .structures import _HiddenParam
from .structures import EMPTY_PARAMETERSET_OPTION
from .structures import get_empty_parameterset_mark
from .structures import HIDDEN_PARAM
from .structures import Mark
from .structures import MARK_GEN
from .structures import MarkDecorator
@ -33,6 +35,7 @@ if TYPE_CHECKING:
__all__ = [
"HIDDEN_PARAM",
"MARK_GEN",
"Mark",
"MarkDecorator",
@ -48,7 +51,7 @@ old_mark_config_key = StashKey[Optional[Config]]()
def param(
*values: object,
marks: MarkDecorator | Collection[MarkDecorator | Mark] = (),
id: str | None = None,
id: str | _HiddenParam | None = None,
) -> ParameterSet:
"""Specify a parameter in `pytest.mark.parametrize`_ calls or
:ref:`parametrized fixtures <fixture-parametrize-marks>`.
@ -72,7 +75,14 @@ def param(
:ref:`pytest.mark.usefixtures <pytest.mark.usefixtures ref>` cannot be added via this parameter.
:param id: The id to attribute to this parameter set.
:type id: str | Literal[pytest.HIDDEN_PARAM] | None
:param id:
The id to attribute to this parameter set.
.. versionadded:: 8.4
:ref:`hidden-param` means to hide the parameter set
from the test name. Can only be used at most 1 time, as
test names need to be unique.
"""
return ParameterSet.param(*values, marks=marks, id=id)

22
src/_pytest/mark/structures.py
View File

@ -10,6 +10,7 @@ from collections.abc import Mapping
from collections.abc import MutableMapping
from collections.abc import Sequence
import dataclasses
import enum
import inspect
from typing import Any
from typing import final
@ -39,6 +40,16 @@ if TYPE_CHECKING:
EMPTY_PARAMETERSET_OPTION = "empty_parameter_set_mark"
# Singleton type for HIDDEN_PARAM, as described in:
# https://www.python.org/dev/peps/pep-0484/#support-for-singleton-types-in-unions
class _HiddenParam(enum.Enum):
token = 0
#: Can be used as a parameter set id to hide it from the test name.
HIDDEN_PARAM = _HiddenParam.token
def istestfunc(func) -> bool:
return callable(func) and getattr(func, "__name__", "<lambda>") != "<lambda>"
@ -69,14 +80,14 @@ def get_empty_parameterset_mark(
class ParameterSet(NamedTuple):
values: Sequence[object | NotSetType]
marks: Collection[MarkDecorator | Mark]
id: str | None
id: str | _HiddenParam | None
@classmethod
def param(
cls,
*values: object,
marks: MarkDecorator | Collection[MarkDecorator | Mark] = (),
id: str | None = None,
id: str | _HiddenParam | None = None,
) -> ParameterSet:
if isinstance(marks, MarkDecorator):
marks = (marks,)
@ -89,8 +100,11 @@ class ParameterSet(NamedTuple):
)
if id is not None:
if not isinstance(id, str):
raise TypeError(f"Expected id to be a string, got {type(id)}: {id!r}")
if not isinstance(id, str) and id is not HIDDEN_PARAM:
raise TypeError(
"Expected id to be a string or a `pytest.HIDDEN_PARAM` sentinel, "
f"got {type(id)}: {id!r}",
)
return cls(values, marks, id)
@classmethod

54
src/_pytest/python.py
View File

@ -25,6 +25,7 @@ import types
from typing import Any
from typing import final
from typing import Literal
from typing import NoReturn
from typing import TYPE_CHECKING
import warnings
@ -56,7 +57,9 @@ from _pytest.fixtures import FuncFixtureInfo
from _pytest.fixtures import get_scope_node
from _pytest.main import Session
from _pytest.mark import ParameterSet
from _pytest.mark.structures import _HiddenParam
from _pytest.mark.structures import get_unpacked_marks
from _pytest.mark.structures import HIDDEN_PARAM
from _pytest.mark.structures import Mark
from _pytest.mark.structures import MarkDecorator
from _pytest.mark.structures import normalize_mark_list
@ -473,7 +476,7 @@ class PyCollector(PyobjMixin, nodes.Collector, abc.ABC):
fixtureinfo.prune_dependency_tree()
for callspec in metafunc._calls:
subname = f"{name}[{callspec.id}]"
subname = f"{name}[{callspec.id}]" if callspec._idlist else name
yield Function.from_parent(
self,
name=subname,
@ -884,7 +887,7 @@ class IdMaker:
# Used only for clearer error messages.
func_name: str | None
def make_unique_parameterset_ids(self) -> list[str]:
def make_unique_parameterset_ids(self) -> list[str | _HiddenParam]:
"""Make a unique identifier for each ParameterSet, that may be used to
identify the parametrization in a node ID.
@ -905,6 +908,8 @@ class IdMaker:
# Suffix non-unique IDs to make them unique.
for index, id in enumerate(resolved_ids):
if id_counts[id] > 1:
if id is HIDDEN_PARAM:
self._complain_multiple_hidden_parameter_sets()
suffix = ""
if id and id[-1].isdigit():
suffix = "_"
@ -919,15 +924,21 @@ class IdMaker:
)
return resolved_ids
def _resolve_ids(self) -> Iterable[str]:
def _resolve_ids(self) -> Iterable[str | _HiddenParam]:
"""Resolve IDs for all ParameterSets (may contain duplicates)."""
for idx, parameterset in enumerate(self.parametersets):
if parameterset.id is not None:
# ID provided directly - pytest.param(..., id="...")
yield _ascii_escaped_by_config(parameterset.id, self.config)
if parameterset.id is HIDDEN_PARAM:
yield HIDDEN_PARAM
else:
yield _ascii_escaped_by_config(parameterset.id, self.config)
elif self.ids and idx < len(self.ids) and self.ids[idx] is not None:
# ID provided in the IDs list - parametrize(..., ids=[...]).
yield self._idval_from_value_required(self.ids[idx], idx)
if self.ids[idx] is HIDDEN_PARAM:
yield HIDDEN_PARAM
else:
yield self._idval_from_value_required(self.ids[idx], idx)
else:
# ID not provided - generate it.
yield "-".join(
@ -1001,12 +1012,7 @@ class IdMaker:
return id
# Fail.
if self.func_name is not None:
prefix = f"In {self.func_name}: "
elif self.nodeid is not None:
prefix = f"In {self.nodeid}: "
else:
prefix = ""
prefix = self._make_error_prefix()
msg = (
f"{prefix}ids contains unsupported value {saferepr(val)} (type: {type(val)!r}) at index {idx}. "
"Supported types are: str, bytes, int, float, complex, bool, enum, regex or anything with a __name__."
@ -1019,6 +1025,21 @@ class IdMaker:
and the index of the ParameterSet."""
return str(argname) + str(idx)
def _complain_multiple_hidden_parameter_sets(self) -> NoReturn:
fail(
f"{self._make_error_prefix()}multiple instances of HIDDEN_PARAM "
"cannot be used in the same parametrize call, "
"because the tests names need to be unique."
)
def _make_error_prefix(self) -> str:
if self.func_name is not None:
return f"In {self.func_name}: "
elif self.nodeid is not None:
return f"In {self.nodeid}: "
else:
return ""
@final
@dataclasses.dataclass(frozen=True)
@ -1047,7 +1068,7 @@ class CallSpec2:
*,
argnames: Iterable[str],
valset: Iterable[object],
id: str,
id: str | _HiddenParam,
marks: Iterable[Mark | MarkDecorator],
scope: Scope,
param_index: int,
@ -1065,7 +1086,7 @@ class CallSpec2:
params=params,
indices=indices,
_arg2scope=arg2scope,
_idlist=[*self._idlist, id],
_idlist=self._idlist if id is HIDDEN_PARAM else [*self._idlist, id],
marks=[*self.marks, *normalize_mark_list(marks)],
)
@ -1190,6 +1211,11 @@ class Metafunc:
They are mapped to the corresponding index in ``argvalues``.
``None`` means to use the auto-generated id.
.. versionadded:: 8.4
:ref:`hidden-param` means to hide the parameter set
from the test name. Can only be used at most 1 time, as
test names need to be unique.
If it is a callable it will be called for each entry in
``argvalues``, and the return value is used as part of the
auto-generated id for the whole set (where parts are joined with
@ -1322,7 +1348,7 @@ class Metafunc:
ids: Iterable[object | None] | Callable[[Any], object | None] | None,
parametersets: Sequence[ParameterSet],
nodeid: str,
) -> list[str]:
) -> list[str | _HiddenParam]:
"""Resolve the actual ids for the given parameter sets.
:param argnames:

2
src/pytest/__init__.py
View File

@ -33,6 +33,7 @@ from _pytest.legacypath import Testdir
from _pytest.logging import LogCaptureFixture
from _pytest.main import Dir
from _pytest.main import Session
from _pytest.mark import HIDDEN_PARAM
from _pytest.mark import Mark
from _pytest.mark import MARK_GEN as mark
from _pytest.mark import MarkDecorator
@ -91,6 +92,7 @@ set_trace = __pytestPDB.set_trace
__all__ = [
"HIDDEN_PARAM",
"Cache",
"CallInfo",
"CaptureFixture",

125
testing/python/metafunc.py
View File

@ -19,6 +19,7 @@ from _pytest import python
from _pytest.compat import getfuncargnames
from _pytest.compat import NOTSET
from _pytest.outcomes import fail
from _pytest.outcomes import Failed
from _pytest.pytester import Pytester
from _pytest.python import Function
from _pytest.python import IdMaker
@ -2143,3 +2144,127 @@ class TestMarkersWithParametrization:
"*= 6 passed in *",
]
)
class TestHiddenParam:
"""Test that pytest.HIDDEN_PARAM works"""
def test_parametrize_ids(self, pytester: Pytester) -> None:
items = pytester.getitems(
"""
import pytest
@pytest.mark.parametrize(
("foo", "bar"),
[
("a", "x"),
("b", "y"),
("c", "z"),
],
ids=["paramset1", pytest.HIDDEN_PARAM, "paramset3"],
)
def test_func(foo, bar):
pass
"""
)
names = [item.name for item in items]
assert names == [
"test_func[paramset1]",
"test_func",
"test_func[paramset3]",
]
def test_param_id(self, pytester: Pytester) -> None:
items = pytester.getitems(
"""
import pytest
@pytest.mark.parametrize(
("foo", "bar"),
[
pytest.param("a", "x", id="paramset1"),
pytest.param("b", "y", id=pytest.HIDDEN_PARAM),
("c", "z"),
],
)
def test_func(foo, bar):
pass
"""
)
names = [item.name for item in items]
assert names == [
"test_func[paramset1]",
"test_func",
"test_func[c-z]",
]
def test_multiple_hidden_param_is_forbidden(self, pytester: Pytester) -> None:
pytester.makepyfile(
"""
import pytest
@pytest.mark.parametrize(
("foo", "bar"),
[
("a", "x"),
("b", "y"),
],
ids=[pytest.HIDDEN_PARAM, pytest.HIDDEN_PARAM],
)
def test_func(foo, bar):
pass
"""
)
result = pytester.runpytest("--collect-only")
result.stdout.fnmatch_lines(
[
"collected 0 items / 1 error",
"",
"*= ERRORS =*",
"*_ ERROR collecting test_multiple_hidden_param_is_forbidden.py _*",
"E Failed: In test_func: multiple instances of HIDDEN_PARAM cannot be used "
"in the same parametrize call, because the tests names need to be unique.",
"*! Interrupted: 1 error during collection !*",
"*= no tests collected, 1 error in *",
]
)
def test_multiple_hidden_param_is_forbidden_idmaker(self) -> None:
id_maker = IdMaker(
("foo", "bar"),
[pytest.param("a", "x"), pytest.param("b", "y")],
None,
[pytest.HIDDEN_PARAM, pytest.HIDDEN_PARAM],
None,
"some_node_id",
None,
)
expected = "In some_node_id: multiple instances of HIDDEN_PARAM"
with pytest.raises(Failed, match=expected):
id_maker.make_unique_parameterset_ids()
def test_multiple_parametrize(self, pytester: Pytester) -> None:
items = pytester.getitems(
"""
import pytest
@pytest.mark.parametrize(
"bar",
["x", "y"],
)
@pytest.mark.parametrize(
"foo",
["a", "b"],
ids=["a", pytest.HIDDEN_PARAM],
)
def test_func(foo, bar):
pass
"""
)
names = [item.name for item in items]
assert names == [
"test_func[a-x]",
"test_func[a-y]",
"test_func[x]",
"test_func[y]",
]

51
testing/test_assertion.py
View File

@ -218,10 +218,36 @@ class TestImportHookInstallation:
assert result.ret == 0
@pytest.mark.parametrize("mode", ["plain", "rewrite"])
@pytest.mark.parametrize("disable_plugin_autoload", ["env_var", "cli", ""])
@pytest.mark.parametrize("explicit_specify", ["env_var", "cli", ""])
def test_installed_plugin_rewrite(
self, pytester: Pytester, mode, monkeypatch
self,
pytester: Pytester,
mode: str,
monkeypatch: pytest.MonkeyPatch,
disable_plugin_autoload: str,
explicit_specify: str,
) -> None:
monkeypatch.delenv("PYTEST_DISABLE_PLUGIN_AUTOLOAD", raising=False)
args = ["mainwrapper.py", "-s", f"--assert={mode}"]
if disable_plugin_autoload == "env_var":
monkeypatch.setenv("PYTEST_DISABLE_PLUGIN_AUTOLOAD", "1")
elif disable_plugin_autoload == "cli":
monkeypatch.delenv("PYTEST_DISABLE_PLUGIN_AUTOLOAD", raising=False)
args.append("--disable-plugin-autoload")
else:
assert disable_plugin_autoload == ""
monkeypatch.delenv("PYTEST_DISABLE_PLUGIN_AUTOLOAD", raising=False)
name = "spamplugin"
if explicit_specify == "env_var":
monkeypatch.setenv("PYTEST_PLUGINS", name)
elif explicit_specify == "cli":
args.append("-p")
args.append(name)
else:
assert explicit_specify == ""
# Make sure the hook is installed early enough so that plugins
# installed via distribution package are rewritten.
pytester.mkdir("hampkg")
@ -250,7 +276,7 @@ class TestImportHookInstallation:
import pytest
class DummyEntryPoint(object):
name = 'spam'
name = 'spamplugin'
module_name = 'spam.py'
group = 'pytest11'
@ -275,20 +301,29 @@ class TestImportHookInstallation:
check_first([10, 30], 30)
def test2(check_first2):
check_first([10, 30], 30)
check_first2([10, 30], 30)
""",
}
pytester.makepyfile(**contents)
result = pytester.run(
sys.executable, "mainwrapper.py", "-s", f"--assert={mode}"
)
result = pytester.run(sys.executable, *args)
if mode == "plain":
expected = "E AssertionError"
elif mode == "rewrite":
expected = "*assert 10 == 30*"
else:
assert 0
result.stdout.fnmatch_lines([expected])
if not disable_plugin_autoload or explicit_specify:
result.assert_outcomes(failed=2)
result.stdout.fnmatch_lines([expected, expected])
else:
result.assert_outcomes(errors=2)
result.stdout.fnmatch_lines(
[
"E fixture 'check_first' not found",
"E fixture 'check_first2' not found",
]
)
def test_rewrite_ast(self, pytester: Pytester) -> None:
pytester.mkdir("pkg")

32
testing/test_capture.py
View File

@ -446,6 +446,38 @@ class TestCaptureFixture:
)
reprec.assertoutcome(passed=1)
def test_capteesys(self, pytester: Pytester) -> None:
p = pytester.makepyfile(
"""\
import sys
def test_one(capteesys):
print("sTdoUt")
print("sTdeRr", file=sys.stderr)
out, err = capteesys.readouterr()
assert out == "sTdoUt\\n"
assert err == "sTdeRr\\n"
"""
)
# -rN and --capture=tee-sys means we'll read them on stdout/stderr,
# as opposed to both being reported on stdout
result = pytester.runpytest(p, "--quiet", "--quiet", "-rN", "--capture=tee-sys")
assert result.ret == ExitCode.OK
result.stdout.fnmatch_lines(["sTdoUt"]) # tee'd out
result.stderr.fnmatch_lines(["sTdeRr"]) # tee'd out
result = pytester.runpytest(p, "--quiet", "--quiet", "-rA", "--capture=tee-sys")
assert result.ret == ExitCode.OK
result.stdout.fnmatch_lines(
["sTdoUt", "sTdoUt", "sTdeRr"]
) # tee'd out, the next two reported
result.stderr.fnmatch_lines(["sTdeRr"]) # tee'd out
# -rA and --capture=sys means we'll read them on stdout.
result = pytester.runpytest(p, "--quiet", "--quiet", "-rA", "--capture=sys")
assert result.ret == ExitCode.OK
result.stdout.fnmatch_lines(["sTdoUt", "sTdeRr"]) # no tee, just reported
assert not result.stderr.lines
def test_capsyscapfd(self, pytester: Pytester) -> None:
p = pytester.makepyfile(
"""\

61
testing/test_config.py
View File

@ -6,6 +6,7 @@ import dataclasses
import importlib.metadata
import os
from pathlib import Path
import platform
import re
import sys
import textwrap
@ -1314,14 +1315,13 @@ def test_plugin_preparse_prevents_setuptools_loading(
)
@pytest.mark.parametrize(
"parse_args,should_load", [(("-p", "mytestplugin"), True), ((), False)]
)
@pytest.mark.parametrize("disable_plugin_method", ["env_var", "flag", ""])
@pytest.mark.parametrize("enable_plugin_method", ["env_var", "flag", ""])
def test_disable_plugin_autoload(
pytester: Pytester,
monkeypatch: MonkeyPatch,
parse_args: tuple[str, str] | tuple[()],
should_load: bool,
enable_plugin_method: str,
disable_plugin_method: str,
) -> None:
class DummyEntryPoint:
project_name = name = "mytestplugin"
@ -1342,23 +1342,60 @@ def test_disable_plugin_autoload(
attrs_used = []
def __getattr__(self, name):
assert name == "__loader__"
assert name in ("__loader__", "__spec__")
self.attrs_used.append(name)
return object()
def distributions():
return (Distribution(),)
monkeypatch.setenv("PYTEST_DISABLE_PLUGIN_AUTOLOAD", "1")
parse_args: list[str] = []
if disable_plugin_method == "env_var":
monkeypatch.setenv("PYTEST_DISABLE_PLUGIN_AUTOLOAD", "1")
elif disable_plugin_method == "flag":
monkeypatch.delenv("PYTEST_DISABLE_PLUGIN_AUTOLOAD")
parse_args.append("--disable-plugin-autoload")
else:
assert disable_plugin_method == ""
monkeypatch.delenv("PYTEST_DISABLE_PLUGIN_AUTOLOAD")
if enable_plugin_method == "env_var":
monkeypatch.setenv("PYTEST_PLUGINS", "mytestplugin")
elif enable_plugin_method == "flag":
parse_args.extend(["-p", "mytestplugin"])
else:
assert enable_plugin_method == ""
monkeypatch.setattr(importlib.metadata, "distributions", distributions)
monkeypatch.setitem(sys.modules, "mytestplugin", PseudoPlugin())
config = pytester.parseconfig(*parse_args)
has_loaded = config.pluginmanager.get_plugin("mytestplugin") is not None
assert has_loaded == should_load
if should_load:
assert PseudoPlugin.attrs_used == ["__loader__"]
else:
assert PseudoPlugin.attrs_used == []
# it should load if it's enabled, or we haven't disabled autoloading
assert has_loaded == (bool(enable_plugin_method) or not disable_plugin_method)
# The reason for the discrepancy between 'has_loaded' and __loader__ being accessed
# appears to be the monkeypatching of importlib.metadata.distributions; where
# files being empty means that _mark_plugins_for_rewrite doesn't find the plugin.
# But enable_method==flag ends up in mark_rewrite being called and __loader__
# being accessed.
assert ("__loader__" in PseudoPlugin.attrs_used) == (
has_loaded
and not (enable_plugin_method in ("env_var", "") and not disable_plugin_method)
)
# __spec__ is accessed in AssertionRewritingHook.exec_module, which would be
# eventually called if we did a full pytest run; but it's only accessed with
# enable_plugin_method=="env_var" because that will early-load it.
# Except when autoloads aren't disabled, in which case PytestPluginManager.import_plugin
# bails out before importing it.. because it knows it'll be loaded later?
# The above seems a bit weird, but I *think* it's true.
if platform.python_implementation() != "PyPy":
assert ("__spec__" in PseudoPlugin.attrs_used) == bool(
enable_plugin_method == "env_var" and disable_plugin_method
)
# __spec__ is present when testing locally on pypy, but not in CI ????
def test_plugin_loading_order(pytester: Pytester) -> None:

68
testing/test_debugging.py
View File

@ -52,6 +52,16 @@ def custom_pdb_calls() -> list[str]:
def interaction(self, *args):
called.append("interaction")
# Methods which we copy docstrings to.
def do_debug(self, *args): # pragma: no cover
pass
def do_continue(self, *args): # pragma: no cover
pass
def do_quit(self, *args): # pragma: no cover
pass
_pytest._CustomPdb = _CustomPdb # type: ignore
return called
@ -75,6 +85,16 @@ def custom_debugger_hook():
print("**CustomDebugger**")
called.append("set_trace")
# Methods which we copy docstrings to.
def do_debug(self, *args): # pragma: no cover
pass
def do_continue(self, *args): # pragma: no cover
pass
def do_quit(self, *args): # pragma: no cover
pass
_pytest._CustomDebugger = _CustomDebugger # type: ignore
yield called
del _pytest._CustomDebugger # type: ignore
@ -965,6 +985,34 @@ class TestPDB:
child.sendeof()
self.flush(child)
def test_pdb_wrapped_commands_docstrings(self, pytester: Pytester) -> None:
p1 = pytester.makepyfile(
"""
def test_1():
assert False
"""
)
child = pytester.spawn_pytest(f"--pdb {p1}")
child.expect("Pdb")
# Verify no undocumented commands
child.sendline("help")
child.expect("Documented commands")
assert "Undocumented commands" not in child.before.decode()
child.sendline("help continue")
child.expect("Continue execution")
child.expect("Pdb")
child.sendline("help debug")
child.expect("Enter a recursive debugger")
child.expect("Pdb")
child.sendline("c")
child.sendeof()
self.flush(child)
class TestDebuggingBreakpoints:
@pytest.mark.parametrize("arg", ["--pdb", ""])
@ -1288,6 +1336,16 @@ def test_pdbcls_via_local_module(pytester: Pytester) -> None:
def runcall(self, *args, **kwds):
print("runcall_called", args, kwds)
# Methods which we copy the docstring over.
def do_debug(self, *args):
pass
def do_continue(self, *args):
pass
def do_quit(self, *args):
pass
""",
)
result = pytester.runpytest(
@ -1354,6 +1412,16 @@ def test_pdb_wrapper_class_is_reused(pytester: Pytester) -> None:
def set_trace(self, *args):
print("set_trace_called", args)
# Methods which we copy the docstring over.
def do_debug(self, *args):
pass
def do_continue(self, *args):
pass
def do_quit(self, *args):
pass
""",
)
result = pytester.runpytest(str(p1), "--pdbcls=mypdb:MyPdb", syspathinsert=True)

6
testing/test_mark.py
View File

@ -1170,7 +1170,11 @@ def test_pytest_param_id_requires_string() -> None:
with pytest.raises(TypeError) as excinfo:
pytest.param(id=True) # type: ignore[arg-type]
(msg,) = excinfo.value.args
assert msg == "Expected id to be a string, got <class 'bool'>: True"
expected = (
"Expected id to be a string or a `pytest.HIDDEN_PARAM` sentinel, "
"got <class 'bool'>: True"
)
assert msg == expected
@pytest.mark.parametrize("s", (None, "hello world"))