pytest is a popular Python testing framework that provides a number of command line options to help users better control the test execution process. When testing with pytest, it is important to familiarize yourself with pytest's command line options, as this will help reduce errors and increase test efficiency. This article describes pytest's command line options in detail and gives examples.
I. Classification of pytest command line options
pytest's command line options can be divided into three categories:
- Environment Configuration Options: These options are used to set up pytest's environment configuration, such as setting up logging levels, overriding configuration files, setting up test modes, and so on.
- Test Filtering Options: These options are used to filter and select test cases, e.g., specify a test directory, select a specific test module, run a specified test function, etc.
- Test Execution Options: These options are used to control pytest's test execution process, such as retrying failed tests, generating test reports, executing tests in parallel, and so on.
Below, we will go through each of these command line options one by one.
II. Environment configuration options
--version: Print the pytest version number.
-h, --help: Display pytest command line options and usage instructions.
--verbose, -v: Increase the level of detail in the test result output.
--quiet, -q: Reduce the level of detail in the test result output.
--traceconfig: Displays detailed information when parsing and loading configuration files.
--pdb: Access the Python debugger when a test fails or an exception occurs.
--pdbcls: customize the type of debugger.
--capture: set pytest's standard output stream capture method, three values are available: sys, fd, no.
--norecursedirs: Set which directories are not searched for test cases.
--rootdir: Set the root directory of pytest.
--maxfail=n: Set to stop test execution after the nth test failure.
--junit-xml=PATH: outputs test results to a JUnit XML file.
III. Testing filtering options
path: Specifies the test directory or file, which can be a relative or absolute path. For example: pytest tests/.
-m: Selects a specific tagged test case for execution. Example: pytest -m "slow".
-k: Selects test cases containing a certain keyword for execution. Example: pytest -k "add or subtract".
-x: Stop test execution when a test case failure is encountered.
--pdb: Access the Python debugger when a test fails or an exception occurs.
--lf: rerun only the test cases that failed the last test.
--ff: Rerun only the test cases that failed in the last test, and re-run those test cases again after all the tests are finished.
--sw: Rerun the last modified test module.
--last-failed-no-failures: rerun only the last failed test cases (if there are no failed test cases, no tests are executed).
--collect-only: Execute only the collection phase of the test case, do not run the execution phase of the test case.
--pdbcls: customize the type of debugger.
--trace: show pytest's internal trace information.
--count: Run the specified number of test cases. Example: pytest -v --count=10.
IV. Test execution options
-n: Run the test cases in parallel, which can be followed by a number to specify the degree of concurrency. For example: pytest -n 4.
-x: Stop test execution when a test case failure is encountered.
--maxfail=n: Set to stop test execution after the nth test failure.
--last-failed: only rerun the last failed test case.
--failed-first: Run the previously failed test case first.
--reruns=n: reruns the test n times if the test case fails.
--pdb: Access the Python debugger when a test fails or an exception occurs.
--pdbcls: customize the type of debugger.
--junit-xml=PATH: outputs test results to a JUnit XML file.
--html=PATH: outputs the test results to an HTML file.
--tb=long/short/line/native/no: Set the format of the output error message.
--capture=no: Disable capturing standard output and standard errors and output them directly to the terminal.
--capture=sys/stdout/stderr: set pytest's standard output stream capture method, there are three optional values: sys, fd, no.
--show-capture=all/failed/no: Controls whether to show the captured standard output stream.
--disable-warnings: Disable pytest's warning messages.
The above is an introduction to pytest's command line options. By using these options, you can better control pytest's test execution process, reduce errors and improve test efficiency. Next, I'll give you some code examples of the command line options and how they work.
V. Code samples and running effects
1. --version: Print the pytest version number.
This option can be used by executing the pytest --version command as shown in the code example below:
$ pytest --version This is pytest version 5.0.1, imported from /usr/local/lib/python3.6/site-packages/
2. -h, --help: Display pytest command line options and usage instructions.
This option can be used by executing the pytest --help command with the following code example:
$ pytest --help usage: pytest [options] [file_or_dir] [file_or_dir] [...] positional arguments: file_or_dir test file or root directory to search for tests optional arguments: -h, --help show this help message and exit --version display pytest version and information about plugins ...
3. --verbose, -v: Increase the level of detail in the test result output.
This option can be used by executing the pytest -v command with the following code example:
$ pytest -v test_example.py::test_add PASSED [ 50%] test_example.py::test_subtract PASSED [100%] ============================ 2 passed in 0.03s =============================
4. --quiet, -q: Reduce the level of detail in the test result output.
This option can be used by executing the pytest -q command with the following code example:
$ pytest -q .. 2 passed in 0.03s
5. --traceconfig: Displays detailed information when parsing and loading configuration files.
This option can be used by executing the pytest --traceconfig command with the following code example:
$ pytest --traceconfig using: pytest-5.0.1, py-1.8.0, pluggy-0.12.0 active plugins: * pytest_cov-2.7.1 at /Users/user/anaconda3/envs/test/lib/python3.6/site-packages/pytest_cov/ disabled plugins: collector: <_pytest. object at 0x107cbe3c8>
6. --pdb: Access to the Python debugger when a test fails or an exception occurs.
This option can be used by executing the pytest --pdb command, which automatically enters the pdb debugger in the event of an error in the test case, as shown in the following code example:
$ pytest --pdb
test_example.py::test_add PASSED [ 50%]
test_example.py::test_subtract FAILED [100%]
=================================== FAILURES ===================================
________________________________ test_subtract ________________________________
def test_subtract():
assert subtract(4, 3) == 1
> assert subtract(2, 3) == -1
E assert 2 == -1
E + where 2 = subtract(2, 3)
test_example.py:10: AssertionError
----------------------------- PDB -----------------------------
> /Users/user/example/test_example.py(10)test_subtract()
...
(Pdb) 7. --capture: set pytest's standard output stream capture method, there are three values available: sys, fd, no.
This option can be used by executing the pytest --capture=sys command to redirect the standard output stream and standard error output to pytest's logging system, as shown in the following code example:
$ pytest --capture=sys
test_example.py::test_add PASSED [ 50%]
test_example.py::test_subtract FAILED [100%]
=================================== FAILURES ===================================
________________________________ test_subtract ________________________________
def test_subtract():
assert subtract(4, 3) == 1
> assert subtract(2, 3) == -1
E assert 2 == -1
E + where 2 = subtract(2, 3)
test_example.py:10: AssertionError
[Capture] capturing logcall output to 'log': sys8. --norecursedirs: Sets which directories are not searched for test cases.
This option can be used to exclude the examples directory from the path to search for test cases by executing the pytest --norecursedirs=examples command, as shown in the code example below:
$ pytest --norecursedirs=examples test_example.py::test_add PASSED [ 50%] test_example.py::test_subtract PASSED [100%] ============================ 2 passed in 0.03s =============================
9. --rootdir: Set the root directory of pytest.
This option can be used by executing the pytest --rootdir=/path/to/project command to set the root directory of pytest to /path/to/project, as shown in the following code example:
$ pytest --rootdir=/path/to/project test_example.py::test_add PASSED [ 50%] test_example.py::test_subtract PASSED [100%] ============================ 2 passed in 0.03s =============================
10. --maxfail=n: Set to stop test execution after the nth test failure.
This option can be used by executing the pytest --maxfail=1 command, which stops the test execution when the first test case fails, as shown in the code example below:
$ pytest --maxfail=1
test_example.py::test_add PASSED [ 50%]
test_example.py::test_subtract FAILED [100%]
=================================== FAILURES ===================================
________________________________ test_subtract ________________________________
def test_subtract():
assert subtract(4, 3) == 1
> assert subtract(2, 3) == -1
E assert 2 == -1
E + where 2 = subtract(2, 3)
test_example.py:10: AssertionError
========================== 1 failed, 1 passed in 0.04s ==========================11.--junit-xml=PATH: Outputs test results to a JUnit XML file.
This option can be used by executing the pytest --junit-xml=test_results.xml command to output the test results to the test_results.xml file, as shown in the code example below:
$ pytest --junit-xml=test_results.xml test_example.py::test_add PASSED [ 50%] test_example.py::test_subtract PASSED [100%] ============================ 2 passed in 0.03s =============================
: Specifies the test directory or file, either a relative or absolute path. For example: pytest tests/.
This option can be used by executing the pytest tests/ command to run only the test cases in the tests directory, as shown in the code example below:
$ pytest tests/ test_example.py::test_add PASSED [ 50%] test_example.py::test_subtract PASSED [100%] ============================ 2 passed in 0.03s =============================
13. -m: Selects a specific tagged test case for execution. Example: pytest -m "slow".
This option can be used by using the @ tag in the test case, for example:
import pytest
@
def test_slow():
pass
def test_not_slow():
pass
Then execute the pytest -m "slow" command to run only the test cases marked with the @ tag, as shown in the following code example:
$ pytest -m "slow" test_example.py:3: PytestUnknownMarkWarning: Unknown - is this a typo? You can register custom marks to avoid this warning - for details, see /en/latest/ @ no tests ran in 0.00s PytestUnknownMarkWarning: Unknown - is this a typo? You can register custom marks to avoid this warning - for details, see /en/latest/
14. -k: Selects test cases containing a certain keyword for execution. For example: pytest -k "add or subtract".
This option can be used by executing the pytest -k "add or subtract" command to run only those test cases that contain the string "add" or "subtract" in the test case name, as shown in the following code example:
$ pytest -k "add or subtract" test_example.py::test_add PASSED [ 50%] test_example.py::test_subtract PASSED [100%] ============================ 2 passed in 0.03s =============================
15. -x: Stop test execution when a test case failure is encountered.
This option can be used by executing the pytest -x command, which stops test execution when the first test case fails, as shown in the following code example:
$ pytest -x
test_example.py::test_add PASSED [ 50%]
test_example.py::test_subtract FAILED [100%]
=================================== FAILURES ===================================
________________________________ test_subtract ________________________________
def test_subtract():
assert subtract(4, 3) == 1
> assert subtract(2, 3) == -1
E assert 2 == -1
E + where 2 = subtract(2, 3)
test_example.py:10: AssertionError
========================== 1 failed, 1 passed in 0.04s ==========================16. --lf: rerun only the test cases that failed the last test.
This option can be used by executing the pytest --lf command to automatically rerun a failed test case if the last test case had an error, as shown in the following code example:
$ pytest --lf
test_example.py::test_add PASSED [ 50%]
test_example.py::test_subtract FAILED [100%]
=================================== FAILURES ===================================
______________________ test_subtract[2-3] ______________________
x = 2
y = 3
def test_subtract(x, y):
assert subtract(x, y) == -1
tests/test_example.py:10: AssertionError
------------------------------ rerun test call -------------------------------
test_example.py::test_subtract[x17. --ff: Rerun only the test cases that failed the last test.
This option can be used by executing the pytest --ff command to automatically rerun the last failed test case when the last test case had an error, as shown in the following code example:
$ pytest --ff
test_example.py::test_add PASSED [ 50%]
test_example.py::test_subtract FAILED [100%]
=================================== FAILURES ===================================
________________________________ test_subtract ________________________________
def test_subtract():
assert subtract(4, 3) == 1
> assert subtract(2, 3) == -1
E assert 2 == -1
E + where 2 = subtract(2, 3)
test_example.py:10: AssertionError
------------------------------- Captured stdout --------------------------------
subtraction of 2 and 3 is -1
========================== 1 failed, 1 passed in 0.04s ==========================18. --failed-first: Runs the test case that failed the last test before all test cases.
This option can be used by executing the pytest --failed-first command, which will run the test cases that failed in the last test before running the rest of the test cases, as shown in the code example below:
$ pytest --failed-first
test_example.py::test_subtract FAILED [100%]
test_example.py::test_add PASSED
=================================== FAILURES ===================================
________________________________ test_subtract ________________________________
def test_subtract():
assert subtract(4, 3) == 1
> assert subtract(2, 3) == -1
E assert 2 == -1
E + where 2 = subtract(2, 3)
test_example.py:10: AssertionError
----------------------------- Captured stdout -----------------------------
subtraction of 2 and 3 is -1
========================== 1 failed, 1 passed in 0.03s ==========================19. -n, --numprocesses: Specify the number of concurrent processes.
This option can be used by executing the pytest -n 4 command to assign the test cases to run in 4 concurrent processes, as shown in the code example below:
$ pytest -n 4 test_example.py::test_add PASSED [ 25%] test_example.py::test_add PASSED [ 50%] test_example.py::test_subtract PASSED [ 75%] test_example.py::test_subtract PASSED [100%] ============================ 4 passed in 0.02s =============================
20. -d, --dist: Enable distributed testing.
This option can be used by executing the pytest --dist=loadfile command, by loading multiple Python processes and distributing the test run across all of them, as shown in the following code example:
$ pytest --dist=loadfile test_example.py::test_add PASSED [ 25%] test_example.py::test_add PASSED [ 50%] test_example.py::test_subtract PASSED [ 75%] test_example.py::test_subtract PASSED [100%] ============================ 4 passed in 0.02s =============================
21. --durations=n: Sorts test cases by runtime and displays the runtime of the slowest n test cases.
This option can be used by executing the pytest --duration=2 command to display the runtime of the two test cases with the slowest runtime, as shown in the code example below:
$ pytest --duration=2 test_example.py::test_add PASSED [ 50%] test_example.py::test_subtract PASSED [100%] ======================== slowest 2 test durations ========================= 0.01s call test_example.py::test_subtract 0.01s call test_example.py::test_add ============================ 2 passed in 0.02s =============================
22. --show-capture: Show the test case standard output stream and standard error output in the result.
This option can be used by executing the pytest --show-capture command to display the test case's standard output stream and standard error output in the test results, as shown in the code example below:
$ pytest --show-capture test_example.py::test_add PASSED [ 50%] test_example.py::test_subtract PASSED [100%] ================================== Captured stdout ================================= subtraction of 4 and 3 is 1 subtraction of 2 and 3 is -1 ============================ 2 passed in 0.03s =============================
The above is a detailed introduction to pytest command line options and the corresponding code examples, I hope to help you better use pytest for testing. For more information about Pytest command line options, please search my previous articles or continue to browse the following related articles I hope you will support me more in the future!