SoFunction
Updated on 2024-11-16

pytest realize multi-process and multi-threaded running super nice plugin

preamble

If you want to execute use cases in a distributed manner, the use case design must follow the following principles:

1. The use cases are all independent of each other.
2. Use case a should not rely on use case b.
3. The use cases are not prioritized.
4, random can be executed each use case can be run independently of the success of each use case can be repeated to run, does not affect the other use cases
This is the same as our usual multiple manual tests, use cases are independent, can be randomly assigned to different people to execute, do not rely on each other, there is no order of precedence between use cases

I. pytest-parallel

Installation:pip install pytest-parallel

Common parameter configurations:

  • --workers=n: this parameter is required for multi-process operation, n is the number of processes. The default is 1.
  • --tests-per-worker=n: multi-threaded need to add this parameter, n is the number of threads

If both parameters are configured, it is process parallelism, up to n threads per process, number of bus threads: number of processes * number of threads

Note: On windows the process count is always 1.

Requires the use of the ifname == “main"::, running it in dos will report an error

#!/usr/bin/env python
# _*_ coding: utf-8 _*_
# @project : API_Service
# @File    : test_1.py
# @Date : 2021/6/15 3:07 PM
# @Author : Li Wenliang


# demo:
import pytest

def test_01():
    print('Test case 1 operation')

def test_02():
    print('Test case 2 operation')

def test_03():
    print('Test case 3 operation')

def test_04():
    print('Test case 4 operation')


def test_05():
    print('Test case 5 operation')


def test_06():
    print('Test case 6 operation')


def test_07():
    print('Test case 7 operations')


def test_08():
    print('Test case 8 operation')

if __name__ == "__main__":
    (["-s", "test_1.py",'--workers=2', '--tests-per-worker=4'])

在这里插入图片描述

II. pytest-xdist

Installation:pip install pytest-xdist

Multi-threading is not supported

Common parameter configurations:

  • -n=*: * represents the number of processes

Multi-cpu parallel execution of the use case, directly add a -n parameter can be, after the num parameter is the number of parallelism, such as num set to 3

  • -n auto automatically detects the number of CPUs in the system.
  • -n num Specify the number of processor processes running the test

III. Comparative notes

pytest-parallel is relatively better than pytst-xdist, with more feature support.

pytst-xdist doesn't support multithreading, while pytest-parallel supports python 3.6 and above, if you want to do multi-process concurrency do it on linux or mac, it doesn't work on Windows (workers=1), if you do multithreading linux/mac/windows platforms support it, the process number is the value of workers.

Common configuration commands for pytest-parallel are as follows

  • -workers (optional) *: this parameter is required for multi-process operation, * is the number of processes. The default is 1.
  • -tests-per-worker (optional) *: multithreaded, * is the maximum number of concurrent threads per worker to run. The default is 1

pytest --workers 3: 3 processes running
pytest --tests-per-worker 4: 4 threads running
pytest --workers 2 --tests-per-worker 4: 2 processes in parallel and up to 4 threads running per process, i.e. up to 8 threads running in total.

IV. Special attention

1, pytest-parallel's workers parameter is always 1 under windows, and can take different values under linux and mac.
2, pytest-parallel added multi-thread processing, the final execution time is the time of the longest running thread.
3, in windows want to use multi-process selection pytst-xdist; want to use multi-threaded selection pytest-parallel

to this article on pytest to achieve multi-process and multi-threaded running super plug-ins are introduced to this article, more related pytest multi-process and multi-threaded running plug-ins, please search for my previous posts or continue to browse the following related articles I hope you will support me in the future more!