SoFunction
Updated on 2024-11-16

python implementation of multi-process concurrency control Semaphore and mutual exclusion lock LOCK

I. Understanding locks

Application Scenario Examples: Lock Mutual Exclusion Lock: Example - there are three colleagues at the same time need to go to the toilet, but there is only one toilet, the colleagues compared to the process, more than one process and fight to seize a toilet, we want to ensure that the order of priority, one by one, then it must be serial, first-come, first-served, someone to use it, lock, the end of the rest of the people continue to scramble!

1、Use Lock to deal with

Simulate three coworkers grabbing a toilet

from multiprocessing import Process
from multiprocessing import Lock
import time
import random

def task1(p, lock):
    # Locked up
    ()
    print(f'{p} start to excrete')
    ((1, 3))
    print(f'{p} end of draining')
    # Unlocked
    ()

def task2(p, lock):
    ()
    print(f'{p} start to excrete')
    ((1, 3))
    print(f'{p} end of draining')
    ()

def task3(p, lock):
    ()
    print(f'{p} start to excrete')
    ((1, 3))
    print(f'{p} end of draining')
    ()


if __name__ == '__main__':
    # Instantiate a lock object
    mutex = Lock()
    # Pass the lock as a parameter to the process object
    p1 = Process(target=task1, args=('task1', mutex,))
    p2 = Process(target=task2, args=('task2', mutex,))
    p3 = Process(target=task3, args=('task3', mutex,))

    ()
    ()
    ()

Implementation results:

# Output result: three processes start to scramble for the mutual exclusion lock, the first one to grab it executes first, after the execution is finished, releases the lock, and the remaining ones continue to scramble.
Task 1 Start of defecation
Task 1 End of excretion
task2 Start of defecation
Task 2 End of excretion
task3 Starting to defecate
Task 3 End of excretion

1. Take note:

  • Mutually exclusive locks are in functions where () locks once and has to be () unlocked once, writing the code that needs to be executed between locking and unlocking.
  • If you lock more than two times in a row, you will have a deadlock and the code will not continue to execute. It must be locked once and unlocked once.

2. lock and join are compared:

  • The common thread ------ can turn parallel into serial, guaranteeing the order of execution
  • Difference ------join is setting the order artificially, lock is letting it scramble the order to ensure fairness

2, the use of reflection, to optimize the above code

Although the above code plays the effect of FIFO, one in one out, but it is not perfect. As we all know, we go to the toilet is whoever grabs it first, not that according to the code in the order of start() execution. It's more reasonable to be restricted by the process that got there first.

from multiprocessing import Process
from multiprocessing import Lock
import time
import random
import sys

def task1(p, lock):
    # Locked up
    ()
    print(f'{p} Start Printing')
    ((1, 3))
    print(f'{p} End of Printing')
    # Unlocked
    ()

def task2(p, lock):
    ()
    print(f'{p} Start Printing')
    ((1, 3))
    print(f'{p} End of Printing')
    ()

def task3(p, lock):
    ()
    print(f'{p} Start Printing')
    ((1, 3))
    print(f'{p} End of Printing')
    ()


if __name__ == '__main__':
    slock = Lock()
    for i in range(1,4):
       p = Process(target=getattr([__name__], f'task{i}'), args=(f'task{i}', slock))
       ()

Output results:

task2 Start Printing
task2 End of print
task3 Start Printing
task3 End of print
task1 Start Printing
task1 End of print

II. Process concurrency control semaphore

semaphore: used to control the number of accesses to shared resources, can control the number of concurrent processes at the same time
Signal volume: also a lock, but does not guarantee data security, open multiple threads at the same time, but provides for concurrent execution of the upper limit, the back of how much to go, how much to enter. (used to control the number of concurrent)

1. Example of multi-process control (1)

# Example: a toilet has 5 pits and a maximum of 5 people can go to the toilet at the same time, at the current moment 20 people want to go to the toilet but only 5 can go in, then how many come out and how many go in?

# Comma-separated references to multiple functions from a single module
from threading import Semaphore, Thread, currentThread
import time
import random

sem = Semaphore(3)             # Concurrent executions set to 5

def task():
    ()
    print(f'{currentThread().name}')
    ((1,3))
    ()

if __name__ == '__main__':
    for i in range(20):
        t = Thread(target=task)
        ()

Execution result: the first concurrency is 3, after the first to grab the lock first execution

Thread-1
Thread-2
Thread-3

Thread-4
Thread-5

Thread-6
Thread-7

Thread-8

Process finished with exit code 0

2. Examples of multi-process control (2)

import multiprocessing
import time

def worker(s, i):
    ()
    print(('%Y-%m-%d %H:%M:%S'), multiprocessing.current_process().name + " Seize and acquire the lock, run ")
    (i)
    print(('%Y-%m-%d %H:%M:%S'), multiprocessing.current_process().name + " End of run, release the lock.")
    ()

if __name__ == '__main__':
    s = (2)
    for i in range(8):
        p = (target=worker, args=(s, 1))
        ()

Implementation results:

The concurrent execution of processes can be seen more clearly by pressing the Enter key for each block in the terminal where the execution results are output.
As can be seen from the following execution results, at the same moment, two processes are executing
2021-05-18 22:50:37 Process-1 seize and acquire lock, running
2021-05-18 22:50:37 Process-2 seize and acquire lock, running

2021-05-18 22:50:38 Process-1 End of run, releasing locks
2021-05-18 22:50:38 Process-3 seize and acquire lock, running
2021-05-18 22:50:38 Process-2 End of run, releasing locks
2021-05-18 22:50:38 Process-4 seize and acquire lock, running

2021-05-18 22:50:39 Process-3 End of run, releasing locks
2021-05-18 22:50:39 Process-5 seize and acquire lock, run
2021-05-18 22:50:39 Process-4 End of run, releasing locks
2021-05-18 22:50:39 Process-6 seize and acquire lock, running

2021-05-18 22:50:40 Process-5 End of run, releasing locks
2021-05-18 22:50:40 Process-7 seize and acquire lock, running
2021-05-18 22:50:40 Process-6 End of run, releasing locks
2021-05-18 22:50:40 Process-8 seize and acquire lock, running

2021-05-18 22:50:41 Process-7 End of run, releasing locks
2021-05-18 22:50:41 Process-8 End of run, releasing locks

Process finished with exit code 0

Third, the process synchronization LOCK

Concurrent execution of multiple processes to improve resource utilization, thereby improving efficiency, but sometimes we need to have only one process at a certain time to access a shared resource, then we need to use the lock LOCK

1. Example without LOCK

import multiprocessing
import time

def task1():
    n = 4
    while n > 1:
        print(f'{("%Y-%M-%d %H:%M:%S")}  task1 output message')
        (1)
        n -= 1

def task2():
    n = 4
    while n > 1:
        print(f'{("%Y-%M-%d %H:%M:%S")}  task2 output message')
        (1)
        n -= 1

def task3():
    n = 4
    while n > 1:
        print(f'{("%Y-%M-%d %H:%M:%S")}  task3 output message')
        (1)
        n -= 1

if __name__ == '__main__':
    p1 = (target=task1)
    p2 = (target=task2)
    p3 = (target=task3)
    ()
    ()
    ()

Implementation results:

2021-59-18 22:59:46 task1 output message
2021-59-18 22:59:46 task2 output message
2021-59-18 22:59:46 task3 Output message

2021-59-18 22:59:47 task1 output message
2021-59-18 22:59:47 task2 output message
2021-59-18 22:59:47 task3 Output message

2021-59-18 22:59:48 task1 output message
2021-59-18 22:59:48 task2 output message
2021-59-18 22:59:48 task3 Output message

Process finished with exit code 0

2. Example of adding LOCK

There are two ways to add a lock: first, lock = () generates the lock object lock

  1. with lock: with starts the lock before execution and closes it at the end of execution.
  2. () ... () : Note that the two must correspond one to the other.
import multiprocessing

import time

def task1(lock):
    with lock:
        n = 4
        while n > 1:
            print(f'{("%Y-%M-%d %H:%M:%S")}  task1 output message')
            (1)
            n -= 1

def task2(lock):
    ()
    n = 4
    while n > 1:
        print(f'{("%Y-%M-%d %H:%M:%S")}  task2 output message')
        (1)
        n -= 1
    ()

def task3(lock):
    ()
    n = 4
    while n > 1:
        print(f'{("%Y-%M-%d %H:%M:%S")}  task3 output message')
        (1)
        n -= 1
    ()

if __name__ == '__main__':
    lock = ()
    p1 = (target=task1, args=(lock,))
    p2 = (target=task2, args=(lock,))
    p3 = (target=task3, args=(lock,))
    ()
    ()
    ()

Implementation results

2021-11-18 23:11:37 task1 output message

2021-11-18 23:11:38 task1 output message

2021-11-18 23:11:39 task1 output message

2021-11-18 23:11:40 task2 output message

2021-11-18 23:11:41 task2 output message

2021-11-18 23:11:42 task2 output message

2021-11-18 23:11:43 task3 output message

2021-11-18 23:11:44 task3 output message

2021-11-18 23:11:45 task3 output message

Process finished with exit code 0
 

to this article on the python implementation of multi-process concurrency control Semaphore and mutual exclusion lock LOCK article is introduced to this, more related python multi-process Semaphore and LOCK content, please search for my previous articles or continue to browse the following related articles I hope you will support me in the future!