SoFunction
Updated on 2024-11-20

Hands-on understanding of python multi-process, multi-threading

clarification

See the link for the corresponding learning video, this article will only summarize the highlights.

请添加图片描述

请添加图片描述

multiprocess

Highlights (just look at the main function in the code below)

1. Creation

2. How to open a daemon

3. Multi-process, high overhead, call multiple processes with a for loop, the background cpu goes up at once

import time
import multiprocessing
import os
def dance(who,num):
    print("danceparent process:{}".format(()))
    for i in range(1,num+1):
        print("number:{}————{}dances。。。{}".format((),who,i))
        (0.5)
def sing(num):
    print("singparent process:{}".format(()))
    for i in range(1,num+1):
        print("number:{}----sing a song。。。{}".format((),i))
        (0.5)
def work():
    for i in range(10):
        print("Work in progress.")
        (0.2)
if __name__ == '__main__':
    # print("main main process {}".format(())))
    start= ()
    #1 Process creation and startup
    # # 1.1 Create process objects, note that dance cannot be enclosed in parentheses
    # # dance_process = (target=dance) #1. no parameters
    # dance_process=(target=dance,args=("lin",3))#2. with args=metazoan approach
    # sing_process = (target=sing,kwargs={"num":3})#3. in kwargs={} dictionary fashion
    # # 1.2 Starting the process
    # dance_process.start()
    # sing_process.start()
    #2. Default - main process and child process are separated, main process can be completed in 1s, child process needs 2s, main process will wait for all child processes to finish execution, then exit
    # 2.1 child daemon main process, when the main but finished, the child will be disconnected (such as qq a closed, all the chat window is gone). daemon=True
    work_process = (target=work,daemon=True)
    work_process.start()
    (1)
    print("The main process is complete!")#The main process and child processes are separate, the main process can be completed in 1s, the child processes need 2s, the main process will wait for all child processes to finish execution, then exit
    print("Main main process time spent:",()-start)
    #

multi-threaded

请添加图片描述

recount (e.g. results of election)

1. Creation

2. Guardian threads

3. Thread safety issues (multiple people grabbing tickets, will grab the same one)

import time
import os
import threading
def dance(num):
    for i in range(num):
        print("Process number:{},Thread number:{}————dances。。。".format((),threading.current_thread()))
        (1)
def sing(count):
    for i in range(count):
        print("Process number:{},Thread number:{}----sing a song。。。".format((),threading.current_thread()))
        (1)
def task():
    (1)
    thread=threading.current_thread()
    print(thread)
if __name__ == '__main__':
    # start=()
    # # sing_thread =(target=dance,args=(3,),daemon=True) # set to guard main thread
    # sing_thread = (target=dance, args=(3,))
    # dance_thread = (target=sing,kwargs={"count":3})
    #
    # sing_thread.start()
    # dance_thread.start()
    #
    # (1)
    # print("Process number: {} main thread ended... Time spent {}".format(((), (()-start))))
    for i in range(10):#Execution between multiple threads is unordered and scheduled by cpu
        sub_thread = (target=task)
        sub_thread.start()

thread safety

Since threads proceed directly in an unorganized manner and they share all the resources of the same process, thread-safety issues can arise (e.g., multiple people grabbing tickets online and buying the same one)

请添加图片描述
请添加图片描述

#The following code will sell 0 votes when there is no lock lock, with lock it works fine

import threading
import time
lock =()
class Sum_tickets:
    def __init__(self,tickets):
        =tickets
def window(sum_tickets):
    while True:
        with lock:
            if sum_tickets.tickets>0:
                (0.2)
                print(threading.current_thread().name,"pick up tickets{}".format(sum_tickets.tickets))
                sum_tickets.tickets-=1
            else:
                break
if __name__ == '__main__':
    sum_tickets=Sum_tickets(10)
    sub_thread1 = (name="Window 1",target=window,args=(sum_tickets,))
    sub_thread2 = (name="Window 2",target=window,args=(sum_tickets,))
    sub_thread1.start()
    sub_thread2.start()

Highly concurrent copies (multi-process, multi-threaded)

import os
import multiprocessing
import threading
import time
def copy_file(file_name,source_dir,dest_dir):
    source_path = source_dir+"/"+file_name
    dest_path =dest_dir+"/"+file_name
    print("The current process is:{}".format(()))
    with open(source_path,"rb") as source_file:
        with open(dest_path,"wb") as dest_file:
            while True:
                data=source_file.read(1024)
                if data:
                    dest_file.write(data)
                else:
                    break
    pass
if __name__ == '__main__':
    source_dir=r'C:\Users\Administrator\Desktop\Attention'
    dest_dir=r'C:\Users\Administrator\Desktop\test'
    start = ()
    try:
        (dest_dir)
    except:
        print("Target file already exists.")
    file_list =(source_dir)
    count=0
    #1 Multi-process
    for file_name in file_list:
        count+=1
        print(count)
        sub_processor=(target=copy_file,
                                args=(file_name,source_dir,dest_dir))
        sub_processor.start()
        # (20)
    print(()-start)
# Here there is a master process and child processes, through the printout can be seen, the master process in the creation of 1, 2, 3, 4,,, 21 process, the child process has already begun to execute, that is to say, each process does not affect each other
# 9
# 10
# 11
# 12
# 13
# Current processes are: 2936 (when the main process creates the 13th, at which point the first child process starts working)
# 14
# Current process is: 10120
# Current process: 10440
# 15
# Current process: 9508
    # 2 Multi-threading
    # for file_name in file_list:
    #     count += 1
    #     print(count)
    #     sub_thread = (target=copy_file,
    #                                             args=(file_name, source_dir, dest_dir))
    #     sub_thread.start()
    #     # (20)
    # print(() - start)

summarize

That's all for this post, I hope it helped you and I hope you'll check back for more from me!