SoFunction
Updated on 2024-11-07

Introduction to python multithreading and queue management shell programs

First of all, let's describe the environment, there are many JAVA programs on the machine, we configure a start | stop | restart script in each JAVA program

An example:

We now want to run these scripts at the same time, to achieve rapid startup of all JAVA programs, if we only use multi-threaded, the thread is not returned to the parent process message, how can we know that these programs are started successfully?

So we use a queue to manage it.

"""I tried gevent, but it blocks here in command""""

The gevent code is as follows If any of you know how to optimize it, could you please let me know?

#!/usr/bin/python2.7
# -*- coding:utf-8 -*-
import os,sys
from datetime import datetime
import commands
import 
.patch_os()
import gevent
 
def Servers():
  servers=('''ls /data/program/payment/ | grep 'payment' ''')
  servers=('\n')
  return servers
 
def handle(servername):
  if [1] == 'start' or [1] == 'stop' or [1] == 'restart':
    print '\033[1;31;40m'
    print '========================>>>go to handle %s<<<=========================' %servername
    print '\033[0m'
    r=('''su - tomcat -c "/data/program/payment/%s/bin/ %s &" ''' %(servername,[1]))  # It will block here and we won't be able to find the right place to make the co-programming switch
    (0)        # Wherever you put it, it blocks either before or after the switch.
    print r
  else:
    print 'Please Use start | stop | restart To Handle The Command'
    (1)
   
if __name__ == '__main__':
  s=Servers()
  threads=[]
  for i in s:
    ((handle,i))
#  print threads
  (threads)

The multi-threaded code is as follows

#!/usr/bin/python2.7
# -*- coding:utf-8 -*-
 
from datetime import datetime
import commands
from Queue import Queue
from threading import Thread
 
_sentinel = object()
 
def Servers():
  servers=('''ls /data/program/payment/ | grep 'payment' ''')
  servers=('\n')
  return servers
 
def producer(servername,out_q):
  if [1] == 'start' or [1] == 'stop' or [1] == 'restart':
    print '\033[1;31;40m'
    print '========================>>>put %s in Queue<<<=========================' %servername
    print '\033[0m'
    out_q.put_nowait(('''su - tomcat -c "/data/program/payment/%s/bin/ %s &" ''' %(servername,[1])))  # Objects placed in the queue
     
  else:
    print 'Please Use start | stop | restart To Handle The Command'
    (1)
 
def consumer(servername,in_q):
  n=len(servername)
  while n > 0:             # Loop through the queue to fetch results until the end of the loop
    data=in_q.get()
    n -= 1
    print '\033[1;31;40m'
    print data
    print '\033[0m'
  print '\033[1;31;40m'
  print 'consumer was done!!!!!!!'
  print '\033[0m' 
 
if __name__ == '__main__':
  s=Servers()
  q = Queue()
  t1 = Thread(target=consumer, args=(s,q,))      # Consumers fetch the result in the queue, which has been looped inside the previous function to fetch the
  for i in s:
    t2=Thread(target=producer, args=(i,q,))     #Talking about threads being managed and put into queues
    ()                   #Start the producer thread
# () # Start the producer after giving up checking whether the thread is finished or not, for concurrency, because we are putting the thread into the queue for management, so do not have to wait for the end of the thread here, if we use join here will block our program. When the thread ends, the consumer notifies the parent process that the thread has ended.
  ()                     #Start the consumer thread
  ()                      #Thread blocking until fetch is complete

A quick note on the join method:

The call will cause the calling thread to block until the called thread finishes running or times out. The parameter timeout is a numeric value indicating the timeout period; if this parameter is not supplied, then the calling thread will block until the called thread finishes.

The above mentioned is the whole content of this article, I hope you will enjoy it.