Co-threading, also known as micro-threading, fiber-threading. English name Coroutine.
Concurrent threads are another way to implement multitasking in Python, except that they are smaller than threads and take up a smaller execution unit (understood as the resources needed).
Why it's an execution unit is because it comes with a CPU context. This allows us to switch from one co-program to another at the right time. As long as the CPU context is preserved or restored in the process, the program can still run.
Common understanding: a function in a thread, you can save the current function anywhere in some temporary variables and other information, and then switch to another function in the implementation, note that not by calling the function of the way to do so, and the number of times to switch and when to switch to the original function by the developer to determine their own.
Differences between concatenation and threading
When implementing multitasking, thread switching is much more than just saving and restoring the CPU context at the system level.
The operating system in order to run the program efficiently each thread has its own cache Cache and other data, the operating system will also help you do these data recovery operations, so the thread switching is very performance-consuming.
But the switching of concatenation is simply manipulating the context of the CPU, so the system can withstand millions of switches a second.
We've talked about the yield keyword before, and now we'll use it to implement multitasking.
Example:
import time
def task_1():
while True:
print("--1--")
(0.5)
yield
def task_2():
while True:
print("--2--")
(0.5)
yield
def main():
t1 = task_1()
t2 = task_2()
while True:
next(t1)
next(t2)
if __name__ == "__main__":
main()
Run the process:
First let t1 run for a while, when t1 encounters a yield, then return to the main() loop, then execute t2, when it encounters a yield, switch to t1 again, so that t1 and t2 are running alternately, which finally realizes multitasking and concurrency.
Run results:
greenlet
In order to make better use of concurrency for multitasking, the greenlet module in Python encapsulates it, making it easier to switch tasks.
First you need to install the greenlet module.
pip3 install greenlet
from greenlet import greenlet
import time
def test1():
while True:
print("---A--")
()
(0.5)
def test2():
while True:
print("---B--")
()
(0.5)
gr1 = greenlet(test1)
gr2 = greenlet(test2)
# Switch to gr1 and run
()
Run results:
It's basically the same as what we've done with yield before. greenlet is actually a simple encapsulation of yield.
The greenlet implementation of multitasking is simpler than yield, but we won't use it in the future.
The delay in the above example is 0.5 seconds, if the delay is 100 seconds, then the program will be stuck for 100 seconds, and even if there are other tasks that need to be performed, the system will not switch over to them, and these 100 seconds cannot be utilized.
This issue is addressed below.
gevent
greenlet has already implemented concurrency, but you still have to do manual switching, isn't that too much trouble.
Python also has a module called gevent that is more powerful than greenlet and can switch tasks automatically.
gevent is a repackaging of greenlet.
The principle is that when a greenlet encounters an IO (meaning input output input output, such as network, file operations, etc.) operation, such as accessing the network, it automatically switches to another greenlet, and waits until the IO operation is completed, and then switches back at the appropriate time to continue execution.
Since IO operations are very time-consuming and often leave the program in a waiting state, having gevent automatically switching between concatenations for us ensures that there is always a greenlet running instead of waiting for IO.
You still have to install gevent first.
pip3 install gevent
Example:
import gevent
def f(n):
for i in range(n):
print((), i)
g1 = (f, 3)
g2 = (f, 3)
g3 = (f, 3)
()
()
()
Results.
<Greenlet at 0x35aae40: f(3)> 0
<Greenlet at 0x35aae40: f(3)> 1
<Greenlet at 0x35aae40: f(3)> 2
<Greenlet at 0x374a780: f(3)> 0
<Greenlet at 0x374a780: f(3)> 1
<Greenlet at 0x374a780: f(3)> 2
<Greenlet at 0x374a810: f(3)> 0
<Greenlet at 0x374a810: f(3)> 1
<Greenlet at 0x374a810: f(3)> 2
As you can see, the three greenlets run sequentially rather than alternately.
It's not possible to tell if gevent is multitasking; the best case scenario is when 0 1 2 appear out of order in the run result.
In the gevent concept, we mentioned that gevent automatically switches tasks when it encounters a delay.
So, let's add a delay to the above example and then see the effect.
import gevent
import time
def f(n):
for i in range(n):
print((), i)
(0.5)
g1 = (f, 3)
g2 = (f, 3)
g3 = (f, 3)
()
()
()
Run results:
<Greenlet at 0x36aae40: f(3)> 0
<Greenlet at 0x36aae40: f(3)> 1
<Greenlet at 0x36aae40: f(3)> 2
<Greenlet at 0x384a780: f(3)> 0
<Greenlet at 0x384a780: f(3)> 1
<Greenlet at 0x384a780: f(3)> 2
<Greenlet at 0x384a810: f(3)> 0
<Greenlet at 0x384a810: f(3)> 1
<Greenlet at 0x384a810: f(3)> 2
After adding the delay, the results of the run did not change.
Actually, it's not the delay of () that gevent wants, it's the delay of ().
import gevent
def f(n):
for i in range(n):
print((), i)
(0.5)
g1 = (f, 3)
g2 = (f, 3)
g3 = (f, 3)
()
()
()
There is a simpler way to write join.
import time
import gevent
def f(n):
for i in range(n):
print((), i)
(0.5)
([
(f, 3),
(f, 3),
(f, 3)
])
It's usually written in this latter style.
Results.
<Greenlet at 0x2e5ae40: f(3)> 0
<Greenlet at 0x2ffa780: f(3)> 0
<Greenlet at 0x2ffa810: f(3)> 0
<Greenlet at 0x2e5ae40: f(3)> 1
<Greenlet at 0x2ffa780: f(3)> 1
<Greenlet at 0x2ffa810: f(3)> 1
<Greenlet at 0x2e5ae40: f(3)> 2
<Greenlet at 0x2ffa780: f(3)> 2
<Greenlet at 0x2ffa810: f(3)> 2
Now we finally have multitasking, and gevent automatically switches to another task when it encounters a delay.
This replaces sleep in time with sleep in gevent.
Then if you have a network program, there are also many blockages in the network program, such as connect, recv, accept, do you need to replace them with the corresponding methods in gevent.
Theoretically, you have to switch. If you want to use gevent, then you have to replace all the delay operations and blocking functions with the corresponding methods in gevent.
So here's a question, in case I've already written 100,000 lines of code, how does this switch up break ......
Is there any way to do this without manually modifying it, yes, with a patch.
import time
import gevent
from gevent import monkey
# Required for time-consuming operations
# Replace the time-consuming operations used in the program with modules implemented in gevent itself.
monkey.patch_all()
def f(n):
for i in range(n):
print((), i)
(0.5)
g1 = (f, 3)
g2 = (f, 3)
g3 = (f, 3)
()
()
()
monkey.patch_all() will automatically go through the code and replace all methods that cause delay blocking with methods from gevent.
Running results:
<Greenlet at 0x3dd91e0: f(3)> 0
<Greenlet at 0x3dd9810: f(3)> 0
<Greenlet at 0x3dd99c0: f(3)> 0
<Greenlet at 0x3dd91e0: f(3)> 1
<Greenlet at 0x3dd9810: f(3)> 1
<Greenlet at 0x3dd99c0: f(3)> 1
<Greenlet at 0x3dd91e0: f(3)> 2
<Greenlet at 0x3dd9810: f(3)> 2
<Greenlet at 0x3dd99c0: f(3)> 2
Summary:
Making use of all that time by using the delay to do other tasks is the biggest point of concurrency.
To this point this article on the concurrent Python multi-tasking resource-consuming way to achieve the smallest way to this article, more related Python multi-tasking resource-consuming way to the smallest way to search for my previous articles or continue to browse the following related articles I hope you will support me in the future more!