Today we will talk about a topic that every developer cares about - how to make Python code run faster! For many Python beginners, they may not have been exposed to many performance optimization techniques, only knowing that "Python is slow", but in fact, you can use some simple optimization techniques to make your code fly, bringing huge improvements to yourself and your team.
Next, let’s see how to improve the speed of Python code!
1. Avoid unnecessary loops and redundant calculations
First, let’s start with the most basic ones. Have you found that some code seems to do a lot of repetitive calculations when executed, and these calculations are completely unnecessary?
example:
Suppose we have a program that needs to sum a set of data, and the code is written like this:
def calculate_sum(data):
total = 0
for i in range(len(data)):
total += data[i]
return total
It seems that nothing is wrong, right? But we pass every timedata[i]To access elements, this is actually very slow. Can you guess how to optimize?
optimization:
We can usesum()Function, which is built in Python, is much faster than manual calculations:
def calculate_sum(data):
return sum(data)
Simple? It doesn't seem special, but you'll find that running this kind of code will increase the speed of orders of magnitude!
principle:
sum()It is a built-in function in Python, and it has done a lot of optimizations internally, such as implementing it in C language, which is why it is much faster than the loop we handwritten. Therefore, for most simple mathematical operations, Python built-in functions are usually the optimal solution.
2. Use List Comprehensions instead of ordinary loops
Let's take a look at a common optimization scenario: if you use a normal for loop to create a list, it will actually be slower than list comprehension.
example:
Suppose we need to generate a list of 1 million square numbers:
def generate_squares(n):
squares = []
for i in range(n):
(i**2)
return squares
This method is correct, but it is a bit "cumbersome". Can you think of a simpler and more efficient way?
optimization:
We can replace ordinary loops with list generation:
def generate_squares(n):
return [i**2 for i in range(n)]
principle:
Behind the list generation formula is actually accelerated through efficient memory management. It has much less unnecessary memory allocation than traditional loop operations, so the speed is faster.
3. Avoid frequent memory allocation
The overhead of memory allocation is very large for programs. Frequent allocation and recycle memory is like your moving house, which is troublesome and wastes time. If your code frequently creates temporary objects or data structures in certain places, doing so will degrade performance.
example:
Suppose we need to create a list with a length of 1 million, we useforLoops continuously add elements to the list:
def create_list(n):
result = []
for i in range(n):
(i)
return result
This method is a little problem because every callappend()When Python needs to dynamically resize the list in memory.
optimization:
We can pre-allocate the list size at one time, which avoids the overhead of multiple memory allocations:
def create_list(n):
result = [None] * n # Preallocate memory for i in range(n):
result[i] = i
return result
principle:
By allocating memory in advance, you reduce the need for frequent memory re-allocation and significantly improve performance.
4. Use generator instead of list
Speaking of memory overhead, another noteworthy thing is: if you process a very large data set, you don't need to load all the data into memory at once. In this case, generators are a very good choice.
example:
Suppose you want to read a large file and process the data line by line:
def read_file(filename):
with open(filename, 'r') as f:
data = () # Read all lines into memory at once return data
If the file is particularly large, it is not ideal to load all the data into memory at once.
optimization:
It can be changed to a generator to load data as needed, so that only one row is loaded at a time, saving memory:
def read_file(filename):
with open(filename, 'r') as f:
for line in f:
yield line # Return one row of data each timeprinciple:
The advantage of the generator is that it is lazy to load and will only read a line from the file if you need data. This avoids loading all data into memory at once, saving a lot of memory.
5. Use multi-threading or multi-processing to process tasks in parallel
Finally, let’s talk about how to make Python more “capable” to do multiple things at the same time. As you may know, Python's GIL (global interpreter lock) will limit concurrent execution of multiple threads, but this does not mean that you cannot use multi-threading or multi-processing to improve performance!
example:
Suppose we need to download multiple files, and if we download them one by one in order, the speed will be very slow:
import time
def download_file(url):
(1) # Simulation download print(f"Download completed: {url}")
urls = ["url1", "url2", "url3", "url4"]
for url in urls:
download_file(url)
This method is very intuitive, but it takes 1 second to download a file. If there are hundreds of documents, wouldn’t it take a long time to wait?
optimization:
We can use multiple threads or multiple processes to download multiple files at the same time to quickly complete tasks:
import threading
def download_file(url):
(1)
print(f"Download completed: {url}")
urls = ["url1", "url2", "url3", "url4"]
threads = []
for url in urls:
thread = (target=download_file, args=(url,))
()
(thread)
for thread in threads:
()
principle:
Through multithreading, programs can execute multiple tasks at the same time, reducing the waiting time. If you want to speed up further, consider using multi-processes, especially in compute-intensive tasks.
Summarize
Optimizing Python code is not a masterpiece. By reducing unnecessary computation, using Python's built-in tools, making rational use of memory and threads, etc., you can greatly speed up your code. The 5 tips we discussed today seem simple, but each can bring a significant improvement.
If you still think these tips seem useless now, try using them to optimize the code you have at hand and you will definitely see the results.
This is the article about 5 Python performance optimization tips sharing (code speedup by 300%). For more related Python performance optimization content, please search for my previous articles or continue browsing the related articles below. I hope everyone will support me in the future!