Share eight practical tips to speed up Python

1. Use built-in functions and libraries

Python's built-in functions are implemented in C language and run much faster than pure Python code.

# Slow writingresult = []
for item in iterable:
    (func(item))
    
# Quick writing - Use map functionresult = list(map(func, iterable))

# Or use list comprehensionresult = [func(item) for item in iterable]

2. Utilize the JIT compiler - Numba

Numba is a JIT (instant) compiler that can compile Python functions into machine code.

from numba import jit
import numpy as np

@jit(nopython=True)
def sum_array(arr):
    total = 0.0
    for i in range([0]):
        total += arr[i]
    return total

large_array = (10000000)
print(sum_array(large_array))

3. Use multi-process to handle CPU-intensive tasks

Python has GIL (global interpreter lock), multithreading is not suitable for CPU-intensive tasks, and multiprocessing is a better choice.

from multiprocessing import Pool

def process_data(data):
    # Data processing logic    return result * 2

if __name__ == '__main__':
    data = [1, 2, 3, 4, 5, 6, 7, 8, 9, 10]
    with Pool(4) as p:  # Use 4 processes        results = (process_data, data)
    print(results)

4. Compile Python into C with C

Cython allows you to write C extension modules to significantly improve performance.

# Save asdef compute(int n):
    cdef int i
    cdef double res = 0.0
    for i in range(n):
        res += i * i
    return res

Then create:

from  import setup
from  import cythonize

setup(ext_modules=cythonize(''))

Compile and install:

python  build_ext --inplace

5. Use efficient data structures

Choosing the right data structure can greatly improve performance.

# Frequent member checks to use sets instead of listslarge_list = list(range(1000000))
large_set = set(large_list)

#Slow speedif 999999 in large_list:  # O(n)
    pass
    
# fastif 999999 in large_set:  # O(1)
    pass

6. Use NumPy and Pandas for vectorization operations

Avoid Python-level loops and use vectorized operations.

import numpy as np

#Slow - Python Loopdef slow_dot(a, b):
    result = 0
    for x, y in zip(a, b):
        result += x * y
    return result

# Fast - NumPy vectorizationdef fast_dot(a, b):
    return (a, b)

a = (1000000)
b = (1000000)

%timeit slow_dot(a, b)  # About 500ms%timeit fast_dot(a, b)  # About 2ms

7. Use lru_cache to cache the function results

For calculation-intensive functions that frequently use the same parameters, using caches can avoid repeated calculations.

from functools import lru_cache

@lru_cache(maxsize=128)
def expensive_function(x, y):
    # Simulate complex calculations    result = 0
    for i in range(x):
        for j in range(y):
            result += i * j
    return result

# The first call will perform calculationsprint(expensive_function(100, 100))
# Call the same parameter again and will directly return the cached resultprint(expensive_function(100, 100))

8. Avoid unnecessary global variable access

Local variable access is much faster than global variables.

#Slow - Frequent access to global variablesglobal_var = 10

def slow_func():
    total = 0
    for i in range(1000000):
        total += global_var
    return total

# Quick - Using local variablesdef fast_func():
    local_var = global_var
    total = 0
    for i in range(1000000):
        total += local_var
    return total

%timeit slow_func()  # About 80ms%timeit fast_func()  # About 50ms

Summarize

Preferred use of built-in functions and libraries

Logarithmic calculations use Numba JIT

CPU-intensive tasks use multi-process

Cython compiled

Select an efficient data structure

Vectorization operations using NumPy/Pandas

Cache function results to avoid repeated calculations

Reduce global variable access

Choose the right optimization method according to your specific application scenario, which can usually bring performance improvements of several times to hundreds of times! Remember to analyze performance bottlenecks before optimizing, and use tools like cProfile to find out what really needs to be optimized.

The above is the detailed content of eight practical techniques to speed up Python. For more information about Python acceleration running skills, please follow my other related articles!