How to write extension libraries for Python in C code (Cython)

A previous article mentioned the use of Cython to compile Python, this time to talk about how to use Cython to write extension libraries for Python.

Programming in a mixture of both languages, where the most important thing is the passing of types.

Let's get started with a simple example: this time the goal is to write a Numpy addition and element multiplication module in C. In this example, Numpy's array is passed inside the C module as a two-dimensional array.

1. Headers:

#ifndef _MAIN_H
#define _MAIN_H
void plus(double *a, double *b, double *r, int n, int m); // Matrix addition
void mul(double *a, double *b, double *r, int n, int m); // Multiply matrices by their elements
void main(double *a, double *b, double *r, int n, int m, int times); // main function for testing
#endif

2. Write the main code in:

#include ""
 
/***********************************
* Addition of matrices
* Taking advantage of the fact that arrays are stored sequentially, *
* Access 2D arrays by dimensionality reduction! *
* r
***********************************/
void plus(double *a, double *b, double *r, int n, int m)
{
  int i, j;
  for(i = 0; i < n; i++)
  {
    for(j = 0; j < m; j++)
      *(r + i*m + j) = *(a + i*m + j) + *(b + i*m + j);
  }
}
 
/***********************************
* Element-wise multiplication of matrices
* Taking advantage of the fact that arrays are stored sequentially, *
* Access 2D arrays by dimensionality reduction! *
* r
***********************************/
void mul(double *a, double *b, double *r, int n, int m)
{
  int i, j;
  for(i = 0; i < n; i++)
  {
    for(j = 0; j < m; j++)
      *(r + i*m + j) = *(a + i*m + j) * *(b + i*m + j);
  }
}
 
/***********************************
* main function
* Taking advantage of the fact that arrays are stored sequentially, *
* accesses a two-dimensional array by dimensionality reduction! *
* r
***********************************/
void main(double *a, double *b, double *r, int n, int m, int times)
{
  int i;
  // Cycle times
#pragma omp parallel for
  for (i = 0; i < times; i++)
  {
    // Addition of matrices
    plus(a, b, r, n, m);
    
    // Multiply matrices by their elements
    mul(a, b, r, n, m);
  }
}

In this implementation of the matrix addition, matrix multiplication by elements of the function, the data structure used is a two-dimensional array, but because the C language to the function to pass a two-dimensional array is more troublesome, here with the method of dimensionality reduction to achieve. In addition, in the main() function, a loop is used to test the performance.

3. The following write a file to call the above C function (note that the suffix is .pyx oh): detailed knowledge in the notes written out ~ ~

# Both import numpy, and cimport numpy
import time
import numpy as np
cimport numpy as np
 
# Using the Numpy-C-API
np.import_array()
 
# cdefine C function
cdef extern from "":
  void plus(double *a, double *b, double *r, int n, int m)
  void mul(double *a, double *b, double *r, int n, int m)
  void main(double *a, double *b, double *r, int n, int m, int times)
 
"""
# Define a "wrapper function", used to call the C main function, call example: plus_fun(a, b, r)
# Here it is important to note the type declaration of the parameters passed to the function. double means that the elements of the array are of type double.
# ndim = 2 means the dimension of the array is 2
# When calling the main function, force the python variables to be converted to the appropriate type (to ensure no errors), e.g. <int>.
# Of course, basic types such as int can be written without explicitly, as in the following [0], [1]
"""
def main_func([double, ndim=2, mode="c"] a not None,
           [double, ndim=2, mode="c"] b not None, 
           [double, ndim=2, mode="c"] r not None,
           times not None):
  main(<double*> np.PyArray_DATA(a),
        <double*> np.PyArray_DATA(b),
        <double*> np.PyArray_DATA(r),
        [0],
        [1],
        <int> times)

4. To compile the above code in Cython, we create a file:

import numpy
from  import setup
from  import Extension
from  import build_ext
 
filename = 'test' # Source file name
full_filename = '' # Source file name with suffix
 
setup(
  name = 'test',
  cmdclass = {'build_ext': build_ext},
  ext_modules=[Extension(filename,sources=[full_filename, ""],
         include_dirs=[numpy.get_include()])],
)

5. The above, , must be placed in the same folder. At this point in that folder hold down the shift key and then right-click the mouse to open the cmd or PowerShell console and run the following command in the console for Cython compilation:

python build_ext --i

Or:

python build_ext --inplace

Example of a successful compilation:

At this time in the same directory will generate "test.cp36-win_amd64.pyd" binary code file, it is closed source, but can be directly imported with python. the following test code to test:

import test
import time
import numpy as np
 
start_time = ()
a = (100, 100) * 2 - 1 # Generate 300*300 random matrices
b = (100, 100) * 2 - 1
r = np.empty_like(a) # Create an empty matrix to store the results of the calculation
test.main_func(a, b, r, 500000) # Call main_func to test
end_time = ()
print(end_time - start_time) # Output time
print(r) # Output the results of the run

Implementation results:

As we can see with this example: placing the loop in a C module instead of the native Python improves execution efficiency.

This is the whole content of this article.