Python source code learning PyObject and PyTypeObject

preamble

Python is a C implementation, so thePython objects at the C level should be aconstructor It organizes the memory occupied by the objects. Different types of objects may have different data and behavior, so you can hazard a guess:Different types of objects are represented by different structures。

Objects also have some commonalities, such as the need for each object to have areference countthat is used to implement the garbage collection mechanism. Therefore, further speculation is possible:Structures representing objects have a common header。

I. Cornerstones of Instance Objects - PyObject and PyVarObject

PyObject and PyVarObject are essentially objects of theHeader information。

1.1 PyObject Structures

Python objects are all controlled by thePyObjectStructures are represented, and object references are pointersPyObject *。 PyObjectThe structure is defined in the header file at pathInclude/The code is as follows

typedef struct _object {
    _PyObject_HEAD_EXTRA
    Py_ssize_t ob_refcnt;
    struct _typeobject *ob_type;
} PyObject;

describing the elements in the structure.

1.2 Definition of Macros

#ifdef Py_TRACE_REFS
/* Define pointers to support a doubly-linked list of all live heap objects. */
#define _PyObject_HEAD_EXTRA            \
    struct _object *_ob_next;           \
    struct _object *_ob_prev;

#define _PyObject_EXTRA_INIT 0, 0,

#else
#define _PyObject_HEAD_EXTRA
#define _PyObject_EXTRA_INIT
#endif

in the event thatPy_TRACE_REFSis defined, the macro expands to two pointersob_nextcap (a poem)ob_prevUsed to implement bi-directional chained tables. The note explains that bidirectional linked tables are used to keep track of all active heap objects and are generally not enabled and are not covered in depth.

1.3 PyVarObject Structures

for the purpose of expressingvariable length object(used form a nominal expression)PyVarObjectStructures are created in thePyObjectstructure is based on adding length information.

typedef struct {
    PyObject ob_base;
    Py_ssize_t ob_size; /* Number of items in variable part */
} PyVarObject;

compareobjectThe structure adds theob_sizefield is used to record the number of elements.

1.4 Two header information macro definitions and their initialization

Depending on whether its memory size is fixed or not, a concrete instance object is either a fixed-length object or a variable-length object. Accordingly, it needs to have header informationPyObjectmaybePyVarObject。

Therefore, the header file prepares two macro definitions for header informationPyObject_HEADcap (a poem)PyObject_VAR_HEADthat facilitates the use of the object.

#define PyObject_HEAD          PyObject ob_base;
#define PyObject_VAR_HEAD      PyVarObject ob_base;

Macro Definition Description.

#define PyObject_HEAD PyObject ob_base;
Indicates that other occurrences of the code will bePyObject_HEADlocalities，replace withPyObject ob_base;

1.4.1 Fixed-length object implementation

The implementation of a floating-point class with a fixed memory size only needs to be implemented in thePyObjectThe header base is implemented with a double-precision floating-point number, double.

typedef struct {
    PyObject_HEAD

    double ob_fval;
} PyFloatObject;

1.4.2 Variable-length object implementation

List objects with variable memory sizes need to be added to thePyVarObjectThe header is implemented as a dynamic array of objects contained in the list, i.e., the PyObject pointer.

typedef struct {
    PyObject_VAR_HEAD

    PyObject **ob_item;
    Py_ssize_t allocated;
} PyListObject;

The underlying PyListObject is implemented as an array with the following 3 key fields.

Python automatically expands the list when it runs out of capacity, see the list source code for an explanation of the mechanism.

1.4.3 Header information macro initialization

PyObject_HEAD_INITuse fortarget of fixed lengthHeader information initialization. Initialize the reference countob_refcntSet to 1 and set the object typeob_typeSet to the given type.

#define PyObject_HEAD_INIT(type)        \
    { _PyObject_EXTRA_INIT              \
    1, type },

PyVarObject_HEAD_INITuse forvariable length objectHeader information initialization. Further setting of the length field on top of the formerob_size。

#define PyVarObject_HEAD_INIT(type, size)       \
    { PyObject_HEAD_INIT(type) size },

These two macro definitions are often seen in the source code.

II. The Cornerstone of Type Objects - PyTypeObject 2.1 PyTypeObject Contained Information

PyObjectRecords information common to all objects in Python. Such as reference counts, type pointers, and the number of elements specific to variable-length objects. But there are a few more details to consider that

• How to know the memory information required for an object when creating different types of objects
• Given an object, how to determine what operations it supports

These are used as objects of themeta-information , should be kept by a separate entity, closely related to the type to which the object belongs.PyObjectinclusiveob_typePointer to atyped objectThe Type ObjectPyTypeObjectalso inInclude/as defined in the following key fields.

typedef struct _typeobject {
    PyObject_VAR_HEAD
    const char *tp_name; /* For printing, in format "<module>.<name>" */
    Py_ssize_t tp_basicsize, tp_itemsize; /* For allocation */

    /* Methods to implement standard operations */
    destructor tp_dealloc;
    printfunc tp_print;

    getattrfunc tp_getattr;
    setattrfunc tp_setattr;

    // ...
    /* Attribute descriptor and subclassing stuff */
    struct _typeobject *tp_base;

    // ......
} PyTypeObject;

typed objectPyTypeObjectis a variable-length object containing information about the header of the variable-length objectPyObject_VAR_HEADand proprietary fields.

PyTypeObjectjust liketyped objectThe Python representation corresponds to the object-oriented notion of "class".PyTypeObjectThe structure is complex, and for now it's enough to know that it holds meta-information about the object, describing the type of the object.

2.2 Relationship between type and instance objects in memory

Examining the morphology and relationship between type objects and instance objects in memory, using float as an example.

>>> float
<class 'float'>
>>> pi = 3.14
>>> e = 2.71
>>> type(pi) is float
True

• Two floats.instance objectallPyFloatObjectconstructorexcept for the public header fieldob_refcntcap (a poem)ob_typeProprietary Fieldsob_fvalThe corresponding values are saved.
• typed objectanPyTypeObjectconstructor, which holds type names, memory allocation information, and floating-point-related operations.instance object of theob_typeFields point to type objectsPython can then determine the type of the object and learn meta-information about the object.
• Variables such as float, pi, and e are simply a pointer to the actual object.

The chart above is not entirely correct, see a later blog post for a more in-depth explanation.

to this article on Python source code to learn PyObject and PyTypeObject article is introduced to this, more related to PyObject and PyTypeObject content, please search for my previous articles or continue to browse the following related articles I hope you will support me in the future more!