SoFunction
Updated on 2024-11-17

python decorator deep dive

What is a decorator

In our software products upgrade, often need to add new features to each function, and in our software products, the same function may be called hundreds of times, this situation is very common, if we modify one by one, that our code farmers do not want to hang up (some people say , you stupid ah, modify the definition of the function is not on the line! Students, you wake up, if you want to add a new function will modify the parameters, or return value?) . At this time, it is our decorator time to show their power. Decorator can be achieved without changing the original function call form (i.e., transparent processing of the function), to the function of the new function. How to achieve this, as well as the principle of realization, will be explained in detail below.

Principles followed by decorators

Decorator, as the name suggests, is to play a decorative role, since it is decorative, then the object to be decorated is what kind is what kind, can not have the slightest change.In this case, we are writing a decorator because we have to be sure that we can't modify the source code of the function being modified.How to follow this iron rule, we still need to also do some padding, must first understand three concepts, as follows:

Function names are "variables"

In python, a function name is actually like a function pointer in C. It represents the address of our function, and only when the interpreter gets that address will it execute the code in that block of memory. Therefore, in essence, the function name is no different from a different variable, except that the function name is used differently from the block of memory referred to by the normal variable. These are all determined by the underlying interpreter mechanism, which is transparent to the programmers, so we can assume that there is no difference between the two.

(math.) higher order function

What constitutes a higher-order function is actually quite simple, and it's good to grasp two principles:

  • Formal parameters are function names
  • The return value has the function name

As long as it satisfies one of these two principles, it can be called a higher-order function. Turning back, the function name we talked about above appears here, and realizing that, aren't we treating it as a real parameter here?

nested lettercriticize (i.e. enumerate shortcomings)

What is a nested function is actually very simple to grasp a principle:

  • Defining a function within the body of another function.

It is important to emphasize here that the function body is not executed when the function is defined, in the same way that defining a variable does not read the contents of the variable. This is a crucial point that is very helpful in understanding the principles of decorator implementation.

How to write a decorator

With the above out of the way, it's much easier to understand how to write a decorator when it comes to detailing it now.

Decorator Essence

Actually, a decorator is essentially a function, which also has a function name, arguments and return value. But in python, we use "@auth" for that.

@auth # Its equivalent: func = auth(func)
def func():
 print("func called")

This example is how to modify the format of the func function in python, but of course we haven't implemented our decorator function yet. What we have to pay attention to is what is written in the comments, as we can see:

  • A decorator function is actually a higher-order function (the arguments and return value are function names).
  • "auth(func)" is calling our decorator function, i.e., the function body of the decorator function will be executed, make sure you remember this.

Design Ideas

Since a decorator is a function and has the equivalence introduced above, we can design our decorator in this way:

  • Define a new function within the body of our decorator function, call the modified function within this newly defined function, and at the same time, add new functionality in the context of the modified function. Finally, use the return value of the decorator function to return the function name of our newly defined function.
  • It follows that the return value func in "func = auth(func)" represents the function name of the newly defined function in the decorator.

A lot of padding was done earlier to reveal the implementation mechanism of the decorator here, which is nothing really, it's very simple:

  • The decorator mechanism changes the address data represented by the function name of the modified function.To be clear, before the modification, the function name represents memory block A; after the modification, the function name represents memory block B; it is just that, when executing memory block B, it will call memory block A. The code in memory block B is the new function we added.This mechanism is realized using the mechanism of "higher-order functions" and "nested functions".
  • The net effect is that, but when calling the modified function, you are actually calling not the original block of memory, but a newly requested block of memory by the modifier.

Step one:Designing Decorator Functions

Decorator function definition and ordinary function definition is not much different, the key is the function body how to write the problem. Here, to make it easier to understand, the first parameterless decorator function is used to illustrate.

# Decorator function definition format
def deco(func):
 '''Function body...'''
return func

By no arguments, we mean no arguments other than "func".
The hard part is the writing of the function body, and the following example shows you why the second step is necessary in the first place:

# Use the syntactic sugar @ to decorate the function, equivalent to "myfunc = deco(myfunc)"
def deco(func):
 print("before myfunc() called.")
 func()
 print("after myfunc() called.")
 return func
 
@deco
def myfunc():
 print("myfunc() called.")
 
myfunc()
myfunc()
 
#output:
before myfunc() called.
myfunc() called.
after myfunc() called.
myfunc() called.
myfunc() called. 

As you can see from the output, our decorator didn't take effect. Don't tell me that the decorator only works once, that's because people ignore the equivalent mechanism of "@deco". The interpretation of "@deco" is "myfunc = deco(myfunc)". Note that, as I mentioned earlier, the deco function is actually being called here, so the deco function will be executed. So the first three lines of output are not the effect of the call to the myfunc function, so how can we say that the decorator took effect once? The second step is to solve the problem of the decorator not taking effect.

Step two:Wrapping Modified Functions

# Basic format
def deco(func):
 def _deco()
 #New Features
 #...
 #...
 func() #Alternate modifier function calls
 return_deco

An example is given below:

# Use inline wrapper functions to ensure that each new function is called.
#The formal parameters and return values of an embedded wrapper function are the same as those of the original function, and the decorated function returns the embedded wrapper function object.
 
def deco(func):
 def _deco():
 print("before myfunc() called.")
 func()
 print("after myfunc() called.")
 # Instead of returning func, you should actually return the return value of the original function
 return _deco
 
@deco
def myfunc():
 print("myfunc() called.")
 return 'ok'
 
myfunc()
 
#output:
before myfunc() called.
myfunc() called.
after myfunc() called.

Step Three:Transparent Handling of Modified Function Arguments and Return Values

When the second step is completed, the decorator has actually completed the main part, the following is the processing of the parameters and return values of the modified function. This can really realize the irony of the decorator. Without further ado, directly on the code:

# Basic format
def deco(func):
 def _deco(*args, **kwargs) # Parameter transparency
 #New Features
 #...
 #...
 res = func(*args, **kwargs) #Alternate modifier function calls
 return res # Return value transparency
 return_deco

Know by the above analysis:

Parameter Transparency:When we call the decorated function, we are actually calling the _deco function here. So, we'll just add variable parameters to the _deco function and pass the resulting variable parameters to the func function.
Return value transparency:In the same way as parameter transparency, defining a return value for the _deco function and returning the return value of func is sufficient.

Transparent processing is that simple! At this point, we are done writing our decorator. Let's give an example:

# Decorating functions with parameters.
#The formal parameters and return values of an embedded wrapper function are the same as those of the original function, and the decorated function returns the embedded wrapper function object.
 
def deco(func):
 def _deco(*agrs, **kwagrs):
 print("before myfunc() called.")
 ret = func(*agrs, **kwagrs)
 print(" after myfunc() called. result: %s" % ret)
 return ret
 return _deco
 
@deco
def myfunc(a, b):
 print(" myfunc(%s,%s) called." % (a, b))
 return a + b
 
print("sum=",myfunc(1, 2))
print("sum=",myfunc(3, 4))
 
#output:
before myfunc() called.
 myfunc(1,2) called.
 after myfunc() called. result: 3
sum= 3
before myfunc() called.
 myfunc(3,4) called.
 after myfunc() called. result: 7
sum= 7

Decorator Advanced

parameterized decorator

Decorators are also functions, so we can pass parameters to them. I'm talking about "@auth(auth_type = 'type1')" here. Let's start with the code:

# Basic format
def deco(deco_type)
 def _deco(func):
 def __deco(*args, **kwargs) # Parameter transparency
  #New Features
  #...
  #...
  print("deco_type:",deco_type) #Use decorator parameters
  res = func(*args, **kwargs) #Alternate modifier function calls
  return res # Return value transparency
 return __deco
 return_deco 

To be clear, it is based on the original decorator and then in the outermost set of a deco function, and use it to receive decorator parameters. Because it is in the outermost set of a function, then the scope of this function's formal parameter is the function body inside, so inside the definition of the function to be used casually, it is so arbitrary.
So how to understand the parsing process of the interpreter? Here, if we understand one thing, that is: "@auth(auth_type = 'type1')" is equivalent to "func = auth(auth_type = 'type1')(func)"The interpreter will first translate "auth(auth_type = 'type1')", then treat its return value (assuming it is given the non-existent function name of _func) as a function pointer, where the _func function name stands for __deco, and then go on to execute " func = _func(func)", and this func function name represents actually __deco.

So far, the purpose of passing a parameter through a decorator has been achieved. Let's give an example:

# Example 7: Building on Example 4, have the decorator take parameters.
# Compared to the previous example there is an extra layer of packaging on the outer layer.
# Decorative function names should actually make more sense.
 
def deco(deco_type):
 def _deco(func):
 def __deco(*args, **kwagrs):
  print("before %s called [%s]." % (func.__name__, deco_type))
  func(*args, **kwagrs)
  print(" after %s called [%s]." % (func.__name__, deco_type))
 return __deco
 return _deco
 
@deco("mymodule")
def myfunc():
 print(" myfunc() called.")
 
@deco("module2")
def myfunc2():
 print(" myfunc2() called.")
 
myfunc()
myfunc2()
 
#output:
before myfunc called [mymodule].
 myfunc() called.
 after myfunc called [mymodule].
before myfunc2 called [module2].
 myfunc2() called.
 after myfunc2 called [module2].

Multi-decorator modifier functions

If what I've said above is understood, then this thing, it's so simple. Isn't it just taking the function that's being modified in our is decorator and decorating it? But what I also want to say here is that we look at it from a different perspective. We focus on the original function that is modified, we will find that NB ah, I can add a number of functions to it. Let's give an example:

def deco(deco_type):
 def _deco(func):
 def __deco(*args, **kwagrs):
  print("before %s called [%s]." % (func.__name__, deco_type))
  func(*args, **kwagrs)
  print(" after %s called [%s]." % (func.__name__, deco_type))
 return __deco
 return _deco
 
@deco("module1")
@deco("mymodule")
def myfunc():
 print(" myfunc() called.")
 
@deco("module2")
def myfunc2():
 print(" myfunc2() called.")
 
myfunc()
 
#output:
before __deco called [module1].
before myfunc called [mymodule].
 myfunc() called.
 after myfunc called [mymodule].
 after __deco called [module1].

Notice the result yo, @deco("module1"), to modify the deco("mymdule")'s, just as we thought, perfect!

Get the dry goods:Video tutorials for learning python with zero beginner's knowledge

This is the whole content of this article.