Debugger is a very important part of a programming language. A debugger is a tool used to diagnose and fix code errors (or bugs), which allows developers to step-by-step view and analyze the state and behavior of the code while the program is executing, it helps developers to diagnose and fix code errors, understand the behavior of the program, and optimize the performance. Regardless of the programming language, the debugger is a powerful tool that plays an active role in improving development efficiency and code quality.
In this article mainly to introduce you to the python language in the implementation of the debugger principle, through the understanding of the implementation of a language debugger principle we can more in-depth understanding of the operation of the whole language mechanism, can help us to better understand the execution of the program.
Stop the program.
If we need to debug a program one of the most important points is if we stop the program, only by stopping the execution of the program we can observe the state of the program execution, for example, we need to debug the 99 multiplication table:
def m99():
for i in range(1, 10):
for j in range(1, i + 1):
print(f"{i}x{j}={i*j}", end='\t')
print()
if __name__ == '__main__':
m99()
Now execute the commandpython -m pdb It will be possible to debug the above program:
(py3.8) ➜ pdb_test git:(master) ✗ python -m pdb
> /Users/xxxx/Desktop/workdir/dive-into-cpython/code/pdb_test/(3)<module>()
-> def m99():
(Pdb) s
> /Users/xxxx/Desktop/workdir/dive-into-cpython/code/pdb_test/(10)<module>()
-> if __name__ == '__main__':
(Pdb) s
> /Users/xxxx/Desktop/workdir/dive-into-cpython/code/pdb_test/(11)<module>()
-> m99()
(Pdb) s
--Call--
> /Users/xxxx/Desktop/workdir/dive-into-cpython/code/pdb_test/(3)m99()
-> def m99():
(Pdb) s
> /Users/xxxx/Desktop/workdir/dive-into-cpython/code/pdb_test/(4)m99()
-> for i in range(1, 10):
(Pdb) s
> /Users/xxxx/Desktop/workdir/dive-into-cpython/code/pdb_test/(5)m99()
-> for j in range(1, i + 1):
(Pdb) s
> /Users/xxxx/Desktop/workdir/dive-into-cpython/code/pdb_test/(6)m99()
-> print(f"{i}x{j}={i*j}", end='\t')
(Pdb) p i
1
(Pdb)
Of course you can also debug in the IDE:
According to our debugging experience, it is easy to know that in order to debug a program, the first and most important thing is that the program needs to be able to stop at the location where we set the breakpoint.
cpython king bomb mechanism -- tracing
Now the question is, how does the above program stop while the program is executing?
According to the previous study, we can understand that the execution of a python program first needs to be compiled into python byte code by the python compiler, and then handed over to the python virtual machine for execution. If the program needs to stop, it must need the virtual machine to provide an interface to the upper python program, so that the program can be executed in the execution of the program can be known to the current position program to know where it is now. This mysterious mechanism is hidden in the sys module, which in fact does almost all of the interfacing with the python interpreter. A very important function that implements the debugger is the function that will set up a trace function for the thread, which will be executed when the virtual machine has a function call, when it executes a line of code, or even when it executes a bytecode.
Setting up a system trace function allows a Python source code debugger to be implemented in Python. This function is thread-specific; to support multithreaded debugging, you must register a trace function for each thread being debugged, either with settrace() or with ().
The trace function should have three arguments: frame, event, and arg. frame is the current stack frame. event is a string: 'call', 'line', 'return', 'exception', 'opcode', 'c_call', or 'c_exception'. arg depends on the event type. arg depends on the event type.
The trace function is called each time it enters a new local scope (with the event set to 'call'); it should return a reference to the local trace function for the new scope, or None if it does not want to trace in that scope.
If any error occurs in the trace function, it will be unsettled, just like calling settrace(None).
The meaning of the event is as follows:
- call, which calls a function (or enters another block of code). Calls a global trace function; arg is None; the return value specifies a local trace function.
- line, a new line of code will be executed, and the value of the argument arg is None.
- return, the function (or other block) is about to return. Calls the local trace function; arg is the value to be returned, or None if the event was caused by a raised exception. the return value of the trace function is ignored.
- exception, an exception has occurred. Calls the local traceback function; arg is a tuple (exception, value, traceback); the return value specifies the new local traceback function.
- opcode, the interpreter is about to execute a new bytecode instruction. Invoke the local trace function; arg is None; the return value specifies the new local trace function. By default, per-opcode events are not emitted: they must be explicitly requested by setting f_trace_opcodes to True on the frame.
- c_call, a c function is about to be called.
- c_exception, an exception was thrown when the c function was called.
Do-it-yourself implementation of a simple debugger
In this section we will implement a very simple debugger to help you understand the principle of debugger implementation. Debugger implementation code is shown below, only a few dozen lines can help us go deeper to understand the principle of the debugger, we first look at the effect of the implementation of the latter in the text and then go to analyze the specific implementation:
import sys
file = [1]
with open(file, "r+") as fp:
code = ()
lines = ("\n")
def do_line(frame, event, arg):
print("debugging line:", lines[frame.f_lineno - 1])
return debug
def debug(frame, event, arg):
if event == "line":
while True:
_ = input("(Pdb)")
if _ == 'n':
return do_line(frame, event, arg)
elif _.startswith('p'):
_, v = _.split()
v = eval(v, frame.f_globals, frame.f_locals)
print(v)
elif _ == 'q':
(0)
return debug
if __name__ == '__main__':
(debug)
exec(code, None, None)
(None)
The following is used in the above program:
- Type n to execute a line of code.
- p name Prints the variable name.
- q Exit debugging.
Now we execute the above program for program debugging:
(py3.10) ➜ pdb_test git:(master) ✗ python
(Pdb)n
debugging line: def m99():
(Pdb)n
debugging line: if __name__ == '__main__':
(Pdb)n
debugging line: m99()
(Pdb)n
debugging line: for i in range(1, 10):
(Pdb)n
debugging line: for j in range(1, i + 1):
(Pdb)n
debugging line: print(f"{i}x{j}={i*j}", end='\t')
1x1=1 (Pdb)n
debugging line: for j in range(1, i + 1):
(Pdb)p i
1
(Pdb)p j
1
(Pdb)q
(py3.10) ➜ pdb_test git:(master) ✗
You can see that our program is truly debugged.
Now let's analyze our own implementation of the simple version of the debugger, in the previous section we have already mentioned the function, the call to this function needs to pass a function as a parameter, the incoming function needs to accept three parameters:
- frame, the stack frame currently being executed.
- event, the category of the event, as mentioned in the previous document.
- arg, parameter which was also mentioned earlier.
- It should also be noted that this function also needs to have a return value, the python VM will call the returned function when the next event occurs, if it returns None then it will not call the tracing function when the event occurs, this is the reason why the code in debug returns debug.
We only process the line event, and then we do a dead loop that only executes the next line when the n instruction is typed, and then prints the line being executed, which exits the debug function, and the program continues to execute. python has a built-in eval function that can retrieve the value of a variable.
python official debugger source code analysis
The official python debugger is pdb, which comes with the python standard library.python -m pdb Go to the debug file . Here we only analyze the core code:
Code Location: The following Bdb class
def run(self, cmd, globals=None, locals=None):
"""Debug a statement executed via the exec() function.
globals defaults to __main__.dict; locals defaults to globals.
"""
if globals is None:
import __main__
globals = __main__.__dict__
if locals is None:
locals = globals
()
if isinstance(cmd, str):
cmd = compile(cmd, "<string>", "exec")
(self.trace_dispatch)
try:
exec(cmd, globals, locals)
except BdbQuit:
pass
finally:
= True
(None)
The above function is mainly a tracing operation using a function to capture events as they occur. In the above code, the tracing function is self.trace_dispatch. Let's look at the code for this function:
def trace_dispatch(self, frame, event, arg):
"""Dispatch a trace function for debugged frames based on the event.
This function is installed as the trace function for debugged
frames. Its return value is the new trace function, which is
usually itself. The default implementation decides how to
dispatch a frame, depending on the type of event (passed in as a
string) that is about to be executed.
The event can be one of the following:
line: A new line of code is going to be executed.
call: A function is about to be called or another code block
is entered.
return: A function or other code block is about to return.
exception: An exception has occurred.
c_call: A C function is about to be called.
c_return: A C function has returned.
c_exception: A C function has raised an exception.
For the Python events, specialized functions (see the dispatch_*()
methods) are called. For the C events, no action is taken.
The arg parameter depends on the previous event.
"""
if :
return # None
if event == 'line':
print("In line")
return self.dispatch_line(frame)
if event == 'call':
print("In call")
return self.dispatch_call(frame, arg)
if event == 'return':
print("In return")
return self.dispatch_return(frame, arg)
if event == 'exception':
print("In execption")
return self.dispatch_exception(frame, arg)
if event == 'c_call':
print("In c_call")
return self.trace_dispatch
if event == 'c_exception':
print("In c_exception")
return self.trace_dispatch
if event == 'c_return':
print("In c_return")
return self.trace_dispatch
print(': unknown debugging event:', repr(event))
return self.trace_dispatch
As you can see from the code above, each event has a corresponding handler, and in this article we will analyze the dispatch_line function, which handles the line event.
def dispatch_line(self, frame):
"""Invoke user function and return trace function for line event.
If the debugger stops on the current line, invoke
self.user_line(). Raise BdbQuit if is set.
Return self.trace_dispatch to continue tracing in this scope.
"""
if self.stop_here(frame) or self.break_here(frame):
self.user_line(frame)
if : raise BdbQuit
return self.trace_dispatch
This function will first determine whether you need to stop at the current line, if you need to stop you need to enter the user_line function, followed by the call chain function is relatively long, we look directly at the final implementation of the function, according to our experience with pdb, the final must be a while loop allows us to continue to enter the instructions for processing:
def cmdloop(self, intro=None):
"""Repeatedly issue a prompt, accept input, parse an initial prefix
off the received input, and dispatch to action methods, passing them
the remainder of the line as argument.
"""
print("In cmdloop")
()
if self.use_rawinput and :
try:
import readline
self.old_completer = readline.get_completer()
readline.set_completer()
readline.parse_and_bind(+": complete")
except ImportError:
pass
try:
if intro is not None:
= intro
print(f"{ = }")
if :
(str()+"\n")
stop = None
while not stop:
print(f"{ = }")
if :
line = (0)
else:
print(f"{ = } {self.use_rawinput}")
if self.use_rawinput:
try:
# The core logic is right here Constantly asking for input and then processing it #
line = input() # = '(Pdb)'
except EOFError:
line = 'EOF'
else:
()
()
line = ()
if not len(line):
line = 'EOF'
else:
line = ('\r\n')
line = (line)
stop = (line) # This is the function that handles our input strings like p, n, and so on.
stop = (stop, line)
()
finally:
if self.use_rawinput and :
try:
import readline
readline.set_completer(self.old_completer)
except ImportError:
pass
def onecmd(self, line):
"""Interpret the argument as though it had been typed in response
to the prompt.
This may be overridden, but should not normally need to be;
see the precmd() and postcmd() methods for useful execution hooks.
The return value is a flag indicating whether interpretation of
commands by the interpreter should stop.
"""
cmd, arg, line = (line)
if not line:
return ()
if cmd is None:
return (line)
= line
if line == 'EOF' :
= ''
if cmd == '':
return (line)
else:
try:
# Based on the following code, we can analyze and understand that if we execute the command p, the function to be executed is do_p.
func = getattr(self, 'do_' + cmd)
except AttributeError:
return (line)
return func(arg)
Now let's look again at how do_p printing an expression is accomplished:
def do_p(self, arg):
"""p expression
Print the value of the expression.
"""
self._msg_val_func(arg, repr)
def _msg_val_func(self, arg, func):
try:
val = self._getval(arg)
except:
return # _getval() has displayed the error
try:
(func(val))
except:
self._error_exc()
def _getval(self, arg):
try:
# I've solved the case here # # Isn't this the same way that we've implemented our own pdb to get variables? # # Both #
# Use the global and local variables of the current execution stack frame to hand off to the eval function and output its return value.
return eval(arg, .f_globals, self.curframe_locals)
except:
self._error_exc()
raise
summarize
In this post we analyze the principle of debugger implementation in python, and implement a very simple debugger with a few dozen lines of code, which can help us understand the details of debugger implementation, which allows us to deepen our knowledge of the programming language a little more. Finally, we briefly introduced python's own debugger, pdb, but unfortunately pdb does not support direct debugging of python bytecode, but there are bytecode debugging events in the python virtual machine, so we believe that we should be able to debug bytecode directly in the future.
Remember from our discussion of frameobjects that there is a field, f_trace, that points to a function that we pass to the VM to call when an event occurs?
The above is an in-depth understanding of the Python virtual machine debugger implementation principles and source code analysis of the details, more information about the Python virtual machine debugger please pay attention to my other related articles!