I spent past two months doing heavy writing (the thesis, a paper…) and a thing that have seemed more scary to me than it was really — digging into Python codebases.

The lack of proper debugger always irked me about that language, along with not displaying faulty values in error messages. There is little recourse when encountering code which is not well documented, which is, honestly, most code that exists. I don’t like being forced to use some IDE, since leaving the keyboard-centric terminal already slows me down as a programmer, at least subjectively.

Now, I knew that there is the pdb package, but actually putting it into use is another matter.

Interactive mode

Simply running python (or in my case python3) from console with -i flag makes it dump you to the REPL after finishing — which may happen just by successfully executing all the code there is, or by breaking along the way with an error. (So you can break arbitrarily wherever you want by raising some uncatched exception or writing something like fail(), provided that it isn’t a defined function). Then you can fiddle with values living in the global scope at the end of program’s life.

Beautiful landscape
I’m writing this in the Gutenberg editor’s demo and I like this image, so I’m leaving it there.

This is not that optimal. More often that not, you need to examine some local scope of variables buried deep in the code. But it turns out you can get even there.

Post mortem pdb

Once the program broke, there is in fact the whole stack preserved for your inspection. To put your hands on it, you just need some incantations:

import pdb
pdb.pm()

Of course if you’re savvy, you may have imported the package already. The function pm() is a shorthand for “post mortem” — you’re reviving the state when the program died. Then you can do all debugger things you would expect from gdb for instance.

One caveat is that the underlying code — should you run it from the pdb prompt — can sys.exit() on you, for instance, and pdb won’t catch it. All the stack nuked, back to the console or something. This is precisely because pdb is just a package observing things in the Python interpreter, existing alongside other packages, and not an external sandbox as aforementioned gdb.

And this is all there is to this really. In the manual they even tell you how to start the program in debugging mode, set up some breakpoints and avoid nasty hacks with placing fake exceptions beforehand. I won’t be replicating the manual here, although I notice that there is little in the way of tutorials on that (there is some, to be fair).

This is a shame. Many tools building upon Python, such as AllenNLP, don’t give you an option of running the underlying interpreter in the interactive mode, or inside a debugger. So we’re back to stone age or maybe forced to use some third-party tools that can get around that. Maybe most people are just better than me in masterminding cryptic code with no step-by-step debugger, but I suspect it is just a strange lack of widespread knowledge that this side of Python even exists and how to use it.

Leave a Reply

Your email address will not be published. Required fields are marked *