Something's Wrong!

Python, debugging, and finding things out the easy way

Download it!

Download this presentation from:

http://www.beaugunderson.com/debugging/

QR Code

As soon as we started programming, we found to our surprise that it wasn't as easy to get programs right as we had thought. Debugging had to be discovered. I can remember the exact instant when I realized that a large part of my life from then on was going to be spent in finding mistakes in my own programs.

Maurice Wilkes

Debugging Python code

print statements

For example: def add(arg1, arg2): print "%s, %s" % (arg1, arg2) return arg1 + arg2 Barrier to entry is tiny: Usually less than 15 characters.
Useful for visualizing program flow and seeing how things play out sequentially: a = [0, 1, 2] for i in len(a) + 1: print a[i]
Output: 0 1 2 Traceback (most recent call last): File "", line 2, in IndexError: list index out of range

Logging

The logging module in the standard library is very featureful in recent versions of Python. + Format strings with useful arguments (time, line number, etc.) + Circular logging + Verbosity levels + Multiple loggers
All it takes to use logging: import logging logger = logging.getLogger("main") logger.setLevel(logging.DEBUG) handler = logging.StreamHandler() handler.setLevel(logging.DEBUG) formatter = logging.Formatter("%(levelname)s %(asctime)s " + "%(funcName)s:%(lineno)d %(message)s") handler.setFormatter(formatter) logger.addHandler(handler) def main(): logger.debug("Some useful information") if __name__ == "__main__": main()

plac: command line arguments

Adding a debugging flag to `my.py`: import plac @plac.annotations( debug=("Display debug output", "flag")) def main(debug=False): if debug: print "Some useful information" if __name__ == "__main__": plac.call(main)
Output: $ ./my.py -h usage: test.py [-h] [-d] optional arguments: -h, --help show this help message and exit -d, --debug Display debug output

ipython

An improved REPL with colors, tab-completion, etc. This line will kick you into ipython for any uncaught exception your program encounters: from IPython.Shell import IPShellEmbed ipshell = IPShellEmbed() ipshell() # this call anywhere in your program will start IPython You can achieve the same effect by running your .py file with ipython: $ ipython your.py

Exploring with ipython

Let's say we're playing with a new API: flickr.authenticate() picture = flickr.get_a_picture_object('unique-identifier') ipshell() # drop into an ipython shell At this point you can manually invoke an ipython shell to interactively play with that variable using tab-completion: In [1]: picture.<TAB> picture.author picture.date picture.description picture.location picture.tags picture.title

The ? operator

An easy way to get a high-level description of an object: In [1]: a = [1, 2, 3] In [1]: a? Type: list Base Class: String Form: [1, 2, 3] Namespace: Interactive Length: 3 Docstring: list() -> new empty list list(iterable) -> new list initialized from iterable's items

Debugging things with Python

Debugging with WinAppDbg: def action_callback(event): stack = event.get_thread().get_sp() address = event.get_process().read_pointer(stack) pid = event.get_pid() tid = event.get_tid() message = "kernel32!CreateFileW called from %s by thread %d " + "at process %d" print message % (HexDump.address(address), tid, pid) class MyEventHandler(EventHandler): def load_dll(self, event): module = event.get_module() if module.match_name("kernel32.dll"): pid = event.get_pid() address = module.resolve("CreateFileW") event.debug.break_at(pid, address, action_callback)

Interoperability

Disassembly

De-obfuscation

The Web

TCP & UDP

SQL

LDAP

Windows Internals

Linux Internals

SystemTap

SystemTap provides free software (GPL) infrastructure to simplify the gathering of information about the running Linux system.

"Gathering of information?"

Like...

Forensics

Better tools!

Think like a debugger