Defensive Programming & unit testing with Python

Luis Pedro Coelho


On twitter: @luispedrocoelho

Scientific code must not just produce nice looking output, but ideally should be correct

You should be able to do more than say it just works...

Why can things go wrong?

  • Your code is correct, but input files are wrong/missing/..., the network goes down...
  • Your code is buggy.

Defensive programming

Mistakes will be made, file formats will change, computers will crash, ...

Never fail silently

  • Worst possible thing: you work on data that is wrong without realizing it.
  • Give your code the Michael Bay treatment
  • More seriously, though: check for errors.
  • Good error messages are important.
  • The Unix tradition is that success is silent, error are noisy.

Check for files, errors, &c


from os import path
fname = 'my-input-file.txt'
if not path.exists(fname):
    raise ValueError("File not found '{}'".format(fname))

This is more important in shell scripts! (we will talk about it in the next session).

Check for possible bugs

Assertion as a programming tool

  • assert means that you, the programmer, are assuming that something is true.
  • When an assertion fails, this means you have a bug.

Assertions in other languages

  • C/C++ #include <assert.h>
  • D assert and static assert
  • C# Debug.assert
  • Java assert
  • Matlab assert()
  • ...

Assertions versus error checking

  • Error handling protects against outside events; assertions protect against programmer mistakes.
  • Assertions should never be false.

Some concepts and nomenclature

  1. pre-conditions. What must be true before calling a function.
  2. post-conditions. What will be true after calling a function.
  3. invariants. What the function does not change.

Do you test your code?

Of course you test your code, the goal of this session is to teach you how to do it in a more formal/structured fashion.

The big advantage is that this can then be kept and automated.

Again, we use Python/nose as a technology, but there are similar tools in all programming languages.

Example


def add_double(x, y):
    '''Returns the double of the sum of its inputs'''
    return 2. * (x + y)

A smoke test


In [1]: from add_double import add_double

In [2]: add_double(1, 1)
Out[2]: 4.0

Yeah, it ran.

Using nosetests


def add_double(x, y):
    '''Returns the double of the sum of its inputs'''
    return 2. * (x + y)

def test_smoke():
    assert add_double(1, 1) == 4

Now, run on the *Terminal*: nosetest -v.

Case tests


def test_basic():
    assert add_double(1, 1) == 4
    assert add_double(2, 1) == 6
    assert add_double(1, 2) == 6

Just check a few easy cases (which you can compute by hand)

Alternative: use a small test file (part of a larger one).

Edge/corner cases


def test_corner():
    assert add_double(0, 0) == 0
    assert add_double(-1, -1) == 2

Great, I have more things to type now!

  • When you are debugging, you can repeatedly run a test
  • Avoid the short blanket problem
    Fix a problem here, break something over there...
  • True story: updating code to parse a file after the file format changed.

Regression testing

Make sure bugs only appear once!

Bugs often cluster together.

Building a test is often the first step in debugging (either your bug or someone else's).

Testing philosophies

  • Write tests first (All code is guilty until proven innocent)
  • Test everything. Test it twice
  • Continuous integration
  • Regression testing only
  • Ad-hoc testing

Practical session

  1. Either start from scratch or take the file from github
  2. Write tests for a function which computes a robust mean, i.e., the mean value of a set of numbers except the maximum and minimum values.

Types of tests (summary)

  • Smoke test just check it runs
  • Case testing test a "known case" (like a control in the wet lab)
  • Corner/edge cases check "complex" cases.
  • Regression testing create a test when you find a bug.
  • Integration test test that different parts work together.

Writing testable code

  1. Using testing makes your code look different
    (Mostly better, but also just different)
  2. Split data-loading/computation/plotting (hard to test plotting)

Summary

  • Defensive programming means you prepare for inevitable mistakes
  • Automate testing to avoid wasted/repeated effort
  • Testing forces you to be more precise about what your functions do

Thank You