When tests don't

Symptoms

While writing some test cases for a Python module using nose [1], I started to come across instances where individual test cases seemed to be silent. For those for came in late, nose sweeps across a file or directory, looking for python functions with special names (i.e. starting with test_). It then executes these tests, catches and reports any errors. You can also write a test class that sets up some data, runs any test methods and tears down the test data. So I had a case:

class test_drawstring (object):
    def setUp (self):
        self.rc = RpmCanvas (300, 350)

    def test_drawstring_1 (self):
        self.rc.draw_string ("drawstring1", (self.rc.width / 2, 90))

    def test_drawstring_left (self):
        pass

    def test_drawstring_right (self):
        pass

    def tearDown (self):
        self.rc.save ('test/out/test_drawstring.png')

This just tests drawing to a canvas. I was going to write three test methods, checking out aligning text in different ways, and so filled in the first method, leaving the other two blank for later. It was a surprise when I found that test_drawstring_1 was never called. After some detective work, the setUp and tearDown functions were clearly being called, but none of the test methods were. Embedding the body of test_drawstring_1 in setUp resulted in it being called without any problems. So why was test_drawstring_1 disappearing?

Solution

On a hunch, I deleted test_drawstring_left and test_drawstring_right. And the remaining test method was found, called and worked.

It seems that empty methods cause nose to miss any other test cases within a test class. So deleting the empty method fixes the problem:

class test_drawstring (object):
    def setUp (self):
        self.rc = RpmCanvas (300, 350)

    def test_drawstring_1 (self):
        self.rc.draw_string ("drawstring1", (self.rc.width / 2, 90))

    def tearDown (self):
        self.rc.save ('test/out/test_drawstring.png')

Oddly, "filling in" the empty methods with a dummy print statement doesn't work:

class test_drawstring (object):
    def setUp (self):
        self.rc = RpmCanvas (300, 350)

    def test_drawstring_1 (self):
        self.rc.draw_string ("drawstring1", (self.rc.width / 2, 90))

    def test_drawstring_left (self):
        print "dsfsdfdsf"

    def test_drawstring_right (self):
        print "dsfsdfdsf"

    def tearDown (self):
        self.rc.save ('test/out/test_drawstring.png')

It is a mystery.

Technical details

Python 2.5.2, nosetests 0.10.3.

References