We've expanded our Perl news coverage and improved our search! Search for all things Perl across O'Reilly!

An Introduction to Testing
by chromatic | Pages: 1, 2

Enter Test::More

Several modules exist to make testing easier and almost enjoyable. Test ships with modern Perl distributions and plays well with Test::Harness. The Perl-Unit suite reimplements the popular JUnit framework in Perl. The rather new Test::More module adds several features beyond those of Test. (I admit a particular bias toward the latter, though these and other modules are fine choices.)

Test::More has its own ok() function, but it is rarely used in favor of more specific functions. is() compares two expressions. For example, testing an addition function is as simple as:

        is( add(2, 2), 4 );

This handles strings and numbers equally well. Since version 0.36, it also distinguishes between 0, '' (the empty string), and undef -- we hope.

like() applies a regular expression to a scalar. This is also useful for trapping fatal errors:

        $self->eat('garden salad'):

        eval { $self->write_article() };
        like( $@, qr/not enough sugar/ );

The second argument can be either a regular expression compiled with the qr// operator (introduced in Perl 5.005), or a string that resembles a regular expression. Modifiers are allowed. If you absolutely must rewrite the previous example to run on Perl 5.004 and to use StudlyCaps, then you can:

        eval { $self->write_article() };
        like( $@, '/NoT eNoUgH sUgAr/i' );

That's too cute/hideous for a real test, but the regex form is completely valid.

Test::More Makes Debugging Nicer

That's useful enough already, but there's more to Test::More.

Test::More supports test numbering just as Test::Harness does, and automatically provides the numbers. This is a big win in two cases: where the test suite may accidentally fail (from a die() call, a segmentation fault, or spontaneous combustion), or if the tests can accidentally repeat (due to an improper chdir(), unexpected input in a loop condition, or a time warp). Test::Harness is happy to warn whether the number of tests actually run do not match its expectations. You just have to tell it how many should run. This is usually done when using Test::More:

        use Test::More tests => 50;

When writing new tests, you may not know how many there will be. Use the no_plan option:

        use Test::More 'no_plan';

Extreme Programming recommends this game-like approach: add a test, run it, write code to pass the test, repeat. When you've finished, update the use line to reflect the actual number of tests.

Test::More also handles failures gracefully. Given the following file with a final, doomed test:

        use Test::More tests => 4;

        is( 1, 1 );
        is( !0, 1 );
        is( 0, 0 );
        is( undef, 1 );

Test::More run on its own, not through Test::Harness, produces:

        1..4
        ok 1
        ok 2
        ok 3
        not ok 4
        #     Failed test (numbers.t at line 6)
        #          got: undef
        #     expected: '1'
        # Looks like you failed 1 tests of 1.

The error message provides the name of the file containing the tests, the number of the failed test, the line number containing the failed test, and expected and received data. This makes for easier debugging. Count tests to find an error just once, and you'll prefer this approach.

Test::Harness also supports optional test comments attached to test messages. That is, it allows raw tests to say:

        print "ok 1 # the most basic test possible\n";

Nearly all Test::More functions support this as an optional parameter:

        ok( 1, 'the number 1 should evaluate to true' );
        is( 2 + "2", 4, 'numeric strings should numerify in addition' );
        like( 'abc', qr/z*/, '* quantifier should match zero elements' );

These names are required by nothing except social convention. Think of them as little test comments. If the test is wrong, or exposes a fixed bug that should never reoccur, then a descriptive name makes it clear what the test should be testing. Test::Harness silently eats the names, but they're present when run manually:

        ok 1 - the number 1 should evaluate to true
        ok 2 - numeric strings should numerify in addition
        ok 3 - * quantifier should match zero elements

Manual test runs make for improved bug reports. Ignore these convenient tools at your own peril.

Intermediate Test::More Features

If the previous features weren't enough, Test::More supports still more! One such is the notion of skippable tests. Occasionally, the presence or absence of certain criteria obviate the need to test a feature. Consider the qr// operator explained earlier. A module that needs to be backward compatible to Perl 5.004 can gradefully degrade its test suite with skippable tests:

        SKIP: {
                skip( 'qr// not supported in this version', 2 ) unless $] >= 5.005;

                my $foo = qr/i have a cat/;
                ok( 'i have a caterpillar' =~ $foo,
                        'compiled regex should match similar string' );

                ok( 'i have a cold' !~ $foo,
                        'compiled regex should not match dissimilar string' );
        }

There's a lot to digest. First, the skippable tests are contained in a labelled block. The label must be SKIP. (Don't worry: you can have several of these within a file.) Next, there should be a condition that governs whether to skip the tests. This example checks the special variable perlvar to find the current Perl version. skip() will only be called when run with an older version.

The skip() function always confuses me with its unique parameter order. The first argument is the name to display for each skipped test. The second argument is the number of tests to skip. This must match the number of tests within the block, at the risk of certain confusion. Run on Perl 5.004, the above test produces:

        ok 1 # skip qr// not supported in this version
        ok 2 # skip qr// not supported in this version

Though the message says ok, Test::Harness will see skip and report the tests as skipped, not passed. This should only be used for tests that absolutely will not run due to platform or version differences. For tests you just can't figure out yet, use todo().

Though everything is built on Test::Builder::ok(), other functions offer helpful shortcuts. use_ok() and require_ok load and optionally import the named file, reporting the success or error. These verify that a module can be found and compiled, and are often used for the first test in a suite. The can_ok() function attempts to resolve a class or an object method. isa_ok() checks inheritance:

        use_ok( 'My::Module' );
        require_ok( 'My::Module::Sequel' );

        my $foo = My::Module->new();
        can_ok( $foo->boo() );
        isa_ok( $foo, 'My::Module' );

They produce their own test names:

        ok 1 - use My::Module;
        ok 2 - require My::Module::Sequel;
        ok 3 - My::Module->can(boo)
        ok 4 - object->isa('My::Module')

Other functions and features are documented in the Test::More documentation. As well, the Test::Tutorial manpage explains similar things with a different wit.

Finally, do not forget good programming practices. Test functions are simply standard subroutines. Tests are just Perl code. Use loops, variables, helper subs, map(), and anything else, when they make things easier. For example, basic inherited interface testing can be made easier with:

        # see if IceCreamBar inherits these methods from Popsicle
        my $icb = IceCreamBar->new();

        foreach my $method (qw( fall_off_stick freeze_tongue drip_on_carpet )) {
                can_ok( $icb, $method, "IceCreamBar should be able to $method()" );
        }

That beats writing several individual can_ok() tests. Interpolating things into the test name is also handy.

Conclusion

Testing is unfortunately often neglected, especially among free software projects. Think of it as getting plenty of sleep, eating vegetables, and working out regularly. It may cramp your style at first, but will improve things immensely if you do it consistently. (Results may vary if you're adding tests to a huge system that doesn't have them, like, say, Perl itself.)

One goal of Perl is to make your life easier. Perl 5.8 will include Test::More and its hearty brethren, the Test::Simple manpage and the Test::Builder manpage. They exist to make writing tests less of a hassle, and even more pleasant. Consider them.

The easier it is to write and maintain tests, the more likely people will do it. More and better tests improve software portability, maintainability, and reliability. You may currently compare testing to broccoli, brussel sprouts, and wind sprints. Try Test::More or another framework, and you may grow to see them as oranges, sweet potatoes with marshmallows, and a trip to the sauna. It really is good for you.