Building Testing Libraries
by Casey West
|
Pages: 1, 2
Don't Let POD go Unchecked
Documentation is just as important as code, or tests. There are several ways to care for POD in your test suite. First, it's important to keep it well-formed. For this, we turn to Test::Pod. This Perl module takes all the work out of testing POD with a useful function all_pod_files_ok(). Simply create a new test program with the following contents.
use Test::More;
plan skip_all => "Test::Pod 1.00 required for testing POD"
unless eval "use Test::Pod 1.00";
all_pod_files_ok();Yes, it really is that simple. When you run this program, it will test all the POD it finds in your blib directory.
Another simple test we can run on the documentation is coverage analysis. What good is documentation if it doesn't document completely? Test::Pod::Coverage is the right module for the job, yet another gem that hides all the hard work from us with a simple function, all_pod_coverage_ok(). Again, we'll create a new test program.
use Test::More;
plan skip_all => "Test::Pod::Coverage 1.08 required for testing POD coverage"
unless eval "use Test::Pod::Coverage 1.08";
all_pod_coverage_ok();Coverage is only half of the battle. Remember, Test::Pod::Coverage can't tell you if your documentation is actually correct and thorough.
In both of these examples, we use the plan function exported from Test::More to allow us to "bail out" of our tests if the appropriate Perl module isn't installed. This makes our POD tests optional. If you don't want them to be optional, remove that line and be sure to list them as prerequisites for building and installing your software.
Know What You're Testing
One of the biggest testing mistakes is to assume that you know what you're testing. Tests are designed to exercise your software. Let your test exercise the good and bad portions of your software. Make it succeed and, most importantly, make it fail. Superior test coverage digs deep into every line of code you've written. How do you know if your tests are amazing? Coverage analysis.
Code coverage isn't something you can guess; you need good tools. Perl has a good tool: Devel::Cover. This module creates a database that maps actual execution to your source code. It analyzes statements, branches, conditions, subroutines, and even POD and execution time. It then provides a total for all of these areas, as well as a total for each Perl module. It's very simple to use, adding just a little to your make test process.
> cover -delete
> HARNESS_PERL_SWITCHES=-MDevel::Cover make testThe first command deletes any existing coverage database. On the second line we set an environment variable for Test::Harness, HARNESS_PERL_SWITCHES to a Perl command-line switch that imports Devel::Caller. This is all that's required of you. Each of your test programs will now run with Devel::Caller loaded and analyzing execution in the background.
To see your coverage database on the command line, issue one command.
> cover
---------------------------- ------ ------ ------ ------ ------ ------ ------
File stmt branch cond sub pod time total
---------------------------- ------ ------ ------ ------ ------ ------ ------
blib/lib/List/Group.pm 94.7 66.7 33.3 100.0 100.0 100.0 81.6
Total 94.7 66.7 33.3 100.0 100.0 100.0 81.6
---------------------------- ------ ------ ------ ------ ------ ------ ------
Writing HTML output to ~/cvs/perl/modules/List-Group/cover_db/coverage.html ...
done.As you can see, I could've done better. But what did I fail to test? Notice that cover wrote some HTML output. That is the diamond in the rough; the HTML output details everything. Each module has its own series of web pages detailing each of the coverage groups. I did particularly poorly on the conditional coverage -- let's see how.
Now it's become clear. My tests never allow either of the two statements in this condition to succeed. All of my tests make the first statement fail; the second is never executed. I need to update my tests with at least two more for 100.0 conditional coverage. The first test will supply a non-number for the $number variable. The second will supply a value for the $group_by variable that doesn't exist in the list for which grep is looking.
Testing for coverage is a noble goal. I find this method very useful when writing tests for existing software. There are many situations you may think you're testing well. Don't guess; know. Coverage analysis is equally useful for new development. If you've adopted the "test first" method and your coverage isn't 100 percent, something is wrong. Either your tests need help, or you've written more code than originally required.
Keep Test Files Organized
Perl software distributions follow several widely adopted guidelines concerning tests. The rules are simple: test files should reside in a t/ directory, and each test file ends in a .t extension. Test::Harness understands these rules and make test will run every file that abides by them.
The filename can be anything you like. It's a good idea to use descriptive filenames instead of just digits or numerical words. Good examples are pod-coverage.t, software-class-api.t, and compile.t. Sometimes it's desirable to determine the order in which your test files will be run. In these cases, prefix the filename with a number. If you want compilation tests to run first and POD tests last, name them accordingly as 00-compile.t and 99-pod-coverage.t.
Looking Ahead
Testing can be a tedious, difficult job. By this point, you have a number of helpful tools to make the task easier. There are many more testing modules on the CPAN that could have been covered here; I encourage you to explore them all.
