We've expanded our Perl news coverage and improved our search! Search for all things Perl across O'Reilly!

Web Testing with HTTP::Recorder

by Linda Julien
June 04, 2004

HTTP::Recorder is a browser-independent recorder that records interactions with web sites and produces scripts for automated playback. Recorder produces WWW::Mechanize scripts by default (see WWW::Mechanize by Andy Lester), but provides functionality to use your own custom logger.

Why Use HTTP::Recorder?

Simply speaking, HTTP::Recorder removes a great deal of the tedium from writing scripts for web automation. If you're like me, you'd rather spend your time writing code that's interesting and challenging, rather than digging through HTML files, looking for the names of forms an fields, so that you can write your automation scripts. HTTP::Recorder records what you do as you do it, so that you can focus on the things you care about.

Automated Testing

We all know that testing our code is good, and that writing automated tests that can be run again and again to check for regressions is even better. However, writing test scripts by hand can be tedious and prone to errors. You're more likely to write tests if it's easy to do so. The biggest obstacle to testing shouldn't be the mechanics of getting the tests written — it should be figuring out what needs to be tested, and how best to test it.

Part of your test suite should be devoted to testing things the way the user uses them, and HTTP::Recorder makes it easy to produce automation to do that, which allows you to put your energy into the parts of your code that need your attention and your expertise.

Automate Repetitive Tasks

When you think about web automation, the first thing you think of may be automated testing, but there are other uses for automation as well:

• Check your bank balance.
• Check airline fares.
• Check movie times.

How to Set It Up

Use It with a Web Proxy

One way to use HTTP::Recorder (as recommended in the POD) is to set it as the user agent of a web proxy (see HTTP::Proxy by Phillipe "BooK" Bruhat). Start the proxy running like this:

    #!/usr/bin/perl

    use HTTP::Proxy;
    use HTTP::Recorder;

    my $proxy = HTTP::Proxy->new();

    # create a new HTTP::Recorder object
    my $agent = new HTTP::Recorder;

    # set the log file (optional)
    $agent->file("/tmp/myfile");

    # set HTTP::Recorder as the agent for the proxy
    $proxy->agent( $agent );

    # start the proxy
    $proxy->start();

    1;

Then, instruct your favorite web browser to use your new proxy for HTTP traffic.

Other Ways to Use It

Since HTTP::Recorder is a subclass of LWP::UserAgent, so you can use it in any way that you can use its parent class.

How to Use It

Once you've set up HTTP::Recorder, just navigate to web pages, follow links, and fill in forms the way you normally do, with the web browser of your choice. HTTP::Recorder will record your actions and produce a WWW::Mechanize script that you can use to replay those actions.

The script is written to a logfile. By default, this file is /tmp/scriptfile, but you can specify another pathname when you set things up. See Configuration Options for information about configuring the logfile.

HTTP::Recorder Control Panel

The HTTP::Recorder control panel allows you to use to view and edit scripts as you create them. By default, you can access the control panel by using the HTTP::Recorder UserAgent to access the control URL. By default, the control URL is http://http-recorder/, but this address is configurable. See Configuration Options for more information about setting the control URL.

The control panel won't automatically refresh , but if you create HTTP::Recorder with showwindow => 1, a JavaScript popup window will be opened and refreshed every time something is recorded.

Goto Page. You can enter a URL in the control panel to begin a recording session. For SSL sessions, the initial URL must be entered into this field rather than into the browser.

Current Script. The current script is displayed in a textfield, which you can edit as you create it. Changes you make in the control panel won't be saved until you click the Update button.

Update. Saves changes made the script via the control panel. If you prefer to edit your script as you create it, you can save your changes as you make them.

Clear. Deletes the current script and clears the text field.

Reset. Reverts the text field to the currently saved version of the script. Any changes you've made to the script won't be applied if you haven't clicked Update.

Download. Displays a plain text version of the script, suitable for saving.

Close. Closes the window (using JavaScript).

Pages: 1, 2, 3

Next Page