Web Testing with HTTP::Recorder
by Linda Julien
|
Pages: 1, 2, 3
This page has several Field/Operator/Value groupings. Left to its
own devices, the default HTTP::Recorder::Logger will
record every field for which a value has been set:
$agent->form_name("BuildQuery");
$agent->field("ActorOp", "=");
$agent->field("AndOr", "AND");
$agent->field("TimeOp", "<");
$agent->field("WatcherOp", "LIKE");
$agent->field("QueueOp", "=");
$agent->field("PriorityOp", "<");
$agent->field("LinksOp", "=");
$agent->field("idOp", "<");
$agent->field("AttachmentField", "Subject");
$agent->field("ActorField", "Owner");
$agent->field("PriorityField", "Priority");
$agent->field("StatusOp", "=");
$agent->field("DateField", "Created");
$agent->field("TimeField", "TimeWorked");
$agent->field("LinksField", "HasMember");
$agent->field("WatcherField", "Requestor.EmailAddress");
$agent->field("AttachmentOp", "LIKE");
$agent->field("ValueOfAttachment", "foo");
$agent->field("DateOp", "<");
$agent->submit_form(form_name => "BuildQuery");
But on this page, there's no need to record setting the values of fields (XField) and operators (XOp) unless a value (ValueOfX) has actually been set. We can do this with a custom logger that checks for the presence of a value, and only records the value of the field and operator fields if the value field has been set:
package HTTP::Recorder::RTLogger;
use strict;
use warnings;
use HTTP::Recorder::Logger;
our @ISA = qw( HTTP::Recorder::Logger );
sub SetFieldsAndSubmit {
my $self = shift;
my %args = (
name => "",
number => undef,
fields => {},
button_name => {},
button_value => {},
button_number => {},
@_
);
$self->SetForm(name => $args{name}, number => $args{number});
my %fields = %{$args{fields}};
foreach my $field (sort keys %fields) {
if ( $args{name} eq 'BuildQuery' &&
($field =~ /(.*)Op$/ || $field =~ /(.*)Field$/) &&
!exists ($fields{'ValueOf' . $1})) {
next;
}
$self->SetField(name => $field,
value => $args{fields}->{$field});
}
$self->Submit(name => $args{name},
number => $args{number},
button_name => $args{button_name},
button_value => $args{button_value},
button_number => $args{button_number},
);
}
1;
Tell HTTP::Recorder to use the custom logger like
this:
my $logger = new HTTP::Recorder::RTLogger;
$agent->logger($logger);
And it will record a much more reasonable number of things:
$agent->form_name("BuildQuery");
$agent->field("AndOr", "AND");
$agent->field("AttachmentField", "Subject");
$agent->field("AttachmentOp", "LIKE");
$agent->field("ValueOfAttachment", "foo");
$agent->submit_form(form_name => "BuildQuery");
Control panel. By default, you can access the
HTTP::Recorder control panel by using the Recorder to get
http://http-recorder. You can change this URL with the
$recorder->control([$value]) method.
Logger Options
Agent name. By default, HTTP::Recorder::Logger
outputs scripts with the agent name $agent:
$agent->follow_link(text => "Foo", n => 1);
However, if you prefer a different agent name (in order to drop
recorded lines into existing scripts, conform to company conventions,
etc.), you can change that with the
$logger->agentname([value]) method:
$recorder->agentname("mech");
will produce the following:
$mech->follow_link(text => "Foo", n => 1);
How HTTP::Recorder Works
The biggest challenge to writing a web recorder is knowing what the
user is doing, so that it can be recorded. A proxy can watch requests
and responses go by, the only thing you'll learn is the URL that was
requested and its parameters. HTTP::Recorder solves this
problem by rewriting HTTP responses as they come through, and adding
additional information to the page's links and forms, so that it can
extract that information again when the next request comes through.
As an example, a page might contain a link like this:
<a href="http://www.cpan.org/">CPAN</a>If the user follows the link, and we want to record it, we need to know all of the relevant information about the action, so that we can produce a line of code that will replay the action. This includes:
- the fact that a link was followed.
- the text of the link.
- the URL of the link.
- the index (in case there are multiple links on the page of the same name).
HTTP::Recorder overloads LWP::UserAgent's
send_request method, so that it can see requests and
responses as they come through, and modify them as needed.
HTTP::Recorder rewrites the link so that it looks like this:
<a href="http://www.cpan.org/?rec-url=http%3A%2F%2Fwww.cpan.org%2F&rec-action=follow&rec-text=CPAN&rec-index=1">CPAN</a>
So, with the rewritten page, if the user follows this link, the request will contain all of the information needed to record the action.
Forms are handled likewise, with additional fields being added to
the form so that the information can be extracted later.
HTTP::Recorder then removes the added parameters from the
resulting request, and forwards the request along in something close
to its originally intended state.
Looking Ahead
HTTP::Recorder won't record 100% of every script you
need to write, and while future versions will undoubtedly have more
features, they still won't write your scripts for you. However, it
will record the simple things, and it will give you example code that
you can cut, paste, and modify to write the scripts that you need.
Some ideas for the future include:
- Choosing from a list of simple tests based on the fields on the page and their current values.
- "Threaded" recording, so that multiple sessions won't be recorded in the same file, overlapped with each other.
- "Add script header" feature.
- Supporting more configuration options from the control panel.
- Other loggers.
- JavaScript support.
Where to Get HTTP::Recorder
The latest released version of HTTP::Recorder is available at CPAN.
Contributions, Requests, and Bugs
Patches, feature requests, and problem reports are welcomed at http://rt.cpan.org.
You can subscribe to the mailing list for users and developers of HTTP::Recorder at http://lists.fsck.com/mailman/listinfo/http-recorder, or by sending email to http-recorder-request@lists.fsck.com with the subject "subscribe".
The mailing list archives can be found at http://lists.fsck.com/piper-mail/http-recorder.
See Also
WWW::Mechanize by Andy Lester.
HTTP::Proxy by Phillipe "BooK" Bruhat.
