Sign In/My Account | View Cart  
advertisement


Listen Print Discuss

Charting Data at the Bottom of the World
by Alex Gough | Pages: 1, 2

Displaying Data

I now have hold of the weather data and have forced it into a form that I can follow. Now I get to show it to someone else. I did this in two different ways: as raw data through a web page and as a pre-generated chart embedded in the page.

In each case, the code has to read in files to obtain the necessary data:

 my @weather_records = Z::Weather->from_file('weather.data.dat');

Then it needs to produce the web page:

 use Template;
 my $template = Template->new();

 print "Content-type: text/html\n\n";
 
 $template->process(\*DATA, {
                       now => $weather_records[-1],
                       records => \@weather_records,
                             })
    || die "Could not process template: ".$template->error()."\n";

This isn't really all that interesting. In fact, it looks almost like this does nothing at all. I've pulled in the Template module, told it to build and output a template defined after the __END__ of the script, and given it two template variables to play with. The template looks something like:

 __END__
 <html><head><title>Weather</title></head>
 <body>
 <h2>Latest weather data at [% now.time %]<a name="/h2">
 
 <P>T: [% now.temp %] &deg;C
    P: [% now.pressure %] kPa
    W: [% now.wind %] kts
    D: [% now.dir %] &deg;</p>
 
 <P><img src="/weather_chart.png"><br>
    <img src="/mast_chart.png"</p>

 <table>
 <tr><th> Time </th><th> Temp </th><th> Wind </th></tr>
 [% FOREACH rec IN records %]
 <tr>
  <td>[% rec.time %]</td>
  <td>[% rec.temp %]</td>
  <td>[% rec.wind %]</td>
 </tr>
 [% END %]
 </table>
 </body></html>

The template uses the syntax of the Template-Toolkit, a general-purpose templating framework. It's useful because it allows the separation of display and formatting of data from the code that generates it. There's no Perl code in the template, and no HTML will appear in any of my Perl code. While the output generated now is ugly and basic, it will be easy to make it flashy later, once I have the program working, without having to change anything in the program itself to do so. As I've prepared our data carefully as objects with sensible methods, I can just hand a bunch of these over to the template and let it suck out whatever it wants to show.

Pretty Pictures

Producing the charts is, again, a simple business (by now, the theme of this article should be emerging). Gone are the days when you'd have to scratch your head figuring out how to draw lines and plot points; gone even are the days when you have to bang your head hard against the confused API of some long-forgotten module. Taking the mast values as an example, I first need to read in the data:

 my @mast_values = Z::Mast->from_file('mast.data.dat');

Because old weather is old news, I throw away any values older than three hours, using DateTime and DateTime::Duration methods in a grep:

 use DateTime;
 use DateTime::Duration;
 
 my $now = DateTime->now();
 my $age = DateTime::Duration->new(hours => 3);
 
 @mast_values = grep { $_->time + $age > $now } @mast_values;

This is so, so much easier than fiddling around with epochs and 3*3600 all over the place. If you find yourself writing 3600 anywhere in your code, you should be using DateTime::Duration instead. Next, I feed the data points into the Chart::Lines module, a part of the Chart distribution. I use this in three phases. First, I create a new Chart and specify how large the resulting graphic should be:

 use Chart::Lines;
 my $chart = Chart::Lines->new($x_size, $y_size);

Then I set up a few options to tweak how the chart will display:

 $chart->set(
    legend          => 'none',
    xy_plot         => 'true',
    grey_background => 0,
    y_label         => 'Wind kts',
    x_label         => 'Hours ago',
    colors          => {
      y_label    => [0xff, 0xee, 0xee],
      text       => [0xff,0xee,0xff],
      dataset0   => [0xff,0,0],
      dataset1   => [0,0xff,0xff],
      dataset2   => [0,0,0xff],
      background => [0x55, 0x00, 0x55],
                },
    );

These are mostly self-explanatory; the Chart documentation covers them in detail. I set xy_plot to true so that the module will use the first dataset as the x values and all of the other datasets as the y values for a line. I set a bunch of rather bright colors, to keep my avid customers cheerful, and set the text used to label the chart.

 my @labels = map {($now->epoch - $_->time->epoch) / 60} @mast_values;

Finally, I used a series of map expressions to extract x and y values from the data. One turns the DateTime times into a number of minutes ago. These values are the x values. y values are the appropriate parameters extracted from the nested Z::Mast and Z::Mast::Label objects. The rest of the code provides the data to the plotting method of the chart, directing it to write out a .png file (Figure 1).

 $chart->png("mast.png",
               [ \@labels,
                [map {$_->values()->[0]->wind} @mast_values],
                [map {$_->values()->[1]->wind} @mast_values],
                [map {$_->values()->[2]->wind} @mast_values],
                  ]);

the resulting chart
Figure 1. The resulting chart

All I need now is a working HTTP server and a crontab entry or two to run the graphic generation programs. It is possible to use the Chart modules to generate CGI output directly using the Chart::cgi method, but I found that this was too slow once lots of different clients accessed the weather data at the same time. It was a simple task to instead switch to a crontab-based approach for the graphs, with a CGI script still providing real-time access to the current values.

Conclusions

The Chart family of modules provides more than just an x-y plot. Pie, bar, Pareto, and mountain charts, amongst others, are available through the same API as I discussed in this article. They are just as easy to whip into being to satisfy even the most demanding of data consumers.

The Template Toolkit is used mainly for more complicated websites and content management systems, but it will simplify the production of simple sites and pages, allowing you to concentrate on the detail of the problem by separating data and its presentation. Even though a problem is simple and allows a fast solution, you can reach your goal faster still by pulling in big tools to do little jobs.

As for the DateTime module, I simply wouldn't go anywhere without it. These days, I find myself automatically typing use DateTime; along with warnings and strict at the head of every Perl program I write.

Class::Accessors makes the creation of data representation objects faster than typing in a C struct, provides some level of documentation about what the data you're dealing with, and allows for reuse. You could just stick everything into layers of nested hashes and arrays, but this is a certain path to eventual confusion. Class::Accessors will keep you sane and save your fingers at the same time.

IO::All should be a part of your day-to-day toolkit; the new idioms it provides will soon see you using it everywhere, even in one-liners.

One of the many joys of programming lies in the satisfaction we receive when we make someone's life that little bit better. Perl makes it easy, fast, and fun for us to tread that path. Perl's greatest strength, the rock upon which its greatness is founded, is the speed with which we can take a problem, or a cool idea, and structure our half-formed thoughts into a soundly built solution.

Download the example code for this article.