July 2004 Archives

Giving Lightning Talks

After the keynotes, the most popular sessions at most Perl conferences tend to be the lightning talks. Each session consists of about 10 to 15 lightning talks — talks typically lasting individually no more than five minutes — back to back. As well as being tremendously interesting and entertaining for attendees, the conference organizers recognize that they offer an unequalled opportunity for new speakers to present for the first time without going to the lengths required for a longer talk.

Which is why I was surprised that at this year's YAPC someone raised the point in the town hall meeting that there's no document that stands as a basic guide for pointing out the common pitfalls to new speakers. Armed with my experiences of YAPCs and organizing many, many London Perl Monger tech meets I thought I'd have a go at explaining what you need to know.

Lightning talks are a great way to start speaking, but they do come with their own potential problems. Even the seasoned presenter makes mistakes when presenting and has had lightning talks go horribly, horribly wrong. Luckily, Perl audiences are very forgiving when this happens — after all, it's happened to half the audience at sometime too, so it's nothing to get too worked up about. However, there are a few things you can consider when writing your talk that will prevent you from sabotaging your own talk before it even begins.

Make your Point

First of all, let's look at the point of a lightning talk. The point is to make a point, and explain it as quickly as possible. That's it. Understand?

Too many lightning talks forget this. They get caught up in the whole idea of providing background information or explaining other issues. They've got a full five minutes, damn it, and they're gonna use them.

Here's the secret; No one cares if your lightning talk only lasts four minutes. No one cares if it lasts three. I've seen talks that last two minutes and the crowd loved it. But everyone cares if it lasts six — especially if the important point you need to make happens at five minutes and 10 seconds. At a YAPC or other event with strict time keeping you'll be cut off at five, and no one will hear what you were actually trying to say.

One of the best tactics is to make your point as early in the talk as humanly possible. You might need to set up the problem space, explain why you were doing what you were doing before you can explain your point. Fine, but do it quickly. You really shouldn't spend more than a minute explaining the background before you make your first point. If you haven't explained the main point by minute three you're probably up the proverbial creek without the paddle.

Actually, that bit about not explaining much before your first point isn't really true. You see, if you're explaining the background to the problem, that itself has to be your first point. You must carry your audience along, keeping them interested until you hit what you consider to be the actual point of the presentation. You know how to do this — you know to generalize your problem into the larger problem that everyone has and therefore cares about, right?

Most Details Don't Matter

This brings me nicely to the bane of most lightning talks. The details. Here's the rub; No one cares, and even if they do, certainly no one will remember the little details 10 minutes after your talk has ended. This means that unless those details are absolutely necessary to explain the point you're trying to make, then they're just sucking up time. Even if they're necessary, stop and think, can you replace these with a shorter summary, something that you can explain easier? Remember, you're talking to an intelligent crowd here who can normally move from A to B without having to be handled through each and every step.

One of the biggest mistakes people make about the details is misunderstanding that just because they've put something on the slides it means that they have to explain it. For example, you might need to show slides to demonstrate that one of the things your spiffy new function allows you to do is rewrite a huge chunk of code into a much smaller piece of code. This means that you probably need a before and after slide with one slide containing a mountain of code and one containing very little, but it doesn't mean you have to explain how they're different in great detail. That should be self-evident (or if it's not, your new function isn't nearly spiffy enough.) Someone can always download your slides later if they really want to read the example carefully.

Practice

When it comes down to it there's only one way to make sure that your talk will fit in the time-slot, and that's to practice it. Read it out aloud several times. Present it to the cat. Try to convince your colleague/flatmate/significant other to listen to it. If you're presenting at a conference, see if you can present it at your local Perl Monger meeting beforehand as a dry run. Not only will this give you the most accurate understanding of how long it'll take to say everything (and believe me, you'll be surprised at what bits go quickly and what bits drag), but it'll also help you realize what bits can be cut and replaced. It'll also give you confidence and experience in the talk, so you can actually present it slightly quicker.

Having covered the contents of your talk, let's move onto the practicalities of how you'll do it.

Slides

Let's talk about slides. Firstly, do you actually need slides at all? You'll only talk for five minutes. Crib notes on paper work well. One of my favorite lightning talks had someone just talking from a bunch of index cards, each with one word on it to remind him what he was going to talk about next,. And each one he'd theatrically throw over his shoulder after he was done with it. Remember, the slides are to show things to the audience, not to help you remember what you're talking about. Think of the lightning talk as a narrative with visual aids — someone once told me presenting a lightning talk is just like having a conversation with the audience where they don't speak for five minutes.

Prepare your Slides

All this said, let's assume you've decided that you need slides. Remember, you have five minutes. This means that you have zero time to faff about with your slides, setting up your computer, or otherwise doing anything that isn't entertaining your audience. Preparation is everything.

The audience bores easily so it's important that you don't keep them waiting before you start speaking. If you have your slides on your laptop make sure everything is set up so you can start talking as soon as it's plugged in. Set up and load all the apps you're going to use first — no one wants to be caught on stage in an awkward silence with nothing to say while they wait the 20 seconds it takes Mozilla to load. While you're at it don't forget to close down instant-messenger clients, calendaring apps, and anything else that might randomly pop up dialogs when you're trying to speak.

It's absolutely critical that you take the time to check that your laptop works with the projector before the lightning talk session. Make sure that you don't have to do anything other than plug the projector in, that you don't have to remember some weird key combo to make the external display work, and that you have enough power to last the entire talk (including the time your laptop will be switched on while you're waiting your turn).

Of course, the best way to get around this faffing about with switching laptops is to work out who's talking before you and arrange to use their computer. One of the major drawbacks to this is, of course, that you might not be familiar with their setup. This means that you should, at the very least, run through your presentation. Don't assume that one web browser is like any other — they all have slightly different ways of laying out pages and having plugins installed. Try it first to find problems while no one is watching, rather than trying to debug your presentation setup on stage. Have your own laptop ready in case theirs suddenly fails for some unknown reason.

Presentation Software

When it comes to actually writing a lightning talk, there are many choices of presentation software. Although there isn't one right piece of software for everyone, there are definitely wrong choices. That wrong choice is anything that requires any thought on your part while you're presenting. It's stupidly hard to operate a computer and talk at the same time — especially when you're not sitting in front of the machine but standing at a weird angle that makes it almost impossible to operate a mouse or track pad. During a lightning talk it's vital that you keep your rapport with the audience, rather than diverting your attention to the computer every slide change. Unlike a 40-minute talk, you don't have time to take breaks between slides.

For this reason I prefer software that you just hit one key or click the mouse to go to the next slide — I follow the rule that anything that requires you to move the mouse is probably a failure. So if you must present from a web browser make sure that the links to move between places are in the same physical location on the page, preferably in the top left corner. And make sure that you don't have to scroll your slides. Remember kids, HTML isn't a presentation language so the only way to ensure that it doesn't give you unexpected surprises — suddenly scrolling, missing graphics, crazy fonts — is to run through the presentation with the web browser you'll use on the machine you want to use beforehand. All web browsers do not render the same.

Unexpected Problems

Even when you take every precaution, you'll still encounter unexpected problems. Your demo won't work. You'll discover you're using a slightly older version of the slides. Your software will use the wrong font. The computer will burst into flames and burn down the venue. Remember that you don't have time to recover from any of these problems. Even if it takes you 30 seconds to locate a new version of the slides and remember where you were, that's a tenth of your talk gone. More importantly, the audience has lost interest while you're doing this and has slipped into their daydream world where you'll have to work really hard to reacquire their attention.

If something goes wrong, you really have no choice but to move on and skip over it. As I said, the audience is normally a forgiving bunch so tell them something has gone wrong and take them into your confidence; explain what the demo would have shown and press on regardless.

Concluding your Talk

In an ideal world you'd be able to complete your lightning talk with time to spare, take a few questions, and clear up anything you didn't manage to cover. In reality, that's probably won't happen. So it's important to put something in your slide to allow people to find out more about what you're talking about and provide a way of contacting you once you're done speaking. The biggest mistake I've made in my talks in the past is putting this information on the last slide, which of course only appears for 10 seconds and no one has time to copy down. Now I place a simple URL in the bottom corner of every slide.

This just goes to show that I'm still learning and finding ways of improving my lightning talks. Remember that in the end, it doesn't really matter if it all goes well or not; it's all a learning experience. Good luck with your talk.

See Also

Conference Presentation Judo

This Week on Perl 6, Week Ending 2004-07-18

Following last week's bizarrely dated summary (I misplaced a day) we're back with the correct week-ending date, but I'm ashamed to admit that I've slipped to writing on a Tuesday again. My head hangs in shame and I am filled with the direst remorse.

It will probably happen again though; life's like that.

Pie-thon!

Anyone would think that the possibility of not even being able to run the Piethon benchmark (let alone getting a pie in the face for running it too slowly) weighs heavily on Dan.

The bulk of the week's traffic related to getting to the point where we can at least run all the tests.

Python::Bytecode, the module that parses, well, python bytecode, shot up in version numbers; I think it reached 2.7 by the end of the week.

Dan released a Piethon translator, a tool that uses Python::Bytecode to translate Python to Parrot. So has Leo. Dan's is on his web site (see the link below); you'll find Leo's in languages/python/pie-thon.pl.

http://groups.google.com/groups?threadm=40F26AD7.40000@toetsch.at -- Parrot runs a Python lambda

http://groups.google.com/groups?threadm=a06110407bd1c65dd4c0e@[10.0.1.3] -- Notes on pie-thon.pl

http://groups.google.com/groups?threadm=a06110400bd1cf1d2b965@[10.0.1.3] -- Namespaces for built-ins

http://www.sidhe.org/~dan/piethon/translator.pl -- Dan's translator

Semi-Stack Code

In a quest for speed when emulating python's stack manipulation ops, Dan outlined a plan for manipulating a pseudo-stack using some of Parrot's PMC register (18-29, with P31 used for overflow). Leo wasn't convinced it was needed.

http://groups.google.com/groups?threadm=a06110402bd1ae870b2ad@[10.0.1.3]

string_to_num Precision

Leo's got some problems with the precision of the string_to_num function found in string.c. He doctored things to use atof, but Dan's not so keen on that idea because atof doesn't have standard behavior across platforms. He recommended grabbing Perl 5's string_to_num code (which he thought we'd already done). Nicholas Clark wasn't convinced that this was a good idea; he reckoned that Perl 5's code wasn't good enough. I think Nicholas convinced Dan that atof is actually the right thing to use (with a little preprocessing to deal with C99's atof being incompatible with C89's -- Parrot expects C89 behavior).

http://groups.google.com/groups?threadm=40F28C82.3010904@toetsch.at

GMP Licensing

Armin Obersteiner reported his investigations into the licensing requirements of the GMP maths library. It appears that the FSF's 'GPL Compliance Engineer' reckons we won't need to include the GMP's source code with binary distributions. We just have to make it available from the same source as the distribution. Which is handy.

http://groups.google.com/groups?threadm=20040713210952.GC2452@elch.elche

Parrot Configuration -- A Small Perl Task

Leo asked for volunteers to fix up Configure.pl in order to make it less dependent on the host Perl's configuration. Brent Royal-Gordon provided a teeny-tiny patch that's actually shorter than Leo's description of the problem.

http://groups.google.com/groups?threadm=40F76AE3.4000506@toetsch.at

Meanwhile, in perl6-language

Scalar Subscripting

Remember Guatam Gopalakrishnan's question about subscripting scalars? Discussion continued. One Piers Cawley (who he?) suggested that it should be possible to specify a postcircumfix:[] is rw method in the String class to make string subscripting behave in a helpful fashion. He also suggested pulling this (and other similar behaviors) out into a Sequence trait. Juerd thought the proposed method name was a little verbose compared to Ruby's syntax for doing the same thing and showed off some rather pretty Ruby metaphors.

Another sub-thread discussed interpolation in strings. Larry's changed his mind so that "$file.ext" is now interpreted as "$object.method". You need to do "${file}.ext" or "$( $file ).ext". Or maybe "$«file».ext" by analogy with %foo«bar». James Mastros pointed out that . is rather ambiguous in a literal string; sometimes a full stop is just a full stop.

http://groups.google.com/groups?threadm=a3ada2d104070804124184a442@mail.gmail.com

Pushing Lazy Lists

This thread continues to resemble the lists it is nominally discussing: Infinitely long and possibly endless.

http://groups.google.com/groups?threadm=200407121213.12812.philipp.marek@bmlv.gv.at

Perl 6 and Parrot Essentials, 2nd Edition

Leo pointed us at an O'Reilly press release announcing that the second edition of Perl 6 and Parrot Essentials is available. Huzzah! Let joy be unconfined.

http://groups.google.com/groups?threadm=40F26C12.6030704@toetsch.at

Rules to Match Abbreviations

Hans Ginzel wondered wondered about adding something to the rules system to easily match specified abbreviations of a string. Luke Palmer came up with a (pretty ugly) solution that he then wrapped very neatly in a rule.

http://groups.google.com/groups?threadm=20040713071222.GC338@matfyz.cz

Enhancing open

There's a good deal of mileage to be had talking about how to improve the open function it seems.

Larry continues to come up with new and interesting ways of designing the language in such a way that almost every decision about how to do something can be deferred. Speaking as someone who wishes he could just write Application.new.run and then debug his way to working code, I'm all for this kind of late binding.

It's also amazing how much use Larry's getting out of his colon. (The character, obviously).

http://groups.google.com/groups?threadm=20040713132455.GC27097@babylonia.flatirons.org

Cartesian Products and Iterations

Michele Dondi wondered whether Perl 6 will support Cartesian products between lists. Short answer: Yes. It's probably called outer.

http://groups.google.com/groups?threadm=Pine.LNX.4.58.0407131513280.30821@q.pcteor1.mi.infn.it

Do We Really Need Filehandles?

Dave Whipp thought the unthinkable (but in a good way) and wondered if we actually need user-visible filehandles, given the tools we have for slicing and dicing strings, manipulating lazy data structures, etc. He proposed being able to write:

    my $text is TextFile("/tmp/foo.txt");
    for $text.lines -> {
        ...
    }

Which is rather neat, isn't it? Personally I'm not sure we can get rid of filehandles, but I do like the idea of a library that implements something like this. Alex Shitov pointed out that there would still be a need for methods like $text.flush() or $text.close().

http://groups.google.com/groups?threadm=20040718172834.89363.qmail@onion.perl.org

Announcements, Apologies, Acknowledgements

OK, so the interview was on Tuesday the 13th of July. It went well; I'm going to be a maths teacher.

If you find these summaries useful or enjoyable, please consider contributing to the Perl Foundation to help support the development of Perl. You might also like to send me feedback at mailto:pdcawley@bofh.org.uk

http://donate.perl-foundation.org/ -- The Perl Foundation

http://dev.perl.org/perl6/ -- Perl 6 Development site

Building Applications with POE

Earlier, we talked about the fundamental principles of application design with POE. Now it's time to put my money where my mouth is and build some actual working code.

To make life a bit easier, let's lay out a very simple problem. Let's say we would like accept and parse data that resembles CGI query strings. This data will be key value pairs in which the key and value are separated by ='s and the pairs themselves are delimited by &. An example string we'll use throughout this article is as follows:

  foo=bar&baz=1&bat=2

By the time we're done, we will have a working Filter and Component to handle this incoming data.

Step 1: A Filter

The first step is building a simple filter to parse this incoming data. As we discussed earlier, filters are much easier to deal with because they are unaware of their environment and the POE context in which they are run. Our filter is made even easier since we are just parsing incoming data and not generating an outgoing datastream.

First off, we need the basics of any good module.

  package POE::Filter::SimpleQueryString;

  use warnings;
  use strict;

  use Carp qw(carp croak);

Next we need a constructor.

  sub new { 
    my $class = shift;
    my $self = bless {}, $class;
    return $self;
  }

This is about the simplest constructor possible. This very simple filter requires no parameters to operate. It is perfectly reasonable, however, to demand parameters of filter users. For instance, if the filter could rot13 the incoming data before parsing, and a parameter could turn that feature on.

get()

Now we need the ability to parse data. We will be using the newer and much simpler get/put version of the Filter API. This version of the standard POE Filter API requires get() and put() methods with the ability to transform multiple record sets per invocation.

get()'s job is transform raw data into cooked record sets. The example string above (foo=bar&baz=1&bat=2) will become a hash:

  $VAR1 = {
    'foo' => 'bar',
    'baz  => '1',
    'bat  => '2',
  };

A POE Filter is just a normal Perl object with a defined interface.

  sub get {
    my $self = shift;
    my $buffer = shift;

The buffer can and probably will contain multiple records. The size of the buffer is determined by the POE Driver being used and the operating system in question. $buffer will always be an array reference. While it is generally sensible to test $buffer to make sure it conforms to the standard interface, for the purpose of this exercise, we will just trust POE.

In our super-easy format, an individual record is terminated by a \n. Key value pairs are delimited by & and key and value themselves are separated by an =. Note that we aren't dealing with issues like character escaping or data taint. Production quality code will need to deal with these issues.

    my @chunks; 

Each parsed line makes up a chunk of data. We want to represent each record as a distinct entity to the user.

    foreach my $record (@$buffer) { 
      $record =~ s/\x0d\x0a$//;
      my @pairs = split(/&/, $record);
    
      my %chunk;
      foreach my $pair (@pairs) {
        my ($key, $value) = split(/=/, $pair, 2);

So what happens if there is more than one instance of a given key in a record? Simple. We make an array reference. The user will need to inspect the value of each key to determine if they have more than one value:

        if(defined $chunk{$key}) {
          if(ref $chunk{$key} eq 'ARRAY') {
            push @{ $chunk{$key} }, $value;
          } else {
            $chunk{$key} = [ $chunk{$key}, $value ],
          }
        } else {
          $chunk{$key} = $value;
        }
      }
      push @chunks, \%chunk;
    }
    return \@chunks;
  }

put()

We now have a simple query-string-like data parser. This is fine for read-only servers but it makes sense to allow our users to send data back and forth in the same format. To allow for that, we need a put() method. put()'s job is take the cooked form of our records and translate it to the raw form. In this case we will be taking a hash reference that looks like:

  $VAR1 = {
    'foo' => 'bar',
    'baz' => '1',
    'bat' => '2',
  };

And transforming it into:

  foo=bar&baz=1&bat=2

So that whatever Wheel our user has chosen can put the data onto the wire. Like get(), put() is a normal method call on a normal perl object.

  sub put { 
    my $self = shift;
    my $records = shift;

Like get(), the data to act on is passed in as a parameter to the method call. It is always an array reference of records to translate.

Basically, our put() method performs an exact reversal of the get() algorithm. Take each key/value pair and concatenate them with an =. Each pair is then joined with a &.

The only real error condition we are checking for is the presence of non-array data. Since we haven't defined behavior for this condition, we warn the user about the data and skip it:

    my @raw;
    foreach my $record (@$records) {
      my @chunks;
      foreach my $key (sort keys %$record) {
        if(ref $record->{$key}) {
          if(ref $record->{$key} eq 'ARRAY') {
            foreach my $value ( @{ $record->{$key} } ) {
              push @chunks, $key."=".$value;
            }
          } else {
            carp __PACKAGE__." cannot handle data of type 
            ".ref $record->{$key};
          }
        } else {
          push @chunks, $key."=".$record->{$key};
        }
      }
      push @raw, join('&',@chunks)."\x0d\x0a";
    }
    return \@raw;
  }

The raw data is returned to the caller as an array reference of data chunks. The caller has the responsibility of putting the data on the wire in the appropriate fashion.

Step 2: A Wheel

Chances are that our users want to send data in one of the standard UNIX methods -- sockets, pipes, and so on. Lucky for us, POE already has Wheels to deal with just about any methodology of data transfer you can imagine. Let's work with a method that should work just about anywhere, TCP sockets. POE::Wheel::SocketFactory provides the functionality we need. First, we need a session to plug the wheel into. (Remember that wheels mutate sessions to provide new functionality.)

  POE::Session->create(
    inline_states => {
      _start      => \&start,
      factory_success => \&factory_success,
      
      client_input  => \&client_input,
      client_error  => \&client_error,

      fatal_error   => sub { die "A fatal error occurred" },
      _stop       => sub {},
    },
  );

  POE::Kernel->run();

This session will be our controller for the wheels we need to perform socket operations. Each wheel-based event provides a unique identifier so it is possible to handle more than one client per session.

When the session starts up, we spin up the SocketFactory wheel. With the Reuse flag on, SocketFactory will continuously listen on the specified port and address, handing us events for each client. The unique id passed to the SuccessEvent identifies each client.

  sub start {
    $_[HEAP]->{factory} = POE::Wheel::SocketFactory->new(
      BindAddress   => '127.0.0.1',
      BindPort    => '31337',
      SuccessEvent  => 'factory_success',
      FailureEvent  => 'fatal_error',
      SocketProtocol  => 'tcp',
      Reuse       => 'on',
    );
  }

When a client makes a connection, the SocketFactory lets us know. It is our job to figure out what to do with the filehandle SocketFactory built for us. In this case, we want read/write functionality using the filter we built above. POE::Wheel::ReadWrite provides this functionality, including the ability to plug in our filter.

  sub factory_success {
    my( $handle, $wheel_id ) = @_[ARG0, ARG1];
    $_[HEAP]->{clients}->{ $wheel_id }  =
      POE::Wheel::ReadWrite->new(
        Handle    => $handle,
        Driver    => POE::Driver::SysRW->new(),
        Filter    => POE::Filter::SimpleQueryString->new(),
        InputEvent  => 'client_input',
      );
  }

Now the data path is set up. We have the ability for programs to connect to a port and provide data in our simple format. What to do with the data though? Let's simply print it out and echo it back to the client.

  sub client_input {
    my ($input, $wheel_id) = @_[ARG0, ARG1];

    use Data::Dumper;
    print Dumper $input;

    $_[HEAP]->{clients}->{ $wheel_id }->put( $input );
  }

Data::Dumper handles printing out the structure for us. The put() call puts the structure back out onto the wire. If our algorithms are correct, we should get the same data back that we put in.

  sungo@cthulu% telnet localhost 31337
  Trying 127.0.0.1...
  Connected to localhost.
  Escape character is '^]'.
  foo=bar

The server prints out:

  sungo@cthulu% perl -Ilib examples/server.pl
  $VAR1 = { 
    'foo' => 'bar'
  };

And then echoes back to us:

  foo=bar

We're in business.

Step 3: A Component

Man, that was a lot of code to get a simple TCP server up and running. Surely this can be simplified. Again, POE itself comes to the rescue. POE ships with a component specifically designed to simplify TCP server creation. We can replace all that code above with a simple call to the component's constructor.

  POE::Component::Server::TCP->new(
    Address => '127.0.0.1',
    Port  => '31337',

    ClientFilter => "POE::Filter::SimpleQueryString",
    ClientInput => sub {
      my $input = $_[ARG0];
      use Data::Dumper;
      print Dumper $input;

      $_[HEAP]->{client}->put($input);
    }

And we're done. The downside is that Server::TCP doesn't allow for argument passing to the filter's constructor and we lose the flexibility of doing things by hand. For a lot of problems, however, this component does the trick quite nicely.

We can make this even easier for our users by making our own component. For the purpose of this example, we're going to wrap the smaller code above instead of the larger wheel based example. There is no reason why you couldn't use the wheel-based code in your component, however.

  package POE::Component::SimpleQueryString;

  use warnings;
  use strict;

  use POE;
  use POE::Component::Server::TCP;

  use POE::Filter::SimpleQueryString;

  use Carp qw(croak carp);

  sub new {
    my $class = shift;
    my %args = @_;

    my $addr = delete $args{ListenAddr} || croak "ListenAddr required";
    my $port = delete $args{ListenPort} || croak "ListenPort required";
    my $input_event = delete $args{InputEvent} || 
      croak "InputEvent required";

    my $server = POE::Component::Server::TCP->new(
            Address => $addr,
            Port => $port,

            ClientInput => $input_event,
            ClientFilter => "POE::Filter::SimpleQueryString",
           );
    
    return $server;
  }

  1;

Now our users can just load up the component like so:

  POE::Component::SimpleQueryString->new(
    ListenAddr => '127.0.0.1',
    ListenPort => '31337',
    InputEvent => sub {
      my $input = $_[ARG0];
      use Data::Dumper;
      print Dumper $input;

      $_[HEAP]->{client}->put($input);       
    },
  );

Conclusion

We have seen how to build POE filters and components and combine them with wheels and custom code to create flexible and maintainable programs. The code examples provided above may be downloaded under the BSD License.

Accessible Software

Last year Perl.com published an article about pVoice. You learned what pVoice is, why it was built, and where it was supposed to go. Maybe you wanted to do something for disabled people yourself, by writing something like a mail client, a simple game like memory, or an instant-messaging client, but you didn't implement it because you felt it was too much work. In that case there's news for you.

Part of pVoice was a set of modules that was actually the engine for the user interface. A few months ago I pulled those modules from the pVoice project and made them into a CPAN distribution. This distribution is called AAC::Pvoice. And no, they have nothing to do with Apple's digital audio format. They're all about Augmentative and Alternative Communication. Although, I like to think that the "C" stands for "Computing," which makes it into a better description of what those modules are for.

You can use the AAC::Pvoice modules to create GUI applications for people who have difficulty using conventional input devices like a mouse or a keyboard. Instead, those people can use your applications using only one or two keystrokes, mouse buttons, or switches on a wheelchair. You can also enhance the use of a conventional mouse or touch screen by highlighting objects on the screen as the mouse cursor hovers over them. AAC::Pvoice does not only handle the input for you, but it also provides an accessible graphical user interface.

Think of the possibilities: there are so many modules on CPAN that in conjunction with AAC::Pvoice can result in simple, accessible applications. For example, a module like Games::Battleship allows you to create a battleship game, AAC::Pvoice allows you to create the GUI and takes care of the input for you, so with very little effort you can create a game accessible for people who can't play regular games that need full control over a mouse and/or keyboard.

This article will explain how to use the AAC::Pvoice modules step by step. It's up to you what kind of functionality will go into the application you build.

The API of the AAC::Pvoice modules is still subject to changes, but you can trust me when I tell you that the behavior of existing methods won't change too much, because I don't like to go over my own code over and over again to adjust it to a new version of an API.

The Concept

The graphical user interface of AAC::Pvoice is based upon wxPerl classes, so it won't hurt if you already know a bit about wxPerl. But since the modules handle most of the GUI for you, as you can see below, it's much simpler to write AAC::Pvoice applications than using plain wxPerl.

Basically an AAC::Pvoice-based application consists of three entities: a Panel, a Row and a Button. A panel is a subclass of a Wx::Panel, on which one or more rows, or in fact any other Wx::Window subclass, can be placed. A row is also a subclass of a Wx::Panel, but a row can contain multiple Wx::BitmapButton objects. Those Wx::BitmapButtons however are created from a data structure you feed to the row's constructor. These buttons don't respond to mouse clicks on them like you're used to, but will be invoked when the user generates a 'Select' event, unless of course you use a normal mouse to control the GUI. The 'Select' event will be discussed later.

There is a special class called AAC::Pvoice::Input. You normally won't call any of this class' methods directly. Instead, it's called from the AAC::Pvoice::Panel. The Input class checks for 'normal' mouse, mouse buttons, keyboard or parallel port input (depending on the selected device) and calls whatever the Panel defined it to do when a 'Next' or 'Select' event occurs. If only one switch is used, that switch will only generate a 'Select' event, and a 'Next' event will automatically happen every n milliseconds, depending on your settings. To understand what those events are, you need to understand how the so called 'row/column scanning' mechanism works. Since we've got multiple rows of buttons, and we have only two switches to indicate what we want to do, we can't move the mouse cursor over the screen. Instead, we'll use one switch to indicate 'go to next row' and one to indicate 'select this row'. When we've selected a row, again we need the same two switches to indicate 'go to next button in this row' and 'select this button'.

As you will see in the demo application below, you don't have to worry about row/column scanning when you're creating your application. The AAC::Pvoice::Panel takes care of that. You only have to define what your script will do when a certain button is invoked.

Creating a Demo

To demonstrate how you can use AAC::Pvoice to create your own pVoice-like applications, I'll show you how to build a simple demo application. This application will use the Microsoft Agent (using the Win32::MSAgent module) to move the Merlin character over the screen and let it pronounce a phrase. It's a very simple, silly example, but then again, it's only to demonstrate how the AAC::Pvoice modules work. You'll see how easy it is.

First of all, because an AAC::Pvoice based application is actually a wxPerl-based application, we have to start with a standard wxPerl framework, where we define the application and the frame, and start a main loop. This can be something like this:


use strict;
use warnings;

# Create a wxApp object and start the mainloop
my $obj = pMerlin->new();
$obj->MainLoop;

#----------------------------------------------------------------------
# the pMerlin package implements the Wx::App subclass
package pMerlin;
use Wx;
use base 'Wx::App';

sub OnInit
{
    my $self = shift;
    # Set an AppName and VendorName
    $self->SetAppName('Fun With Merlin');
    $self->SetVendorName("pVoice Applications - Jouke Visser");
    
    # Create a frame and show it
    my $frame = pMerlinFrame->new( undef, -1, 'pMerlin');
    $frame->Show(1);
}

#----------------------------------------------------------------------
# the pMerlinFrame package implements the Wx::Frame subclass
package pMerlinFrame;
use Wx qw(:everything);
use AAC::Pvoice;
use base 'Wx::Frame';

sub new
{
    my $class = shift;
    # Call the superclass' constructor
    my $self = $class->SUPER::new(@_);
    
    # Set the white background colour
    $self->SetBackgroundColour(wxWHITE);
    
    return $self;
}

So far, nothing special. I only used a few calls that aren't really necessary for a default wxPerl script: I'm setting the AppName, VendorName, and I'm telling the frame to have a white background. The reason for setting the AppName and VendorName will be explained later.

Using AAC::Pvoice

Like I said, so far everything looks normal, if you're used to wxPerl programming. But in fact this is where the actual work begins. We now need to add some code to the new() subroutine of pMerlinFrame. The first thing we need to do now is to maximize the pMerlinFrame we just defined and get the dimensions of that maximized state. We need to do this to pass the correct width and height of the frame to the AAC::Pvoice::Panel we want to create. We can define a margin around the AAC::Pvoice::Panel, which looks nicer, but it's not necessary. In this example I'm using a margin of 50 pixels on each side of the panel.


# Get the dimensions of the maximized window
$self->Show(1);
$self->Maximize(1);
my ($width, $height) = ($self->GetClientSize->GetWidth,
                        $self->GetClientSize->GetHeight);
$self->Show(0);

# define the margin
my $margin = 50;

# define the itemspacing
my $itemspacing = 4;

# And create a pVoice panel
$self->{panel} = AAC::Pvoice::Panel->new(
                         $self,              # parent
                         -1,                 # id
                         [$margin, $margin], # position
                         [$width-2*$margin, $height-2*$margin],
                                             # size
                         wxNO_3D,            # style
                         1,                  # hide textrow
                         $itemspacing,       # spacing between items
                         3);                 # selectionborderwidth
# The grey background for the panel
my $panelbackground = Wx::Colour->new(220,220,220);
$self->{panel}->BackgroundColour($panelbackground);

# A title at the top of the panel
$self->{panel}->AddTitle("Fun with Merlin");

As you can see I've also defined a background colour for the panel and a title on top of it. Let's take a closer look at the parameters of the AAC::Pvoice::Panel constructor. The first parameters look exactly like those used for Wx::Panel. The AAC::Pvoice::Panel, however, needs three new parameters. The first indicates if we should hide the (default) text row (a Wx::TextCtrl) at the bottom of the panel. For an application like pVoice, where you create phrases, this text row is of course needed. For our demo we don't need a text row. Furthermore, we need to define the spacing between items on the panel. Effectively this is the spacing between the rows, since those will be placed directly upon this panel. Finally we need to define the width of the border around a selected row or item. It's a good idea to make this value lower than the item spacing, otherwise the border will not be fully visible.

The next step is to create the bitmaps we want to display. For this purpose we can use AAC::Pvoice::Bitmap. This class provides an easy way to create a bitmap of a certain size, put a caption underneath it, and has some other useful features. Before we create the bitmaps, we calculate the maximum X and Y size they can use by dividing the useable X and Y size of the panel by the number of buttons per row and the number of rows respectively. I'll also subtract twice the item spacing used in the application because otherwise the buttons will be a little bit too large.


# The maximum dimensions of a button
my $maxX = int(($self->{panel}->xsize)/4)-2*$itemspacing;
my $maxY = int(($self->{panel}->ysize)/3)-2*$itemspacing; 

# Define the bitmaps. 
# The parameters are the filename, maxX and maxY size and the caption
my $leftbutton     = AAC::Pvoice::Bitmap->new('img/left.png',
                                              $maxX, $maxY, 'left');
my $rightbutton    = AAC::Pvoice::Bitmap->new('img/right.png',
                                              $maxX, $maxY, 'right');
my $upbutton       = AAC::Pvoice::Bitmap->new('img/up.png',
                                              $maxX, $maxY, 'up');
my $downbutton     = AAC::Pvoice::Bitmap->new('img/down.png',
                                              $maxX, $maxY, 'down');
my $questionbutton = AAC::Pvoice::Bitmap->new('img/question.png',
                                              $maxX, $maxY, 'question');
my $rowselbutton   = AAC::Pvoice::Bitmap->new('img/rowsel.png',
                                              $maxX, $maxY, 'rowsel');

Now that we've got those bitmaps, we can create the AAC::Pvoice::Rows that will use those bitmaps, and know what coderefs to invoke when the user selects one of the buttons. We don't need to create individual Wx::BitmapButtons. The AAC::Pvoice::Row takes care of that, and attaches the correct actions to the BitmapButtons in such a way that they can be invoked using any of the available input methods.

Before define the rows, we create a listref for every button, containing the button ID, the bitmap and the callback. These listrefs we put in another list of lists. That LoL defines the whole panel, and every list inside it defines the layout of each every row. Let's see how we achieve this:


# Define the buttons on the pVoice rows
#               the id,      the bitmap,     the callback
my $left     = [Wx::NewId(), $leftbutton,    sub {$self->MoveLeft}];
my $right    = [Wx::NewId(), $rightbutton,   sub {$self->MoveRight}];
my $up       = [Wx::NewId(), $upbutton,      sub {$self->MoveUp}];
my $down     = [Wx::NewId(), $downbutton,    sub {$self->MoveDown}];
my $question = [Wx::NewId(), $questionbutton,sub {$self->Speak}];
my $rowsel   = [Wx::NewId(), $rowselbutton,
                sub {$self->{panel}->ToRowSelection}];

# The definition of the screenlayout
my $content = [ [ $rowsel,    undef,    $up,       undef  ],
                [ $rowsel,    $left,    $question, $right ],
                [ $rowsel,    undef,    $down,     undef  ] ];

You can see that the callbacks actually invoke some methods of pMerlinFrame that I haven't discussed yet. That's because they have nothing to do with the AAC::Pvoice modules themselves. I'm trying to keep this demo as simple as possible. At the bottom of this article you'll find a link to a zip file that contains the complete source and images used in this demo.

The final step in creating the panel is creating the rows based upon our data structure, and then add those rows to the panel. After that there is a magic method called 'Finalize' we need to call on the panel, and the panel is done. First the code, then an explanation:


# Add the buttons to the rows and the rows to the panel
foreach my $row (@$content)
{
  $self->{panel}->Append(AAC::Pvoice::Row->new(
                                 $self->{panel},     # parent
                                 scalar(@$row),      # max
                                 $row,               # items
                                 wxDefaultPosition,  # pos
                                 wxDefaultSize,      # size
                                 $maxX,              # X-size
                                 $maxY,              # Y-size
                                 $itemspacing,       # spacing
                                 $panelbackground)); # background
}
# Finish the panel
$self->{panel}->Finalize();

The constructor of a row has a few unusual parameters. The first parameter is the parent, like almost every wxPerl class needs to know the parent window. The second parameter however, is the maximum number of items in a row. The row will add empty Wx::BitmapButtons to the end of a row if the third parameter (the items we want to put on the row) contains less items than the defined maximum. This ensures an even sized row. In our case we have rows with 4 items each, including some undefined items, so passing scalar($@row) is enough here.

The $row parameter contains the list of items we want to have in our row. We also have to pass the $maxX, $maxY and $itemspacing to this row, since it has to use the correct spacing between the buttons, and the empty buttons it needs to create (for the undef values in the row) need to have the correct size. The final parameter, $panelbackground, is the background color for this row, which we define to be the same as the panel background to let it look like there's no difference between the rows and the panel.

The final thing we have to do is to define how the user can use the application. Are we going to use keystrokes, mouse buttons, the Adremo wheelchair, or normal mouse input? We can define this by setting the registry-key Device to respectively keys, icon, adremo, or mouse. This needs to be set before the AAC::Pvoice::Panel is created. I usually set this in the OnInit method of my Wx::App subclass like this:


# Set the device we use to 'icon' (left and right mouse buttons)
# other possibilities are 'keys' (keystrokes, where the space 
#                           and enter key are the default keys)
#                         'adremo' (electrical wheelchair) and
#                         'mouse' (normal mouse, touch screen, etc.)
my $config = Wx::ConfigBase::Get;
my $Device = $config->Read('Device', 'icon');
$config->Write('Device',   $Device);

If we do this after we define the AppName and the VendorName, the key will appear under HKEY_CURRENT_USER\Software\<VendorName>\<AppName>\Device. There are more possible keys to define, like Buttons (either 1 or 2, defining the number of switches). You can find more information on this in the documentation of AAC::Pvoice::Input.

This is all there is to it. We now have a simple demo of an application that almost everyone is able to use. The screenshot you see on this page is the result of these simple pieces of code. As you could see, the API is extremely simple, so you can concentrate on the actual functionality you want to put in your application.

Platform Issues

The AAC::Pvoice modules can be installed on Windows as well as on Linux. However, it doesn't look too well on Linux. If someone is interested in making the modules work correctly on Linux: patches are welcome! I would also like to see someone trying to port everything to OS X. wxPerl runs on a lot of different platforms, but in the AAC::Pvoice modules I've used some Win32-specific options that may have some workarounds for other platforms. I just haven't had the opportunity to make it work on those.

Hopefully this short article inspires you to build something for people who can't quite accomplish everything you can. After all, this is what programming should really be about -- making useful applications for people who can't build it themselves.

You can download the zipfile containing the complete script and images here.

Autopilots in Perl

X-Plane is an ultra-realistic flight simulator used by aviation pioneers like Burt Rutan, who uses it in his simulator for SpaceShipOne. The latest revision sports a flight model so accurate that the FAA has decided that your time spent in the simulator can count toward a pilot's license.

We'll learn how to monitor and control our virtual plane, and display our virtual aircraft's status in real-time in a Curses window. Our display will be data-driven so that adding a single entry to a hash reference is all you need to display a new data element.

Software Requirements

Curses on OS X

Curses v1.06 seems to have some issues building on the author's machine running OS X 10.3. Replacing Perl_sv_isa with sv_isa on line 275 of Curses.c, along with commenting out #define C_LONGNAME and #define C_TOUCHLINE in c-config.h lets Curses compile with enough functionality to run the monitor program.

Configuring the Sim

Throughout this article, we assume that you're using X-Plane version 7.41 or higher (the demo version has all the functionality we need), and you're flying the standard Boeing 777. This has retractable landing gears and brakes, two features we'll eventually control from inside the monitor program, X-Plane.pl.

Once you're settled in your airplane's cockpit, choose the Settings menu at the top of the screen and choose Data Input & Output under that menu. A window should open to the default tab, Data Set. This is where we tell the simulator which channels of data to transmit over the network where our monitor can listen for them.

Currently, X-Plane.pl displays gear up/down status, throttle percentage, latitude and longitude, pitch and roll, and the simulation frame rate. To enable these channels for display, click the checkboxes just to the right of rows 12, 23, 18, 16, and 0 (respectively).

If you like, you can also click the checkboxes on the far right of each column to display the channel data on the plane's windshield, as well. This is handy for debugging, especially if you're attempting to monitor new data and don't know what the values look like. Also, the channel numbers seem to change version to version, and if you're using a version other than 7.41, you may need to change the code to correspond to the simulator.

Finally, the simulator has to be told where to send the monitoring packets. Choose the Inet2 tab, and click on the checkbox next to IP address of data receiver. Fill in the left-hand box with the IP address of the machine you're running X-Plane.pl on, and fill in the right-hand box with 49999, the UDP port on which X-Plane.pl listens.

Incidentally, even though the monitor program acts as an instruction console, you don't need to turn on the Instructor Console option listed here. It may be necessary, should you want to deal with more advanced communication such as triggering system failures, but for current purposes all we need to configure is the data receiver.

By default, X-Plane.pl listens to packets sent to 127.0.0.1 over port 49999, and sends packets to the same host on port 49000. Passing command-line parameters to X-Plane.pl can override these settings, but the documentation for the simulator seems to indicate that it's hardcoded to listen on port 49000 for its commands, so be warned if you try to change this particular default.

Running the Monitor

Communicating at the UDP layer is a different, and much simpler, world than dealing with a full TCP/IP packet. The main differences here are that we don't get any acknowledgement that a UDP packet is sent, and can't even guarantee that the packet was sent at all. The practical upshot of this is that we can start and stop X-Plane.pl at any time, without the need for fancy socket tear-down protocols, and we don't need to stop the simulator every time we change X-Plane.pl's code.

Once we've started the simulator and X-Plane.pl it should start displaying the plane's position (longitude/latitude), orientation (pitch/roll), speed, and current throttle settings. If it doesn't, switch back to the simulator window and make sure the cockpit is being displayed. The gear display mimics an indicator lamp, with [#] representing a lit lamp, and [ ] being a dark lamp.

The simulator will only send out UDP packets when the cockpit is being displayed, and if you're running in demo mode, it decides to interrupt your flight six minutes in with two dialog boxes that effectively stop packet transmission. If it's still displaying the cockpit, make sure that the simulator is transmitting data to the right port on the right machine, and if all else fails, watch the network with a utility such as tcpdump to make sure UDP packets are being sent out. Press the G key to raise and lower the gear and the I and K keys to advance and retard the throttles, even on the runway. Note that the gear indicator lamp on the monitor will change before the gear handle does. Also, if running the simulator on the same machine as the monitor, you may need to bring the window with the simulator to the foreground before changes are registered.

By the way, the Boeing 777 is equipped with thrust reversers, so negative throttle settings actually make sense. Also, the flight profile dictates running the engines up to 110% of rated power on takeoff, so throttle settings beyond 100% are also legitimate.

Talking to X-Plane

Communicating with the simulator is done entirely via UDP packets. While the simulator can send and receive various types of packets, we'll focus on one type of packet in particular, the DATA packet. The simulator communicates using packets like that described in Figure 1. The first 4 bytes of the header are the ASCII characters naming the type of packet, and the actual data is surrounded by a zero byte on both ends.

In between the zero bytes, we get one 36-byte chunk for every channel the simulator sends out. Each 36-byte chunk is broken down into two sections. The first section is a four-byte index corresponding to the channel number on the "Data Set" screen, and the rest of the chunk contains eight four-byte elements.

The actual data types of the individual elements has changed from version to version, but 7.41 seems to have settled on one layout for inbound, and a different one for outbound. When transmitting data, the simulator sends all elements as floating-point types, but receives a mixture of floating-point and long integers.

Sending Data

Handling the mixture of data types is largely the job of the core data structure in X-Plane.pl, the $DATA_packet hash reference. Starting on line 92, this data structure encapsulates everything we want to know about the simulator packet.

Given that there are currently 113 channels of data that the simulator can send out, and that the author is lazy, we're not going to type in all 113 channels' worth of data. Instead, we'll store just the channels we want to display in what computer scientists call a "sparse array," but Perl programmers call it a "hash reference."

One useful feature the monitoring program doesn't have is controlling the brakes. In the simulator, brakes can be applied with a variable pressure, but for our purposes they only have two settings: "Screeching halt" and "Let it roll." The first order of business is to find out what channel brakes are displayed on.

Going back to the "Data Set" tab in the simulator, we look for "brakes" on the list. In version 7.41 they're on channel 12, "gear/brakes." Conveniently enough, we're already displaying the gear setting, so it shouldn't be much work to add brakes to the list.

Now that we know what channel brakes are on, we need to know where in the channel they're stored. After all, we have eight elements to look through. So, click on the checkbox on the far right of row 12 to display that channel's contents on the windshield and close the window.

At the upper right-hand corner we should see the channel contents being displayed. We're mostly curious about the wbrak setting, which is element number 1 (counting from zero, of course.) The simulator also tells us that it's of part type, which means that it's a floating point value.

We'll first need to display the brake setting. We do this at line 112, duplicating the block for the gear status and creating the following entry, which specifies that element 1 of channel 12 is a floating-point number representing brakes:

    1 => { type => 'f', label => 'Brakes',
           label_x => 0, label_y => 4,
           x => 7, y => 4 },

Restart the monitor program, and you should have a new entry for "Brakes" followed by a 0 or 1. Clicking on the "BRAKES" light in the cockpit should toggle the value. Once you've verified that the new brake display works, it's time to add the code that actually sets/releases the brakes. On line 270, insert the code that transmits the new brake status to the simulator, that looks roughly like this:

  elsif($ch eq 'b') {
    transmit_DATA(
          $socket,
          12,
          -999,
          $DATA_buffer->{12}{1} ? 0 : 1,
          (-999) x 6);
  }

Since the gear and brakes are on the same channel, we have to tell the simulator to ignore the gear while letting us set the brake value, so we use the placeholder value of -999. The old brake value is saved in $DATA_buffer-{12}{1}>, so we set the opposite of whatever was there before.

Rerun the monitor program, and pressing "b" should clear and set the "brake" light on the simulator's indicator panel. When the light is out, the aircraft should naturally start rolling, and the simulated light on the Curses window should go out.

Of course, not all types of variable are as simple to deal with as the brakes and gears. The throttle settings should give you an idea of how to work with more advanced data types, and of course you're not restricted to a Curses display. It just happens that Curses is fairly convenient and low-bandwidth, but nothing is preventing you from transplanting the code into a GTK event loop.

While you can change values in every channel, the simulator may not react to those changes. For instance, you can't change the latitude and longitude of the plane and expect it to teleport to the new location.

The values for latitude and longitude are actually computed from the values in channel 19, the X, Y and Z coordinates. Changing these will actually affect the plane's position in space. However, there doesn't appear to be a simple reference that will describe what values are considered read-only.

The official UDP documentation doesn't contain this, and the only UDP FAQ on the 'net for X-Plane appears not to have been updated since version 6. Of course, some of the channels make no sense on certain aircraft. For instance, the Cessna 172 (which the author has several hours' experience in) doesn't have retractable gear.

Receiving Data

Receiving data is also done through the $DATA_packet sparse array. In this case, the element type is just used for display purposes, as the channel is transmitted in floating point, and not the mixed format. The entry for "True Speed", which is displayed in miles per hour, looks like this:

 104 2 => {
 105   0 => { type => 'mph', label => 'True Speed',
 106          label_x => 0, label_y => 1,
 107          x => 12, y => 1 },
 108 },

When time comes to walk the DATA packet, this entry tells us to see if we've received data channel 2 (line 104). If we have, look at element 0, and display the element at (12,1) in the window (line 107.) Since the type is "mph" (line 105), we know that we have to format that specially.

Special types like "mph" and "deg" are stored in another hash reference, back on lines 81-85. When the time comes to display the actual data, we look into this hash reference to pull out the format string to use when sprintf()'ing the data. The len hash key gets used as well at this time, to create a mask of space characters that we use to erase the old value completely before displaying the new value.

  81 my $typedef = {
  82   deg => { format => "%+03.3f", len => 8 },
  83   mph => { format => "%03.3f", len => 7 },
  84   pct => { format => "%+01.3f", len => 6 },
  85 };

A special type bool is used for indicator lamps (since indicator lamps are either on or off), but is handled specially. The pseudo-types l and f aren't represented here, but are used when we need to return data to the simulator. While the simulator sends out only floating-point numbers, it receives a mixture of floating-point and integer values, and the mixture changes on a per-channel basis.

Instead of making the programmer create the format strings that we'll use later on to pack() and unpack() packets, we pre-compute them in the function create_pack_strings(). Since the individual elements may occur anywhere in the eight-element array, we may need to skip over elements, and that's done with liberal use of x4 in the format, which tells pack() and unpack() to ignore 4 bytes' worth of data.

 164 sub create_pack_strings {
 165   for my $row (values %$DATA_packet) {
 166     $row->{unpack} = 'x4';
 167     $row->{pack} = 'l';
 168     for my $j (0..DATA_max_element) {
 169       if(exists $row->{$j}) {
 170         my $col = $row->{$j};
 171         $row->{pack} .=
 172           (grep { $col->{type} eq $_ } @float_formats) ? 'f' : 'l';
 173         $row->{unpack} .= 'f';
 174       }
 175       else {
 176         $row->{pack} .= 'f';
 177         $row->{unpack} .= 'x4';
 ...

Starting at line 164, create_pack_strings() handles this tedious job for us, by walking the two-dimensional hash reference $DATA_packet. Line 166 starts the unpack() string with x4, which tells the unpack() function to skip over the index long in the packet. We have to unpack the index beforehand in order to know how to deal with the data, so we just ignore that.

Line 167 starts the pack() string with a long, l for the inbound index. Lines 168 onward create the individual elements. The unpack() strings are f if the element is in use, x4 if it's not. This means that the format strings only extract the data we need, which makes it easier later on when time comes to actually call unpack() on the actual data.

Lines 171-2 and 176 create the pack() format string, using f for floating-point formats and l for integer types. Since there's no special way to tell the simulator what elements we're updating, we have to send back every element. Unused elements are filled with a sentinel value of -999 to say "Do not update this value."

In the end, we've added pack and unpack hash keys to every channel in our $DATA_packet structure. Unpacking a channel structure with this format returns to us only the data we're interested in, and skips over unused elements so we don't have to account for empty elements in the array that pack() returns to us.

Likewise, the pack hash key gives us a string to create an entire channel's worth of data, with the proper data types. This is important, even in what should be a simple channel like the gear and brake display. While gears get set to an integer 1 or 0, brakes have to be set to a float from 0 through 1, to account for variable pressure.

Pulling Apart the Packet

All of the heavy lifting gets done in the receive_DATA() function, from lines 193-234. This function accepts the message sent over the UDP port and breaks it into individual channel-sized packets. The adventure starts on line 196, after clearing a small internal buffer we use to record the last packet received.

  196   for (my $i = 0;
  197        $i < (length($message)-&DATA_header_size-1) / DATA_element_size;
  198        $i++) {
  199    my $channel = substr($message,
  200                         $i * DATA_element_size + DATA_header_size,
  201                         DATA_element_size);

Line 197 computes the number of channels this particular message contains (we're doing this on-the-fly in case you want to change the channel selection while the simulator is running.) The substr() call breaks the message into chunks of DATA_element_size bytes, and gives us back a channel's worth of data.

  203    my $index = unpack "l", $channel;
  204    next unless exists $DATA_packet->{$index};

Next, we extract the index (the first byte) of the channel so that we can unpack the data appropriately. If we don't know anything about this channel (i.e., if it isn't present in the $DATA_packet hashref), we reject it and move on. This makes us somewhat immune to changes in format.

  206    my $row = $DATA_packet->{$index};
  207    my @element = unpack $row->{unpack}, $channel;

Next, we get back the elements we're interested in here by unpacking with the format string that got calculated in create_pack_strings(). The format string skips the index and unpacks just the elements we wanted. So, now we walk our proffered hash and extract the individual elements:

  208     my $ctr = 0;
  209     for my $j (0..DATA_max_element) {
  210       next unless exists $row->{$j};
  211       my $col = $row->{$j};
  212       $DATA_buffer->{$index}{$j} = $element[$ctr];

Line 208 initializes a counter so that we can keep track of which element we've extracted. Line 209 loops through each possible element in order, so we take into account the possibility that we haven't used a data element.

Lines 210-212 skip unused elements in the sparse array, and saves the content in $DATA_buffer, so we can have keys like g toggle the gear setting, rather than having one key to raise the gear and one key to lower them.

Finally, we display each element based on its type. Boolean types are displayed as either [#] or [ ] depending upon whether the element is a floating one or zero. They're still written out as an integer, but displayed as floating.

Types such as deg and mph with a registered format are handled specially. We first wipe out the old data completely by overwriting with spaces. This prevents a potential issue with the old value being "3.14159", and the new value being "2.7". If we didn't overwrite the old value, it would be displayed as "2.74159", with the extra digits remaining from the old display.

If we've not been told what to do with this (which is the case with Frame Rate), simply print the value and go onward.

Data-Driven Design

In the current monitor program, each channel in a DATA packet corresponds to a set of fields onscreen. Instead of creating a function to display the onscreen labels and another one to extract the information from the packets, we've elected to combine the label information with the channel and field, in one easy-to-use format.

Since the number of channels and their layout varies between versions of the simulator, we've chosen to represent the channels in a two-tiered hash reference. The outer tier represents a channel, and the inner tier represents the fields inside that channel.

This means that if a channel changes index number (as the pitch and roll channel did from version 7.1 to 7.4), we simply update the channel number in the hashref rather than cutting out a block of code and repositioning it in an array, making maintenance easier.

Since this data likely won't change over the lifetime of the program, we'll store it in a global hash reference, $DATA_packet starting at line 92. We can reference a given element in an arbitrary channel with the code fragment $DATA_packet-{$channel}{$element}>, but the code usually ends up iterating by channel and by element.

The sample for the gear display starts at line 109, and the entire channel/element reference looks like:

  109  12 => {
  110    0 => { type => 'bool', label => 'Gear',
  111           label_x => 0, label_y => 3,
  112           x => 6, y => 3 },
  113  },

This is the only element in channel 12, and sits at element 0. Boolean types are displayed with [#] and [ ] representing indicator lamps, and it specifies the screen coordinates of the label (label_x and label_y) and where the actual indicator goes (x and y).

After Curses initializes, the setup_display() function iterates over this two-tiered hash and draws the label strings that won't change. Lines 148-156 take care of this, and show how to iterate over the data structure:

  148 sub setup_display {
  149   for my $channel (values %$DATA_packet) {
  150     for my $element (values %$channel) {
  151       $win->addstr($element->{label_y},
  152                    $element->{label_x},
  153                    $element->{label}) if $element->{label};
  ...

Note that $win-addstr()> takes the Y coordinate followed by the X coordinate, in accordance with long-standing tradition. Later on, we'll use the type hash key to tell us how to display this data, but that's handled when we receive a message.

Initialization and Shutdown

We start by handling the usual command-line options on lines 24-32, including a -h option to display usage. -x sets the X-Plane IP address to something other than the default of "127.0.0.1," -r changes the port the monitor listens on from 49999 to something else.

Incidentally, the listening port cannot be 49000, as that's where the simulator listens for its commands. -t tells the monitor to transmit on a different port than 49000, although the simulator is documented to listen to only port 49000. -d is there in case this gets run with earlier versions than 7.4, where the packet format varied depending upon the operating system the simulator was running on.

After command-line configuration is processed and defaults overridden, we create the UDP sockets, on lines 48-65. Instead of the usual TCP protocol, we open UDP ports as that's what the simulator communicates with. If we did this after initializing Curses, our error text would be eaten by the terminal, so we place this first.

Startup and Shutdown of Curses

The Curses startup proceeds fairly normally starting on line 314, with the call to noecho() and cbreak() stopping the terminal from echoing key presses and suppressing carriage returns. $win->timeout(1); lets us read key presses without blocking, so we can display packets as they come in real-time without having to wait for key presses.

Displaying Text

Since Curses implementations vary widely in functionality, we limit ourselves to making addstr() and getch() calls to ensure maximum compatibility across platforms. Inside setup_display() we draw the static labels such as "Gear" and "Pitch," and do the job of displaying the actual values inside receive_DATA().

The Main Loop

This makes the main loop on lines 301-311 pretty straightforward. We poll the keyboard, and display the latest data packet if there isn't one. Otherwise, check to see if the user pressed "q", in which case we quit. UDP sockets don't require any special tear down, so all that's left is to call endwin();. If the user pressed a command key, create an appropriate packet and send that out.

With just the DATA packet, you can create your own customized cockpit display, even take this sample source and turn it into a Gtk application that lets you monitor your plane graphically. The VEHA packet type adds even more possibilities. You could read a friend's virtual location and add his plane as traffic in your virtual world.

Even better, scrape HTML from a flight-tracker service and add real traffic to your virtual world! The FAIL and RECO packet types can cause simulated system failures, so you can create your own in-flight emergencies in Perl! You can even go all the way, and use the SNAP packet type to completely override X-Plane's flight model, telling X-Plane how you think the aircraft should fly.

Hopefully now that I've demystified some of X-Plane's internal workings you'll be inspired to create your own tools, maybe even design and build your own fly-by-wire plane, all in Perl.

This Week on Perl 6, Week Ending 2004-07-04

Another week, another Perl 6 Summary. This is becoming a habit.

Let's do perl6-internals first, shall we?

Japhy's Perl 6 Rules Parser

Last week Jeff 'Japhy' Pinyan announced that he'd be working on a Perl 6 rules parser once he'd finished his Perl 5 regex parser. This week saw a deal of discussion on how his parser worked. The answer seemed to be, "It works the Right Way." Which is good. With any luck Steve Fink will be able to get this parser and the current (sketchy) Perl 6 compiler playing well together.

http://groups.google.com/groups?selm=Pine.LNX.4.44.0406262313030.28219-100000@perlmonk.org

Complexities

Last week Leo pointed out that we'd need support for complex numbers in order to get the Piethon stuff running properly. This week there was some discussion of what was actually needed.

Later in the week Ion Alexandru Morega posted a patch implementing them. I think Leo spoke for us all when he said, "Whee, great, thanks -- applied."

http://groups.google.com/groups?selm=20040627215622.37698.qmail@onion.perl.org

New Mailing Lists

There was some discussion of how perl6-internals should be broken up. It looks like we'll see new lists called parrot-internals, parrot-compilers, and parrot-library (for internals hackers, compiler implementers, and library builders respectively). The original perl6-internals will carry on, but will be focused on the actual internals of Perl 6.

Some, or all, of these lists will be covered in these summaries for as long as I can keep up.

http://groups.google.com/groups?selm=Pine.LNX.4.58.0406280851280.16872@sprite.sidhe.org

Stack Language for Parrot

In as impressive a delurk as I've seen in some time, Michael Pelletier announced that he'd been lurking for a while, playing with Parrot, and had implemented a little stack language not entirely unrelated to Forth, which he'd christened Parakeet. He posted his implementation and got promptly Warnocked.

http://groups.google.com/groups?selm=1088570711.2678.648.camel@heinlein

GMP and Licensing Issues

Scott Bronson announced that he'd taken a look at GMP's license and he didn't think it was incompatible with Parrot's licenses. Dan wasn't convinced. Scott's still trying to convince him though.

Robert Spier attempted to resolve the problem by pointing everyone at another possibly suitable library called IMath.

http://groups.google.com/groups?selm=1088572790.3306.100.camel@lea.rinspin.com

http://groups.google.com/groups?selm=m3wu1o8cb1.wl_rspier@pobox.com

Meanwhile, in perl6-language

Argh! Strings!

Discussion of the various ways of slicing and dicing strings in Perl 6 continued. The issue is that, especially in a Unicode world, there are many ways of looking at a string, all of which are useful in different contexts. However, because you can look at a string as a sequence of bytes, code points, graphemes, or whatever, then functions like substr get a little weird. If you were to say:

    $substring = $a_string, 5

Then what does the '5' mean?

And that's just one example of where conceptual problems can arise.

I confess that, whenever Unicode comes up, my gut reaction is to keep my head down and trust that Larry's going to get it right. Certainly the current formulation seems decently sane; things only get problematic if you're trying to do something well out of the ordinary.

http://groups.google.com/groups?selm=20040626202059.GA1747@wall.org

if, loop, and Lexical Scope

Discussion of Perl 6's new scoping rules (things are generally more consistent than they are in Perl 5; if a variable is declared inside a block, or in that block's signature in the case of -> $a, $b {...} type blocks, then that block is its scope) continued. Some people like the new rules, others don't.

http://groups.google.com/groups?selm=20040627211611.GB3013@babylonia.flatirons.org

Different OO Models

Jonadab the Unsightly One had wondered about having objects inheriting behavior from objects rather than classes in Perl 6. The gist of the answers he received was that it wouldn't be in the core of the language, but neither should it be too hard to implement something that worked how he wants.

http://groups.google.com/groups?selm=n02nraxl.fsf@jonadab.homeip.net

Undo?

Michele Dondi wondered if Perl 6 would have a built-in undo function for rolling back the history of a scalar (say). Rafael Garcia-Suarez pointed out that the concept wasn't as simple as it first appeared, especially in the context of threads, closures, side effects. He suggested that a better approach would be to implement a suitable transaction/rollback library that handled such things in an application-appropriate fashion rather than using a "half-baked kludge built into the base language." Mark Biggar pointed out that the language already had hypothetical values, which did pretty much the right thing.

Elsewhere in the thread, Luke Palmer attempted to explain continuations again. A sandwich was involved.

http://groups.google.com/groups?selm=Pine.LNX.4.56L0.0406291740180.10615@worf.pcteor1.mi.infn.it

Luke makes a continuation sandwich --
http://groups.google.com/groups?selm=20040629233129.GA3979@babylonia.flatirons.org

If Not , Then What?

Alexey Trofimenko triggered your Summarizer's sense of déjà vu when he asked what was happening to the C-style comma (we're keeping it, more or less) and proposing a then keyword for use in the contexts where Perl 6's new , didn't quite work the same way as in Perl 5. Which is pretty much the same as the proposal Luke Palmer made some months ago and which Larry rejected.

Jonathan Lang pointed out how you'd implement then if you needed it, though.

The thread got slightly silly after this (Sideways semicolons! I ask you!). I'm not sure what chromatic was driving at when he suggested a meh operator, though.

http://groups.google.com/groups?selm=opsafpwqtgsspxk8@s

map, grep and Laziness

That man Alexey also had some questions about lazy evaluation. Luke Palmer attempted to allay his fears by saying that Perl 6 would be essentially lazy unless you told it otherwise.

http://groups.google.com/groups?selm=opsajn0im2sspxk8@s

Announcements, Apologies, Acknowledgements

Woohoo! Another weekly summary.

If you find these summaries useful or enjoyable, please consider contributing to the Perl Foundation to help support the development of Perl. You might also like to send me feedback at mailto:pdcawley@bofh.org.uk

http://donate.perl-foundation.org/ -- The Perl Foundation

http://dev.perl.org/perl6/ -- Perl 6 Development site

This Week on Perl 6, Week Ending 2004-06-27

What's this? No! It can't be! It's a weekly Perl 6 Summary. What is the world coming to?

Sorry, I can't answer that one, so I'll tell you what's been happening this week in perl6-internals.

Bignums, licenses, pie

As you are no doubt aware, Python has bignums. And that means that, unless we want Dan to get a face full of Pie at OSCON this year, Parrot needs bignums too. Work on several fronts continued towards getting a bignum implementation for Parrot. Ideally, we'd like to just include GMP, but the license seems to require that the full source of GMP sip with any Parrot binary. Which isn't good when you're writing a VM that you'd like to be able to install on, say, a Palm.

Scott Bronson suggested getting in touch with the GMP developers to see if there was anything that could be done about it. He volunteered to be the liaison, assuming Dan was cool with it.

Meanwhile, Uri Guttman offered his 'stillborn' bignum implementation as a basis for our own bignum library. Leo commented that it might be a good idea in the medium to long term, but the Pie-thon is looming...

http://groups.google.com/groups?selm=a0611040bbcefb9fb77e6@[10.0.1.3]

http://groups.google.com/groups?selm=Pine.LNX.4.58.0406231105390.10311@sprite.sidhe.org

Ion's Strings

Remember Ion Alexandru Morega's string pmc that got Warnocked? Well, it got un-Warnocked this week. Dan liked it, so he checked it in. Congratulations Ion, don't forget to send in a patch to the CREDITS file.

http://groups.google.com/groups?selm=20040620075716.37958.qmail@onion.perl.org

Mmm... Pie-thon

Dan reminded everyone of the URL of the benchmark that's going to be run for the Pie-thon. If Parrot doesn't run it faster than the C implementation of Python, then Dan's going to get a pie in the face and he'll have to spring for a round of drinks for the Python Cabal (is there one? Could be a cheap round...). Of course, the plan is to have Parrot be faster than Python so Dan'll get to chuck a Pie at Guido. Either way, if you're at OSCON this year it should be fun.

Right now, Parrot is some way from, well, running it.

Leo announced that he will start a series of postings discussing the main features (and requirements) of the various benchmarks. He started with some notes on b6.py.

http://groups.google.com/groups?selm=Pine.LNX.4.58.0406210949440.10788@sprite.sidhe.org

http://groups.google.com/groups?selm=40D82C8F.1010903@toetsch.at -- Python needs exception objects

http://groups.google.com/groups?selm=40D84647.40407@toetsch.at -- Pie-thon and the recursion limit

Class name hashes

Leo wondered about the current implementation of class name hashes. We have two such hashes, one for PMC base classes and one for dynamic PMCs. Leo thought this was suboptimal (and liable to make one of the Pie-thon benchmarks problematic.) Dan agreed that unifying the hashes (and renaming PMC classes to have Parrot prefixes (ParrotFixedSizeArray anyone?)) was the way to go.

http://groups.google.com/groups?selm=40D85C12.8010502@toetsch.at

Confusing parameter order

Nicholas Clark was somewhat bemused by the choice of parameter order for the Parrot_PMC_set_intval_intkey function, which reverses the order of some parameters from the vtable version. He suggested unifying the two. Leo agreed that changing things round was the best bet and suggested that Nicholas go ahead and make the change to the extend.h interface. Dan didn't pipe up to say it was a bad idea. In fact, Dan didn't pipe up at all.

http://groups.google.com/groups?selm=20040624140935.GH81272@plum.flirble.org

Perl 6 regex parser

Jeff 'japhy' Pinyan announced that he was currently completing work on an extensible, regex-specific, parsing module. Once it's finished, he's going to start work on writing a subclass that parses Perl 6 rules. He thought it might come in handy for people implementing Perl 6.

Luke Palmer thought it would be too; the grammar of Perl 6 is being specified in Perl 6 patterns. Initially it was hoped that Damian's Perl6::Rules module would be what was needed to bootstrap us, but 'serious bugs in the Perl 5 regex engine having to do with executing code within regexes' meant that Perl6::Rules can't do the job for us.

So, good luck Japhy, we're depending on you.

But, you know, no pressure mate.

http://groups.google.com/groups?selm=Pine.LNX.4.44.0406262313030.28219-100000@perlmonk.org

Leo's tasks for the interested

Leo asked for volunteers to implement a class to handle complex numbers (needed for Python) and to implement sort method(s) for the various *Array PMCs.

http://groups.google.com/groups?selm=40DEF36A.3080101@toetsch.at

Meanwhile, in perl6-language

Command line like switches for functions and operators

Michele Dondi proposed syntax for adding 'command line' type switches to functions and methods which look almost, but not quite, entirely unlike Perl 6's named parameters. Brent Royal-Gordon pointed out that Michele's proposed

    rename -v => 1, $orig, $new;

could be written, rather more Perl6ishly as:

    rename $orig, $new :verbose;

Luke Palmer pointed out that wrappers could be used to add such extra named arguments to built ins, so the proposed behaviour for rename could be implemented as:

    &*rename.wrap -> $orig, $new, *%opt {
        say "Renaming '$orig' to '$new'" if %opt{'verbose' | 'v'};
        call;
    }

It turns out that neither of these were what Michele was driving at; it wasn't so much that new syntax was wanted, but that every built in have some kind of verbose version.

Meanwhile, as the discussion drifted in typical perl6-language style, Larry talked about 'allowing (at least) a single closure parameter to come at the end after the list parameter'. Simon Cozens doesn't seem to be maintaining his Ruby-o-meter any more, but if Larry were to make this change, the needle would definitely swing further in Ruby's direction.

http://groups.google.com/groups?selm=Pine.LNX.4.56L0.0406221118150.1885@worf.pcteor1.mi.infn.it

User defined operators

Michele Dondi wondered if Perl 6 would have a way of defining infix operators. Answer: Yes. The discussion pointed up some interesting scoping issues to do with macros too. But macros do that (as a look at any text on Common Lisp macros will show you).

http://groups.google.com/groups?selm=Pine.LNX.4.56L0.0406241206170.2865@q.pcteor1.mi.infn.it

Definitions of truth

Paul Hodges doesn't like the fact that a null character ("\0") is has a boolean 'true' value in Perl 5. He hoped that Perl 6 would change this. It appears his hopes are in vain. The consensus appeared to be that this kind of thinking is a (potentially) dangerous fossil from C type thinking. Jonadab the Unsightly One muttered something about 'comparing apples and orange leotards'.

The actually discussion was a little more discursive, of course.

http://groups.google.com/groups?selm=2820B295B95836498EED98D7A065C4F104DBD5D4@bremocmg-55

Slicing

Rod Adams wondered how hash slicing will work in 'the glorious age of Perl 6'.

So do we all Rod. We hope it'll work really well. Apocalypse N will have detailed answers.

http://groups.google.com/groups?selm=40DB63D1.4070804@rodadams.net

Postfix modifiers

The core Perl 6 language will allow one, and only one, postfix modifier per statement. Larry has said this several times. If you want more, modify the grammar yourself.

If that was the last time I find myself having to say this before we get a Perl 6 beta then I'll gladly eat a hat at the appropriate OSCON. You can hold me to this (but you'll have to pay my travelling expenses).

The .bytes/.codepoints/.graphemes methods

Argh! Unicode! But for once it's not moaning about Unicode operators, so that's a result.

Brent Royal-Gordon wondered about the behaviour of the .bytes, .codepoints, and .graphemes methods on the String object. He suggested that, in a list context, they return a list of the appropriate things in the string. Larry looked upon Brent's proposal and saw that it was good.

http://groups.google.com/groups?selm=40DDCE2A.1080806@brentdax.com

if, loop and lexical scope

Alexey Trofimenko wondered about the lexical scoping rules associated with Perl 6's various control structures.

If you're unsure about them yourself, Luke Palmer's reply is an excellent overview of how such scoping works and of some of the rationale behind it.

http://groups.google.com/groups?selm=opr99iqr0esspxk8@s

Announcements, Apologies, Acknowledgements

Hopefully this summary marks a return to the good old days of weekly summaries, posted to the mailing lists near the beginning of the week.

Well, we can always hope can't we?

If you find these summaries useful or enjoyable, please consider contributing to the Perl Foundation to help support the development of Perl. You might also like to send me feedback at mailto:pdcawley@bofh.org.uk

http://donate.perl-foundation.org/ -- The Perl Foundation

http://dev.perl.org/perl6/ -- Perl 6 Development site

Application Design with POE

Day in and day out, I write large applications in perl. I'm cursed I tell you. While large scale, long-running applications in pure perl may sound fairly easy to write, they are not. Perl, beyond a certain size and complexity, gets really difficult to manage if one is not extremely careful. The proper choice of an application framework helps to minimize this difficulty. For many applications, apache and mod_perl make a lot of sense. This is an excellent choice for user interface applications and data display systems. However, HTML and the WWW simply don't make sense for many forms of long-running applications, particularly network based servers. Apache certainly isn't the right choice for syslog monitoring or edge host traffic analysis.

My framework of choice is POE. POE is a single-threaded, event driven, cooperative multitasking environment for perl. Basically, POE is an application framework in which a single threaded perl process waits for events to occur so it can act accordingly. This event loop comprises the core of a POE process.

If all POE offered was an event loop, there would not be much to talk about though. Nor would POE be particularly special. Several event loop modules already exist on CPAN. Event, Coro, IO::Events, and IO::Poll all offer similar functionality. However, any worthwhile application demands more than a simple set of actions.

SESSIONS

POE programs begin with a 'session'. Each session represents a cooperatively multi-tasked state machine.

    POE::Session->create(
        inline_states => {
            _start => \&start,
            _stop => \&stop,

            do_something => \&do_something,
        },
        heap => {
            'some' => 'data',
        },
    );

Sessions are slightly analogous to threads in that they have a unique runtime context and a semi-private data store (called the "heap"). Each session operates independently from other sessions, receiving time-slices from the POE kernel. It is important to remember that, despite the similarity to threads, all POE sessions run in the same single-threaded process.

Sessions provide very simple, easy to understand building blocks on which to build more complex applications. POE provides a way to give sessions names, called aliases, which uniquely address the session from outside the session itself. $poe_kernel->alias_set($alias) sets an alias for the current session. Any POE session in the process can then send events to that session using the named identifier.

    if($door_bell) {
        $poe_kernel->post( $alias => 'pizza' );
    }

Remote addressing provides the ability to have a service-like model inside an application. Different sessions provide different services to the application. One session may provide DNS resolution while another provides data storage. Using commonly known names, perhaps stored in a config file, the central application becomes much smaller and easier to manage.

COMPONENTS

POE components provide an abstract api to service-like POE sessions. Rather than duplicating the session construction call and the accompanying subroutines every time you find a new use for your sessions, it is a better idea to roll all that code into a perl module.

    package POE::Component::MyService;

    sub create {
        POE::Session->create(
            # ...
        );
    }

    sub start {
        $poe_kernel->alias_set(__PACKAGE__);
    }
    
    
    ####
    
    
    #!/usr/bin/perl
    use POE;
    use POE::Component::MyService;

    POE::Component::MyService->create();
    POE::Kernel->run();

The POE community has created a standard namespace of POE::Component for these modules. Typically they have a constructor called create() or spawn() and provide a service to the POE application via a session. Apart from these few simple rules, components are free to do whatever is necessary to fulfill their purpose. POE::Component::Server::Syslog, for instance, spawns a UDP listener and provides syslog data via callbacks. POE::Component::RSS accepts RSS content via an alias and calls specially named events to deliver data. POE::Component::IRC follows a similar model.

WHEELS

For some tasks, a full session is unnecessary. Sometimes, it makes more sense to alter the abilities of an existing session to provide the desired functionality. POE has a special namespace called POE::Wheel for modules which mutate or alter the abilities of the current session to provide some new functionality.

    package POE::Wheel::MyFunction;

    sub new {
        # ...
    }

    ####

    #!/usr/bin/perl
    use POE;
    use POE::Wheel::MyFunction;

    POE::Session->create(
        #...
        foo => \&foo,
    );

    POE::Kernel->run();

    sub start {
        POE::Wheel::MyFunction->new(
            FooState => 'foo'
        );
    }

Where components often use subroutine callbacks in the same way as POE::Session, wheels use local event names to provide functionality. Internally, they create wrappers around calls to these events which build the context necessary for a POE event to occur.

Wheels are much more complex to create, for good reason. Wheels share their entire operating context with the user's session but share very little of the niceties. Wheels do not have their own heap and cannot create aliases for themselves. In many ways, they are like a parasite clinging to the side of the user's code. As long as they don't get in the way and they provide a useful function, they are allowed to exist.

The development overhead is made up for, however, by the loss of internal POE overhead. Sessions require a certain amount of maintenance to keep running. POE checks sessions to see if they still have work to do, if there are timers or alarms outstanding for them, if they should be garbage collected, etc etc. The more sessions that exist in a system, the more that overhead grows. This overhead is especially noticeable in time sensitive applications. Wheels have none of this overhead. They piggyback on top of the user's session so, apart from any events they may trigger as part of their normal operation, there is no inherent internal POE overhead in using a wheel.

FILTERS

Many wheels handle incoming and outgoing data. They exist to help the user get data from some strange source (say, HTTP) into a format the user can analyze or take apart in perlish ways. POE::Wheel::SocketFactory, for instance, handles all the scariness of nonblocking socket creation and maintenance. For most of us, however, SocketFactory doesn't go far enough. I don't want to have to worry about pack calls or http headers or whatever other nonsense is necessary to take a transaction off the wire and make it palatable. Special modules in the POE::Filter namespace handle this drudgery.

    package POE::Filter::MyData;

    sub new {
        # ...
    }
    
    sub put {
        # ...
    }

    sub get {
        # ...
    }

Filters are very simple data parsing modules. Most POE filters are limited enough to be used outside of a POE environment. They know nothing of POE or of the running POE environment. The standard interface requires three methods: new(), the constructor; get(), the input parser; and put() the output generator. get() takes a stream of data and returns parsed records, which may be hashes, arrays, objects, or anything else one might desire. put() takes user generated records and converts them to raw data.

Design

With these four simple building blocks, POE applications can grow to meet almost any need while still being maintainable. The key is to break the application up into small chunks. This is beneficial for two main reasons: 1) the individual chunks are more easily understood by a new staff member or someone else looking at the code six months from now. 2) Smaller blocks of code spend less time ... well, blocking.

As noted above, a POE application is a single-threaded process that pretends to perform asynchronous actions through a technique called cooperative multitasking. At any given time, only one subroutine inside a POE application is executing. If that one subroutine has a sleep 60; inside of it, the entire application will sleep for 60 seconds. No alarms will be triggered; no actions will occur. Smaller blocks of code mean that POE gets a chance to do other actions like cleaning up sessions that are shutting down or executing another event.

Even long-running for-loops can be broken down into small POE events.

    while(@data) {
        # ... process, process
    }

can become

    $poe_kernel->yield('process_data' => $_) for @data;

This gives POE time to read from sockets, do internal housekeeping, and so on,between each bit of processing time. If @data is large enough, however, this method can lead to resource depletion - spewing out 5000 events to process @data may get the job done and allow POE to do housekeeping, but it means that for the next 5000 event invocations, POE is doing nothing but processing that array.

POE's event queue is a FIFO (First In First Out). Events are processed in the order they are invoked. There are two major exceptions to this. Signals can trigger immediate event processing, and using call() instead of yield() or post() will cause immediate event processing. Beyond those two exceptions, every event happens in order, all of the time.

In the example above, we asked POE to push a large number of events on the queue. While POE can still read off whatever socket we're getting data from inbetween those yields, the events triggered by that socket read will not be invoked until after we're done processing our giant array. We can break that pattern out very easily.

If we don't need to process @data in any timely fashion, we can stagger the processing out further:

    $poe_kernel->delay_add('process_data' => $hence++ => $_) for @data;

This will process one chunk of @data every second. Not very efficient or timely but other events can take place between invocations. One second is by no means the smallest time value accepted by delay_add(). Use of Time::HiRes allows for microsecond delay values:

    use Time::HiRes;
    use POE;

The use of Time::HiRes before importing POE causes POE to use Time::HiRes' time() instead of perl's built-in time(). While Time::HiRes has much greater resolution on time values, it may or may not be the most accurate time keeper on your particular platform. Do your homework and make the choice that best suits your situation and needs.

Conclusion

POE is a flexible application framework appropriate for long-running large-scale perl applications. It provides standard interfaces for task abstraction and forces the coder to think about their software in smaller, more maintainable chunks.

POE is available on CPAN (http://search.cpan.org/dist/POE) and has a rich, community-maintained website (http://poe.perl.org).

Visit the home of the Perl programming language: Perl.org

Sponsored by

Monthly Archives

Powered by Movable Type 5.13-en