October 2000 Archives

Critique of the Perl 6 RFC Process

Table of Contents
Problems with Proposals of Major Changes

Problems with Proposals of Minor Changes

Miscellaneous Problems

Overall Problems

Bottom Line

A Very Brief Summary

Discussion of major changes is pointless in the absence of a familiarity with internals and implementations.

Discussion of minor changes leads to a whole bunch of minor changes which don't end up signifying a whole lot.

Discussion was frequently disregarded by proposal authors.

Problems with Proposals of Major Changes

Somewhere in the middle of the RFC process, I posted a satirical RFC proposing that cars should get 200 miles per gallon of gasoline. I earnestly enumerated all the reasons why this would be a good idea. Then I finished by saying

I confess that I'm not an expert in how cars work. Nevertheless, I'll go out on a limb and assert that this will be relatively easy to implement, with relatively few entangling side-issues.

This characterizes a common problem with many of the RFCs. Alan Perlis, a famous wizard, once said:

When someone says ``I want a programming language in which I need only say what I wish done,'' give him a lollipop.

Although it may sometimes seem that when we program a computer we are playing with the purest and most perfectly malleable idea-stuff, bending it into whatever shape we desire, it isn't really so. Code is a very forgiving medium, much easier to work than metal or stone. But it still has its limitations. When someone says that they want hashes to carry a sorting order, and to always produce items in sorted order, they have failed to understand those limitations. Yes, it would be fabulous to have a hash which preserves order and has fast lookup and fast insert and delete and which will also fry eggs for you, but we don't know how to do that. We have to settle for the things that we do know how to do. Many proposals were totally out of line with reality. It didn't matter how pretty such a proposal sounds, or even if Larry accepts it. If nobody knows how to do it, it is not going to go in.

Back in August I read the IMPLEMENTATION section of the 166s RFC that were then extant.

15/166 had no implementation section at all. 14/166 had an extensive implementation section that neglected to discuss the implementation at all, and instead discussed the programming language interface. 16/166 contained a very brief remark to the effect that the implementation would be simple, which might or might not have been true.

34 of 166 RFCs had a very brief section with no substantive discussion or a protestation of ignorance:

``Dammit, Jim, I'm a doctor, not an engineer!''

``I'll leave that to the internals guys. :-) ''

``I've no real concrete ideas on this, sorry.''

RFC 128 proposed a major extension of subroutine prototypes, and then, in the implementation section, said only ``Definitely S.E.P.''. (``Someone Else's Problem'')

I think this flippant attitude to implementation was a big problem in the RFC process for several reasons.

It leads to a who-will-bell-the-cat syndrome, in which people propose all sorts of impossible features and then have extensive discussions about the minutiae of these things that will never be implemented in any form. You can waste an awful lot of time discussing whether you want your skyhooks painted blue or red, and in the RFC mailing lists, people did exactly this.

It distracts attention from concrete implementation discussion about the real possible tradeoffs. In my opinion, it was not very smart to start a perl6-internals list so soon, because that suggested that the other lists were not for discussion of internals. As a result, a lot of the discussion that went on on the perl6-language-* lists bore no apparent relation to any known universe. One way to fix this might have been to require every language list to have at least one liaison with the internals list, someone who had actually done some implementation work and had at least a vague sense of what was possible.

Finally, on a personal note, I found this flippancy annoying. There are a lot of people around who do have some understanding of the Perl internals. An RFC author who knows that he does not understand the internals should not have a lot of trouble finding someone to consult with, to ask basic questions like ``Do you think this could be made to work?'' As regex group chair, I offered more than once to hook up RFC authors with experienced Perl developers. RFC authors did not bother to do this themselves, preferring to write ``S.E.P.'' and ``I have no idea how difficult this would be to implement'' and ``Dunno''. We could have done better here, but we were too lazy to bother.

Problems with Proposals of Minor Changes

Translation Issues Ignored or Disregarded

Translation issues were frequently ignored. Larry has promised that 80% of Perl 5 programs would be translatable to Perl 6 with 100% compatibility, and 95% with 95% compatibility. Several proposals were advanced which would have changed Perl so as to render many programs essentially untranslatable. If the authors considered such programs to be in the missing 5%, they never said so.

Even when the translation issues were not entirely ignored, they were almost invariably incomplete. For example, RFC 74 proposed a simple change: Rename import and unimport to be IMPORT and UNIMPORT, to make them consistent with the other all-capitals subroutine names that are reserved to Perl. It's not clear what the benefit of this is, since as far as I know nobody has ever reported that they tried to write an import subroutine and then were bitten by the special meaning that Perl attaches to this name, but let's ignore this and suppose that the change is actually useful.

The MIGRATION section of this RFC says, in full:

The Perl5 -> Perl6 translator should provide a import alias for the IMPORT routine to ease migration. Likewise for unimport.

It's not really clear what that means, unless you suppose that the author got it backwards. A Perl 5 module being translated already has an import routine, so it does not need an import alias. Instead, it needs an IMPORT alias that points at import, which it already has. Then when it is run under perl6, Perl will try to call the IMPORT routine, and, because of the alias, it will get the import routine that is actually there.

Now, what if this perl5 module already has an IMPORT subroutine also? Then you can't make IMPORT an alias for import because you must clobber the real IMPORT to do so. Maybe you can rename the original IMPORT and call it _perl5_IMPORT. Now you have broken the following code:

        $x = 'IMPORT';
        &$x(...);

because now the code calls import instead of the thing you now call _perl5_IMPORT. It's easy to construct many cases that are similarly untranslatable.

None of these cases are likely to be common. That is not the point. The point is that the author apparently didn't give any thought to whether they were common or not; it seems that he didn't get that far. Lest anyone think that I'm picking on this author in particular (and I'm really not, because the problem was nearly universal) I'll just point out that a well-known Perl expert had the same problem in RFC 271. As I said in email to this person, the rule of thumb for the MIGRATION ISSUES section is that 'None' is always the wrong answer.

An Anecdote About Translation Issues

Here's a related issue, which is somewhat involved, but which I think perfectly demonstrates the unrealistic attitudes and poor understanding of translation issues and Larry's compatibility promises.

Perl 5 has an eval function which takes a string argument, compiles the string as Perl code, and executes it. I pointed out that if you want a Perl 5 program to have the same behavior after it is translated, eval'ed code will have to be executed with Perl 5 semantics, not Perl 6 semantics. Presumably the Perl 6 eval will interpret its argument as a Perl 6 program, not a Perl 5 program.

For example, the Memoize module constructs an anonymous subroutine like this:

    my $wrapper = eval "sub $proto { unshift \@_, qq{$cref}; 
                                     goto &_memoizer; }";

Suppose hypothetically that the unshift function has been eliminated from Perl 6, and that Perl 5 programs that use unshift are translated so that

        unshift @array, LIST

becomes

        splice @array, 0, 0, LIST

instead. Suppose that Memoize.pm has been translated similarly, but the unshift in the statement above cannot be translated because it is part of a string. If Memoize.pm is going to continue working, the unshift in the string above will need to be interpreted as a Perl 5 unshift (which modifies an array) instead of as a Perl 6 unshift (which generates a compile-time error.)

The easy solution to this is that when the translator sees eval in a Perl 5 program, it should not translate it to eval, but instead to perl5_eval. perl5_eval would be a subroutine that would call the Perl5-to-Perl6 translator on the argument string, and then the built-in (Perl 6) eval function on the result.

A number of people objected to this, and see if you can guess why: Performance!

I found this incredible. Apparently these people all come from the planet where it is more important for the program to finish as quickly as possible than for it to do what it was intended to do.

Tunnel Vision

Probably the largest and most general problem with the proposals themselves was a lack of overall focus in the ideas put forward. Here is a summary of a typical RFC:

Feature XYZ of Perl has always bothered me. I do task xyzpdq all the time and XYZ is not quite right for it; I have to use two lines of code instead of only one. So I propose that XYZ change to XY'Z instead.

RFCs 148 and 272 are a really excellent example of this. They propose two versions of the same thing, each author having apparently solved his little piece of the problem without considering that the Right Thing is to aim for a little more generality. RFC 262 is also a good example, and there are many, many others.

Now, fixing minor problems with feature XYZ, whatever it is, is not necessarily a bad idea. The problem is that so many of the solutions for these problems were so out of proportion to the size of the problem that they were trying to solve. Usually the solution was abetted by some syntactic enormity.

The subsequent discussions would usually discover weird cases of tunnel vision. One might say to the author that the solution they proposed seemed too heavyweight to suit the problem, like squashing a mosquito with a sledgehammer. But often the proponent wouldn't be able to see that, because for them, this was an unusually annoying mosquito. People would point out that with a few changes, the proposal could also be extended to cover a slightly different task, xyz'pdq, also, and the proponent would sometimes reply that they doesn't consider that to be an important problem to solve.

It's all right to be so short-sighted when you're designing software for yourself, but when you design a language that will be used by thousands or millions of people, you have to have more economy. Every feature has a cost in implementation and maintenance and documentation and education, so the language designer has to make every feature count. If a feature isn't widely useful to many people for many different kinds of tasks, it has negative value. In the limit, to accomplish all the things that people want from a language, unless most of your features are powerful and flexible, you have to include so very many of them that the language becomes an impossible morass. (Of course, there is another theory which states that this has already happened.)

This came as no surprise to me. I maintain the Memoize module, which is fairly popular. People would frequently send me mail asking me to add a certain feature, such as timed expiration of cached data. I would reply that I didn't want to do that, because it would slow down the module for everyone, and it would not help the next time I got a similar but slightly different request, such as a request for data that expires when it has been used a fixed number of times. The response was invariably along the lines of ``But what would anyone want to do that for?'' And then the following week I would get mail from someone else asking for expiration of data after it had been used a fixed number of times, and I would say that I didn't want to put this in because it wouldn't help people with the problem of timed expiration and the response would be exactly the same. A module author must be good at foreseeing this sort of thing, and good at finding the best compromise solution for everyone's problems, not just the butt-pimple of the week. A language designer must be even better at doing this, because many, many people will be stuck with the language for years. Many of the people producing RFCs were really, really bad at it.

Miscellaneous Problems

Lack of Familiarity with Prior Art

Many of the people proposing features had apparently never worked in any language other than Perl. Many features were proposed that had been tried in other language and found deficient in one way or another. (For example, the Ruby language has a feature similar to that proposed in RFC 162.) Of course, not everyone knows a lot of other languages, and one has to expect that. It wouldn't have been so bad if the proponents had been more willing to revise their ideas in light of the evidence.

Worse, many of the people proposing new features appeared not to be familiar with Perl. RFC 105 proposed a change that had already been applied to Perl 5.6. RFC 158 proposed semantics for $& that had already been introduced in Perl 5.000.

Too Much Syntax

Too many of the proposals focused on trivial syntactic issues. This isn't to suggest that all the syntactic RFCs were trivial. I particularly appreciated RFC 9's heroic attempt to solve the reference syntax mess.

An outstanding example of this type of RFC: The author of RFC 274 apparently didn't like the /${foo}bar/ syntax for separating a variable interpolation from a literal string in a regex, because he proposed a new syntax, /$foo(?)bar/. Wonderful, because then when Perl 7 comes along we can have an RFC that complains that "${foo}bar" works in double-quoted strings but "$foo(?)bar" does not, points out that beginners are frequently confused by this exception, and proposes to fix it by making "(?)" special in double-quoted strings as well.

This also stands out as a particularly bad example of the problem of the previous section, in which the author is apparently unfamiliar with Perl. Why? Because the syntaxes /$foo(?:)bar/ and /$foo(?=)bar/ both work today and do what RFC 274 wanted to do, at the cost of one extra character. (This part of the proposal was later withdrawn.)

Working Group Chairs Useless

Maybe 'regex language working group chair' is a good thing to put on your résumé, but I don't think I'll be doing that soon, because when you put something like that on your résumé, you always run the risk that an interviewer will ask what it actually means, and if that happened to me I would have to say that I didn't know. I asked on the perl6-meta list what the working group chair's duties were, and it turned out that nobody else knew, either.

Working group chairs are an interesting idea. Some effort was made to chose experienced people to fill the positions. This effort was wasted because there was nothing for these people to do once they were appointed. They participated in the discussions, which was valuable, but calling them 'working group chairs' did not add anything.

Overall Problems

Discussion was of Unnecessarily Low Quality

The biggest problem with the discussion process was that it was essentially pointless, except perhaps insofar as it may have amused a lot of people for a few weeks. What I mean by 'pointless' is that I think the same outcome would have been achieved more quickly and less wastefully by having everyone mail their proposals directly to Larry.

Much of the discussion that I saw was of poor quality because of lack of familiarity with other languages, with Perl, with basic data structures, and so forth.

But I should not complain too much about this because many ill-informed people were still trying in good faith to have a reasonable discussion of the issues involved. That is all we can really ask for. Much worse offenses were committed regularly.

I got really tired of seeing people's suggestions answered with 'Blecch'. Even the silliest proposal does not deserve to be answered with 'Blecch'. No matter how persuasive or pleasing to the ear, it's hard to see 'Blecch' as anything except a crutch for someone who's too lazy to think of a serious technical criticism.

The RFC author's counterpart of this tactic was to describe their own proposal as 'more intuitive' and 'elegant' and everything else as 'counter-intuitive' and 'ugly'. 'Elegant' appears to be RFCese for 'I don't have any concrete reason for believing that this would be better, but I like it anyway.'

Several times I saw people respond to technical criticism of their proposals by saying something like ``It is just a proposal'' or ``It is only a request for comments''. Perhaps I am reading too much into it, but that sounds to me like an RFC author who is becoming defensive, and who is not going to listen to anyone else's advice.

One pleasant surprise is that the RFCs were mostly free of references to the 'beginners'; I only wish it had been as rare in the following discussion. One exasperated poster said:

``Beginners are confused by X'' is a decent bolstering argument as to why X should be changed, but it's a lousy primary argument.

A final bright point: I don't think Hitler was invoked at any point in the discussion.

Too Much Criticism and Discussion was Ignored by RFC Authors

Here's probably the most serious problem of the whole discussion process: Much of the criticism that wasn't low-quality was ignored anyway; it was not even incorporated into subsequent revisions of the RFCs. And why should it have been? No RFC proponent stood to derive any benefit from incorporating criticism, or even reading it.

Suppose you had a nifty idea for Perl 6, and you wrote an RFC. Then three people pointed out problems with your proposal. You might withdraw the RFC, or try to fix the problem, and a few people did actually do these things. But most people did not, or tried for a while and then stopped. Why bother? There was no point to withdrawing an RFC, because if you left it in, Larry might accept it anyway. Kill 'em all and Larry sort 'em out!

As a thought experiment, let's try to give the working group chairs some meaning by giving them the power to order the withdrawal of a proposal. Now the chair can tell a recalcitrant proposer that their precious RFC will be withdrawn if they don't update it, or if they don't answer the objections that were raised, or if they don't do some research into feasible implementations. Very good. The proposal is forcibly withdrawn? So what? It is still on the web site. Larry will probably look at it anyway, whether or not it is labeled 'withdrawn'.

So we ended up with 360 RFCs, some contradictory, some overlapping, just what you would expect to come out of a group of several hundred people who had their fingers stuck in their ears shouting LA LA LA I CAN'T HEAR YOU.

Bottom Line

I almost didn't write this article, for several reasons: I didn't have anything good to say about the process, and I didn't have much constructive advice to offer either. I was afraid that it wouldn't be useful without examples, but I didn't want to have to select examples because I didn't want people to feel like I was picking on them.

However, my discussions with other people who had been involved in the process revealed that many other people had been troubled by the same problems that I had. They seemed to harbor the same doubts that I did about whether anything useful was being accomplished, and sometimes asked me what I thought.

I always said (with considerable regret) that I did not think it was useful, but that Larry might yet prove me wrong and salvage something worthwhile from the whole mess. Larry's past track record at figuring out what Perl should be like has been fabulous, and I trust his judgment. If anyone is well-qualified to distill something of value from the 360 RFCs and ensuing discussion, it is Larry Wall.

That is the other reason to skip writing the article: My feelings about the usefulness of the process are ultimately unimportant. If Larry feels that the exercise was worthwhile and produced useful material for him to sort through, then it was a success, no matter how annoying it was to me or anyone else.

Nevertheless, we might one day do it all over again for Perl 7. I would like to think that if that day comes we would be able to serve Larry a little better than we did this time.

This Week on p5p 2000/10/30



Notes

You can subscribe to an email version of this summary by sending an empty message to p5p-digest-subscribe@plover.com.

Please send corrections and additions to simon@brecon.co.uk

This was a bit of a Unicode-heavy week; maybe it's because that's what I particularly notice, or maybe it's because it's the most broken bit.

sprintf Parameter Re-ordering

Jarkko got around to implementing a sprintf which lets you reorder the parameters, so that you can now say:

        printf "%2\$d %1\$d\n", 12, 34;           
        # will print "34 12\n"

There was some discussion as to whether that's the right way to do it, but that's the way libc seems to do it, so we should too. Read about it.

The Dangers (and bugs) of Unicode

Jarkko mentioned an article about Unicode security in Bruce Schneier's Counterpoint. It's a load of scaremongering about how Unicode can never be secure, apparently.

Now, I have to admit that I think this is bogus, but Jarkko also pointed out Markus Kuhn's Unicode information which is really worth reading. There was some discussion about exactly how bogus it was.

[ Dominus here: I didn't think Bruce was scaremongering. To understand Bruce's point, consider a CGI program written in Perl and running in taint mode. It gets a user input, which it plans to use as a filename, and it wants to untaint the input. The usual advice you get is to have a list of acceptable characters, say [0-9A-Za-z] and to reject the input if it contains some other sort of character.

[ Now, as I understand it, Bruce's point is that this strategy is going to be a lot more dangeous in the Unicode world, because there will be many occasions on which restricting inputs to be just [A-Za-z0-9] will be unacceptable. Restricting input to "just letters and digits" is much, much more complicated under Unicode, because there are thousands of different characters that could be letters or digits. Bruce also points out that although we have decades of experience in dealing with the (relatively few) oddities and escape codes that are found in ASCII, we have little experience with the much more complicated semnatics of Unicode, which includes issues like byte ordering and normalization. I now return you to Simon Cozens. ]

In other Unicode news, Jarkko also noted that \x{some number} should always produce a UTF8 string, no matter whether or not use utf8 is in effect. I had a horrible feeling of deja vu, and churned out a patch. There was some discussion from Jarkko and Andreas about the use of the use utf8 pragma; basically, it's supposed to become a no-op, so we shouldn't be adding any more functionality to it right now.

Self-Tying Is Broken

Steffen Beyer has noticed that removing the capacity to tie an object to itself breaks his Data::Locations module: he was using it to make filehandles which were also objects. (A really cool idea, but undocumented and unsupported.)

There followed a long and fairly acrimonious thread, but a sensible conclusion: Jarkko reallowed self-ties on scalars. Marc Lehmann tried to stir up trouble by asking what should be done about pseudohashes. Jarkko got it right:

Yes. They should die. How's that for a polemic statement? :-)

Unsurprisingly, nobody disagreed.

Configure Confused By Changing Architectures

Nicholas Clarke found that if you reuse your Policy.sh between updating your source tree, things break. Doctor, it hurts when I do that.

He also noted that if you change your architecture (for instance, from using threads to not using threads) then sometimes Configure doesn't pick up the change. Don't reuse your config.sh if you do this.

On a vaguely similar note, Merijn Brand found that doing a make okfile would cause Perl to be rebuilt; he and Nicholas Clark did some debugging, and Nicholas eventually found the problem and fixed it - a little problem with auto-generated files and dependencies.

Encode

Work on the Encode module to convert character sets continues, and it's really looking good now. (Everyone say "thank you" to Nick, who's also doing superb work on line disciplines!)

[ Dominus again: What Nick is doing is so interesting that I thought it deserved special mention. Nick wrote a replacement standard I/O package and embedded it into Perl. This continues Perl's trend towards providing its own functionality in areas traditionally covered by the C library, and removing dependence on the various broken libraries that are provided by vendors. This has happened already with sprintf and qsort. Last week's item about the limit of 256 open files under Solaris shows that even basic functions on major platforms can be impaired.

[ In any case, it became clear a while ago that to support Unicode properly, Perl was going to have to have a custom stdio package, and Nick's work is a big first step in that direction. Read more. -- D. ]

Peter Prymmer's did an excellent job and created an EBCDIC->Unicode mapping with it; Nick came up with a POD translation of the documentation on how Encode's mapping files work. (We stole them from Tcl! Ha!)

The thread wandered off into discussion of what the Unicode characters 0x0000 and 0xFFFF mean. Don't just guess, see the Unicode FAQ!

Switch

Jarkko considered adding Damian Conway's switch module into core; the module simulates the switch statement you'll see in many other languages. Damian's old version is available from CPAN, but he should be working on a new version in line with his Perl 6 RFC. Tim worried about Perl 5 appearing to bless a particular switch semanting before Larry had decided anything - Jarkko said that Damian would get it right anyway, and Andy pointed out that it would encourage people to play with it.

As he mentioned, there are three ways we could do it: use the Perl module and a source filter, convert the module to XS and use a source filter, or hack at the tokeniser and parser. Nobody wanted to do the latter option, since Hugo pointed out that it probably wouldn't be worth it due to the emergence of Perl 6. It might not happen, but if it does, it'll probably happen with an XS module.

Various

There was one flame this week. It was from me. Oops. Sorry, Steffen!

Oh, and I messed up last week - I said that sysopen used fopen, but as Mark-Jason Dominus explains:

Contrary to what you say, sysopen() doesn't call fopen(). It calls through Perl_do_open9, and the real open()ing occurs at doio.c:161 in the PerlLIO_open3 call, which is a macro that (on unix systems) invokes the true open(), not fopen(). This open() call succeeds, and returns a file descriptor. The problem behavior occurs later, at line 188, when Perl calls fdopen() to associate a standard I/O stream with the open descriptor. This is the call that fails.

Until next week I remain, your humble and obedient servant,


Simon Cozens

Last Chance to Support Damian Conway

$16,500 pledged, another $11,000 needed to liberate Damian Conway.

As reported earlier, the Yet Another Society (YAS) is putting together a grant to Monash University, Australia. The grant will fund Damian Conway's full-time work on Perl for a year. YAS founder Kevin Lenzo said, "The response from the Perl community has been just overwhelming."

Damian Conway, the author of many award-winning Perl Conference papers as well as the Quantum::Superpositions and Parse::RecDescent modules, is a professor in Monash's Computer Science department. His day job involves administrating a degree course, teaching undergraduates, and assisting postgraduate students with research. It doesn't include Perl. The Perl work he has done so far has been in his spare time, wedged around his >60-hour Monash weeks.

The wider Perl community has pledged over US$16,000 so far. That only US$11,000 to be raised before a matching pledge from BlackStar, a UK-based video-retail company, takes it up to the US$55,000 needed. Kevin Lenzo said "Blackstar stepped up quite early to help out. Their business benefits from Perl in innumerable ways, and they're really being good citizens by contibuting back this way, in a real and substantial way that will help everyone."

Unlike the previous kinds of corporate sponsorship of Perl developers, where O'Reilly (which runs this site) employs Larry Wall and ActiveState employed Gurusamy Sarathy (the last pumpking), Damian would merely be funded to work on Perl. Said Damian, "I'd be working on projects of my own devising: Perl modules, the development of perl6, and making presentations to any Perl Monger groups I pass on my travels, with no other demands on my time."

However, time is running out. Damian must let Monash know by the end of this week whether he'll be teaching next year. Says Kevin Lenzo, "This is a new kind of community patronage -- community-funded open source development. If this works well, we may have more ability to fund people for core work on Perl and other projects."

To pledge money towards the grant, go to registration.yapc.org. Any sized pledge is accepted. YAS has applied for non-profit status with the IRS, so contributions may be tax-deductible.

State of the Onion 2000

Hear Larry Wall's Keynote

(Because of the time length, we've broken up Larry Wall's keynote at O'Reilly's Open Source Roundtable into four seperate audio files.)

 Audio Part One
MP3     RealPlayer

 Audio Part Two
MP3     RealPlayer

 Audio Part Three
MP3     RealPlayer

 Audio Q & A Session
MP3     RealPlayer

Today, I'll need all of the support I can get. I'm running an experiment today in sleep deprivation. Actually, I've been running the experiment in sleep deprivation for the past six months or so. If what I say today comes out like puddle mush, well, you can just assume that I used up all my pitiful supply of good writing for that little book some of you have been carrying around.

I decided to talk about music this year because I talked about chemistry last year. The two are naturally associated in my brain for some reason. Of course, all music can be viewed as better living through chemistry - brain chemistry that is. Here, have some neurotransmitters. [Plays music] Feel happier? This talk will mostly be auditory, but for you people who are visually oriented I have a sop. I have some new tokens that we'll be adding to Perl. Actually, these are some of the Unicode characters that are in the process of being approved up in the surigate area, but, you know, UPSA can handle that. No, I can't wait until I can overload some of these as operators. I'm not entirely sure what "dollar A trill dollar B" would mean, but it will do something to your neurotransmitters.

But neurotransmitters aside, chemistry and music are also associated in my mind because, although I eventually graduated in computers and linguistics for my first two years in college, I was pursuing a double major in chemistry and music. Kind of the story of my life that I've always been interested in too many things. Jack of all trades and master of maybe one, I don't know. Anyway, I've uttered that phrase chemistry and music so often that it's almost a Pavlovian response. "Hey Larry, you talked about chemistry last year. What are you going to talk about this year?" [Sound of bell] Slobber, slobber, slobber. I'll talk about music.

It wasn't until three days ago, though, that I actually sat down and asked my left brain, "Why do you want to talk about music this year?"

"Dah, I don't know. Ask my right brain."

"I am your right brain, stupid."

I think part of the answer is that I worry about how Perl culture is going to grow up. If any of you have kids, you know that kids are complex and they get more complex as they get older. Fortunately, there's only so much complexity that any one kid can hold and either they eventually go insane or, if you're really lucky, they go sane. Even so, understanding any individual completely is impossible. The Perl culture is composed of many people and so the complexity of Perl culture can grow without bounds. No one person can understand Perl culture completely. If the complexity of Perl culture is going to continue to grow beyond the ability of any one person to fathom, even me, how can we continue to think constructively about it? I believe that we must learn to apply constructive analogies from other systems that are too complicated to understand completely. On the one hand, we can pull ideas from those scientists that have had to deal with overwhelming complexity, such as chemistry, biology, neurophysiology and so on. On the other hand, the more humanistic pursuits have always been overwhelmed by complexity, whether you're talking about sociology or political science. You can study literature all your life and not even read everything you're supposed to read, let alone what you want to read.

Music is the same way. Many of us know thousands of songs or musical pieces, but nobody can know them all. Even if you could somehow combine all the songs that are currently playing somewhere in the world, it would just come out as pink noise. Terribly musical. And that's exactly how your brain feels if you've read too many Perl mailing lists all at once. Only God can hear all the songs that are being played simultaneously and only God can read all the simultaneous mailing lists, newsgroups and Web sites of the world. Pretty soon, only God will be able to understand the CPAN. As humans, we have to simplify. In fact, we must oversimplify. We all specialize. We can focus in one style of music or on a particular piece of music or on a particular instrument. We can focus on rhythm or harmony or melody. We can focus on any of the ways that music affects our moods, whether tied directly to words or subtly as background music in a movie or background music in an elevator. We can focus on any of who, what, where, why, when or how. On a good day, we can focus on several things. But, we can never focus on all of them. So for the rest of this talk, I'd like to oversimplify Perl culture by looking at it through the lens of music culture.

Now the problem with music culture is, of course, that although it makes a nice analogy, it's also too complicated to talk about or even think about. So I brought along a few props. You can think of them as extension modules. Extension modules come in all shapes and sizes. Of course, you can program without extension modules. Those of you who have evolved far enough to have opposable thumbs have a built-in percussion instrument. [Snapping fingers] OK, that's a snap, of course. With a little work, you can develop a crackle and a pop as well. To do the crackle, you first learn to snap twice with each hand and then you combine them in sequence. OK, that's your crackle. To do a pop, you just make a little resonant cavity with your hand and you go [popping sounds], something like that. And then you get, you know, [sounds of snapping and popping combined], things like that.

Long, long ago at a campfire far away, somebody discovered that pigs have spare ribs. At least they're spare after all the meat is gone. Well, at least the pig doesn't need them anymore. Now, if you work these just right, you can get a kind of a triple rhythm. You know, they say it's all in the wrist. [Sounds of triple rhythm] Well, I'm going to have to bone up on that. [Drum & cymbal] Here's a more intuitive interface. These are called claves. At least these would be called claves if they were from Spain, but they're not. They're from Papua-New Guinea, so I have no idea what they're called there. I've asked various people from Papua-New Guinea what they're called and they have no idea either. They're probably called about 750 different things, since that's about how many languages there are in New Guinea. Every time you go over a hill the next tribe speaks an entirely different language. It's like being in your typical computer science department where every professor wants you to learn their favorite computer language which is different from the other 750 favorite computer languages of all the other professors. I suppose you could think of these as opposable sticks, though, in the history of our species, sticks have usually been opposed to crania.

Eventually our ancestors got tired of grilled spare ribs so they figured out how to boil pigs. Not long after that, they discovered soup. And not long after that, they invented the spoon. Technology was developing really fast back then. Anyway, shortly after they invented the spoon, they invented two spoons because that way they didn't have to share. [Sounds of two spoons] Now, this module is actually rather awkward to use, kind of like a Perl 4 module. That's the same sort of, well, you know, I've got a better thing. Here's the Perl 5 version of the same module. [Sounds] OK, I am going to move this over. We have the technology. [Laughter] We don't need the technology. [Laughter] OK. Now we're missing the technology. OK.

OK, back to our regularly scheduled program. This is the same sort of noise you get when two people butt heads against each other in Perl 5 quarters, you know, bonk, bonk. That's what oppositional behavior sounds like. The entire field of percussion is based on oppositional behavior - two objects trying to occupy the same space at the same time. So this is definitely an object-oriented module. You notice the relationship between the two spoons has been encapsulated so that the user no longer has to specify it explicitly. Actually, this is the second version of this module. Unfortunately, one of my kids broke the encapsulation on the first version. But, in fact, all percussion instruments are object oriented. After all, they indicate the rhythm, right? And rhythm is object oriented. You look like you don't believe me. Rhythm really is object oriented. Surely you've all heard of the rhythm method. [Laughter. Drums and cymbal.] Furthermore, you'll note that people who use the rhythm method are frequently members of the Lamaze class. [Laughter. Drums and cymbal.] Personally, I've been through three Lamaze classes, so has my wife. Now, many of you know that we have four kids. For our fourth kid they told us not to come back, since it was obvious we already knew all there was to know about heavy breathing. Actually, the Lamaze techniques are an interesting application of rhythm for the purpose of distracting the participants with pantings of various sorts. "OK, honey, now do sixes, twos, threes, sevens, and fives, not necessarily in that order." "Nurse, take my husband, please."

In the abstract, rhythm is about even programming, kind of like a Geiger counter. Here's our current background radiation. [Sounds.] Here's if you sit too close to the TV in your hotel room. [Sounds.] Those of you who know Morse code probably know what that said. I don't. If you don't care to sit too close to your TV, just move to Boulder or stay here and wait for the Diablo Canyon reactor to melt down. Random events aren't really all that interesting, however. It's get more interesting when you program which events happen when. [Sounds] You can do all sorts of, don't do that - [Sounds.] Enough of that. Other than little games like that, there's really only one sound that spoons can make. The conga drums, by contract, provide a richer interface. You can think of the interface as parameterized in several different dimensions, most of which are infinitely variable, but for all its richness, it's still an event-driven module. Perl also has an event module and it's maybe a little more portable than these, but it could use a little more work on that. But you can do some various interesting event loops on these things. [Conga drums] Now as a linguist trained in Tagnue, I tend to think of these things in terms of fields, waves and particles. Harmony is a field while melody is a wave. I think of rhythm as particles where the events are the points at which things happen. Actually, it's the events themselves that are particles, but rhythm is a higher level of abstraction, sort of connecting the dots in our head. So event questions start with "when?" while rhythm questions tend to start with "how often?" like, "How often should we have a Perl conference?" [Rhythm] "How often should we have a Perl Whirl cruise?" [Faster Rhythm - Laughter] Actually, it's like this. [Rhythm] It's a three against two rhythm, a sort of hemiola. You know, the Perl Conference is once a year, while the Perl cruise is scheduled to be once every 18 months, which gives you three beats in one hand to two beats in the other.

Here's three beats against four. [Rhythm] Here's three against five. [Rhythm.] If you want to figure these out for yourself, it's really just a bit of math. Multiply the two numbers to find out how many subdivisions you need. With three against four, you need 12 subdivisions and with three against five, 15, and you just chart it out.

Now, in reality the rhythm of Perl culture is more of a fractile with many smaller interactions for all of the larger ones. You know, here's the Perl conference. [Rhythm] All sorts of stuff going on in the interest of these. But, speaking of fractiles, I noticed an interesting thing about the rhythm of releases. As the size of the Perl core and libraries gets bigger, it takes longer to rev a major release, so it naturally gets slower. Perl 1, Perl 2, Perl 3, Perl 4, Perl 5. [Sound of beat on drum gets slower] Then we start getting subreleases and then they get slower. Then we get sub-sub versions, and they get slower and slower. Eventually the subversions are taking as long as the original Perl 1 - Perl 2 thing, but that's because they are actually accomplishing just as much. Anyway, I just thought that was interesting. In compensation - up until now, we've seen instruments that specialize in rhythm, which is a particle effect. In contrast, there are other instruments that specialize in harmony and melody. Here is an auto harp. You know, it's always on the wrong side. [Harp sounds] Well, you push buttons and you get harmony. It's almost a pure harmony instrument with a little bit of rhythm on the strum a little.

But harmony is an abstraction, a construct we manufacture in our own minds. Unlike rhythm, harmony is spread out in the pitch dimension and behaves as a field. By that, I mean it seems to fill space in a way that neither rhythm nor melody does. You can play rhythms and melodies simultaneously and they tend to keep their individual identities. But if you tried playing two chords simultaneously, you either get a different chord from either of them or you just get mush. Interestingly, you can't actually add two chords together on an auto harp because an auto harp builds harmonies by subtraction not by addition. You know, it's like trying to mix paints when you ought to be mixing light, and the more you try to mix the less you end up with. So we [Harp sound] subtract the most to get what we want and we subtract - we get fewer and fewer notes. Eventually we get no notes and that's not the way to make harmony. To actually mix harmony you need an additive device like a keyboard. [Sound of electronic keyboard] Something else again entirely.

We talk a lot about harmony in Perl culture and actually we yell at each other a lot about harmony and Perl culture, but it's very harmonious yelling. Remember that harmony tends to monopolize your mental space, but that's kind of an illusion. It's easy when you hear two people arguing in a public forum to think that the entire whole forum is bogus, but if you look carefully there's usually still a background of nonfighting going on as well. Normally people fight all the time. It just seems that way when we try to fit too many notes into the same mental space. You don't actually have to harmonize every note everywhere all at once. We have different locations. Different chords can happen in different places. Different pieces have different standards for dissonance and that's fine. Maybe a Perl friend's mailing list would be like Mozart and comp.lang.perl.misc is like Schoenburg mixed with John Cage with Metallica thrown in for good measure. Well, five quarters is slightly more civilized. It's a bit like late Mahler where part of the time the music is atonal and tortured and the rest of the time the music is tonal but still tortured. Actually, I like Mahler an awful lot. He's my favorite composer and this is no coincidence. Mahler once said he always tried to put the whole world into each of his symphonies. Know also that my favorite author is Tolkin, who also put an entire world into his work, so perhaps this is kind of natural that I try to hook up the entire world to Perl one way or another. Of course, you can't have the whole world in one spot without accepting a certain amount of dissonance.

But there's another form of abstraction which we call melody. In some ways, it's the most mysterious because, in fact, it really is object oriented and in a deep way. A melody is a sequence of notes that we perceive to have been played or performed by a single object, which we often call a voice even if it isn't one. Object permanence is something we learn at a young age. That's why we play peek-a-boo: to figure out that mommy didn't actually disappear when she went behind the towel. Similarly, you can take temporally separate notes and creatively imagine that they came from the same instrument. How many of you have ever played the computer game The Seventh Guest? You may recognize this melody, which is permanently burned into my personal E-prom. And then you solve that particular puzzle. I hope I didn't give anything away. That game intentionally makes it really hard to follow the melody since it treats the notes as discreet, but some instruments make it really easy to follow a melody by making the transitions continuous.

Life is always interesting, isn't it?

OK. Wonder how you play one of these things. Actually, you need a double inflection point, a double inflection here, or a double curve and you find the inflection point in the middle, maybe. We'll try for "Mary Had a Little Lamb." [Plays something - laughter] Wow, learn something new everyday. Now, I don't know who first came up with the idea of playing a saw. It's not what you would call obvious. On the other hand, some things are so obvious that if one person didn't invent it, the next person would. For example, whistling. [Whistles "We Wish You a Merry Christmas"] Another rather obvious invention I think is the bottle whistle. [Whistles in bottle] Perrier works really great for this. As W. C. Fields once said, "I'd rather have a bottle in front of me than a frontal lobotomy." Actually, a bottle in front of you doesn't have a lot to recommend it either. In particular, the melodies you can play with this module are rather monotonic, not to mention monotonous. Here's "The William Tell Overture" on the bottle. [Plays bottle] Other wind instruments can at least vary the pitch parameter, but most wind instruments are by nature melodic in that they can only produce one single note at a time. There are exceptions, of course. Take the bagpipes, please. I don't think the bagpipes are an obvious interface either. I don't know whether playing your hands would be considered obvious or not. I don't know why but you have to wet your whistle to do this. It doesn't make sense to me. [Whistles hands] I discovered that one by accident myself. One day I was leaning on a table with my hands folded and I just happened to blow in my hands suddenly to warm or something. [Tries blowing] Can't do it anymore, can I? [Tries blowing] Well, this is actually the hard way - with your fingers interlaced.

Now, but this actually illustrates a very important musical technique. One that many of us have had to learn repeatedly. Oddly, this technique is called unlearning. It's like back-tracking in real life. Sometimes you have make negative progress in order to go forward in the long run. For instance, when I started taking private lessons on the violin, I had to unlearn a year's worth of bad habits I'd picked up in school. Now in the case of playing of my hands, I had to learn to hold my hands a different way if I wanted to have greater pitch range. So once I'd relearned how to play my hands, I could get almost an octave. That leaves out "The Star Spangled Banner," but there's lots of melodies that will fit into an octave. [Plays hands] I almost have that octave. Let me tune it up here a little bit. [Plays hands] Which is, of course, "How Much Is That Camel in the Window?" Of course, you can do much the same thing with an extension mechanism that's official. Anybody want to play the box? [Plays something] Easy does it, Larry. [Plays something] One and three. Thank you. This is my wife's. [Plays.] OK. Obviously, that's my main instrument. And certainly one of the most obvious wind instruments is your voice, at least we certainly produce a lot of wind with our voice, but - no, I'm not going to sing anything operatic for you and, although I'm preaching to the choir, I'm not going to make you sing either. That should play something on my computer. We have ways to make you talk. [Computer sings "Allelulia"] Hey, a percussion instrument. Yeah, I know. I'm a sound engineer at my church, I'm allowed to do that.

Anyway, but you'll recall that melody is an abstraction of object permanence. As I mentioned earlier, these instruments call such an object a voice whether it's a really a human voice or any of the other instruments, but the really interesting thing to me is the relationship of melody to harmony and it has to do with what we call voice leading. Here's a guitar. You can play the guitar in many styles, but the most notable feature of the guitar is how much it's used for harmony rather than melody regardless of the style. I don't want the pick yet. [Plays guitar] You know, that's a little more Spanish. Here's something else. [Plays guitar] Something like that. [Applause] Of course, if you're a classical guitarist, you'd do a lot of melody, too, but the very basis of harmony is all the little melodies going on in the middle of the chords and you can actually hear the little waves in there if you listen. You can hear a chord change. [Plays] But you hear things like this happening in the middle and things like that - little voices - and so the harmony is actually, in a reductionist fashion, it's just a bunch of little harmonies, but holistically, melodies. But, holistically, you don't perceive it that way. It's perceived as a field, you know, so when you hear chords, you don't actually hear the individual notes here, you somehow intuit the whole - whatever it is.

You ever notice that music is sometimes hard to talk about? I never have any trouble talking about Perl. Anyway, the bottom note of a chord is kind of in a privileged position. It behaves more like a melody of its own, but it's a funny kind of melody in that it's perceived to drive the rhythm and the harmony. If you take the bottom four strings of a guitar and drop them an octave, you get one of these. [Shows something] Yeah. Got to get the slouch right here. Ready? [Plays something] Now, some of you will think that that's "Mission Impossible," but that's actually "Man from Uncle," which was my favorite show when I was young. Here's "Mission Impossible." [Plays] Recognize that? Bass guitar is really fun even if you're not good at it. Sometimes I think I'm the bass guitarist of Perl culture. I play the strange melody and then a whole bunch of other people start playing these strange rhythms and harmonies around me, but now I'm going to go back to hitting things again. This is what is known as a hammer dulcimer. It actually belongs to my wife, whom I like because she lets me borrow her things. The one thing about this that is harder on other instruments, especially wind instruments, is it's actually multithreaded. It's the beginnings of multithreading. It has two separate threads of control, so - [Plays] What shall I play? OK. [Plays] Nope. [Plays "Oh Suzanna"] OK. [Applause] "It rained all night the day I left, the weather was dry. Got so hot I froze to death, Suzanna, don't you cry."

Musicians delight in contradictions, but so do language designers. You have to be able to see both sides of every question when you design a language. It helps to have multiple-personality disorder. It also helps to have a multithreaded interface like the piano. [Plays "Oh Suzanna" on piano] You know, a piano is just a fancier interface for a dulcimer in a way. It still has hammers - a real piano does - that go out and strike the strings in another form of oppositional behavior, but it's a percussion instrument, in other words, but it happens to have a pitch parameter as well. It also has a more developed multithreading model. It supports up to 10 threads in two groups of five. Advanced implementations can handle 20 threads, but that requires two CPUs. You know, I'm afraid the piano module must still be considered experimental.

Still, the interface is somewhat user friendly. [Plays "Chopsticks" on piano] Etc. I don't actually have time to talk about counterpoint, except to say that this simple tune illustrates two contrapuntle principles we see in the Perl community: contrary motion and parallel motion. Contrary motion is when you have two programmers, or that is melodies, going in opposite directions. [Plays] Parallel motion is when the programmers agree on how to get where they're going. [Plays "Chopsticks"] Actually, I joked about it being experimental, but the piano interface is actually one of the most standard interfaces we have. Unfortunately, organs are not so standard. Once you get away from the keyboard itself, how you set one particular stop really depends on the kind of organ you have. In pipe organs, you might pull out one of the stops puller-outerers. On a Hammond organ, you might just adjust the draw bar. On this organ, I push D-35. [Plays organ] You got to love Bach. The next time someone says Perl is baroque, thank them for the compliment. [Plays organ]

They say it's easy to get a composer out of bed in the morning. All you have to do is go over to the piano - or the organ in this case - and play an unresolved chord and then they have to get out of bed and resolve the chord. [Laughter] It's hard. I can't stand it. [Plays] Now, I promised I'd bring my violin and, as you can see, I didn't break my arm in Aikido, so I guess I'll have to play it some. The violin is one of those traditional standard instruments. As I say, nobody ever got fired for buying a symphony orchestra. But, if you did get fired, your orchestra can play for you. [Plays violin] That's very sad. There's lots of happy music, too.

But, a violin is actually two different instruments and I actually brought it here to illustrate polymorphism. People ask what's the difference between a violin and a fiddle. There's no difference really, it only depends on which class you call the method from. Here's the fiddle interface. You just choke up on the bow a little bit here and a - [Plays fiddle] [Applause] Funny how people who claim to be tone deaf can nonetheless recognize various styles of music when they hear them. Our pattern-matching capabilities are usually much better than we admit. In fact, Perl's design banks on that. That last little bit of music actually can stand on its own. [Plays] That's usually known as "Shave and a Haircut, Two Bits," so music also has its one-liners.

Here's another piece known as "The Mouse Trap Concerto." [Plays one note] [Laughter and applause] Actually, I've played a lot of serious music on my violin. It was my privilege to spend six years in the Seattle Youth Symphony under the direction of Wilhelm Sokle. There wasn't anything that we couldn't play, but we just had to work at it a little longer than a professional orchestra would, which reminds me of Perl development sometimes. One thing I learned while playing in various orchestras is the importance of faking it. You have to be able to fake playing an instrument before you can really play it and I'm faking most of these instruments. My whole first year in the youth symphony I was petrified that I might get called upon during rehearsal to play a part that I wasn't ready to play. Fortunately, I was never called upon. My second year I made a startling discovery. I just learned the music thoroughly and then I didn't have to worry about whether anybody called on me to play it.

How does this play out in Perl culture? Well, we have to be willing to let people fake it for a while. If Perl is getting their job done, then that's fine, but we also have to find ways of encouraging people to upgrade their abilities when they're ready for that step, and we don't do that by beating them over the head. We do it by showing the positive benefits of learning Perl for real. You know, I probably could have actually been a professional violinist. Had I been only interested in music, I might have been, but then I wouldn't be up here waving around a violin at you. But I'd also like to use this violin to illustrate reusability. [Plays portion of "William Tell Overture" on violin] Or something like that. Well, I don't think William Tell would have minded the Lone Ranger using his music, but when I was growing up there were still cigarette commercials on TV and sometimes you heard, "Have a Lark, have a Lark, have a Lark, today." Later on it was "Have a pizza, have a pizza, have a pizza roll." I'm sure William Tell would just love to shoot a pizza roll off of someone's head.

Finally, I'd like to finally introduce officially my synthesizer here, Korg 5S, meet Perl hackers. Perl hackers meet Korg 5S. I've been pretending it's a piano and an organ and any number of other things, but it's really just a bunch of switches and oscillators and such. Like Perl, it can be viewed as a tool that got out of hand. Like Perl, it can be viewed as just another tool in the toolbox as well. I joke that the piano interface was experimental, but this interface really is experimental. The keyboard interface is pretty standard, it's an awful lot of fun to use this thing in its current state, kind of like Perl. It does a pretty good job of emulating some other tools in the toolbox. For instance - [Plays on keyboard] - easier here than there, unless you're my wife. She does this easier here too. Now I'm going to get in trouble. The thing that is really cool about this keyboard is that it can play various different styles of music, Tim Tody and all that. My favorite button is the one that locks in the different styles into the same tempo so that you can go from one style to another and see how the same tune sounds in different cultural contexts. For example, how would "Pachabel's Canon" come out if it were played by Mick and Keith or by John, Paul, George, and Ringo, or by Elvis? Well, we can find out. [Plays] [Applause] Well, one could go on all day with that.

One of the things we love about Perl is that it supports many different styles of programming. That's something we never want to lose with Perl. There's also just the intrinsic joy in making music that has nothing to do with whether we're using the music for some other purpose. Likewise, there's an intrinsic joy in programming in Perl that has nothing to do with the purpose we're putting it to. That is also something we never want to lose. In fact, there are many features we want to conserve in Perl, but music is continually re-inventing itself and so is Perl culture. Most music is evolutionary, not revolutionary. People don't usually riot over new music - Stravinsky's "Rite of Spring" being the exception that proves the rule, but people don't usually riot over new Perl modules either. But occasionally there does come a time when we have to think like revolutionaries. Someone has to throw the tea into the Boston Harbor. Someone has to decide that it was time to write the document starting out, "When in the course of human events it becomes necessary" etc., etc., but before that someone had to decide to alter the course of human events.

Yesterday, a bunch of us radicals decided that it was time to alter the course of human events. Some of you may have heard rumors of this. So, today, I'd like to announce to the world that the effort to write Perl 6 has begun in earnest. [Applause] And I'd like to use the synthesizer to make an important point. If you manufacture something like this, you eventually come to a point where you say, "This is a really neat gismo, but we can do something better. Do we continue to make small improvements in the current design or do we redesign the interface to let us do what we would really do down the road? And, if we do a redesign, can we keep everything people like about the old design while getting rid of all the things people don't like about the thing they have right now?" Well, that's kind of the state Perl is in right now. We really, really like what we have. We like it a lot, but we can think of lots of ways we can do it better and the things we'd like to do better come in several categories.

First, the language itself could use some revision. I'm allowed to admit that. There are many historical warts on Perl that wouldn't have been there if I'd known what I was doing, but, hey, I was faking it back then. You didn't know that, did you? I'm more of a competent language designer than I was 13 years ago and I have a lot more help these days, plus it's time to steal all the good ideas we can from those other languages that developed in the last decade. One of the things I realized yesterday was that we're actually in a much better position than when I designed Perl 5. Nowadays, we have code back-ins, such as B::B Parse, that can spit out the Perl code corresponding to the compiled syntax tree. If you think about that, it means that it would be relatively easy to make it spit out a closely related language, such as Perl 6.

Perl has always been designed to evolve but now we actually have the capability to be evolving a little faster. This means that for the first time in history we have the opportunity to make some incompatible fixes to Perl while preserving a migration path for the current code. I really couldn't do that when I designed Perl 5. We had to make almost everything upward compatible, or backward compatible, whichever one it is. But now it's the first chance to make that sort of changes and, since it is the first chance, it probably is also the last chance, so I think we should. Of course, we are not interested in breaking things just to break things, but I'm sure you can think of things you might have done differently. Myself, I really wish I'd made the system call return "true" on success rather than "false." I wish I'd made local time return the actual year and not the year minus 1900. I'd really love to throw out select file handle and there's general consensus that type gloves may have outlived their usefulness, and a number of simple but potentially powerful features have already been put on the table for consideration. That's not to say we're going to do all of them. My overriding goal for the redesign of Perl's language is that easy things should stay easy, hard things should get easier, and impossible things should get hard, as it were.

Another place we'd like to do better is in the implementation of the language as opposed to the language itself. I think I did a pretty good job with the design of Perl 5 and making it extensible at the language level, but the internal APIs necessary to write extension modules could really use to be cleaned up. Some of you may have noticed that. We could scrap Excess for something better, and, of course, we want the chord to be smaller and faster, always. I'd like to run Perl on my Palm, but perhaps more importantly we could design the extension system so that installing a new version of Perl doesn't break all your existing extension modules. We have many other ideas for improving the implementation as well and these will come filtering out, but neither language changes nor implementation changes will happen unless we also reinvent how we do things.

So we've already started a redesign of Perl culture, trying to keep the good aspects and leaving behind the nonproductive aspects. We intend to abandon the Perl 5 porter's model of development, which demonstrably leads to a lot of talk but little action. Instead we'll break down the design of Perl 6 and the maintenance of Perl 5 into manageable tasks given to meaningful working groups with meaningful charters and meaningful goals. We have collectively resolved to make these working groups work, and where they do not work to work at making them work until they do work. We will continue to refine all aspects of our development model until every itch is scratched as efficiently as possible.

We are really jazzed about this. It is our belief that if Perl culture is designed right, Perl will be able to evolve into the language we need 20 years from now. It's also our belief that only a radical rethinking of both the Perl language and its implementation can energize the community in the long run. In the long run means 10 and 20 years down the road. Finally, it is our belief that Perl 5 will be better supported than it would be if we merely tried to guard what we already have. The best defense is a good offense. Now, this is not going to happen quickly. We expect to have alpha code a year from now, or some definition of alpha. We might even ship it, but we expect it to be well-designed alpha code.

In the meantime, we are not abandoning Perl 5 anytime soon. We all like Perl 5 a lot. We all use it a lot. Many commercial interests will guarantee that Perl 5 continues to be well-maintained and stabilized for quite a few years to come, and we fully expect, given the history of Perl 4, that five years from now a lot of people will still be using Perl 5. We do expect the rate of new development in Perl 5 to taper off, of course, and that can be viewed as a feature, but no, open-source software specifically rejects the get-big-quick philosophy of the typical Web startup. Such rapid growth tends to fragment the culture and, in the long run, leads to ruin. Instead, we intend to proceed at the fastest speed at which we can efficiently propagate our cultural values to newcomers in our culture, but no faster. This is the healthy way forward and the only way to compete in a competitive space.

We have to be better, not just get there faster. Part of being better is making sure the stragglers don't get left behind. We are determined to do the right thing by everyone. To this we pledge our lives, our fortunes, and our sacred honor, as it were, what there is of it. Many more details of our plans will be coming out in the next few days and weeks, and we'll tell you who has taken responsibility for what. We'll set out a road map or chart if you're into music of where we'd like to be when. Look at www.perl.org for more as time goes on. Things should be showing up there today even. But right now I would like to call each of you to play your part, whatever that part is. You, yourselves, are individual melodies. Your being here today may be an event that changes the course of your tune. Certainly your being here with everyone else who is here today makes a kind of harmony, a lost chord that will never be played exactly the same again. Together we perform a contrapuntal jazz improvisation that can only be recorded imperfectly. Music has always been an ephemeral art, and even with CDs and DVDs people still go to live concerts. So remember, you were here when a new thing was born.

Every Perl conference is a cool event because Perl people are the best people in the world. In this age of mailing lists and Web pages, it's really nice to get personally acquainted with all the folks that you've met on the Net. But this conference is not just about getting together with your buds. It's also about finding new friends, forming new bands, creating cool new sounds, maybe landing a recording contract on the CPAN. Sometimes it's about more than that. Today it's about more than than. We're really serious about reinventing everything that needs reinventing. The way I look at it, Perl 5 was a composition largely by a single composer - me. It's a fine classical composition, but in essence it's one person's view of how to make music. If you work with Perl 5 you have to follow the score pretty closely. Perl 6 is going to be designed by the community. We're going to be doing some jamming. I'll still be exercising some artistic control over the language itself, but instead of playing off a score, I'm going to be playing off charts now and you're going to be seeing a lot of people improvising melodies of their own and interweaving them creatively in ways that will make Perl 6 much better than Perl 5, just as Perl 5 was much better than Perl 4, and if you know anything about me, you know I take the promised land quite seriously. We're all going to march in there someday. [Plays music.] I'm jazzed. [Applause.]

Narrator: John has very kindly offered to let Larry do a Q&A for the next 15 minutes on the grounds that there is not enough time to cover the full details of how to rebuild civilization after the Apocalypse. Larry would you be willing to do a Q&A?

Larry: If you'll help, I'd like to introduce our interim program manager for Perl 6. [Applause.] Now that we've got that out of the way.

Larry: Is Perl 6 going to be in C++? Maybe. Chip has a lot of experience with thinking about Perl and C++ and we intend to use the lessons he's learned one way or another.

Question: When do you expect the Perl 6 plan to appear and when can we start work on it?

Larry: Well, it will come out in stages. We are, in cultural terms, we are starting working on it already, and over the next month or so you are going to see the chart, the road map, come out and my own personal goal - for some reason, they wanted me to take the position of language designer - of course, I'll have a lot of help on that, but my goal for that part of it - I'm giving a talk at Linux World Expo in Atlanta in October and they wanted me to do a keynote there and I didn't know what I was going to talk about. Well, now I do, and I'll talk about the new Perl and where it's going as a language. The language design is now going to be separate from the implementation design and we've got a number of other positions that we've named names for and you'll see those if you look at the press release, but the schedule is not nailed down yet, but we'll try to act a bit like pointy-haired bosses and do some of that scheduling.

Question: Will Perl 6 have specs and then you could implement something that looks like this, or will it be kind of like now, where the system won't work?

Larry: Yeah. I don't know how strict a spec it will be from the language design point of view. I'm not really big on that sort of spec and there is some value to using the reference implementation approach and what we currently have is a reference implementation will no second implementation, well, unless you count the JVM work, but, obviously there are benefits to having things justified well enough that you could implement another one even if you didn't want to, so we'll definitely be working in that direction and - do you have something? And there was something else I was going to say - what we particularly want to stress in terms of - is not perhaps so much the spec as developing our current regression test. Well, we call them regression tests, but they're almost more acceptance tests, but, we developed our acceptance into real regression tests then you'd further develop the real regression tests into a validation test of what the language actually means and actually go out and explore all the nooks and crannies and say, "This is Perl, this is not Perl," and then we actually have a machine-readable spec. And to me that's actually a lot more important than what the verbiage on the human readable thing says.

Question: I think that one of the problems with the P5P model is the infrastructure of mailing lists tends to push people into a "I say this," "but I say that," "but I say this," "but I say that." Kind of heads banging against each other and not really resolving anything. There's various kinds of work. Horscht Ritter did his Ivis project and there's Wicki's and things like that that are used to try and resolve things and give people a place to put their argument as best possible and then they can move on to something else, so you might want to look into some of that.

Larry: Yes, we're already planning to do some of that. Each working group will have somebody in charge who makes the final decisions. We will have mailing lists which are at least two tiered in that they will have official inner ringers but anybody can listen in and contribute indirectly if they want to contribute to the actual working group - the people who are actually on the working group. We want to have an official RFC sort of kind of mechanism for not just this sort of off-the-top-of-your-head "Oh, wouldn't it be nice if this," "Wouldn't it be nice if that." If you have a real proposal for a feature, make an official proposal in an official place with all the things that make it an official proposal, and, you know - I guess that's it about that one.

Question: Thank you for taking on this endeavor. Can you hint at any language changes that you're considering?

Larry: I hinted at some of them. Everything is negotiable, but everything will not be traded away. On a philosophical level, I have the profound feeling that if I like something, other people will like it and if I don't like something, other people probably don't like it so much, and so I really trust my instincts on where things will be going. I'll be getting a lot of feedback on that, too. We've been getting a lot of feedback for the past 10 years on things that people think are kind of grouty. For instance, there's really no reason why formats should be in the core anymore. They should be a, you know, come in as a module. There are things that could be done perhaps to clean up ambiguities and indirect object impacts. Basically what we are saying at this point is if we are going to bite the bullet and require translation of Perl 5 to Perl 6, that really means that we can consider anything that still allows us to translate most scripts. Now we do not expect to be able to translate a 100 percent, but if we can translate with 95-percent accuracy 95 percent of the scripts, and 100-percent accuracy 80 percent of the scripts, then that's getting into the ballpark, but on the other hand, sometimes you have to break a few somethings or other to make an omelette. Other specific features, can you remember any of the ones we named?

Question: One of the problems with language translators is you lose all the comments and formatting and things like that.

Larry: Not the Ox to Perl translators. I wrote that.

Audience member: Well, can you make sure that will be the way it works in the Perl 5 to Perl 6.

Larry: It may, it may lose some of the formatting information, but there's ways to annotate a syntax tree with additional information. And if we have to get a little incestuous with the compiler and turn off certain optimizations to get a more pristine syntax tree to translate that -- lots of things like that can be done. This is subject that -- I enjoy doing things like that, so I don't think you need to worry about that not getting done reasonably adequately.

Question: I somewhat disagree with your bracketing and indentation style, but respect your right to observe your particular religion. With the automatic language translation that you're having, will I have, within reason, the ability to observe my religion? How long have you been thinking about doing this? How long have you been stewing on it?

Larry: Since yesterday.

Audience member: Are you contemplating any changes to Pod?

Larry: Everything is negotiable.

Audience member: You said you wish to steal from some languages. Which languages in particular over the last decade?

Larry: Cobalt. Identification Division. I always wanted an identification divisioner. No, really, I don't want an identification division. The problem with identification division is it really puts a crimp in Perl's poetry, or in Cobalt poetry. How many poems can you start off identification division? One. What's your favorite language, besides Perl? You know, lots of languages do more of a byte code thing. Some of these things, a lot of these things are not borrowing from a specific language. There's multiple languages that use the byte code thing. There's various languages with a cleaner object interface to their IL and such. There's lots of languages that, I don't know, do various things, but we - when something like - when you see all the new languages coming out and they all have a garbage collector and that helps them fit together into browsers and things like that better, you start thinking, maybe we ought to think about that.

Question: Larry, more and more people, of course, are using Perl today to write larger and larger software. Is there anything in Perl 6 that you can think of off hand that might make that easier? Large software tend to have - tend to appreciate things like stronger touch up checking, for example.

Larry: Yes, certainly. We already got the hooks in there to start putting some things in there optionally, and you could have used strict-type checking if you wanted to or you could use Stricter or something. You can use bondage and discipline - whatever you want to call the module. Also, as part of the redesign - here's a biggie - we intend to get rid of quite a few of those strange global variables or the strange one. We will certainly get rid of dollar sharp.

These Weeks on p5p 2000/10/23



Notes

You can subscribe to an email version of this summary by sending an empty message to p5p-digest-subscribe@plover.com.

Please send corrections and additions to simon@brecon.co.uk.

For family reasons, I wasn't able to get the summary out last week; please note that this is a special bumper double issue, covering the last two weeks.

What is our Unicode model?

The big debate these past couple of weeks has been what our Unicode model actually means: What use bytes should really do, whether it's all right to upgrade values to UTF8 without telling anyone, and so on. If you're at all interested in how Perl's Unicode strategy works - or, as some would claim, doesn't - take a look at this thread. There were nearly a hundred messages in the thread, and it went off into discussion of line disciplines, the mechanism by which Unicode data will be read from external sources such as files. The thread also sparked up a bit of private email, and the following things were resolved:

  • It doesn't matter if data gets upgraded to UTF8 internally; if there is a place where it does matter, that's a bug.

  • Line disciplines need to happen before anything else.

  • use bytes does make sense. Honest.

A lot of the thread was taken up with technical details about how line disciplines should be implemented; having line disciplines will mean we'll almost certainly need our own version of stdio or something very like it.

Why not use sfio?

The obvious alternative to implementing our own version of stdio is, of course, steal someone else's, and so the Nicks (Clarke and Ing-Simmons) began looking at ATT's sfio library.

sfio claims to do what we want from it, and its developers (David Korn - yes, him - and Phong Vo) posted to p5p to reassure us that it would be suitable, and that the new license allows it to be distributed with Perl; so it looks like that's a very big possibility.

It will need to be ported to VMS and some other platforms, but between us and the sfio guys, this shouldn't be a problem, and it would benefit all the other sfio users out there too. Let's get to it!

More than 256 Files / sysopen

Piotr Piatkowski reported that you can't open more than 256 files on Solaris. Well, that turns out to be standard Solaris behaviour, but you should be able to get around it with sysopen, since that calls the underlying operating system's open directly, right? However, it turns out that isn't the case - sysopen actually calls fopen, which is wrong! A bug report wasn't opened for this, and there wasn't a fix.

What if cc changes?

Jarkko noted that since people are now shipping Perl with their commercial Unix distributions, the compilers they compile Perl under - say, Sun cc - may well not be the compiler that the user ends up with - typically, something like FSF gcc. This causes problems when adding CPAN modules - Sarathy suggested the answer was to hack the Config.pm, and also to have packaged module distributions. (Like the module .debs and .rpms) ActiveState's PPM was suggested as a natural way to do this, and it has recently been released onto CPAN.

Unicode on EBCDIC

Perl 5.6 on EBCDIC is slowly getting together; Ignasi Roca noticed a bug, and attempted a fix, but Peter Prymmer and I were already on to it. Peter also contributed a lot of miscellaneous patches to places where the test suites implied ASCII.

AIX Is Confused

Guy Hulbert helped us get AIX on its feet after it seemed to be having some difficulties about the contents of struct tm. Merijn then contributed README.aix, which had been mysteriously left unwritten. Guy also identified a few more issues, which I believe have been fixed but I'm not sure.

Unicode split fixed!

Last time, I reported problems with split and Unicode; the good news is, Jarkko has fixed it. I also contributed a fix to the wide-character- ~ issue identified last time, which didn't work on 64-bit operating systems. Jarkko and I simultaneously discovered this was a bug in pp_chr which was restricted to 32-bit values because it took a U32 off the stack - when you're dealing with characters these days, anything up to and including a UV is fair game.

In other Unicode news, there was a bug in which doing something like "$1$utf8" caused a read-only variable error; Jarkko fixed it, and Nicholas Clark produced some tests.

Fixes to Carp

Ben Tilly produced a big patch to address some errant and confusing behaviour in Carp; unfortunately, this appeared to cause more of the test suits to fail than before. The patch hasn't been revisited as far as I can tell, so someone might want to take a look at it.

open() might fail

Jarkko was evidently in "astute" mode this fortnight, asking "what to do when open/fopen/fdopen fail?". This potentially interesting thread got bogged down by a lot of language-lawyering about errno. This usually happens with errno.

Perl 5 is 5!

Perl 5 became 5 years old sometime last week. I say "sometime" because there was a little debate as to when birthdays are celebrated with respect to timezone. Yes, I thought it was incredulous, too, but here's the original release message as kept by Jeff Okamoto.

Flip-flop bug

Jeff Pinyan noticed that the flip-flop operator in scalar context ( if 1...20 and so on) has interesting problems when there isn't an open filehandle. I fixed this, badly, but Hugo fixed it tidily; Mark-Jason Dominus called for a documentation patch, which hasn't appeared yet. If someone wants to read the thread and make the suggested change, that would be lovely!

Exiting a subroutine via next strikes again!

Alexander Farber found a small File::Find example which caused segfaults. There's a workaround in place in the current development Perl, but the fact remains that exiting a subroutine via next can, and does, cause segfaults. This would be a good thing for someone who's feeling bold to investigate.

New constant sub mechanism

John Tobey came up with a reimplementation of newCONSTSUB. What's that, then? Well, when you use constant or create a sub like like sub foo () { 42 }, Perl optimizes this and creates a special subroutine which is inlined at compile time. His new version seems to save about 70% of space, but of what space and under what conditions I couldn't tell you. This one, however, also optimises sub () { return 42; } by creating a "constant" flag for a CV. Adding the extra flag meant that things like dump.c and B::* had to be updated, which John duly did. Nice, thorough job.

Regex segfaulting

There's been rather a lot of segfaults this time. Marc Lehmann noticed one with regexps. Andreas did the regression testing, and Hugo fixed it; Ilya came in to help explain the details and the pitfalls.

use vars lets you do naughty things

Robert Spier was doing a bit of bug archaeology, and found that use vars lets you do naughty things; in particular, it lets you create variables like $f; that you can't easily access. He provided a patch which caused use vars when combined with use strict q/vars/ to croak on unreasonable variable names. This raises interesting philosophical issues - should we be allowed to say

use vars q($x $y;$z);

What do you think?

Various

In other news, the usual collection of bug reports, bug fixes, non-bug reports, questions, answers, and no spam. No flamage; or at least, nothing memorably amusing.

Until next week I remain, your humble and obedient servant,


Simon Cozens

Programming GNOME Applications with Perl

Table of Contents

Architecture Introduction
Hello, World
Adding a Menu Bar
Adding an About Box
Adding More Chrome
Final Program

GNOME is the Unix desktop. It's a framework for writing graphical applications with Unix, providing drag-and-drop, interapplication communication, CORBA components (what's called ``OLE'' in the Windows world) a standard, good-looking interface, and all the other features that you'd expect from modern graphical applications.

And it's available for Perl, which means Perl programmers can create really neat applications, too. Except there's one slight barrier ...

    % perldoc GNOME
    No documentation found for "GNOME".

I recently needed to write a GNOME application and hit this barrier, and I had to figure the whole thing out pretty much for myself. So, I decided to write these tutorials so that you, dear reader, don't have to. In this first episode, we'll create an extremely simple application, but one with a full, standard GNOME interface.

Architecture Introduction

The GNOME is a complicated beast and made up of many different libraries and components. Thankfully, for the purposes of this tutorial and a reasonable amount of your programming, you only need to know about two parts: GTK+ and GNOME.

You might have heard about Tk, the ``other'' graphical toolkit Perl people use. Tk's role in life is to do the laborious job of talking to the X server and telling it how to draw buttons, menus, controls and dialog boxes, and then firing off Perl routines in response to the user's actions. It's an intermediary between the raw power of the X server and the comfort of Perl.

GTK+ performs a similar job, but it does so with arguably more beauty. GTK+ will be providing all the windows, the buttons, the text labels, the text inputs, all of the graphical elements for our application. It'll also, critically, provide the main event loop that connects the user's actions with our code.

The Gnome library places another layer of abstraction over GTK+, providing us with higher level graphical objects, such as main application windows, about boxes, button panels, dialog boxes, color and font selection boxes, and it also gives us the ``glue'' to interact with other parts of the GNOME environment - spelling checkers, calculators and resources from other applications.

  • It's worth pointing out at this stage that there's a Visual Basic style drag-and-drop IDE for the GNOME, called Glade. It can produce Perl code, and some of you may find it a lot easier to construct applications using that; however, you'd be advised to keep reading so that you can understand what the code that Glade produces actually does.

Hello, World

We'll show two versions of the classic ``Hello, World'' application here: first, a version which just uses GTK+, and then a GNOME version.

Here's the GTK+ version:

   1 #!/usr/bin/perl -w
   2
   3 use strict;
   4 use Gnome;
   5 
   6 my $NAME = 'Hello World';
   7
   8 init Gnome $NAME;
   9
  10 my $w = new Gtk::Window -toplevel;
  11
  12 my $label = new Gtk::Label "Hello, world";
  13 $w->add($label);
  14
  15 show_all $w;
  16 
  17 main Gtk;

On line 4, we load the main Gnome module; this'll load up the GTK+ module for us. Line 8 sets up everything we need in this session and registers the application. We pass the application's name to the init method.

On line 10, we create our main window. This is a top level window, meaning it's not a sub-window of anything else. Next, we need to create the message label that'll say ``Hello, world''; any text we want to place on a window has to be in a Gtk::Label object, so we create an object, and put it in $label. Now, as of line 12, this label isn't doing anything - it's created, but it doesn't live anywhere. We want it to appear on our window, so we call the window's add method and add the label object.

We next decide what we're going to show at the start of the program. We'll show everything the window and everything attached to it - in our case, the label. So, we call the window's show_all method. Note that this doesn't actually put the window on the screen yet; it just dictates what gets shown initially.

Finally, the statement that kicks off the action is main Gtk; - this passes control over to GTK+'s main event loop, which first paints the window and the label on the screen and then waits for something to happen.

  • Once we've said main Gtk; our program has given up control - everything that happens after that occurs in reaction to the user's actions. Instead of the normal, procedural approach where we, as programmers, have control over what the program does, we now have to take a passive, reactive approach, providing responses to what the user does. The way we do this is through callbacks, and we'll see examples of this later on. But it's important to note that main Gtk; is where our job finishes and GTK+'s job begins.

Next, here's the Gnome version:


     1  #!/usr/bin/perl -w
     2  
     3  use strict;
     4  use Gnome;
     5  
     6  my $NAME = 'Hello World';
     7  
     8  init Gnome $NAME;
     9  
    10  my $app = new Gnome::App $NAME, $NAME;
    11  
    12  my $label = new Gtk::Label "Hello, world";
    13  $app->set_contents($label);
    14  
    15  show_all $app;
    16  
    17  main Gtk;

It's the same length, and most of it is the same. This is what we've changed:

    10  my $app = new Gnome::App $NAME, $NAME;

Instead of creating a window, we're now shifting up a level and saying that we're creating an entire application. We pass the application's name to the new method twice - once as the window's title, and once to register it with the GNOME environment.

We've also changed the line which adds the label to the window:

    13  $app->set_contents($label);

Why is it ``set contents'' here and not add? The answer lies in the way GTK+ puts graphical elements (``widgets'') inside windows, which is based on the idea of containers. Simply put, you can only have one widget in a window; thankfully, some widgets can contain other widgets. What we're saying above is that the main contents of this window, the one widget we're allowed, is the label.

Now, one thing you may have noticed if you've been exiting these examples using the ``Close'' button on your window manager is that the Perl application doesn't finish; we have to break out of it using ^C or similar. When a GNOME application receives notification from the window manager that it is to close, GNOME sends us a signal; not a true Unix signal which is implemented by the kernel, but a GNOME signal which is purely a feature of GNOME. We need to catch this signal and install a signal handler which cleanly shuts down the program. Here's how to do this:

    my $app = new Gnome::App $NAME, $NAME;
    signal_connect $app 'delete_event',
                         sub { Gtk->main_quit; return 0 };

We're connecting a handler to the ``delete event'' signal, which tells us to clean up and go home, and we catch it with an anonymous subroutine. This subroutine calls the main_quit method of GTK+, which terminates the main loop.

Now our application should cleanly close down. But it still doesn't do much.

Adding a Menu Bar

As I mentioned earlier, the benefits of GNOME over GTK+ are that most of the standard things we expect of an application are ready-made for us. Let's create some standard menus for our application. Add this after the signal_connect line:

      $app->create_menus(
        {type => 'subtree',
         label => '_File',
         subtree => [
                {type => 'item',
                 label => 'E_xit',
                 pixmap_type => 'stock',
                 pixmap_info => 'Menu_Quit'
                }
                    ]
        },
        {type => 'subtree',
         label => '_Help',
         subtree => [
                {type => 'item', 
                 label => '_About...',
                 pixmap_type => 'stock',
                 pixmap_info => 'Menu_About'
                }
                    ]
        }
      );

We pass to create_menus a series of anonymous hashes, one for each of the main menu tabs we want to create. The type of subtree for each tab means that there will be other menus beneath this one. In the label option, we put an underscore before the character which is to be the key accelerator for the menu item; Alt-F will open the File menu. The subtree option is an anonymous array of menu items; here, we only put one item in each of our two subtree arrays and each item has an anonymous hash.

For these hashes, the type this time is item - an ordinary menu item, rather than the start of a submenu. The menu items have little icons before the names. We use the stock GNOME icon library by saying that pixmap_type is 'stock', and we use the Menu_Quit and Menu_About to get standard quit and ``about box'' icons suitable for display in menus.

If you run your application again, you should see a menu bar. Now, want to see something really impressive? I said GNOME did all the work for you. Try this:

   LANG=fr_FR perl hello.pl

If all goes well, the menus should appear - but this time translated into French. Where was the code for that? GNOME did it. Try a few others - pt_PT for Portuguese, de_DE for German, el_GR (if you've got the fonts for it) for Greek. Magical!

There's one small problem, though: Our menu doesn't do anything. Let us first fix that Exit item since we already know how to shut down a GTK+ application. Change the item's hash so it looks like this:

                {type => 'item',
                 label => 'E_xit',
                 pixmap_type => 'stock',
                 pixmap_info => 'Menu_Quit',
                 callback => sub {Gtk->main_quit; return 0 }
                }

What we've said is that when the menu item gets selected, GNOME should ``call us back'' by executing the code we give it. We specify a subroutine reference to be called when the user selects the item.

Adding an About Box

Now let's fix up the other menu item by adding an about box. Once again, GNOME has done the work for us. We'll add a callback to the About... menu option, and we'll make it a subroutine reference:

                {type => 'item',
                 label => '_About...',
                 pixmap_type => 'stock',
                 pixmap_info => 'Menu_About',
                 callback => \&about_box
                }

Our subroutine will create and display the box:

           sub about_box {
               my $about = new Gnome::About $NAME, "v1.0",
                  "(C) Simon Cozens, 2000", ["Simon Cozens"],
                "This program is released under the same terms as Perl itself";
             show $about;
         }

The Gnome::About class gives us a ready-made about box: we just have to supply the name of our application, its version, copyright information, an anonymous array of the authors' names and any other comments. Then we show the box just as we showed the main window before. When we click the ``OK'' button, the window is automatically removed.

Adding More Chrome

GNOME applications also have two other distinctive interface features: a toolbar and a status bar. We'll first add the toolbar. Put this after the menu code:

   $app->create_toolbar(
    {
        type => 'item',
        label => 'Exit',
        pixmap_type => 'stock',
        pixmap_info => 'Quit',
        hint => "Click here to quit",
        callback => sub { Gtk->main_quit },
    }, {
        type => 'item',
        label => 'About...',
        pixmap_type => 'stock',
        pixmap_info => 'About',
        hint => "More information about this app",
        callback => \&about_box
    }
   );

Once more, we're passing a series of anonymous hashes, and most of the entries should be familiar to you now. The hint is what is displayed when the mouse pointer lingers over the button. Our callbacks and pixmaps are the same as before.

Next, the status bar:

    my $bar = new Gnome::AppBar 0,1,"user" ;
    $bar->set_status("   Welcome   ");
    $app->set_statusbar( $bar );

First, we create a new AppBar object, an application status bar. Then we write our initial status onto it using the set_status method. Again, this bar now exists but it doesn't appear on the screen as it doesn't have a home. We connect it to the application using the app's set_statusbar method, and it'll now appear at the bottom of our main window.

Final Program

Here's what you should have ended up with at the end of this tutorial:

    #!/usr/bin/perl -w
    use strict;
    use Gnome;
    my $NAME = 'Hello World';
    init Gnome $NAME;
    my $app = new Gnome::App $NAME, $NAME;
    signal_connect $app 'delete_event', sub { Gtk->main_quit; return 0 };
    $app->create_menus(
               {type => 'subtree',
                label => '_File',
                subtree => [
                    {type => 'item',
                     label => 'E_xit',
                     pixmap_type => 'stock',
                     pixmap_info => 'Menu_Quit',
                     callback => sub { Gtk->main_quit; return 0 }
                    }
                       ]
               },
               {type => 'subtree',
                label => '_Help',
                subtree => [
                    {type => 'item', 
                     label => '_About...',
                     pixmap_type => 'stock',
                     pixmap_info => 'Menu_About',
                     callback => \&about_box
                    }
                       ]
               }
              );
    $app->create_toolbar(
                 {
                  type => 'item', 
                  label => 'Exit', 
                  pixmap_type => 'stock', 
                  pixmap_info => 'Quit', 
                  hint => "Click here to quit",
                  callback => sub { Gtk->main_quit }, 
                 }, {
                 type => 'item',
                 label => 'About...', 
                 pixmap_type => 'stock',
                 pixmap_info => 'About',
                 hint => "More information about this app",
                 callback => \&about_box
                }
                );
    my $label = new Gtk::Label "Hello, world";
    $app->set_contents($label);
    my $bar = new Gnome::AppBar 0,1,"user" ;
    $bar->set_status("   Welcome   ");
    $app->set_statusbar( $bar );
    show_all $app;
    main Gtk;
    sub about_box {
      my $about = new Gnome::About $NAME, "v1.0", 
      "(C) Simon Cozens, 2000", ["Simon Cozens"], 
      "This program is released under the same terms as Perl itself";
      show $about;
    }


=head1 Summary

So, we've now created our first application using GNOME/Perl. It complies with the GNOME interface standards, it's got standard menus, a toolbar, a status bar and an about box. It looks, feels and acts like a real GNOME application, and all in about 70 lines of Perl.

Next time, we'll start to create a more useful application, a recipe organizer, and we'll use some slightly more sophisticated widgets such as containers, input areas, scroll bars and list boxes.

Beginner's Introduction to Perl

Editor's note: this venerable series is undergoing updates. You might be interested in the newer versions, available at:

First, a Little Sales Pitch

Table of Contents

Part 2 of this series
Part 3 of this series
Part 4 of this series
Part 5 of this series
Part 6 of this series

A Word About Operating Systems
Your First Perl Program
Functions and Statements
Numbers, Strings and Quotes
Variables
Comments
Loops
The Miracle of Compound Interest
Play Around!

Welcome to Perl.

Perl is the Swiss Army chainsaw of scripting languages: powerful and adaptable. It was first developed by Larry Wall, a linguist working as a systems administrator for NASA in the late 1980s, as a way to make report processing easier. Since then, it has moved into a large number of roles: automating system administration, acting as glue between different computer systems; and, of course, being one of the most popular languages for CGI programming on the Web.

Why did Perl become so popular when the Web came along? Two reasons: First, most of what is being done on the Web happens with text, and is best done with a language that's designed for text processing. More importantly, Perl was appreciably better than the alternatives at the time when people needed something to use. C is complex and can produce security problems (especially with untrusted data), Tcl can be awkward and Python didn't really have a foothold.

It also didn't hurt that Perl is a friendly language. It plays well with your personal programming style. The Perl slogan is ``There's more than one way to do it,'' and that lends itself well to large and small problems alike.

In this first part of our series, you'll learn a few basics about Perl and see a small sample program.

A Word About Operating Systems

In this series, I'm going to assume that you're using a Unix system and that your Perl interpreter is located at /usr/local/bin/perl. It's OK if you're running Windows; most Perl code is platform-independent.

Your First Perl Program

Take the following text and put it into a file called first.pl:

     #!/usr/local/bin/perl
     print "Hi there!\n";

(Traditionally, first programs are supposed to say Hello world!, but I'm an iconoclast.)

Now, run it with your Perl interpreter. From a command line, go to the directory with this file and type perl first.pl. You should see:

     Hi there!

The \n indicates the ``newline'' character; without it, Perl doesn't skip to a new line of text on its own.

Functions and Statements

Perl has a rich library of functions. They're the verbs of Perl, the commands that the interpreter runs. You can see a list of all the built-in functions on the perlfunc main page. Almost all functions can be given a list of parameters, which are separated by commas.

The print function is one of the most frequently used parts of Perl. You use it to display things on the screen or to send information to a file (which we'll discuss in the next article). It takes a list of things to output as its parameters.

   print "This is a single statement.";
   print "Look, ", "a ", "list!";

A Perl program consists of statements, each of which ends with a semicolon. Statements don't need to be on separate lines; there may be multiple statements on one line or a single statement can be split across multiple lines.

    print "This is "; print "two statements.\n"; print "But this ",
          "is only one statement.\n";

Numbers, Strings and Quotes

There are two basic data types in Perl: numbers and strings.

Numbers are easy; we've all dealt with them. The only thing you need to know is that you never insert commas or spaces into numbers in Perl. always write 10000, not 10,000 or 10 000.

Strings are a bit more complex. A string is a collection of characters in either single or double quotes:

   'This is a test.'
   "Hi there!\n"

The difference between single quotes and double quotes is that single quotes mean that their contents should be taken literally, while double quotes mean that their contents should be interpreted. For example, the character sequence \n is a newline character when it appears in a string with double quotes, but is literally the two characters, backslash and n, when it appears in single quotes.

    print "This string\nshows up on two lines.";
    print 'This string \n shows up on only one.';

(Two other useful backslash sequences are \t to insert a tab character, and \\ to insert a backslash into a double-quoted string.)

Variables

If functions are Perl's verbs, then variables are its nouns. Perl has three types of variables: scalars, arrays and hashes. Think of them as ``things,'' ``lists,'' and ``dictionaries.'' In Perl, all variable names are a punctuation character, a letter or underscore, and one or more alphanumeric characters or underscores.

Scalars are single things. This might be a number or a string. The name of a scalar begins with a dollar sign, such as $i or $abacus. You assign a value to a scalar by telling Perl what it equals, like so:

    $i = 5;
    $pie_flavor = 'apple';
    $constitution1776 = "We the People, etc.";

You don't need to specify whether a scalar is a number or a string. It doesn't matter, because when Perl needs to treat a scalar as a string, it does; when it needs to treat it as a number, it does. The conversion happens automatically. (This is different from many other languages, where strings and numbers are two separate data types.)

If you use a double-quoted string, Perl will insert the value of any scalar variables you name in the string. This is often used to fill in strings on the fly:

    $apple_count = 5; 
    $count_report = "There are $apple_count apples.";
    print "The report is: $count_report\n";

The final output from this code is The report is: There are 5 apples..

Numbers in Perl can be manipulated with the usual mathematical operations: addition, multiplication, division and subtraction. (Multiplication and division are indicated in Perl with the * and / symbols, by the way.)

    $a = 5;
    $b = $a + 10;       # $b is now equal to 15.
    $c = $b * 10;       # $c is now equal to 150.
    $a = $a - 1;        # $a is now 4, and algebra teachers are cringing.

You can also use special operators like ++, --, +=, -=, /= and *=. These manipulate a scalar's value without needing two elements in an equation. Some people like them, some don't. I like the fact that they can make code clearer.

   $a = 5;
   $a++;        # $a is now 6; we added 1 to it.
   $a += 10;    # Now it's 16; we added 10.
   $a /= 2;     # And divided it by 2, so it's 8.

Strings in Perl don't have quite as much flexibility. About the only basic operator that you can use on strings is concatenation, which is a $10 way of saying ``put together.'' The concatenation operator is the period. Concatenation and addition are two different things:

   $a = "8";    # Note the quotes.  $a is a string.
   $b = $a + "1";   # "1" is a string too.
   $c = $a . "1";   # But $b and $c have different values!

Remember that Perl converts strings to numbers transparently whenever it's needed, so to get the value of $b, the Perl interpreter converted the two strings "8" and "1" to numbers, then added them. The value of $b is the number 9. However, $c used concatenation, so its value is the string "81".

Just remember, the plus sign adds numbers and the period puts strings together.

Arrays are lists of scalars. Array names begin with @. You define arrays by listing their contents in parentheses, separated by commas:

    @lotto_numbers = (1, 2, 3, 4, 5, 6);  # Hey, it could happen.
    @months = ("July", "August", "September");

The contents of an array are indexed beginning with 0. (Why not 1? Because. It's a computer thing.) To retrieve the elements of an array, you replace the @ sign with a $ sign, and follow that with the index position of the element you want. (It begins with a dollar sign because you're getting a scalar value.) You can also modify it in place, just like any other scalar.

    @months = ("July", "August", "September");
    print $months[0];   # This prints "July".
    $months[2] = "Smarch";  # We just renamed September!

If an array doesn't exist, by the way, you'll create it when you try to assign a value to one of its elements.

    $winter_months[0] = "December";  # This implicitly creates @winter_months.

Arrays always return their contents in the same order; if you go through @months from beginning to end, no matter how many times you do it, you'll get back July, August and September in that order. If you want to find the length of an array, use the value $#array_name. This is one less than the number of elements in the array. If the array just doesn't exist or is empty, $#array_name is -1. If you want to resize an array, just change the value of $#array_name.

    @months = ("July", "August", "September");
    print $#months;         # This prints 2.
    $a1 = $#autumn_months;  # We don't have an @autumn_months, so this is -1.
    $#months = 0;           # Now @months only contains "July".

Hashes are called ``dictionaries'' in some programming languages, and that's what they are: a term and a definition, or in more correct language a key and a value. Each key in a hash has one and only one corresponding value. The name of a hash begins with a percentage sign, like %parents. You define hashes by comma-separated pairs of key and value, like so:

    %days_in_summer = ( "July" => 31, "August" => 31, "September" => 30 );

You can fetch any value from a hash by referring to $hashname{key}, or modify it in place just like any other scalar.

    print $days_in_summer{"September"}; # 30, of course.
    $days_in_summer{"February"} = 29;   # It's a leap year.

If you want to see what keys are in a hash, you can use the keys function with the name of the hash. This returns a list containing all of the keys in the hash. The list isn't always in the same order, though; while we could count on @months to always return July, August, September in that order, keys %days_in_summer might return them in any order whatsoever.

    @month_list = keys %days_in_summer;
    # @month_list is now ('July', 'September', 'August') !

The three types of variables have three separate namespaces. That means that $abacus and @abacus are two different variables, and $abacus[0] (the first element of @abacus) is not the same as $abacus{0} (the value in abacus that has the key 0).

Comments

Notice that in some of the code samples from the previous section, I've used code comments. These are useful for explaining what a particular piece of code does, and vital for any piece of code you plan to modify, enhance, fix, or just look at again. (That is to say, comments are vital for all code.)

Anything in a line of Perl code that follows a # sign is a comment. (Except, of course, if the # sign appears in a string.)

   print "Hello world!\n";  # That's more like it.
   # This entire line is a comment.

Loops

Almost every time you write a program, you'll need to use a loop. Loops allow you run a particular piece of code over and over again. This is part of a general concept in programming called flow control.

Perl has several different functions that are useful for flow control, the most basic of which is for. When you use the for function, you specify a variable that will be used for the loop index, and a list of values to loop over. Inside a pair of curly brackets, you put any code you want to run during the loop:

     for $i (1, 2, 3, 4, 5) {
         print "$i\n";
     }

This loop prints the numbers 1 through 5, each on a separate line.

A handy shortcut for defining loops is using .. to specify a range of numbers. You can write (1, 2, 3, 4, 5) as (1 .. 5). You can also use arrays and scalars in your loop list. Try this code and see what happens:

    @one_to_ten = (1 .. 10);
    $top_limit = 25;
    for $i (@one_to_ten, 15, 20 .. $top_limit) {
        print "$i\n";
    }

The items in your loop list don't have to be numbers; you can use strings just as easily. If the hash %month_has contains names of months and the number of days in each month, you can use the keys function to step through them.

    for $i (keys %month_has) {
        print "$i has $month_has{$i} days.\n";
    }
    for $marx ('Groucho', 'Harpo', 'Zeppo', 'Karl') {

        print "$marx is my favorite Marx brother.\n";
    }

The Miracle of Compound Interest

You now know enough about Perl - variables, print, and for() - to write a small, useful program. Everyone loves money, so the first sample program is a compound-interest calculator. It will print a (somewhat) nicely formatted table showing the value of an investment over a number of years. (You can see the program at compound_interest.pl)

The single most complex line in the program is this one:

    $interest = int (($apr / 100) * $nest_egg * 100) / 100;

$apr / 100 is the interest rate, and ($apr / 100) * $nest_egg is the amount of interest earned in one year. This line uses the int() function, which returns the integer value of a scalar (its value after any fractional part has been stripped off). We use int() here because when you multiply, for example, 10925 by 9.25%, the result is 1010.5625, which we must round off to 1010.56. To do this, we multiply by 100, yielding 101056.25, use int() to throw away the leftover fraction, yielding 101056, and then divide by 100 again, so that the final result is 1010.56. Try stepping through this statement yourself to see just how we end up with the correct result, rounded to cents.

Play Around!

At this point you have some basic knowledge of Perl syntax and a few simple toys to play with - print, for(), keys(), and int(). Try writing some simple programs with them. Here are two suggestions, one simple and the other a little more complex:

  • A word frequency counter. How often does each word show up in an array of words? Print out a report. (Hint: Use a hash to count of the number of appearances of each word.)
  • Given a month and the day of the week that's the first of that month, print a calendar for the month. (Remember, you need \n to go to a new line.)

This Week on p5p 2000/10/08



Notes

First, the meta-news this week is that you have a new author. I've taken over producing the perl5-porters summary from Mark-Jason Dominus, so from now on, please send your corrections and additions to simon@brecon.co.uk.

This week was relatively quiet, with just a couple of hundred mails.

Self-tying is broken

Alan Burlison noted that you can't create a self-tie anymore without Perl going into nasty recursion. What's a self-tie? As you may guess, tieing a variable to itself goes like this:

    package MyTie;
    sub TIEARRAY { bless $_[1], $_[0] }

    package main;
    my (@self);
    tie(@self, "MyTie", \@self);

The problem comes with how tying is implemented. Perl uses a structure called "magic" attached to each tied variable to contain subs like FETCH. When you fetch a variable's value, Perl checks to see whether there's a FETCH sub stored in the variable's magic. If there is, Perl calls it to determine the variable's value. If this sub tries to look at the variable itself, Perl then calls the FETCH sub stored in the variable's magic to try and get its value, which, in turn, tries to look at the variable, and things end up in a mess. Alan then investigated and tentatively blamed Sarathy, but this turns out to have been a bad call; the problem's much deeper than that and nobody could think of a way of fixing it without breaking nested ties or threading. Part of the problem appears to be that nobody really knows what self-ties are supposed to do anyway. No patch was forthcoming, and the discussion fizzled out.

Elsewhere, Daniel Chetlin fixed some bugs with tied filehandles. He makes some other interesting points about what happened while he was working on it. Read about it.

Virtual values

Jarkko came up with a new idea for copy-on-write sharing of scalars, but said that it may have to wait until Perl 6 before anyone implements it. Ilya popped up and characteristically said that he'd already done it.

Why is unshift slow?

Ben Tilly started a discussion on why unshift is so slow, which didn't get much attention but contains a lot of juicy goodies for people thinking about how to make array handling faster. No patches or benchmarks came out of the discussion, but there's an opportunity for someone to try some of the ideas there and see whether there's any improvement.

More Perl hacking guidelines

Mark Fisher contributed a helpful patch for perlhack on using Purify to debug Perl, prompting people to promise to explain how to use their favourite code checkers. None of the other patches has materialized, and the discussion moved on to the fine print of PERL_DESTRUCT_LEVEL.

Integer and floating-point handling.

Currently, putting together two variables which contain integers is actually a floating-point operation, meaning that variables have to get upgraded to hold floating-point values when they don't necessarily need to. It was hard to see a good way around this, but Sarathy came up with a suggestion. Someone with a few hours to spare should consider applying Sarathy's idea and coming back with some benchmarks.

In other floating-point news, Nick Clark found some 64-bit bugs to do with the conversion between UVs and NVs. There was some discussion, but no patch.

printf %v format bug fixed

[This section of the report was written by Dominus.]

In an earlier report, I mentioned a bug in the %v format specifier for printf and suggested that this was a good candidate for fixing by someone who was just starting out fixing Perl bugs. Avi F. stepped up to the challenge and provided a patch, which was accepted. Unfortunately, Avi did this right after I stopped doing p5p summaries, so never received proper credit. Thank you very much, Avi!

Read about it.

[We now return you to Simon Cozens.]

Jarkko impersonates me

I noticed a bug in split which caused Unicode values to be denatured; Jarkko's worsening influenza produced a delirium that caused him to not only babble to himself but also to impersonate me.

But at least he fixed the bug. If I hack Unicode too long, will this happen to me too? Find out next week!

Various

The usual collection of bug reports, test results, bug fixes, test results, esoterica and more test results. The clown who was sending duplicate copies of CCs to people has been ejected from the premises.

Until next week, I remain your humble and obedient servant,


Simon Cozens

Report from YAPC::Europe


This is written for those poor souls who couldn't make it to give them aflavor of the wonderful event they missed.

Despite being an ex-Londoner, I got on the wrong tube Friday morning. The train was packed and boiling hot; there weren't any bomb scares which was a nice change, but elevators weren't working in some stations, which were therefore closed. I arrived rather late in the first part of Johan Vromans talk. I'd opted for this instead of Stas Bekman's talk because I'd been to all Stas' excellent talks at Apachecon in March and have read his mod_perl Guide many times. The talk was interesting and solid, moving rapidly through the material. The threat of being dropped into a bucket of water silenced the last mobile phone.

During the lunch break I had a chance to go out into the hot sunshine; the predicted weather for the weekend was ``warm and wet,'' but it was sweltering, sunny and dry, quite uncharacteristic. I also met some people I knew through e-mail; it's amazing how different people are in person. After lunch, the first dilemma arose: Writing Solid Perl or The Template Toolkit. I ended up going to the Toolkit talk by Andy Wardley and Simon Matthews. The talk was excellent; there are many similar tools available in Perl -- we've all written our own (I know I have), but I was so impressed with the Toolkit that I expect to use it in my projects in future.

Most people gathered into clusters and went off into the evening to pubs, clubs and the like to enjoy London's lowlights. I'm not one for socializing with more than a handful of people at once, so at the end of the day I left, exhausted by all the input I had received, and met my wife. She spent most of the entire three days up the road at the National Gallery with 2,000 of the world's finest paintings, free entrance and a nice cafe; she was in heaven. Back at our accommodation, I finally got to look through the goodie bag. The T-shirt is something to take rock-climbing or mountaineering -- it's orange color is so bright it could save your life. BSDI thoughtfully enclosed not just a CD, but a nice pair of red plastic horns. I tried them on, but they didn't seem to do anything. When my wife tried them, well, it was like my birthday all over again. O'Reilly thoughtfully enclosed Johan's Perl 5 reference book which was most appreciated.

By Saturday, I was beginning to get an idea about who was who. Greg McCarroll is a big man, both physically and in personality; he's full of energy and fun, and spent endless hours rescheduling in the light of last minute changes. His name tag read ``grep,'' indicating he was either too tired to spell his own name or he's read one too many Unix books. Dave Cross, London.pm's chieftain, was also involved as an organizer and a sponsor. He's the type of man who gives consultants a bad reputation. His obvious sincerity, honesty and competence will unduly raise client expectations, ruining customers for those consultants who follow him. A few Americans also made it; some guy called brian d foy tried to attack grep; apparently grep had spoken Brian's name in uppercase. There was also an insect who grep called Nathanial. The entire conference was haunted by an Australian spirit called Damian that keeps telling Larry Wall to be objective when reading the RFCs.

The next talk was Art and Open Source by Simon Cozens. It was sheer synchronicity; he put up a slide of a Seurat painting while up the road my wife was looking at a real Seurat in the National. The talk was interesting, entertaining and thought-provoking. Simon likened modern artists who are no longer bound by patronage to produce flattering and `nice' art to suit their patrons to open source software developers who are not constrained by their employers but can build the software they want in the way they want.

Kevin Lenzo talked about YAS (Yet Another Society) and hoped for future YAPC's in North America, Europe (Amsterdam?) and beyond. It was clear that despite being thousands of miles away, Kevin had been of considerable help to Leon Brocard and London.pm in getting things off the ground. Next up was Honza Pazdziora from the Czech Republic. He's written software that allows a user on a Linux box to convert Word or Excel documents into more tractable formats by executing Word or Excel on a single Windows box and getting the results back on their own machine. For those of us who see Windows as a legacy OS, this looks useful. BRIAN D FOY (who's too far away to hit me :) ) gave a summary of what the Perl 6 process is and a taste of some of the issues it will address. There is unlikely to be any progress on the Do::WhatIWant module, but many other issues will be addressed.

One or two people had a minor interest in Perl 6, so there was an impromptu lunchtime discussion with Nathan Torkington on problems people have with Perl 5 and what they want from Perl 6. My own contribution to the discussion was so exemplary that gnat said that he had memorized every one of my ideas and that he would personally implement all of them without me even having to go through the bother of writing an RFC; needless to say everyone else's ideas were either barmy or impractical, or both. With less than a week to go until the RFC deadline, it isn't too late to get the full richness of COBOL syntax into Perl.

Lunch was on O'Reilly, but I opted for sunshine in case we didn't get any more for another year. After lunch it was lightning talk time. They were fast furious and fun. Some serious, some not, but each one was worth listening to. Because Kevin made us keep to time (no one argues with a whip) we managed to get some extra ones in, the last one being gnat's Python talk in which he gave a wonderful overview of Python's virtues in a calm and balanced manner, pointing out one or two minor drawbacks in passing. In a conversation afterward, it was clear that gnat really likes the RFCs which amount to ``make Perl 6 into Python.''

Charlie Stross gave a somewhat calmer talk on his NetServer::Generic module. This talk reminded me of the adage: ``Don't do it until you look in CPAN!'' I've written quite a few programs recently that could have been made far more robust and considerably simplified had I used this module. Dave Cross followed with his talk on data munging, plus a shameless plug for his new book on the subject. Piers ``Heckler'' Cawley rounded off the day by confessing to some of his Perl sins. Pier's sin was to reinvent the wheel, and to do so thoroughly and with an exemplary expenditure of time and effort over several weeks. He had to produce a sophisticated sort function that would work on a variety of data, some strings, some numbers and some combinations. He ended up finding the perfect algorithm to convert his data into a form that would compare properly in all cases from Knuth's third volume. He later realised that the entire function could be replaced by a simple call to pack().

Piers asked others to come up to the stage and confess along with him. What was sad about this was, no matter how awful the sin, no one was surprised because we are so jaded by seeing bad programming practices.

For the evening, Dave invited everyone to walk around Southwark with an occasional refreshment stop; by all accounts this was a popular trip. I arrived at 10 a.m. Sunday to find a new schedule (first event, 9 a.m. -- bah!). I went to Marc Mosthav's talk on IrDA with Perl and Windows even though most people ran to the other talk rather than face the Win32 API. It's amazing what you can get working on toy OSs. Chris Young from the BBC described how his little bit of the BBC makes extensive use of Perl, surrounding perfectly good live TV with text boxes filled with facts and journalistic jottings -- it was almost enough to make me rush out and buy a digital TV. Leon popped up again with a talk on Graphing Perl -- I wasn't looking forward to it, bar and pie charts don't give me much of a thrill but it turned out the talk was about proper graphs, the nodes and edges sort; Leon presented visualizations not just of Perl data structures, but of Perl programs, including graphical representations of a profile of a program which uses color and position to identify hot spots. It was an excellent presentation given with dry, reticent humor.

Because of the popularity of Saturday's Perl 6 discussion, another discussion took place before lunch. Gnat is a charming and engaging man, but one can't help wondering when his brain would explode from all the conflicting demands. Things will doubtless be better once Larry has decreed the language that we shall speak and the energy can go into implementation.

Foolishly, I went for a walk at lunchtime and got soaked; warm rain is unusual and it took hours for me to dry off. Kevin Lenzo's talk on Speech and Perl followed lunch; it was heartening to see that the open-source world is keeping up with both speech recognition and speech synthesis, and to see the amazing processing work that Kevin is doing in Perl. My final talk was Benjamin Holzman's on XML::Generator where he showed that what should be a simple task has many subtleties and isn't straightforward. The winding up time was conducted by grep, who will never get a job at Sotheby's; auctions aren't supposed to lead to rugby scrums, bloodshed and fighting, but this auction came close. Buffy finally made an appearance, in the form of videos provided by sponsor BlackStar. It was a fitting end to three great days; I certainly had a wonderful time and I'm sure that must be true for just about everyone. Congratulations to all concerned.

References: YAPC::Europe site Nouns, My lightning talk

How Perl Helped Me Win the Office Football Pool

Everyone who has read the Camel Book knows that the three great virtues of programming are laziness, impatience and hubris. Most of the columns that appear here at www.perl.com have to do, in one way or another, with either laziness or impatience; i.e. getting things done faster or more simply than before. While I'll touch on those virtues, this is primarily a column about hubris. And aside from programming, there are few greater sources of hubris in the workplace than the office football pool.

A football pool looks something like this. (Please note that any relationship between this and a real football pool is completely accidental.)

FAVORITE SPREAD UNDERDOG
Thursday
TENNESSEE 3 Indianapolis
Sunday
BUFFALO 7 Washington
TAMPA BAY 8 St. Louis
PITTSBURGH 5 ½ Dallas
GREEN BAY 4 Cincinnati
Miami 10 ½ SEATTLE
Atlanta 1 ½ KANSAS CITY
NEW ENGLAND 9 N. Y. Giants
ARIZONA 8 Detroit
NEW ORLEANS 3 Baltimore
Philadelphia 9 ½ CAROLINA
Cleveland 3 N. Y. JETS
SAN DIEGO 1 San Francisco
Minnesota 5 OAKLAND
Monday
DENVER 9 Chicago

The home team is printed in capital letters. The pool is traditionally marked For Amusement Only so management will not suspect that there is gambling taking place in the office.

The object is to pick the most winning teams. The team marked as the favorite must win by at least the number of points in the point spread.

There are two basic approaches to picking winners in a football pool. One is to spend hours researching the teams, studying scouting reports, teams' records against their opponents and against the spread, looking up injury reports on the Internet, and so on. This method is often used by the ex-jocks and DBA's in the department. The other approach is to pick the teams randomly, based on, perhaps, the color of the teams' uniforms. This approach is typically used by the less analytical members of the department; for example, the boss' secretary or Visual Basic programmers.

Surprisingly, my informal studies have shown that both methods are about equally successful. I suspect this is because the folks making the betting lines in Las Vegas know much more about football than the average office worker, and are good at picking point spreads that give each team an equal chance of winning.

Several of us in the office realized this a few years ago. Since then, we have amused ourselves with devising more and more creative ways to randomly pick our pools each week. For example:

  • Pick the team with the longest or shortest name.
  • Around Christmas, pick the team closest to the North Pole.
  • Pick the team with the most ex-Philadelphia Eagles (since they always seem to do better after leaving town).

Well, that was fun for a while, but it's tough devising new ways to pick the teams each week. Even flipping a coin all those times becomes tiresome. There had to be a better way.

Perl, of course, makes it easy to be lazy. Why flip a coin by hand when you can have perl flip a coin for you? So I wrote pick_pool.pl:

#!/usr/local/bin/perl -w
use strict;

# Most weeks have 15 games, but allow the user to specify a lower
# number for bye weeks
my $num_games = $ARGV[0] || 15;

for (1..$num_games) {
     printf "Game %2d: ", $_;
     print ((rand(2) < 1) ? "FAVORITE" : "underdog");
     print "\n";
     print "\n" if ($_ % 5 == 0);
}
Here's a sample run:
$ pick_pool.pl
Game  1: underdog
Game  2: FAVORITE
Game  3: FAVORITE
Game  4: underdog
Game  5: underdog

Game  6: FAVORITE
Game  7: FAVORITE
Game  8: FAVORITE
Game  9: FAVORITE
Game 10: underdog

Game 11: underdog
Game 12: underdog
Game 13: FAVORITE
Game 14: underdog
Game 15: FAVORITE
$

This remarkably simple program actually won the pool for me the second week I used it, much to the annoyance of the other people in the pool.

But it turns out we can do even better. Using Damian Conway's Quantum::Superpositions module, it's possible to pick all the games simultaneously, in constant time:

#!/usr/local/bin/perl -w
use strict;
use Quantum::Superpositions;

my @picks;

my $num_games = $ARGV[0] || 15;

for (1..$num_games) {
     push @picks, any("FAVORITE", "underdog");
}

foreach my $pick (@picks) {
    if ($pick eq "FAVORITE") {
 print "yes\n";
    }
}

print @picks;

What's more, thanks to the magic of quantum mechanics, this program will get each game right each week! That's because, just as Schoedinger's cat is both dead and alive until the box is opened, each pick remains in both a "FAVORITE" and an "underdog" state until it is observed. All you need to do is prevent the person running the pool from looking at your picks until after all the games have been played.

As proof, let's compare the @picks generated by the quantum superpositions code to a sample set of winners:

my @winners = qw(underdog
   FAVORITE
   underdog
   underdog
   underdog
   underdog
   FAVORITE
   FAVORITE
   FAVORITE
   underdog
   FAVORITE
   FAVORITE
   FAVORITE
   underdog
   FAVORITE);

foreach my $i (0..$#picks) {
    printf "Game %2d: ", $i+1;
    if ($picks[$i] eq $winners[$i]) {
 print "winner\n";
    } else {
 print "loser\n";
    }
}

As expected, the output is

Game  1: winner
Game  2: winner
Game  3: winner
Game  4: winner
Game  5: winner
Game  6: winner
Game  7: winner
Game  8: winner
Game  9: winner
Game 10: winner
Game 11: winner
Game 12: winner
Game 13: winner
Game 14: winner
Game 15: winner
Visit the home of the Perl programming language: Perl.org

Sponsored by

Monthly Archives

Powered by Movable Type 5.13-en