Recently in Larry Wall Category

Programming is Hard, Let's Go Scripting...

I think, to most people, scripting is a lot like obscenity. I can't define it, but I'll know it when I see it. Here are some common memes floating around:

    Simple language
    "Everything is a string"
    Rapid prototyping
    Glue language
    Process control
    Compact/concise
    Worse-is-better
    Domain specific
    "Batteries included"

...I don't see any real center here, at least in terms of technology. If I had to pick one metaphor, it'd be easy onramps. And a slow lane. Maybe even with some optional fast lanes.

Easy Onramps

But basically, scripting is not a technical term. When we call something a scripting language, we're primarily making a linguistic and cultural judgment, not a technical judgment.

I see scripting as one of the humanities. It's our linguistic roots showing through. So speaking of roots...

The Past

Suppose you went back to Ada Lovelace and asked her the difference between a script and a program. She'd probably look at you funny, then say something like: Well, a script is what you give the actors, but a program is what you give the audience. That Ada was one sharp lady...

Since her time, we seem to have gotten a bit more confused about what we mean when we say scripting. It confuses even me, and I'm supposed to be one of the experts.

So I'm afraid all I can do is give you my own worm's eye view of the past, the present, and the future. Let me warn you that I am not without a few prejudices here and there.

BASIC

Now, however it was initially intended, I think BASIC turned out to be one of the first major scripting languages, especially the extended version that DEC put onto its minicomputers called BASIC/PLUS, which happily included recursive functions with arguments. I started out as a BASIC programmer. Some people would say that I'm permanently damaged. Some people are undoubtedly right.

But I'm not going to apologize for that. All language designers have their occasional idiosyncracies. I'm just better at it than most. :-)

RSTS BASIC/PLUS

Anyway, when I was a RSTS programmer on a PDP-11, I certainly treated BASIC as a scripting language, at least in terms of rapid prototyping and process control. I'm sure it warped my brain forever. Perl's statement modifiers are straight out of BASIC/PLUS. It even had some cute sigils on the ends of its variables to distinguish string and integer from floating point.

But you could do extreme programming. In fact, I had a college buddy I did pair programming with. We took a compiler writing class together and studied all that fancy stuff from the dragon book. Then of course the professor announced we would be implementing our own language, called PL/0. After thinking about it a while, we announced that we were going to do our project in BASIC. The professor looked at us like were insane. Nobody else in the class was using BASIC. And you know what? Nobody else in the class finished their compiler either. We not only finished but added I/O extensions, and called it PL 0.5. That's rapid prototyping.

Unix?

I remember one day our computer center got a letter from Bell Labs telling us that we could get a tape of Unix V6 for cheap, only $100 because they were coming out shortly with V7. We all looked at each other and said, Why would we ever want to use this thing called Unix? We have RSTS.

JAM (no not that one)

My first scripting language was written in BASIC. For my job in the computer center I wrote a language that I called JAM, short for Jury-rigged All-purpose Meta-language. Story of my life...

JAM was an inside-out text-processing language much like PHP, except that HTML hadn't been invented yet. We mostly used it as a fancy macro processor for BASIC. Unlike PHP, it did not have 3,000 functions in one namespace. We wouldn't have had the memory, for one thing.

LISP

For good or ill, when I went off to grad school, I studied linguistics, so the only computer language I used there was LISP. It was my own personal McCarthy era.

Is LISP a candidate for a scripting language? While you can certainly write things rapidly in it, I cannot in good conscience call LISP a scripting language. By policy, LISP has never really catered to mere mortals.

And, of course, mere mortals have never really forgiven LISP for not catering to them.

Pascal, Ada

Once I got into industry, I wrote a compiler in Pascal for a discrete event simulator, and slavered over the forthcoming Ada specs. As a linguist, I don't think of Ada as a big language. Now, English and Japanese, those are big languages. Ada is just a medium-sized language.

Unix, shell

After several years I finally became acquainted with Unix and its various scripting languages. OK, to be more precise, BSD, and csh.

BSD, csh

Yeah, yeah, I know. More brain damage...

I also learned a little C.

C

That's because a little C is all there is. I'm still learning those libraries though.

shell + awk + sed + find + expr...

But the frustrations of Unix shell programming led directly to the creation of Perl, which I don't really have time to tell. But essentially, I found that shell scripting was intrinsically limited by the fact that most of its verbs are not under its control and hence largely inconsistent with each other. And the nouns are impoverished, restricted to strings and files, with who-knows-what typology.

C xor shell

More destructive was the mindset that it was a one-dimensional universe: you either programmed in C or you programmed in shell, because they're obviously at opposite ends of the One True Continuum. Perl came about when I realized that scripting did not always have to viewed as the opposite of programming, but that a single language could be pretty good for both. That opened up a huge ecological niche. Many of you have seen my old clamshell diagram, with the two dimensions of manipulexity and whipuptitude.

Tcl

After Perl came Tcl, which in a sense is a purer scripting language than Perl. Perl just pretends that everything is a string when it's convenient, but Tcl really believes that as a controlling metaphor. The string metaphor tends to have bad performance ramifications, but that's not why Tcl languished, I think. There were two reasons for that.

First, Tcl stayed in the Unix mindset that controlling tools was the opposite of creating tools, so they didn't optimize much. The fast parts can always be written in C, after all.

The second reason was the lack of a decent extension mechanism, so you ended up with separate executables for expect, incr-tcl, etc.

I must say, though, that I've always admired Tcl's delegational model of semantics. But it fell into the same trap as LISP by expecting everyone to use the One True Syntax. Speaking of the One True Syntax:

Python

After Tcl came Python, which in Guido's mind was inspired positively by ABC, but in the Python community's mind was inspired negatively by Perl. I'm not terribly qualified to talk about Python however. I don't really know much about Python. I only stole its object system for Perl 5. I have since repented.

Ruby

I'm much more qualified to talk about Ruby--that's because a great deal of Ruby's syntax is borrowed from Perl, layered over Smalltalk semantics. I've always viewed Ruby as a much closer competitor for Perls ecological niche, not just because of the borrowed ideas, but because both Perl and Ruby take their functional programming support rather more seriously that Python does. On the other hand, I think Ruby kind of screwed up on its declaration syntax, among other things.

*sh

Meanwhile, the Bourne shell was extended into the Korn shell and bash. I didn't have much to do with those either. Thankfully. I will say that the continued evolution of the shell shows just how crufty a language can get when you just keep adding on ad hoc syntactic features.

PHP

We've also seen the rise of PHP, which takes the worse-is-better approach to dazzling new depths, as it were. By and large PHP seems to be making the same progression of mistakes as early Perl did, only slower. The one thing it does better is packaging. And when I say packaging, I don't mean namespaces.

JavaScript

Then there's JavaScript, a nice clean design. It has some issues, but in the long run JavaScript might actually turn out to be a decent platform for running Perl 6 on. Pugs already has part of a backend for JavaScript, though sadly that has suffered some bitrot in the last year. I think when the new JavaScript engines come out we'll probably see renewed interest in a JavaScript backend.

Monad/PowerShell

I've looked a bit at Microsoft's Monad, and I'm pleased to note that it has object pipes like Perl 6. I just hope they don't patent it.

Lua, AppleScript

There are other scripting languages in wide use. Sadly, I must confess I never looked closely at Lua or AppleScript, probably because I'm not a game designer with a Mac.

Actually, I suspect it runs deeper than that, which brings us up to the present time.

The Present

When I look at the present situation, what I see is the various scripting communities behaving a lot like neighboring tribes in the jungle, sometimes trading, sometimes warring, but by and large just keeping out of each other's way in complacent isolation.

I tend to take an anthropological view of these things. Many of you here are Perl programmers, but some of you come from other programming tribes. And depending on your tribal history, you might think of a string as a pointer to a byte array if you're a C programmer, or as a list if you're a functional programmer, or as an object if you're a Java programmer. I view a string as a Text, with a capital T.

Text

I read that word from a postmodern perspective. Of course, the term Postmodern is itself context-sensitive. Some folks think Postmodernism means little more than the Empowerment of the Vulgar. Some folks think the same about Perl.

But I take Postmodernism to mean that a Text, whether spoken or written, is an act of communication requiring intelligence on both ends, and sometimes in the middle too. I don't want to talk to a stupid computer language. I want my computer language to understand the strings I type.

Perl is a postmodern language, and a lot of conservative folks feel like Postmodernism is a rather liberal notion. So it's rather ironic that my views on Postmodernism were primarily informed by studying linguistics and translation as taught by missionaries, specifically, the Wycliffe Bible Translators. One of the things they hammered home is that there's really no such thing as a primitive human language. By which they mean essentially that all human languages are Turing complete.

When you go out to so-called primitive tribes and analyze their languages, you find that structurally they're just about as complex as any other human language. Basically, you can say pretty much anything in any human language, if you work at it long enough. Human languages are Turing complete, as it were.

Human languages therefore differ not so much in what you can say but in what you must say. In English, you are forced to differentiate singular from plural. In Japanese, you don't have to distinguish singular from plural, but you do have to pick a specific level of politeness, taking into account not only your degree of respect for the person you're talking to, but also your degree of respect for the person or thing you're talking about.

So languages differ in what you're forced to say. Obviously, if your language forces you to say something, you can't be concise in that particular dimension using your language. Which brings us back to scripting.

How many ways are there for different scripting languages to be concise?

How many recipes for borscht are there in Russia?

Language designers have many degrees of freedom. I'd like to point out just a few of them.

early binding / late binding

Binding in this context is about exactly when you decide which routine you're going to call for a given routine name. In the early days of computing, most binding was done fairly early for efficiency reasons, either at compile time, or at the latest, at link time. You still tend to see this approach in statically typed languages. With languages like Smalltalk, however, we began to see a different trend, and these days most scripting languages are trending towards later binding. That's because scripting languages are trying to be dwimmy (Do What I Mean), and the dwimmiest decision is usually a late decision because you then have more available semantic and even pragmatic context to work with. Otherwise you have to predict the future, which is hard.

So scripting languages naturally tend to move toward an object-oriented point of view, where the binding doesn't happen 'til method dispatch time. You can still see the scars of conflict in languages like C++ and Java though. C++ makes the default method type non-virtual, so you have to say virtual explicitly to get late binding. Java has the notion of final classes, which force calls to the class to be bound at compile time, essentially. I think both of those approaches are big mistakes. Perl 6 will make different mistakes. In Perl 6 all methods are virtual by default, and only the application as a whole can tell the optimizer to finalize classes, presumably only after you know how all the classes are going to be used by all the other modules in the program.

single dispatch / multiple dispatch

In a sense, multiple dispatch is a way to delay binding even longer. You not only have to delay binding 'til you know the type of the object, but you also have to know the types of all rest of the arguments before you can pick a routine to call. Python and Ruby always do single dispatch, while Dylan does multiple dispatch. Here is one dimension in which Perl 6 forces the caller to be explicit for clarity. I think it's an important distinction for the programmer to bear in mind, because single dispatch and multiple dispatch are philosophically very different ideas, based on different metaphors.

With single-dispatch languages, you are basically sending a message to an object, and the object decides what to do with that message. With multiple dispatch languages, however, there is no privileged object. All the objects involved in the call have equal weight. So one way to look at multiple dispatch is that the objects are completely passive. But if the objects aren't deciding how to bind, who is?

Well, it's sort of a democratic thing. All the routines of a given name get together and hold a political conference. (Well, not really, but this is how the metaphor works.) Each of the routines is a delegate to the convention. All the potential candidates put their names in the hat. Then all the routines vote on who the best candidate is, and the next best, and the next best after that. And eventually the routines themselves decide what the best routine to call is.

So basically, multiple dispatch is like democracy. It's the worst way to do late binding, except for all the others.

But I really do think that's true, and likely to become truer as time goes on. I'm spending a lot of time on this multiple dispatch issue because I think programming in the large is mutating away from the command-and-control model implicit in single dispatch. I think the field of computation as a whole is moving more toward the kinds of decisions that are better made by swarms of insects or schools of fish, where no single individual is in control, but the swarm as a whole has emergent behaviors that are somehow much smarter than any of the individual components.

eager evaluation / lazy evaluation

Most languages evaluate eagerly, including Perl 5. Some languages evaluate all expressions as lazily as possible. Haskell is a good example of that. It doesn't compute anything until it is forced to. This has the advantage that you can do lots of cool things with infinite lists without running out of memory. Well, at least until someone asks the program to calculate the whole list. Then you're pretty much hosed in any language, unless you have a real Turing machine.

So anyway, in Perl 6 we're experimenting with a mixture of eager and lazy. Interestingly, the distinction maps very nicely onto Perl 5's concept of scalar context vs. list context. So in Perl 6, scalar context is eager and list context is lazy. By default, of course. You can always force a scalar to be lazy or a list to be eager if you like. But you can say things like for 1..Inf as long as your loop exits some other way a little bit before you run into infinity.

eager typology / lazy typology

Usually known as static vs. dynamic, but again there are various positions for the adjustment knob. I rather like the gradual typing approach for a number of reasons. Efficiency is one reason. People usually think of strong typing as a reason, but the main reason to put types into Perl 6 turns out not to be strong typing, but rather multiple dispatch. Remember our political convention metaphor? When the various candidates put their names in the hat, what distinguishes them? Well, each candidate has a political platform. The planks in those political platforms are the types of arguments they want to respond to. We all know politicians are only good at responding to the types of arguments they want to have...

There's another way in which Perl 6 is slightly more lazy than Perl 5. We still have the notion of contexts, but exactly when the contexts are decided has changed. In Perl 5, the compiler usually knows at compile time which arguments will be in scalar context, and which arguments will be in list context. But Perl 6 delays that decision until method binding time, which is conceptually at run time, not at compile time. This might seem like an odd thing to you, but it actually fixes a great number of things that are suboptimal in the design of Perl 5. Prototypes, for instance. And the need for explicit references. And other annoying little things like that, many of which end up as frequently asked questions.

limited structures / rich structures

Awk, Lua, and PHP all limit their composite structures to associative arrays. That has both pluses and minuses, but the fact that awk did it that way is one of the reasons that Perl does it differently, and differentiates ordered arrays from unordered hashes. I just think about them differently, and I think a lot of other people do too.

symbolic / wordy

Arguably APL is also a kind of scripting language, largely symbolic. At the other extreme we have languages that eschew punctuation in favor of words, such as AppleScript and COBOL, and to a lesser extent all the Algolish languages that use words to indicate blocks where the C-derived languages use curlies. I prefer a balanced approach here, where symbols and identifiers are each doing what theyre best at. I like it when most of the actual words are those chosen by the programmer to represent the problem at hand. I don't like to see words used for mere syntax. Such syntactic functors merely obscure the real words. That's one thing I learned when I switched from Pascal to C. Braces for blocks. It's just right visually.

Actually, there are languages that do it even worse than COBOL. I remember one Pascal variant that required your keywords to be capitalized so that they would stand out. No, no, no, no, no! You don't want your functors to stand out. It's shouting the wrong words: IF! foo THEN! bar ELSE! baz END! END! END! END!

Anyway, in Perl 6 we're raising the standard for where we use punctuation, and where we don't. We're getting rid of some of our punctuation that isn't really pulling its weight, such as parentheses around conditional expressions, and most of the punctuational variables. And we're making all the remaining punctuation work harder. Each symbol has to justify its existence according to Huffman coding.

Oddly, there's one spot where we're introducing new punctuation. After your sigil you can add a twigil, or secondary sigil. Just as a sigil tells you the basic structure of an object, a twigil tells you that a particular variable has a weird scope. This is basically an idea stolen from Ruby, which uses sigils to indicate weird scoping. But by hiding our twigils after our sigils, we get the best of both worlds, plus an extensible twigil system for weird scopes we haven't thought of yet.

We think about extensibility a lot. We think about languages we don't know how to think about yet. But leaving spaces in the grammar for new languages is kind of like reserving some of our land for national parks and national forests. Or like an archaeologist not digging up half the archaeological site because we know our descendants will have even better analytical tools than we have.

Really designing a language for the future involves a great deal of humility. As with science, you have to assume that, over the long term, a great deal of what you think is true will turn out not to be quite the case. On the other hand, if you don't make your best guess now, you're not really doing science either. In retrospect, we know APL had too many strange symbols. But we wouldn't be as sure about that if APL hadn't tried it first.

compile time / run time

Many dynamic languages can eval code at run time. Perl also takes it the other direction and runs a lot of code at compile time. This can get messy with operational definitions. You don't want to be doing much file I/O in your BEGIN blocks, for instance. But that leads us to another distinction:

declarational / operational

Most scripting languages are way over there on the operational side. I thought Perl 5 had an oversimplified object system till I saw Lua. In Lua, an object is just a hash, and there's a bit of syntactic sugar to call a hash element if it happens to contain code. Thats all there is. They don't even have classes. Anything resembling inheritance has to be handled by explicit delegation. That's a choice the designers of Lua made to keep the language very small and embeddable. For them, maybe it's the right choice.

Perl 5 has always been a bit more declarational than either Python or Ruby. I've always felt strongly that implicit scoping was just asking for trouble, and that scoped variable declarations should be very easy to recognize visually. Thats why we have my. It's short because I knew we'd use it frequently. Huffman coding. Keep common things short, but not too short. In this case, 0 is too short.

Perl 6 has more different kinds of scopes, so we'll have more declarators like my and our. But appearances can be deceiving. While the language looks more declarative on the surface, we make most of the declarations operationally hookable underneath to retain flexibility. When you declare the type of a variable, for instance, you're really just doing a kind of tie, in Perl 5 terms. The main difference is that you're tying the implementation to the variable at compile time rather than run time, which makes things more efficient, or at least potentially optimizable.

immutable classes / mutable classes

Classes in Java are closed, which is one of the reasons Java can run pretty fast. In contrast, Ruby's classes are open, which means you can add new things to them at any time. Keeping that option open is perhaps one of the reasons Ruby runs so slow. But that flexibility is also why Ruby has Rails.

Perl 6 will have an interesting mix of immutable generics and mutable classes here, and interesting policies on who is allowed to close classes when. Classes are never allowed to close or finalize themselves, for instance. Sorry, for some reason I keep talking about Perl 6. It could have something to do with the fact that we've had to think about all of these dimensions in designing Perl 6.

class-based / prototype-based

Here's another dimension that can open up to allow both approaches. Some of you may be familiar with classless languages like Self or JavaScript. Instead of classes, objects just clone from their ancestors or delegate to other objects. For many kinds of modeling, it's actually closer to the way the real world works. Real organisms just copy their DNA when they reproduce. They don't have some DNA of their own, and an @ISA array telling you which parent objects contain the rest of their DNA.

The meta-object protocol for Perl 6 defaults to class-based, but is flexible enough to set up prototype-based objects as well. Some of you have played around with Moose in Perl 5. Moose is essentially a prototype of Perl 6's object model. On a semantic level, anyway. The syntax is a little different. Hopefully a little more natural in Perl 6.

passive data, global consistency / active data, local consistency

Your view of data and control will vary with how functional or object-oriented your brain is. People just think differently. Some people think mathematically, in terms of provable universal truths. Functional programmers don't much care if they strew implicit computation state throughout the stack and heap, as long as everything looks pure and free from side-effects.

Other people think socially, in terms of cooperating entities that each have their own free will. And it's pretty important to them that the state of the computation be stored with each individual object, not off in some heap of continuations somewhere.

Of course, some of us can't make up our minds whether we'd rather emulate the logical Sherlock Holmes or sociable Dr. Watson. Fortunately, scripting is not incompatible with either of these approaches, because both approaches can be made more approachable to normal folk.

info hiding / scoping / attachment

And finally, if you're designing a computer language, there are a couple bazillion ways to encapsulate data. You have to decide which ones are important. What's the best way to let the programmer achieve separation of concerns?

object / class / aspect / closure / module / template / trait

You can use any of these various traditional encapsulation mechanisms.

transaction / reaction / dynamic scope

Or you can isolate information to various time-based domains.

process / thread / device / environment

You can attach info to various OS concepts.

screen / window / panel / menu / icon

You can hide info various places in your GUI. Yeah, yeah, I know, everything is an object. But some objects are more equal than others.

syntactic scope / semantic scope / pragmatic scope

Information can attach to various abstractions of your program, including, bizarrely, lexical scopes. Though if you think about it hard enough, you realize lexical scopes are also a funny kind of dynamic scope, or recursion wouldn't work right. A state variable is actually more purely lexical than a my variable, because it's shared by all calls to that lexical scope. But even state variables get cloned with closures. Only global variables can be truly lexical, as long as you refer to them only in a given lexical scope. Go figure.

So really, most of our scopes are semantic scopes that happen to be attached to a particular syntactic scope.

You may be wondering what I mean by a pragmatic scope. That's the scope of what the user of the program is storing in their brain, or in some surrogate for their brain, such as a game cartridge. In a sense, most of the web pages out there on the Internet are part of the pragmatic scope. As is most of the data in databases. The hallmark of the pragmatic scope is that you really don't know the lifetime of the container. It's just out there somewhere, and will eventually be collected by that Great Garbage Collector that collects all information that anyone forgets to remember. The Google cache can only last so long. Eventually we will forget the meaning of every URL. But we must not forget the principle of the URL. That leads us to our next degree of freedom.

use Lingua::Perligata;

If you allow a language to mutate its own grammar within a lexical scope, how do you keep track of that cleanly? Perl 5 discovered one really bad way to do it, namely source filters, but even so we ended up with Perl dialects such as Perligata and Klingon. What would it be like if we actually did it right?

Doing it right involves treating the evolution of the language as a pragmatic scope, or as a set of pragmatic scopes. You have to be able to name your dialect, kind of like a URL, so there needs to be a universal root language, and ways of warping that universal root language into whatever dialect you like. This is actually near the heart of the vision for Perl 6. We don't see Perl 6 as a single language, but as the root for a family of related languages. As a family, there are shared cultural values that can be passed back and forth among sibling languages as well as to the descendants.

I hope you're all scared stiff by all these degrees of freedom. I'm sure there are other dimensions that are even scarier.

But... I think its a manageable problem. I think its possible to still think of Perl 6 as a scripting language, with easy onramps.

And the reason I think its manageable is because, for each of these dimensions, it's not just a binary decision, but a knob that can be positioned at design time, compile time, or even run time. For a given dimension X, different scripting languages make different choices, set the knob at different locations.

You can't even think about X!
There's only one way to do X!
There's more than one way to do X!
There are too many ways to do X!

You may recognize some slogans in here.

Curling Up

So I'm not suggesting that all scripting languages have to take all these dimensions into account, even if Perl 6 tries to. The scripting paradigm is not any one of these dimensions. According to various theories the universe may be laid out in ten or twenty dimensions, but generally we get by with only about three and a half of those dimensions. The rest are said to be curled up. Maybe we live in a scripting universe.

Most of the scripting languages we call Perl 6 will have most of these dimensions curled up most of the time. But unlike the real universe, where it takes huge machines to uncurl these dimensions, we'll make the dimensions uncurl just by keeping our declarations straight. Well, we'll try. And where that fails, we'll rely on the culture to keep things straight.

For example, that's exactly what happened already with Perl 5. We have the declarations, use strict; use warnings;. But it's the culture that decided to enforce the use of them. So much so that we've decided that they should be the default for most of Perl 6. It was one of those decisions by the hive. In this case the swarm turned out to be smarter than the language designer. And that's as it should be.

The Future

Well, so what's the future of scripting?

In my completely unbiased opinion, that would be Perl 6. :-)

Seriously though, it's always safe to predict that the ecological landscape will end up with many small languages and a few dominant ones. Some languages like AppleScript have particular ecological niches and are unlikely to grow out of them. Other languages get used outside their original niche. There will always be the generalists, like crows and mockingbirds, and the specialists, like penguins and dodos. (Well, maybe not always the dodos...)

Among the generalists, the conventional wisdom is that the worse-is-better approach is more adaptive. Personally, I get a little tired of the argument: My worse-is-better is better than your worse-is-better because I'm better at being worser! Is it really true that the worse-is-better approach always wins? With Perl 6 we're trying to sneak one better-is-better cycle in there and hope to come out ahead before reverting to the tried and true worse-is-better approach. Whether that works, only time will tell.

The State of the Onion 10

Welcome to the tenth State of the Perl Onion. For those of you who are unfamiliar with my methods, this is the annual speech wherein I ramble on about various things that are only marginally related to the state of Perl. I've gotten pretty good at rambling in my old age.

In the Scientific American that just came out, there's an article on chess experts, written by an expert, on what makes experts so expert. This expert claims that you can become an expert in just about anything if you study it persistently for ten years or so. So, since this is my tenth State of the Onion, maybe I'm about to become an expert in giving strange talks. One can only hope (not).

Speaking of chess, how many of you recognize this?

Does this help?

This is, of course, the mnemonic for the old Linnean taxonomy of biological classification.

Those of you who understand computers better than critters can think of these as nested namespaces.

This is all about describing nature, so naturally, different languages care about different levels.

For instance, PHP isn't much into taxonomy, so everything in PHP is just its own species in a flat namespace. Congratulations, this is your new species name:

Ruby, of course, is interested primarily in Classes.

Python, as the "anti-Perl," is heavily invested in maintaining Order.

Now, you might be smart enough to program in Haskell if you've received a MacArthur Genus award.

... used to be I couldn't spell "genus," and now I are one ...

Moving right along toward the other end of the spectrum, we have JavaScript that kind of believes in Phyla without believing in Classes.

And at the top of the heap, playing king of the mountain, we have languages like C# and Java. The kingdom of Java only has one species.

The kingdom of C# has many species, but they all look like C#.

Well, that leaves us with families.

I expect I have a pretty good excuse for thinking a lot about families lately, and here is my excuse:

This is Julian, my grandson. Julian, meet the open source hackers. Open source hackers, meet Julian.

Many of you will remember my daughter Heidi from previous OSCONs. A couple years ago she married Andy, and Julian is the result. I think he's a keeper. Julian, I mean.

Well, and Andy too.

Andy obviously has his priorities straight. I would certainly recommend him as a son-in-law to anyone. (Wait, that doesn't quite work ...)

There are many definitions of family, of course. Here's a mommy and a daddy truck. They live on a truck farm, and raise little trucks.

Out in California, the word "family" keeps leaping out at me from various signs. People use the word "family" in some really weird ways.

There was a Family Fun Center, with a miniature golf course. I believe that sign. At least for the golf. As a parent, I'm not sure the game arcade is for the whole family. I'm an expert in staying out of loud places.

But the sign that said "Farmers Feed America--Family Water Alliance" ... I suspect the word "family" is in there more for its PR value than anything else.

And, of course, "family planning" is for when you plan not to have a family. Go figure.

All of my kids were unplanned, but that doesn't mean they were unwanted.

Many of you know that I have four kids, but in a strange way, I really have five, if you count Perl.

Geneva thinks of Perl as more or less her twin sister, since they were both born in 1987. But then, Geneva is strange.

Some people think Perl is strange too. That's okay--all my kids are a little strange. They come by it naturally.

Here's a self-portrait of the other end of Geneva.

Here's what you usually see of Lewis.

Here's some of Aron, pulling the door that says "push."

And here's Heidi.

She always was a pale child.

Actually, here's the real picture.

You can see she's actually quite sane. Compared to the rest of us.

Here's a picture of my wife Gloria.

Here's another picture of my wife. Well, her arms. The feet are my mom's. Actually, this is really a picture of my granddog, Milo. He's the one on the right.

I've talked before about how the stages in Perl's life are very much like that of a kid. To review:

This extended metaphor can be extended even further as necessary and prudent. Actually, it's probably unnecessary and imprudent, but I'll extend it anyway, because I find the metaphor useful. Perl, my fifth child, is showing various signs that she is about to grow up, and as a pseudo-parent, that makes me pseudo-proud of her. But there are other ways the metaphor makes me happy. For instance, it gives me another argument about the name of Perl 6.

From time to time, people have suggested that Perl 6 is sufficiently different from Perl 5 that she should be given a new name. But we don't usually rename our kids when they grow up. They may choose to rename themselves, of course. For the moment I think Perl would like her name to stay Perl.

Now, I know what some of you are thinking: in anthropomorphizing Perl this way, Larry has gone completely off the deep end. That's not possible--I started out by jumping off the deep end, and I haven't noticed the water getting any shallower lately.

But in justification of my metaphor, let me just say that when I say "Perl" here, I'm not just talking about the language, but the entire culture. There are a lot of people who worked hard to raise Perl up to where she is today, and a bunch more people working hard to send her off to college. It's the collective aspirations of those people that is the real personality of Perl.

When we first announced the Perl 6 effort back in 2000, we said it would be the community redesign of Perl. That continues to be the case today. It may look like I'm making all these arbitrary decisions as the language designer, but as with a teenager, you somehow end up making most of your decisions consistent with what they want. With what the Perl community wants, in this case.

If a teenager doesn't want to listen to you, you can't make 'em.

The fact is, Perl would be nothing without the people around her. Here's a new acronym:

or if you like:

It really helps to have an extended family to raise a kid well. American culture has been somewhat reductionist in this respect, but a lot of other cultures around the world understand the importance of extended family. Maybe it's just because Americans move around so much. But it's a healthy trend that young people these days are manufacturing their own extended families. At the church I go to, we call it "Doing Life Together." Here in the extended Perl family, we're doing life together too.

We have people in our family like Uncle Chip and Aunt Audrey. There's Cousin Allison, and Cousin Ingy, and Cousin Uri, and our very own Evil Brother Damian. I think Randal occasionally enjoys being the honorary black sheep of the family, as it were.

It all kind of reminds me of the Addams family. Hmm.

I watched The Addams Family a lot when I was young. Maybe you should call me Gomez, and call Gloria, Morticia. I must confess that I do love it when my wife speaks French. It gives me déjà vu all up and down my spine.

It's okay for me to tell you that because I live in a fishbowl.

I'm not sure who gets to be Lurch. Or Thing. Anybody wanna volunteer? We're always looking for volunteers in the Perl community. Don't be scared. The Addams family can be a little scary, and so can the Perl family, but you'll notice we're also affectionate and accepting. In a ghoulish sort of way.

We could take this TV family metaphor a lot further, but fortunately for you I never watched the Partridge Family or The Brady Bunch or All in the Family or Father Knows Best. Those of you who were here before know I mostly watched The Man From U.N.C.L.E.

I also watched Combat, a World War II show. But I was kind of a gruesome little kid that way.

I like gruesome shows. Maybe that explains why I liked the Addams family. Hmm. I once sat on the lap of the Santa Claus at Sears and asked for all five toy machine guns listed in the Sears catalog that year. For some reason I didn't get any of them. But I suppose my family loved me in spite of my faults. My role models in parenting obviously didn't come from TV. Or maybe they did. You know, that would explain a lot about how my family turned out. In actual fact, the picture above is another self-portrait done by my daughter Geneva.

Anyway, I love my own family, even if they're kind of peculiar at times. Last month we were staying at a Motel 6 in Medford, Oregon. Gloria kindly went off to fetch me a cup of coffee from the motel lobby, and then she came to this door and stood there for a while wondering how to pull the door open with her hands full. Then she realized that the door must have been designed by someone who thinks there should be only one obvious way to do it. Because, the fact is, you can either pull or push this door, despite what it says. I suggested we should start marking such pushmepullyu doors with a P*. We obviously need more globs in real life.

Anyway, back to my weird family--this summer as we were driving around, we had a great literary discussion about how Tolstoy debunks the Great Man theory of history in War and Peace. After discussing the far-too-heavily overloaded namespace in Russian novels and the almost complete absence of names in the Tale of Genji, we tried to decide if the Tale of Genji was the first novel or not, and decided that it was really the first soap opera. Of course, then there had to be a long discussion of what really was the first novel--Tale of Genji, Madame Bovary, or Sense and Sensibility. Then there's the first romance, first mystery, first fantasy, first science fiction, first modern novel, etc. One interesting fact we noted was that the first in a genre almost always has to officially be some other genre too. For example, the Tale of Genji was written in the existing form of explication of some haiku. Transitional forms are important in biological evolution as well, as one species learns to become another species. That's why we explicitly allow people to program babytalk in Perl. The only way to become smart is to be stupid first. Puts a new spin on the Great Man theory of history.

So then, as we were driving we saw a cloud formation resembling Thomas Jefferson, which led us to speculate on the Great Documents theory of history. "Liberty, Equality, Fraternity" brought up the Great Slogans theory of history.

Back to Tolstoy: "Moscow didn't burn because Napoleon decided to burn it. Moscow burned because it was made of wood." Those of you who attended YAPC Chicago may recognize that as the Great Cow theory of history. Or maybe the lantern was really kicked over by a camel, and there was a coverup.

Anyway, back to the family again, presuming the house hasn't burned down. They say that "A family is where, when you have to go there, they have to take you in." Arguably, regardless of your viewpoint, many people have been, um, taken in by Perl culture.

Sorry. I have a low taste for taking people in with puns.

But hey, taking people in is good. And stray kitties.

Some families just naturally accumulate strays. My wife and I were both fortunate enough to grow up in families that took in strays as a matter of course. We have a number of honorary members of our own family. I think a good family tends to Borg people who need to be taken in. It's a lot like the way Audrey hands out commit bits to Pugs left and right. It all one big happy hivemind. Er, I mean family.

Now, it's all well and good to get people in the door, but that's only the beginning of accessibility. Whenever you get someone new in the family, either by birth or by adoption, where do you go from there? You have to raise your kids somehow, and they're all different. Raising different kids requires different approaches, just like computer problems do.

So, then, how do we raise a family according to the various computing paradigms?

Imperative programming is the Father Knows Best approach. It only works at all when Father does know best, which is not all that often. Often Mother knows "bester" than Father. Hi, Gloria. And a surprising amount of the time, it's the kids who know "bestest."

For some reason the Von Trapp family comes to mind. I guess you have to structure your family to make the Sound of Music together.

"Look, if you hit your sister, she will hit you back. Duh."

Obviously anyone who doesn't program their family functionally has a dysfunctional family. But what does it mean to have a functional family? "Being hit back is a function of whether you hit your sister." On the surface, everything appears to be free of side effects. Certainly when I tell my kids to mind their manners it often seems to have no lasting effect. Really, though, it does, but in the typical family, there's a lot of hidden state change wound in the call stack. We first learn lazy evaluation in the family.

"Don't take the last piece of candy unless you really want it."
"Please define whether you really care, and exactly how much you care."
"I'm sure I care more than you do."

That's almost a direct quote from Heidi when she was young: "But I want it more than you do."

Functional programming tends to merge into declarative programming in general. I married into a family where you have to declare whether you want the last piece of cheesecake, or you're unlikely to get it.

Unfortunately, I grew up in more of a culture where it was everyone's responsibility to let someone else have the cheesecake. This algorithm did not always terminate. After several rounds of, "No, you go ahead and take it, no you take it, no you take it ..."

In the end, nobody was really sure who wanted the cheesecake. I guess you say it was a form of starvation. But when I married into my wife's family I found out that I definitely wouldn't get the cheesecake until I learned to predeclare.

Let's see, inheritance is obviously important, or you wouldn't have a family in the first place. On the other hand, the family is where culture is handed down in the form of design patterns. A good model of composition is important--a lot of the work of being a family consists of just trying to stay in one spot together. As a form of composition, we learn how to combine our traits constructively by playing various roles in the family. Sometimes those are fixed roles built at family composition time, and sometimes those are temporary roles that are mixed in at run time. Delegation is also important. I frequently delegate to my sons: "Lewis, take the trash out."

That's Design By Contract. "Keep your promises, young man!"

Metaprogramming. "Takes one to know one!"

Aspected-oriented programming comes up when we teach our kids to evaluate their methods in the broader context of society:

"Okay kid, now that you've passed your driver's test, you still have to believe the stop signs, but when the speed limit sign says 65, what it really means is that you should try to keep it under 70. Or when you're in Los Angeles, under 80."

But I think the basic Perl paradigm is "Whatever-oriented programming."

Your kid comes to you and says, "Can I borrow the car?"

You say: "May I borrow the car?"

They say: "Whatever ..."

Should I push the door or pull it?

Actually, "whatever" is such an important concept that we built it into Perl 6. This is read, "from one to whatever."

You might ask why we can't just say "from one to infinity":

The problem is that not all operators operate on numbers:

Not all operators are ranges. Here's the sibling argument operator, which repeats the same words an arbitrary number of times:

Perl has always been about letting you care about the things you want to care about, while not caring about the things you don't want to care about, or that maybe you're not quite ready to care about yet. That's how Perl achieves both its accessibility and its power. We've just baked more of that "who cares?" philosophy into Perl 6.

A couple of years ago, Tim O'Reilly asked me what great problem Perl 6 was being designed to solve. This question always just sat in my brain sideways because, apart from Perl 0, I have never thought of Perl as the solution to any one particular problem. If there's a particular problem that Perl is trying to solve, it's the basic fact that all programming languages suck. Sort of the concept of original sin, applied to programming languages.

As parents, to the extent that we can influence the design of our kids, we design our kids to be creative, not to solve a particular problem. About as close as we get to that is to hope the kid takes over the family business, and we all know how often that sort of coercion works.

No, instead, we design our kids to be ready to solve problems, by helping them learn to be creative, to be socially aware, to know how to use tools, and maybe even how to manufacture the tools for living when they're missing. They should be prepared to do ... whatever.

Trouble is, it takes a long time to make an adult, on the order of 20 years. Most insects don't take 20 years to mature.

Apparently it takes you ten years to become an expert in being a kid, and then another ten years to become an expert in not being a kid. Some people never manage the second part at all, or have a strange idea of adulthood. Some people think that adulthood is when you just bake all your learning into hardware and don't learn anything new ever again, except maybe a few baseball scores. That's an oversimplified view of reality, much like building a hardwired Lisp machine. Neoteny is good in moderation. We have to be lifelong learners to really be adults, I think.

No, adulthood is really more about mature judgment. I think an adult is basically someone who knows when to care, and how to figure out when they should care when they don't know offhand. A teenager is forever caring about things the parents think are unimportant, and not caring about things the parents think are important. Well, hopefully not forever. That's the point. But it's certainly a long process, with both kids and programming languages.

In computer science, it is said that premature optimization is the root of all evil. The same is true in the family. In parenting terms, you pick your battlefields, and learn not to care so much about secondary objectives. If you can't modulate what you care about, you're not really ready to parent a teenager. Teenagers have a way of finding your hot buttons and pushing them just to distract you from the important issues. So, don't get distracted.

There are elements of the Perl community that like to push our collective hot buttons. Most of them go by the first name of Anonymous, because they don't really want to stand up for their own opinions. The naysayers could even be right: we may certainly fail in what we're trying to do with Perl 6, but I'd just like to point out that only those people who put their name behind their opinions are allowed to say "I told you so." Anonymous cowards like the "told you so" part as long as it doesn't include the "I." Anonymous cowards don't have an "I," by definition.

Anyway, don't let the teenage trolls distract you from the real issues.

As parents we're setting up some minimum expectations for civilized behavior. Perl should have good manners by default.

Perl should be wary of strangers.

But Perl should be helpful to strangers.

While we're working on their weaknesses, we also have to encourage our kids to develop where they have strengths, even if that makes them not like everyone else. It's okay to be a little weird.

Every kid is different. At least, all my kids are really different. From each other, I mean. Well, and the other way too.

I guess my kids are all alike in one way. None of them is biddable. They're all arguers and will happily debate the merits of any idea presented to them whether it needs further discussion or not. They're certainly unlikely to simply wander off to the slaughter with any stranger that suggests it.

This is the natural result of letting them fight as siblings, with supervision. It's inevitable that siblings will squabble. Your job as parent is to make sure they fight fair. It helps a lot if the parents have already learned how to fight fair. What I mean by fight fair is that you fight about what you're fighting about--you don't fight the other person. If you find yourself dragging all sorts of old baggage into an argument, then you're fighting the person, you're not fighting about something anymore. Nothing makes me happier as a parent than to hear one of my kids make a logical argument at the same time as they're completely pissed off.

If you teach your kids to argue effectively, they'll be resistant to peer pressure. You can't be too careful here. There are a lotta computer languages out there doing drugs. As a parent, you don't get into a barricade situation and isolate your kids from the outside world forever. Moving out and building other relationships is a natural process, but it needs some supervision.

Perl is learning to care deeply about things like:

This final point is crucial, if you want to understand the state of Perl today. Perl 6 is all about reconciling the supposedly irreconcilable.

Reconciling the seemingly irreconcilable is part of why Perl 6 taking so long. We want to understand the various tensions that have surfaced as people have tried to use and extend Perl 5. In fact, just as Perl 1 was an attempt to digest Unix Culture down into something more coherent, you can view Perl 6 as an attempt to digest CPAN down into something more coherent. Here are some of the irreconcilables we run into when we do that:

OO brings us a world of choices:

Do we even have classes at all?

And if we do, how do they inherit and dispatch?

Is our type system more general than our class system?

Plus a grab bag of other issues:

And finally, the biggie:

Reconciling these known conflicts is all well and good, but our goal as a parent must be a bit larger than that.

Just as a child that leaves the house today will face unpredictable challenges tomorrow, the programming languages of the future will have to reconcile not only the conflicting ideas we know about today, but also the conflicting ideas that we haven't even thought of yet.

We don't know how to do that. Nobody knows how to do that, because nobody is smart enough. Some people pretend to be smart enough. That's not something I care about.

Nevertheless, a lot of smart people are really excited about Perl 6 because, as we go about teaching Perl how to reconcile the current crop of irreconcilables, we're also hoping to teach Perl strategies for how to cope with future irreconcilables. It's our vision that Perl can learn to care about what future generations will care about, and not to care about what they don't care about.

That's pretty abstruse, I'll admit. Future-proofing your children is hard. Some of us get excited by the long-term potential of our kids. But it's also exciting when you see their day-to-day progress. And we've make a lot of progress recently.

In terms of Audrey's Perl 6 timeline, we're right at that spot where it says "hack, hack, hack." In a year or so we'll be up here saying, "What's the big deal?"

This is the year that Perl 6 will finally be bootstrapped in Perl 6, one way or another. Actually, make that one way and another. There are several approaches being pursued currently, in a kind of flooding algorithm. One or another of those approaches is bound to work eventually.

Now, anyone who has been following along at home knows that we never, ever promise a delivery date for Perl 6. Nevertheless, I can point out that many of us hope to have most of a Perl 6 parser written in Perl 6 by this Christmas. The only big question is which VM it will compile down to first. There's a bit of a friendly race between the different implementations, but that's healthy, since they're all aiming to support the same language.

So one of the exciting things that happened very recently is that the Pugs test suite was freed from its Haskell implementation and made available for all the other implementations to test against. There are already roughly 12,000 tests in the test suite, with more coming every day. The Haskell implementation is, of course, the furthest along in terms of passing tests, but the other approaches are already starting to pass the various basic sanity tests, and as many of you know, getting the first test to pass is already a large part of the work.

So the plan is for Perl 6 to run consistently on a number of platforms. We suspect that eventually the Parrot platform is likely to be the most efficient way to run Perl 6, and may well be the best way to achieve interoperability with other dynamic languages, especially if Parrot can be embedded whole in other platforms.

But the other virtual machines out there each have their own advantages. The Haskell implementation may well turn out to be the most acceptable to academia, and the best reference implementation for semantics, since Haskell is so picky. JavaScript is already ubiquitous in the browsers. There are various ideas for how to host Perl 6 on top of other VMs as well. Whatever.

But the VM that works the best for Perl right now is, in fact, Perl 5. We've already bootstrapped much of a Perl 5 compiler for Perl 6. Here's a picture of the approach of layering Perl 6 on Perl 5.

Here in the middle we have the Great Moose theory of history.

Other stuff that's going on:

In addition to lots of testing and documentation projects, I'm very happy that Sage La Torra is working on a P5-to-P6 translator for the Google Summer of Code. Soon we'll be able to take Perl 5 code, translate it to Perl 6, and then translate it back to Perl 5 to see how well we did.

Another bootstrapping approach is to take our current Haskell codebase and translate to Perl 6. That could be very important long term in keeping all the various implementations in sync.

There are many, many other exciting things going on all the time. Hang out on the mailing lists and on the IRC channels to find out more.

If you care.

Perl is growing up, but she's doing so in a healthy way, I think. Those of us who are parents tend to try to discourage our kids from getting married too young, because we know how much people change around their twentieth year. Around the age of 19 or 20 is when we start that last major rewiring of our brains to become adults. This year, Perl will be 19 going on 20. She's due for a brain rewiring.

In previous years, Perl was just trying to act grownup by ignoring her past. This year, I'm happy to report that instead of just trying to act grownup, Perl is going back and reintegrating her personality to include the positive aspects of childhood and adolescence. I don't know where Perl will go in the next ten or twenty years. It's my job to say, "I don't care anymore," and kick her out of the house. She's a big girl now, and she's becoming graceful and smart and wise, and she can just decide her future for herself.

Whatever. Thanks for listening, and for learning to care, and for learning to not care. Have a great conference! I don't care how!

The State of the Onion 9

figure 1

For the last couple of years, we've been homeschooling our two youngest kids. Gloria has been making sure they learn the easy subjects like history and mathematics. I've been making sure they also learn the hard subjects like, um, cinematography. So I've been making sure they view some of the great classics.

figure 2
James Bond materials copyright 1962 - 2005 United Artists Corporation and Danjac, LLC

Home schooling works best if the parents learn alongside the children, so I've been forced to watch the Bond corpus along with my kids.

figure 3

Or is that the Bond corpses? Seems like there are an awful lot of them.

figure 4

Anyway, it's a large body of work.

Though some of the bodies are larger than others. If you know what I mean, and I think you do.

Anyway, now that I've been wading through the Bond corpus again, I've noticed something I've never noticed before about the show. It's just not terribly realistic. I mean, come on, who would ever name an organization "SPECTRE?" Good names are important, especially for bad guys. A name like SPECTRE is just too obvious. SPECTRE. Boo! Whooo!! Run away.

You know, if I were going to name an evil programming language, I certainly wouldn't name it after a snake. Python! Run away, run away.

When I was young I actually preferred Man from U.N.C.L.E.

figure 5

Now "THRUSH," that's is a decent name for an evil organization.

Oh, and then there's Get Smart.

figure 6

Unlike James Bond, it's highly realistic. I can believe in an evil organization with a name like "KAOS." After we're done with the James Bond series, I plan to show my kids Get Smart. I want to make sure my kids score high on intelligence tests. Ba dump bump.

I'm a child of the Cold War. We didn't go quite as far as to build a bomb shelter, but we actually thought about it before deciding our house would probably burn up anyway. Back then people thought you could win a nuclear war, or at least try real hard not to lose one. Eventually we all figured out that imperfect knowledge was a feature, and so we settled on a national policy of Fear, Uncertainty, and Doubt, though they couldn't bring themselves to call it the FUD doctrine, so they called it MAD instead. Probably because they read too much MAD magazine. Hmm, that puts a whole new twist on the "Spy vs. Spy" comics.

figure 7

Anyway, as a child of the Cold War, I know that seeing a mushroom cloud is a good thing, since it means you haven't been vaporized just yet. Sort of the same principle that you should never be scared of thunder, since the shocking part is already over with. Or more subtly, if you live under the flight path of several airports, like me, you're always wondering if the Blue Angels going 100 feet overhead are going to run into your house. But as soon as you hear the Doppler shift dropping in pitch, you know that they're probably going to miss your house, because if they were on a collision course with your house, the pitch would stay the same until impact. As I said, that's one's subtle.

Of course, you should also plan ahead. Where there's one plane, there's likely to be another one coming along after it. And lightning does strike twice in the same place. And, if you see a mushroom cloud, I would suggest that you start replanning your short-term future. And your new future plans should probably take into account not only your future but the future plans of about a million other people who just saw the same mushroom cloud and are suddenly replanning their futures.

Anyway, planning is good. Well, some planning.

Everyone my age and older knows that Five-Year Plans are bad for people, unless of course you're someone like Josef Stalin, in which case they're just bad for other people. All good Americans know that good plans come in four-year increments, because they mostly involve planning to get reelected.

I probably shouldn't point this out, but we've been planning Perl 6 for five years now.

figure 8

Comrades, here in the People's Republic, the last five years have seen great progress in the science of computer programming. In the next five years, we will not starve nearly so many programmers, except for those we are starving on purpose, and those who will starve accidentally. Comrades, our new five-year mission is to boldly go where no man has gone before! Oh wait, wrong TV show.

You might say that Perl grew out of the Cold War. I've often told the story about how Perl was invented at a secret lab that was working on a secret NSA project, so I won't repeat that here, since it's no secret. Some of you have heard the part about my looking for a good name for Perl, and scanning through /usr/dict/words for every three- and four-letter word with positive connotations. Though offhand, I can't explain how I missed seeing Ruby. So anyway, I ended up with "Pearl" instead.

But it's a little known fact that one of the three-letter names I considered for quite a while was the word "spy." Now, those of you who took in Damian's session on Presentation Aikido are now wondering whether I'm just making this up to make this speech more interesting. And in this particular case, I'm not. You can ask my brother-in-law, who was there. On the other hand, please don't ask him to vouch for anything else in this speech.

But wouldn't "Spy" be a great name to give to a language whose purpose was pattern matching and reporting? Hmm. And spies are also called "agents of change." "Practical extractions are one of our specialties."

Instead of a warn operator, it'd have to be the warn off operator. Instead of having a die operator, we might have had the let die operator. Then we'd get Perl poetry, I mean, Spy poetry, with phrases like live or let die.

How history might have been different! Those of you who are Perl programmers might instead be attending the 9th annual Spy conference. And maybe Ruby would not have been named Ruby, but instead have been named Spook, or Agent. And Python might not have been named after Monty Python, but after some other comedy troupe. It might be called Stooge, or Muppet, or something.

Even if you're not a child of the Cold War, you've been affected. My kids have been affected. Lately my son Aron and my daughter Geneva have been designing a game. No, not a computer game. It's a kind of a board game involving spies and cool gadgets. It's not done yet, so don't pester them over when it's going to be done. At least, don't pester them any more than you pester me about Perl 6. Heh, heh.

But as soon as I saw their cast of characters, I knew I had my theme for this year's talk. For some reason, any time I see a really diverse set of characters, I think of the open source community in general, and the Perl community in specific.

So if you're new to the department, I'd like to introduce you to a few of the other spooks working for the organization.

figure 9

The cards for this game list various stats for each character's strengths and weaknesses, as well as their favorite gadget. Now, as it happens, these spies all happen to be Perl programmers as well, so in this talk maybe we'll get to see what their favorite Perl gadgets are.

figure 10

Ace is the quintessential spy. All the fake spies like James Bond are based on him. And like Bond, Ace is such a good spy that he doesn't care if everyone knows he's a spy. Ace knows he's the hero of the story, and therefore invincible.

So Ace does spying simply because it's fun. That's also why he does Perl programming. He knows it's all just a big game, and it's fun to win with the hand you are dealt. On the other hand, it's also fun to win by changing the rules. It's especially fun if you can win using someone else's money.

You know, when you think about it, most open source software is written using someone else's money. Most Perl programmers are not paid directly to hack on Perl, or Pugs, or on CPAN modules. A lucky few are paid to have fun, but most of us have to make a living some other way, and our bosses kindly let us spend part of our time working on things that are mutually beneficial to the organization and to the world in general. And if we have fun doing that, they don't seem to mind.

But on the flip side, the Aces of the world all seem to know how to create fun wherever they go, or at least they know how to go places where people know how to have fun. For this reason, Ace is one of the most important people in the Perl community.

This last year, we were starting to lose our sense of fun in the Perl community. Though we tried to be careful about not making promises, everyone knew in their hearts that five years is an awfully long time to wait for anything. People were getting tired and discouraged and a little bit dreary.

Then Autrijus Tang showed up. Maybe we should call him "Ace" Tang. He basically said, "Look, we'll never get this done unless we optimize for fun." So fun is exactly what the Pugs project is optimized for. Mind you, Autrijus's idea of fun is to learn Haskell and then write a prototype of Perl 6 in it. Now, for those of you who don't know, Haskell is one of those pure functional languages that doesn't allow any side effects (except, of course, when it does). Really way-out-there stuff, compared to the thinking of the average Perl programmer.

Furthermore, Autrijus thinks it's fun to persuade other functional programmers that it would fun to bootstrap Perl 6 in Haskell. These folks proudly call themselves the "lambdacamels."

But when this happened, a few skeptical people in the Perl community thought they knew what was going to happen next. The cabal would say that this was just too crazy, and we already had Parrot, and why duplicate effort. You know, your basic turf-protection reaction.

But that's not what happened. Instead, the cabal said, "Yay. We can work this problem from both ends now. Let's give Autrijus all the help we can. Parrot will work bottom-up, and Pugs will work top-down."

Did I say "work"? I meant "play." That's what Ace does best.

figure 11

But not everyone is an Ace. Some people are naturally sneaky. Every organization needs a second-story man, and people like The Cat prefer to have their fun in private. I strongly suspect certain Perl programmers of being retired jewel thieves. You watch the version tree, and things mysteriously disappear in one place and appear somewhere else. Like a real cat, Le Chat's ego is not involved in any kind of public way. That's not to say that cats don't have egos--just that they don't care whether you notice.

Cats know how to get into and out of places they're not supposed to be able to get into and out of. Cats seem to know how to levitate, and to pass through supposedly impermeable barriers. Even the stupidest cat knows how to make you think they're reading your mind, but it's all a trick.

In Perl culture, Cats also do sneaky things. Sneakiness is a good quality when you're playing Perl golf, for example. Sneaky Perl programmers like to do sneaky things with overloading and with source filters. In fact, Le Chat is looking forward with glee to the day he can change the Perl 6 grammar on the fly and write yet another set of ACME modules. Oh, hi Damian--didn't see you there.

figure 12

This is Miss Engles. She's a librarian. Like a jewel thief, she also moves things from place to place, but for very different reasons. A jewel thief moves things from where they belong, while a librarian moves things to where they belong. A place for everything, and everything in its place. Miss Engles likes those aspects of Perl 6 that support literate programming, and that let her index the documentation in various ways.

Miss Engles has never cracked a smile because, oddly enough, not cracking a smile is what makes her happy. She is a librarian all the way to the bone. Or that's what she'd like you to believe.

But in fact, as we all know from the movies, librarians take off their glasses and let down their hair when they get off work, and become completely different people. Librarians instinctively understand paradigm shifts, having perused most of the history section in their spare time, not to mention a great deal of the psychology section. So nothing ever surprises a librarian, least of all themselves. If a librarian ever says "I'm shocked," you know they're being completely sarcastic.

figure 13

Eric has a problem with dyslexia, so he's never going to be a librarian. But that's okay, since he's not terribly interested in the things librarians are interested in. Now, since Miss Engles is officially interested in almost everything, that makes it a little tough for people like Eric, since it forces him to be interested in almost nothing. But that's okay--give him a fishing pole and a tent, and he's happy. Oh, he wouldn't mind a suitcase nuke, either.

Perl culture is full of easy-going, straightforward people. Actually, now that I look at it, this looks like Eric O'Reilly, long-lost cousin to Tim O'Reilly. Just kidding. But Tim has always been a straight dealer, and so is Eric, in his own way. I think with Tim it's a matter of choice, but with Eric--well, that's just the way he is.

Eric is the sort of agent you send skiing over the mountain to count enemy soldiers. Pick your term: he's a trooper, or a SEAL, or a Marine. You know he'll almost certainly come back alive, eventually, but you don't quite know in advance whether he'll have to kill all the enemy soldiers in order to count them. What you do know is that if he does, he certainly won't hold it against any of them. Nothing personal. That's just the way it is.

And when he reports in, you'll just get the facts, without much interpretation. Eric doesn't put many comments in his code. He thinks that if you have to comment a piece of code, you haven't written it clearly enough. He is looking forward to working in Perl 6 because a lot of the magical cruft has been cleaned out of the language, or at least moved into places where he doesn't have to worry about it, such as function signatures.

But the thing he loves most about Perl 6 is the multimethod dispatch, precisely because those crufty signatures also allow him to say what he wants without extra words.

figure 14

Jezebel isn't really a bad girl. She's just drawn that way.

For all we know, this might be Miss Engles on her day off.

That being said, I wouldn't mind it if there more female programmers, especially female Perl programmers. And no, I don't mean it like that, or my wife wouldn't let me say it. But I think we need some spies to tell us what things in our culture appeal to women, and what don't. And it kinda goes without saying that these spies need to be women. Well, look, the guys all have a lot of great ideas, but you know, guys tend to be rather, well, idea-oriented. In theory, Perl culture is supposed to be more cooperative than competitive, but it's kind of hard to argue for that viewpoint when the vast majority of us are standing and pounding our chests like big gorillas. I include myself in that category. Er, the gorilla category, not the Jezebel category. Just thought I'd clear that up.

figure 15

I think Mama represents the older generation of Perl programmers. Mama is wise to all the stupid tricks of the young'uns, and not afraid to tell 'em off for it. Mama tends to be strict, but she has good reasons for it, because she takes the long view.

A lot of Mamas must hang out on PerlMonks--they're the ones who are always saying "use strict; use warnings;" or you'll grow up to be sorry you didn't. And wipe your feet when you come in.

Mama kinda likes the fact that Perl 6 is growing up to become a strict language by default, but she isn't quite sure what she's gonna do after the kids are all grown up. It makes her happy and sad at the same time.

figure 16

Natasha gets to represent the next wave of international programmers, particularly those from developing countries. Natasha's mother probably worked for a large eastern-bloc TLA, and I don't mean IBM. But Natasha is actually doing a bit of industrial work here. All in the interests of capitalism, of course. Though whose capital is perhaps a bit unclear at times.

A lot of people like Natasha are trying to figure out how to make a living in the new economic realities, and one of those economic realities is that the traditional western powers are trying to vacuum up all intellectual property rights on behalf of various corporate interests. And she wonders, rightfully so, if there will ever be any place in that economy for her, and for people like her.

So when our friend "Mad Dog" comes to town, and preaches the gospel that free and open source software is the only path to freedom, the only way for the rest of the world to push back--well, she can see the appeal of that notion.

And there are a lot of Natashas in the world, and potential Natashas. Over the long haul, she may well be the most important member of this list.

figure 17

Oliver represents the next generation of programmers, who don't even know it yet. Oliver is not ready to learn computer science. He is just starting to think about programming because he'd like to be a video game designer someday. He doesn't know that what he likes best about Perl is that it will let him learn what he needs to know one concept at a time, without forcing him to learn a bunch of abstract concepts all at once before he really needs them. When he drives on the freeway, he drives in the slow lane.

figure 18

On the other hand, Pixie drives in the fast lane. She is what you might call an extreme programmer.

As an extreme programmer, Pixie loves testing. Preferably testing to destruction. The best defense is a good offense. She'll get the job done, but she's determined to have a lot of fun doing it.

Pixie works well in small teams, especially with pair programming. A team of two is the perfect size for her because wherever she's aiming, her partner can always stand on the opposite side of her.

As part of a rapid response unit, Pixie is very much into rapid prototyping. So Pixie's favorite Perl 6 feature is the yadayadayada operator, which lets her stub out new functions and blow up their interfaces even before the function bodies are written.

figure 19

As the quintessential English banker, Mr. Radcliffe is a firm believer in reliability, with a dash of style. For our talk today, he gets to represent the business interests surrounding open source software. Mr. Radcliffe knows that businesses have different set of goals than most open source programmers, but he also knows that there is a great deal of overlap in those goals, and that the clever businessman can exploit that overlap to the betterment of both business and programmer.

You see, Mr. Radcliffe understands that one thing can have multiple functions. His umbrella is almost certainly multifunctional. Mr. Radcliffe's favorite part of Perl 6 is that nearly every feature is multifunctional, though not completely orthogonal. That doesn't bother Mr. Radcliffe, because nobody who rides cabs around in London expects complete orthogonality. He just expects to get where he's going.

figure 20

What can I say? Perl is also used by script kiddies. We just hope Oliver doesn't grow up to be a script kiddie like r0u73r. Or if he does, we hope it's just a passing phase.

And in fact, r0u73r used to be a script kiddie, but now uses his 1337 skills for good. To some extent, most of us were cargo culters as we learned how to program. We were reusing code, which is good, but we just didn't always understand why we were reusing the code. But the Perl community has always had a soft spot for cargo culters, and seeks to educate them until they learn the real reasons for things being the way they are. Then they're ready to join the real cult. Er, I mean, the real culture.

Anyway, as a vestige of his former ways, r0u73r looks forward to using the introspection capabilities of Perl 6, particularly when he can introspect someone else's data structures.

figure 21

Tina seems like a girl who just wants to have fun, but she's really aspiring to be Mata Hari, except for the part about getting caught and executed. As a dancer, and perhaps an actress someday, Tina understands about playing roles. She knows that the role she's currently playing is not who she really is. In Perl 6 terms, she understands the difference between an "isa" relationship and a "does" relationship. So she's very much into the Perl 6 concepts of roles, traits, properties, and mixins.

figure 22

Wheelbarrow is a scavenger. That is to say, he's a sysadmin.

Wheelbarrow loves to sift through the trash, which in his case consists primarily of discarded HTTP logs. Wheelbarrow loves the strong pattern-matching skills of Perl, and wonders how much better he'll be at scavenging useful information with Perl 6 rules.

As a strong believer in ecology, Wheelbarrow also loves CPAN, and hopes that Perl 6's version of CPAN will be even better at helping him reduce, reuse, and recycle.

figure 23

Wraith has trained herself to be good at hiding. She's particularly good at information hiding and various forms of encapsulation.

Wraith also loves all the crazy new Perl 6 operators, especially the ones that will allow her to express parallel operations implicitly. She is willing to train herself in their skillful use because she values efficiency of expression. Why toss a single throwing star when you can throw eight at once?

figure 24

Wu-Li is the old guy who is interested in Unicode, and is looking forward to Perl 6's built-in support for efficient and ubiquitous Unicode processing. Wu-Li likes to think about the various positive and negative aspects of various strange philosophies, whether Eastern, Western, Northern, Southern, or somewhere in-between.

Wu-Li isn't actually Chinese. He only thinks he's Chinese because when he was young his parents told him that every third child born into the world was Chinese, and he was a third child.

As a foreteller of the future, Wu-Li is the only person who knows when Perl 6 is coming out, and I'm not telling. Er, he's not telling.

Wu-Li is also known for giving peculiar speeches from time to time.

figure 25

And finally we come to Mr. X. I don't know much about Mr. X, because nobody knows anything about Mr. X except Mr. X himself. We can only guess.

Mr. X seems to be highly placed in his organization, because all his information is of high quality, and of strategic value. Mr. X seems to believe in clean interfaces, with no extraneous information. He's a master of the deaddrop, and other forms of message passing. He seems to understand Perl 6's concept of delegation, so he's probably in management. Perhaps he's the CIO. Or maybe he's the CFO's assistant. Who knows?

In any event, he may be someone with some decision-making power, but he can't afford to compromise his position by being overtly in favor of open source. At least, not yet. Nevertheless, he may be the most important player in the eventual success of open source. Mr. X is future-oriented, and as enigmatic as the future itself. We sincerely hope he turns out to be a nice person.

figure 26

That's our organization in a nutshell. We sincerely hope you'll join up. Unfortunately, if you don't join, we'll have to liquidate you.

Well, enough of that. If this were an ordinary State of the Onion speech, I would now go into my standard spiel about how diverse the open source community is and how it's such a great thing that we can pool our various strengths and produce something greater than any of us can do alone. And if this conference were still in California, I might say it again anyway, since diversity in California is not just encouraged, it's actually required, culturally speaking. Californians have gotten to the point of being completely intolerant of non-diversity. But we're not in California, so let's just assume I said all that again and go on to something else.

I'd like to leave you with one thought, along with all these pretty pictures.

As I was thinking about the intelligence community and its recent obvious failures, it kinda put a new spin onto the phrase, "Information wants to be free," or my own version of it, which is that "Information wants to be useful."

We often think that intelligence failures are caused by having too little information. But often, in retrospect, we find that the problem is too much information, and that in fact, we had the data available to us, if only it had been analyzed correctly.

So I'm just wondering if we're getting ourselves into a similar situation with open source software. More software is not always better software. Google notwithstanding, I think it's actually getting harder and harder over time to find that nugget you're looking for. This process of re-inventing the wheel makes better wheels, but we're running the risk of getting buried under a lot of half-built wheels.

And there are two take-home lessons from that. The first is that, as an open source author, you should be quick to try to make someone else's half-built wheel better, and slow to try to make your own. We're making progress in this realm in the Perl community, but I don't think any open source community ever gets good enough at harmonizing the dissimilar interests that sometimes lead to project forks. We can always improve there.

The second take-home lesson is this. Pity your poor intelligence analyst back at headquarters. He's not all that intelligent, after all. The intelligence of the intelligence community is distributed, and it's often the Tinas and the Wheelbarrows of the world that know when they've got a piece of hot information. But somehow that meta-information gets lost on transmission back to headquarters.

So my plea to all you agents out there today is to use your own initiative in figuring out which things to bother us with, and which things to work out for yourself. You're smart, and the worst that can happen is that we tell you that you've wasted some effort. Just think of it as a kind of commit and rollback mechanism. Recent studies in multithreading show that hard locks do not scale as well as Software Transactional Memory, which is just such a commit/rollback mechanism.

Look around you. We are a multithreaded organization, so the same is true socially. It's easy to get offended or discouraged when a rollback happens, but just don't. The whole community will function more efficiently that way.

But if you get rolled back on something you know is important, just keep pushing. Those of us back at headquarters try to stay flexible and open-minded, but we don't always succeed. So keep that good intel coming in, because good analysts can change their minds occasionally, too. At least, that's what I think this week, and this year.

This talk will self-destruct in five seconds. Thank you all for listening.

figure
Visit the home of the Perl programming language: Perl.org

Sponsored by

Powered by Movable Type 5.02