December 2007 Archives

Memories of 20 Years of Perl

Proving Them Wrong

Around 1991 I wrote a very useful program, in C, which took a bunch of files and then sorted them into groups according to which files had identical contents. A lot of sysadmins at the time wrote to thank me for it. But when I boasted about it at Usenix that year, people told me "oh, you should have written then in Perl."

That was pretty annoying, so I got the Camel Book (pink in those days) so that I could learn Perl and prove that they were wrong. But it turned out that they were right.

Mark Dominus is the author of Higher-Order Perl

My First CGI Program

It was the year 2000, and I was working at a software startup in San Francisco. I was tasked with writing a simple form handler with an auto thank you email. I had been a C programmer for several years, a Fortran programmer for a few, and this was essentially my first Perl program. It was your standard CGI gateway which presented a form to the user, did some error checking, and sent a thank you email to the user.

After a few hours of learning Perl and putting my form handler together, it was put live on our website. I was delighted that I was able to pick up this language so quickly and produce results in a short period of time. I never like programming C that much (although that has changed), due the fact that it got in my way. Perl just worked.

I came into work the next day and reviewed how my program was doing. It turns out that my first bug had surfaced; the thank you email function managed to get caught in a loop. One poor soul who filled out my form had received 800 thank you emails! I was able to quickly fix the bug.

In honor of my first Perl program, I would like to extend a hearty 800 thank yous to the Perl community! I have been using Perl ever since and love it.

Fred Moyer is just another mod_perl hacker

Perl and the University Student

One cannot imagine how useful Perl proves sometimes to a university student. I can recall several occasions in which I used Perl to facilitate a task or check my homework. Of them, there is one that I still remember very clearly.

It was the course "Introduction to Computer Networks" and we learned about the various variations of networking protocols (Stop-and-wait, Go-back-N, and Selective-Repeat). We were given a simulation of these protocols written in C and compiled to run on Windows. The simulation could be ran with several parameters and would output a verbose file with the parameters of the simulation, the simulation itself and then some statistics of the simulation.

We ran the program several times and got several files in return. Now we had to somehow insert the statistics into Excel so we can analyze them, process them, and create charts out of them. But the statistics were scattered over several different files, all with the same format, but nothing that Excel can understand (at least not without a massive amount of Visual Basic for Applications code).

Without thinking for a moment, I started writing a Perl script that will process the files, extract the corresponding data and output a tab-delimited file that can be inputted into Excel. It took some time to write the script, and meanwhile my partner decided it may be faster to do it by hand. Thus, he occupied the nearby station, and started extracting the data himself. I finished a few minutes after that, though, (while he was just beginning in his manual labour) and we were able to input the data into Excel and continue the assignment. It took about 15 minutes or less, all in all.

Later on I talked to a few fellow students about the assignment. One of them claimed it took him 3 hours to input everything into Excel. (!) Another said it took him one hour, which is still much worse than 15 minutes. Needless to say, none of them knew Perl.

Enough said.

(Originally published at Perl Success Story, Israel.pm.)

Shlomi Fish has worked with Perl since 1996 and considers himself a happy user, developer and advocate of Perl and other open-source technologies.

How To Become a Guru

In early 1999 I started a new job as a system administrator. In my previous position I'd taught myself Unix and GNU/Linux, and ended up writing a small tracking application for a customer service group in Java.

As a new SA, I took over a pile of work from my predecessor, including some small Perl programs he'd downloaded, installed, and modified to add his name to the comments. Over the next couple of months, I picked up the Camel and the Perl Cookbook, and taught myself enough Perl that I could skim comp.lang.perl.moderated and answer some of the questions in my head.

About that time, I started to do a little work on the Everything Engine -- not much, but a little bit -- and so I was the second external person to register on PerlMonks when it started. In those days there was no voting, no XP, and there were just a few people racing to reach the milestone of a hundred posts.

In between troubleshooting problems at work, I'd play with little programs, read whatever tutorials or books I could get, and answer any question I could on the site, and so I learned Perl that way.

I remember the rush to find an idea -- any idea -- worthy of putting on the CPAN, and thinking in 2000 that every problem that anyone could solve, someone had already solved. I remember my first patch to Perl 5, then realizing that I hadn't actually run the tests, and resolving to improve the tests because they didn't actually do what they said they should.

I remember getting job offers from my postings, and meeting some of the top Perl programmers in the world for the first time, and being accepted because I did (some of the) things I said I would, just because no one else was doing them.

That, I think, is the secret to become a contributing member of any community. Look for something that needs someone to do it and do it. You don't have to have permission, just a little bit of determination and stubbornness and some time.

I'm a little sad that I missed the first eleven years of Perl's life, but I'm glad to have caught up in the past nine years.

chromatic does a lot of things, some of them even sometimes productive.

How an English Major Saved Christmas

Right before Christmas of 1998 I was a fairly new employee at Amazon.com. Not a CS grad hacker with 30,000 shares, but an English grad customer service rep with 250. I knew about the 29,750 share disparity from picking up a fax for a star employee in the apps group. Instead of letting it get to me, I started to look into why it was so. I bought Learning Perl and spent two of the most painful weeks of self-edification in my life discovering how the lack of chmod +x was preventing me from getting through Chapter 2.

Free at last I wrote, in two days, a badly needed and overlooked tax + shipping costs calculator for customer service for the new product tab launching that week. It was the kind of script that would take any decent Perl hacker 30 minutes. A former art critic saved hundreds of reps and tens of thousands of customers a lot of time and aggravation. I got the company's "Just Do It" Award. If it had been C or Java or anything but Perl I wouldn't have been able to do it.

If I'd come to anything but Perl, I would not have returned to coding--I dabbled in BASIC and Assembly as a kid--and I wouldn�t be a software developer today.

Ashley Pond V is a New Mexican writer turned Seattlite software developer, currently working with Catalyst applications, who credits Perl with saving his soul as he'd probably have gone into marketing otherwise.

Smells Like Wet Camel

Standing out in my memory is the day in college (either in late 1993 or early 1994) when my grandmother had emergency eye surgery. Originally, she only had a regularly scheduled checkup, and my mother could take her to the appointment before work began, but not pick her up. The doctor was one street over from the college (more or less) and I was conscripted to go over and take her home after her appointment and my first class. The day was rainy, increasing in intensity as the day grew older.

Everything changed when I arrived at the doctor's office, because the doctor had found something that required immediate attention. She had to be taken to a specialist immediately, and I began improvising. Each eye appointment took a long time, and they would only get longer as my grandmother was worked in to the specialist's schedule as an emergency patient. So I had time to take her to the next appointment, leave her to wait for what might be hours, go to my next class, eat lunch, and come back and get her.

I was trying to keep up with my college work, and brought my O'Reilly Perl book along so I could work on my computer science project, figuring I might as well do something useful while I was sitting around. My project involved writing an e-mail processing system in Perl, so I had bought what was for me at the time an almost impossibly expensive book to help me learn the language. On the way to the car, in the hardest and coldest rain I can ever remember, I was trying to help my grandmother and juggle the umbrella, car keys, car door, and everything else. The book slipped out from under my arm and landed in a puddle. Somehow, it landed on its edge, and had about an inch of muddy water soak into it. My new book! Ruined! Nothing to do but keep going, to the next appointment, and back to my class. I knew that to leave the college after eight a.m. was a guarantee of not being able to park anywhere near the building for the rest of the day, because the only parking spaces left were in the lower area of an overflow lot far from any building I needed to go to. Without even a sidewalk near this lot, I had plenty of time to think about my ruined book and what was happening to my grandmother as I trudged through the mud, in the pouring rain, to get to my next class.

I also, in these days before mobile phones, had to find a pay phone to tell my mother about the abrupt change of plans. My grandmother eventually got settled in the hospital, where it was at least dry, and she pulled through the eye surgery fine. My waterlogged book with a brown bottom and hastily scribbled notes on the blank pages in the back was a good enough starting point; I graduated.

Scott McMahan has been writing Perl code since 1991.

"I Couldn't Believe That Perl Even Worked"

My first exposure to Perl was a web server with -- I think -- Perl 4.036 installed. This would be 1995 or so. I wanted to write CGI scripts so I started reading everything I could find about Perl. I nearly lost heart when I read that the parser was, effectively, heuristic. Coming from a background in Pascal and C I couldn't believe that Perl even worked.

Fortunately Perl was the only option for my script. I persevered and discovered that -- not only did Perl work -- I rather enjoyed it. Within two weeks I had a CGI script that implemented a kind of ad-hoc PHP: chunks of Perl embedded in HTML. It was ugly -- but Perl had made it possible.

At some time between then and now -- after digressions into Java and even LotusScript -- Perl became my main language. At the end of 2006 I decided to concentrate on Perl, release some modules, proactively seek out things I didn't know about the language and learn them.

As a result 2007 has been the happiest year of my professional career. I've written loads of code, most of which works. I attended my first YAPC in Vienna and came home with a bunch of new friends and a renewed enthusiasm for cranking out code.

I've still got plenty to learn. Perl may be easy to pick up but mastery takes years. And if you love programming that's part of the fun. However good you think you are there's always a way to improve.

I dabble with other languages -- because if you take programming seriously you must. What do they know of Perl that only Perl know? There are things about Perl that grate. It's not perfect but it's, well, loveable I suppose.

Thank you Perl community. Thank you Larry. Thank you for a lovely language.

Andy Armstrong is a compulsive Perl abuser based in Cumbria, UK.

From awk to perl

In early 1990, I was working with a large set of data that needed to be massaged and formatted so that it could be statistically analyzed.

I started the task in awk, but quickly ran into trouble because awk could only open one file at a time. A quick search through the Usenet comp.lang group found Perl 3.0, which had just recently been released.

I had to get the source code and build it on my machine, but it compiled cleanly and I was able to try some simple stuff. Worked real good too. As I had already a large awk program, that I didn't want to re-edit for Perl, I ran it through a2p and the perl version produced the same results. I was hooked. When I got stuck, asking questions on comp.lang.perl almost always got instant answers. There has been an active perl community for a long time, and they were fabulous! (Just like now). I subsequently re-factored my code for perl and produced vast quantities of data to be analyzed. I have been using Perl ever since.

Roe McBurnett is a systems engineer for a telecommunications company and has been working on telephony related projects as a developer, systems engineer, and software tester since 1985.

Programming is Hard, Let's Go Scripting...

I think, to most people, scripting is a lot like obscenity. I can't define it, but I'll know it when I see it. Here are some common memes floating around:

    Simple language
    "Everything is a string"
    Rapid prototyping
    Glue language
    Process control
    Compact/concise
    Worse-is-better
    Domain specific
    "Batteries included"

...I don't see any real center here, at least in terms of technology. If I had to pick one metaphor, it'd be easy onramps. And a slow lane. Maybe even with some optional fast lanes.

Easy Onramps

But basically, scripting is not a technical term. When we call something a scripting language, we're primarily making a linguistic and cultural judgment, not a technical judgment.

I see scripting as one of the humanities. It's our linguistic roots showing through. So speaking of roots...

The Past

Suppose you went back to Ada Lovelace and asked her the difference between a script and a program. She'd probably look at you funny, then say something like: Well, a script is what you give the actors, but a program is what you give the audience. That Ada was one sharp lady...

Since her time, we seem to have gotten a bit more confused about what we mean when we say scripting. It confuses even me, and I'm supposed to be one of the experts.

So I'm afraid all I can do is give you my own worm's eye view of the past, the present, and the future. Let me warn you that I am not without a few prejudices here and there.

BASIC

Now, however it was initially intended, I think BASIC turned out to be one of the first major scripting languages, especially the extended version that DEC put onto its minicomputers called BASIC/PLUS, which happily included recursive functions with arguments. I started out as a BASIC programmer. Some people would say that I'm permanently damaged. Some people are undoubtedly right.

But I'm not going to apologize for that. All language designers have their occasional idiosyncracies. I'm just better at it than most. :-)

RSTS BASIC/PLUS

Anyway, when I was a RSTS programmer on a PDP-11, I certainly treated BASIC as a scripting language, at least in terms of rapid prototyping and process control. I'm sure it warped my brain forever. Perl's statement modifiers are straight out of BASIC/PLUS. It even had some cute sigils on the ends of its variables to distinguish string and integer from floating point.

But you could do extreme programming. In fact, I had a college buddy I did pair programming with. We took a compiler writing class together and studied all that fancy stuff from the dragon book. Then of course the professor announced we would be implementing our own language, called PL/0. After thinking about it a while, we announced that we were going to do our project in BASIC. The professor looked at us like were insane. Nobody else in the class was using BASIC. And you know what? Nobody else in the class finished their compiler either. We not only finished but added I/O extensions, and called it PL 0.5. That's rapid prototyping.

Unix?

I remember one day our computer center got a letter from Bell Labs telling us that we could get a tape of Unix V6 for cheap, only $100 because they were coming out shortly with V7. We all looked at each other and said, Why would we ever want to use this thing called Unix? We have RSTS.

JAM (no not that one)

My first scripting language was written in BASIC. For my job in the computer center I wrote a language that I called JAM, short for Jury-rigged All-purpose Meta-language. Story of my life...

JAM was an inside-out text-processing language much like PHP, except that HTML hadn't been invented yet. We mostly used it as a fancy macro processor for BASIC. Unlike PHP, it did not have 3,000 functions in one namespace. We wouldn't have had the memory, for one thing.

LISP

For good or ill, when I went off to grad school, I studied linguistics, so the only computer language I used there was LISP. It was my own personal McCarthy era.

Is LISP a candidate for a scripting language? While you can certainly write things rapidly in it, I cannot in good conscience call LISP a scripting language. By policy, LISP has never really catered to mere mortals.

And, of course, mere mortals have never really forgiven LISP for not catering to them.

Pascal, Ada

Once I got into industry, I wrote a compiler in Pascal for a discrete event simulator, and slavered over the forthcoming Ada specs. As a linguist, I don't think of Ada as a big language. Now, English and Japanese, those are big languages. Ada is just a medium-sized language.

Unix, shell

After several years I finally became acquainted with Unix and its various scripting languages. OK, to be more precise, BSD, and csh.

BSD, csh

Yeah, yeah, I know. More brain damage...

I also learned a little C.

C

That's because a little C is all there is. I'm still learning those libraries though.

shell + awk + sed + find + expr...

But the frustrations of Unix shell programming led directly to the creation of Perl, which I don't really have time to tell. But essentially, I found that shell scripting was intrinsically limited by the fact that most of its verbs are not under its control and hence largely inconsistent with each other. And the nouns are impoverished, restricted to strings and files, with who-knows-what typology.

C xor shell

More destructive was the mindset that it was a one-dimensional universe: you either programmed in C or you programmed in shell, because they're obviously at opposite ends of the One True Continuum. Perl came about when I realized that scripting did not always have to viewed as the opposite of programming, but that a single language could be pretty good for both. That opened up a huge ecological niche. Many of you have seen my old clamshell diagram, with the two dimensions of manipulexity and whipuptitude.

Tcl

After Perl came Tcl, which in a sense is a purer scripting language than Perl. Perl just pretends that everything is a string when it's convenient, but Tcl really believes that as a controlling metaphor. The string metaphor tends to have bad performance ramifications, but that's not why Tcl languished, I think. There were two reasons for that.

First, Tcl stayed in the Unix mindset that controlling tools was the opposite of creating tools, so they didn't optimize much. The fast parts can always be written in C, after all.

The second reason was the lack of a decent extension mechanism, so you ended up with separate executables for expect, incr-tcl, etc.

I must say, though, that I've always admired Tcl's delegational model of semantics. But it fell into the same trap as LISP by expecting everyone to use the One True Syntax. Speaking of the One True Syntax:

Python

After Tcl came Python, which in Guido's mind was inspired positively by ABC, but in the Python community's mind was inspired negatively by Perl. I'm not terribly qualified to talk about Python however. I don't really know much about Python. I only stole its object system for Perl 5. I have since repented.

Ruby

I'm much more qualified to talk about Ruby--that's because a great deal of Ruby's syntax is borrowed from Perl, layered over Smalltalk semantics. I've always viewed Ruby as a much closer competitor for Perls ecological niche, not just because of the borrowed ideas, but because both Perl and Ruby take their functional programming support rather more seriously that Python does. On the other hand, I think Ruby kind of screwed up on its declaration syntax, among other things.

*sh

Meanwhile, the Bourne shell was extended into the Korn shell and bash. I didn't have much to do with those either. Thankfully. I will say that the continued evolution of the shell shows just how crufty a language can get when you just keep adding on ad hoc syntactic features.

PHP

We've also seen the rise of PHP, which takes the worse-is-better approach to dazzling new depths, as it were. By and large PHP seems to be making the same progression of mistakes as early Perl did, only slower. The one thing it does better is packaging. And when I say packaging, I don't mean namespaces.

JavaScript

Then there's JavaScript, a nice clean design. It has some issues, but in the long run JavaScript might actually turn out to be a decent platform for running Perl 6 on. Pugs already has part of a backend for JavaScript, though sadly that has suffered some bitrot in the last year. I think when the new JavaScript engines come out we'll probably see renewed interest in a JavaScript backend.

Monad/PowerShell

I've looked a bit at Microsoft's Monad, and I'm pleased to note that it has object pipes like Perl 6. I just hope they don't patent it.

Lua, AppleScript

There are other scripting languages in wide use. Sadly, I must confess I never looked closely at Lua or AppleScript, probably because I'm not a game designer with a Mac.

Actually, I suspect it runs deeper than that, which brings us up to the present time.

The Present

When I look at the present situation, what I see is the various scripting communities behaving a lot like neighboring tribes in the jungle, sometimes trading, sometimes warring, but by and large just keeping out of each other's way in complacent isolation.

I tend to take an anthropological view of these things. Many of you here are Perl programmers, but some of you come from other programming tribes. And depending on your tribal history, you might think of a string as a pointer to a byte array if you're a C programmer, or as a list if you're a functional programmer, or as an object if you're a Java programmer. I view a string as a Text, with a capital T.

Text

I read that word from a postmodern perspective. Of course, the term Postmodern is itself context-sensitive. Some folks think Postmodernism means little more than the Empowerment of the Vulgar. Some folks think the same about Perl.

But I take Postmodernism to mean that a Text, whether spoken or written, is an act of communication requiring intelligence on both ends, and sometimes in the middle too. I don't want to talk to a stupid computer language. I want my computer language to understand the strings I type.

Perl is a postmodern language, and a lot of conservative folks feel like Postmodernism is a rather liberal notion. So it's rather ironic that my views on Postmodernism were primarily informed by studying linguistics and translation as taught by missionaries, specifically, the Wycliffe Bible Translators. One of the things they hammered home is that there's really no such thing as a primitive human language. By which they mean essentially that all human languages are Turing complete.

When you go out to so-called primitive tribes and analyze their languages, you find that structurally they're just about as complex as any other human language. Basically, you can say pretty much anything in any human language, if you work at it long enough. Human languages are Turing complete, as it were.

Human languages therefore differ not so much in what you can say but in what you must say. In English, you are forced to differentiate singular from plural. In Japanese, you don't have to distinguish singular from plural, but you do have to pick a specific level of politeness, taking into account not only your degree of respect for the person you're talking to, but also your degree of respect for the person or thing you're talking about.

So languages differ in what you're forced to say. Obviously, if your language forces you to say something, you can't be concise in that particular dimension using your language. Which brings us back to scripting.

How many ways are there for different scripting languages to be concise?

How many recipes for borscht are there in Russia?

Language designers have many degrees of freedom. I'd like to point out just a few of them.

early binding / late binding

Binding in this context is about exactly when you decide which routine you're going to call for a given routine name. In the early days of computing, most binding was done fairly early for efficiency reasons, either at compile time, or at the latest, at link time. You still tend to see this approach in statically typed languages. With languages like Smalltalk, however, we began to see a different trend, and these days most scripting languages are trending towards later binding. That's because scripting languages are trying to be dwimmy (Do What I Mean), and the dwimmiest decision is usually a late decision because you then have more available semantic and even pragmatic context to work with. Otherwise you have to predict the future, which is hard.

So scripting languages naturally tend to move toward an object-oriented point of view, where the binding doesn't happen 'til method dispatch time. You can still see the scars of conflict in languages like C++ and Java though. C++ makes the default method type non-virtual, so you have to say virtual explicitly to get late binding. Java has the notion of final classes, which force calls to the class to be bound at compile time, essentially. I think both of those approaches are big mistakes. Perl 6 will make different mistakes. In Perl 6 all methods are virtual by default, and only the application as a whole can tell the optimizer to finalize classes, presumably only after you know how all the classes are going to be used by all the other modules in the program.

single dispatch / multiple dispatch

In a sense, multiple dispatch is a way to delay binding even longer. You not only have to delay binding 'til you know the type of the object, but you also have to know the types of all rest of the arguments before you can pick a routine to call. Python and Ruby always do single dispatch, while Dylan does multiple dispatch. Here is one dimension in which Perl 6 forces the caller to be explicit for clarity. I think it's an important distinction for the programmer to bear in mind, because single dispatch and multiple dispatch are philosophically very different ideas, based on different metaphors.

With single-dispatch languages, you are basically sending a message to an object, and the object decides what to do with that message. With multiple dispatch languages, however, there is no privileged object. All the objects involved in the call have equal weight. So one way to look at multiple dispatch is that the objects are completely passive. But if the objects aren't deciding how to bind, who is?

Well, it's sort of a democratic thing. All the routines of a given name get together and hold a political conference. (Well, not really, but this is how the metaphor works.) Each of the routines is a delegate to the convention. All the potential candidates put their names in the hat. Then all the routines vote on who the best candidate is, and the next best, and the next best after that. And eventually the routines themselves decide what the best routine to call is.

So basically, multiple dispatch is like democracy. It's the worst way to do late binding, except for all the others.

But I really do think that's true, and likely to become truer as time goes on. I'm spending a lot of time on this multiple dispatch issue because I think programming in the large is mutating away from the command-and-control model implicit in single dispatch. I think the field of computation as a whole is moving more toward the kinds of decisions that are better made by swarms of insects or schools of fish, where no single individual is in control, but the swarm as a whole has emergent behaviors that are somehow much smarter than any of the individual components.

eager evaluation / lazy evaluation

Most languages evaluate eagerly, including Perl 5. Some languages evaluate all expressions as lazily as possible. Haskell is a good example of that. It doesn't compute anything until it is forced to. This has the advantage that you can do lots of cool things with infinite lists without running out of memory. Well, at least until someone asks the program to calculate the whole list. Then you're pretty much hosed in any language, unless you have a real Turing machine.

So anyway, in Perl 6 we're experimenting with a mixture of eager and lazy. Interestingly, the distinction maps very nicely onto Perl 5's concept of scalar context vs. list context. So in Perl 6, scalar context is eager and list context is lazy. By default, of course. You can always force a scalar to be lazy or a list to be eager if you like. But you can say things like for 1..Inf as long as your loop exits some other way a little bit before you run into infinity.

eager typology / lazy typology

Usually known as static vs. dynamic, but again there are various positions for the adjustment knob. I rather like the gradual typing approach for a number of reasons. Efficiency is one reason. People usually think of strong typing as a reason, but the main reason to put types into Perl 6 turns out not to be strong typing, but rather multiple dispatch. Remember our political convention metaphor? When the various candidates put their names in the hat, what distinguishes them? Well, each candidate has a political platform. The planks in those political platforms are the types of arguments they want to respond to. We all know politicians are only good at responding to the types of arguments they want to have...

There's another way in which Perl 6 is slightly more lazy than Perl 5. We still have the notion of contexts, but exactly when the contexts are decided has changed. In Perl 5, the compiler usually knows at compile time which arguments will be in scalar context, and which arguments will be in list context. But Perl 6 delays that decision until method binding time, which is conceptually at run time, not at compile time. This might seem like an odd thing to you, but it actually fixes a great number of things that are suboptimal in the design of Perl 5. Prototypes, for instance. And the need for explicit references. And other annoying little things like that, many of which end up as frequently asked questions.

limited structures / rich structures

Awk, Lua, and PHP all limit their composite structures to associative arrays. That has both pluses and minuses, but the fact that awk did it that way is one of the reasons that Perl does it differently, and differentiates ordered arrays from unordered hashes. I just think about them differently, and I think a lot of other people do too.

symbolic / wordy

Arguably APL is also a kind of scripting language, largely symbolic. At the other extreme we have languages that eschew punctuation in favor of words, such as AppleScript and COBOL, and to a lesser extent all the Algolish languages that use words to indicate blocks where the C-derived languages use curlies. I prefer a balanced approach here, where symbols and identifiers are each doing what theyre best at. I like it when most of the actual words are those chosen by the programmer to represent the problem at hand. I don't like to see words used for mere syntax. Such syntactic functors merely obscure the real words. That's one thing I learned when I switched from Pascal to C. Braces for blocks. It's just right visually.

Actually, there are languages that do it even worse than COBOL. I remember one Pascal variant that required your keywords to be capitalized so that they would stand out. No, no, no, no, no! You don't want your functors to stand out. It's shouting the wrong words: IF! foo THEN! bar ELSE! baz END! END! END! END!

Anyway, in Perl 6 we're raising the standard for where we use punctuation, and where we don't. We're getting rid of some of our punctuation that isn't really pulling its weight, such as parentheses around conditional expressions, and most of the punctuational variables. And we're making all the remaining punctuation work harder. Each symbol has to justify its existence according to Huffman coding.

Oddly, there's one spot where we're introducing new punctuation. After your sigil you can add a twigil, or secondary sigil. Just as a sigil tells you the basic structure of an object, a twigil tells you that a particular variable has a weird scope. This is basically an idea stolen from Ruby, which uses sigils to indicate weird scoping. But by hiding our twigils after our sigils, we get the best of both worlds, plus an extensible twigil system for weird scopes we haven't thought of yet.

We think about extensibility a lot. We think about languages we don't know how to think about yet. But leaving spaces in the grammar for new languages is kind of like reserving some of our land for national parks and national forests. Or like an archaeologist not digging up half the archaeological site because we know our descendants will have even better analytical tools than we have.

Really designing a language for the future involves a great deal of humility. As with science, you have to assume that, over the long term, a great deal of what you think is true will turn out not to be quite the case. On the other hand, if you don't make your best guess now, you're not really doing science either. In retrospect, we know APL had too many strange symbols. But we wouldn't be as sure about that if APL hadn't tried it first.

compile time / run time

Many dynamic languages can eval code at run time. Perl also takes it the other direction and runs a lot of code at compile time. This can get messy with operational definitions. You don't want to be doing much file I/O in your BEGIN blocks, for instance. But that leads us to another distinction:

declarational / operational

Most scripting languages are way over there on the operational side. I thought Perl 5 had an oversimplified object system till I saw Lua. In Lua, an object is just a hash, and there's a bit of syntactic sugar to call a hash element if it happens to contain code. Thats all there is. They don't even have classes. Anything resembling inheritance has to be handled by explicit delegation. That's a choice the designers of Lua made to keep the language very small and embeddable. For them, maybe it's the right choice.

Perl 5 has always been a bit more declarational than either Python or Ruby. I've always felt strongly that implicit scoping was just asking for trouble, and that scoped variable declarations should be very easy to recognize visually. Thats why we have my. It's short because I knew we'd use it frequently. Huffman coding. Keep common things short, but not too short. In this case, 0 is too short.

Perl 6 has more different kinds of scopes, so we'll have more declarators like my and our. But appearances can be deceiving. While the language looks more declarative on the surface, we make most of the declarations operationally hookable underneath to retain flexibility. When you declare the type of a variable, for instance, you're really just doing a kind of tie, in Perl 5 terms. The main difference is that you're tying the implementation to the variable at compile time rather than run time, which makes things more efficient, or at least potentially optimizable.

immutable classes / mutable classes

Classes in Java are closed, which is one of the reasons Java can run pretty fast. In contrast, Ruby's classes are open, which means you can add new things to them at any time. Keeping that option open is perhaps one of the reasons Ruby runs so slow. But that flexibility is also why Ruby has Rails.

Perl 6 will have an interesting mix of immutable generics and mutable classes here, and interesting policies on who is allowed to close classes when. Classes are never allowed to close or finalize themselves, for instance. Sorry, for some reason I keep talking about Perl 6. It could have something to do with the fact that we've had to think about all of these dimensions in designing Perl 6.

class-based / prototype-based

Here's another dimension that can open up to allow both approaches. Some of you may be familiar with classless languages like Self or JavaScript. Instead of classes, objects just clone from their ancestors or delegate to other objects. For many kinds of modeling, it's actually closer to the way the real world works. Real organisms just copy their DNA when they reproduce. They don't have some DNA of their own, and an @ISA array telling you which parent objects contain the rest of their DNA.

The meta-object protocol for Perl 6 defaults to class-based, but is flexible enough to set up prototype-based objects as well. Some of you have played around with Moose in Perl 5. Moose is essentially a prototype of Perl 6's object model. On a semantic level, anyway. The syntax is a little different. Hopefully a little more natural in Perl 6.

passive data, global consistency / active data, local consistency

Your view of data and control will vary with how functional or object-oriented your brain is. People just think differently. Some people think mathematically, in terms of provable universal truths. Functional programmers don't much care if they strew implicit computation state throughout the stack and heap, as long as everything looks pure and free from side-effects.

Other people think socially, in terms of cooperating entities that each have their own free will. And it's pretty important to them that the state of the computation be stored with each individual object, not off in some heap of continuations somewhere.

Of course, some of us can't make up our minds whether we'd rather emulate the logical Sherlock Holmes or sociable Dr. Watson. Fortunately, scripting is not incompatible with either of these approaches, because both approaches can be made more approachable to normal folk.

info hiding / scoping / attachment

And finally, if you're designing a computer language, there are a couple bazillion ways to encapsulate data. You have to decide which ones are important. What's the best way to let the programmer achieve separation of concerns?

object / class / aspect / closure / module / template / trait

You can use any of these various traditional encapsulation mechanisms.

transaction / reaction / dynamic scope

Or you can isolate information to various time-based domains.

process / thread / device / environment

You can attach info to various OS concepts.

screen / window / panel / menu / icon

You can hide info various places in your GUI. Yeah, yeah, I know, everything is an object. But some objects are more equal than others.

syntactic scope / semantic scope / pragmatic scope

Information can attach to various abstractions of your program, including, bizarrely, lexical scopes. Though if you think about it hard enough, you realize lexical scopes are also a funny kind of dynamic scope, or recursion wouldn't work right. A state variable is actually more purely lexical than a my variable, because it's shared by all calls to that lexical scope. But even state variables get cloned with closures. Only global variables can be truly lexical, as long as you refer to them only in a given lexical scope. Go figure.

So really, most of our scopes are semantic scopes that happen to be attached to a particular syntactic scope.

You may be wondering what I mean by a pragmatic scope. That's the scope of what the user of the program is storing in their brain, or in some surrogate for their brain, such as a game cartridge. In a sense, most of the web pages out there on the Internet are part of the pragmatic scope. As is most of the data in databases. The hallmark of the pragmatic scope is that you really don't know the lifetime of the container. It's just out there somewhere, and will eventually be collected by that Great Garbage Collector that collects all information that anyone forgets to remember. The Google cache can only last so long. Eventually we will forget the meaning of every URL. But we must not forget the principle of the URL. That leads us to our next degree of freedom.

use Lingua::Perligata;

If you allow a language to mutate its own grammar within a lexical scope, how do you keep track of that cleanly? Perl 5 discovered one really bad way to do it, namely source filters, but even so we ended up with Perl dialects such as Perligata and Klingon. What would it be like if we actually did it right?

Doing it right involves treating the evolution of the language as a pragmatic scope, or as a set of pragmatic scopes. You have to be able to name your dialect, kind of like a URL, so there needs to be a universal root language, and ways of warping that universal root language into whatever dialect you like. This is actually near the heart of the vision for Perl 6. We don't see Perl 6 as a single language, but as the root for a family of related languages. As a family, there are shared cultural values that can be passed back and forth among sibling languages as well as to the descendants.

I hope you're all scared stiff by all these degrees of freedom. I'm sure there are other dimensions that are even scarier.

But... I think its a manageable problem. I think its possible to still think of Perl 6 as a scripting language, with easy onramps.

And the reason I think its manageable is because, for each of these dimensions, it's not just a binary decision, but a knob that can be positioned at design time, compile time, or even run time. For a given dimension X, different scripting languages make different choices, set the knob at different locations.

You can't even think about X!
There's only one way to do X!
There's more than one way to do X!
There are too many ways to do X!

You may recognize some slogans in here.

Curling Up

So I'm not suggesting that all scripting languages have to take all these dimensions into account, even if Perl 6 tries to. The scripting paradigm is not any one of these dimensions. According to various theories the universe may be laid out in ten or twenty dimensions, but generally we get by with only about three and a half of those dimensions. The rest are said to be curled up. Maybe we live in a scripting universe.

Most of the scripting languages we call Perl 6 will have most of these dimensions curled up most of the time. But unlike the real universe, where it takes huge machines to uncurl these dimensions, we'll make the dimensions uncurl just by keeping our declarations straight. Well, we'll try. And where that fails, we'll rely on the culture to keep things straight.

For example, that's exactly what happened already with Perl 5. We have the declarations, use strict; use warnings;. But it's the culture that decided to enforce the use of them. So much so that we've decided that they should be the default for most of Perl 6. It was one of those decisions by the hive. In this case the swarm turned out to be smarter than the language designer. And that's as it should be.

The Future

Well, so what's the future of scripting?

In my completely unbiased opinion, that would be Perl 6. :-)

Seriously though, it's always safe to predict that the ecological landscape will end up with many small languages and a few dominant ones. Some languages like AppleScript have particular ecological niches and are unlikely to grow out of them. Other languages get used outside their original niche. There will always be the generalists, like crows and mockingbirds, and the specialists, like penguins and dodos. (Well, maybe not always the dodos...)

Among the generalists, the conventional wisdom is that the worse-is-better approach is more adaptive. Personally, I get a little tired of the argument: My worse-is-better is better than your worse-is-better because I'm better at being worser! Is it really true that the worse-is-better approach always wins? With Perl 6 we're trying to sneak one better-is-better cycle in there and hope to come out ahead before reverting to the tried and true worse-is-better approach. Whether that works, only time will tell.

Visit the home of the Perl programming language: Perl.org

Sponsored by

Monthly Archives

Powered by Movable Type 5.13-en