December 2003 Archives

Blosxoms, Bryars and Blikis

Recently we heard from Kake Pugh about the OpenGuides project, a wiki-based collaborative city guide system; previously, we heard from Brian Ingerson about his Kwiki wiki implementation. Guides, wikis, blogs ... the new fashion in software engineering at the moment is the use of software to help organize, document, and facilitate collaboration -- the social software movement is gaining momentum, and Perl is one of the best languages for it.

In this article we'll look not just at some of the existing social software tools in Perl (focusing, naturally, on my own Bryar blog toolkit), but we'll look at some ways to break, bend, and embed them in other tasks.

Basic Blogging with Blosxom

When I finally decided that the world would benefit from hearing my internal monologue, I naturally looked around for nice, simple blogging programs. Blogs are, essentially, simple things, and so I didn't want all-singing, all-dancing web-based database-backed content management systems. I knew that if I were going to blog, it would have to be easy for me to do so, and there's nothing easier than firing up a text editor in the next available terminal window and doodling my thoughts into it.

Thankfully, Rael Dornfest has the same sort of laziness as I, and created Blosxom, a very simple blogging tool, which simply reads a bunch of files from the file system, finds the most recent and relevant, packages them up together, and sends them at a browser. That was how I saw blogging.

Getting Blosxom up and running is, in keeping with the whole theme of Blosxom, quite simple. You need to download the blosxom zip file, unpack it, drop the blosxom.cgi file in your web server's cgi-bin directory, and then edit the first few lines of it to tell it the name of your blog and where the entries will live, and you're done.

Posting into the blog is just a matter of creating a file called something.txt in the blog directory. It doesn't even matter what the something is. All that matters is the first line of the file, which becomes the title, and the rest, which becomes the story in raw HTML:


First post!

<p> Today I set up a blosxom blog. It was really easy! </p>

You can then, if you like, style your blog with custom header and footer files, CSS, custom HTML for each blog post, and so on.

Enter Bryar

Of course, back when I started using Blosxom, things were not quite so easy, and templating required hacking the Blosxom CGI file itself, and things just didn't seem as neat as I wanted them to be. I liked Blosxom's simplicity, but didn't want to pay the price for it in terms of flexibility.

At the same time, I was learning the wonders of the Template Toolkit, and thought that a well-designed blog should merely collect Post objects and send them to a user-defined template to do what it wants with them.

So I started writing Bryar, which is designed to be a modular, extensible replacement for Blosxom. To make this happen, I needed to provide a default set of components that simply Does The Right Thing. Let's take a look at that default set before we go any further.

When we install Bryar from CPAN, we're encouraged to run a command to set up a new blog:

You probably want to run bryar-newblog in a likely home for
your blog once we've finished installing.
    

The bryar-newblog command simply sets up a decent default set of templates, creates a first blog post, and drops in a bryar.cgi driver, similar to blosxom. At this point, we're ready to go.

Bryar's default collection of modules consists of a Bryar::DataSource::FlatFile, which emulates Blosxom's use of text files in the file system as a database of blog posts; there are Bryar::Frontend::CGI and Bryar::Frontend::mod_perl, which describe how URL parameters are retrieved from the server translated into a request for documents, Bryar::Document which represents each post, and Bryar::Collector which ties it all together. With the benefit of hindsight, that should have been called a Bryar::Controller, since the whole thing can be described as a Model-View-Controller application. But more on that later!

Since all of these things can be overridden, we need something to keep track of which classes we're using, and the Bryar::Config class does that, reading an optional configuration file but otherwise providing mostly sensible defaults.

In the rest of this article we're going to look at some interesting ways to override those classes, and build some more funky tools out of this social software framework.

Blogging the Past

Now it may surprise some people to find that I'm not really a programmer, although it may not come as a surprise to some of my colleagues.

I was trained in modern and classical Japanese language and literature, with a side order of linguistics. Programming was a hobby that turned out to make a bit more cash than conjugating 10th century verbs.

But as I began thinking a bit about blogging, I found an old Japanese diary of a lady-in-waiting to the Emperor, called the "Pillow Book", by Sei Shonagon. It was something I'd read before and found quite interesting, but picking it up again in a blog context put it in a completely new light. It read just like a 10th century LiveJournal.

Here's a sample entry:

Small children and babies should be fat. Provincial governors should be fat too, and other people who have done well for themselves. If they're too thin, it makes you think they're miserable.

As well as thoughts, observations, tales of life at the court, the book contains a large number of random lists, which really made me think of LiveJournal polls and surveys. Here's another example:

Things I think are elegant
  • A white coat over a violet waistcoat
  • Duck's eggs
  • Sherbet ice, liana syrup, in a nice new silver bowl
  • Rock crystal
  • Wisteria
  • Snow on plum blossoms
  • A pretty little girl eating strawberries

I decided that it would be quite fun to translate the blog -- uh, book -- in a fairly modern, relaxed style, use fictitious but plausible dates for the entries, and run it as a real blog.

Naturally, I wanted to do this with Bryar, but there was going to be a slight problem; Bryar takes its entry dates from the Unix timestamp of the file, and try as I might, I could not persuade Unix to recognize epoch times a thousand years before the epoch. I had to find another way of getting the times.

But since Bryar is extensible, this is really easy -- I just write a new data source class that gets the entry timestamp from somewhere else. I simply created a subclass of Bryar::DataSource::FlatFile called ...FlatFile::Dated, which takes its date from the first line, and title from the next line.

Telling the Bryar blog to use the new data source was simply a matter of writing a little config file called bryar.conf:


source: Bryar::DataSource::FlatFile::Dated
name: Notes on a Pillow - Sei Shonagon's blog
baseurl: http://blog.simon-cozens.org/shonagon/bryar.cgi

And the rest, of course, is just a simple matter of classical Japanese translation.

Blogging the House

But as well as a linguist, I'm also a geek, and I use Perl to control various things around the house -- the house network, the division of the phone bill, and so on. This is all driven by Template Toolkit and Class::DBI.

It seemed useful, at the time, to add a blog into this great mass of centralized control, so we could pass phone messages around. Of course, in the end, it was easier to just put a big whiteboard in the lounge. XP's "the simplest thing that could possibly work" wins again.

But it was fun, and it allowed me to use Bryar in a new way. This time, we're using the original data source, collector, and other modules, but we're plugging them into an existing templating system. We already have code to deal with the web server, handle the URL processing, and the output; we already have code to transform objects into HTML; all we need is to collect the recent documents and hand them on to the templates.

I did this by adding a Bryar object to our template variables, and calling the collect_current method on the Bryar collector, using Richard Clamp's Template::Plugin::Class module. This gave a bunch of Bryar::Documents, which the template system could manipulate however it desired:

[% INCLUDE header %]
[% PROCESS blogmacros %]

<h2> Recent blog postings</h2>

[%
USE c = Class("Bryar::Collector");
FOR post = c.collect_current(blog);
    display_post(post);
END;
%]

Everything's a Wiki

So much for blogs, for the time being. The other movement in social software at the moment is Ward Cunningham's wiki. A wiki has three important characteristics: a simple mark-up language, the ability for the viewer to modify the page, and the ability to create links from one wiki page to another.

Despite initial predictions that they'd be unusable due to widespread graffiti, vandalism, and other kinds of kicking down the sandcastle, wikis have turned out to work quite well in practice, with sites like Wikipedia collecting a great store of information from contributors both regular and occasional.

CPAN has a fair collection of wiki implementations, including Kwiki, CGI::Wiki, Apache::MiniWiki, and others. However, there may well be times when it makes sense to create a new implementation, particularly when the wiki is to be embedded in another application.

What does it mean to embed a wiki? An example will demonstrate it best. Imagine a web-mail system like IMP or SquirrelMail. Now we want to add the ability to write a short note or annotation on a given message, so that it can be referred back to later. If I get a mail from my friend Al arranging to meet up for a drink at the pub, I'd want to annotate the mail with Al's mobile phone number, directions to the pub, and the fact that I need to take the book that I borrowed from him last month. Then, in an ideal world, the mail system would collect all the notes, work out the date and time of the meeting, and tell my calendaring software about it, but that's another story for a different time.

The fact is that once we have these sticky notes for mail, we might want to use them to refer to other mail, group them together in fluid topics, and so on. For instance, if I mentioned in the annotation that this was an AlMeeting, I would expect that text to be magically marked up as a link to all the other AlMeeting mails. Here we've got the three elements of a wiki -- annotation editing, a simple markup language, and the ability to link.

While the editing is a simple matter of linking your CGI parameters with your database, something that should be second nature to those who want to avoid writing code, the other two components take a little bit of work. Thankfully, chromatic has written a handy module, Text::WikiFormat.

With WikiFormat, you can pass a text string marked up in wiki format to Text::WikiFormat::format, and it will return you HTML:


print Text::WikiFormat::format(<<EOF);
= A Test Post =

I found this '''very''' interesting WikiLink recently.

EOF

__DATA__
<h1>A Test Post</h1>

<p> <br />
I found this <strong>very</strong> interesting 
<a href="WikiLink">WikiLink</a> recently. <br />
  </p>

This gives us the simple markup, but it doesn't do much about the linkability; for an embedded wiki, we're going to need finer-grained control over how the links are constructed. For instance, we might want to pass the link name to a particular URL with a particular CGI parameter. Thankfully, Text::WikiFormat supports two optional parameters in addition to the text to be formatted: the first is a hash reference overriding how the wiki text is marked up to HTML, and the second is a hash reference of options, such as the URL prefix for links.

So, if we have an application that displays a page given a parameter in the URL, we can just say:

 
Text::WikiFormat::format($text, {},  
        { prefix => "http://www.myhost.int/wiki?page=" }); 

And there we have it, an instant wiki.

Of course, in the context of our webmail application, we may have to do something more convoluted. For instance, we might want a link to SimonCozens to resolve to all mail from that person, if it can be understood as a person who sends email, or to a similarly marked-up set of messages, like an AlMeeting, if it can't. For this, we need to introduce a bit of logic and decide how to form the link.

As we mentioned above, Text::WikiFormat allows us to specify replacements for the way markup is translated to HTML. WikiFormat uses a hash-ref internally to turn wiki "tags" into HTML, and we can, in fact, put in a subroutine reference for the link tag and have that emit the appropriate HTML link.

 
sub markup { 
    my $text = shift; 
    Text::WikiFormat::format(shift, { link => \&my_link });
} 

sub my_link {
    my $linkname = shift;
    my $name = $linkname;
    ($name=$linkname) =~ s/(?<=[a-z])(?=[A-Z])/ /g;
    if (have_mail_from($name)) {
       # Valid correspondent, link to page of their mail
       my $id = correspondent_id($name);
       return qq{<A HREF="/mail/search?correspondent=$id">}.$name.
              "</A>";
    }
    # Not a correspondent, just a wiki page
    return qq{<A HREF="/wiki?$linkname">}.$linkname.
           "</A>";
}

What's going on here? First, we receive the link name in StudlyCaps every time format sees a link. We then convert it to a real name, by inserting a space in the gaps between a lowercase character and an uppercase one. Now we have a name like "Simon Cozens" that we can search our mailstore for, and if we find someone, we can return the URL to a page about their mail. Otherwise, we return a wiki page. We've used our webmail annotations to refer to either real people or to other annotations, stealthily embedding a wiki in the application, with minimal effort.

Creating the Bliki

With that example of how to create a custom wiki from an existing application, let's return to Bryar and our blogging software.

The concept of a bliki -- a cross between a wiki and a blog -- may seem like a bizarre one at first, but it's such a natural synergy that it's been independently invented by multiple people, including Martin Fowler and many others.

Depending on how you look at it, a bliki is either an ordinary wiki with a blog-style front-end showing the latest posts, or an ordinary blog with wiki-like markup and interconnections between the posts. The only difference between these views is how easy it is to edit the posts. We'll stick with the "ordinary blog" viewpoint, since editing posts over the Web is a boring and solved problem, and one not really related to the technology we're looking at today.

How are we going to extend Bryar to become a bliki? It's obvious that Text::WikiFormat is going to be involved somewhere, and it would be helpful to use the Bryar::DataSource::FlatFile class too. First, we inherit from the FlatFile class, and override the make_document method, which is responsible for turning a filename into a Bryar::Document:


package Bryar::DataSource::FlatFile::Bliki;
use base 'Bryar::DataSource::FlatFile';
use strict;
use Text::WikiFormat;

sub make_document {
    my ($self, $filename) = @_;
}

Here the format will be slightly different -- the filename (such as "MyWikiPost.txt") will be used to form both the title and the ID of the document, and the content of the document will be transformed via WikiFormatter wholesale.


sub make_document {
    my ($self, $file) = @_;
    return unless $file and open my $in, $file;
    my $when = (stat $file)[9];
    my $who = getpwuid((stat $file)[4]);
    $file =~ s/\.txt$//;

    my $dir = dirname($file);
    $dir =~ s{^\./?}{};
    my $category = $dir || "main";

    local $/;
    my $text = <$in>;
    
    return Bryar::Document->new(
        title    => $file,
        content  => Text::WikiFormat::format($text, {}, {prefix => "id_"}),
        epoch    => $when,
        author   => $who,
        id       => $file,
        category => $category,
    );
}

We change the prefix for links to be a relative link of the form id_PostName, since this is how Bryar deals with references to other blog posts, by default. (Of course, we could change this by modifying the FrontEnd class, but this seems to work fine.)

It took 30 lines of code to convert Bryar into a simple bliki, many of which were stolen from the superclass. Now let's look at something a bit different.

Everything's a Blog

I mentioned when introducing Bryar that it was based on the Model-View-Controller (MVC) pattern, although I didn't realize this while designing it.

The MVC pattern is described in this entry in Ward Cunningham's own wiki, as being made up of three parts:

  • A model, therefore is an object representing data or even activity, e.g. a database table or even some plant-floor production-machine process.
  • A view is some form of visualization of the state of the model.
  • A controller offers facilities to change the state of the model.

All kinds of things follow the MVC design. In Bryar, the model is the data source object; the view is the templating object; the controller is the Collector object. A web browser might be described as a controller that contains all its logic, a view which contains the user interface, and the pane of rendered HTML, and the model of the entire Worldwide Web.

But some web-based applications follow Bryar's MVC approach particularly closely. Imagine, for instance, a company's product catalog on the Web. The catalog data will be stored in some kind of database, and will be presented via a templating system, with a controller that will decide which products to show. Isn't that exactly the same as reading some blog posts from a database (albeit potentially one that looks a lot like a filesystem), rendering them via templating system, with a controller deciding which posts to show?

Indeed, many of the views you will want of the catalog data are exactly what you'd want from a blog: display the most recent products, display all products, search by product name or description, and a detailed display of a single product.

We've seen it take 30 lines to turn a blog into a bliki. To turn a blog into a complete product catalog web site takes a subclass of the Bryar::DataSource::DBI class (to talk to the product database) and the Bryar::Document class (to take more details, a URL to a photo, and so on). The rest is a matter of templating, and that's a designer's job!

Social Software

I consider the emergence of interest in social software to be one of the most fascinating trends in software engineering this year. Two of the most powerful and popular aspects of this, wikis and blogs, are particularly well-suited for extension and embedding, and Perl is a particularly well-suited language for achieving this.

Although part of the point of this article was to demonstrate Bryar, there were several other important points. First, that there are plenty of Perl implementations of both wikis and blogs that you can choose from; second, that Perl makes it really easy to create your own blog or wiki and customize to your own purposes, including embedding them in an existing application.

But finally, the point was to encourage you to think about good design and the power of extensible applications; if you can create a tool that is both powerful and generalizable -- just like Perl itself -- it may end up doing wildly different things to what you initially intended!

This week on Perl 6, week ending 2003-12-14


It looks like things are starting to slow down slightly as we run up to Christmas, but the quality of discussion remains high. We'll start with the usual trawl through perl6-internals.

Testing for null

Dan ruled on last week's discussion of testing for undef in Parrot. We're getting a new isnull branch op along with a new Undef PMC. Leo Tötsch implemented it with his usual alacrity.

http://groups.google.com/

How many comparison ops do we need again?

One of the annoyances of writing something that's really going to be used is that you can't get away with the equivalent of outlining your proof on the blackboard, waving your hands and saying "The details are left as an exercise for the interested reader". A case in point is Parrot's array of comparison branch operators. For a while now we've been getting by with a sparsely populated array of such beasties. This week saw people (well, okay then, Leo, but he has the productivity of people) doing the detail work to get the full range implemented.

http://groups.google.com/

Incorrect scoping of constants in IMCC

Dan wasn't happy with the scoping of the .const directive in IMCC (essentially constants are sub/block scoped, which makes them pretty much useless for the things constants are normally used for). Melvin Smith made the fix.

http://groups.google.com/

Objects and Namespaces

Dan had mandated that the Parrot internal equivalent of the Perlish Package::Name should be Package\0Name. Nobody (Dan included) liked the embedded nulls mandated in Dan's original spec. After some discussion he posted a description of a new hierarchical namespace design.

Dan pointed out that the use of hierarchical namespaces would probably mean that the semantics (and syntax) of find_global would need to be adjusted.

http://groups.google.com/

http://groups.google.com/ -- The New Namespace

Dan Sugalski, IMCC Bugfinder General

Dan's been writing more PIR code by hand, and is discovering infelicities as he goes.

http://groups.google.com/

http://groups.google.com/

Macros, PIR and PASM

Following his bout of IMCC bug spotting, Dan announced a decision about macros in PIR and PASM code. Essentially, Parrot's built in PIR and PASM parsers do not need to do macros as 'macro assemblers are for people, not compilers'. Because of this, Dan would like to see any macro processing moved out of the core binary and into an external tool. (Which can always be invoked via the hypothetical macro-parrot.pl). Melvin Smith suggested that it would probably be worth extracting the existing macro processor from imcc.l and moving it into a separate library.

http://groups.google.com/

Meanwhile, in perl6-language

The language list continues to fascinate (and not in the 'car crash' way it has done on occasions in the past) with an ongoing conversation between Larry on the one hand (seemingly designing on the hoof and certainly doing a great deal of thinking aloud) and everyone else on the other hand getting clarification or extrapolating to the point where Larry has to step in and do a little bit more design.

I'm finding it hard to do a proper summary of what's going on because so much is changing (and because so much of what seems to be set in quick drying concrete now is so novel.)

Essentially the discussion revolves around Roles, which are sort of like Java interfaces, sort of like mixins and nothing like either. In his 'Vocabulary' post, Luke Palmer describes a Role as 'A collection of methods [and/or attributes] to be incorporated into a class sans inheritance [...]. Used with does'. In this new world, a property ($bar but Red etc) is a kind of degenerate role.

At the same time though, we're getting other gems. For instance, Perl 6 is going to get something a little like final, but instead of it being a declaration made in a type's implementation ("This is the one, the only Integer and you may not inherit from it or ever alter it!"), it will become a promise made by a type's user not to attempt to monkey with it come run time, which should set the code generator free to use optimizations that aren't safe when everything's dynamic.

So, if you're interested in how Perl 6's OO system is going to work, now is the time to get involved in the language list. If there's something you've always wanted to do or something about Perl's current OO that you love and want to retain, it's time to speak up for it. The odds are good that you'll be able to do what you want using the mechanisms being designed, but you'll be kicking yourself later if it turns out you can't.

If you do join in, you'll see references to the 'Traits Paper'. This is where the original idea for what Larry's calling 'Roles' came from. When you read it, bear in mind that the authors of the paper use the term 'trait' for what we're calling a 'role' (We already have traits you see).

http://www.cse.ogi.edu/~black/publications/TR_CSE_02-012.pdf -- Traits paper

Iterating through two arrays at once

Joe Gottman asked for some clarification about which was more right, Apocalypse 4 (for @a ; @b -> $a ; $b {...}) or Perl 6 Essentials (for zip @a, @b -> $a, $b {...}) when talking about Essentials> iterating through two (or more) arrays at once. The rule of thumb for this seems to be 'latest publication date wins' so P6E is the most correct. He went on to wonder what happens when one array is shorter than the other (an undefined value is passed into the loop when the array runs out of values until the last array is exhausted). He wondered what would happen if any of the arrays contained undef; how would the body of the loop be able to differentiate between a 'valid' undef and one created by the loop itself.

Larry pointed out that, in Perl 6, undef isn't necessarily unique and may well contain an unthrown exception with extra information hidden in a property.

http://groups.google.com/

Vocabulary

If you're even vaguely interested in the workings of Perl 6's object system, you need to read the referenced post.

Luke Palmer, worrying about people using Object related vocabulary in subtly inconsistent ways, posted a glossary explaining how OO terminology is used in a Perl 6 context. Casey West wrapped it up in a POD, which will, I hope, end up on dev.perl.org soon.

Of course, there were a few corrections for subtleties, a few rethinks of the design so far, and general gratitude for at least having a baseline document that people could refer to.

http://groups.google.com/

http://groups.google.com/ -- Casey's POD

Enums and bitenums

Stéphane Payrard wondered if Perl 6 would have enums and bitenums directly supported in the language (he pointed out that it wouldn't be hard to implement them using macros of course). Larry speculated that enums might just end up being special cases of roles/subtypes.

Further down the thread, Uri Guttman commented that Larry seemed to be working on a grand unification and wondered if string theory had any place in it. (Actually, it's been apparent for ages now that a great deal of the power that Larry's putting into Perl 6 arises from the unifications he's making, but that's an article for my weblog I think).

http://groups.google.com/

Acknowledgements, Apologies, Announcements

For the duration of the panto season, Leon Brocard will be appearing in the role of Pumpking in a Perl 5 Porters summary near you.

Still no link shortening. I appear to have managed to subtly screw my Perl installation and now I can't get the link shortening module installed. I shall try and make time to fix it this week.

Unless someone beats me to it (please someone, beat me to it) I'm going to try and put together a 'state of play' post, summarizing the current state of the Perl 6 Object system with a little more background in it than Luke Palmer's excellent 'Vocabulary' post. All being well it'll be appearing in next week's summary.

If you find these summaries useful or enjoyable, you might like to contribute to the Perl Foundation to help support the ongoing development of Perl. I also welcome feedback at p6summarizer@bofh.org.uk or at my website.

http://donate.perl-foundation.org/ -- The Perl Foundation

http://www.bofh.org.uk:8080/ -- My website, Just a Summary

This week on Perl 6, week ending 2003-12-07

Another week, another late summary. Luckily it's been a quiet week so I should get this written faster than usual. As is traditional, we start with perl6-internals

Parrot build system tinkering

Andy Dougherty and other discussed extending the Parrot build system to allow for building parrot itself outside the main source directory (useful when Parrot's mounted read only via NFS for example. It's not (quite) possible to do this just yet because of a couple of generated files in the IMCC tree. Leo suggested only generating those when Configure.pl --maintainer is run, on the grounds that a maintainer shouldn't be building from a read only source. Dan asked for a volunteer to fix things up.

http://groups.google.com/

Object stuff

I (and others I presume) got ready to do the "Happy! Happy! Joy! Joy!" dance of delight; this was the week that Parrot finally got objects. Or, as Dan described them, 'objects-ish'. Parrot now has single inheritance objects. They're not the final word on the matter, but it's definitely better than a poke in the eye with a sharp stick.

Dan went on to discuss what needs to be done with them before we're ready for Parrot 0.1 (Objects are one of the criteria for making a point release).

Dan then went rather quiet, apparently because he got mugged by flu, or something like it, and spent the rest of the week wishing he were dead.

Meanwhile, Leo lived up to his Patchmonster nickname by checking in a bunch of object related stuff.

http://groups.google.com/ -- Rumblings

http://groups.google.com/ -- Announcement

http://groups.google.com/ -- The way forward

http://groups.google.com/ -- Dan gets mugged by flu

http://groups.google.com/ -- Leo's patches

Raising exceptions

Cory Spencer wondered about 'proper exception raising' and wondered what the state of play was, and if there were any good examples. Leo Tötsch pointed him at t/pmc/exception.t

http://groups.google.com/

Compilation units and the boundaries of IMCC

Pete Lomax had wondered about package/file local variables in IMCC. At the moment IMCC variables can't be file or package scoped, but Melvin Smith is working on it. As he said this, he asked for suggestions for other potential IMCC features. Cory Spencer wanted to be able to write

   if _some_test() goto LABEL

but that was vetoed by Dan (and Melvin) as being rather too high level for an intermediate language that's meant to be a compiler target.

http://groups.google.com/

String manipulation

Melvin Smith put forward a convincing argument that IMCC shouldn't concern itself with doing string interpolation as the expected behaviour is too 'high level language' dependent, but that Parrot should have a printf/sprintf equivalent, implemented in C for speed. Dan seemed to agree and reckoned it should be addressed as soon as objects were out of the way.

http://groups.google.com/

IMCC pending changes

Melvin Smith posted a list of pending IMCC changes, most of which were concerned with tightening up IMCC behaviour (making it both stricter nd more forgiving...). Discussion ensued.

A few days later, Melvin committed some of his changes.

http://groups.google.com/

http://groups.google.com/

PMC Compiler 2nd edition

Leo applied a 'really long pending patch' which should simplify upcoming vtable changes. Melvin wondered if the time had come to replace the existing ops2c and pmc2c with the newer versions. Leo thought that pmc2c2 was definitely stable enough, but wasn't too sure about ops2c2. Jonathan Worthington pointed out a Win32 bug that was quickly fixed.

http://groups.google.com/

get_pmc_keyed() in PerlString

Sterling Hughes noted that, in PHP it's valid to index a string as if it were an array. He wondered it would be possible to implement get_pmc_keyed() on the PerlString vtable to allow for similar tricks with the standard string PMC. I'm confused as to what the decision was. I think Leo agreed that it was a good idea, but that it should probably be implemented in a new PMC for the time being...

http://groups.google.com/

Symbolic vs. Named variable register allocation

Pete Lomax stumbled over a bug in IMCC's register allocation. Melvin thinks the problem is that much of IMCC's flow analysis code doesn't yet know about the Parrot Calling Conventions and went to have a look to see if there was a quick fix available.

http://groups.google.com/

Testing for null

Jürgen Bömmels is working on having the Parrot IO system's open return a PMCNULL on failure instead of an invalid IO-Object (sounds like a good idea to me). The catch is, there doesn't seem to be any way for the bytecode to detect that this has happened. There was some debate about whether what Jürgen was doing was the right thing, or if we shouldn't introduce a new null.pmc, or some other possibility. The consensus in the end seemed to be that using PMCNULL was probably the way to go, but an appropriate test operator hadn't been finally decided upon (but if_null and unless_null are the front runners).

http://groups.google.com/

Determining PMC memory addresses

After last week's kerfuffle about the different types of equality Cory Spencer still needed a test for testing if two PMCs had the same address so he posted a patch to add an eq_addr opcode. Melvin Smith still wasn't sure that the name was quite right though and asked for a description of why Cory needed it (he needs it for implementing a Lisp interpreter).

http://groups.google.com/

Fixing the Makefile dependencies

Harry the Surnameless has gone through the makefiles in search of the broken dependencies that meant that parallel makes didn't work and posted a patch. However, he worries that the patch may well introduce other problems.

http://groups.google.com/

Meanwhile in perl6-language

The only game in town was the ongoing discussion of Properties, which morphed into a discussion of Larry's tantalizing role based OO formulation. People are generally impressed and excited by what Larry has so far revealed and, in the continuing absence of Apocalypse 12 are speculating like mad. It's fascinating, but it's also next to impossible to summarize beyond saying "People talked excitedly about OO".

Oh yes, if you've not been following, ^op (ie, the vector operators) has become >>op<< which is, if nothing else, a right swine to write in a POD C<> escape.

http://groups.google.com/

http://groups.google.com/

Acknowledgements, Apologies, Announcements

Nope, still no link shortening.

Um... many apologies to Leon Brocard. Last week I stated that he'd become the new Pumpking for Perl 5.004 and would not be appearing as a running joke in these summaries until he relinquished the pumpkin. However, what I should have said is that he has become the new Pumpking for Perl 5.005_04 and will not be appearing as a running joke in these summaries until he relinquishes said pumpkin.

Any suggestions that last week's mistake was deliberate so I could continue the joke for one more week are utterly without foundation.

It was just an 'amusing' consequence of my stupidity, obviously.

If you find these summaries useful or enjoyable, please consider contributing to the Perl Foundation to help support the Perl 6 effort. I also welcome feedback at or at my website.

http://donate.perl-foundation.org/ -- The Perl Foundation

http://dev.perl.org/perl6/ -- Perl 6 Development site

http://www.bofh.org.uk:8080/ -- My website, "Just a Summary"

How Perl Powers the Squeezebox

Slim Devices made their name in 2001 with the SLIMP3, a networked MP3 and Internet radio player. The SLIMP3 won a five-out-of-five mouse rating in Macworld magazine, and was featured in GQ magazine and on TechTV's The Screen Savers. This year, they're trying to repeat the success with Squeezebox, an 802.11-enabled version.

The interesting thing about the SLIMP3 and the Squeezebox from our point of view, though, is that the server which drives the player is written in Perl, and developers are allowed -- in fact, encouraged -- to hack on it and make the devices do interesting things. A developer community has sprung up around the SLIMP3, and some interesting third-party hacks have been produced.

We caught up with Dean Blackketter from Slim Devices, who took some time out from moving Squeezeboxes to answer a few of our questions.


perl.com: What made you decide, first of all, to write the server code in Perl?

Dean: Sean [Adams, Slim Devices founder] wrote the very first version of the software in Perl because he was able to get it up and running quickly. When I first encountered Sean and SLIMP3, I was a little afraid of Perl, having been primarily a C and C++ programmer, mostly because when I tried to read it, I got confused by idiomatic usage.

I first decided to rewrite the thing in C, but I couldn't really get started until I could read the existing code. So I bought a copy of Learning Perl and read it in two evenings. Halfway through the first chapter something clicked and I said, "That's so cool!" Then, about every 10 pages after I'd repeat, "That's so cool!" again, but a little louder. By the end, I was sold. At that point I dug in and started making the software better.

What features of Perl were particularly helpful when you were writing the SlimServer?

The greatest feature of Perl, for us, is that so many people know it. Since our software is open source, we benefit from as many users as possible being able to read and improve our code base. Anybody who's thrown together a little Perl CGI or script is a potential contributor to our software.

Second, the ability for us to offer our software on a wide array of platforms (Mac OS X, Windows 98/ME/NT/2000/XP, BSD, Linux, Solaris) and also be deployed on server appliances makes it possible for us to get our product running as broadly as possible.

Third, even though our product's primary feature is playing back audio, the bulk of the software really is processing text. Our built-in HTTP server, command-line interface, device-user interface, and even music meta-information is all text-based, and Perl is nearly the perfect language for this.

What issues did you have to face when considering how to distribute a Perl-based consumer application, and how did you approach them?

Distributing a Perl application on the traditional Unix-like platforms (BSD, Solaris, Linux) is easy. Things got a bit harder on Mac OS X, since command-line solutions aren't acceptable to Mac users. Initially, I wrote a simple graphical application (in AppleScript!) that acted as a launcher for our server software, but this was really not good enough for Mac users. One of our Open Source contributors, Dave Nanian, came to the rescue and built us a System Preferences pane and installer application that work as they should on a Mac.

Windows is the hardest platform for us to support. Luckily we had another Open Source contributor, Sam Saffron, who decided that building a shell application for the server software would be a great way to learn MFC, and he contributed a nice little app for this. Another contributor, Chris Eastwood, put together an installer for us. But it was the fine folks at ActiveState with their Perl Dev Kit that really made it possible for us to release on Windows with their PerlApp (for making Perl-based .exe files) and PerlSvc (for making Perl-based Windows Services).

Even with all this help, Perl on Windows is different enough in subtle ways to make it a real pain to maintain. Maybe if we had more hardcore in-house Windows experience things would be easier, but it's a struggle.

For consumer devices like the SLIMP3, the manufacturer has a choice between open and hackable, and "sealed box." You guys went for the hackable approach, GPLing the code and setting up developers' forums. What was the rationale behind that?

When I got into this, the total of my Open Source experience was running a Linux server in my basement. Sean initially released the SLIMP3 software under the GPL, which made it possible for me to help out. I quickly realized that our customers have a lot of talent and energy that they are willing to contribute to make the product better. We're lucky to have a business model where we can make money on the hardware and give the software away. When our customers contribute, they make their own SLIMP3s (and now Squeezeboxes) more valuable to themselves and make the new units we sell to new customers more valuable as well. Everybody wins.

How did you go about building a developer community behind the SlimServer project?

I like to think that the community built itself. We provided a forum (our discussion lists) and Sean and I participate on a daily basis; acting on the requests, suggestions, bug reports, and patches that people post. We're spending more effort lately setting up our own list server and CVS system, as we had problems with our previous forums (Yahoo Groups and SourceForge, respectively), but these will give us a little more control of our virtual spaces.

How do you decide which developer extensions to keep as third-party projects and which to integrate?

If a contribution will make the product better for a substantial portion of our customers and won't make the product worse for any of them, then, generally speaking, it's included. Submitted features that make the software harder or less intuitive to use, require obscure or platform-specific software or hardware to be installed, generally don't make it in. Contributions that are useful to a large fraction of our customers and don't diminish the product's ability to play back music, make it in.

I act as final arbiter and (hopefully) benevolent dictator, and the community has been really supportive of this.

Finally, we notice that you've decided to donate 10 percent of your profits to the EFF. What's the motivation behind this?

Digital freedoms are important to the folks at Slim Devices, both personally and at a corporate level. We've been frustrated by not being able to offer our customers the ability to play the music that they've bought due to digital rights management systems and onerous licensing fees for various patents. The EFF shares our goal to free our customers' music.

The Squeezebox is available from Slim Devices and their resellers, and the server code can be downloaded from their downloads site.

Visit the home of the Perl programming language: Perl.org

Sponsored by

Monthly Archives

Powered by Movable Type 5.13-en