Sign In/My Account | View Cart  
advertisement


Listen Print

Improving mod_perl Sites' Performance: Part 8
by Stas Bekman | Pages: 1, 2

Reducing the Number of stat() Calls Made by Apache

If you watch the system calls that your server makes (using truss or strace) while processing a request, then you will notice that a few stat() calls are made. For example, when I fetch http://localhost/perl-status and I have my DocRoot set to /home/httpd/docs I see:


  [snip]
  stat("/home/httpd/docs/perl-status", 0xbffff8cc) = -1 
                      ENOENT (No such file or directory)
  stat("/home/httpd/docs", {st_mode=S_IFDIR|0755, 
                                 st_size=1024, ...}) = 0
  [snip]

If you have some dynamic content and your virtual relative URI is something like /news/perl/mod_perl/summary (i.e., there is no such directory on the web server, the path components are only used for requesting a specific report), then this will generate five(!) stat() calls, before the DocumentRoot is found. You will see something like this:


  stat("/home/httpd/docs/news/perl/mod_perl/summary", 0xbffff744) = -1 
                      ENOENT (No such file or directory)
  stat("/home/httpd/docs/news/perl/mod_perl",         0xbffff744) = -1
                      ENOENT (No such file or directory)
  stat("/home/httpd/docs/news/perl",                  0xbffff744) = -1
                      ENOENT (No such file or directory)
  stat("/home/httpd/docs/news",                       0xbffff744) = -1
                      ENOENT (No such file or directory)
  stat("/home/httpd/docs", 
                      {st_mode=S_IFDIR|0755, st_size=1024, ...})  =  0

How expensive are those calls? Let's use the Time::HiRes module to find out.


  stat_call_sample.pl
  -------------------
  use Time::HiRes qw(gettimeofday tv_interval);
  my $calls = 1_000_000;
  
  my $start_time = [ gettimeofday ];
  
  stat "/app" for 1..$calls;
  
  my $end_time = [ gettimeofday ];
  
  my $elapsed = tv_interval($start_time,$end_time) / $calls;
  
  print "The average execution time: $elapsed seconds\n";

This script takes a time sample at the beginning, then does 1,000,000 stat() calls to a nonexisting file, samples the time at the end and prints the average time it took to make a single stat() call. I'm sampling a million stats, so I'd get a correct average result.

Before we actually run the script, one should distinguish between two different situations. When the server is idle, the time between the first and the last system call will be much shorter than the same time measured on the loaded system. That is because on the idle system, a process can use CPU very often, and on the loaded system lots of processes compete over it and each process has to wait for a longer time to get the same amount of CPU time.

So first we run the above code on the unloaded system:


  % perl stat_call_sample.pl
  The average execution time: 4.209645e-06 seconds

So it takes about 4 microseconds to execute a stat() call. Now let's start a CPU intensive process in one console. The following code keeps the CPU busy all the time.


  % perl -e '1**1 while 1'

And now run the stat_call_sample.pl script in the other console.


  % perl stat_call_sample.pl
  The average execution time: 8.777301e-06 seconds

You can see that the average time has more than doubled (about 8 microseconds). And this is obvious, since there were two processes competing for the CPU. Now if we run 4 occurrences of the above code:


  % perl -e '1**1 while 1' &
  % perl -e '1**1 while 1' &
  % perl -e '1**1 while 1' &
  % perl -e '1**1 while 1' &

And when running our script in parallel with these processes, we get:


  % perl stat_call_sample.pl
  2.0853558e-05 seconds

about 20 microseconds. So the average stat() system call is five times longer now. Now, if you have 50 mod_perl processes that keep the CPU busy all the time, the stat() call will be 50 times slower and it'll take 0.2 milliseconds to complete a series of call. If you have five redundant calls as in the strace example above, then they add up to 1 millisecond. If you have more processes constantly consuming CPU, then this time adds up. Now multiply this time by the number of processes that you have and you get a few seconds lost. As usual, for some services, this loss is insignificant, while for others a very significant one.

So why does Apache make all these redundant stat() calls? You can blame the default installed TransHandler for this inefficiency. Of course, you could supply your own, which will be smart enough not to look for this virtual path and immediately return OK. But in cases where you have a virtual host that serves only dynamically generated documents, you can override the default PerlTransHandler with this one:


  <VirtualHost 10.10.10.10:80>
    ...
    PerlTransHandler  Apache::OK
    ...
  </VirtualHost>

As you see it affects only this specific virtual host.

This has the effect of short circuiting the normal TransHandler processing of trying to find a filesystem component that matches the given URI -- no more 'stat's!

Watching your server under strace/truss can often reveal more performance hits than trying to optimize the code itself!

For example, unless configured correctly, Apache might look for the .htaccess file in many places, even if you don't have one, and make many unnecessary open() calls.

Let's start with this simple configuration. We will try to reduce the number of irrelevant system calls.


  DocumentRoot "/home/httpd/docs"
  <Location /app/test>
    SetHandler perl-script
    PerlHandler Apache::MyApp
  </Location>

The above configuration allows us to make a request to /app/test and the Perl handler() defined in Apache::MyApp will be executed. Notice that in the test setup there is no file to be executed (like in Apache::Registry). There is no .htaccess file as well.

This is a typical generated trace.


  stat("/home/httpd/docs/app/test", 0xbffff8fc) = -1 ENOENT 
        (No such file or directory)
  stat("/home/httpd/docs/app",      0xbffff8fc) = -1 ENOENT 
        (No such file or directory)
  stat("/home/httpd/docs", 
        {st_mode=S_IFDIR|0755, st_size=1024, ...}) = 0
  open("/.htaccess", O_RDONLY)                 = -1 ENOENT 
        (No such file or directory)
  open("/home/.htaccess", O_RDONLY)            = -1 ENOENT 
        (No such file or directory)
  open("/home/httpd/.htaccess", O_RDONLY)      = -1 ENOENT 
        (No such file or directory)
  open("/home/httpd/docs/.htaccess", O_RDONLY) = -1 ENOENT 
        (No such file or directory)
  stat("/home/httpd/docs/test", 0xbffff774)    = -1 ENOENT 
        (No such file or directory)
  stat("/home/httpd/docs", 
        {st_mode=S_IFDIR|0755, st_size=1024, ...}) = 0

Now we modify the <Directory> entry and add AllowOverride None, which among other things disables .htaccess files and will not try to open them.


  <Directory />
    AllowOverride None
  </Directory>

We see that the four open() calls for .htaccess have gone:


  stat("/home/httpd/docs/app/test", 0xbffff8fc) = -1 ENOENT 
        (No such file or directory)
  stat("/home/httpd/docs/app",      0xbffff8fc) = -1 ENOENT 
        (No such file or directory)
  stat("/home/httpd/docs", 
        {st_mode=S_IFDIR|0755, st_size=1024, ...}) = 0
  stat("/home/httpd/docs/test", 0xbffff774)    = -1 ENOENT 
        (No such file or directory)
  stat("/home/httpd/docs", 
        {st_mode=S_IFDIR|0755, st_size=1024, ...}) = 0

Let's try to shortcut the app location with:


  Alias /app /

Which makes Apache to look for the file in the / directory and not under /home/httpd/docs/app. Let's run it:


  stat("//test", 0xbffff8fc) = -1 ENOENT (No such file or directory)

Wow, we've got only one stat call left!

Let's remove the last Alias setting and use:


    PerlTransHandler  Apache::OK

as explained above. When we issue the request, we see no stat() calls. But this is possible only if you serve only dynamically generated documents, i.e. no CGI scripts. Otherwise, you will have to write your own PerlTransHandler to handle requests as desired.

For example, this PerlTransHandler will not lookup the file on the filesystem if the URI starts with /app, but will use the default PerlTransHandler otherwise:


  PerlTransHandler 'sub { return shift->uri() =~ m|^/app| \
                        ? Apache::OK : Apache::DECLINED;}'

Let's see the same configuration using the <Perl> section and a dedicated package:


  <Perl>  
    package My::Trans;
    use Apache::Constants qw(:common);
    sub handler{
       my $r = shift;
       return OK if $r->uri() =~ m|^/app|;
       return DECLINED;
    }

    package Apache::ReadConfig;  
    $PerlTransHandler = "My::Trans";
  </Perl>

As you see we have defined the My::Trans package and implemented the handler() function. Then we have assigned this handler to the PerlTransHandler.

Of course you can move the code in the module into an external file, (e.g. My/Trans.pm) and configure the PerlTransHandler with


  PerlTransHandler My::Trans

in the normal way (no <Perl> section required).

References