We've expanded our Perl news coverage and improved our search! Search for all things Perl across O'Reilly!

Improving mod_perl Sites' Performance: Part 4
by Stas Bekman | Pages: 1, 2, 3, 4, 5

Calculating Real Memory Usage

I have shown how to measure the size of the process' shared memory, but we still want to know what the real memory usage is. Obviously this cannot be calculated simply by adding up the memory size of each process because that wouldn't account for the shared memory.

On the other hand we cannot just subtract the shared memory size from the total size to get the real memory usage numbers, because in reality each process has a different history of processed requests, therefore the shared memory is not the same for all processes.

So how do we measure the real memory size used by the server we run? It's probably too difficult to give the exact number, but I've found a way to get a fair approximation, which was verified in the following way. I calculated the real memory used by a technique you will see in the moment, and then stopped the Apache server and saw that the memory usage report indicated that the total used memory went down by almost the same number I've calculated. Note that some OSs do smart memory pages caching so you may not see the memory usage decrease as soon as it actually happens when you quit the application.

This is a technique I've used:

1. For each process sum up the difference between shared and system memory. To calculate a difference for a single process use:

   
     use GTop;
     my $proc_mem = GTop->new->proc_mem($$);
     my $diff     = $proc_mem->size - $proc_mem->share;
     print "Difference is $diff bytes\n";

2. Now if we add the shared memory size of the process with maximum shared memory, we will get all the memory that actually is being used by all httpd processes, except for the parent process.
3. Finally, add the size of the parent process.

Please note that this might be incorrect for your system, so you use this number on your own risk.

I've used this technique to display real memory usage in the module Apache::VMonitor (see the previous article), so instead of trying to manually calculate this number you can use this module to do it automatically. In fact in the calculations used in this module there is no separation between the parent and child processes, they are all counted indifferently using the following code:

  use GTop ();
  my $gtop = GTop->new;
  my $total_real = 0;
  my $max_shared = 0;
  # @mod_perl_pids is initialized by Apache::Scoreboard,
  # irrelevant here
  my @mod_perl_pids = some_code();
  for my $pid (@mod_perl_pids)
    my $proc_mem = $gtop->proc_mem($pid);
    my $size     = $proc_mem->size($pid);
    my $share    = $proc_mem->share($pid);
    $total_real += $size - $share;
    $max_shared  = $share if $max_shared < $share;
  }
  $total_real += $max_shared;

So as you see we that we accumulate the difference between the shared and reported memory:

    $total_real  += $size-$share;

and at the end add the biggest shared process size:

  $total_real += $max_shared;

So now $total_real contains approximately the really used memory.

Pages: 1, 2, 3, 4, 5

Next Page