MP2 + Apache2 Memory usage

[prev] [thread] [next] [Date index for 2004/12/15]

We're implementing a mod_perl module for a pretty high traffic site, 
I've run some benchmarks and our mod_perl code, doing the same job as 
our current (proprietry module) is 30-40% faster.  However, the memory 
usage of mod_perl is substantially larger**. 

Having tweaked mod_perl with some preloading and other things a pmap on 
solaris shows our apache processes using 4-5Mb of private memory (mainly 
heap), the full process size is around 20Mb.

The proprietry module we currently have has process sizes of around 
16Mb, but their private data section on pmap is around 200-300k.

I know perl is somewhat more memory intensive, however I remain 
convinced that each of our modules is not using 4-5 Mb of real memory 
(7Mb before I started preloading modules), and that actually most of the 
heap is still actually "copy-on-write" and the perl processes are merely 
sharing their code on the heap, however I have no idea how to actually 
get a dump of the memory status of a process and tell which segments of 
it are still "copy-on-write". 

If anyone can give me any pointers how I might track all this info down, 
or alternatively any good ways of reducing the memory usage any further, 
I would be very grateful.  The issue arises because  we need to run up 
to 800 apache process on each of our frontend webservers at peak times 
(mainly to deal with slow connections), which would probably require (at 
1-2Gb real memory per CPU) 16-32Gb swap space.  Our sysadmin is 
understandably a little nervous about this.


One other issue I had was that I can preload all of the modules in 
startup.pl, however each module still retains the extra memory unless I 
actually remove the uses/requires from the module which is using it.  
This is fine but since we are using objects in perl this causes an error 
(unless I'm missing something obvious, which is possibly the case).

Thanks
Gareth

** Yes the obvious solution to this is to go with threaded, however 
thats the "next step" and requires some more work on making our "other" 
apache modules reentrant.  I'm pretty sure making our perl code 
threadsafe is fairly easy, however we have several other apache modules 
we've converted to apache2, but are still written in C.

        -- 
        Report problems: http://perl.apache.org/bugs/
Mail list info: http://perl.apache.org/maillist/modperl.html
List etiquette: http://perl.apache.org/maillist/email-etiquette.html