Re: [BENCHMARK] Server Resources

[prev] [thread] [next] [Date index for 2005/01/14]

Hey there

It strikes me that your problem is directly related to memory usage.  
Linux machines have a bad problem in that if your apache children get 
swapped out ->for any reason at all<-, they will lose any shared memory 
that they may have had before being swapped out.  When they are swapped 
back in, they take up more RAM than before, so the system swaps out 
another Apache process, and so on, until the system enters the 
"downward spiral of death".

You need to memory-limit your apache processes using Apache::SizeLimit 
to make sure that they NEVER go over a certain size.  Then you have to 
make sure that

MaxClients * Apache::SizeLimit < Available core memory

which will make you miserable because you have MySQL running on the 
same machine.  I would recommend moving MySQL off if you can, which 
will greatly simplify your memory calculations.  If you only have 768MB 
on your machine, then you need to subtract the OS's basic footprint, 
and anything that's running, subtract a bit more for breathing room, 
then divide what's left by the Apache::SizeLimit number that you have 
found (through experimentation and observation) to be the stable size 
of your apache children, and that will give you the MaxClient number.

Then, you pre-load all of your perl modules and as many of your perl 
scripts as possible, and you should be fine.  I have seen this happen 
in almost every mod_perl (<1.99, admittedly) environment I have ever 
worked on.  It takes some tweaking but once you find the magic numbers, 
it will not give you any more problems.

Also, I don't know if this is an option for you, but switching to 
FreeBSD made our lives much much easier, from almost any way you look 
at it.  Food for thought.

I also highly recommend the idea of having two separate servers, a 
lightweight tiny apache that handles static content, and proxies 
dynamic requests to a heavier, mod_perl apache.  You can save huge 
amounts of memory that way, and it tends to increase performance 
significantly.  It's also incredibly easy to set up.

Hope this helps!

Kyle
Central Park Software


> We have recently added a site to our server which hosts 30 other sites.
> The new site uses modperl & MySQL. After adding the new site, the 
> server
> we were on (shared server at a national server farm) had major resource
> problems (too many connections & load avg up to 30 not infrequently) &
> eventually crashed. We moved to a new dedicated server which gave us 
> more
> resources, but we're still experiencing intermittent sluggishness - 
> load
> avg approaching 10 - and this is before this new site really becomes
> active. Maybe 20 logins per day currently - in 10 days we'll have ~ 
> 4000.
>
> I've made changes to Apache config, following suggestions in the 
> mod_perl
> performance tuning docs:
> MinSpareServers 5
> MaxSpareServers 10
> MaxClients=75
> MaxRequestsPerChild 500
>
> I've made efforts to optimize the database & queries, and have made
> changes to the my.cnf file following suggestions in the High 
> Performance
> MySQL O'Reilly book & a couple of postings that seemed to have some
> similarity to our situation:
> set-variable = key_buffer_size=128M
> set-variable = table_cache=1024
> set-variable = join_buffer=1M
> set-variable = sort_buffer=2M
> set-variable = record_buffer=1M
> set-variable = wait_timeout=20
> set-variable = thread_cache=8
>
> But before I go any further with testing & modifying configurations &
> perhaps code, I'd like to have some good idea that the server we 
> currently
> are running on should be able to handle this new site & the 30 others 
> - 2
> running WebGUI, 12 use MySQL (4-5 fairly heavily), 4 modperl.
>
> The server is a Pentium 3 800 MHz with 768M RAM (of which we can use 
> ~640,
> according to the server sysadmin)
>
> It's running Ensim 4.0 with Fedora Core 1 with Embedded Perl version
> v5.8.1 for Apache/1.3.31 (Unix) (Red-Hat/Linux) mod_perl/1.29 PHP/4.3.8
> FrontPage/5.0.2.2635 mod_ssl/2.8.18 OpenSSL/0.9.7a
> MySQL version 3.23.58
>
> Any constructive suggestions are very welcome.
>
> Thanks.
>
> Sys
>