AW: Logging user's movements

[prev] [thread] [next] [Date index for 2005/02/04]

Hi Leo,

I have a very similar app running in mod_perl with about 1/2 mio hits a day.=
 I need to do some optimisation, so I'm just interessted what optimisations t=
hat you are using brought you the best improvements.
Was it preloading modules in the startup.pl or caching the 1x1 gif image, or=
 maybe optimising the database cache ( I'm using mysql ).
I'm sure you are also having usage peaks, so it would be interessting how ma=
ny hits(inserts)/hour can a single server  machine handle approx.


Thanks

Denis





-----Urspr=FCngliche Nachricht-----
Von: Leo Lapworth [mailto:ranguard@xxxxx.xxx]=20
Gesendet: Freitag, 4. Februar 2005 10:37
An: ben syverson
Cc: modperl@xxxx.xxxxxx.xxx
Betreff: Re: Logging user's movements


H
On 4 Feb 2005, at 08:13, ben syverson wrote:

> Hello,
>
> I'm curious how the "pros" would approach an interesting system design =20
> problem I'm facing. I'm building a system which keeps track of user's =20
> movements through a collection of information (for the sake of =20
> argument, a Wiki). For example, if John moves from the "dinosaur" page =20
> to the "bird" page, the system logs it -- but only once a day per =20
> connection between nodes per user. That is, if Jane then travels from =20
> "dinosaur" to "bird," it will log it, but if "John" travels moves back =20
> to "dinosaur" from "bird," it won't be logged. The result is a log of =20
> every unique connection made by every user that day.
>
> The question is, how would you do this with the least amount of strain =20
> on the server?
>
I think the standard approach for user tracking is a 1x1 gif, there are =20
lots of ways of doing it, here are 2:

Javascript + Logs - update tracking when logs are processed
------------------------------------------------------------------------=20
---------

Use javascript to set a cookie (session or 24 hours) - if there isn't =20
already one. Then use javascript to do a document write to the gif.

so /tracker/c.gif?c=3D<user_session_id>&page=3Ddinosaur

It should then be fast (no live processing) and fairly easy to extract =20
this information from the logs and into a db.

Mod_perl - live db updates
-------------------------------------
Alternatively if you need live updates create a mod_perl handle that =20
sits at /tracker/c.gif, processes the parameters and puts them into a =20
database, then returns a gif (I do this, read the gif in and store it =20
as a global when the module starts so it just stays in memory). It's =20
fast and means you can still get the benefits of caching with squid or =20
what ever.

I get about half a million hits a day to my gif.

I think the main point is you should separate it from your main content =20
handler if you want it to be flexible and still allow other levels of =20
caching.

Cheers

Leo


____________
Virus checked by G DATA AntiVirusKit
Version: AVK 15.0.2702 from 26.01.2005
Virus news: www.antiviruslab.com