[Fwd: Re: /proc/*/statm, exactly what does "shared" mean?]

[prev] [thread] [next] [Date index for 2005/02/12]

--=-ajcD/PuMS8nkoqiDWQBj
Content-Type: text/plain
Content-Transfer-Encoding: quoted-printable


Hello Perrin and other modperl users,

I have forwarded an email from Hugh Dickens as a follow up on recent
discussion of mod_perl+apache forked children and copy-on-write pages.

As I had suspected, for our interest, the 'shared' figure represented by
TOP, ps, and other tools (based on /proc/<pid>/statm) does not and never
really has been a useful way to count how 'shared' our apache+mod_perl
processes are.  It seems that in 2.2 and 2.4 that figure at least
included COW pages, but it was polluted with other types such as shared
libraries and mmaps etc.  In recent 2.6 kernels it is more refined, but
does NOT include nor report COW pages.

Whoops, I keep on falling into a Linux only mindset. Let me the clear,
this is NOT necessarily the case with other un*x-es.

If I read between the lines, Hugh's message implies that counting COW's
may be reasonably significant performance hit, thus is why it's not
done.

Perrin, when I get a chance this week, I'll look into the GTop method I
have been using and see if it's based on /proc/<pid>/statm as well and
thus useless for this purpose.

I'll also continue my thread on the Linux kernel list to see if there is
a way to report on COW pages from the command line.

Best,

Richard F. Rebel

-------- Forwarded Message --------
> From: Hugh Dickins <hugh@xxxxxxx.xxx>
> To: Richard F. Rebel <rrebel@xxxxx.xxx>
> Cc: linux-kernel@xxxx.xxxxxx.xxx
> Subject: Re: /proc/*/statm, exactly what does "shared" mean?
> Date: Sat, 12 Feb 2005 13:06:39 +0000 (GMT)
> On Fri, 11 Feb 2005, Richard F. Rebel wrote:
> >=20
> > I can't seem to find clear documentation about the 'share' column
> > from /proc/<pid>/statm.
> >=20
> > Does this include pages that are shared with forked children marked as
> > copy-on-write?
> >=20
> > Does this only reflect libraries that are dynamically loaded?  What
> > about shared memory segments/mmaps (ala shmat or mmmap)?
> >=20
> > If there is a place where I might find documentation that is more clear
> > beyond the proc.txt in the kernel docs and then man pages for procfs,
> > I'd welcome a pointer.
>=20
> You may not be entirely happy with this answer.
> It is a count of "pages of the process" which are "shared" in some sense.
> But precisely what that means has changed from time to time: depending on
> our perception of what we can safely afford the overhead of counting.
>=20
> You may want to look at fs/proc proc_pid_statm() source for the release
> of interest, and follow that back to see exactly what is being counted.
>=20
> Throughout 2.4 (and 2.2 too I think) it was the count of those pages
> instantiated in the process address space which currently have a page
> count greater than 1.  That would include private pages shared with
> forked children, pages from the pagecache (including pages mapped
> from executable or library or shared memory or file mmap), those
> private pages which currently have swap allocated (so they're also
> in the swapcache), and any pages which transitorily have page count
> raised for whatever reason (they'd likely already be in one of the
> above categories).  A ragbag of meanings, but that's all you can
> get from looking at page count.
>=20
> Counting up that not very meaningful number, at frequent intervals
> on large process address spaces, is a waste of valuable time.
>=20
> From 2.5.37 to 2.6.9, it's the total extent of file-backed areas
> (file including executable or library or shared memory) in the
> process address space: a total extent (in pagesize units),
> not a count of instantiated pages.  Much quicker to calculate.
>=20
> But there were complaints about that, and a need to revert from
> total extent to count of instantiated pages.
>=20
> From 2.6.10 onwards, for the foreseeable future, it is the count
> of those pages instantiated in the process address space which are
> shared with a file (including executable or library or shared memory)
> i.e. those pages which are not anonymous, not private.  That count
> does not include private pages shared with forked children, nor
> does it include private pages which happen to have swap allocated.
>=20
> Hugh
--=20
Richard F. Rebel <rrebel@xxxxx.xxx>
WhenU.com

--=-ajcD/PuMS8nkoqiDWQBj
Content-Type: application/pgp-signature; name=signature.asc
Content-Description: This is a digitally signed message part

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.2.4 (GNU/Linux)

iD8DBQBCDg7vx1ZaISfnBu0RAgbJAJwLuRRrE81zsZZl6gR5Hcm7KrdIEQCfbC7g
4v0tY0cp4dPTvQP89ST/DnM=
=Ok8p
-----END PGP SIGNATURE-----

--=-ajcD/PuMS8nkoqiDWQBj--