Re: Question: Limiting Filetypes Uploaded

[prev] [thread] [next] [Date index for 2005/03/03]

--=-rnZgF1DwUGCSnurQgIlg
Content-Type: text/plain
Content-Transfer-Encoding: quoted-printable

On Wed, 2005-03-02 at 23:48 -0500, jonathan vanasco wrote:
> On Mar 2, 2005, at 7:21 PM, Richard F. Rebel wrote:
> > I worked for a company who did mass free hosting and I can tell you=20
> > that
> > browser supplied mime types are not enough in many situations.
> I agree - my question was if they tended to call a 'valid' file=20
> invalid, or an invalid file valid -- the idea being that if the browser=20
> claimed something was a pdf, and I only want jpgs, its safe to throw=20
> out without doing additional testing (ie the magic byte to image app=20
> tests).   if the browser says its a jpg, then i do actual file analysis=20
> to see if it is indeed a jpg. (the idea being to conserve cpu)

yes, I would say yes, it's safe, except in the case of octet-stream
which you might permit and test afterwards.

> > I had to resort to file magic byte testing (technique used by the unix
> > 'file' command), and then further to ensuring that tar's, rar's,=20
> > bzip's,
> > pkzip's etc all passed consistency checks and had no padded data on the
> > end, as well as check the contents.  We also ended up having to run
> > JPEG's and GIFS through image libraries to ensure the same as people
> > were uploading copies of windows in hundreds of JPEGS which had proper
> > headers but the remainder was added on CAB files.
> That sounds awful, and unfortunately like something I must do.
>=20
> > Also, if I remember correctly you must write a mod_perl content handler
> > in order to acurately stop a 'claimed' 100k upload that is in actuality
> > a 100MB upload.  We did this by counting bytes as we read them and
> > aborting once over a given size.  Also, I think some browsers don't
> > correctly report the size up uploaded files, but I am kinda hazy on=20
> > this
> > aspect.  This was apache 1.3 btw.
>=20
> Ok, this is a more general question then -- shouldn't POST_MAX handle=20
> that?

For absolute max, yes. Depends on your quota needs.

> > 2 ensure they cannot upload larger than claimed/quota amounts by a
> > writing a handler in mod_perl or c that counts the bytes.  TIME OUT=20
> > slow
> > connections... eg, upload 100k, wait, slow down, wait some more, send
> > some bytes, etc. this is an easy way to DOS you.
> ok, now i'm scared.  does anyone know of a handler out there that does=20
> something like this?  or can point me in the right direction on how to=20
> make one?  i def. don't have the knowledge  to code this alone, but=20
> would be eager to  work on one.

The module I wrote is owned by my prior employer, sorry.  I don't know
of any other module either.  It wasn't that hard to write tho.

The part about timing of connections is relatively simple as well, if
you don't get X bytes over X amount of time, the connection is too slow.
If you don't get ANY bytes over N amount of time, connection is dead/too
slow.  Stuff like that.  In apache 1.3 you can use alarm and/or selects.
In Apache2 with a threaded mpm you are limited to selects.

> all of  this upload stuff is for a 'profile' image for  a community=20
> style site i'm working on.  i'm only allowing limited numbers of=20
> uploads, but you've alerted me to a whole area of  problems that are=20
> best addressed by doing this right and building handlers/etc to prevent=20
> issues like  this from possibly arising.

Well, maybe I sounded a little paranoid.  If you cover the 'basic' stuff
you'll rule out most of it.  I worked for a large hosting site that had
millions of free web sites, it was very attractive to try and abuse them
as they had lots and lots of resources and bandwidth.

--=20
Richard F. Rebel

cat /dev/null > `tty`

--=-rnZgF1DwUGCSnurQgIlg
Content-Type: application/pgp-signature; name=signature.asc
Content-Description: This is a digitally signed message part

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.2.4 (GNU/Linux)

iD8DBQBCJztpx1ZaISfnBu0RArLqAJ9/73pzhYVL4yldXF/+ToATm07JlQCaAr6r
KpEsy1zun37zv6GtIOuqnZ4=
=WhtR
-----END PGP SIGNATURE-----

--=-rnZgF1DwUGCSnurQgIlg--