John Abbe wrote:
> Well, the NCDD wiki was found, and spammed by a robot before we even
> have gone public. I'm trying to nudge the team off a sudden interest
> in HardSecurity. At the same time, on Wiki:ReverseLinkDisabled
> there's mention of turning away IPs with a high request rate.
> Can anyone offer good starting settings for such a protection - how
> many requests in how short a time to trigger it?

On Sensei's Library (http://senseis.xmp.net/) which is one of the
largest non-Wikipedia wikis I have a 3-step meassure:

* limit requests/minute: anything beyond 30 requests within 60 seconds
and the IP address is disabled for 5 minutes. If after that the maximum
gets exceeded again within an hour then the IP address is disabled for
24 hours.

* shield resource intensive requests (or edit links etc.) by checking
for a HTTP referer header that originates from your site. Effective as
well. Some browsers (privacy proxys, ...) supress the referer header.
Those people have to set a (preference) cookie in order to access those
functions.

* one of the most effective meassures is adding a "trap link". I.e. if
the link is followed the IP address is immediatly added to the block
list (at Sensei's for 48 hours). Mark this link as "Disallow" in your
robots.txt file so compliant robots don't follow the link. At Sensei's
look at source and search for "Blockme" to find the trap link - users
are not able to active it, as it contains no link-text.

In my experience of running this high traffic site, the trap link in
combination with the referer header is most effective. The
requests/minute is only there so that people don't mirror the wiki with
wget or some other tool.

See also: http://senseis.xmp.net/?AccessBlocked

regards
/Arno