Blocking access based on referrer...?

September 3, 2006 12:16 PM

Is it possible to restrict access to a web page (or whole site) based on the referrer?

I noticed that a site I manage (which includes a phpbb forum) has been getting traffic off of the google search "inurl posting.php?mode=newtopic". I'm pretty sure this is one of the ways spammers and/or spambots are finding the forum.

Specific to my problem, is there any way to restrict access to visits coming from that search? Through scripting? .htaccess? etc?

And out of general interest, is there a way to block access based on referrers?

posted by poweredbybeard to Computers & Internet (10 answers total)

.htaccess:

RewriteEngine on
# Options +FollowSymlinks
RewriteCond %{HTTP_REFERER} inurl posting\.\?mode\=newtopic [NC]
RewriteRule .* - [F]

That will block anyone trying to access the site from a URL with the string you mention above. I think.
posted by Civil_Disobedient at 12:35 PM on September 3, 2006

Oh, remove the # sign if your website gives you "500 Service" errors.
posted by Civil_Disobedient at 12:37 PM on September 3, 2006

Response by poster: Awesome, thanks - I'll try that out later.
posted by poweredbybeard at 12:39 PM on September 3, 2006

Note that you should only ever exclude certain referrers, not make a certain referrer a condition of entry. So it's fine to block 'posting.php?mode=newtopic', for example, but don't set it so that everyone coming there has to have a referrer from 'www.yoursite.com/[whatever]', as that will block legitimate traffic that has decided not to send you a referrer for some reason.
posted by reklaw at 12:41 PM on September 3, 2006

Spambots don't give (accurate) referer information.
posted by winston at 1:12 PM on September 3, 2006

Response by poster: Winston - I kind of thought they might not, but I can't think of another reason for why anyone would search for that specific string. All it does is find you a bunch of phpbb boards, and I doubt if the one I admin is even in the first twenty results pages.

But mostly it just made me want to learn if this was possible, and if so, learn how.
posted by poweredbybeard at 2:25 PM on September 3, 2006

Danger, Master Robinson: whenever you mess around with this, make damned sure to test it afterwards. Any mistake whatever will take your entire site down; .htaccess is extremely unforgiving, and the syntax is arcane.

Some syntax errors in the .htaccess file can result in every access to your site returning 503 errors, for instance.
posted by Steven C. Den Beste at 6:10 PM on September 3, 2006

Response by poster: Thanks, Steven. Duly noted. Still haven't had time to get my (virtual) hands dirty yet, but I'll certainly take care.
posted by poweredbybeard at 12:19 PM on September 4, 2006

If you run into any problems, drop me an email.
posted by Civil_Disobedient at 3:35 PM on September 4, 2006

Note that the referrer information can be easily spoofed, and shouldn't be relied on for real security. For example: RefSpoof.
posted by owen at 8:28 AM on September 6, 2006