Block every shred of traffic from a referrer

February 4, 2008 7:45 PM

How do I block absolutely all traffic from a specified referrer?

I've got a handful of referrers I need to block completely - even pings. I want anything from the referrer ignored, and not to register as a single byte of traffic, if possible.

I've put PHP and .htaccess blocks in place, but they still seem to register a sliver of traffic on each hit.

Any way to block these completely? I have a dedicated server running Fedora, so I can install whatever I need wherever it needs to be.

posted by JWright to computers & internet (18 comments total)

What do you mean? A referrer is a website that points to your website. So if you click, for example this link the referrer will be this.

That is, your browser with your IP address will send an HTTP request to google (using TCP). One of the headers will read (something like)

   referrer: http://ask.metafilter.com/82811/Block-every-shred-of-traffic-from-a-referrer

However, if you took the link, and pasted it into your browser's address bar, it would not send a referrer header, and Google wouldn't know where you got the link from.

Most protocols don't even have a way of sending a referrer, and ping certainly doesn't.

Did you just mean you want to block all traffic from other hosts? Meaning, if you have the IP Address of machine A, you want to prevent A from even pinging you?

That should be easy to do using a personal firewall. What OS are you using? I think Vista has that feature built in.
posted by delmoi at 7:56 PM on February 4, 2008

Referrers are a feature of HTTP. In order to detect where someone is coming from in order to reach your server (their referrer), this bit of the HTTP header has to be read by your server. By definition, by completely blocking "referrers" you would, by definition, be unable to detect where people are being referred to you from.
posted by Jimbob at 7:57 PM on February 4, 2008

Maybe delmoi explains it better. Referrers. That word. I do not think it means what you think it does.
posted by Jimbob at 7:58 PM on February 4, 2008

Sounds like what you want is a firewall rule/setup that specifically blocks ip addresses from hitting your site.

Here's an easy iptables generator script i use when i have to venture into iptables land.

@delmoi: he mentioned fedora on a dedicated server in the OP.
posted by jbroome at 8:07 PM on February 4, 2008

Yeah, I'm not sure about the referrer thing, but if you just want to deny all traffic sourcing from a particular IP address, you can just put an entry in /etc/hosts.deny like this:

ALL: 12.34.567.789

...using the IP address you want blocked.
posted by team lowkey at 8:08 PM on February 4, 2008

If you're using Fedora, you're probably already running iptables. The straight forward thing to do would be to put the rules in there. I'd be hesitant about putting too much in hosts.deny - I've always treated it as a place for broad server use configuration (i.e., blocking all traffic from outside the local network).
posted by devilsbrigade at 8:56 PM on February 4, 2008

Nope. It's not IP addresses. It's referrers. It's not traffic coming from a web site. It's traffic coming from an app that hits web sites. So it's multiple IPs. The only way I can identify them in my logs seems to be by referrer.
posted by JWright at 9:16 PM on February 4, 2008

The same directives that go in .htaccess can go in your apache config files. If it really is referrer (which seems strange, because that's an incredibly easy one to change) a line in the apache config should take it out completely.

If that fails, you can make your firewall block any packets with that string, though that's quite a bit more invasive and work-intensive to set up.
posted by devilsbrigade at 9:26 PM on February 4, 2008

OK, so you want to block all traffic at the firewall from any IP address that makes a request at your Web site with a certain Referer header. In .htaccess:

RewriteEngine On
RewriteCond %{HTTP_REFERER} www.bad-referer.com
RewriteRule .* /block-ip.php [L]

and in block-ip.php:

<?
system("sudo /sbin/iptables -I INPUT -s ". $_SERVER["REMOTE_ADDR"] . " -j DROP 1>&2");
header("Location: ". $_SERVER["HTTP_REFERER"]);
?>

Of course you'll have to edit /etc/sudoers to allow the httpd user to sudo /sbin/iptables with NOPASSWD.
posted by nicwolff at 9:43 PM on February 4, 2008

PS: I didn't test that, and I don't really know PHP.
posted by nicwolff at 9:43 PM on February 4, 2008

Looks to me like what nicwolff suggested would permaban all traffic from the same host that the objectionable web app was run from, so go carefully if that's not what you want.
posted by flabdablet at 10:22 PM on February 4, 2008

I've got a handful of referrers I need to block completely - even pings. I want anything from the referrer ignored, and not to register as a single byte of traffic, if possible.

What are you trying to accomplish here? Are you just trying to keep your traffic bandwidth levels down? There isn't any way to do that, since you have to at least read enough of the HTTP request to tell what the referrer actually says. And absent some serious apache hacking I'm pretty sure you need to read the whole request in order to do that. That's going to take some bytes.

Other then that, Nicwolff's solution should prevent the machine that hit you from ever hitting you again (I'm assuming, I can tell what it's supposed to do but I haven't, like, tested it). If that's what you want.
posted by delmoi at 10:30 PM on February 4, 2008

flab, the RewriteRule->iptables hack will block traffic not from the host running the offending Web site, but from any host that says it is following a link from that site. (Unless I fucked it up!) And it's the only way to do it that will block "even pings" as JWright requested - he clearly doesn't just want to block the referred requests themselves.

delmoi, the RewriteRule would save almost all the response traffic, since all you're sending is a Location header redirecting the user back to the referer site. And a GET request is normally headers-only, so that's not a lot of traffic. And it'll save connections, too, since the user won't be making the image or object subrequests that the page would have triggered if it had been sent.
posted by nicwolff at 12:15 AM on February 5, 2008

Yeah. My point was really that not only would it block computers that got to JWright's server via such sites, it would leave them blocked - permaban them.

All it would take, for my computer to become totally unable to communicate with JWright's site forever, is for me (or somebody else using my computer, or even another computer behind the same NAT router) to visit one dodgy web page once.

That strikes me as overzealous. It seems to me that what JWright already has in place - web server rules that cause him not to respond to requests bearing the undesired Referer header - is probably about as good as he'll get, and he'll just have to live with the sliver of traffic caused by the fact that the requests have to actually arrive before he can decide how to respond to them.
posted by flabdablet at 12:54 AM on February 5, 2008

Blocking by referer is famously fraught with gotchas. Referer doesn't have to be emitted by a client, and there's various add-ons for browsers that'll prevent it from going out on any request. Having said that, good luck!
posted by boo_radley at 9:24 AM on February 5, 2008

flab, he knows how to block HTTP requests by referer, and he asked how to block the traffic "absolutely" and "completely". If he's getting DOS'ed, blocking all traffic from those hosts is better than being unavailable to all hosts, even sporadically. And it's not "forever", he can watch his traffic at the interface or the router and purge his iptable rules when the attack is over.
posted by nicwolff at 10:16 AM on February 5, 2008

Cannot begin to describe just how potentially dangerous it would be to allow your web server user to execute iptables.

You're one remote execution flaw in some crappy PHP app like Wordpress away from someone dropping all your rules. Or maybe adding '0.0.0.0' to the ban list. That'd be fun.
posted by genghis at 5:13 PM on February 5, 2008

True, and a good point - better to queue the IPs to be blocked and let a root daemon or cron job sanity-check them and call iptables.
posted by nicwolff at 5:40 PM on February 5, 2008