Help me stop referral log spam

August 18, 2006 8:10 AM

Referral log spam. I've been noticing some strange things..

I know there is a wealth of info on this phenomena on the internets but I'd like your perspective/advice. My host uses AWStats to track referring sites to my weblog and lately I've noticed a huge uptick in the number of other web logs linking to me. This is weird because I offer no compelling content nor do I have any porn or mp3s posted. Its just plain ol' text with some Flickr pics. When I visit the sites that have apparently linked to me, they look like legit weblogs but I'll be damned if I can find a link to me. I'm going on the assumption that they are doing something funky to boost their own site or something else nefarious is happening. How do they do this? Is there any way to prevent it? Any ideas would be appreciated.

posted by KevinSkomsvold to Technology (9 answers total)

Is it possible they are hotlinking (or "leeching") to some image(s) on your site? If an image or file is showing a disproportionately high number of requests, that's probably the culprit.
posted by justkevin at 8:22 AM on August 18, 2006

A referrer is just a small string of text that is sent by your browser to the site you visit. If you were to use a custom tool (and spammers do), it would be trivial to set the referrer to whatever you wanted. In fact, there are a number of web downloading tools that will allow you to specify a referrer, as some sites won't allow you to download something unless you activated the link from their page.

Spammers use these techniques in order to get in your referrer logs and hopefully be displayed to the world, kind of like getting free advertising, not to mention it would count as a link to their page and probably affect PageRank and so on.

As far as fixes, I'm not sure. If your logs aren't publically viewable and you have no "top referrers" page on your blog it probably has no effect. Otherwise there probably is some blog spam detectors that would handle this type of thing, but I'm not into the blog scene so I wouldn't know where to point you.

On preview: yes, an image link could definitely cause this also. Go to the page that referred to yours, view the source and search for your blog url (preferably part of it) there. There should also be some way to know exactly what page was requested from your blog with that referrer. Not sure about AWStats, I know I just go to my raw logs to get that info when I need it.
posted by splice at 8:23 AM on August 18, 2006

Response by poster: Good point kevin. I double-checked and remembered that I have hot-linking disabled in cPanel (with the exception of a few sites I use).
posted by KevinSkomsvold at 8:25 AM on August 18, 2006

Response by poster: Interesting splice. I was afraid it was that easy :(

So it would be pointless of me to email the offending sites and ask them what they are linking to since they're probably unaware?
posted by KevinSkomsvold at 8:30 AM on August 18, 2006

Kevin: chances are they are VERY aware, and are in fact the spammers doing it, and won't give a fig if you email them or not. Do the so called "real" blogs referring to you have Google Ads on them? Some sites make their referral logs public "ie a 'last 5 referrals' tab", so this is why spammers will just bulk-referral-spam every blog they can find. My referral logs are JAMMED with tons of these faux-blogs with their google ads. It's annoying.
posted by antifuse at 9:08 AM on August 18, 2006

Best answer: Typically when you have someone hotlinking to an image, awstats will report 0 pages and ### hits for a picture etc under the "links from an external page" section.



As I look through my referrer logs from awstats, apparently I was linked from the homepage at www.disney.com for a while... enough to get 18 hits. :P

The Bad Behavior plugin has done good things for stopping a lot of my spam recently. You might check it out.
posted by jeversol at 9:10 AM on August 18, 2006

Realistically, this is the reason why referrer (or "referer", in misspelled-RFC-speak) logs are now useless, and won't ever be useful again. There's no good way to prevent referrer spam unless you are (a) willing to modify your webserver process to implement some sort of check on the referrer string, and (b) willing to take the processor and network traffic hit that such a check would necessitate. In the end, it's not worth it, not by a long shot.
posted by delfuego at 9:27 AM on August 18, 2006

Also, some browsers (older versions of IE) have a bug where you will get the referer of a the page the user's currently looking at when they click a bookmark or even type your URL in.
posted by kindall at 10:14 AM on August 18, 2006

Response by poster: Thanks jeversol. Since I use WordPress, I'll give that plugin a try.

Delfuego. Yeah, I was afraid of that.
posted by KevinSkomsvold at 10:40 AM on August 18, 2006