How to divert unwanted traffic from website?

February 7, 2023 7:30 PM

I need help in setting up an .htaccess file to block traffic to my site.

When I was setting up my website, I added a forum with some settings that attracted a lot of traffic. The forum didn't require registration to post and bots went wild bombarding it with spam. I turned the forum off and eventually deleted it and thought that was sorted out. However, I checked in on it recently and the traffic apparently has kept coming.

I am somewhat ignorant about these things, so I'm hoping someone more knowledgeable can help. At the advice of my hosting provider, I enabled access logs. In those logs, I saw several GET requests for http://www.[mydomain].com/forum/[other stuff]/. I've already shut down and removed the forum, so any urls of this type do not exist. My hosting provider says that even with removing the forum, the traffic can still be causing me to be charged and they suggested "divert those away from WordPress with a separate rewrite rule". I'm not sure if this means setting up an .htaccess file or something else. Can anyone help guide me through how to handle this? The traffic is unfortunately causing me to be charged much more for hosting, so I need to find a way to handle this.

posted by NoneOfTheAbove to Computers & Internet (10 answers total)

My solution has been to use a reverse proxy with blacklist/whitelist rules and scripts to extract malicious/spam IPs from the logs to add to the blacklist. (I use relayd and OpenBSD httpd.) I wrote the log analysis scripts but I also use pf-badhost which runs on *BSD and linux and is pretty effective.
posted by sudogeek at 8:21 PM on February 7, 2023 [1 favorite]

Response by poster: I should have mentioned that I'm pretty naïve about this kind of stuff. My hosting is on NearlyFreeSpeech. Is this something I could install there?
posted by NoneOfTheAbove at 8:27 PM on February 7, 2023

I am not an expert, but this StackOverflow post should cover the basics of RewriteEngine

https://stackoverflow.com/questions/17477460/block-urls-in-htaccess-using-regular-expression

You basically change the bits that says {thewordsImustblock)|(morewords)|(yetmorewords)

with just (whatIneedtoblockmyself)
posted by kschang at 8:46 PM on February 7, 2023 [1 favorite]

An easy way and cheap way to do this might be to put the free tier of cloudflare in front of your domain. I dont know offhand if you can use it to block certain paths, but in general its pretty good about blocking suspicious traffic. It may also make your wordpress site faster via caching, which means you may also pay less in hosting costs.
posted by cgg at 10:32 PM on February 7, 2023 [1 favorite]

The htaccess route above should work, but I was going to also suggest the Cloudflare approach above, as it could likely help with your server costs in other ways.

You can absolutely block all paths matching a pattern using a Cloudflare Page Rule. Free accounts get up to 3 page rules.
You could setup a page rule to match:
www.example.com/forum*
and then the setting for that rule could be one of:
"Forwarding URL" (to forward to another page or even another website)
"Security Level = High" (to block even mildly suspicious activity to that section)
posted by Gomez_in_the_South at 3:38 AM on February 8, 2023

Vote #3 for the free tier of Cloudflare.

I use it for a number of sites to solve this exact issue. Create a page rule that matches to any form and have it always trigger a "managed challenge." It is a brief javascript based check that seems to eliminate just about every bot out there.
posted by SegFaultCoreDump at 6:10 AM on February 8, 2023

My hosting provider says that even with removing the forum, the traffic can still be causing me to be charged and they suggested "divert those away from WordPress with a separate rewrite rule". I'm not sure if this means setting up an .htaccess file or something else. Can anyone help guide me through how to handle this? The traffic is unfortunately causing me to be charged much more for hosting, so I need to find a way to handle this.

What are you being charged for? CPU utilization? Bandwidth? Unless you have a massive amount of bots hitting you, serving up a 404 error shouldn't consume that much of either. A rewrite rule isn't going to save you much on the bandwidth side because you're still going to be getting the requests. A well written bot would respect the 404 (at least after a while) and stop making any requests to those URIs but most spam bots are poorly written. I've run big websites and bots would hit old URIs a decade after they'd been discontinued.

If the cost is the bandwidth, Cloudflare is probably your best free bet. It'll get a bunch of the bots from ever connecting to your server.
posted by Candleman at 6:38 AM on February 8, 2023 [1 favorite]

...serving up a 404 error shouldn't consume that much of either.

Using modrewrite/.htaccess to send a 301 redirect should technically be the same amount of bandwidth as a 404 unless you have some fancy custom 404 error page. I don't think the person at the hosting company really understood the problem.
posted by AzraelBrown at 7:46 AM on February 8, 2023 [2 favorites]

With the forum deleted, the bandwidth use will be coming from your 404 page. If you create and publish a 0 byte file, you can then redirect the traffic to that with something like:

redirect 301 /forum/some-page https://yourdomain.com/zero.txt

You can also create a very stripped down 404 page to save on bandwidth.

Also if most of the traffic is coming from a few ip addresses, you can block them completely with a rule like:

order allow,deny
allow from all
deny from 123.456.89.7
deny from 123.456.89.8
posted by Lanark at 7:59 AM on February 8, 2023

Also, do you have a dedicated server or are you on a shared one? Firewall suggestions aren't going to help if you're on a shared system without access (or permission) to block IPs.
posted by Candleman at 8:20 AM on February 8, 2023