ISP hijacks invalid URLs—sometimes

April 22, 2008 9:37 AM

NetworkFilter: I have a bizarre and arcane-seeming problem with my home network, the Internet, and my ISP's DNS. Invalid URLs are being redirected to the ISP's search page—in some cases.

If I type in a partial URL like "amazon", normally I'd expect my browser to figure out that I meant amazon.com and go there. And indeed, it worked that way until about two weeks ago.

At that point (with no changes at my end), I started getting redirected to a special page run by my ISP (grandecom.com) that attempts to monetize my laziness by showing me ads. I realize there's been a lot of this kind of thing with a lot of different ISPs lately.

I have two Macs running 10.5, connected via cat-5, and one Mac running 10.4 connected via wifi, all running into the same old-ish Netgear router (which has current firmware). What I've just figured out is that the wifi-connected Mac does not encounter this redirect; when I connect any of my computers directly to the cable modem, I also don't encounter it.

I've spoken to my ISP's tech support a couple of times. One guy had never even heard of this redirect page and expressed surprise that it exists. The other had heard of it, but claimed there hadn't been a lot of complaints relating to it. And because the problem went away when I took my router out of the loop, he's inclined to blame it.

Clearly the ISP's DNS is implicated, and I am not satisfied with this response, or lack of solution. I'm wondering if anyone has encountered anything like this, or can shed light on it.

posted by adamrice to computers & internet (22 answers total)

If you change your DNS servers to Opendns.com (or any free dns server) and you should avoid those search pages + ads.
posted by sharkfu at 9:49 AM on April 22, 2008

I know this doesn't solve your problem but with most web browsers if you type in amazon in the address bar and press CTRL+ENTER, it will automatically add www. and .com to the address.
posted by JaredSeth at 10:02 AM on April 22, 2008

I honestly don't know if it's related or not, but maybe this recent MeFi post can shed some light on what's going on?
posted by carsonb at 10:06 AM on April 22, 2008

Seconding OpenDNS. Great free service.
posted by jaythebull at 10:16 AM on April 22, 2008

Does the ISP's redirect page have an opt-out option? My ISP's page does.
posted by box at 10:18 AM on April 22, 2008

Run your own DNS server if you can. If you have the know-how, set it up and forget about getting shafted by your ISP. Even OpenDNS redirects non-existing domains to their own search page these days.

Link to a wikipedia page listing several DNS servers

I've been running BIND on one of my linux boxes for a while now, and wouldn't settle for anything less.
posted by cyanide at 10:19 AM on April 22, 2008

If I type in a partial URL like "amazon", normally I'd expect my browser to figure out that I meant amazon.com and go there. And indeed, it worked that way until about two weeks ago.

This is pretty inefficient. Your web browser is making a DNS lookup for "amazon," which could resolve either to a local machine or to a really odd domain, and when it fails your browser was smart enough to try tacking on a ".com". Now, your ISP's DNS server is returning their search page for invalid domains.

Options:
1. Find a DNS server that doesn't try to game that system to serve you advertisements.
2. Find a DNS server that games the system more favorably, like OpenDNS does.
3. Stop being so lazy and learn to type a real domain, because such gaming could lead you to a phishing site some day.
posted by mikeh at 11:01 AM on April 22, 2008

If it doesn't do it on the wire, but does on wireless, I would just double check that you are using your wireless connection and not accidentally on some other wireless connection that does this kind of thing.
posted by advicepig at 11:08 AM on April 22, 2008

box: The ISP does offer an opt-out page. Opting out means I still get redirected, but don't get the suggested links. Gee thanks, ISP.

advicepig: the redirect does happen on the wire, not on the wireless, and I've tripled-checked that my wifi connection is to my own node.

mikeh: wow, your answer manages to be both unhelpful and condescending.
posted by adamrice at 11:14 AM on April 22, 2008

Check the network setting on the mac that doesn't do this to see what it is using for a dns server. If it's different than the other computers, you should be able to change the other computers to match.
posted by advicepig at 11:31 AM on April 22, 2008

mikeh has the right answer, even though you don't want to hear it. Take the half-second and type ".com", or whatever. Yeesh.
posted by catkins at 12:34 PM on April 22, 2008 [1 favorite]

The fact that only your 10.5 computers is having this problem raises my suspicions.

Are you having problems not just with the fact that your computer is taking you to these pages, but that it keeps taking you to these pages? Like, for a while typing amazon works. Then, once, it doesn't work -- and every time you try for the next hour, it doesn't work. Then it works again. If so, then see this post, in particular my comment about using dscacheutil -flushcache, for talk about an issue with how Leopard deals with pages it can't find. Essentially, you fix the problem by telling your computer "really, I think that page is there, please try again instead of always dumping me to the error page." On its own it won't try again for an hour.
posted by wyzewoman at 1:48 PM on April 22, 2008

mikeh and catkins: just so you know, this is also happening to me (and others) even when we DO type the ".com".

adamrice: I am having similar problems, and I switched to OpenDNS in an attempt to solve the problem--and just get directed to their search page instead. I actually DO type the ".com" and still have similar problems. The redirecting happens only sometimes, not every time. I actually suspect two things: the OS version and the browser (firefox vs safari, paying attention to the version as well). I'm still trying to track down the actual issue, but clearing cookies worked for me a couple of times, and using a different, fresh (no extensions) Firefox worked once, too. I haven't had the problem again lately but that doesn't mean it's really solved yet! Good luck.
posted by rio at 7:54 PM on April 22, 2008

don't mean to hijack, but wyzewoman: I actually tried that (I saw your post!) and the command wasn't accepted for some reason. I'm not at my mac now to post the actual error, but any idea why the command dscacheutil -flushcache wouldn't work?
posted by rio at 7:58 PM on April 22, 2008

rio: What system are you using? I'm pretty sure that command only works on 10.5; there is an equivalent (but different) command for previous systems. If you're on 10.5, then I have no idea! (You could try "sudo dscacheutil -flushcache", but I don't seem to need that on my my machine...)
posted by wyzewoman at 8:14 PM on April 22, 2008

wyzewoman: I'm on 10.5 and I tried it again when I got home--and it worked this time! No idea what happened before (PEBKAC), I'm just thankful it works now. Thanks!
posted by rio at 1:30 AM on April 23, 2008

My apologies, I didn't mean to be condescending, but I missed the following paragraph (which really puts a weird spin on the situation) and if this is an issue with your ISP's DNS server, then routing your DNS traffic elsewhere or switching providers may be the key. In any case:


I have two Macs running 10.5, connected via cat-5, and one Mac running 10.4 connected via wifi, all running into the same old-ish Netgear router (which has current firmware). What I've just figured out is that the wifi-connected Mac does not encounter this redirect; when I connect any of my computers directly to the cable modem, I also don't encounter it.


Can you check what DNS servers you're using with each system, on the router, and whether you're picking them up via DHCP or if they're typed in? Can you also confirm that the wifi-connected Mac, if plugged into the router, does then get the redirect?

Due to the fact that it's inconsistent across wired and wifi connections, I'd first rule out that it's not a function of the computer and that your wireless connection is actually connecting to your own router. After that, it gets blurry -- does your router have different options for wired versus wireless connections?
posted by mikeh at 8:41 AM on April 23, 2008

And this is really reaching, but the "caching negative DNS responses" issue mentioned by wyzewoman might actually be helping your wireless connection. I've had old-timey modem and wireless connections consistently miss DNS server responses due to packet loss, causing really erratic behavior.
posted by mikeh at 8:44 AM on April 23, 2008

Curiouser and curiouser.

Here's some more/clearer information.
- Any computer plugged directly into my cable modem (bypassing my router) does not encounter this redirect.
- My 10.4 computer, via wifi or ethernet into my router, does not encounter this redirect
- My 10.5 computers, via wifi or ethernet into my router, do encounter this redirect.
- I am definitely, unmistakably, unambiguously connecting to my own router via wifi in each case.
- Tried the dns cache flush. No change.
- All my computers show "192.168.1.1" as the DNS -- obviously this is my router. I got the DNS addresses from my ISP and tried putting them in directly into a 10.5 computer. No change. The router is set to obtain DNS automatically via DHCP. My computers (ordinarily, except when I tried putting in the DNS manually) get all their configuration from my router via DHCP.

So my router is partly implicated, 10.5 is partly implicated, and my ISP is partly implicated. Wired vs wireless connection appears to be a red herring.
posted by adamrice at 9:50 AM on April 23, 2008

adamrice, could you change the settings in your router to have it use OpenDNS instead of pulling the ISP's DNS server automatically?
posted by wyzewoman at 12:01 PM on April 23, 2008

wyzewoman--I have thought of using OpenDNS (or someone else) but that does nothing to solve this puzzle.
posted by adamrice at 10:59 AM on April 24, 2008

I got updated DNS addresses for my ISP and the problem went away.
posted by adamrice at 1:02 PM on January 28, 2009