I sure would love it if they'd stop throttling me.

November 13, 2005 11:48 PM

How can I detect if my ISP is using traffic-shaping software to throttle my P2P connections?

I've noticed in the last while that not one file sharing application is coming close to using my connection to its full potential anymore. The most obvious offender is Bittorrent, which on a torrent w/ 30 seeds and 45 peers connected right now is only getting 13.5 k/sec, while my ISP's speed test gives me a connection speed of about 500 k/sec down, 50 k/sec up.

Obviously this is a bit... off.

So, I realized that one possible reason for this would be traffic-shaping software being run upstream, throttling the speed of connections that are clearly P2P traffic. Now, the question I have is how can I detect this? Does a tool exist to verify it? It'd be better if it were a Linux tool, as I don't have access to a Windows machine anymore, but I could throw something together in a pinch.

In a potentially-related direction, is anyone else experiencing this? The latest Family Guy episode is liable to take 4-5 hours to download, when only a year ago I could get a half-hour episode onto my machine in about 2.5 minutes. Something has changed, and I'm not too happy about it.

posted by ChrisR to computers & internet (21 comments total)

You could try setting the default port for BitTorrent to a different port number. I'd guess only ports above 1024 would be subject to traffic shaping, so you might try a lower port.

If you have somewhere at your disposal to try such things, you could setup Apache on somewhere remote to listen to ports 80 and 6881 and try both of those.

You might also look at using Azureus with a TOR router, and see if that changes anything.
posted by stovenator at 11:58 PM on November 13, 2005

Would the < 1024 port really make a difference? i mean, i'm leery of it because i'd have to run my bt client as root -- not my favourite idea in the world, to say the least.br>
Also: The maintainers of the Tor network have long been on record saying that it is not meant for data traffic for bittorrent -- that kind of volume would kill it. I'm pretty sure that Tor is only used for metadata transmission.

Your middle suggestion, though, is interesting -- what, exactly, do you mean by it? How would I make use of Apache running remotely to tell if my traffic is being shaped? I'm not sure I can run it, but I might be able to work something out.
posted by ChrisR at 12:05 AM on November 14, 2005

If they are using deep packet inspection then switching ports won't do a thing. (And the business about ports below 1024 is bunk as far as I know - if you're going to shape traffic by port you do it by whatever ports are in use by the programs you're going to block, you don't do it based on some ancient and arbitrary "ephemeral / non-emphemeral" distinction.) Bittorrent already has a very diverse port profile because long ago some trackers started blacklisting the stock ports and forcing users to pick a new, arbitrary port. If you look at the ports in use with netstat you will see that it's essentially random these days, because the protocol was designed tol use whatever the end user configures his client for. So, if they are successfully throttling BT traffic then that strongly implies DPI because it would be nearly impossible to effectively block BT traffic based on port alone.

Really the only thing that will absolutely work if indeed your ISP is using DPI is to tunnel your traffic over an encrypted link. This could be as simple as ssh port forwarding, or a full blown VPN. This requires a machine outside of your ISP that you have access to, which provides the endpoint of your VPN and proxies all your p2p traffic. So this is not a very practical solution for most people, unless you know someone that runs a server and doesn't mind blowing their bandwidth, or you can use a machine at your school/work -- though they probably don't want you using their bandwidth for your warez.

Even then, they can mark and throttle all packets that look like ssh / VPN traffic, regardless of their actual payload, which still doesn't make this a slam dunk. (Their logic here is, "if you're connecting to a VPN then you should be paying for the business-class connection anyway", so they throttle away.)

Really if an ISP wants to do bandwidth management and they have invested enough cash in the hardware to do it properly, then there's nothing you can do about it -- at least nothing simple like enable "bypass Packeteer" in options somewhere.
posted by Rhomboid at 1:00 AM on November 14, 2005

And, I should point out that EVEN IF no port below 1024 were throttled, that still wouldn't mean changing your listen port to one below 1024 would avoid throttling. All that you have control over is the port that your client will listen on -- every other user can set this to anything as well. So just because your client is listening on a <1024 port for incoming connections, doesn't change the fact that you'll still have to initiate outbound connect to clients who aren't necessarily on a port <1024.
posted by Rhomboid at 1:06 AM on November 14, 2005

For your ISP to do any kind of traffic shaping, they have limited options. They can limit based on

A. Source or Destination IP address
B. Protocol (TCP or UDP)
C. Packet Content
D. Total Throughput
E. Port number


Option A isn't practical, as they would need to know what IP addresses throughout the world are using PTP.

Option B isn't practical, and doesn't really make any sense. This would kill many other worthwhile programs.

Option C isn't practical, as they would need to open every packet, and determine the content. Hardware equipment to evaluate packets based on content isn't common, so they would need to use a packet sniffer. They likely wouldn't have the CPU to do this.

Option D is possible. If your throughput for a month hits a certain limit, they could limit you. This would likely affect all your traffic, unless they combined this with one of the other methods (i.e. only apply Option C to customers who have surpassed 20GB transfer or similar.)

Option E seems the most likely. Well known HTTP and FTP ports (80 and 21) are not likely to be limited, and you could easily verify this by downloading something big from a fast site (like Firefox, Kernel source, a Linux ISO, etc.). If those are a lot faster, then they probably aren't using lower ports.

My suggestion to try Apache on multiple ports was to try to determine if they are rate limiting BitTorrent ports (or higher ports in general). You could setup Apache somewhere on port 80 and try downloading. Then set it up on 6881 and try it again. If speeds are way faster on port 80, you would know they are rate limiting port 6881.

If that's the case, you can try changing the default BitTorrent port. If they are traffic shaping, I would guess they are only doing it above port 1024, but using a lower port is more of a security risk (root privileges are necessary under 1024 , but you can use setuid). There may be ports above 1024 they aren't rate limiting, but that's likely a trial and error thing (if you determine they are rate limiting at all).

You could also perhaps setup a proxy server, but for P2P or Torrent applications, that would be hairy. Port forwarding and use of the iptables firewall might also be an option.
posted by stovenator at 1:12 AM on November 14, 2005

And what Rhomboid said is correct. If they are inspecting the packets for payload (what I deemed Option C), there isn't much you can do other than an ssh tunnel, or vpn.
posted by stovenator at 1:15 AM on November 14, 2005

Word on DSL reports is that Rogers cable in Ontario is inspecting packets. In fact, the consensus seems to be that Rogers even unthrottles content coming from certain larger torrent seeders that distribute Linux - presumably because it is "legal". Of course P2P music sharing is still "legal" in Canada, for now.
posted by Chuckles at 1:38 AM on November 14, 2005

If your using Shaw: they've been using a Ellacoya switch and throttling BT traffic in Vancouver (was big news awhile back) maybe they've installed one in Edmonton? I'm pretty sure they said they were going to extend it to other areas to "recoup bandwidth". I've had problems with Shaw, a recent torrent posted to the blue had oodles of seeds and peers but I crawled along at about 5 to 10K up and 15K down, was not a happy camper.
posted by squeak at 3:13 AM on November 14, 2005

well i'm wary of saying this in the face of stovenators obvious expertise here (good answer by the way), but i have read (on digg) that using bitcomets "protocol header encrypt" function has brought many users back to their unthrottled high speeds. Your mileage may vary.
posted by kev23f at 5:51 AM on November 14, 2005

Interesting stuff - I've had the same issue recently, but since I've never used P2P before (and hence didn't know how fast it can actually go!) I didn't realise that my ISP could be restricting my traffic...

I shall have a word with them to find out!
posted by Chunder at 7:08 AM on November 14, 2005

squeak writes "If your using Shaw: they've been using a Ellacoya switch and throttling BT traffic in Vancouver (was big news awhile back) maybe they've installed one in Edmonton? I'm pretty sure they said they were going to extend it to other areas to 'recoup bandwidth'. I've had problems with Shaw, a recent torrent posted to the blue had oodles of seeds and peers but I crawled along at about 5 to 10K up and 15K down, was not a happy camper."

Okay, you say that as if it were a past-tense issue, and somehow you've restored your speed to what it used to be. How might that be, assuming I'm correct in my assumption?
posted by ChrisR at 7:13 AM on November 14, 2005

the solution to your problem may be as simple as not maxing out your upstream connection. if you max the upstream, then tcp acks for the incoming data get dropped and cause the tcp sessions to throttle back.

tell your bittorrent client to rate limit the upstream data, and make sure the sum of all upstreams is ~20% less than your measured upstream bandwidth as measured by one of those speed tools and see what happens.
posted by joeblough at 8:59 AM on November 14, 2005

Option C isn't practical, as they would need to open every packet, and determine the content.

They don't have to read very much of the packet to determine its type. They could slow BitTorrent down to a crawl if they wanted to, without having to read every packet in its entirety.
posted by oaf at 9:00 AM on November 14, 2005

... and it turns out that they're doing just that.

I'm seeing a lot of indications that BT has been nailed from one end of Shaw to the other, to a degree that is flat-out offensive. Dialup is faster than this crap.

Okay, well, I guess that's the answer to my question. I'm chatting with the devs for my bittorrent client now to try hash out a way to dodge this. Who knows, maybe a protocol upgrade to BT is coming?
posted by ChrisR at 9:35 AM on November 14, 2005

btw i design networking hardware for a living and its very easy to design hardware that looks at all sorts of packet data. back in 1997 we designed chips that could look all the way into the TCP headers and make decisions at line-rate for gigabit ethernet.

more and more switches are microcoded these days, making it even easier to design custom firmware that can do packet filtering...
posted by joeblough at 9:40 AM on November 14, 2005

Just FYI for all you Shaw users: I am using Telus ADSL in Vancouver and regularly getting 210k down and 80k up, even sharing the connection with two other computers in the house. It looks like Telus isn't throttling yet. They also aren't regularly enforcing bandwidth caps. So while it's not as fast as when I was on Shaw Xtreme in Victoria, at least they don't come after me when I go over 50 GB in a month.

And it is often faster to download TV shows from alt.binaries.multimedia whenever you can.
posted by solid-one-love at 10:41 AM on November 14, 2005

Fixed it? Err ... nope. I wish I new of a workaround but never found one.
posted by squeak at 11:06 AM on November 14, 2005

Huh, so that's why BT is so slow (fellow Shaw subscriber). I pay for a premium usenet service and I get between 512k - 768k downloads from them through Shaw's pipes.

But that 50 GB cap's kind of a pain.
posted by PurplePorpoise at 11:48 AM on November 14, 2005

I didn't realize they were actually doing that these days. Huh. While I've had my head buried in the sand, they're getting more invasive every day. Guess it's time to start reading some more.
posted by stovenator at 12:22 PM on November 14, 2005

Another Telus DSL user, interior BC: 150kb/s down, 50kbs up, and downloads are not capped nor charged at this point. Telus desperately wants into the media distribution business, so I expect it believes the enemy of its enemy (torrent users vs. cable media, aka Rogers and Shaw) is its friend. I use Azureus, and I have manually set the port to a very high number.
posted by five fresh fish at 3:45 PM on November 14, 2005

Option C isn't practical, as they would need to open every packet, and determine the content. Hardware equipment to evaluate packets based on content isn't common, so they would need to use a packet sniffer. They likely wouldn't have the CPU to do this.

Sorry but that may have been true 10 years ago but it is most definitely not true now. There are dozens of companies (such as Check Point, Netscreen, Packeteer) that offer hardware solutions to do just this. They do everything in hardware and can analyze traffic at line rate. You can essentially buy one of these boxes, plug it into your network, tell it what p2p apps to block and then walk away. They are terribly effective, because they do not depend on what port the application is using, and they come with a comprehenisve list of almost all known types of traffic. With p2p traffic approaching three quarters of all internet traffic, organizations such as universities, high schools, corporations, and even ISPs are buying these things like hotcakes. It is most definitely not "uncommon".
posted by Rhomboid at 9:55 PM on November 15, 2005