How fat it was, how fat

April 11, 2007 10:19 AM

Help me understand bandwidth, servers and net speed in general please!

I'm looking around for a colo just now, and pricing at the bottom end of the scale. Some places price at 1Mbit/s connections, others at 100GB/mo. On this, I am confused. How do the two relate: could I get the 100GB at varying rates, or is it simply another way of expressing the same thign?

Additionally, 1Mb seems slow: My home broadband (ADSL) for instance, is 8Mbit down, 1Mbit up. Serving from it is way too slow. Is this the same sort of speed I'd be buying from the 1Mbit connection service? T1 lines are 1.5Mb/s, aren't they? They used to be spoken of in hushed tones of awe as being killer for speed -- surely they're not the same thing as my modem is trickling up the pipe? (Ignoring contention ratios here)

Can you clear this up for me?

posted by fightorflight to computers & internet (17 comments total)

When they say 1Mbit/s what they're talking about is the cap on transfer speed. A 100GB/mo plan is a statement about billing, more or less. It doesn't tell you how fast data will go out; what it says is that when you've reached 100 GB either they charge you more, or they shut your site down until the beginning of the next month.

If the only limit they quote you is 1Mbit/s, then the presumption is that you could saturate that 24 hours per day for the entire month. Myself, I have 768 Kbit/s with no monthly cap I've ever been told about, which if you do the math means I top out at about 8 gigabytes per day. It isn't possible for me to exceed this. But that also means I don't get hit with extra charges, and I don't get my site shut down near the end of particularly popular months.

1 MB/s is a lot faster than you realize, but whether it's adequate for your site depends entirely on what you're going to be hosting and how many visitors you expect.
posted by Steven C. Den Beste at 10:29 AM on April 11, 2007

Thanks, that helps. You said 1MB/s is faster -- do you mean a megabyte a second? I'm sure that would be fast enough, yes. The quotes I see are for 1Mb/s, which is similar to my ADSL and is really too slow even just for me browsing my home server from outside.

Of course there may be other bottlenecks in that pipe.
posted by fightorflight at 10:38 AM on April 11, 2007

For reference, our school's Internet connection is about 3Mbps. We top out at about 30-40 simultaneous connections and host two web servers. As long as our mail server doesn't get flooded, we rarely have slow network speeds. However, it's not very reliable for real-time video conferencing and distance-learning on a large scale basis which will present problems very shortly.
posted by jmd82 at 10:46 AM on April 11, 2007

The advertised upstream speed on your DSL is likely the maximum - have you checked to see that you can actually attain this speed? I know a lot of tests out there test download speed, I don't know how many test upload speed. If you have access to an remote server with good download bandwidth you might try to upload a large file and time it, and figure the actual speed. 1Mb/s is not blazingly fast but it's OK for a lot of things (like, it's probably OK for a medium traffic web site formed of html, but not great for transferring lots of large files)
posted by RustyBrooks at 10:47 AM on April 11, 2007

SCDB has it right. However, the hosts I've worked with (Serverbeach, EV1) quote in both the speed of the connection and your monthly cap.

FWIW, in my experience, 100Mbit is much more responsive to clients than 10Mbit, even when there isn't that much bandwidth being used. Granted, this could be due to a number of factors, including how loaded your provider is, your connection on the client end, etc. But on both my home connection (8Mbit down, 384Kbps up) and work (multiple T1s) I've noticed a big difference. This is on some pretty large sites, though, and your needs may vary.

On preview, SCDB probably means 1Mbit/s (small b, bit).
posted by kableh at 10:47 AM on April 11, 2007

What are you serving, what are your existing traffic patterns?

As Steven mentioned, there are lots of ways to price "bandwidth"

It used to be that it was priced on the 95 percentile of traffic. These days it's common to see it priced either on the maximum transfer speed permitted, or the total amount of data transfered. In either case, there might be other variables. Sometimes transfer speed limited plans also have some total monthly data transfer limit. More often, total transfer limit plans will let you pay extra for higher peak transfer rates.

A 1 Mbps pipe in a colo situation should perform better than the a 1Mbps up ADSL connection, Even so, for applications like serving a web site, it is probably too slow because your viewers will probably be able to receive data at a faster rate, and you won't be able to deliver to them.

If you are going to be serving a lot of larger files, where a second here or there isn't going to make much difference, then buying by the size of the straw can make sense as long as it's desirable to throttle traffic to limit your costs.

t1s are 1.5Mbps per second. They were fast, a long time ago. They still have advantages over DSL in that they are symmetrical, lower latency, and aren't typically as oversold as cheaper connections. A colo better have a lot more than a couple T1s coming into it, though I imagine some still sell connectivity in T1 equivalent increments.

Depending on your application, skill level, and dedication to being a sysadmin, you might want to look at high volume shared hosting, semi-managed hosting, or managed hosting, rather than putting your own box in colo
posted by Good Brain at 10:59 AM on April 11, 2007

We're currently serving a lot of plain text, but nothing more -- no images or big files. We go through about 35GB a month, according to the host's estimates.

I'm looking at colo mostly because we are already running on hardware we own -- but the donated pipe we use is in peril.

From what I gather here, a host that talks about 100GB/mo is not telling me what sort of speeds I can expect, right? Whereas the 1Mb/s people are guaranteeing a (relatively slow) speed.
posted by fightorflight at 11:36 AM on April 11, 2007

Hi. I used to sell colo for a living.

The 1 meg is probably your constant rate, meaning similar to the 8 up, 1 down that you have at home. The place I worked for priced it at 95th percentile, meaning that if you made a graph of your network usage, we would throw out the top 5% of usage. So say your machine does a steady rate of 512 MB and has a couple of spikes to, say, 2 MB, as long as 95% of your usage was below 1MB, then you don't pay the dreaded overage cost. 1MB usually refers to the up and down stream.

By 1000 GB, the colo probably means total usage for the month. 1000 GB refers to the total amount of data transfer, up and down from your sever.

These number definitions may vary from colo to colo.
posted by frecklefaerie at 11:39 AM on April 11, 2007

Please just remember the following about units, and understand you may need to ask for clarification if something sounds too good to be true:

1 MB/s = 8 Mb/s

1 MB/s = 8192 Kbps

Read aloud, one megabyte per second is equal to eight megabits per second.
posted by odinsdream at 12:15 PM on April 11, 2007

Does this site really need it's own hardware? You can get very reliable shared hosting for $10/month that has 40+ GB of monthly transfer and can burst at 10-100Mbps, can run PHP & Perl apps, etc AND you don't have to worry about admining the machine. Windows, Java, Python, etc hosting are also available for similar prices.

To answer your last question, you should look at the providers SLA and terms of service to find out what they are really guaranteeing. Someone who is quoting you 100GB/month may burst data at 100Mbps, or even 1Gbps, or they may be cagey and cap the transfer rate so that it will be impossible to reach 100GB/month with typical web traffic. A lot depends on their pricing and their business model.

Just as one example, we had an unmanaged server at serverbeach for serving large media files. It came with 2TB of transfer a month. We paid a little extra for a 100Mbps connection, which they did nothing to throttle. I think we did most of our 2TB in the space of a couple of days when we got a big traffic spike. They didn't cap.
posted by Good Brain at 12:17 PM on April 11, 2007

Yes, that was a typo. 1 megabit per second is faster than you realize, but it might not be fast enough for you if your site is carrying video or if you're expecting tens of thousands of visitors per day.
posted by Steven C. Den Beste at 12:17 PM on April 11, 2007

Also, you can test your actual transfer rate with Iperf, which is a little tool that you can run as a server at home. On another outside system, run the tool in client mode to connect to the server you started and you can run several types of bandwidth tests. You can have the program report your transfer rate in convenient units, as well.

This may give you a better idea of whether your assumption that your DSL is too slow is accurate. You may only be getting a fraction of the actual rate you were promised at sign-up.
posted by odinsdream at 12:26 PM on April 11, 2007

Does this site really need it's own hardware? You can get very reliable shared hosting for $10/month that has 40+ GB of monthly transfer and can burst at 10-100Mbps, can run PHP & Perl apps, etc AND you don't have to worry about admining the machine. Windows, Java, Python, etc hosting are also available for similar prices.

It definitely needs a dedicated server for software reasons, including apache jiggery-pokery. I looked at VPSs, but they were offering lower specs than our current box (particularly in terms of disk space) for a bit more cash than colo-ing our existing box. Should I shop around for a VPS a bit more?

Thank you very much everyone who answered so far, I really appreciate it!
posted by fightorflight at 12:44 PM on April 11, 2007

This may give you a better idea of whether your assumption that your DSL is too slow is accurate. You may only be getting a fraction of the actual rate you were promised at sign-up.

I can't stress this enough -- what that means is that your DSL line is shared pipe. Ok, pipes analogy: You've got a small, high pressure pipe running out of your house for sewage. You've got a storm drain system running out front of your house for storm water.

In a big long thunderstorm, (i.e. when everyone's up at midnight surfing porn and downloading movies on bittorrent), your storm sewer will likely back up into the street.

However, even if everyone flushed their toilet at the same time, your sewage line is very unlikely to back up.

The storm seweres are your DSL line -- it's common with everyone in your neighbourhood, and even though you were promised 8 megs down and 1 meg up, that's the MOST you'll ever get -- and everyone on your local loop has to share that. (Of course, they don't tell you this when you're spending $50/mo for the service. That's what we call marketing, and it's similar to what you flushed down the toilet.)

The high pressure sewage lines are what you get from a colo or from a dedicated T-1 line -- that means that they guarantee that you will (except in the event of an accident) be able to use that bandwidth. 1 meg up/down on a dedicated line is pretty damned fast. 1 meg up/down on a residential DSL line is almost too slow.
posted by SpecialK at 2:06 PM on April 11, 2007

Actually, Special K, that's not accurate. Even at a colo there is a OC48 or whatever providing bandwidth to the facility, and you're also limited by the switches.
posted by frecklefaerie at 2:19 PM on April 11, 2007

SpecialK seems to be describing a cable modem topology. What's ignored is that cable providers have more flexibility with their infrastructure than the department of waste. They can break up oversubscribed segements if they need to, or they can allocate more bandwidth on the cable to data services. Even with cable modems, each shared segment has way more capacity than whatever they are marketing to consumers.

With DSL, the infrastructure is dedicated between your home and the central office (other than your analog voice service which may be sharing the same wire). Everything is pooled at the central office, where it gets on the phone company's fat pipes which interconnect with your ISP.

fightorflight, I don't know what colo or VPS rates you've been quoted, so I can't say whether you should be looking for more VPS options. The upside to VPS is that they are often hosted on nice reliable hardware, thought resource wise, it may not compete with hardware you already own on price/capacity. At the same time, more resources can often be acquired on demand, without having to deal with your own hardware in a colo.
posted by Good Brain at 3:09 PM on April 11, 2007

Yes, a T1 is 1.544 Mbps, to be more precise.

The rule of thumb is that a 1 Mbps link, at 100% utilization for a whole month, would move 324 GB. (Assuming, incorrectly, that there was no overhead.)

With my dedicated server, I'm allowed to use up to 10 Mbps, with a maximum transfer capacity of 1,000 GB.

The big thing is that cable modems are oversold, so you might not be getting out nearly 1 Mbps in uplink, whereas a dedicated 1 Mbps uplink would (or, at least, should) would let you use it non-stop.

Does this help any?
posted by fogster at 3:43 PM on April 11, 2007