Quality Dedicated Hosting that allows Bittorrent Tracker? Have Budget.

December 18, 2007 6:01 PM

I will soon be distributing a bunch of large scientific and research datasets, all entirely legal for redistribution, many as large as a few gigabytes. Bittorrent is the best solution for this. Many respectable webhosting sites don't allow Bittorrent trackers; too many of those that do seem fly-by-night. I have a budget for this and want something respectable and reliable. Recommendations?

My current host (Bluehost) has 6000GB/transfer a month, but I hear that these large caps can often be illusory; I want to be prepared for one to ten TB of traffic per month. I also need to be able to receive similarly large datasets, which I think will be much easier over bittorrent than asking folks to POST >1GB over ADSL and have it crap out after 3 hrs transfer (not to mention hitting CPU limits).

Amazon S3, at $0.18/GB, would be ~ $1000/mo for 6000 GB. I've reviewed most of the previous hosting q'ns, including Dealing with spikes in website popularity, but none of those suggestions make this work financially.

This clearly calls for bittorrent, but I've had real trouble finding dedicated hosting that will allow me to install a tracker. Bitrot has happened to the last question along these lines, and too many of the directories for tracker sites seem populated by rug merchants and fly-by-night operations.

I'm willing to spend money -- preferably in the $100-$200/mo range -- and want something respectable and reliable. If you tell me it will cost more than that then so be it. I'm figuring that the bittorrent solution (combined with throttling, ratios and a cooperative userbase) will keep me in the 1 TB/month range.

Can you recommend dedicated hosting that will allow me to run a tracker?

posted by mrflip to computers & internet (20 answers total)

Almost any host will probably allow an exception to stated policy for this. The only problem may be finding a human being willing to consider it, rather than an automaton of the e-mail or phone variety. But once you find that, you have found a good host anyway.
posted by rokusan at 6:21 PM on December 18, 2007

Have you bugged your it/comp phys people? For $200/mo you might get a node on the university's sweet sweet backbone connection.
posted by a robot made out of meat at 6:24 PM on December 18, 2007

I'm a researcher and would not use bittorrent for this purpose. It's a virus nightmare for large institutions. The common solution is an ftp site with a guest login.
posted by fshgrl at 6:28 PM on December 18, 2007

that would be an ftp site on a large institutional network, for free or cheap.
posted by fshgrl at 6:37 PM on December 18, 2007

fshgrl, how is the bittorrent protocol a virus nightmare? Or is it just that large institutions see all p2p as virus risks?
posted by Asymptote at 6:42 PM on December 18, 2007

If you are associated with a university or college, try contacting your library. Digital archiving, especially of scientific data sets, is increasingly one of the roles that academic libraries play in the grand academic game.
posted by rachelpapers at 6:51 PM on December 18, 2007

As far as the cost of S3, according to this blog post, you can upload a file to S3, download the .torrent, seed the file from your local machine, and then delete file from S3.

That way, you have almost no S3 download costs, but get to use the S3 tracker.
posted by Asymptote at 6:53 PM on December 18, 2007

Yah, the best solution for this for academic stuff isn't bittorrent. It's putting them on an ftp server on the internet2 backbone (which UT is on). Is there some pressing reason to do this through your own hosting? And why you think you should pay for it?

Internet2 connections are crazy awesome. Last time I played with it, I could download linux ISOs in like 20 seconds.
posted by ROU_Xenophobe at 7:33 PM on December 18, 2007

I've had no personal experience with any of the things I'm about to link to, but:

MoveDigital sounds like exactly what you want (a professional dedicated torrent host). They're apparently reputable enough for John Edwards. Also, from what I can tell, they grew out of Prodigem, which used to provide free torrent hosting to Creative Commons-licensed files.

It looks like Legit Torrents would act as a free tracker, but you'd still have to figure out seeding.

If you do end up running your own server, you might want to try Snakebite (which handles both tracking and seeding) or Diferior (which is meant to create an entire community site that includes a tracker, hosting, and the ability for users to upload their own torrents).
posted by Asymptote at 7:37 PM on December 18, 2007 [2 favorites]

I'm thinking that BitTorrent wouldn't be a good choice for you (unless I'm misunderstanding your question).

Due to the way it works, BitTorrent would only be effective if there were others seeding. I'm not sure how you would force other people to seed your database. If you are the only seeder, all of the traffic is still coming from you, and therefore, you pay for all of it.

BitTorrent is not a magical way to reduce bandwith cost, it just distributes it amongst the seeders.
posted by ArgentCorvid at 7:48 PM on December 18, 2007

I agree that the more reasonable solution is to put it on a university machine for plain old download and accept FTP uploads to a writable-but-not-readable "incoming files" directory. That kind of thing is why your university has an internet connection in the first place, after all. (I'm assuming that whatever this data is is university-related.) If UTA's people won't let you do that, maybe you have a collaborator at some institution which will.

The only aspect that that solution doesn't cover is allowing people to upload datasets over an unreliable connection. BT is good at recovering from interruptions, and you could presumably ask your IT folks if you can run a tracker in that case (or if you expect "flash mobs" of many simultaneous downloaders of the same data); BitTorrent is no more of an inherent "virus nightmare" than HTTP or email are.
posted by hattifattener at 8:21 PM on December 18, 2007

I've got some spare cycles and bandwidth. E-mail me if you'd like a mirror.
posted by yellowbkpk at 11:06 PM on December 18, 2007

I'm about to leave school, which will make things harder -- but hosting some of these at school to insure against burst demand is something I'm looking into. The other problem is that a fat pipe alone doesn't cut it -- my department was the origin of the famous Shear thickening in vibrated cornstarch video; when that got Farked it brought the whole server down. Luckily we had a connection in the NOC who knew the right Apache cantrips to make it work for that burst traffic. More importantly, the data files will come from my next project (after I've left school), which deals with visualization and organization of large data sets, and will be work- and not university- related.

Asymptote -- thanks for the links, this was the next thing I had to look into!

I've poked around with using domain tools and alexa to find the ISP for sites that seem to have figured this out, but it's harder for dedicated hosts. I may just resort to emailing one or two to ask their experience.

So I'd still like to find hosted service that allows bittorrent... If you have any more ideas, keep them coming. Thanks everyone for your generous help!
posted by mrflip at 11:34 PM on December 18, 2007

If you've identified a few hosts that would work if not for the tracker restrictions, you could just spend a few minutes collecting sales email addressess, then fire off an email to all of them briefly explaining the situation.
posted by MetaMonkey at 12:24 AM on December 19, 2007

Determining Which Company Physically Hosts a Website. The missing piece was using RIPE/ARIN to find the owner of that IP block. Note that if an IP comes back as "global" it probably means it belongs to a differen top-level Regional Registrar.

I'll report back on my findings. Thanks all!
posted by mrflip at 12:39 AM on December 19, 2007

Slicehost gives you a dedicated Linux VM. Their most expensive standard package is currently 2 Gigs RAM, 80GB storage, 800GB transfer for $140 per month. You can do anything that's legal (in the US, so libel British tabloids all you want!). If you don't need that much storage and RAM you can ask them about upping the transfer allowance on one of their cheaper packages.

I haven't installed a bt tracker or client on my server but I've had a great experience with them.
posted by XMLicious at 1:04 AM on December 19, 2007

2nding Slicehost - I've only got a 256slice, but it's been absolutely solid, and their "anything, as long as it's legal" policy about network use seems ideal here.
posted by doofsmack at 2:17 AM on December 19, 2007

I'm checking that out XMLicious, this looks good. If I go with them I'll make sure to put you down for the referral bonus.
posted by mrflip at 2:36 AM on December 19, 2007

Be sure to let us know what you finally went with.
posted by Asymptote at 3:49 PM on December 21, 2007

I asked around to several hosting companies, by thinking of small businesses that push a lot of bits and then using ARIN/RIPE to find their ISP. Two hosting companies got back to me same day, slicehost and The Planet (Actually, slicehost got back to me within the hour, that hour being 5am.) They both allow bittorrent for legitimate use and seem to have good recommendations.

The Planet is more towards the high end -- they have a package with 250GB storage/~1.5TB cumulative bandwidth in the $150/mo range. They offer more modest plans but the price/performance falls off for the smaller packages.

Planet also offers unmetered server packages: you get unlimited bandwidth, but you are throttled to a hard limit of 10Mbps peak throughput (or more as you go up in price), and it's much more expensive -- in the ~$350+ range depending on your yada yada.

Slicehost seems to scale better at the low end: you can start with a pretty burly package for $20 and move up. You can also buy multiple slices easily, and there's no setup charge.

• $20.00/month - 10GB HD, 100GB BW
• $38.00/month - 20GB HD, 200GB BW
• $70.00/month - 40GB HD, 400GB BW
• $140.00/month - 80GB HD, 800GB BW

Keep in mind please that this is what I've looked up -- I don't yet have any experience with either company (well, outside the courtship period I don't).

I'm going to go with slicehost for now, and reevaluate if I get up to the $100/mo ballpark. Thanks to all of you for your generous help!
posted by mrflip at 7:56 PM on December 23, 2007