My web site's getting crushed. How to keep it alive for another day or two...

July 28, 2006 5:25 PM

Help! I'm participating in tomorrow's Blogathon and my site got mentioned today on Pitchfork because I'll be giving away music during the event. My site's getting pounded and was down for about 3 hours right after the article appeared. The thing is, I haven't even put the tracks up yet! I fear the site may go down during the 'thon or, if it stays up, my bandwidth bill will be murderous. Suggestions? Questions inside.

I'm a total n00b when it comes to server shit. I'm wondering:

- way to make it so that people can't hotlink to the tracks?
- way to make it so that robot thingies can't be used to search for the tracks?
- temporary offsite places to host the music that are affordable and don't limit number of downloads?

Any other suggestions on how I can keep the site up during the event?

I'm on Dreamhost if that makes any diff. I appealed to them earlier in the week to see if they'd donate bandwidth considering the event is for charity. I got a boilerplate "we can't do that" response.

I already spent $200 on an iPod to give away in a draw for donors so don't really have more cash to throw at this event.

posted by dobbs to computers & internet (30 comments total)

Couldn't you host the tracks at yousendit or dropload or somesuch site?
posted by bob sarabia at 5:44 PM on July 28, 2006

scratch dropload, I just realized they only give you one download. yousendit should work though.
posted by bob sarabia at 5:45 PM on July 28, 2006

Do you do anything like Google AdSense? It might help you offset the costs...
posted by twiggy at 5:46 PM on July 28, 2006

On robots and indexing exclusions

Free offsite hosting of music tracks (and other files) exists but tends to be annoying to use—there are often clickthroughs, registration, or other such hurdles that come as part of the compromise.

There are certainly ways to prevent hotlinking—I can make a couple of ad hoc suggestions—but I don't know offhand what the standard techniques are. You could prevent fileserver when there's a non-local referrer, I suppose, similar to the old "don't leach my images" issue.
posted by cortex at 5:47 PM on July 28, 2006

you were down because dreamhost was down for three hours this afternoon, about 4:30pm-7pm e.s.t... see http://status.dreamhost.com
posted by noloveforned at 5:47 PM on July 28, 2006

Oh, for the hotlinking issue, the correct google terms are "htaccess hotlinking"
posted by bob sarabia at 5:49 PM on July 28, 2006

Arrgh. I feel like a dolt.

I just got word that DH itself went down at exactly the time I thought it was just me. I knew it was impossible that PFM was that popular.

Still, if anyone has answers for my hotlink question I'd appreciate it.

On preview...
posted by dobbs at 5:50 PM on July 28, 2006

Note that those results talk about hotlinking images. You can change the extension being used for your own purposes.
posted by bob sarabia at 6:05 PM on July 28, 2006

I would simply zip the files together and host them on a bittorrent tracker. You can be the first seed. You shouldnt have to worry about bandwidth. Note, youll be seeding with your PCs connection not your dreamhost account.

Heres some info to get you started.
posted by the ghost of Ken Lay at 6:21 PM on July 28, 2006

What about those free podcast-hosting services that let you embed flash thingies in the page?
posted by reklaw at 6:26 PM on July 28, 2006

Thanks for all the answers thus far.

Ken Lay... I'm trying to avoid zips and BT as I don't want people to come and just download everything and go. I want them to revisit (the tracks are being doled out one at a time over the event) and hopefully donate to the charity if they hear something they like.

I may BT them post event--like next week or something.

Also, does anyone know how to make it so that when people go to a directory on my server they can't see the contents? For instance, if I put the music in the music folder they can't just punch in mydomain.com/music and then see all the folders? What's that called, diong that, and how is it done? Is it just a matter of setting a certain chmod?
posted by dobbs at 6:32 PM on July 28, 2006

The simple way to do that is make a file called index.html and putting it under /music. Now they cant browse the directory. The server will force them to view index.html.
posted by the ghost of Ken Lay at 6:36 PM on July 28, 2006

I believe this'll do it:

chmod a-r .

from within the directory in question. Should throw a 403 at that point.

Advanced question for those who Know Things: would that stop a wget?
posted by cortex at 6:36 PM on July 28, 2006

Ken Lay, I can't believe I didn't think of that. *slaps forehead*

cortex, what is a-r in numbers, do you know?
posted by dobbs at 6:39 PM on July 28, 2006

Advanced question for those who Know Things: would that stop a wget?

Yes. wget is subject to the same restrictions as any other web browser.
posted by evariste at 6:44 PM on July 28, 2006

Some anti-hotlinking info from Dreamhosts's own Wiki. Looks like you might be able to do it from your control panel.

If you decide you are worried about bandwidth use after-all you might check out CoralCache. All you do is add ".nyud.net:8080." to the hostname in the URLs you link to the music and it handles the rest. The main downside, as far as I can tell, is that you don't get download stats.
posted by Good Brain at 6:49 PM on July 28, 2006

goddam numbers! I believe you'd want 711 but I recommend you not take my word for it: reference material
posted by cortex at 6:53 PM on July 28, 2006

Make a different torrent for each song, and publish the torrent whenever you were going to publish the song. As long as people have bittorrent software already it should be pretty much equivalent for them to you posting the actual file, but wayyyy cheaper for you, in terms of bandwidth.
posted by aubilenon at 7:15 PM on July 28, 2006

And totally alienating to anybody who doesn't have a bittorrent client, which is most people.
posted by cortex at 7:33 PM on July 28, 2006

Since you're transferring large files, I suggest that you immediately familiarize yourself with Coral Cache. By simply modifying the URLs that you use to link to those songs, you can immediately alleviate the bulk of your traffic.
posted by waldo at 7:42 PM on July 28, 2006

D'oh. Sorry. I loaded this page and forgot to check on preview whether that advice had already been offered. Consider it a second endorsement.
posted by waldo at 7:43 PM on July 28, 2006

How about hosting them on a MySpace account? You can allow the files to be downloaded and the bandwidth hit is zilch to you. I believe you're limited to 4 files so maybe it's not such a hot idea.
posted by KevinSkomsvold at 7:56 PM on July 28, 2006

If they're your songs, I say post them to Odeo.com and link to them there from your site.
posted by mathowie at 8:14 PM on July 28, 2006

I think the bandwidth thing isn't going to be an issue--it was really just an odd coincidene that PFM ran the article and within an hour my site was down due to DH's power outage.

Also, cortex has kindly offered to mirror if need be and the CoralCache option should also make it go smooth.

Also, though I have permission to share the tracks, they're not technically my own, but were donated by various bands so on reflection, I probably shouldn't use a third-party host.
posted by dobbs at 8:31 PM on July 28, 2006

I'm baffled by the chmod thing. I can either enter an octal number (which I can't figure out from cortex's link) or ...

--- --- ---

with r, w, and x as the options. What do I want it to read in order for me to be able to link to a file but not have someone be able to browse the file or use wget?
posted by dobbs at 8:40 PM on July 28, 2006

If you don't mind putting them under a creative commons license archive.org can fit the bill of hosting large files for free.
posted by psychobum at 8:45 PM on July 28, 2006

Thanks, all. Cortex and I got it good to go in email!
posted by dobbs at 9:02 PM on July 28, 2006

dobbs, if you want people to be able to read the files you link to, you have to make them readable.

If you don't want people finding them except by following your links, you need to make the directory they're in unreadable. Since the only way anybody can possibly read the directory from outside your server is by submitting an http request, and since your web server won't hand out a directory listing over http if there's an index.html file already there, that's a much simpler thing to do than messing with permissions.

There's nothing magical about wget; it's just another http client. If you can download a file with a web browser, you can download it with wget. If what you're aiming to do is to stop people guessing your downloadable's URL and grabbing it without visiting your web page on the way, just include a few random digits in the filename.
posted by flabdablet at 9:38 PM on July 28, 2006

To echo flab:

To prevent people from browsing your directories of music, put an index.html file in each and every directory that contains music. You can make it entirely blank, or yell at people for trying to hotlink. Those crazy permissions tweaks above have a good chance of breaking other things.
posted by JZig at 1:01 AM on July 29, 2006

"Crazy permissions tweaks"? I respect the simple kluge of dropping a blank or cookie-cutter standard index.html, but those are not crazy persmissions tweaks, that's like elementary school UNIX, and familiarizing one's self with permissions structure is an extremely good idea.

The permissions tweaks should furthermore not have any chance of "breaking other things"—a specific permissions modification aimed at a specific directory isn't going to go running wild.

Get to know your filesystem at your earliest opportunity. It is your friend, your partner, your first responder.
posted by cortex at 6:38 AM on July 29, 2006