Best way to store 4 TB of media?

November 18, 2009 6:58 PM

I'm looking for reliable, scalable storage, around 4 TB.

I've got the usual setup: several computers, home network, lots of media to share. Right now, I've got most of the media on a NAS (Infrant NV), which holds ~2 TB with RAID. However, I've got another 2 TB floating around, and I'd rather use the NAS for backup. I don't want to lose the media, but the current setup is expensive and doesn't scale well (i.e. cheaply). I'd be willing to add external storage to an existing computer or build a new file server, but I'm not sure what will scale best over the next couple of years.

posted by heliostatic to Computers & Internet (17 answers total)

I feel your pain. The closest I've gotten is linux software raid5. Just add drives when I start running out of room. Sadly, I've maxed my drive holding capacity and now I need to make those drives bigger. My last few incarnations have been along the lines of start with two drives mirrored, add a third and convert the array, add a fourth and convert the array. Then start over because at that point the hardware probably needs to be refreshed anyway.

I'm thinking my next scenario will be some kind of USB enclosure setup. When I need more space, plug in another hard drive. That will scale (logically) but I don't know how well realistically. I'm really hoping that ata-over-ethernet thing catches on. Seems like a really good solution. IF one can get their hands on small, cheap ata to ethernet adapters.

But more theoretically, the idea of having one repository of data ("The Server") works way better for me. Workstations can have any garbage drive I want, because they are all backed up. I just can't make it work when my data is spread over different machines.
posted by gjc at 7:25 PM on November 18, 2009

Response by poster: @gjc Absolutely. Single repository for anything shared. Haven't looked at ata-over-ethernet much, though I did see the pogoplug recently. Still, I like the idea of a server (or server device) with all the storage, as you said.
posted by heliostatic at 7:30 PM on November 18, 2009

What's not cheap and scalable about the ReadyNAS? You just put new, larger hard drives in it, replacing your old drives one-by-one.
posted by zsazsa at 8:43 PM on November 18, 2009

Seconding zsazsa.
Here is a guide to upgrade the NV:
http://www.readynas.com/forum/viewtopic.php?f=24&t=33757&view=next
posted by tresbizzare at 9:01 PM on November 18, 2009

Response by poster: @zsazsa and @tresbizzare I've upgraded once before. The expensive part is that all drives have to be upgraded at once. Also, RAID is expensive for data that doesn't need it. But it is a good solution insofar as I already have the hardware...
posted by heliostatic at 9:54 PM on November 18, 2009

Build a Windows Home Server machine? Cheap, easy to grow your data store and I believe WHS supports up to 28 drives.
posted by wongcorgi at 10:14 PM on November 18, 2009

Response by poster: @wongcorgi I was looking at WHS. Decent multi-platform support?
posted by heliostatic at 10:20 PM on November 18, 2009

Just one perspective -- you may want to look at storing backups outside your house. That way if you have a fire, flood damage, burglary, that kind of thing (the most common threats in a home environment) your data will be safe. I considered redundancy with RAID myself but I decided to go with an off-site backup scheme and keep my "hot" files that change often in special directories which are backed up more frequently, locally, on USB or DVD-R.
posted by crapmatic at 11:12 PM on November 18, 2009

Response by poster: @crapmatic Agreed. I have offsite storage for the files I care about. But for the onsite sharing, I'm not sure what to do.
posted by heliostatic at 11:15 PM on November 18, 2009

Best answer: 4TB is a ton of data. Friend, it's time to kick it up a notch. A proper RAID card will give you the flexibility you need and will scale quite well. Additionally, you are covered in case of controller failure (which does happen, from time to time) since you can replace a failed Areca card with any other Areca card that can support the same amount of disks and your raid set will appear fully intact.

If you are technically inclined and feel comfortable, I'd say build a low power file server around Openfiler, an Areca ARC-1220 (with optional battery backup unit) and 4x 1.5TB disks. This will yield 4.5TB of pretty redundant, reasonably fast storage. As you need more space, you can add more disks (up to 8 in total on the ARC-1220) and then expand your volume set. You can even do the expansion online, so no downtime while the disks rebuild, which is pretty cool actually. I also recommend a nice big case with lots of 5.25" bays and a pair of disk enclosures, but I'm fancy like that.

If you feel like that is a bit too complicated, a simple low power server (for NAS, not needed if you use only iSCSI) + Drobo Pro offers many of the same benefits and is easy to manage. It costs more and you are basically screwed if the unit fails, but it is much simpler and, admittedly, pretty nice looking.
posted by tracert at 11:24 PM on November 18, 2009

If you want something that will grow, check out UNRaid. It does carry a single-drive worth of overhead for parity, but it can be expanded a drive at a time, and if two drives fail you only loose one drives worth of files, rather than the whole damn array.
posted by Good Brain at 11:53 PM on November 18, 2009

I used to be a big fan of RAID and logical volume management and all those other lovely things that would let me set up a single, massive file system that I could just keep growing by throwing more and more drives at it and using some kind of Expand option. But lately I'm finding myself kind of over all that.

A media collection doesn't need RAID for speed, and RAID isn't backup, and big big file systems rely on all the underlying drives staying working if you're not to risk major, major data loss from a single drive failure.

My current setup is the low-powered 733MHz Pentium III Linux box I was using for all the RAID and LVM things, but it's now just got a 500GB, 1TB and 1.5TB drive attached for media storage, with one filesystem per physical drive. I mount these under /home/drives/0 through /home/drives/2. When the time comes to add more storage, I'll just add a 2TB drive, copy the current contents of the 500GB drive onto it, and retire the 500GB drive.

If a drive fails, only the files on that drive are threatened; the others remain fully functional. Also, file recovery is a lot easier, and there are a lot more tools available to do it with, if the filesystem needing recovery is on a simple, single drive with no complicated partitioning and striping schemes.
posted by flabdablet at 12:06 AM on November 19, 2009 [3 favorites]

Have a look at the Acer easyStore H340.
posted by devnull at 4:21 AM on November 19, 2009

The need for RAID is in uptime. Proper backups are great, but when you are talking filesystem sizes that just aren't possible with single drives, you will have catastophic failures if one drive goes bad.

And RAID specifically prevents any data loss from the loss of a single drive. One drive fails, you plug a new one in and it automagically restores redundancy. Without RAID, you have to restore everything from backup. The hassle of the restore is reduced, you only have to do it when you have the unlikely filesystem failure, or more than one drive failure.

Also, cost can easily be managed. Instead of having to buy the biggest and most expensive drive that comes out, you can buy cheaper commodity drives. Since speed isn't really an issue for a home server, software raid is fine. You don't need a dedicated card. (And, unless you are talking enterprise level, if your raid card fails, you have to find another compatible one in order to see your data. With software raid, you just have to plug them into a new machine.

My outlook is that the best kind of backup is the one you never have to use. And planning to rely on file recovery software is not a plan.

(That said, I don't like LVM. Too much obfuscation. I've seen it do weird, weird things. Even in enterprise level hardware/software.)
posted by gjc at 5:46 AM on November 19, 2009

I came in to recommend Drobo. I've had one for a year and have been happy so far. It serves media to my home network and works well for me.
posted by rglasmann at 6:59 AM on November 19, 2009

Cheap option: 5 bay drive caddy and software RAID 5 - whack in 4 x 1.5 TB drives and you have 4.5 TB of redundant storage. You only have to pay for the drives and when you need more space, just whack in another 1.5 TB drive. You should have spare bays in your tower for the next 3 TB or so. 1.5 TB is a good dollar/gig pricepoint and it should scale cheaply and effortlessly up to at least 9 TB.

Expensive option: QNAP TS-639 NAS and RAID 6 - throw in 3 x 2TB drives and you've got room for another 6 TB later. Great media server abilities plus you get a proper linux server. If you want to be able to throw in any capacity drive, look at the Drobo instead but it's probably even more expensive.
posted by turkeyphant at 7:23 AM on November 19, 2009

Best answer: I just went through this.

I ended up getting this unit. It's got 4 drive bays, is totally hackable, and you can install Windows Home Server using instructions here (see the comments on that page for specific instructions for this server).

MeFiMail me if you want more details.
posted by sciencemandan at 10:44 AM on November 19, 2009 [3 favorites]