Small Production Office Needs a File Server

January 5, 2009 9:04 AM

I work in a very small office, less than 10 people, that is mac based doing primarily Final Cut Pro editing. I know a lot about the mac but not much about putting a server together. Here are our needs:

- We will continue to work on independent systems but would like the ability to keep some files centrally located. These files will be copied to the seperate workstations as needed.
- a fast transfer protocol. Currently we are using ethernet which is not fast or efficient enough to tranfer large video files.
- We'd love the ability to harnass the prossecor power of all the computers for rendering, particularly with renders and encodes. (Is a render farm even possible using Final Cut? And how would you do it? I know this is a seperate issue from the files sharing above but we'd like to accomplish both tasks.)
- No one is particularly UNIX savy so if actual server software is avoidable all the better. But certainly not a deal breaker.

So to reiterate: our two primary goals are the ability to make quick transfers between machines and two, a way to harnass the processor power during renders and encodes.

posted by captainscared to computers & internet (7 comments total)

For file transfers and shared media, you're probably looking for a Fibre Channel SAN setup. Speeds are much faster and latency much lower than using ethernet (even gigabit). The setup generally does involve a separate unit, not necessarily a server per se, but a bunch of hard disks connected to a controller box, with skinny little fiber optic cables running from that box to each computer. This is not an inexpensive solution or a turnkey setup at all, and would involve bringing in an outside consultant if you don't have a strong sense of what you're doing. Hardware cost for 5 systems would be upwards of $10K, before hard drives.

As far as distributed rendering goes, Final Cut Studio includes a distributed rendering/compression system called QMaster. However, as far as I'm aware, only Compressor and Shake can actually use it for distributed rendering.

Unfortunately, I don't know that there's an inexpensive, easily configurable middle ground between using Fibre Channel and Gigabit Ethernet. Maybe 10GigE or 100GiGE, if and when they ever materialize on Apple hardware. Barring all that, you could just sneakernet everything with some nice, big, fast FW800 or eSATA drives.
posted by kid_dynamite at 9:28 AM on January 5

Almost everyone is going to recommend Xsan to you, which is Apple's video production editing / storage system. Xsan is fibre-channel based, so it's definitely faster than ethernet. In fact, it's virtually impossible to do video-editing across ethernet. At the TV station where I worked for awhile, we tried to cobble together an ethernet-based editing system (Final Cut Pro on workstations, hi-end NAS on the storage side) and discovered that even with highly-optimized networking (jumbo frames, acceleraters, etc.) we started getting dropped frames after adding the 3rd or 4th computer to the mix...in other words we could get 2-3 machines editing from the SAN, but any more, and the performance degradation started to affect all workstations.

However, Xsan is fairly expensive. You might not be able to afford it; you'll have to decide. If you can't afford a proper fibre-channel SAN, then your best bet is to have each FCP workstation set up with a large attached FWHD which acts as the media repository and then to set up a simple file-server (Mac OS X Server) to be a shared space where you can swap files between users.

FCP supports rendering using Xgrid, which is now built into Mac OS X 10.5 and up. It's relatively easy to set up an Xgrid controller on Mac OS X Server and the Xgrid clients (your workstations) to set up a render processing farm.
posted by mrbarrett.com at 9:40 AM on January 5

mrbarrett, do you know if Mac OS X can run virtualized in VMWare View? Teradici looks like a solution just for this kind of situation. I have read cases where non-Mac studios run this with ease. It is OS independent, so if you can get OS X working in ESX you're good to go. Load up on cheap quad-core processors (FCP rendering is CPU not a discrete graphics card?) and you're good to go.
posted by geoff. at 9:58 AM on January 5

phrase of the day: good to go.
posted by geoff. at 9:58 AM on January 5

Thanks for all the input. Final Cut aside as we will not be working with files off the server, what's the best, most efficient way to set up a fast file server?
posted by captainscared at 11:16 AM on January 5

Well, you've got a few ways you can go:

1) the cheap route: A standard Mac OS X 10.4 or 10.5 machine with Sharepoints installed. This will turn the box into a "server" and give you the ability to add/create sharepoints and give you more granular control over user access, etc. Advantages: cheap. Disadvantages: you'll get nowhere near the AFP or SMB performance that the next solution will give you.

2) the more expensive route: Mac OS X Server 10.4 or 10.5 installed on a fairly fast box. It does not have to be an Xserve. A decent Mac Pro will suffice, and I've even seen it installed on a maxxed out Mac Mini. This will give you the granular access control, full ACLs, highly-customizable sharepoint control, the full Open Directory framework, Kerberos and Single-Sign-On support and more. But it's more expensive, of course. It'll definitely perform better--meaning you'll get much faster AFP and SMB throughput and faster copy times--than a regular client Mac OS X with Sharepoints installed.

3) a third and much more complicated solution would be a Windows server with GroupLogic's ExtremeZ-IP installed. This is a really great product, but is expensive. It essentially replaces Windows 2003 Server's AFP service and SMB service with their own updated-and-optimized protocols. It really works well and is fine-tuned for bandwidth performance, so you'll really see great transfer rates. This might matter since you'll be tranferring very large files.

4) A Window Server using the built-in Services for Macintosh (SFM). Really not recommended as Microsoft has done next-to-nothing to improve SFM since Win2K, and even then it was buggy and pretty bad.

5) A Linux server with Netatalk installed. I have limited experience with this, but have Linux friends who have build these and swear by them. This, of course, requires a Linux sysadmin with the right skillset to build, deploy, and maintain.

What I DO NOT recommend: trying to use the built-in personal filesharing that comes with Mac OS X. I suppose it's fine for very simple transfers of files between workstations, but we're likely talking about many-GB-sized video files. You'll quickly hit the limit of what personal filesharing can do, methinks.

You might explore other file transfer protocols. AFP is, of course, your default, but it has some minor processor overhead issues...well, the Finder has some overhead issues when copying files via AFP vs. SMB. If your workstations need to stay lean and fast for video editing, you might want to explore building out an FTP server (Rumpus is awesome!) and then training your users to use a light FTP program to transfer files back and forth to the server. This allows the Finder to wallow in it's own purgatory and lets you continue to get some work done while the FTP program (CyberDuck, Transmit, Forklift) do the copying.

You'll also want to optimize your network. Make sure everything is Gigabit or 1000 BaseT. Your workstations almost certainly all have 1000 BaseT, but check your switches and cabling to make sure they can handle 1000 BaseT, otherwise you may have a bottleneck sitting right in the middle of your file transfer workflow.
posted by mrbarrett.com at 2:38 PM on January 5

Test to make sure that you have a gigabit network. Check if you have a switch that supports link aggregation. Don't get a general purpose server; instead find yourself a network attached storage device like the ReadyNAS Pro.
posted by PueExMachina at 4:45 PM on January 5