Fish outta water...
January 25, 2006 3:54 AM Subscribe
Mac network/backup/storage questions. Any Mac gurus willing to throw their 2 cents at me?
I am volunteering some of my time at our local college/community radio
station, and one of my skill sets is IT stuff. My problem is that I am
a windows guy, and have only supported Windows networks. The
station is 100% MAC. We want to implement some new technology, and I
thought I would run this by the askme crowd for comments, advice.
The main things we want to do is implement a file server. We have one
machine running the MAC server OS, but it has limited disk space. We
also have hopes of converting the library to MP3, and allowing a
workstation in the on-air booth to access the library via iTunes. We
need lots of diskspace for this, estimates are in excess of 1TB.
Lacie makes a device that offers 1TB of storage with RAID options,
which will allow us some redundancy as well. I am leery of trying to
implement a true backup solution with this station, as it is largely
volunteer run, and taking tapes offsite would be a nightmare, so I am
thinking the RAID5 might give us some redundancy without the need for
proper backups.
I guess what I am looking for from you folks is just someone to run
this by. Does this make sense? Is there a better MAC-only solution
that I am not aware of?
Thanks!
I am volunteering some of my time at our local college/community radio
station, and one of my skill sets is IT stuff. My problem is that I am
a windows guy, and have only supported Windows networks. The
station is 100% MAC. We want to implement some new technology, and I
thought I would run this by the askme crowd for comments, advice.
The main things we want to do is implement a file server. We have one
machine running the MAC server OS, but it has limited disk space. We
also have hopes of converting the library to MP3, and allowing a
workstation in the on-air booth to access the library via iTunes. We
need lots of diskspace for this, estimates are in excess of 1TB.
Lacie makes a device that offers 1TB of storage with RAID options,
which will allow us some redundancy as well. I am leery of trying to
implement a true backup solution with this station, as it is largely
volunteer run, and taking tapes offsite would be a nightmare, so I am
thinking the RAID5 might give us some redundancy without the need for
proper backups.
I guess what I am looking for from you folks is just someone to run
this by. Does this make sense? Is there a better MAC-only solution
that I am not aware of?
Thanks!
If you need 1TB then don't get a 1TB RAID device and make it work in RAID 5. You'll get 750GB (I think). Either way the redundancy takes space, you'd have to get larger than a TB for a TB using RAID.
I've been trying to figure out how to get a low cost RAID that I can expand later. I like the Terastation, but it seems inflexible. I may get a pro Terastation, but am unsure whether it works with bigger drives. Not to mention RAID 5 be better when adding more drives, not swapping larger ones into it.
Personally I was thinking of getting this :
http://lime-technology.com/
and using a more standard Linux RAID setup (more advanced setup-wise I guess). It's awesome because you can add 4 drives, get to 1TB, and continue adding dynamically as you go.
Also, search ask mefi for "RAID". Lost of questions for the past few weeks.
posted by Napierzaza at 6:43 AM on January 25, 2006
I've been trying to figure out how to get a low cost RAID that I can expand later. I like the Terastation, but it seems inflexible. I may get a pro Terastation, but am unsure whether it works with bigger drives. Not to mention RAID 5 be better when adding more drives, not swapping larger ones into it.
Personally I was thinking of getting this :
http://lime-technology.com/
and using a more standard Linux RAID setup (more advanced setup-wise I guess). It's awesome because you can add 4 drives, get to 1TB, and continue adding dynamically as you go.
Also, search ask mefi for "RAID". Lost of questions for the past few weeks.
posted by Napierzaza at 6:43 AM on January 25, 2006
You say volunteer run, so I am assuming the budget is close to none? I am not a big fan of LaCie drives. Most of the ones that I have do not work properly anymore (after 1 year or so). They fail on copying and backup and it's not reliable.
posted by _zed_ at 7:04 AM on January 25, 2006
posted by _zed_ at 7:04 AM on January 25, 2006
A data store incorporating redundancy doesn't obviate the need for regular backups. Systems change over time for many reasons, and backups are the recognized way of restoring a known working system state when something goes awry. Generally, redundancy mitigates the effect of single point hardware failure, but it can be used in conjunction with a suitable backup strategy to minimize the time impact of doing large backups. A specific case of this is when a database is operated on a SAN or cluster of servers, and one server in the group is intentionally taken out of service regularly to act as the backup snapshot origin, after which backup is done, it is returned to the group and re-synched with the active database.
Assuming your Mac server is running some version of OS X, if you want a general purpose library solution, I'd suggest you think of putting in a Linux or BSD based file server as your library machine, and adding a couple of SATA disks as your backup volume in that. The value of the *NIX based file server approach is that you can choose a journaling file system that is suitable for your needs, implement and upgrade hardware/software RAID as you choose, have the flexibility of LVM (Logical Volume Management) when (not if) your storage needs grow, and have standardized security and management tools. A journaling file system will save a lot of time at restart should the system experience an unexpected shutdown or failure, and will scale well if you start using it for multi-user applications, such as running a streaming server front end for Internet radio.
Not directly in your OP, but a closely related issue, as brought up by ursus_comiter up thread, is the question of your network capacities. If you are setting up a library function, gigabit Ethernet should be a baseline, including decent switches and NICs with jumbo frame support. It's worth doing your homework on network hardware as you move into gigabit Ethernet, particularly if you do choose a Linux solution base, as some chipset and NIC manufacturers are being sticky with releasing information to the Open Source community needed to create drivers. Broadcom and others supply only binary Linux drivers for some of their products, so if you're planning to use a motherboard based NIC, check to make sure it is going to be compatible with the Linux distro you intend to use, should you choose to go this route.
posted by paulsc at 7:07 AM on January 25, 2006
Assuming your Mac server is running some version of OS X, if you want a general purpose library solution, I'd suggest you think of putting in a Linux or BSD based file server as your library machine, and adding a couple of SATA disks as your backup volume in that. The value of the *NIX based file server approach is that you can choose a journaling file system that is suitable for your needs, implement and upgrade hardware/software RAID as you choose, have the flexibility of LVM (Logical Volume Management) when (not if) your storage needs grow, and have standardized security and management tools. A journaling file system will save a lot of time at restart should the system experience an unexpected shutdown or failure, and will scale well if you start using it for multi-user applications, such as running a streaming server front end for Internet radio.
Not directly in your OP, but a closely related issue, as brought up by ursus_comiter up thread, is the question of your network capacities. If you are setting up a library function, gigabit Ethernet should be a baseline, including decent switches and NICs with jumbo frame support. It's worth doing your homework on network hardware as you move into gigabit Ethernet, particularly if you do choose a Linux solution base, as some chipset and NIC manufacturers are being sticky with releasing information to the Open Source community needed to create drivers. Broadcom and others supply only binary Linux drivers for some of their products, so if you're planning to use a motherboard based NIC, check to make sure it is going to be compatible with the Linux distro you intend to use, should you choose to go this route.
posted by paulsc at 7:07 AM on January 25, 2006
Response by poster: Wow. Thanks for the well thought out replies folks.
paulsc, do you mind if I email you some questions? I am mainly curious as to what you might guess ramp up time for a *nix solution might be with my current level of knowledge, and I don't want to noise-up the thread with my specifics.
I suspect that there are two main ways of looking at this:
1) Low ramp up, low successful redundancy
2) Higher ramp up, greater long term pay offs.
posted by Richat at 7:43 AM on January 25, 2006
paulsc, do you mind if I email you some questions? I am mainly curious as to what you might guess ramp up time for a *nix solution might be with my current level of knowledge, and I don't want to noise-up the thread with my specifics.
I suspect that there are two main ways of looking at this:
1) Low ramp up, low successful redundancy
2) Higher ramp up, greater long term pay offs.
posted by Richat at 7:43 AM on January 25, 2006
Sure. paulsc at mindspring.com works, but if your questions aren't too personally specific, keeping them in thread may benefit others with similar or related concerns. And, you may draw in others with different perspectives, or greater experience. Lots of people are looking for advice these days regarding media storage and retrieval on small networks, and many MeFi regulars are experienced sysadmins and early adopters.
posted by paulsc at 8:33 AM on January 25, 2006
posted by paulsc at 8:33 AM on January 25, 2006
Mac OS X has a journaled file system, excellent backup technologies, and is far easier to deal with as a mortal than BSD or Linux- why wouldn't you use that since it's already on the machines?
I've worked in IT as a volunteer at a lot organizations, and the biggest struggle for the staff of these organizations is that people come in, volunteer for 3 months, get 25% of the way through changing everything, never educate anybody, and then vanish into thin air.
Mac OS X can do software RAID as well. Get a couple internal or external disks, RAID them until you have sufficient size, and then use that. Then, use Carbon Copy Cloner as an excellent (and, once again, usable by mortals) application to back up any data you want to an external disk- LaCie or whoever. (I've not had troubles with LaCie.)
Sticking with Mac OS X (since they already had it) means they'll know more about how to use it. The solution I've proposed involves adding nearly no complexity to their existing solution.
posted by thethirdman at 9:09 AM on January 25, 2006
I've worked in IT as a volunteer at a lot organizations, and the biggest struggle for the staff of these organizations is that people come in, volunteer for 3 months, get 25% of the way through changing everything, never educate anybody, and then vanish into thin air.
Mac OS X can do software RAID as well. Get a couple internal or external disks, RAID them until you have sufficient size, and then use that. Then, use Carbon Copy Cloner as an excellent (and, once again, usable by mortals) application to back up any data you want to an external disk- LaCie or whoever. (I've not had troubles with LaCie.)
Sticking with Mac OS X (since they already had it) means they'll know more about how to use it. The solution I've proposed involves adding nearly no complexity to their existing solution.
posted by thethirdman at 9:09 AM on January 25, 2006
See, already, different perspectives...;-)
Seriously, thethirdman makes good points. Some of what he recommends depends, of course, on what the current Mac server is already used for, and what options you have for expandability in hardware. And budget, etc. Countering his argument for simplicity, are experiences I've had with volunteer operations where more and more functions got incrementally added to existing servers, until nothing worked right. More than once, I've "volunteered" to "look" at some church server on a Saturday when the minister couldn't print his sermon, only to find that the small server in the basement had a disk that was reporting 98% utilization, and no one knew the admin password, except the CPA that came in Friday to update Quickbooks (or some similar scenario) ...
Some friends of mine operate a couple of commercial radio stations in western Nebraska, and although their operation is Windows based, at a general level, their set up may be instructive. In their shop, they do a lot of production of commercials and PSA's for their operations, and have a several different servers for various aspects of their business. They do about 6 hours a day of locally generated content (live shows, news, etc.) and about another 4 to 6 hours a day of feeds; the remaining 12 hours, they play music and commercials via set lists (they want to keep a local voice, and don't want to be yet another Clear Channel clone). At the core, they have a small Dell Windows 2003 machine, that acts as their AD root, file and print services for the accounting and business application packages, and similar general chores. In each studio, they have a "board" machine that runs an application package that emulates a traditional studio "board," provides Internet access, and a connection to the rest of the automation, such as their automated telephone switchboard. One studio is also equipped with additional machines for audio editing and post production, where they mainly do their commercials, newscasts, etc. In a server closet with the domain controller, they have a library machine which contains a couple of hundred commercial songs for their playlist, plus their commercials, PSA's, etc. This library machine has a commercial software package that generates their ASCAP and BMI air stats, and manages the library, and stores the content files in lossless format (maybe proprietary). For security, the only machine that can upload, delete or manage content on the library machine is in the production studio, but most machines on the internal network can download files from it. One production studio machine runs the overnight automated playlist, and pulls data from the library machine to build files for the accounting package for air verification to generate client advertising invoices.
From the standpoint of operations, the key machine for them is their library machine, since without it, they are dead air. Well, except for a couple of old CD decks in the air studio, which probably, nobody remembers how to use, now... But it is still just a midline Dell PowerEdge, with a single storage expansion chassis for an external tape DLT backup library with an autoloader . But, they don't (yet) stream content via the Internet, and they don't keep a lot of old content on the machine. I think they told me they had about 600 gig of storage on the machine, all told, with about 1/2 that being their commercials archive. It's all on a 10/100 Cat 5 network, and they occasionally have some performance problems they've been told are network related. As of fall 2004, they were looking at doing some network upgrades to resolve this.
Yet they feel it's important to keep functions segregated as they do, and it's generally working for them. So, that's one data point for you. Probably more sophistication than what you had in mind, but still a pretty basic setup, that is kind of standard for small commercial station operations these days.
posted by paulsc at 10:18 AM on January 25, 2006
Seriously, thethirdman makes good points. Some of what he recommends depends, of course, on what the current Mac server is already used for, and what options you have for expandability in hardware. And budget, etc. Countering his argument for simplicity, are experiences I've had with volunteer operations where more and more functions got incrementally added to existing servers, until nothing worked right. More than once, I've "volunteered" to "look" at some church server on a Saturday when the minister couldn't print his sermon, only to find that the small server in the basement had a disk that was reporting 98% utilization, and no one knew the admin password, except the CPA that came in Friday to update Quickbooks (or some similar scenario) ...
Some friends of mine operate a couple of commercial radio stations in western Nebraska, and although their operation is Windows based, at a general level, their set up may be instructive. In their shop, they do a lot of production of commercials and PSA's for their operations, and have a several different servers for various aspects of their business. They do about 6 hours a day of locally generated content (live shows, news, etc.) and about another 4 to 6 hours a day of feeds; the remaining 12 hours, they play music and commercials via set lists (they want to keep a local voice, and don't want to be yet another Clear Channel clone). At the core, they have a small Dell Windows 2003 machine, that acts as their AD root, file and print services for the accounting and business application packages, and similar general chores. In each studio, they have a "board" machine that runs an application package that emulates a traditional studio "board," provides Internet access, and a connection to the rest of the automation, such as their automated telephone switchboard. One studio is also equipped with additional machines for audio editing and post production, where they mainly do their commercials, newscasts, etc. In a server closet with the domain controller, they have a library machine which contains a couple of hundred commercial songs for their playlist, plus their commercials, PSA's, etc. This library machine has a commercial software package that generates their ASCAP and BMI air stats, and manages the library, and stores the content files in lossless format (maybe proprietary). For security, the only machine that can upload, delete or manage content on the library machine is in the production studio, but most machines on the internal network can download files from it. One production studio machine runs the overnight automated playlist, and pulls data from the library machine to build files for the accounting package for air verification to generate client advertising invoices.
From the standpoint of operations, the key machine for them is their library machine, since without it, they are dead air. Well, except for a couple of old CD decks in the air studio, which probably, nobody remembers how to use, now... But it is still just a midline Dell PowerEdge, with a single storage expansion chassis for an external tape DLT backup library with an autoloader . But, they don't (yet) stream content via the Internet, and they don't keep a lot of old content on the machine. I think they told me they had about 600 gig of storage on the machine, all told, with about 1/2 that being their commercials archive. It's all on a 10/100 Cat 5 network, and they occasionally have some performance problems they've been told are network related. As of fall 2004, they were looking at doing some network upgrades to resolve this.
Yet they feel it's important to keep functions segregated as they do, and it's generally working for them. So, that's one data point for you. Probably more sophistication than what you had in mind, but still a pretty basic setup, that is kind of standard for small commercial station operations these days.
posted by paulsc at 10:18 AM on January 25, 2006
Response by poster: Thanks again guys.
The other thing that clouds my thinking on this is my own desire to dip my toes into the *nix environment. The thing is, as thethirdman warns, I don't want to be one of those volunteers who bring in a new approach and then bugger off with no one having any clue what was done.
I think I need to get some more details of the current setup in order to properly answer some of the raised questions. Even budget is debatable. We are finally in a position to be able to spend some money, but I am not sure how much. Hence, the LaCie solution.
Thanks for the rundown of your pal's station paulsc. It does help, and I think to some extent, we are hoping to do a similar setup, but with fewer steps. We want the production studio to have the better machine, and to be able to dump material to the library, and for on-air to be able to access the library. Also, we have two imacs in the office, that we would like to keep clean of local data, so I am seeing a third share to setup.
We have no domain right now, only the workgroup, and only local users I think. Networking is handled via CAT5 and a small soho router of one kind or another.
Paulsc, thanks for your email address...thirdman, can I also talk to you some more once I have some more info? I do suspect that we may be wiser to keep it more toward the mac solution, given that I have two small daughters at home who will likely slow down any learning I want to do about linux!
posted by Richat at 10:31 AM on January 25, 2006
The other thing that clouds my thinking on this is my own desire to dip my toes into the *nix environment. The thing is, as thethirdman warns, I don't want to be one of those volunteers who bring in a new approach and then bugger off with no one having any clue what was done.
I think I need to get some more details of the current setup in order to properly answer some of the raised questions. Even budget is debatable. We are finally in a position to be able to spend some money, but I am not sure how much. Hence, the LaCie solution.
Thanks for the rundown of your pal's station paulsc. It does help, and I think to some extent, we are hoping to do a similar setup, but with fewer steps. We want the production studio to have the better machine, and to be able to dump material to the library, and for on-air to be able to access the library. Also, we have two imacs in the office, that we would like to keep clean of local data, so I am seeing a third share to setup.
We have no domain right now, only the workgroup, and only local users I think. Networking is handled via CAT5 and a small soho router of one kind or another.
Paulsc, thanks for your email address...thirdman, can I also talk to you some more once I have some more info? I do suspect that we may be wiser to keep it more toward the mac solution, given that I have two small daughters at home who will likely slow down any learning I want to do about linux!
posted by Richat at 10:31 AM on January 25, 2006
This thread is closed to new comments.
Apple does make a RAID product, XServe RAID, which I have experience with. The ones I've tended to have been stable over the past two years. This may or may not work for your purposes though.
The XServe RAID is going to be more expensive than a simple firewire RAID setup, but it's also a heftier piece of hardware. If you opt for it with 1TB today, you'll have room for expansion later on. You'll also have to deal with getting a fibre channel card on your OS X Server machine and a more complex setup than with a firewire RAID system.
How's the network at the station? If you're planning on relying on a network link as part of your data stream anyway, you could get a terastation, which will save you some money. I've been happy with the one I helped set up, although it's only been in production for a few months at this point. It's got 4 drives that can be formatted as a RAID and it's served over ethernet via TCP/IP. The downside to this would be loading the data onto it initially, especially if you have only got a 10BT network. A review.
posted by ursus_comiter at 5:17 AM on January 25, 2006