Second time's the charm?
December 9, 2009 11:58 AM Subscribe
Please help me rearchitect my Apple-specific wireless home network and iTunes library! Eleven paragraphs of geeky details within (sorry!).
I have a big (~100GB) iTunes library I've been accumulating over at least a decade of ripping all my CDs and, in more recent days, my DVDs as well. That project is still ongoing and may never end. I've invested a lot of hours in encoding and cataloging my physical media, and I want to keep it safe for, well, forever. I'm motivated by a teeny bit of paranoia that my discs will scratch and I'll lose something. I keep a tidy house but I guess you could call me a data hoarder. ;-)
This summer I decided to free up room on my MacBook (occupying that much disk space just for iTunes was pretty ridiculous). I bought a WD My Book Mirror Edition and moved the library to it. I configured RAID mirroring for the dual drives so I'd have a backup in case one failed. Before the move, I regularly backed up the library onto my Time Capsule (using Time Machine) along with all my other non-iTunes stuff; the RAID mirroring made that seem redundant so I deleted the backup and excluded it from all future backups. Also, instead of connecting the MyBook straight to my computer, I plugged it into the Time Capsule USB port so I could carry the laptop around the house and still use iTunes wirelessly.
I was really proud of this setup until a couple issues reared their heads:
Latency. Playing music in iTunes over this network works mostly fine apart from the occasional stutter. There's also often a very long delay (30 seconds or more) between when I tell a new song to play and the audio actually begins. Similar delays between when I "get info" on a file and the info window appears. iTunes beachballs frequently now, completely unresponsive while it does... stuff. If I wait, it eventually takes care of whatever it was doing and becomes usable again. So annoying.
Latency's an even bigger issue when watching movies. By default, the Time Capsule was configured for 802.11b/g/n compatibility and movies were unwatchably laggy. I found that forcing pure 802.11n improved performance a lot and movies ran almost as smoothly as music, with only occasional, forgivable hiccups and long delays for play/pause, scrubbing, etc. The downside of killing 802.11b/g compatibility is that my iPhone 3G can no longer use the network, as it only goes up to g. I really miss that functionality, for things like Airfoil Speakers and iTunes Remote.
Another issue is the unfamiliarity of maintaining a network drive that's out of sight, out of mind. Whenever I want to use iTunes I have to mount the drive (I run a script to do it automatically at logon). Whenever I leave the house I have to unmount it. I've forgotten a couple times and disconnected improperly. There were also a couple times when I had trouble getting online and reset the Time Capsule, forgetting that the disk was mounted. Which leads us to the next sad, sad paragraph...
This past weekend, tragedy struck. I think it was related to an improper disconnect while data was being written. Suddenly iTunes refused to read the drive at all. Time Capsule reported a "problem" with the drive. I plugged it into the MacBook and ran Disk Utility. It failed to verify or repair the disk and said I would have to format it. What's worse, the mirrored drive was identically corrupted too. I lost my whole library and the supposed "failsafe!" Crap! I really botched this one up.
I'm investigating data recovery options. I should be able to restore my files. Not sure how this is going to play out but it makes my heart ache. Any suggestions for cheap-as-in-free ways to get my library back in working condition would be appreciated.
But what I'm really asking for in this question is the best way to prevent this from happening again and in a way that works better than my first attempt. Assuming the drive is not mechanically broken and I'll be able to get my files back, how can I redesign my network to do all of the following:
I have a big (~100GB) iTunes library I've been accumulating over at least a decade of ripping all my CDs and, in more recent days, my DVDs as well. That project is still ongoing and may never end. I've invested a lot of hours in encoding and cataloging my physical media, and I want to keep it safe for, well, forever. I'm motivated by a teeny bit of paranoia that my discs will scratch and I'll lose something. I keep a tidy house but I guess you could call me a data hoarder. ;-)
This summer I decided to free up room on my MacBook (occupying that much disk space just for iTunes was pretty ridiculous). I bought a WD My Book Mirror Edition and moved the library to it. I configured RAID mirroring for the dual drives so I'd have a backup in case one failed. Before the move, I regularly backed up the library onto my Time Capsule (using Time Machine) along with all my other non-iTunes stuff; the RAID mirroring made that seem redundant so I deleted the backup and excluded it from all future backups. Also, instead of connecting the MyBook straight to my computer, I plugged it into the Time Capsule USB port so I could carry the laptop around the house and still use iTunes wirelessly.
I was really proud of this setup until a couple issues reared their heads:
Latency. Playing music in iTunes over this network works mostly fine apart from the occasional stutter. There's also often a very long delay (30 seconds or more) between when I tell a new song to play and the audio actually begins. Similar delays between when I "get info" on a file and the info window appears. iTunes beachballs frequently now, completely unresponsive while it does... stuff. If I wait, it eventually takes care of whatever it was doing and becomes usable again. So annoying.
Latency's an even bigger issue when watching movies. By default, the Time Capsule was configured for 802.11b/g/n compatibility and movies were unwatchably laggy. I found that forcing pure 802.11n improved performance a lot and movies ran almost as smoothly as music, with only occasional, forgivable hiccups and long delays for play/pause, scrubbing, etc. The downside of killing 802.11b/g compatibility is that my iPhone 3G can no longer use the network, as it only goes up to g. I really miss that functionality, for things like Airfoil Speakers and iTunes Remote.
Another issue is the unfamiliarity of maintaining a network drive that's out of sight, out of mind. Whenever I want to use iTunes I have to mount the drive (I run a script to do it automatically at logon). Whenever I leave the house I have to unmount it. I've forgotten a couple times and disconnected improperly. There were also a couple times when I had trouble getting online and reset the Time Capsule, forgetting that the disk was mounted. Which leads us to the next sad, sad paragraph...
This past weekend, tragedy struck. I think it was related to an improper disconnect while data was being written. Suddenly iTunes refused to read the drive at all. Time Capsule reported a "problem" with the drive. I plugged it into the MacBook and ran Disk Utility. It failed to verify or repair the disk and said I would have to format it. What's worse, the mirrored drive was identically corrupted too. I lost my whole library and the supposed "failsafe!" Crap! I really botched this one up.
I'm investigating data recovery options. I should be able to restore my files. Not sure how this is going to play out but it makes my heart ache. Any suggestions for cheap-as-in-free ways to get my library back in working condition would be appreciated.
But what I'm really asking for in this question is the best way to prevent this from happening again and in a way that works better than my first attempt. Assuming the drive is not mechanically broken and I'll be able to get my files back, how can I redesign my network to do all of the following:
- Access my iTunes library wirelessly
- Minimize latency
- Keep a backup of the library that's actually safe and won't give me any more headaches
- Use my iPhone on the network
- Require no additional hardware purchases if at all possible
Opps, I see you already mentioned iphone remote in your post.
posted by travis08 at 12:11 PM on December 9, 2009
posted by travis08 at 12:11 PM on December 9, 2009
Response by poster: Sorry, I know my post was tl;dr! I love Remote but can't connect my iPhone to a purely 802.11n network. I have to run the network at that speed otherwise iTunes doesn't have the bandwidth to play movies. :-(
posted by The Winsome Parker Lewis at 12:14 PM on December 9, 2009
posted by The Winsome Parker Lewis at 12:14 PM on December 9, 2009
Maybe you could switch your network back to b/g/n compatibility, use remote for wireless music access, and simply bring the hard drive with you when you want to watch a movie on your laptop in a different room.
Another great application for the iphone is SimplifyMedia, this lets you stream all the music in your itunes library to the iphone itself. Assuming you currently drag your laptop to different rooms and hook it up to a stereo, you now can just hook up you iphone instead.
posted by travis08 at 12:19 PM on December 9, 2009
Another great application for the iphone is SimplifyMedia, this lets you stream all the music in your itunes library to the iphone itself. Assuming you currently drag your laptop to different rooms and hook it up to a stereo, you now can just hook up you iphone instead.
posted by travis08 at 12:19 PM on December 9, 2009
I can't help you with the data recovery, but I strongly recommend cloning your important hard drive using SuperDuper! (the exclamation mark is part of the name). You could even make multiple clones and store one of them off-site, for example at your place of work. Hard disks are cheap nowadays.
RAID is not backup; it exists to enable your system to keep running when a hard drive fails. This is important for a database used for running a business, but probably not for something like a media library. RAID doesn't protect you from things like accidentally deleting files, your RAID array being stolen, or your house burning down.
posted by Gilbert Osmond at 12:26 PM on December 9, 2009
RAID is not backup; it exists to enable your system to keep running when a hard drive fails. This is important for a database used for running a business, but probably not for something like a media library. RAID doesn't protect you from things like accidentally deleting files, your RAID array being stolen, or your house burning down.
posted by Gilbert Osmond at 12:26 PM on December 9, 2009
Duh, I should have said "RAID doesn't protect your data from things like ...".
posted by Gilbert Osmond at 12:28 PM on December 9, 2009
posted by Gilbert Osmond at 12:28 PM on December 9, 2009
Best answer: First, a little light ribbing....100GB of data does not make one a data hoarder. However, 1.7 terabytes of data (who me?) might classify one as a data hoarder.
I built my setup to get around some of the issues you're discussing.
- AppleTV, flashed with aTV Flash
- ReadyNAS NV+ with 4 x 500GB HDs, RAID 5. Ethernet wired to the AppleTV via a network switch (the LAN switch in an Airport Extreme Base station).
- Airport Extreme Base Station to act as a router for wired and wireless
- an iMac that's wireless and acts as the DVD ripper, controller, configurator, etc. for the network.
The flashed AppleTV allows me to run XBMC as my media center library manager. While iTunes can still be a decent music library, I detest it as a video library manager.
Eventually I'll replace the AppleTV with a Mac mini that acts as my media library manager. The AppleTV is just too underpowered, and XBMC will run on just about anything.
My music library lives on both my iMac (260GB, 60,000 songs) and on my AppleTV (which I upgraded with a 320GB HD), so I allow iTunes to sync the music and my iPhoto library but nothing else.
Access my iTunes library wirelessly
My solution allows me to sync my iTunes music library wirelessly. My iPhone and my iMac can all see it and stream to it. Music is a much lower streaming overhead than HD video.
Minimize latency
The data store on the ReadyNAS is linked via ethernet at gigabit speeds to the device acting as a video player (the AppleTV running XBMC). Virtually no latency at all.
Keep a backup of the library that's actually safe and won't give me any more headaches
While I haven't backed up the 1.x TB of video data on my ReadyNAS, I do have all the music synced and backed up in several places (AppleTV, iMac, the backup HD connected to the iMac). Eventually when 2TB and larger HDs get cheap enough, I'll attach a giant USB drive to the ReadyNAS USB port and set up an rsync between the RAID 5 and the giant USB HD. Eventually.
Use my iPhone on the network
My iPhone and iPod Touch connect with zero problems with my Airport Extreme set to B/G/N.
Require no additional hardware purchases if at all possible
I see your Time Capsule as your weak link. It could easily be repurposed as a backup device, but you should investigate some of the NAS devices out there and install one as your primary media storage point.
My video library lives on the ReadyNAS. XBMC allows me to establish an SMB share connection between the ReadyNAS and the AppleTV over ethernet. I can easily stream videos between the two devices without even involving iTunes or my iMac because the content isn't on the iMac or even on the AppleTV.
posted by mrbarrett.com at 1:31 PM on December 9, 2009 [2 favorites]
I built my setup to get around some of the issues you're discussing.
- AppleTV, flashed with aTV Flash
- ReadyNAS NV+ with 4 x 500GB HDs, RAID 5. Ethernet wired to the AppleTV via a network switch (the LAN switch in an Airport Extreme Base station).
- Airport Extreme Base Station to act as a router for wired and wireless
- an iMac that's wireless and acts as the DVD ripper, controller, configurator, etc. for the network.
The flashed AppleTV allows me to run XBMC as my media center library manager. While iTunes can still be a decent music library, I detest it as a video library manager.
Eventually I'll replace the AppleTV with a Mac mini that acts as my media library manager. The AppleTV is just too underpowered, and XBMC will run on just about anything.
My music library lives on both my iMac (260GB, 60,000 songs) and on my AppleTV (which I upgraded with a 320GB HD), so I allow iTunes to sync the music and my iPhoto library but nothing else.
Access my iTunes library wirelessly
My solution allows me to sync my iTunes music library wirelessly. My iPhone and my iMac can all see it and stream to it. Music is a much lower streaming overhead than HD video.
Minimize latency
The data store on the ReadyNAS is linked via ethernet at gigabit speeds to the device acting as a video player (the AppleTV running XBMC). Virtually no latency at all.
Keep a backup of the library that's actually safe and won't give me any more headaches
While I haven't backed up the 1.x TB of video data on my ReadyNAS, I do have all the music synced and backed up in several places (AppleTV, iMac, the backup HD connected to the iMac). Eventually when 2TB and larger HDs get cheap enough, I'll attach a giant USB drive to the ReadyNAS USB port and set up an rsync between the RAID 5 and the giant USB HD. Eventually.
Use my iPhone on the network
My iPhone and iPod Touch connect with zero problems with my Airport Extreme set to B/G/N.
Require no additional hardware purchases if at all possible
I see your Time Capsule as your weak link. It could easily be repurposed as a backup device, but you should investigate some of the NAS devices out there and install one as your primary media storage point.
My video library lives on the ReadyNAS. XBMC allows me to establish an SMB share connection between the ReadyNAS and the AppleTV over ethernet. I can easily stream videos between the two devices without even involving iTunes or my iMac because the content isn't on the iMac or even on the AppleTV.
posted by mrbarrett.com at 1:31 PM on December 9, 2009 [2 favorites]
Doesn't answer your questions at all, but there is a lot of food for thought about these issues *here* perhaps it can be of some assistance to you and others now or in the future.
posted by VikingSword at 1:47 PM on December 9, 2009
posted by VikingSword at 1:47 PM on December 9, 2009
I can help with the networking part. We have an Airport running with 802.11n and an older one running on 802.11g that's bridged to it. The iPhones run on the g network and the laptops and Apple TV and Roku all run on the n network. You will need a second older Airport, but you should be able to find one on the cheap.
If you need help going this way, MeMail me. My husband set our network up, and he's walked more than one friend through getting their network in order.
posted by immlass at 2:00 PM on December 9, 2009 [1 favorite]
If you need help going this way, MeMail me. My husband set our network up, and he's walked more than one friend through getting their network in order.
posted by immlass at 2:00 PM on December 9, 2009 [1 favorite]
If you replace your Time Capsule with a new Airport Extreme + external HD, you would have the benefit of the AE's dual-band setup -- that is, it will run both n and g networks on separate bands, allowing for the best of both worlds (your macbook would connect to the n, the iphone to the g)...
posted by modernnomad at 2:33 PM on December 9, 2009
posted by modernnomad at 2:33 PM on December 9, 2009
Response by poster: modernnomad: Is this different from the Time Capsule's compatibility mode (or whatever it's called; I don't have it in front of me at the moment)? It communicates just fine with n and g under that mode. It was only when I tried to stream full-screen video through it that I realized I wasn't getting full "n" performance. It seemed strange to me but there appeared to be some throttling going on in that setup, and it was fixed when I forced it into n-mode only.
posted by The Winsome Parker Lewis at 3:44 PM on December 9, 2009
posted by The Winsome Parker Lewis at 3:44 PM on December 9, 2009
RAID != backup, as you have experienced. Errors get replicated, and power spikes or the like usually fry both drives.
rsync isn't backup either, as (human) errors will get replicated as well. Accidentally delete 100 Gb of data? Hope you notice it before the next sync cycle or it'll be gone forever.
Superduper, Time Machine and other options that allow you to copy all your data to another physical drive *and then detach it and store it* do protect you from this kind of errors. Preferably if you've got several drives you use in a rotating scheme, to protect you from dual failures (primary and backup are corrupt) and accidental errors you only notice down the road.
Good luck with your data recovery!
posted by lodev at 7:26 AM on December 10, 2009
rsync isn't backup either, as (human) errors will get replicated as well. Accidentally delete 100 Gb of data? Hope you notice it before the next sync cycle or it'll be gone forever.
Superduper, Time Machine and other options that allow you to copy all your data to another physical drive *and then detach it and store it* do protect you from this kind of errors. Preferably if you've got several drives you use in a rotating scheme, to protect you from dual failures (primary and backup are corrupt) and accidental errors you only notice down the road.
Good luck with your data recovery!
posted by lodev at 7:26 AM on December 10, 2009
I have a wild guess that the latency you're experiencing when moving to a new track in itunes stems from having not only your music files, but also your itunes library database, on the external drive. is that true? If so, moving the database back to your macbook and just pointing to the files on the external drive should fix it. However, if you start itunes without the drive mounted, you'll probably get question marks on all your songs and videos.
I don't know a perfect solution but I have similar concerns, so I'm watching this thread closely.
posted by Chris4d at 10:02 AM on December 10, 2009
I don't know a perfect solution but I have similar concerns, so I'm watching this thread closely.
posted by Chris4d at 10:02 AM on December 10, 2009
This thread is closed to new comments.
Using this would allow you to keep your backup drives wired to the laptop, but still control itunes from anywhere in your house (assuming your iphone is on the same wireless network as computer)
posted by travis08 at 12:07 PM on December 9, 2009