Help me archive my image files!
September 6, 2006 8:59 PM Subscribe
PC-based digital photographer here... Does anyone supplement camera RAW file archives with JPEGs, and/or use any specific browsing/cataloging apps to speed up library searches?
I recently transitioned to using RAW capture on my digital SLR, as opposed to writing JPEG format. I've noticed that 6mp source JPEGs are not the fastest-loading file format out there, but the RAW files are egregiously slow to load in all applications on my PC (Photoshop, Picasa, IrfanView, MS Picture Viewer shell extension, etc). Sure, they load fine, but they take so damn long! I can't just flip-through pictures; I have to wait for them to load and sharpen in nearly all my applications. I feel the need to correct this somehow.
I have disk space (much) + plenty of RAM (1GB), and my processor should be plenty fast (1.8ghz P4) to handle the sort of images we're talking about here. So the performance bottleneck, sans upgrades, is the size/complexity of the RAW files. And I do NOT plan to buy more equiment to fix this.
Anyway, the bigger problem is that I'm at the point where I'd like to take about 70 GB of source images offline, but I'm undecided about the storage media (do I really buy an offline hard disk for this? DVD-RAM is too small and perishable) and the cataloging (if I don't make small local copies of the image files, how will I know what is what exactly?). I'd enjoy best a solution that provides automated offline backups, but that also keeps sufficient local copies so that I can browse through and pull larger files from archives. I'm not sure if Picasa or Adobe Bridge fit these needs on my platform. (FYI, Bridge is dog-slow on my computer)
A potential solution: go through a tedious process of generating 600x400 samples of each RAW image in a replica directory tree. These I can browse through efficiently and store locally. It doesn't link to originals through any one-click solution, but with my existing storage tree, I'm sure I can match the preview/small images to their offline parents pretty easily (as long as I have online access to them... in which case I have an external-chassis 300GB hard drive to store copy #1). And while I have local archive access figured out (USB2.0 300GB HD), having an extra offline HD copy of the originals would be my catastropic recovery solution.
Thoughts?
I recently transitioned to using RAW capture on my digital SLR, as opposed to writing JPEG format. I've noticed that 6mp source JPEGs are not the fastest-loading file format out there, but the RAW files are egregiously slow to load in all applications on my PC (Photoshop, Picasa, IrfanView, MS Picture Viewer shell extension, etc). Sure, they load fine, but they take so damn long! I can't just flip-through pictures; I have to wait for them to load and sharpen in nearly all my applications. I feel the need to correct this somehow.
I have disk space (much) + plenty of RAM (1GB), and my processor should be plenty fast (1.8ghz P4) to handle the sort of images we're talking about here. So the performance bottleneck, sans upgrades, is the size/complexity of the RAW files. And I do NOT plan to buy more equiment to fix this.
Anyway, the bigger problem is that I'm at the point where I'd like to take about 70 GB of source images offline, but I'm undecided about the storage media (do I really buy an offline hard disk for this? DVD-RAM is too small and perishable) and the cataloging (if I don't make small local copies of the image files, how will I know what is what exactly?). I'd enjoy best a solution that provides automated offline backups, but that also keeps sufficient local copies so that I can browse through and pull larger files from archives. I'm not sure if Picasa or Adobe Bridge fit these needs on my platform. (FYI, Bridge is dog-slow on my computer)
A potential solution: go through a tedious process of generating 600x400 samples of each RAW image in a replica directory tree. These I can browse through efficiently and store locally. It doesn't link to originals through any one-click solution, but with my existing storage tree, I'm sure I can match the preview/small images to their offline parents pretty easily (as long as I have online access to them... in which case I have an external-chassis 300GB hard drive to store copy #1). And while I have local archive access figured out (USB2.0 300GB HD), having an extra offline HD copy of the originals would be my catastropic recovery solution.
Thoughts?
I'd enjoy best a solution that provides automated offline backups
But you say this:
And I do NOT plan to buy more equiment to fix this.
Any backup solution will require you to either buy equipment, such as a tape drive, or online data hosting capacity, either with backup software.
but that also keeps sufficient local copies so that I can browse through and pull larger files from archives. I'm not sure if Picasa or Adobe Bridge fit these needs on my platform. (FYI, Bridge is dog-slow on my computer)
You'll need a database of some sort. You might look into options listed in this Ask Metafilter thread.
posted by Blazecock Pileon at 9:18 PM on September 6, 2006
But you say this:
And I do NOT plan to buy more equiment to fix this.
Any backup solution will require you to either buy equipment, such as a tape drive, or online data hosting capacity, either with backup software.
but that also keeps sufficient local copies so that I can browse through and pull larger files from archives. I'm not sure if Picasa or Adobe Bridge fit these needs on my platform. (FYI, Bridge is dog-slow on my computer)
You'll need a database of some sort. You might look into options listed in this Ask Metafilter thread.
posted by Blazecock Pileon at 9:18 PM on September 6, 2006
Best answer: Check out iView MediaPro. I've been using it for years to keep offline catalogs of my photos. Extensis Portfolio is also another product to consider.
Regarding moving 70GB, no matter what medium you move that to, you'll probably need to move them again in a few years anyway. No medium lasts forever.
posted by lunarboy at 10:35 PM on September 6, 2006
Regarding moving 70GB, no matter what medium you move that to, you'll probably need to move them again in a few years anyway. No medium lasts forever.
posted by lunarboy at 10:35 PM on September 6, 2006
Currently cheapest media for large volumes of data: hard disk drives in the 320GB region.
Buy a new drive every couple of years (the new drive will naturally be far more capacious) and transfer your entire collection to it. That way you're online, backed up and future-proofed.
posted by flabdablet at 12:51 AM on September 7, 2006
Buy a new drive every couple of years (the new drive will naturally be far more capacious) and transfer your entire collection to it. That way you're online, backed up and future-proofed.
posted by flabdablet at 12:51 AM on September 7, 2006
When you say "Microsoft Picture Viewer Shell Extension" you don't mean Microsoft's RAW thumbnailer and viewer, do you?
Check it out here.
posted by roomwithaview at 5:40 AM on September 7, 2006
Check it out here.
posted by roomwithaview at 5:40 AM on September 7, 2006
Response by poster: Blazecock Pileon (great name BTW):
Hah, I surely contradicted myself there, but for brevity's sake: Investing in performance at this point (like a new mobo) is out of the question because of budget concerns, but I wasn't referring to backup media when saying I wouldn't spend. I certainly can afford an inexpensive drive to take a new form of backup media (although I have a DVD burner and an external 300GB USB drive) and whatever media necessary.
lunarboy: I'll check both out. And yeah, as someone who's done 15 years of shuffling personal documents from computer to computer, I certainly know this.
flabdablet: If I had to make an executive decision today about backup media, that's exactly what I'd do. But I don't think it would help with the RAW file load time, and I have double backups already, so no rush on this.
roomwithaview: yeah, I've got that! And its the quickest thing I've got besides Picasa, but try flipping through 10,000 photos with that and you'll see what "slow" is. (Even with 6mp JPEGs, it ain't quick)
posted by brianvan at 6:32 AM on September 7, 2006
Hah, I surely contradicted myself there, but for brevity's sake: Investing in performance at this point (like a new mobo) is out of the question because of budget concerns, but I wasn't referring to backup media when saying I wouldn't spend. I certainly can afford an inexpensive drive to take a new form of backup media (although I have a DVD burner and an external 300GB USB drive) and whatever media necessary.
lunarboy: I'll check both out. And yeah, as someone who's done 15 years of shuffling personal documents from computer to computer, I certainly know this.
flabdablet: If I had to make an executive decision today about backup media, that's exactly what I'd do. But I don't think it would help with the RAW file load time, and I have double backups already, so no rush on this.
roomwithaview: yeah, I've got that! And its the quickest thing I've got besides Picasa, but try flipping through 10,000 photos with that and you'll see what "slow" is. (Even with 6mp JPEGs, it ain't quick)
posted by brianvan at 6:32 AM on September 7, 2006
Response by poster: Oh and Brian James: Yeah, if I did that recursively on my photo directory, it would probably work. But I'd have to leave my computer running on it for a day or two...
posted by brianvan at 6:47 AM on September 7, 2006
posted by brianvan at 6:47 AM on September 7, 2006
Would performance be better if you conerted to Adobe's .DNG format?
posted by unmake at 9:41 AM on September 8, 2006
posted by unmake at 9:41 AM on September 8, 2006
Response by poster: .DNG is something I will try out. I'm most worried about loss of functionality through compatibility issues (the various programs I use to handle different parts of my workflow may not read .DNG files if it's an Adobe format), but I think that isn't going to be a problem. I really don't think it'll be much faster, though - maybe in Bridge it'll be faster, but it'll require permanent conversion. I don't have the space to store two types of RAW data. (I shoot about 1,000 - 2,000 photos a month, I'd fill all my hard drives)
A different possiblity is using a program that makes contact sheets, or possibly assigning ratings to files using whatever widely applicable system exists for that. I'm sure contact sheet generation could be automated, but I figure I can't just select a bunch of thumbnails from a contact sheet and drag them into a window to get them to edit/print. As for ratings, I'd do the tedious work of rating (and also of cleaning out the truly useless photos, cutting my archive by about 10% / 7GB) if I knew that I'd be able to use the ratings universally and to use them as a filter. I don't know enough about metadata standards to predict how that would pan out. (Will my ratings in Picasa work in Bridge and vice versa? Can I hide everything that's not rated "best" in any of these various DAM applications?) Questions, questions...
posted by brianvan at 12:10 PM on September 8, 2006
A different possiblity is using a program that makes contact sheets, or possibly assigning ratings to files using whatever widely applicable system exists for that. I'm sure contact sheet generation could be automated, but I figure I can't just select a bunch of thumbnails from a contact sheet and drag them into a window to get them to edit/print. As for ratings, I'd do the tedious work of rating (and also of cleaning out the truly useless photos, cutting my archive by about 10% / 7GB) if I knew that I'd be able to use the ratings universally and to use them as a filter. I don't know enough about metadata standards to predict how that would pan out. (Will my ratings in Picasa work in Bridge and vice versa? Can I hide everything that's not rated "best" in any of these various DAM applications?) Questions, questions...
posted by brianvan at 12:10 PM on September 8, 2006
This thread is closed to new comments.
A potential solution: go through a tedious process of generating 600x400 samples of each RAW image in a replica directory tree.
I basically have done this for my own RAW files, except I created an action in Photoshop to do it for me, and had it run against a whole mess of photos using the Batch feature.
posted by Brian James at 9:12 PM on September 6, 2006