I want to be a curator, not a collector
September 11, 2013 5:11 AM Subscribe
I have a lot of duplicate music and photos on my hard drive - and this is making sorting them out difficult. Is there a good programme out there that will help me out with this?
Music - I have two or three copies of some songs, as I've either ripped various compilation CDs, have downloaded them from different sources over time, or - in the case of The Smiths - the same songs appear on various albums. I also have some songs as studio versions and some as live tracks or taken from the Festive Fifty radio show (complete with John Peel introductions), which I don't want to delete. iTunes will show me duplicates of songs, but this doesn't remove them from my hard drive. At present, my music is all in one 'My Music' folder on my hard drive which is fed into iTunes, and annoyingly, though songs are correctly tagged in the folder they aren't always in iTunes due to having had a lot of old or obscure CDs.
Photos - I am sure that half of my many thousands of photos are duplicate copies, thanks to shifting files round from various hard-drives and SD cards. I did once use a program which identified duplicate file names, but sometimes this isn't reliable, as over the years the cameras I've used have used the same file number eg. IMG0001, on different photos, depending on the camera and memory card I've been using. Cutting out the duplicates will make the task of sorting through these considerably easier.
Is there one or two programmes which will help me do this? Free is good, but if something more powerful and useful exists for a reasonable fee, I'm happy to pony up.
Music - I have two or three copies of some songs, as I've either ripped various compilation CDs, have downloaded them from different sources over time, or - in the case of The Smiths - the same songs appear on various albums. I also have some songs as studio versions and some as live tracks or taken from the Festive Fifty radio show (complete with John Peel introductions), which I don't want to delete. iTunes will show me duplicates of songs, but this doesn't remove them from my hard drive. At present, my music is all in one 'My Music' folder on my hard drive which is fed into iTunes, and annoyingly, though songs are correctly tagged in the folder they aren't always in iTunes due to having had a lot of old or obscure CDs.
Photos - I am sure that half of my many thousands of photos are duplicate copies, thanks to shifting files round from various hard-drives and SD cards. I did once use a program which identified duplicate file names, but sometimes this isn't reliable, as over the years the cameras I've used have used the same file number eg. IMG0001, on different photos, depending on the camera and memory card I've been using. Cutting out the duplicates will make the task of sorting through these considerably easier.
Is there one or two programmes which will help me do this? Free is good, but if something more powerful and useful exists for a reasonable fee, I'm happy to pony up.
We've had a few questions about duplicate file finders on the Green. I asked one last year, and got some good suggestions.
In the end, I went with a program called DupeGuru, which did everything you're looking for. I think the other program that is frequently mentioned is Decloner. Another new entrant is called Gemini.
The programs use algorithms / magic to find duplicate files with the same content, even if they have different names. They work with photos, music, and text files, etc. It was an enjoyable experience culling all the dupes--it felt deliciously productive.
posted by Admiral Haddock at 6:01 AM on September 11, 2013 [7 favorites]
In the end, I went with a program called DupeGuru, which did everything you're looking for. I think the other program that is frequently mentioned is Decloner. Another new entrant is called Gemini.
The programs use algorithms / magic to find duplicate files with the same content, even if they have different names. They work with photos, music, and text files, etc. It was an enjoyable experience culling all the dupes--it felt deliciously productive.
posted by Admiral Haddock at 6:01 AM on September 11, 2013 [7 favorites]
Merlin Mann talked about this earlier in the year. His app recommendations are Mac-centric, but the principles are pretty decent.
You might want to try MusicBrainz to do some kind of canonical tagging.
For photos, it can be trickier because EXIF data can change while the core image data stays the same, so a file looks 'different' from a filesystem perspective. That said, filesystem properties are a good starting point: match on multiple criteria, including file size, creation/modification dates, and MD5 hashes of the file itself.
posted by holgate at 6:06 AM on September 11, 2013
You might want to try MusicBrainz to do some kind of canonical tagging.
For photos, it can be trickier because EXIF data can change while the core image data stays the same, so a file looks 'different' from a filesystem perspective. That said, filesystem properties are a good starting point: match on multiple criteria, including file size, creation/modification dates, and MD5 hashes of the file itself.
posted by holgate at 6:06 AM on September 11, 2013
Response by poster: Sorry, should have said - I'm using Windows 8.
posted by mippy at 6:38 AM on September 11, 2013
posted by mippy at 6:38 AM on September 11, 2013
I've used EasyDuplicateFinder on Widnows and it does successfully find dupes. But the old version I used had a rather poor interface that made it less-than-simple to remove the duplicates. It's fine for a small number of files, but gets overwhelming when you have hundreds. (That is, you have to think about which of each duplicate pair or triple you want to discard, and select the file individually in a possibly-huge list.)
posted by anadem at 12:06 PM on September 11, 2013
posted by anadem at 12:06 PM on September 11, 2013
I've used Duplicate Cleaner Free on Windows 7, and it should work on Windows 8 too.
If you need to be able to compare the actual audio content of files you might have to go for the paid version.
posted by philipy at 1:12 PM on September 11, 2013
If you need to be able to compare the actual audio content of files you might have to go for the paid version.
posted by philipy at 1:12 PM on September 11, 2013
« Older My Magic Wand...Well, It's Not Working | What are essentials for visiting NYC with a baby? Newer »
This thread is closed to new comments.
posted by beerbajay at 5:21 AM on September 11, 2013