Help me clean up a musical mess.
November 21, 2008 6:29 AM Subscribe
A server on our home network has over 100GB of mp3s in various states of disrepair. How can we organize them, get rid of duplicates, losslessly normalize volume, name and tag them, and prepare them for streaming?
A chunk of these mp3s were burned directly from CD, but the majority were not. They were acquired as collections, albums, and single songs. Half of them have no ID3 tags, and some aren't even named properly.
Here are our major concerns: How do we get them all named correctly in a timely and efficient manner?
What is the best way to organize them, especially when we have some CDs that are "Best of" an artist, but have the artist's other albums with the same songs, and those songs are also part of compilations, and similar situations? Is it even possible to get rid of duplicates in this sort of situation? Especially when we have a lot of Electronic Dance Music, but would like to keep different mixes of the same song...
Where does musicbrainz, as mentioned in this thread, fit in? (Or is that even the best option here?)
How can we normalize the volume for the whole collection without losing quality?
What is the best platform for storing and streaming these files? They are currently on a Windows XP machine that I want to reformat with Ubuntu or Kubuntu.
I have searched the past questions, and most people have some sort of organization already for their collections, or aren't facing the same issues.
Oh, finally, what is the best way to create and organize two different collections for an iPhone and a 60GB iPod? Currently the music is on two different PCs running iTunes for two different collections (yay, more duplication!). Is there a better program than iTunes out there for this? Is there a good way to select songs for the devices without either copying xGB of data to other machines, or going through the tedious process of hand-selecting about 8000 out of 26000+ songs in a library?
Thank you in advance!
A chunk of these mp3s were burned directly from CD, but the majority were not. They were acquired as collections, albums, and single songs. Half of them have no ID3 tags, and some aren't even named properly.
Here are our major concerns: How do we get them all named correctly in a timely and efficient manner?
What is the best way to organize them, especially when we have some CDs that are "Best of" an artist, but have the artist's other albums with the same songs, and those songs are also part of compilations, and similar situations? Is it even possible to get rid of duplicates in this sort of situation? Especially when we have a lot of Electronic Dance Music, but would like to keep different mixes of the same song...
Where does musicbrainz, as mentioned in this thread, fit in? (Or is that even the best option here?)
How can we normalize the volume for the whole collection without losing quality?
What is the best platform for storing and streaming these files? They are currently on a Windows XP machine that I want to reformat with Ubuntu or Kubuntu.
I have searched the past questions, and most people have some sort of organization already for their collections, or aren't facing the same issues.
Oh, finally, what is the best way to create and organize two different collections for an iPhone and a 60GB iPod? Currently the music is on two different PCs running iTunes for two different collections (yay, more duplication!). Is there a better program than iTunes out there for this? Is there a good way to select songs for the devices without either copying xGB of data to other machines, or going through the tedious process of hand-selecting about 8000 out of 26000+ songs in a library?
Thank you in advance!
Media Monkey is designed for managing large music collections. I don't know what you can do to save effort in tagging already-ripped MP3's that aren't properly tagged, though (but there are alot of features I haven't explored). MM will re-name all your files acording to a convention you specify (e.g. name all files with artistname_trackname_albumname_tracknumber) but that's after they're tagged.
posted by winston at 6:39 AM on November 21, 2008
posted by winston at 6:39 AM on November 21, 2008
(actually, artistname_albumname_tracknumber_trackname would make more sense)
posted by winston at 6:40 AM on November 21, 2008
posted by winston at 6:40 AM on November 21, 2008
100GB isn't that much, relatively speaking. I had this same issue years ago, and I decided to to re-rip or re-download it all, and never looked back.
posted by Jairus at 6:42 AM on November 21, 2008
posted by Jairus at 6:42 AM on November 21, 2008
You can use MusicBrainz to tag untagged mp3s.
posted by damn dirty ape at 6:44 AM on November 21, 2008
posted by damn dirty ape at 6:44 AM on November 21, 2008
Wow.. big question.
For tagging:
So, I've used a number of different methods in the past as I've had similar issues. I was a big proponent of MusicBrainz as my tagger of choice, though the last time I used it was several years ago. The one thing I like is that as it tags with a certain level of confidence. Meaning, you don't need to look at the ones that are 100% sure, but you'll need to at least screen the 90% - 99% batch and look much closer at the 89% and less batch. There will also be many songs in which it doesn't have much confidence in, and you'll need to match them up based on an auto search. The bulk of it should be finished in a half hour, but prepare for a large amount of time to sort through the remainder and match them up accordingly.
Naming:
I definitely prefer the naming convention that Winston described above. That way, if they are all in the same folder, they are actually "sorted" appropriately by name.
Managing:
I also agree with using a program such as MediaMonkey to manage the big database. Many other managers simply don't have the muscle. However, I've had good experiences with Foobar and with Songbird (the latter is a new open source player/aggregator that has serious promise).
Streaming:
Not a clue what the best way to do this.
Order:
Not only do I like them to be properly tagged, but I like at least a SMALL inkling of order in the folder systems as well. I have several "main" folder headings such as Electronic, ClassicRock, AlternativeRock, World (different language songs), Reggae, Punk/Ska, HipHop, R&B... etc. Though this is more personal style. Mine doesn't get more granular for Blues, or Jazz etc... I just want folders for common music... then I can add AUTO PLAYLISTS to them from my player/aggregator which is great for instant music listening.
Normalizing:
Audacity has worked great for me for years. Not sure if it can do it lossless, but still a great, robust program.
Hope this helps...
posted by namewithhe1d at 7:39 AM on November 21, 2008
For tagging:
So, I've used a number of different methods in the past as I've had similar issues. I was a big proponent of MusicBrainz as my tagger of choice, though the last time I used it was several years ago. The one thing I like is that as it tags with a certain level of confidence. Meaning, you don't need to look at the ones that are 100% sure, but you'll need to at least screen the 90% - 99% batch and look much closer at the 89% and less batch. There will also be many songs in which it doesn't have much confidence in, and you'll need to match them up based on an auto search. The bulk of it should be finished in a half hour, but prepare for a large amount of time to sort through the remainder and match them up accordingly.
Naming:
I definitely prefer the naming convention that Winston described above. That way, if they are all in the same folder, they are actually "sorted" appropriately by name.
Managing:
I also agree with using a program such as MediaMonkey to manage the big database. Many other managers simply don't have the muscle. However, I've had good experiences with Foobar and with Songbird (the latter is a new open source player/aggregator that has serious promise).
Streaming:
Not a clue what the best way to do this.
Order:
Not only do I like them to be properly tagged, but I like at least a SMALL inkling of order in the folder systems as well. I have several "main" folder headings such as Electronic, ClassicRock, AlternativeRock, World (different language songs), Reggae, Punk/Ska, HipHop, R&B... etc. Though this is more personal style. Mine doesn't get more granular for Blues, or Jazz etc... I just want folders for common music... then I can add AUTO PLAYLISTS to them from my player/aggregator which is great for instant music listening.
Normalizing:
Audacity has worked great for me for years. Not sure if it can do it lossless, but still a great, robust program.
Hope this helps...
posted by namewithhe1d at 7:39 AM on November 21, 2008
There's no single "killer app" that is going to solve the situation you find yourself in. (meaning = no matter what you use its going to take some percentage of "manual labor")
I'm also a big fan of MusicBrainz for the following 2 main reasons:
1.) its pretty reliable scanning unknown files and identifying / fixing the ID3 tags
2.) You can set an option inside MusicBrainz to update the filename (to your specifications) at the same time it updates the ID3 info. VERY HANDY and TIMESAVING.
Those 2 features alone probably save 90% of my mp3 file management tasks. I cant answer your other questions because I dont use any "management" program ( I prefer Winamp with Media Library turned OFF) and I dont do any streaming or normalizing.
and for god sakes ... when you get done putting all that work in... make a backup :)
posted by jmnugent at 8:06 AM on November 21, 2008
I'm also a big fan of MusicBrainz for the following 2 main reasons:
1.) its pretty reliable scanning unknown files and identifying / fixing the ID3 tags
2.) You can set an option inside MusicBrainz to update the filename (to your specifications) at the same time it updates the ID3 info. VERY HANDY and TIMESAVING.
Those 2 features alone probably save 90% of my mp3 file management tasks. I cant answer your other questions because I dont use any "management" program ( I prefer Winamp with Media Library turned OFF) and I dont do any streaming or normalizing.
and for god sakes ... when you get done putting all that work in... make a backup :)
posted by jmnugent at 8:06 AM on November 21, 2008
For lossless normalization, you want ReplayGain, which is supported by foobar2000, Winamp, Media Monkey, and many others and can be applied to any type of audio file with tag support. Because it works on the tags, and not the compressed PCM data, it is completely removable and won't permanently change the audio: it will only change the volume level of individual tracks (in relation to albums or your whole library) on playback in players that support it. You can also set players to prevent clipping, which is useful for improving the sound of your music in situations where an album was mastered poorly or mp3 quantization has induced clipping.
posted by Inspector.Gadget at 8:21 AM on November 21, 2008 [1 favorite]
posted by Inspector.Gadget at 8:21 AM on November 21, 2008 [1 favorite]
I use doublekiller.exe to get rid of dupe files. I think it's great.
http://www.portablefreeware.com/?id=136
posted by cp7 at 8:35 AM on November 21, 2008
http://www.portablefreeware.com/?id=136
posted by cp7 at 8:35 AM on November 21, 2008
Many have tackled the file-level issues here to great effect (thanks for the ideas!), so I'll speak to the streaming. The easiest thing would probably be to get a web-based interface that lets you mark albums, create playlists, after which it spits out an m3u to the client which is picked up by whatever media player they use. Andromeda is the classic example, though it's payware, and Zina is a freeware/OSS version of same. There are certainly others out there and it would be fairly easy to roll your own if you wanted, but hopefully that gets you close to what you're looking for.
posted by rhizome at 9:25 AM on November 21, 2008
posted by rhizome at 9:25 AM on November 21, 2008
Most people use artistname/albumname/tracknumber - trackname which keeps the file name short. Overly long file names can be problematic when burning to CD.
Some people use artistname/releaseyear - albumname/tracknumber - trackname so that albums sort chronologically.
I use artistname - albumname/artistname - albumname - tracknumber - trackname because (1) the file name continues to makes sense no matter where you move the files to, and also when emailing, and (2) because I don't like to have two levels of nesting in my directories.
People who often deal with Linux might prefer artistname_albumname/artistname_albumname_tracknumber_trackname because a fair number of Linux programs struggle to handle spaces in file names, starting with Bash.
Compilations albums which gather songs by many different artists can be named releaselabel - albumname/releaselabel - albumname - tracknumber - artistname - trackname, or the same but without releaselabel. Classical music and Jazz have to deal with many more fields, such as the composer and the performers, which can quickly break most system's file name length limit. I have never seen this addressed properly.
If you have the same song twice, once as part of an album and once as part of a compilation, keep both copies. It is worth spending the extra 5 megabytes of disk space necessary to keep both albums complete in their own directory.
posted by gmarceau at 9:40 AM on November 21, 2008
Some people use artistname/releaseyear - albumname/tracknumber - trackname so that albums sort chronologically.
I use artistname - albumname/artistname - albumname - tracknumber - trackname because (1) the file name continues to makes sense no matter where you move the files to, and also when emailing, and (2) because I don't like to have two levels of nesting in my directories.
People who often deal with Linux might prefer artistname_albumname/artistname_albumname_tracknumber_trackname because a fair number of Linux programs struggle to handle spaces in file names, starting with Bash.
Compilations albums which gather songs by many different artists can be named releaselabel - albumname/releaselabel - albumname - tracknumber - artistname - trackname, or the same but without releaselabel. Classical music and Jazz have to deal with many more fields, such as the composer and the performers, which can quickly break most system's file name length limit. I have never seen this addressed properly.
If you have the same song twice, once as part of an album and once as part of a compilation, keep both copies. It is worth spending the extra 5 megabytes of disk space necessary to keep both albums complete in their own directory.
posted by gmarceau at 9:40 AM on November 21, 2008
I do genre/artist/album/tracknumber - trackname. I also treat singles as a genre for those few tracks that don't get an album to go with them.
I've done crazy amounts of tag fixing with easytag. It's probably faster to download the music again. One of the things I like about bittorrent is that you get wohle albums or discographies rather than single tracks, so even if they aren't up to your standard they're usually consistent within themselves.
posted by valadil at 10:00 AM on November 21, 2008
I've done crazy amounts of tag fixing with easytag. It's probably faster to download the music again. One of the things I like about bittorrent is that you get wohle albums or discographies rather than single tracks, so even if they aren't up to your standard they're usually consistent within themselves.
posted by valadil at 10:00 AM on November 21, 2008
I actually had a similar issue with a similar sized library after buying a NAS server and wanting any and all music on it to be correctly tagged and uniformly named.
My 3 step process:
1) Run MusicBrainz on the whole library to do general song identification and tagging
2) With Tag&Rename sort by various fields (genre/album/artist/etc) to look for incomplete or misssing tags and use their Amazon.com info sync functionality to tag files including Cover Art
3) Use The Godfather to rename and restructure files to my chosen hierarchy, which you can customize pretty easily.
Now that my library is already set up I use steps 2 and 3 for any new music before i copy it to the NAS. Once you get all the settings down, it works pretty quickly.
my folder structure looks like this:
Music/[Artist]/[Album]/[Track#] - [TrackTitle].mp3
I use iTunes primarily to play music, and I can just drag the top level "Music" folder into iTunes to update my library. I used to have genre specific folders, but since everything is tagged correctly i don't think its necessary anymore. Also I only have the track number and title on the actual mp3 filename because my car stereo only displays the first 15 or so characters from the filenames when i burn mp3 CDs.
Like winston mentioned above, mediamonkey is probably the best for a large collection, but since my primary music players are apple products (Iphone/nano/shuffle), iTunes works best for me.
Hope this helps, I know it doesn't exactly apply to your situation, especially in regards to the Ubuntu setup. However, my NAS is Linux based and runs Twonky Media Server to stream stuff from my music folder all over the house, and I use Simplify media which works on linux to stream the same folder over the internet to either my phone or offsite iTunes installations.
posted by theDrizzle at 10:08 AM on November 21, 2008 [1 favorite]
My 3 step process:
1) Run MusicBrainz on the whole library to do general song identification and tagging
2) With Tag&Rename sort by various fields (genre/album/artist/etc) to look for incomplete or misssing tags and use their Amazon.com info sync functionality to tag files including Cover Art
3) Use The Godfather to rename and restructure files to my chosen hierarchy, which you can customize pretty easily.
Now that my library is already set up I use steps 2 and 3 for any new music before i copy it to the NAS. Once you get all the settings down, it works pretty quickly.
my folder structure looks like this:
Music/[Artist]/[Album]/[Track#] - [TrackTitle].mp3
I use iTunes primarily to play music, and I can just drag the top level "Music" folder into iTunes to update my library. I used to have genre specific folders, but since everything is tagged correctly i don't think its necessary anymore. Also I only have the track number and title on the actual mp3 filename because my car stereo only displays the first 15 or so characters from the filenames when i burn mp3 CDs.
Like winston mentioned above, mediamonkey is probably the best for a large collection, but since my primary music players are apple products (Iphone/nano/shuffle), iTunes works best for me.
Hope this helps, I know it doesn't exactly apply to your situation, especially in regards to the Ubuntu setup. However, my NAS is Linux based and runs Twonky Media Server to stream stuff from my music folder all over the house, and I use Simplify media which works on linux to stream the same folder over the internet to either my phone or offsite iTunes installations.
posted by theDrizzle at 10:08 AM on November 21, 2008 [1 favorite]
The tool I've found easiest for finding, managing and removing duplicates is Easy Duplicate File Finder (it's free.)
posted by anadem at 1:21 PM on November 21, 2008
posted by anadem at 1:21 PM on November 21, 2008
I recently retagged and renamed 60GB of mp3's using MusicBrainz Picard with the cover art and last.fm tag plugins. I renamed the top-level folder of my old collection as Untagged and had MusicBrainz move files to a new folder named Tagged. At the end of the process, there was only 6GB of music that MusicBrainz couldn't identify.
posted by PueExMachina at 4:00 PM on November 21, 2008
posted by PueExMachina at 4:00 PM on November 21, 2008
If you have any Windows machines, artistname_albumname_tracknumber_trackname is going to cause problems. Windows XP has a 255 character limit on path names, including the file name. The song name can be something like This is a long song title (featuring Johnny the Piano Master, William the Drum Machine and Phil the Beat Box King), which would run you out of characters pretty quickly.
Use tags to identify your music by artist, album and so on. For naming and organizing, I do this:
Music\Artist\Album\Track No - Track Name
I'd also recommend centralizing your collection on a network-accessible drive rather than having different things in different places. Remember to backup your music.
posted by cnc at 10:33 PM on November 21, 2008
Use tags to identify your music by artist, album and so on. For naming and organizing, I do this:
Music\Artist\Album\Track No - Track Name
I'd also recommend centralizing your collection on a network-accessible drive rather than having different things in different places. Remember to backup your music.
posted by cnc at 10:33 PM on November 21, 2008
My favorite program for tagging and file renaming is Mp3tag
posted by jkaczor at 10:36 PM on November 21, 2008
posted by jkaczor at 10:36 PM on November 21, 2008
I don't think that tagging all your music should be too difficult. A lot of other people have mentioned MediaMonkey. I recently tagged my 30 gig music collection using that. It has a cool search amazon feature, which then you can embed the album art in the tags, and figure everything out. You really have to just be diligent, sit through the boredom, but hey, you get to go through all your music and weed out stuff that you can't believe you had in the first place. 30 gigs took me a few hours, but that was also with a lot of distractions and talking to people, so, put a weekend away and just get it done.
Then, you can use the file rename and reorganize feature to name all your MP3s properly, and put them in folders. I used to have some ridiculous folder scheming, like Classic Rock here, Rock here, Classical over here.... MediaMonkey can rename and reorganize all of your folders, so now I just have it as \My Music\Artist\Album\tracknumber - nameofsong.mp3. So much nicer now. Takes a few seconds to set it to what you want, then you can leave it as it moves everything.
Oh and if you are gonna replaygain your collection, and you are like me and are very album centric, please please please do an Album replaygain, not an individual track replaygain. I hate it when you go to listen to an album and now a normally quiet track is blasting because it got gained separately from the others.
Good Luck!
posted by MaHaGoN at 8:21 PM on November 22, 2008
Then, you can use the file rename and reorganize feature to name all your MP3s properly, and put them in folders. I used to have some ridiculous folder scheming, like Classic Rock here, Rock here, Classical over here.... MediaMonkey can rename and reorganize all of your folders, so now I just have it as \My Music\Artist\Album\tracknumber - nameofsong.mp3. So much nicer now. Takes a few seconds to set it to what you want, then you can leave it as it moves everything.
Oh and if you are gonna replaygain your collection, and you are like me and are very album centric, please please please do an Album replaygain, not an individual track replaygain. I hate it when you go to listen to an album and now a normally quiet track is blasting because it got gained separately from the others.
Good Luck!
posted by MaHaGoN at 8:21 PM on November 22, 2008
I'm going out on a limb to say there is a single killer app to organise your music, and it's foobar (with the masstagger component installed). There are of course other ways and other apps, but I know this one so I'll tell you about it.
Tagging:
- Use masstagger's "guess values from filename" function (or specify a filename -> tag template for difficult files.)
- Or, use the "get tags from freedb" function
- Set a consistent tag type for your library (I'd recommend id3v1 + apev2)
Organizing:
- You may want to use multiple schemes. I have separate schemes for Albums (by a single artist), Compilations, and Singles, viz.
\Music\Albums\%artist%\%album%\%artist% - %album% - %track% - %title%
\Music\Compilations\%album%\%album% - %track% - %artist% - %title%
\Music\Singles\%artist% - %title%
- Once you've tagged everything, foobar has a "Move Files" function that lets you specify a folder and filename pattern (such as the examples above) and apply it to your collection.
Normalising:
- Replaygain. Use the "scan selection as albums" function on your whole library. It will take a while.
Removing Duplicates:
- This will be difficult. Many third party apps will find duplicate (ie. identical) files, but your real problem will be copies of the same song where the file content *isn't* identical (because the copies were encoded separately). I suggest you search for duplicate filenames after your library is tagged and organised / auto-named.
Platform:
- Use whatever you're most comfortable with (assuming you mean streaming via file share to other computers on your network).
iPhone & iPod
- Not sure I understand your question. Maybe set a custom tag for files you want on each device, or put the files into their own playlist.
posted by magic curl at 1:02 AM on November 24, 2008
Tagging:
- Use masstagger's "guess values from filename" function (or specify a filename -> tag template for difficult files.)
- Or, use the "get tags from freedb" function
- Set a consistent tag type for your library (I'd recommend id3v1 + apev2)
Organizing:
- You may want to use multiple schemes. I have separate schemes for Albums (by a single artist), Compilations, and Singles, viz.
\Music\Albums\%artist%\%album%\%artist% - %album% - %track% - %title%
\Music\Compilations\%album%\%album% - %track% - %artist% - %title%
\Music\Singles\%artist% - %title%
- Once you've tagged everything, foobar has a "Move Files" function that lets you specify a folder and filename pattern (such as the examples above) and apply it to your collection.
Normalising:
- Replaygain. Use the "scan selection as albums" function on your whole library. It will take a while.
Removing Duplicates:
- This will be difficult. Many third party apps will find duplicate (ie. identical) files, but your real problem will be copies of the same song where the file content *isn't* identical (because the copies were encoded separately). I suggest you search for duplicate filenames after your library is tagged and organised / auto-named.
Platform:
- Use whatever you're most comfortable with (assuming you mean streaming via file share to other computers on your network).
iPhone & iPod
- Not sure I understand your question. Maybe set a custom tag for files you want on each device, or put the files into their own playlist.
posted by magic curl at 1:02 AM on November 24, 2008
MP3Tag will grab data from cddb, freedb, Amazon, filenames (configurable) and probably a bunch of other places. Once you've appropriately tagged your music, you can use it to rename everything into a neatly structured filesystem (ie, d:\music\%artist%\%artist% - %year% - %album%\%track%. %title%)
It still requires effort, however. I think ~100GB could be handled in 5-6hrs over a couple days. Spend the time, do it right, and never have to do it again.
Things that generally require manual attention:
Various Artists/Compilations/Soundtracks
"The" Bands (I store "the killers" under 't' but "the incredible string band" under i, without the "the" in the folder name. Tags always include the "the" however.
Classical music
Live recordings, unofficial recordings, things for which there is no cddb info.
Don't stress about duplicates - even if you've got a GB of them, that's nothing, space-wise. And there's no elegant solution for 'sharing' them between best-of albums and source albums.
posted by unmake at 5:09 PM on June 24, 2009
It still requires effort, however. I think ~100GB could be handled in 5-6hrs over a couple days. Spend the time, do it right, and never have to do it again.
Things that generally require manual attention:
Various Artists/Compilations/Soundtracks
"The" Bands (I store "the killers" under 't' but "the incredible string band" under i, without the "the" in the folder name. Tags always include the "the" however.
Classical music
Live recordings, unofficial recordings, things for which there is no cddb info.
Don't stress about duplicates - even if you've got a GB of them, that's nothing, space-wise. And there's no elegant solution for 'sharing' them between best-of albums and source albums.
posted by unmake at 5:09 PM on June 24, 2009
This thread is closed to new comments.
posted by Nixie Pixel at 6:31 AM on November 21, 2008