Introduction

This is an early version of what is planned to be a comprehensive guide to building and managing a large music library. It's not a walk-through of all possible different ways to do things, but more a list of the qualified choices I have personally made, and why.

Author

This guide was written by Jacob Laursen, November 2010. E-mail: jac...@vindvejr.dk

Background

I have written this guide partly to document my own setup and choices, so I can continue building my library in a consistent way. Also, since I have gone through various considerations, I might as well share these with the world, so others can perhaps skip some of the mistakes I've made and learned from.

Prerequisites

Knowledge of Linux is not required, but some scripts are provided that will only be useful if posessing this knowledge.

Content

This guide cover the following topics:

Ripping the CD's

Hundreds, if not thousands, of different audio grabbers exists out there. One program, however, stands out: Exact Audio Copy for Windows. It will rip a CD almost perfectly, even in case of errors on the CD. And it's free!

By using EAC you will benefit from the online freedb database, so that you don't have to manually tag all your files. However, always remember to check if the tags are correct. Many many errors exist in the freedb database. Correct the mistakes before ripping, this will save you from a lot of headache later on.

After ripping the CD, remember to save the logfile generated by EAC. This can be a help in the future, if you need to find out if there are errors in some of the files, and you need to decide whether to re-rip the CD in question, if presented for the opportunity. Always save the logfile with the same filename, for example `eac.log', so you can perform scripted searches. To find all albums with ripping errors, use:

# grep -r "There were errors" --include eac.log /mnt/diskstation/music/flac/*

Use option "Append Gaps To Previous Track (default)". This is the least wrong way to store gaps (silence) between tracks. If left out, you can't play the album like it was intended - silence between tracks will be completely removed. If appended to next track, it will be the first thing you hear in a track. This is annoying when skipping tracks or making playlists -- after selecting a new song you will not expect a delay because of silence in the beginning of the track.

Encoding the audio files

Many different audio codecs exist, each with different advantages and disadvantages. Important parameters to consider:

Many years ago I decided to rip all my music in mp3PRO format, at 96 kbps. At the time this was a reasonable choice with both software (although limited) and hardware support (portable mp3PRO player). I just forgot to consider one parameter: the future. Today the format is dead, support has disappered and the quality does no longer live up to my expensive Hi-Fi equipment.

The one choice that will never be regretted is a lossless audio codec. It's the only future-proof choice. The files can always be converted to any other lossless format, without having to rip all the music once again, and without losing quality. If the music files are needed on a portable device with limited space, just convert the files to any lossy format the device supports.

I found FLAC to be the best lossless audio codec available. It's not the one with the best compression, but it has the best hardware and software support -- just to name a few products:

See also Monkey's Audio vs. WavPack vs. FLAC.

Adding additional tags

After ripping the CD there may be a need to add additional tags, i.e. multiple genres, barcode, disc number or other stuff. Using program `metaflac' the following command-lines are useful:

  1. Add barcode for a complete album:
    # metaflac --preserve-modtime --set-tag=ean/upn=724354399428 *.flac
  2. List tags in a file:
    # metaflac --list 01*.flac | grep comment

Adding replay-gain

I use the following script in a nightly cronjob to add replay-gain to all newly ripped albums:

#!/bin/bash

MUSIC_LIBRARY="/mnt/diskstation/music/flac"
SCAN_DAYS=1

find $MUSIC_LIBRARY -type d -mtime -$SCAN_DAYS -exec bash -c '\
        if ls "{}"/*.flac >&/dev/null; then \
                metaflac --add-replay-gain --preserve-modtime "{}"/*.flac; \
        fi \
        ' \;
You can't add this when ripping the individual audio files, since the replay-gain for the complete album can only be calculated when all files have been ripped.

Tagging the music

Using Vorbis comments a small number of standard field names are defined. What may have seemed excessive in the ID3v2 specification is now missed, since there is no well-defined way of adding useful information in addition to the standard fields (such as artist, album, title, track number, etc.)

In addition to the standard fields, the following fields are more or less de-facto standards (from different sources):

In Exact Audio Copy, the tags can be added with the following additional command-line options for FLAC (in Compression options -> External Compression):

--compression-level-6 --verify -T "artist=%a" -T "title=%t" -T "album=%g" -T "date=%y" -T "tracknumber=%n" -T "discid=%f" -T "genre=%m" %s -o %d
  
For version 1.0 beta 2 and later:
--compression-level-6 --verify -T "artist=%artist%" -T "title=%title%" -T "album=%albumtitle%" -T "date=%year%" -T "tracknumber=%tracknr%" -T "discnumber=%cdnumber%" -T "discid=%cddbid%" -T "genre=%genre%" %source% -o %dest%
  
For version 1.0 beta 3 and later:
--compression-level-6 --verify -T "artist=%artist%" -T "title=%title%" -T "album=%albumtitle%" -T "date=%year%" -T "tracknumber=%tracknr1%" -T "discid=%cddbid%" -T "genre=%genre%" %source% -o %dest%
  

Genres

I also use the following custom genres in addition to the ones defined in the ID3 specification:

By using these genre tags, it's easy, for instance, to make a playlist with Danish christmas music.

Multiple genres

Storing multiple genres is done by simply repeating the genre tag, i.e.:

metaflac --set-tag=genre=Pop --set-tag=genre=Rap
  
FLAC files with multiple genres are supported by: They are not supported by:

Multiple artists

When a song features additional artist(s), decide on a consistent naming scheme for this. For example, "Edward Maya featuring Vika Jugulina - Stereo Love" may be written on the back cover. Don't include the "featuring" information in the artist tag. Move it to the title tag, i.e. artist "Edward Maya" and title "Stereo Love (feat. Vika Jugulina)". This way you will only have "Edward Maya" as an artist once, and not all kinds of variations. Also write it in a consistent way, i.e. always "(feat. artist 2)", "(ft. artist 2)", "feat. artist 2" or whatever you like.

Multi-disc albums

Unfortunately the non-standard "discnumber" tag is not supported by all players. Hence, to avoid blending of the discs in the index, the disc number has to be included in the album title. Use a consistent naming scheme, for example "Greatest Hits (CD 1)" or "Greatest Hits (Disc 1)". Should the discnumber tag be fully supported one day, this would make it possible to script a conversion.

Directory structure

A nice clean directory structure can help keep an overview of the music collection. Making the right choices here can also help:

I use the following naming convention: "<ROOT_DIRECTORY>/<artist> - <album>/[CD <discnumber>/] <tracknumber> - <artist> - <title>.<extension>" For example: With this structure I can have my DLNA and Squeezebox servers use the most specific cover picture found. First it will attempt to find "Folder.jpg", then "../Folder.jpg". Thus, for multi-disc albums with only one cover, I need to store this cover only once. For multi-disc albums with separate covers for each disc (i.e. box sets) the specific covers are stored separately in each album directory.

Exact Audio Copy can create this basic with naming scheme "%D - %C\%N - %A - %T" (EAC options -> Filename) - or "%albumartist% - %albumtitle%\%tracknr2% - %artist% - %title%" for EAC 1.0 beta 2 and later. However, for multi-disc albums the directories need to be manually renamed and moved.

Hacking Squeezebox Server

To get this multi-disc cover art trick to work, go to the Squeezebox Server settings, page "Advanced" and set Formatting/Artwork to "../Folder.jpg". This will tell the server to look for "../Folder.jpg" first and then attempt the default names afterwards. Since we would prefer it the other way around, let's modify this logic. Look for a file called SqueezeCenter/Slim/Music/Artwork.pm. If you own a Synology NAS, it's placed here: /volume1/@appstore/SqueezeCenter/Slim/Music/Artwork.pm. Apply this patch (for version 7.5.3):

--- /volume1/@appstore/SqueezeCenter/Slim/Music/Artwork.pm_orig
+++ /volume1/@appstore/SqueezeCenter/Slim/Music/Artwork.pm
@@ -298,7 +298,9 @@

        } elsif (defined $artwork) {

-        unshift @filestotry, $artwork;
+        # unshift @filestotry, $artwork;
+        # Jacob Laursen, 06.04.2011: Try built-in filenames first, then custom filename.
+        push @filestotry, $artwork;
        }

        if (defined $artworkDir && $artworkDir eq $parentDir) {
Or manually replace "unshift" with "push" in this line:
unshift @filestotry, $artwork;
For some reason (probably due to a bug in Squeezebox Server) covers won't be found in directories with non-Latin 1 characters (like æ).

Generic work-around using symbolic links

Another option for getting this scheme to work independently of the software being used would be to run a command like this from the root directory:

find . -mindepth 2 -not -path "*@eaDir*" -type d -not -exec test -e "{}/Folder.jpg" ';' -exec test -e "{}/../Folder.jpg" ';' -print0 | xargs -0 -I {} ln -s ../Folder.jpg {}/Folder.jpg
This will create symbolic links named "Folder.jpg" in all directories without one, referencing "Folder.jpg" in the parent directly. If the parent directory doesn't have one either, no link is created. This command line will skip all @eaDir's created by Synology DiskStation (containing thumbnails). If you regret this, remove all symbolic links using the command (warning: recursive - only run it from the music directory):
find . -name Folder.jpg -type l -delete

Storing cover art

The best way to store cover art is in separate images files in the individual album directories. The most adapted convention for storing the front cover is in a file named "Folder.jpg". The is supported by Winamp, Squeezebox, Microsoft Media Player, Windows Explorer, Directory Opus and many more.

Store the cover art in the best possible quality. Don't let your small cellphone display, viewing pane in Winamp or whatever lower your ambition. The picture can always be scaled down, either realtime or in a conversion -- but it can't be scaled up, when you actually need a nice high-quality picture, for example on your big TV screen (through DLNA). Don't go for anything lower than 600x600 pixels.

Sources

If you don't want to scan the cover art yourself, you could try the following sources:

Converting to other formats

My FLAC library consists of hundreds of CD's and has passed 100 GB in size a long time ago. The FLAC format is still not suitable for portable devices with limited storage or bandwidth (for streaming). To solve this problem, I have simply created a complete MP3 mirror of my music library. I use the following scripts:

flac2mp3_all

This script will update my MP3 mirror, i.e. convert all those files not already converted. It will add ID3v2 cover art in 300x300 resolution from the Folder.jpg files.
#!/bin/bash
# Written by Jacob Laursen , 2009.
#
# Requires:
# - ImageMagick, http://www.imagemagick.org/script/index.php
# - Rechosen's 'flac2mp3' script, see http://www.linuxtutorialblog.com/wp-content/uploads/2010/03/flac2mp3
# - LAME, see http://lame.sourceforge.net/

SRCDIR="/mnt/diskstation/music/flac/"
DSTDIR="/mnt/diskstation/music/mp3"
TMPFILE=`mktemp`
LAMEOPTS="-q 0 --add-id3v2"

cd $SRCDIR
find . -type d -print0 | while read -d $'\0' dir
do
	ls "$dir" | grep .flac$ >/dev/null 2>&1
	if [ $? == 0 ]; then
		if [ -e "$dir/Folder.jpg" ]; then
			cover="$dir/Folder.jpg";
		elif [ -e "$dir/../Folder.jpg" ]; then
			cover="$dir/../Folder.jpg";
		else
			cover="";
		fi
		if [ "$cover" != "" ]; then
			convert -define jpeg "$cover" -thumbnail '300x300>' $TMPFILE
			nice -n 19 /root/scripts/flac2mp3 -b 192 -o $DSTDIR -r -l "$LAMEOPTS --ti $TMPFILE" "$dir"
			rm $TMPFILE
		else
			nice -n 19 /root/scripts/flac2mp3 -b 192 -o $DSTDIR -r -l "$LAMEOPTS" "$dir"
		fi
	fi
done

flac2mp3

To get flac2mp3_all to work, I had to tweak the original flac2mp3 script:

--- /root/scripts/flac2mp3.orig 2008-08-25 21:09:58.000000000 +0200
+++ /root/scripts/flac2mp3      2009-09-20 17:02:36.000000000 +0200
@@ -24,6 +24,7 @@
     -o outputdirectory  Specifies the directory to write the mp3s to. If not
                         specified, the mp3s will be written to the directory
                        where the flac files are located.
+    -w                  Overwrite existing files.
     -r                  Makes the script look for flac files recursively. If
                         this option is specified together with an alternative
                        output directory, the mp3s will be written to the
@@ -64,6 +65,10 @@
 optionstopass=( )
 lameoptsarray=( )
 verbose=""
+overwrite="false"
+
+# Set local charset to Latin-1 and let metaflac convert from UTF-8
+LANG=en_DK.iso88591

 # Functions for input validation
 function IsNum() {
@@ -75,7 +80,7 @@

 # Get the passed options
 i=0
-while getopts ":b:d:l:o:rsv" options; do
+while getopts ":b:d:l:o:rsvw" options; do
        optionstopass[$i]=-"$options"
        (( i++ ))
        if [ "$OPTARG" ]; then
@@ -90,6 +95,7 @@
                r) recursive="";;
                s) selfexecute="true";;
                v) verbose="true";;
+               w) overwrite="true";;
                ?) echo $shorthelp; exit 2;;
        esac
 done
@@ -167,7 +173,7 @@
                        elif [ "$option" == "y" ]; then
                                command[$((i+1))]=${command[$((i+1))]:0:4}
                        elif [ "$option" == "g" ]; then
-                               if ! lame --genre-list | grep "${command[$((i+1))]}" >/dev/null; then
+                               if ! lame --genre-list 2>&1 | grep "${command[$((i+1))]}$" >/dev/null; then
                                        command[$((i+1))]=""
                                        [ "$verbose" ] && echo "Nullified the genre tag as the specified genre does not exist."
                                fi
@@ -204,7 +210,12 @@
        fi
        [ "$verbose" ] && echo "\$outputfile is ""$outputfile"". Creating directories if needed..."
        mkdir -p "$(dirname "$outputfile")"
-
+
+       [ "$overwrite" == "false" ] && if [ -e "$outputfile" ]; then
+               echo "Skipping existing file $outputfile..."
+               exit
+       fi
+
        # Prepare the bitrate
        [ "$verbose" ] && echo "Preparing bitrate..."
        if [ -n "$bitrate" -a "$bitrate" != "none" ]; then
@@ -228,9 +239,10 @@
                blocksize=1
        fi
        ddcount=$(( $blocksize / 512 ))
-       if [ $blocksize -le 0 ]; then
+       if [ $ddcount -le 0 ]; then
                ddcount=1
        fi
+       blocksize=$((ddcount * 512 ))
        donesofar=0
        percent=0
        start=$(date +%s)
@@ -241,7 +253,7 @@
        while [ $donesofar -lt $flacsize ]; do
                dd bs=512 count=$ddcount 2>/dev/null
                (( donesofar += $blocksize ))
-
+
                percent=$(( 100 * $donesofar / $flacsize ))
                echo -n -e "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b" 1>&2
                printf "%4s%%" $percent 1>&2

Maintenance

Maintaining a music library is an ongoing process that will never end. Different problems or needs can show up long time after adding music files to the library:

Helpful tools

This script will find either all files with a specific tag, or all files missing a specific tag:
#!/bin/bash

MUSIC_LIBRARY="/mnt/diskstation/music/flac"

find $MUSIC_LIBRARY -name *.flac -print0 | while read -d $'\0' file
do
        if metaflac --show-tag="$1" "$file" | grep "$1" >/dev/null; then
                if [ "$2" != "--inverse" ]; then
                        echo $file":";
                        metaflac --show-tag="$1" "$file" | cut -d '=' -f2
                fi
        else
                if [ "$2" == "--inverse" ]; then
                        echo $file;
                fi
        fi
done

[ Du er ikke logget ind | Tilbage til hjemmeside | Log ind ]