Saturday, October 31, 2009

abcde: CDROM has not been defined or cannot be found

Back to auto CD ripping again after a bit of a break, caused by the power supply of my USB-IDE cable set packing in. This is how I connect a CD drive to my fitPC, so it was a bit of a set back. The ebay vendor kindly sent me a replacement cable set, so I'm back in business.

But when I fed a new music CD to the drive, it didn't rip. On investigation, I found
abcde: CDROM has not been defined or cannot be found
Two useful threads I found were http://ubuntuforums.org/showthread.php?p=7849371 and http://ubuntuforums.org/showthread.php?p=7622858.

I found that for some reason, instead of /dev/cdrom I only had /dev/cdrom2. Editing /etc/abcde.conf, I defined CDROM=/dev/cdrom2. That fixed the problem for abcde.

The second thread showed how to get my CDROM drive at /dev/cdrom again. Following funkydan2's lead I moved /etc/udev/rules.d/70-persistant-cd.rules to one side and rebooted. The file was regenerated and /dev/cdrom reappeared.

That done, I reverted /etc/abcde.conf, commenting out the CDROM=/dev/cdrom2 line.

Saturday, September 12, 2009

Sync files to an mp3 player with WinMerge

To sync files between my PC and my mp3 player, I'm finding WinMerge to be a fast, lightweight solution.

This is a bit geeky, because WinMerge is generally a programmer's tool. Typically you open two different versions of a file or folder in WinMerge; it shows the differences in a left/right view, and lets you copy differences from one side to the other.

But that's exactly what you need to selectively sync a music collection from one place to another, like to an mp3 player.

What I do is to open my music collection folder (on my PC) and my mp3 player's music folder in WinMerge. It quickly scans for differences, then shows me
  • which folders and files are on left or right only (e.g. not on the mp3 player).
  • which folders and files are different (e.g. different folder contents or different file size)
  • which files and folders are identical (i.e. in sync)
Then it's easy to select files to copy to the mp3 player.

Because I'm a geek, I store my music collection on a mini Linux file server and access it using Samba over a wireless network. You do not have to do this! But just for interest, in this environment and using the setup below, WinMerge generates the diff view of my collection of around 5,000 tracks in about 3 seconds. Of course it will work just the same but faster with files stored on your PC.

The following instructions assume you have installed and launched WinMerge and have your mass storage class mp3 player connected to your PC. My player is a Sansa Clip.

1. Configure WinMerge to do fast scans of your music collection

Edit -> Options



Select the Compare category
Tick Ignore time differences less than 3 seconds
File Compare Method: Modified Date and Size

Changing the file compare method from Full Contents to Modified Date and Size is key to achieving fast scan times. You don't need a byte by byte comparison to sync your mp3s. Date and filesize comparison is fast and all you need.

2. Open your music collection and your mp3 player in WinMerge

In Select Files or Folders Dialog



Left: select music folder on your PC
Right: select music folder on your mp3 player
Filter: *.mp3
Tick Include Subfolders
Click OK

3. Customize the view

You will want an Explorer tree view: View -> Tree mode

It can be handy to show file size differences:

Tools -> Customize Columns
Add left size and right size

4. Save all this as a WinMerge project

File -> Save Project



On this page you also have option to make one side read only. This is handy if you want to protect your music collection and only copy files in one direction - to your mp3 player.

Next time you want to sync files, just open the project. All your configuration steps are saved.


5. Start syncing

WinMerge is now showing your music collection in an Explorer tree view. Different icons in the tree clearly show you which folders and files are
  • on left only (i.e. not on mp3 player):

  • on right only (i.e. only on mp3 player)
  • identical:

  • or different (e.g. different folder contents, different file size)
To copy from PC to mp3 player, select the files or folders to copy, then

Right click the folder or file/s
Copy -> Left to Right

... or just hit the "Copy right" icon in the toolbar:




To check result of the merge, click refresh.

Sin é.

Thursday, June 4, 2009

Auto rip audio CDs in Ubuntu Server

I got my headless Linux server set up to auto rip music CDs and add them to my music library. Pop a CD in the tray and it automatically rips and tags the tracks to FLAC files and ejects the CD... and queues up a task to transcode to mp3 and then rescan my Squeezecenter library later. Here's how:

* Detection of the audio CD and kicking stuff off is done by ivman.
* Ripping and tagging is done by abcde.
* Transcoding is done by flac2mp3.
* Refreshing the library is done using the Squeezecenter CLI.

It took me "a while" to get it working end to end. The tough part was getting ivman configured. But that's the essential part for automation.

Ivman

To install
$ sudo apt-get install ivman
which installs a bunch of dependencies (but no X). Then I enabled an action to start a script when ivman detects an audio CD. This is what I added to /etc/ivman/IvmConfigActions.xml:
<ivm:Match name="hal.volume.disc.type" value="cd_rom">
<ivm:Match name="hal.volume.disc.has_audio" value="true">
<ivm:Match name="hal.volume.disc.has_data" value="false">
<ivm:Option name="exec" value="/home/user_x/bin/rip_cd '$hal.block.device$'"/>
</ivm:Match>
</ivm:Match>
</ivm:Match>
The rip_cd script just invokes abcde and directs its output to a log file.

The problem that broke my heart for ages was that ivman didn't have permission to use the cdrom drive. I got this error in the log file from abcde:
cd-discid: /dev/cdrom: open: Permission denied
[ERROR] abcde: CD could not be read. Perhaps there's no CD in the drive?
No such problem running abcde directly from the command line as the same user ivman ran as, since that user is a member of the group "cdrom".

How I eventually got it working consistently was to configure ivman to run explicitly as the group "cdrom".

Ivman requires you to specify both a user and a group for the daemon to run as - in /etc/ivman/IvmConfigBase.xml). Just because the user you specify is a member of a group, e.g. cdrom, doesn't mean that ivman will run as a member of that group.

In my setup, ivman is now solely configured from /etc/ivman and runs as "user_x" and group "cdrom". And abcde is solely configured from /etc/abcde.conf.

It can help to run ivman in debug mode:
$ sudo /etc/init.d/ivman stop
$ sudo ivman -d --nofork

ABCDE

To install
$ sudo apt-get install abcde
This also installs cd-discid and probably cdparanoia (which I had installed already). Getting abcde configured is easy, especially thanks to a great guide on Andrew's Corner.

I included a post-read hook in /etc/abcde.conf to enqueue a transcoding script when the CD is ripped.
post_read ()
{
atq -q x | while read line; do job=`echo $line | sed -e 's/\([0123456789]*\).*/\1/'`; atrm $job; done
at -q x 01:00AM -f /home/user_x/bin/flac2mp3
}
It would probably be better just to run the transcoding script as a cron job rather than "on demand". But this way works too.

The flac2mp3 script invokes Robin Bowes' flac2mp3, directing its output to a log file, and then invokes the Squeezecenter CLI to rescan the music library.

Of course, abcde would happily take care of ripping to FLAC and mp3 at the same time. But I prefer to rip to FLAC and transcode from there. That way, if I need to change tags, I just update the FLACs and run flac2mp3 to update the tags in my mp3s.

Squeezecenter CLI

Squeezecenter has a telnet-based command line interface. One of the things it supports is executing a rescan of the library. So I included the scriptlet below in my transcoding script, after transcoding completes:
echo `date` "requesting squeezecenter rescan..." >> $log
(
sleep 1;
/bin/echo -e "rescan\r";
sleep 1;
/bin/echo -e "exit\r";
sleep 1;
) | telnet localhost 9090 >> $log 2>&1
echo `date` "telnet exit status: " $? >> $log
Directing the output to a file was useful here (again). It helped me diagnose a problem when the "wrong" version of echo got used when the script was run from the at task, causing the telnet to fail. The fix was to use /bin/echo explicitly.

Of course, I could just let Squeezecenter schedule a regular rescan of the music library instead of doing it on demand. But I wanted to give the CLI a try.

Anyway, that's about it.

Just a note on hardware. I'm still using the fit-PC slim as my server. It doesn't have a CD drive. I connect the internal DVD burner from my old PC through a cheapo IDE-USB adapter cable from ebay.

Saturday, February 28, 2009

Installing mediainfo on Ubuntu

There's no single ready made package for mediainfo as far as I can see, so no apt-get install, but the required .deb packages are available and so can be installed using dpkg.

Pick the appropriate packages for your system from the mediainfo sourceforge download page, then install in the correct order using dpkg -i.

To install the CLI version on my fit-pc Ubuntu 8.10 server, I did
$ wget http://downloads.sourceforge.net/mediainfo/mediainfo_0.7.11-1_i386.Debian_5.0.deb
$ wget http://downloads.sourceforge.net/mediainfo/libmediainfo0_0.7.11-1_i386.Debian_5.0.deb
$ wget http://downloads.sourceforge.net/zenlib/libzen0_0.3.8-1_i386.Debian_5.0.deb
$ sudo dpkg -i libzen0_0.3.8-1_i386.Debian_5.0.deb
$ sudo dpkg -i libmediainfo0_0.7.11-1_i386.Debian_5.0.deb
$ sudo dpkg -i mediainfo_0.7.11-1_i386.Debian_5.0.deb
Then it's good to go.

Monday, February 2, 2009

Automount USB disks in Ubuntu Server

I am using the usbmount package to automount a 1TB USB disk connected to my mini linux media server. This allows me to mount and unmount the external disk simply by turning it on and off, which is a convenient and power-saving arrangement.

Since I'm now storing DVD rips for my Popcorn Hour media player, I needed to add more storage to my streaming media server. My fit-PC takes 2.5" IDE/PATA drives, so it is already fully loaded with a 250G drive. So I got a 1TB 3.5" SATA drive (Western Digital GreenPower) and put it in a USB enclosure.

It's a very quiet, low power drive (about 9W generally), but I don't want it on 24/7. Spinning down the disk is one option, and I might investigate that further. But right now I want
  1. To simply switch the disk on when I need it and off when I'm done.
  2. For that to be safe (without potential for losing data).
  3. To include the disk in the network share I export for the Popcorn Hour.
Desktops systems automount USB disks when you plug them in or power them on. This functionality is not available by default in Ubuntu server. (The packages used depend on the desktop environment.) Following a suggestion in the Ubuntu documentation, I installed the usbmount package
This package automatically mounts USB mass storage devices (typically USB pens) when they are plugged in, and unmounts them when they are removed. The mountpoints (/media/usb[0-7] by default), filesystem types to consider, and mount options are configurable. When multiple devices are plugged in, the first available mountpoint is automatically selected. If the device provides a model name, a symbolic link /var/run/usbmount/MODELNAME pointing to the mountpoint is automatically created. [...]
... which sounds ideal. And as far as I can see, it does the job.

I couldn't find much online about how to configure usbmount, but I soon found /etc/usbmount/usbmount.conf. You can configure mount options per filesystem type. The default options are
MOUNTOPTIONS="sync,noexec,nodev,noatime"

Out of the box, it only automounts ext2 and ext3 formatted volumes. I had formatted my new disk with XFS, so I added xfs to the configuration

FILESYSTEMS="ext2 ext3 xfs"

and it seems to work fine. When I switch on the disk, I see these messages in syslog:

kernel: [1034394.920087] usb 1-2: new high speed USB device using ehci_hcd and address 27
kernel: [1034395.064826] usb 1-2: configuration #1 chosen from 1 choice
kernel: [1034395.078845] scsi25 : SCSI emulation for USB Mass Storage devices
kernel: [1034395.085992] usb-storage: device found at 27
kernel: [1034395.086033] usb-storage: waiting for device to settle before scanning
kernel: [1034400.084595] usb-storage: device scan complete
kernel: [1034400.087474] scsi 25:0:0:0: Direct-Access     WDC WD10 EADS-00L5B1      0041 PQ: 0 ANSI: 0
kernel: [1034400.091636] sd 25:0:0:0: [sdb] 1953525168 512-byte hardware sectors (1000205 MB)
kernel: [1034400.094271] sd 25:0:0:0: [sdb] Write Protect is off
kernel: [1034400.094309] sd 25:0:0:0: [sdb] Mode Sense: 03 00 00 00
kernel: [1034400.094328] sd 25:0:0:0: [sdb] Assuming drive cache: write through
kernel: [1034400.104985] sd 25:0:0:0: [sdb] 1953525168 512-byte hardware sectors (1000205 MB)
kernel: [1034400.113058] sd 25:0:0:0: [sdb] Write Protect is off
kernel: [1034400.113094] sd 25:0:0:0: [sdb] Mode Sense: 03 00 00 00
kernel: [1034400.113114] sd 25:0:0:0: [sdb] Assuming drive cache: write through
kernel: [1034400.120193]  sdb: sdb1
kernel: [1034400.137434] sd 25:0:0:0: [sdb] Attached SCSI disk
kernel: [1034400.137980] sd 25:0:0:0: Attached scsi generic sg1 type 0
usbmount[16483]: executing command: mount -txfs -osync,noexec,nodev,noatime /dev/sdb1 /media/usb0
kernel: [1034401.228110] XFS mounting filesystem sdb1
kernel: [1034401.710070] Ending clean XFS mount for filesystem: sdb1
usbmount[16483]: executing command: run-parts /etc/usbmount/mount.d

The disk is mounted:

$ mount
<snip...>
/dev/sdb1 on /media/usb0 type xfs (rw,noexec,nodev,sync,noatime)

And when I switch it off, I see

kernel: [1034259.923629] usb 1-2: USB disconnect, address 26
kernel: [1034259.936251] xfs_force_shutdown(sdb1,0x1) called from line 420 of file /build/buildd/linux-2.6.27/fs/xfs/xfs_rw.c.  Return address = 0xe0a9ddb4
kernel: [1034259.936334] Filesystem "sdb1": I/O Error Detected.  Shutting down filesystem: sdb1
kernel: [1034259.936404] Please umount the filesystem, and rectify the problem(s)
kernel: [1034259.949818] Filesystem "sdb1": xfs_log_force: error 5 returned.
kernel: [1034259.949851] Filesystem "sdb1": xfs_log_force: error 5 returned.
kernel: [1034259.949890] xfs_force_shutdown(sdb1,0x1) called from line 420 of file /build/buildd/linux-2.6.27/fs/xfs/xfs_rw.c.  Return address = 0xe0a9ddb4
kernel: [1034259.968157] Filesystem "sdb1": xfs_log_force: error 5 returned.
usbmount[16307]: executing command: umount -l /media/usb0
kernel: [1034260.050924] Filesystem "sdb1": xfs_log_force: error 5 returned.
kernel: [1034260.057086] Filesystem "sdb1": xfs_log_force: error 5 returned.
kernel: [1034260.061208] xfs_force_shutdown(sdb1,0x1) called from line 420 of file /build/buildd/linux-2.6.27/fs/xfs/xfs_rw.c.  Return address = 0xe0a9ddb4
kernel: [1034260.069650] Filesystem "sdb1": xfs_log_force: error 5 returned.
kernel: [1034260.073675] Filesystem "sdb1": xfs_log_force: error 5 returned.
kernel: [1034260.077672] xfs_force_shutdown(sdb1,0x1) called from line 420 of file /build/buildd/linux-2.6.27/fs/xfs/xfs_rw.c.  Return address = 0xe0a9ddb4
usbmount[16307]: executing command: run-parts /etc/usbmount/umount.d
kernel: [1034260.084706] Filesystem "sdb1": xfs_log_force: error 5 returned.
kernel: [1034260.084735] Filesystem "sdb1": xfs_log_force: error 5 returned.
kernel: [1034260.084820] Filesystem "sdb1": xfs_log_force: error 5 returned.
kernel: [1034260.084864] Filesystem "sdb1": xfs_log_force: error 5 returned.
kernel: [1034260.084891] Filesystem "sdb1": xfs_log_force: error 5 returned.

which doesn't look so great. But after repeatedly mounting and unmounting the disk without any apparent problems, I was getting the feeling that this was not harmful.

I did a bit of searching to see whether XFS supports the sync mount option. The answer wasn't clear, so I raised a question on the XFS mailing list to see what they thought. The initial response was discouraging, but on followup it seems like since...
  • I am sync mounting the filesystem (what usbmount does by default), and

  • my kernel version is later than 2.6.17 (from which point write barrier support is enabled by default in XFS)
... I can keep on doing what I'm doing. I will not get corruption of the file system. (Do read the thread, the XFS FAQ and otherwise satisfy yourself. It's your data.)

The last item on my wish list was to include the disk in the network share I export for the Popcorn Hour. This is easy.

Usbmount mounts the disk to /media/usb[0-7] arbitrarily, but it also creates a symbolic link in /var/run/usbmount using the disk model name, which provides a consistent reference point. I created a link in my media storage directory to that point, e.g.
ln -s /home/shared/movies /var/run/usbmount/WDC_WD10_EADS-00L5B1_1/movies

Using my existing Samba share (/home/shared), this works perfectly. When the disk is on, I can browse into the linked directory from the Popcorn Hour. When the disk is off, the link is invisible.

Friday, January 23, 2009

Installing SwissCenter on Ubuntu Server

I got SwissCenter up and running on my fit-PC Slim, which is now running Ubuntu 8.10 Server.

At first I was working off the official SwissCenter Linux install documentation which is for SuSE 9.3. Later on I found this cheatsheet for installing SwissCenter on Ubuntu which is a bit out of date now, but still very useful.

Install apache2 and mysql

I had these installed already, done in the standard way as in the Ubuntu Server Guide, which is to say
$ sudo apt-get install apache2
$ sudo apt-get install mysql-server
Actually, I had the default mysql instance disabled, so re-enable it now and maybe try converging to a single mysql instance later.
$ cd /etc/rc2.d
$ sudo mv K81mysql S19mysql
$ sudo /etc/init.d/mysql start
Install PHP
$ sudo apt-get install php5 libapache2-mod-php5 php5-mysql php5-gd
Install SwissCenter
$ wget http://www.swisscenter.co.uk/downloads/swisscenter.zip
$ sudo mkdir -p /srv/www/swisscenter
$ cd /srv/www/swisscenter/
$ sudo unzip ~/swisscenter.zip
$ sudo chown -R www-data:www-data .
(Using www-data which is the standard apache user/group in Debian-based distributions, rather than wwwrun as specified in the SwissCenter document.)

Enable the site in Apache

Slightly different from the document here... I appended the following to /etc/apache2/sites-available/default
Listen 8080
NameVirtualHost --myhostname--
<virtualhost>
DocumentRoot "/srv/www/swisscenter"
<directory>
Options Indexes FollowSymLinks MultiViews
AllowOverride None
Order allow,deny
Allow from all
</directory>
<ifmodule>
DirectoryIndex index.html index.php
</ifmodule>
</VirtualHost>
Then restarted apache.

Checking and configuring SwissCenter

Point web browser to the SwissCenter Configuration page at http://<servername>:8080/config. On first attempt, the page showed:
Warning: dl() [function.dl]: Dynamically loaded extensions aren't enabled in /srv/www/swisscenter/base/image.php on line 50

Fatal error: Call to undefined function imagecreatetruecolor() in /srv/www/swisscenter/base/image.php on line 251
After a bit of searching, I found I needed to include php5-gd (included in the install command above. I didn't have it originally), so
$ sudo apt-get install php5-gd
$ sudo /etc/init.d/apache2 restart
Better this time. But the page still shows some errors under Installation Checks.
Unable to locate a TrueType font. The "Now Playing" screen will be missing track information.
Google turned up this, which helped me to find Advanced Options / Miscellaneous in the SwissCenter configuration pages where you can set the path to a TT font. I entered /usr/share/fonts/truetype/freefont/FreeSans.ttf.

That left a warning that database is not created yet. You create it from the SwissCenter Create Database page. On first attempt, I got
Unable to create database - Is your MySQL "Root" password correct?
Umm... Looks like I changed the default mysql root password when I installed mysql. I've forgotten it now. So I had to reset the root password. Now I can create the database successfully.

Back on the main Configuration page,
Unable to write to the SwissCenter playlists directory (playlists)
Under Media Management / Playlists, you enter a full path to a dir (which must be writeable by www-data user), i.e. created something like
$ sudo -u www-data mkdir <some_dir>
After all that, SwissCenter is up and ready to load some content. I pointed it to my photos and music directories for a start.

Accessing SwissCenter from the Popcorn Hour

The simplest way for a start is to add the SwissCenter URL to the Web Services page in the Popcorn Hour, i.e. create an entry pointing to http://servername:8080.

But you can run an additional service on the server to broadcast the SwissCenter's UPnP interface. With that running, SwissCenter shows up automatically in the Popcorn Hour's Media Sources page.

There's a nice custom broadcast script and a Debian init script to start and stop this broadcast service automatically. I modified the init.d script slightly to make it run in the background:
$ diff -upN broadcast.orig broadcast
--- broadcast.orig 2008-09-11 17:02:03.000000000 +0100
+++ broadcast 2009-01-11 02:00:52.011883945 +0000
@@ -46,7 +46,7 @@ do_start()
# 2 if daemon could not be started
start-stop-daemon --start --quiet --pidfile $PIDFILE --exec $DAEMON --test > /dev/null \
|| return 1
- start-stop-daemon --start --quiet --pidfile $PIDFILE --exec $DAEMON -- \
+ start-stop-daemon --start --quiet --pidfile $PIDFILE --exec $DAEMON --background --make-pidfile -- \
$DAEMON_ARGS \
|| return 2
# Add code here, if necessary, that waits for the process to be ready
@@ -95,7 +95,7 @@ do_reload() {

case "$1" in
start)
- [ "$VERBOSE" != no ] && log_daemon_msg "Starting $DESC" "$NAME"
+ log_daemon_msg "Starting $DESC" "$NAME"
do_start
case "$?" in
0|1) [ "$VERBOSE" != no ] && log_end_msg 0 ;;
@@ -103,7 +103,7 @@ case "$1" in
esac
;;
stop)
- [ "$VERBOSE" != no ] && log_daemon_msg "Stopping $DESC" "$NAME"
+ log_daemon_msg "Stopping $DESC" "$NAME"
do_stop
case "$?" in
0|1) [ "$VERBOSE" != no ] && log_end_msg 0 ;;

And that's about it.

Thursday, January 8, 2009

I got a Popcorn Hour

I got a Popcorn Hour A-110 (AKA Networked Media Tank) media player just before Christmas.

I've had my Squeezebox network music player for a couple of years now, and along the way I've got hooked on
  • the idea of storing my digital media centrally

  • running a low power server

  • hacking around with Linux
So now I'm going to have a go at streaming video. The Popcorn Hour seems like a good device to do it with.

My video content is DVDs ripped in full to ISOs using DVDFab (free version). I'm not compressing or discarding any of the non-movie content yet. But as each title takes 6-8GB, I might look into one or both soonish...

So far, I've had perfect success viewing this content on my TV in these ways:
  1. From an external disk (NTFS formatted), connected to a USB port on the Popcorn Hour.

  2. From an NFS share hosted on my fit-PC mini server, with the content stored on it's internal disk.

  3. As previous, but with the content on the external disk, connected to a USB port on the fit-PC.

  4. As previous, but using Samba instead of NFS.
For options 2-4 this is streaming via a pair of Devolo Homeplug AV adapters (AKA powerline networking). One adapter is connected to my router in the "server room". The other is connected to the Popcorn Hour in the TV room.

I hadn't expected the Homeplug setup to work. Quite a few people have reported problems with powerline networking on the Popcorn Hour forums. (And I had previously been unsuccessful streaming this way to a Hauppauge MVP box - it stuttered badly whatever I tried.) So was resigned to somehow running a cable from one end of the house to the other. But happy day, it is working flawlessly for me - at least for this standard-def content.

And my fit-PC handles streaming the video over NFS and Samba without breaking sweat.

Loads of scope for geeky time suckage with this. So more posts may follow.

(It took me absolutely feckin' ages to successfully export an NFS shared from a directory mounted from the NTFS formatted USB disk, but that's a whole other story.)