Ian's Linux Adventure

General Lew Wallace of the U.S. Civil War. I don't look like him. Howdy.

I'm documenting my adventures (and problems) here, so that I remember my mistakes, and that you may learn from my mistakes. I'm not a programmer, nor a computer expert. I'm just a tinkering guy in Milwaukee with a store and three kids to keep me busy.

If you want to respond, or help me out, or you find a way to explain something better, just use the web form. And thanks!

TIP: Use your browser's Find (usually CTRL+F) to look for the topic of your choice. I'm far too lazy to organize this unimportant corner of my life....

TIP: I dont have a bunch of fragmented updates - if an old entry changes, I update it, put a current date on it, and stick it at the head of the queue. So if you're looking for an old entry that vanished, look instead for an updated entry closer to the top.



Video, photos, and music sharing with my phone

July 13, 2009: I have a snazzy new geek-phone, a Shuttle from Virgin Mobile. It has a 4GB MicroSD slot so files can be exchanged with my Xubuntu system.

Using conky for desktop text information

July 12, 2009: Experimenting with conky to put a lot of geek eye candy on the desktop background. I found some good instructions.

  1. Created a new script ~.config/conky/startup to launch the conky processes.
  2. New alias in my .bashrc alias conky='killall conky; .config/conky/startup &' makes manually stopping and starting conky (for testing) a breeze.
  3. New autostart entry in the Login and Sessions control panel to launch conky at login.
  4. Each new conky config file goes in ~.config/conky/ and gets a separate line to start it in the startup script.
  5. The system information conky is from a forum post
  6. Developed a script to read National Weather Service data and reformat it for conky display. I certainly learned a lot about sed! 
    #!/bin/sh
LOCAL_PATH=/home/me/.config/weather/
STATION=KMKE
RADAR_NAME=MKX
ZONE=wiz066
RADAR_IMAGE_TOP_PADDING=15
DISPLAY_WIDTH=48

# BEGIN RADAR IMAGE
LOCAL_NAME="${LOCAL_PATH}radar_image.png"
RADAR_URL="http://radar.weather.gov/lite/N0R/${RADAR_NAME}_0.png"
curl -o $LOCAL_NAME $RADAR_URL
convert $LOCAL_NAME -background none -splice 0x${RADAR_IMAGE_TOP_PADDING} $LOCAL_NAME
DISPLAY=:0.0 xfconf-query -v -c xfce4-desktop -p /backdrop/screen0/monitor0/image-path -s /usr/share/xfce4/backdrops/xfce4logo.png    # workaround - simply refreshing the desktop doesn't update the picture.
DISPLAY=:0.0 xfconf-query -v -c xfce4-desktop -p /backdrop/screen0/monitor0/image-path -s $LOCAL_NAME
# END RADAR IMAGE

# BEGIN CURRENT OBSERVATIONS
DOWNLOAD_URL=http://weather.noaa.gov/pub/data/observations/metar/decoded/${STATION}.TXT
LOCAL_NAME="${LOCAL_PATH}current_conditions.txt"
curl -o $LOCAL_NAME $DOWNLOAD_URL
# Format: Observation Time, Sky Conditions, Temperature, Humidity, and Wind
# These are the 3rd Field of Line 10, The end of Line 5, The first half of the 13th field of line 10, The end of line 8, and the 4th field of line 10.
OBSERVATION_TIME_ZULU=`awk '$1=="ob:" {print $3}' $LOCAL_NAME | cut -c3-7`
OBSERVATION_TIME=`date -d $OBSERVATION_TIME_ZULU +%H:%M`
SKY_CONDITIONS=`grep 'Sky conditions:' $LOCAL_NAME | cut -d: -f2 | sed 's/^[ \t]*//' | sed 's/\<./\u&/'g`
TEMPERATURE=`grep 'Temperature:' $LOCAL_NAME | cut -d: -f2 | cut -d' ' -f2 `
HUMIDITY=`grep 'Humidity:' $LOCAL_NAME | cut -d' ' -f3`
WIND=`grep 'Wind:' $LOCAL_NAME | cut -d' ' -f4,8`
echo "Observations at ${STATION} as of ${OBSERVATION_TIME}" > ${LOCAL_PATH}conky_observation_1
SPACER='   '
echo "${SKY_CONDITIONS} ${SPACER} ${TEMPERATURE}F ${SPACER} ${HUMIDITY} ${SPACER} ${WIND}MPH" > ${LOCAL_PATH}conky_observation_2
# END CURRENT OBSERVATIONS

# BEGIN FORECAST
DOWNLOAD_URL=http://weather.noaa.gov/pub/data/forecasts/zone/wi/${ZONE}.txt
LOCAL_NAME="${LOCAL_PATH}/detailed_forecast.txt"
curl -o $LOCAL_NAME $DOWNLOAD_URL
# Reformat the forecast
# 1) Delete the first 11 lines of header
# 2) Replace the linefeeds with spaces
sed -e '1,11d' $LOCAL_NAME | tr '\012' ' ' > ${LOCAL_PATH}conky_forecast_1
# Reformat the forecast. These are the sed commands, in order:
# 1) Change the whole message to lowercase
# 2) Replace the '...' with ': '
# 3) Reduce double-spaces '  ' to single-spaces ' ' 
# 4) Fix capitalization in the first sentence on new lines (' .thursday' to 
#    '~Thursday'). The '~' is a placeholder for a newline
# 5) Fix capitalization in other sentences on each line ('. low' to '. Low')
# 6) Fix capitalizaation after colons (': mostly' to ': Mostly')
# 7) Strip all newlines that aren't the beginning of a new forecast period (strip
#   '30~mph', keep '~Thursday'
# 8) Delete any '~' before the first text.
sed -e 's/\(.*\)/\L\1/' -e 's/\.\.\./: /g' -e 's/\ \ /\ /g' -e 's/\.\([a-z]\)/~~\U\1/g' -e 's/\(\.\ [a-z]\)/\U\1/g'  -e 's/\(\:\ [a-z]\)/\U\1/g' -e 's/\([a-z1-9]\)~\([a-z1-9]\)/\1\ \2/g' -e 's/^~~//'< ${LOCAL_PATH}conky_forecast_1 > ${LOCAL_PATH}conky_forecast_2
# Reformat the text to the desired width, and put the newlines back in.
cat ${LOCAL_PATH}conky_forecast_2 | fmt -w $DISPLAY_WIDTH | tr '~' '\012' > ${LOCAL_PATH}conky_forecast
# END FORECAST
  7. Unused, but cool, weather URLs: 
    http://weather.noaa.gov/pub/data/forecasts/city/wi/milwaukee.txt
http://weather.noaa.gov/pub/data/forecasts/state/wi/wiz066.txt

Installing Xubuntu 9.04 on an emachines E625-5192

July 7, 2009: Received my new laptop today - I need it for the fall, and got it a little early due to a sale.

What went well: I created a set of Restore DVDs (in case I want Windows back), then removed windows and installed Xubuntu 9.04 full-disk. Copied over my old /home directory, and installed all my favorite apps. E-mail, web, games, .bashrc, most dektop settings, etc. transferred without a hiccup. Recreated my crontab. Wireless networking and video work great. Built-in card reader reads all cards from my cameras and phone. Machine is noticeably faster. FN-key brightness control works. FN-key multimedia controls work.

Solved Problems:

Problems: No built-in webcam. Keyboard is different, and will take time to get used to - many typos in the meantime. TomTom GPSautomatic update doesn't work on Linux. Department of Defense forms and other applications are Windows-specific.

Lossless formats and converting audio using SoundConverter

July 8, 2009: FLAC is a lossless audio storage format. So I'm going to try converting my audio collection to FLAC. Converting lossy (.mp3, .wma) files to a lossless format is a bit of a waste - better to rip them directly to FLAC next time. I use .mp3 files on my phone.

The 'soundconverter' package is an easy and convenient way to convert. From the package description "It reads sound files in any format supported by GStreamer and outputs them in Ogg Vorbis, FLAC, or WAV format, or MP3 format if you have the GStreamer LAME plugin." As far as I can tell, the LAME plugin is included in the ubuntu-restricted-extras package.

The GUI works, but I haven't figured out the command line yet.

Creating Cube Art using The GIMP

July 6, 2009: My spouse wants to create art using carefully-arranged Rubik's Cubes. She needs a tool to do it: She needs to manipluate images to the right color and pixellation to look cube-ready (Solving them to the displayed configuration is another issue).

So let's set up The GIMP to do it.

Preparation: GIMP needs to know what colors to use, so let's create a custom color pallette. (This only needs to be done the first time)

  1. Find an image or two with cube colors (Google is handy for this). Copy the images to your clipboard.
  2. Import the image to GIMP: File -> Create -> From Clipboard
  3. Create the new color palette: Image -> Mode -> Indexed -> Use Custom Palette -> Open The Palette Selection Dialog -> New Palette
  4. Select the six colors: Use the color-picker tool to change the foreground color, then right-click in the palette-editor to add each color.
  5. Save the new palette with a name you will remember, like 'Cube Colors'

Changing an image

  1. Import the image into GIMP
  2. Reduce the image to the six cube colors - Image -> Mode -> Indexed -> Use Custom Palette -> Cube Colors
  3. Reduce the image to manageable size with Image -> Scale Image Pick a small size that is a multiple of 3 (3 rows/colums on each cube)
  4. NEED a way to split the image into 3x3 (blown up to 9x9) squares for each cube.

Scripting for batch-resizing: This looks possible using Imagemagick - see "using pre-defined color maps" for an easy way to get Imagemagick to reduce the colors.

Converting photos for a picture frame using Imagemagick

July 4, 2009: Our store has a 7-inch photo frame, with a 2GB stick in it. So we can just drag-and-drop lots of photos onto the stick.

But how can we use Imagemagick, the command-line photo editor, to batch-resize a whole bunch of images...perhaps as part of a script?

mogrify -sample 480 x 234 *.JPG

Installing Empathy on Xubuntu 9.04

July 3, 2009: Trying Empathy instead of Pidgin.

Installing a Samsung SCX-5725FN printer under Xubuntu 9.04

July 2, 2009: Instead of the installation disks, I used these instructions

Then go into Settings -> Printer -> Add New Printer and let it autodetect the new printer on the network.

Xubuntu user-level startup scripts

July 1, 2009: Here is how to get a custom script to run at XFCE startup:

  1. Create a generic startup script and save it as custom_startup.sh. You can put anything in the script; in there now is only a logger so you know it's working. 
    #!/bin/sh
# This script is run automatically by xfce4-desktop during system startup.
logger -i "Running the custom startup script"
  2. Create the following entry as /home/me/.config/autostart/MyStartup. 
    [Desktop Entry]
Encoding=UTF-8
Version=0.9.4
Type=Application
Name=MyStartup
Comment=
Exec=bash /home/me/Scripts/startup_script.sh
StartupNotify=false
Terminal=true
Hidden=false

Xubuntu desktop picture from a website

June 28, 2009: Living in the midwest, I check the weather radar a lot to protect my laundry drying outside from lots of pesky thunderstorms. So I want to make the radar image my desktop picture in Xubuntu, and to have it automatically update.

To get the image: I'm using http://radar.weather.gov/lite/N0R/MKX_0.png. It is from the United States National Weather Service and updates every 5-6 minutes.

A shell script to refresh the radar image as the desktop picture: 

#!/bin/sh
# This script downloads a radar image, pads the image top so it's not blocked by the menu bar, and refreshes the Xfce4 desktop to make the updated image into the background. 

# This is the path-and-filename of the saved radar image.
PICTURE=/home/me/.config/xfce4/desktop/radar_image.png

# Download the radar image.
curl -o $PICTURE http://radar.weather.gov/lite/N0R/MKX_0.png

# Pad the image to so it doesn't get blocked by the menu bar.
# This is important because the top of the image has the timestamp on it. We want to see that the image is current.
convert $PICTURE -background none -splice 0x20 $PICTURE

# Set the desktop background to another image for just a moment. This is a workaround - simply refreshing the desktop doesn't update the picture.
# The 'DISPLAY=:0.0' prefix is required so cron can process it.
xfconf-query -v -c xfce4-desktop -p /backdrop/screen0/monitor0/image-path -s /usr/share/xfce4/backdrops/xfce4logo.png
# Return the desktop background to the newest radar picture.
xfconf-query -v -c xfce4-desktop -p /backdrop/screen0/monitor0/image-path -s $PICTURE
I'll save this script as radar_background.sh, and make it executable with sudo chmod +x radar_background.sh.
Notes:
  1. The DISPLAY=:0.0 element is explained here.
  2. The xfconf-query command, and how to change the background using DBus, are discussed in the XFCE forums

Updating the desktop picture manually:
Since we have it in a shell script already, we need only create a .bashrc alias to run the script. nano .bashrc opens the .bashrc for editing. Add the line alias radar='/home/me/radar_background.sh' to the bottom of the file and save it. Open a new terminal window (terminals only read the .bashrc upon starting) and try the command radar.

Updating the desktop picture automatically: Since we can update the desktop image using bash commands, let's make a crontab entry to update the desktop image automatically. Here's what it looks like - a crontab entry with just the shell command: 

# m h  dom mon dow   command
*/20 * * * * /home/me/radar_background.sh
Note that the desktop picture will refresh every 20 minutes.

Explaining the cron instructions:
# m h dom mon dow command - That's just part of the crontab
*/20 * * * * - tells the machine to run the script every 20 minutes. */5 * * * * will run the script every five minutes.
> /dev/null at the end (optional) - tells the machine to not e-mail me every time the script runs.
Make sure the command is all on one line (not wrapped), or you'll get crontab errors.

Watching a DVD in Xubuntu 9.04

June 27, 2009: After the previous reinstall two months ago, DVDs stopped working.

Here's how I got it to work:

  1. Add the medibuntu repository, if you haven't already 
    sudo cp /etc/apt/sources.list /etc/apt/sources.list~   # Backup the sources.list file
sudo mousepad /etc/apt/sources.list &                  # Open the sources.list file in an editing window

  ## In sources.list, append the following two lines at the bottom, then save (don't close it)
  ## Medibuntu
  deb http://packages.medibuntu.org/ jaunty free non-free
  ## If you use debtorrent, use: deb debtorrent://localhost:9988/packages.medibuntu.org/ jaunty free non-free

sudo apt-get update
sudo sudo apt-get install medibuntu-keyring
sudo apt-get update
  2. Use these two terminal commands to add the correct software (source): 
    sudo apt-get install totem-xine libxine1-ffmpeg libdvdread4
sudo /usr/share/doc/libdvdread4/install-css.sh
# Note - this installs the libdvdcss2 package, which is *not* in the repositories.
# If you use this list of packages to rebuild your system, for example by using a 
# Jablicator metapackage, it will fail due to this missing dependency.
  3. I installed ubuntu-restricted-extras for unrelated reasons. So I don't *think* it's necessary. 
    sudo apt-get install ubuntu-restricted-extras
  4. Finally, I got an error message when I put in a dvd: "Could not open location; you might not have permission to open the file." This is indeed a permission issue. Fix it with: 
    sudo chmod 755 /media/cdrom0
    And then try opening the DVD from within your player application (Totem).

Reinstalling Xubuntu 9.04

June 27, 2009: About a month ago, audio suddenly stopped working. Rather than troubleshoot, I decided to reinstall...it might be faster. Unlike last time, this time was a complete reinstall to get rid of certain dependency problems that had also cropped up.

Creating a metapackage with jablicator

May 31, 2009: A convenient way to create a metapackage is the jablicator command-line application, included in the package of the same name.

Jablicator creates a metapackage with all current packages as the dependencies. So you can edit the dependencies to limit metapackage, or take a snapshot of your packages for easy restoring after an update.

Launchpad bug #355209 - The Evolution plugin for Mail Notification doesn't work/exist

April 27, 2009: In mail-notification 5.4 in Ubuntu Jaunty (9.04), the following files are installed to /usr/lib/evolution/2.24/plugins/ instead of /usr/lib/evolution/2.26/plugins/:
liborg-jylefort-mail-notification.so
org-jylefort-mail-notification.eplug

    A simple workaround:
  1. Copy or link the files to the correct directory: 
    sudo ln -s /usr/lib/evolution/2.24/plugins/org-jylefort-mail-notification.eplug /usr/lib/evolution/2.26/plugins/
sudo ln -s /usr/lib/evolution/2.24/plugins/liborg-jylefort-mail-notification.so /usr/lib/evolution/2.26/plugins/
  2. Restart evolution and go to Edit -> Plugins -> Jean-Yves Lefort's Mail Notification. Check the box.
  3. Right-click on the Mail-Notification icon -> Properties. Add your evolution e-mailbox.

Upgrading from Xubuntu 8.04 to 9.04

April 25, 2009: It's finally time to reinstall/dist-upgrade, which I haven't done in a year.

Burning the 9.04 CD: Instead of installing Brasero or another burner, I used the command wodim dev=$PATH-TO-DEVICE driveropts=burnfree -v -data $PATH-TO-ISO, so in my case wodim dev=/dev/scd0 driveropts=burnfree -v -data /home/me/Ubuntu\ Images/xubuntu-9.04-desktop-i386.iso. Very easy and fast that way.

Using the 9.04 LiveCD installer: Very simple. One hiccup when automatic partitioning failed. I chose to reuse my existing partition *without* formatting it first, and (COOL!) my /home directory was untouched. All my preferences and saved fies were still there...as if they had been migrated. Networking and sound worked immediately from the default installation.

Enabling Medibuntu and debtorrent: Medibuntu is for non-free packages like skype. Debtorrent is a method of using torrents instead of mirrors to download. Both require changes to the /etc/apt/sources.list file. debtorrent instructions 

sudo cp /etc/apt/sources.list /etc/apt/sources.list~   # Backup the sources.list file
sudo mousepad /etc/apt/sources.list &                  # Open the sources.list file in an editing window

  ## In sources.list, append the following two lines at the bottom, then save (don't close it)
  ## Medibuntu
  deb debtorrent://localhost:9988/packages.medibuntu.org/ jaunty free non-free

sudo apt-get update
sudo sudo apt-get install medibuntu-keyring debtorrent apt-transport debtorrent

  ## In sources.list, substitute each occurrence of the string 'deb http://' prefix with 'deb debtorrent://localhost:9988/'
  ## Save and close the sources.list file.

sudo apt-get update

Bringing back my favorite apps: Using this table, it's pretty easy to figure out what to install and remove. Downloading all this stuff took about 40 minutes.

The droid fonts are nice, but not special.
SubjectPackages I RemovedPackages I AddedInstall Notes
E-Mailthunderbirdevolution
spamassassin
mail-notification-evolution		Evolution needed a couple restarts to start working properly
Scriptingbluefish
d-feet
curl
devscripts
Fontsmsttcorefonts
ttf-droid
Codecsgstreamer0.10-ffmpeg
gstreamer0.10-plugins-ugly
gstreamer0.10-plugins-bad
libdvdcss2		Download-at-first-need for audio also works well.
Printingcups-pdf
GPSgpsd
python-gps
Remote desktopvinagreopenssh-server
xtightvncviewer		xtightvncviewer required manual config: sudo update-alternatives --set vncviewer /usr/bin/xtightvncviewer
Networktransmission-gtk
xchat		deluge
skype
Officeopenoffice.org
openoffice.org-gtk
Otherbluez
bluez-gnome
bluez-cups
gnome-pilot		wine
simutrans
powertop
inkscape
flashplugin-installer
pmount		blues-gnome gets rid of bluetooth-applet
pmount mounts usb drives as user instead of root
Non-Ubuntuskype4pidginhttp://eion.robbmob.com/skype4pidgin.deb

Several launcher icons were missing - the launchers were still in the right place and fully functional, but the application (like bluefish) was gone. After reinstallation, most images came back automatically. A couple needed to be reassociated with the image by right-clicking on 'properties'.

Two important shortcuts were missing.

The crontab was gone and had to be recreated.

The mail-notification icon couldn't find evolution (Bug 355209). The bug report has the simple workaround.

Using rmadison and apt-cache

April 15, 2009: rmadison is part of the devscripts package. It's a fantastic little tool that tells you which version of a package is in which release of Ubuntu or Debian. It's also the quickest way to see if a package is in one of them at all - very handy to check package requests on Launchpad and in Brainstorm.

apt-cache is part of the default Ubuntu install. It's very handy to find the right package name, dependencies, and other clues when tracking down the correct package for Launchpad or Brainstorm.

Using debtorrent to contribute to the community

April 7, 2009: Debtorrent is a way to download packages using a torrent instead of a mirror. It's also a way to contribute to the community by reducing the need for mirrors.

Installing debtorrent: (instructions)

  1. Install from the command line ( sudo apt-get install debtorrent apt-transport-debtorrent ), or synaptic.
  2. Edit the lines in /etc/apt/sources.list to take advantage of debtorrent
    deb http://ftp.us.debian.org/debian etch main contrib non-free                              #OLD
deb debtorrent://localhost:9988/ftp.us.debian.org/debian etch main contrib non-free         #NEW
# Do not modify deb-src lines
  3. Reload the package list with sudo apt-get update

Using debtorrent: Debtorrent runs in the background automatically. You don't need to start it or stop it. To check on what it's doing, use the web interface at http://localhost:9988/

One IM application to rule them all...

April 1, 2009: I use Pidgin, the default IM client on Xubuntu 8.04. But I also use the Department of Defense's AKOIM. And my laptop doesn't have a built-in webcam or microphone, so family uses Skype's IM. I really don't want three IM's open (one tying up a browser window and java!), so here's how I consolidated them into just Pidgin.

AKO Instant Messenger uses a standard xmpp protocol, so Pidgin can talk to it. All your contacts migrate like magic! If you have AKO/DKO access, get the details here. If you don't have AKO/DKO access, then safely ignore this paragraph.

Skype uses a proprietary protocol, so it's integration is limited. The skype4pidgin plugin (.deb package) shares Skype contacts and IM with Pidgin...but Skype must still be running alongside Pidgin to work...though many fewer windows must be open. After installation, BOTH Skype and Pidgin must be restarted. I had to edit the Skype options manually to turn off chat notifications - otherwise both apps whine when a new message arrives.

Scanning for wireless networks

March 22, 2009: Two methods to scan for wireless networks. One requires sudo/root, the other requires Network Manager.

#! /usr/bin/env python
"""This python 2.5 script uses iwlist to scan for nearby wireless networks. It must be run as sudo/root to work."""
import subprocess as SU
command = ['iwlist', 'eth1', 'scan']
output = SU.Popen(command, stdout=SU.PIPE).stdout.readlines()
data = []
for item in output:
    print item.strip()
    if item.strip().startswith('ESSID:'): 
        data.append(item.lstrip(' ESSID:"').rstrip('"\n'))
    if item.strip().startswith('Quality'): 
        data.append(int(item.split()[0].lstrip(' Quality=').rstrip('/100 ')))
    if item.strip().startswith('Encryption key:off'): data.append('OPEN')
    if item.strip().startswith('Encryption key:on'): data.append('encrypted')        
print data


#! /usr/bin/env python
"""This python 2.5 script uses dbus to query Network Manager, which scans regularly for wireless networks. It does NOT require root/sudo."""
import dbus
item = 'org.freedesktop.NetworkManager'
path = '/org/freedesktop/NetworkManager/Devices/eth1'
interface = item + '.Device'

bus = dbus.SystemBus()
data = []
wireless = dbus.Interface(bus.get_object(item, path), interface)
for network_path in wireless.getNetworks():
    network = dbus.Interface(bus.get_object(item, network_path), interface)
    data.append(network.getName())                        # also network.getProperties[1]
    data.append(network.getStrength())                    # also network.getProperties[3]
    if network.getEncrypted(): data.append('encrypted')
    else: data.append('OPEN')
print data

Replacing os.popen() with subprocess.Popen() in Python 2.5

March 11, 2009: In Python, you can execute shell commands using the os.popen method...but it's been deprecated in favor of a whole new command. 

# The old way, which worked great!
import os
shell_command = 'date'
event = os.popen(shell_command)
stdout = event.readlines()
print stdout

# The new way, which is more powerful, but also more cumbersome.
from subprocess import Popen, PIPE, STDOUT
shell_command = 'date'
event = Popen(shell_command, shell=True, stdin=PIPE, stdout=PIPE, 
    stderr=STDOUT, close_fds=True)
output = event.stdout.read()
print output

# The new way, all in one line (a bit uglier)
import subprocess as SU
output = SU.Popen('date', stdout=SU.PIPE).stdout.read()
print output

Using DBus on Xubuntu 8.04

March 8, 2009: DBus is a system that permits different applications to exchange information. Tutorial Reference Other Reference.

Sometimes, DBus crashes upon restart from a suspend or hibernation. These bash commands will help you figure out if it has crashed, and how to restart it. 

$ps -e | grep `cat /var/run/dbus/pid` # Confirm if DBus is running by checking for the PID number in the list of live processes.
                                      # If DBus is running, this will return the process number. 
                                      # If not, it will return nothing.
$sudo rm /var/run/dbus/pid            # Remove the stale pid file so DBus can be restarted.
$sudo dbus-daemon                     # Start DBus again.

A python script uses DBus to see if the network connection is available by asking Network Manager: 

#! /usr/bin/env python
import dbus
bus = dbus.SystemBus()
item = 'org.freedesktop.NetworkManager'
eth0_path = '/org/freedesktop/NetworkManager/Devices/eth0'
eth1_path = '/org/freedesktop/NetworkManager/Devices/eth1'
interface = 'org.freedesktop.NetworkManager.Devices'

# There are two possible network interfaces: eth0 (wired) and eth1 (wireless).
eth0 = dbus.Interface(bus.get_object(item, eth0_path), interface)
if eth0.getLinkActive(): print('The wired network is up') # getLinkActive() is a boolean, TRUE if the network link is active
eth1 = dbus.Interface(bus.get_object(item, eth1_path), interface)
if eth1.getLinkActive(): print('The wireless network is up')
This shell script does exactly the same thing, using the same DBus call: 
# This shell script checks Network Manager if the network is up, using dbus as the communications medium.
# There are two possible network interfaces: eth0 (wired) and eth1 (wireless). Of course, you may need to alter these to meet your own circumstances.
# The basic format of dbus-send is: dbus-send --system --dest=org.freedesktop.NetworkManager /org/freedesktop/NetworkManager/Devices/eth0 --print-reply org.freedesktop.NetworkManager.Devices.eth0.getLinkActive
DEST='org.freedesktop.NetworkManager'
PPATH='/org/freedesktop/NetworkManager/Devices'
DEVICE='org.freedesktop.NetworkManager.Devices'

result_eth0=`dbus-send --system --dest=$DEST $PPATH'/eth0' --print-reply $DEVICE'.eth0.getLinkActive'`
shaved_eth0=`echo $result_eth0 | cut -d ' ' -f8`
if [ $shaved_eth0 = 'true' ]; then echo 'The wired network is up'; fi

result_eth1=`dbus-send --system --dest=$DEST $PPATH'/eth1' --print-reply $DEVICE'.eth1.getLinkActive'`
shaved_eth1=`echo $result_eth1 | cut -d ' ' -f8`
if [ $shaved_eth1 = 'true' ]; then echo 'The wireless network is up'; fi

A Python script that queries Network Manager to get the list of wireless networks. 

import dbus
item = 'org.freedesktop.NetworkManager'
path = '/org/freedesktop/NetworkManager'
interface = item + '.Device'

network_list = []
bus = dbus.SystemBus()

# Create a Network Manager interface and get the list of network devices
event = dbus.Interface(bus.get_object(item, path), interface)

# Create an interface for each device
# Query each interface to see if it's wireless
# Query each wireless interface for the networks it sees
for device in event.getDevices():
    device_interface = dbus.Interface(bus.get_object(item, device), interface)
    if device_interface.getType() == 2:  # 0 unknown, 1 wired, 2 wireless
        network_list.extend(device_interface.getNetworks())

# Reformat the network names in the list to be more readable
if network_list:
    for entry in network_list:
        #print entry    # String of path/to/network_name
        entry_list = entry.split('/')
        print entry_list[-1]  # String of network_name

A Python listener that catches the changes in wireless signal strength using both available methods. 

import dbus, gobject
from dbus.mainloop.glib import DBusGMainLoop

def print_device_strength1(*args):  #Use the received signals
    signal_strength = args[1]
    print ('Signal Strength (Method 1): ' + str(signal_strength) + '%')

def print_device_strength2(*args, **kwargs):  #Use the received signals
    signal_strength = args[1]
    print ('Signal Strength (Method 2): ' + str(signal_strength) + '%')

DBusGMainLoop(set_as_default=True)    # Set up the event loop before connecting to the bus
bus_object = dbus.SystemBus()

# The variables you need. I used the shell command 'dbus-monitor --system' to find this information
sender = 'org.freedesktop.NetworkManager'
path = '/org/freedesktop/NetworkManager'
interface = sender
member = 'DeviceStrengthChanged'

# Method 1 - bus_object.proxy_object.connect_to_signal(method, action, filter, message_parts)
proxy_object = bus_object.get_object(sender, path)
proxy_object.connect_to_signal(member, print_device_strength1, dbus_interface = interface)

# Method 2 - bus_object.add_signal_receiver(action, [filters])
bus_object.add_signal_receiver(print_device_strength2, dbus_interface = interface, member_keyword = member)

# Start the loop
loop = gobject.MainLoop()
loop.run()

Thunar responds beautifully to D-Bus. Introspection is fully set up, so it's easy to use with the d-feet application. Useful for launching programs, opening folders and windows, and manipulating the trash. Launching a program by this method means that the the window manager launches the program, not the script or terminal, so the program can remain open after the script or terminal terminates. 

#!/usr/bin/env python
import dbus
item = ('org.xfce.Thunar')
path = ('/org/xfce/FileManager')
interface = ('org.xfce.FileManager')
event = dbus.Interface(dbus.SessionBus().get_object(item, path), interface)

# These three lines at the end of the script open the file's 'properties' window
display = (':0')         # The current session screen
uri = ('/home/me/dbus_test.py')
event.DisplayFileProperties(uri, display)

# These three lines at the end of the script launch a new application
display = (':0')         # The current session screen
uri = ('/usr/bin/gftp-gtk')
event.Launch(uri, display)

# These four lines at the end of the script open a folder window and optionally select a file
display = (':0')         # The current session screen
uri = ('/home/me/.cron')
filename = ('anacrontab.daily')
event.DisplayFolderAndSelect(uri, filename, display)

A sample hal script. 

#!/usr/bin/python
"""This python 2.5 script uses dbus to check if the lid switch is open.
Based on an original python script at http://schurger.org/wordpress/?p=49"""
import dbus

dest = 'org.freedesktop.Hal'
hal_path = '/org/freedesktop/Hal/Manager'
hal_interface = 'org.freedesktop.Hal.Manager'
udi_interface = 'org.freedesktop.Hal.Device'

# Get the list of possible input switches. The return is a list of paths.
bus = dbus.SystemBus()
hal = dbus.Interface(bus.get_object(dest, hal_path), hal_interface)
list_of_udi_paths = hal.FindDeviceByCapability('input.switch')

# Filter the list for the word 'lid'. Print the status for each one.
for udi_path in list_of_udi_paths:
    udi = dbus.Interface(bus.get_object(dest, udi_path), udi_interface)
    if udi.GetProperty('button.type') == "lid":
        # The button.state.value is FALSE if the lid is open.
        if udi.GetProperty('button.state.value'): print ('Lid is closed')
        else: print ('Lid is open') 
    else: print ('Problem: I could not find the lid switch. Sorry.')

Notes:

Bash and Python scripts to modify an OpenOffice .odt document

March 4, 2009: .odt files are actually containers (you can see one using unzip -l document.odt). Within the container, content is in the content.xml file. Script info source

Here's what I've figured out about opening, modifying, and saving the content of an .odt file:

Five ways to make a notification pop-up

February 27, 2009: There are two notification apps included with Ubuntu - notification-daemon and zenity. I've tried them both.

Notification-daemon sleeps until it is triggered (like every other daemon) by shell command, from python, and from DBus. It is the preferred notification method.

  1. From the command line using the notify-send command. This command is part of the libnotify-bin package, and I haven't tried it. From descriptions, it seems pretty easy. One example.

  2. From python using the pynotify module: 
    #!/usr/bin/env python
"""Creates a System-Tray pop-up bubble"""
import pynotify
title = ('Test Title')
text = ('This is the sample text body')
icon = ('/usr/share/icons/Rodent/48x48/apps/gnome-info.png')
pynotify.init(name)
notification = pynotify.Notification(title, text, icon) 
notification.set_urgency(pynotify.URGENCY_NORMAL)
notification.show()
  3. From DBus using the command line. It should be something really close to this, but I can't get it quite right.
    4. dbus-send --session --dest=org.freedesktop.Notifications --type=method_call --reply-timeout=10000 \
  /org/freedesktop/Notifications org.freedesktop.Notifications string:'Test Application' uint32:0 \
  string: string:'NOTIFICATION TEST' string:'This is a test of the notification system via DBus.' \
  array:string: dict:string: int32:10000
  4. From DBus using the python dbus module: 
    #!/usr/bin/env python
"""This is a python 2.5 script that creates a notification using dbus."""
import dbus
item = ('org.freedesktop.Notifications')
path = ('/org/freedesktop/Notifications')
interface = ('org.freedesktop.Notifications')

icon = ('/usr/share/icons/Rodent/48x48/apps/gnome-info.png')
array = ''
hint = ''
time = 10000   # Use seconds x 1000
app_name = ('Test Application')
title = ('NOTIFICATION TEST')
body = ('This is a test of the notification system via DBus.')

bus = dbus.SessionBus()
notif = bus.get_object(item, path)
notify = dbus.Interface(notif, interface)
notify.Notify(app_name, 0, icon, title, body, array, hint, time)

# The four lines can be consolidated into two:
notify = dbus.Interface(dbus.SessionBus().get_object(item, path), interface)
notify.Notify(app_name, 0, icon, title, body, array, hint, time)

# The 'notify =' line can be made even uglier!
notify = dbus.Interface(dbus.SessionBus().get_object('org.freedesktop.Notifications', '/org/freedesktop/Notifications'), 'org.freedesktop.Notifications')

Zenity is an application, not a daemon. It creates a regular window that can be abused for notifications. It can also create icons in the notification area with mouseover information.

zenity --info --title "Big Title" --text "This is the\\nbody text" &
zenity --notification --text "This is the mouseover text" &

You Are Here on a Google Map

February 24, 2009: The following python script shows your current (GPS-enabled) location on a Google Map. A useful learning experience: 

#!/usr/bin/env python

"""This is a python 2.5 script that plot's a GPS receiver's location on the
Google Maps website.

In order to work, you need a network connection, an attached GPS receiver, and
the GPS daemon (gpsd).
"""
import os
#import subprocess and SU
import gps
import dbus
import sys


def test( ):
    """ Step 1: Test for the existence of a running gpsd, test for the existence of an open network connection, and test for a firefox process.
    If any fail, give an error message, don't try to recover. FUTURE: Could also use DBus to test for firefox and gpsd."""

    process_list = os.popen('ps -e')    # os.popen is deprecated in favort of subprocess.Popen
    #process_list = SU.Popen(['ps','e'], stdout=SU.PIPE).stdout.read()  
    gpsd_existence_flag = 0
    firefox_existence_flag = 0
    for line in process_list.readlines():
        if line.count('gpsd') > 0: gpsd_existence_flag = 1
        if line.count('firefox') > 0: firefox_existence_flag = 1

    if not gpsd_existence_flag:
        print ("gpsd is not running. Use 'gpsd -b /dev/ttyUSB0' to start it, and then try again.")
        sys.exit()       
    else: print ('Checking...found gpsd')

    if not firefox_existence_flag:
        print ("firefox is not running. Please start it and try again.")
        sys.exit()
    else: print ('Checking...found firefox')

    bus = dbus.SystemBus()
    nm_item = ('org.freedesktop.NetworkManager')  # This string gets used a lot
    nm_path = ('/org/freedesktop/NetworkManager')
    nm_device = ('org.freedesktop.NetworkManager.Device')

    list_of_interface_paths = dbus.Interface(bus.get_object(nm_item, nm_path), nm_device).getDevices()

    found_network_flag = 0
    for interface_path in list_of_interface_paths:
        one_interface = dbus.Interface(bus.get_object(nm_item, interface_path), nm_device)
        if one_interface.getLinkActive():   # True if there is an active network on this interface
            if one_interface.getType() == 2: # 0 unknown, 1 wired, 2 wireless
                print('Checking...found the wireless network') 
                found_network_flag = 1
            elif one_interface.getType() == 1: 
                print('Checking...found the wired network')
                found_network_flag = 1
                
    if found_network_flag: return
    else:
        print ("cannot find a network connection. Please connect and try again.")
        sys.exit()    


def get_position_fix( ):
    """Step 2: Get a position fix from gpsd."""
    session = gps.gps('localhost','2947')  # Open a connection to gpsd
    session.query('p')                     # Get the location fix 
    lat = session.fix.latitude
    lon = session.fix.longitude
    print ('Location is ' + str(lat) + ' latitude and ' + str(lon) + ' longitude.')
    return (lat, lon)


def show_map(lat_lon_tuple):
    """Step 3: Submit the position fix to Google Maps. Note that the parentheses '()' in the URL must be escaped '\' to work.
    Sample URL format: http://maps.google.com/maps?q=37.771008,+-122.41175+(You+can+insert+your+text+here)&iwloc=A&hl=en"""
    url_string = ('http://maps.google.com/maps?q=' + str(lat_lon_tuple[0]) + ',+' + str(lat_lon_tuple[1]) + '+\(You+Are+Here\)&iwloc=A&hl=en')
    os.popen('firefox ' + url_string)
    return

# Run this script as a standalone program
if __name__ == "__main__" :
    test()
    location = get_position_fix()
    show_map(location)

GPS and Xubuntu 8.04

February 23, 2009: I'm experimenting with USB GPS receiver (dongle). It's a Canmore GT-730F that I received in January 2009. Here's what I've learned so far.

Manually getting data using the command line (source):

  1. Check dmesg, the kernel log, to find out where the device has been mounted. In my case, it mounts reliably to /dev/ttyUSB0. If it doesn't mount, try the command sudo modprobe pl2303 to load the correct USB driver.
  2. Set the data output rate to 4800 baud using the stty command: stty 4800 > /dev/ttyUSB0
  3. Read the data stream using the cat command: cat /dev/ttyUSB0
  4. You should see a set of data scroll down the screen. Use CTRL+C to end the test.

The Linux GPS Daemon (gpsd) is the central clearinghouse for receiving GPS data from the receiver, buffering it, and forwarding it to the applications that want it. gpsd has features to broadcast to dbus (system bus), update ntpd, and respond to a multitude of specific queries from clients. References: Project home page, gpsd man page, and a great example program

$sudo apt-get install gpsd gpsd-clients  # Installs the daemon (gpsd) and test set (gpsd-clients) packages
$gpsd -b /dev/ttyUSB0                    # Start gpsd, telling it where to find the receiver
$cgps                                    # Current satellite data - great way to test that the receiver and gpsd are working

gpsfake is a gpsd simulator. It tricks gpsd into reading from a logfile instead of a real GPS device. Very handy for testing without actually using a GPS dongle. It is included with the gpsd package, and has a man page for additional reference. To make a logfile, and then to use gpsfake: 

$cat /dev/ttyUSB0 > path/to/testlog  # Create the log file. Use CTRL+C to end the capture.
$gpsfake path/to/testlog             # Run gpsd simulator (not a daemon - it will occupy the terminal)

Python interface to gpsd (python-gps) is a programming tool to build your own gps-aware application. 

>>>import gps                             # Load the module
>>>session = gps.gps('localhost','2947')  # Open a connection to gpsd
>>>session.query('o')                     # See man gpsd(8) for the list of commands
>>>print session.fix.latitude             # Query responses are attributes of the session
>>>dir(session)                           # To see the possible responses
>>>del session                            # Close the connection to gpsd

In this case, it seems that I need a periodic session.query('p'), which just gives lat/lon and timestamp.

Time might be an issue, since the system and the GPS may think the time is different. To see if it's an issue, compare them using the python script below. In my tests, they vary from 0.08 to 1.3 seconds apart, not enough to worry about. GPS timestamps use GMT, not localtime. 

#!/usr/bin/env python
import calendar, time, gps
system_time = calendar.timegm(time.gmtime())  # System time (in seconds)
session = gps.gps('localhost','2947')         # Open a connection to gpsd
session.query('p')                            # See man gpsd(8) for the list of commands
gps_time = session.timings.c_recv_time        # GPS time (in seconds)
print ('The time difference is ' + str(system_time - gps_time) + ' seconds.')

MGRS (military map coordinates) conversion to/from latitude and longitude is not currently available in Ubuntu...that I can find. The dongle documentation doesn't mention MGRS at all. An online converter is available. The proj package looks promising, but I haven't figured it out yet. Perhaps Lat/Lon -> UTM -> MGRS?

DBus access appears to be undocumented...but there are tantalizing hints on Google that examples are out there. I can listen to dBus using cgps to make traffic, then dbus-monitor --system to see it.

The best storage format for tracklogs, routes, and waypoints seems to be GPX format, since it's easy to understand and cgpxlogger, included with gpsd, will create an XML track in GPX 1.1 format. Google's KML is more powerful, but also much more complex. GPSbabel universal data translator is a command-line application that translates one file type to another, and does convert GPX <-> KML. 

$cgpxlogger -i 30 > path/to/logfile         # Save data every 30 seconds to an XML file
$gpsbabel -i gpx -f path/to/gpx_file -x duplicate -o kml -F path/to/kml_file
$#gpsbabel [options] -i INTYPE -f INFILE -x FILTER -o OUTTYPE -F OUTFILE

GPSdrive navigation system looks cool, but I couldn't get maps to load, so it's utility was limited. However, it seems that online plotting of tracklogs, routes, and waypoints is possible on Google Maps (and Yahoo Maps, and others). One example is the cool GPS Visualizer.

Gypsy is an alternative daemon, but not currently in Debian or Ubuntu, so I haven't tried it. Last release 0.6 in March 2008.

GPSman device manager is an app I know nothing about. I couldn't get it to work, so I removed it. The dongle seems small enough and simple enough that it may not need to be 'managed' at all.

Using Python to compare .odt files

February 20, 2009: A quick python script to copy, unzip, and reformat an .odt file. It adds indentations and line breaks. Useful to debug my invoicing script, which muddles with the xml files, by making the files diff-able and easier to read and easier to search.

#!/usr/bin/env python
import os
import xml.etree.ElementTree as ET
odt_path_and_file = 'path/to/file.odt'

# This function was copied from http://effbot.org/zone/element-lib.htm
def indent(elem, level=0):
    i = "\n" + level*"  "
    if len(elem):
        if not elem.text or not elem.text.strip():
            elem.text = i + "  "
        if not elem.tail or not elem.tail.strip():
            elem.tail = i
        for elem in elem:
            indent(elem, level+1)
        if not elem.tail or not elem.tail.strip():
            elem.tail = i
    else:
        if level and (not elem.tail or not elem.tail.strip()):
            elem.tail = i

odt_filename = odt_path_and_file.split('/')[-1]
folder_name = ('Desktop/' + odt_path_and_file.split('/')[-1].rstrip('.odt'))
os.popen('rm -r ' + folder_name) #Delete any old working files
os.popen('mkdir ' + folder_name)
os.popen('cp ' + odt_path_and_file + ' ' + folder_name)
os.popen('unzip ' + folder_name + '/' + odt_filename + ' -d ' + folder_name)
reply = os.popen('ls ' + folder_name)
file_list = [filename.rstrip('\n') for filename in reply.readlines() if filename.count('.xml') > 0]
for file in file_list:
    print ('Parsing ' + folder_name + '/' + file)
    tree = ET.parse(folder_name + '/' + file)
    indent(tree.getroot())
    tree.write(folder_name + '/' + file)
    print ('Completed ' + file)

Bean on Mac OS 10.5

February 10, 2009: Our old copy of Appleworks from a previous Mac is sputtering and becoming unreliable in 10.5, so I'm looking for a replacement. OpenOffice and AbuWord are cluttered and unpleasant (spouse dislikes them), but Bean seems pleasant enough to fit our needs. Happily, it reads and writes simple documents to .rtf, .doc, and .odt.

Writing to syslog on Xubuntu 8.04

January 24, 2009: logger is a simple command to write a string to /var/log/syslog. For example: 

logger -i "This is a test string"
will append the string to the syslog.

Moving Cron jobs to Anacron

January 23, 2009: Cron is a great way to run recurring jobs. But some jobs need to run weekly...and sometimes the computer is turned off, so the cron job doesn't run. So I'm going to migrate some jobs to anacron. Cron runs once each minute, checking if the time matches anything in the crontab list. Anacron, however, runs once each day and checks if the interval since each item was last run is greater than the listed value (more than 6 days, for example).

Anacron is a root/sudo-level command. Running it as a user will fail silently.

Anacron stores the timestamps of each job's last run in /var/spool/anacron/JOBNAME. This is handy to change while testing.

There are two ways to run a command using anacron. You can place the command directly in the anacrontab (/etc/anacrontab), or you can put a script in one of the periodic folders (/etc/cron.daily, /etc/cron.weekly, or /etc/cron.monthly)Here are some examples:

Working on Launchpad bugs using the python API instead of the web page

January 22, 2009: Trying to build a simple tracker of the bugs I've worked in Launchpad.

The API installation and instructions and reference documentation.

Web Scraper for string prices

January 2, 2009: I successfully tested a web scraper in python. It scrapes about 20 web pages for the prices of violin strings, then puts the prices in an OpenOffice document for handy printing. It is structured so I can add other output formats, and I could add an XML file to track prices over time or just print changes.

I'm installing it on the store iMac, and setting it as a daily recurring job. The finished file just pops onto the desktop, marked with the date.

A future version may compare prices from multiple sites.

The script and template live in the standard loation for user scripts, /Users/username/Library/Scripts/scriptname/

Installing Skype on an XO laptop

December 21, 2008: I successfully installed Skype with video and did a test call from my XO laptop (build 720) today. I used a version of these instructions. Note: Skype will probably need to be reinstalled each time the xo laptop is upgraded. Warning: Skype isn't really meant for the xo - this install creates several zombie processes each time Skype is started, and the sound quality is definitely inferior.

  1. Install Skype
    $su
#mkdir /home/olpc/skype
#cd skype
#wget http://skype.com/go/getskype-linux-fc7
#yum --nogpgcheck -y localinstall skype-2.0.0.72-fc5.i586.rpm
#wget ftp://ftp.pbone.net/mirror/atrpms.net/el5-i386/atrpms/testing/#libasound2-1.0.15-33.el5.i386.rpm
#rpm -i libasound2-1.0.15-33.el5.i386.rpm
#wget http://dev.laptop.org/~ffm/gstfakevideo.zip
#unzip gstfakevideo.zip
#chmod +x gstfakevideo
  2. Skype doesn't have a sugar package, so it must be started by root with the command /home/olpc/skype/gstfakevideo. This is cumbersome and hard to remember, so use nano .bashrc to create an alias. Add this line to the 'alias' section of the .bashrc. 
    # User specific aliases and functions
alias skype 'sudo /home/olpc/skype/gstfakevideo'
    Now the user (not root) can start Skype from any prompt with the command skype.

SSH Host Verification Keys

December 21, 2008: I tried to ssh to my XO laptop today, but got the following error:

me:~$ssh user@(IP addr of xo_laptop)
@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@
@    WARNING: REMOTE HOST IDENTIFICATION HAS CHANGED!     @
@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@
IT IS POSSIBLE THAT SOMEONE IS DOING SOMETHING NASTY!
Someone could be eavesdropping on you right now (man-in-the-middle attack)!
It is also possible that the RSA host key has just been changed.
The fingerprint for the RSA key sent by the remote host is
blah:blah:blah:blah:blah:blah:blah:blah:blah:blah.
Please contact your system administrator.
Add correct host key in /home/USER/.ssh/known_hosts to get rid of this message.
Offending key in /home/USER/.ssh/known_hosts:NUMBER
RSA host key for (IP addr of xo_laptop) has changed and you have requested strict checking.
Host key verification failed.
me:~$

Why? Probably because I upgraded the OLPC laptop.

How to fix it?

  1. Use ssh-keygen -R (IP addr) to purge the old key.
  2. Connect again like normal ssh user@xo_laptop to install the new (current) key.

Creating a patch to fix an Ubuntu bug

October 13, 2008: Today I'm preparing a patch to fix Launchpad Bug #117984, which is also Gnome Bug #451734.

References: Ubuntu Wiki, Ubuntu Packaging Guide

It turns out that to do it properly, you need to make two patches. The first one is a .diff file for Debian and upstream. The second creates a .debdiff patch for Ubuntu.

There are circumstances, of course, where one of the patches is not neccessary, and some of these steps can be skipped.

    Note: Use steps 1-9 & 14-15 for just an upstream (.diff) patch. Use steps 1-4,5-7,11-13 & 15 for just an Ubuntu (.debdiff) patch.
  1. Open a terminal window
  2. Create a working directory with the command mkdir working
  3. Move to the working directory with the command cd working
  4. Download the latest source package using the command apt-get source packagename. This method automatically appends an '.orig' suffix, and unpacks the file, too. DON'T download the source from packages.ubuntu.com; instead ADD the repos to your Software Sources control panel using the instructions at the Ubuntu Wiki
  5. Make a copy of the unpacked folder with cp -r package-folder package-folder-orig
  6. Go into the unpacked folder (not the orig) with cd package-folder.
  7. Edit the file using nano path/to/file/to/fix. Fix the file
  8. Return to the working directory with cd ..
  9. Use diff -Nurp package-folder-orig package-folder > upstream-bug#.diff to create the upstream patch.
  10. Go back into the unpacked folder (not the orig) with cd package-folder.
  11. Use the command dch -i to update the changelog. Show the change and list the bug# fixed.
  12. Use debuild -S -us -uc to create the debdiff patch.
  13. Attach the .debdiff patch to the bug in Launchpad.
  14. Attach the upstream .diff patch to the Launchpad bug AND the upstream bug.
  15. Delete the working directory, and all contents.

Cron to be deprecated in favor of Upstart in Ubuntu

September 20, 2008: Ubuntu's Upstart is an init daemon replacement, quite analagous to OS X's launchd. Launchd also replaced cron on OS X - and upstart plans to replace cron on Ubuntu. No telling when, but all my cron jobs will need to be reformatted.

Recurring jobs, launchctl, plist, and shell scripting on OSX

September 17, 2008: OSX is Unix - similar, and yet dissimilar to linux. I'm setting up a few recurring maintenance scripts

First of all, cron is deprecated in favor of launchd, which handles a lot more than just cron jobs.

First, recurring jobs are located in /Users/username/Library/LaunchAgent/ as .plist files.

Sample syntax of a .plist file: 

<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE plist PUBLIC "-//Apple Computer//DTD PLIST 1.0//EN" "http:// www.apple.com/DTDs/PropertyList-1.0.dtd">
<plist version="1.0">
<dict>
   <key>Label</key>
   <string>com.thisismytask</string>
   <key>LowPriorityIO</key>
   <true/>
   <key>Nice</key>
   <integer>1</integer>
   <key>ProgramArguments</key>
   <array>
      <string>/Users/username/thisismytask.sh</string>
   </array>
   <key>StartInterval</key>
   <integer>3600</integer>
</dict>
</plist>

Alternative interval for daily 
   <key>StartCalendarInterval</key>
   <dict>
      <key>Minute</key>
      <integer>0</integer>
      <key>Hour</key>
      <integer>5</integer>
   </dict>

Activate the job using the command launchctl load /Users/username/Library/LaunchAgents/plistname.plist
Deactivate the job the same way (except use unload)

Remote desktopping and administration - Vinagre/VNC and SSH

September 10, 2008: My network, at home and work, are reaching a point where I wish to remotely administer (or work on) some machines. This kind of connection turn out to be terribly useful for doing bookkeeping from home. I set up a VNC Reverse Connection through SSH. As installed below, the VNC connection is automatically routed through ssh.

How To Install It

  1. Administrator's Ubuntu: (source) Use the following commands 
    apt-get install openssh-server xtightvncviewer
sudo update-alternatives --set vncviewer /usr/bin/xtightvncviewer
    Don't use the included 'vinagre' application - it's broken and doesn't work with many VNC servers.
  2. Mac systems: (source) Download Vine Server and put it anyplace convenient.
    I put the original in 'Utilities', and a shortcut on the desktop named 'Ian Help'
  3. Windows systems: (source) Use the following commands: 
    Download and open the latest 'tightvnc' zip file for windows
Copy WinVNC.exe and VNCHooks.dll to someplace convenient (desktop works), trash the rest of the .zip
    Download source

How To Use It - from my (helpdesk) point of view

  1. The first thing I need is my own IP address. I can find my IP easily by using the command curl "http://www.whatismyip.com/automation/n09230945.asp", the return is pure text IP address - no further processing required.
  2. Open ports 22, 5500, 5800, and 5900 on my router and port forward them to my computer. In my case, this uses the router's web interface.
  3. Tell my viewer to listen for the connection on port 5500 with xtightvncviewer -listen 0
  4. Tell the IP address to the other side...their server need it.
    MAC Systems:
Find Vine Server in Finder --> Applications --> Utilities --> Vine 3.0 --> Vine Server
Open Vine Server by double-clicking on it.
Select Server Menu --> Reverse Connection.
    Type in my (me, the helpdesk) IP address. Instant connection! Automated/Scripted Mac connection: echo /Applications/Utilities/Vine3.0/Vine\ Server.app/OSXvnc-server -connectHost $connectionIP -dontdisconnect -nodimming 
    WINDOWS Systems:
Double click WinVNC.exe, type an easy password into the "Password:" box, and press OK. 
Right click on WinVNC icon in system tray, choose "Add New Client", and type in my (me, the helpdesk) IP address.
  5. When finished, they can quit their VNC server (On Mac: Apple Menu --> System Preferences --> Sharing, disable Remote Desktop). They don't need to close anything else - they didn't open anything else. I need to close port 5500 on my router (via web interface) and quit my VNC client.

This is a simple script to find my internet IP, my local (router-assigned) IP, command the router to begin port forwarding, set up a job to disable port forwarding later, and set the tightvncviewer on my xubuntu box to listen. So when somebody calls, I just run the script...

#!/bin/sh
# This script sets up my computer to receive a reverse-VNC connection.
# It does _NOT_ need to be run as root.
#
# Step 1: Find my internet IP and echo it to the screen
# The server (person at the other end) needs this to connect.

# Define Variables
#ip_url="http://www.whatismyip.com/automation/n09230945.asp"

externalIP=`curl -s http://www.whatismyip.com/automation/n09230945.asp`
echo "My IP address is: "$externalIP

# Step 2: Find my local, router-assigned IP
# We need this for Port Forwarding the router

localIP=`ifconfig | grep "inet addr" | grep -v "127.0.0.1" | awk '{ print $2 }' | awk -F: '{ print $2 }'`
echo "My Local address is: "$localIP

# Step 3: Create a pop-up or notification with the IP information

zenity --info --title "IP Addresses" --text "Internet IP Address (the server needs this): "$externalIP"\\nLocal IP Address (for Port Forwarding): "$localIP"\\n\\nMac Instructions: Open Vine Server, File --> Reverse Connection\\n\\nWindows Instructions: Open tightvncserver, right-click on icon in system tray, select 'Add New Client'" &

zenity --notification --text "External: "$externalIP"   Local: "$localIP &

# Step 4: SSH to my router and forward router port 5500 to my computer.
# Step 5: Create a job to shut off port forwarding automatically
# Since I don't know IPtables yet, here's a workaround
# that simply open's the router web interface.

firefox -url 192.168.1.1/Forward.asp &

# Step 5: Set up the VNC client to listen.

xtightvncviewer -listen 0
echo "End!"

Music Score Program

September 9, 2008: Trying music notation programs

mscore - looks great, requires fluid-soundfont-gm to playback (piano)

denemo/lillypond - haven't tried yet.

lime might be promising for the Mac - but Lime for Windows doesn't work under Ubuntu Wine.

Replacing update-notifier with a cron script in Xubuntu

September 9, 2008: Update-notifier does a lot of good things. It shows you when a package manager is working in the background, it automates security updates, it cuased updates to be downloaded in the background, and it provides a convenient icon to click on and launch update-manager. Good stuff.

But it's daemon eats an annoying trickle of ram and resources.

I have it set to update weekly, so the rest of the time it's just eye candy. Much of it's power is wasted on my scheduled update cycle. Time to find a replacement with less overhead.

The simple solution: Really, all I need is to run update-manager weekly. So add the following to your user crontab (crontab -e):

#Reminder bot - this e-mail replaces the update-notifier package
40 7 * * sun /usr/bin/nail -s "Reminder Bot: Run Update-Manager" me@example.com >/dev/null
or instead of using nail (to e-mail the notice), I can simply use zenity to create a pop-up: 
#Reminder bot - this e-mail replaces the update-notifier package
40 7 * * sun DISPLAY=:0.0 /usr/bin/zenity --info --title "Reminder!" --text "It's time to run Update-Manager"
or zenity can also create a notification-area icon: 
#Reminder bot - this e-mail replaces the update-notifier package
40 7 * * sun DISPLAY=:0.0 /usr/bin/zenity --notification --text "Reminder: It's time to run Update-Manager"
and I'll get a reminder e-mail every Sunday morning at 7:40 am...if I'm awake. Then I launch update-manager from the menu or command line...if I want.

The lazy solution: The simple solution has a drawback - none of the packages are downloaded yet, so the update might take a while (zzzzz). An additional cron entry, though, will have everything downloaded and ready.
Edit the root crontab: sudo mousepad /etc/crontab
Add the line 42 6 * * sun root apt-get update && apt-get autoclean && \apt-get -q -d -y -u upgrade (source)
This will download the updated and upgraded packages on Sunday mornings about an hour before the reminder e-mail, plenty of time.

In theory we could try 42 6 * * sun root apt-get update && apt-get autoclean && \apt-get -q -d -y -u upgrade && /usr/bin/nail -s "Reminder Bot: Run Update-Manager" me@example.com and have it all in one place, but I'll find that confusing in another 6 months.

Removing update-notifier and update-notifier-common: Nothing depends on the package in Xubuntu 8.04, so a simple sudo apt-get remove update-notifier update-notifier-common gets rid of the packages, freeing up a whopping 426kb of disk space.

I find I don't miss the old notifier at all.

Using the at command in Xubuntu 8.04

September 9, 2008: Tried using at and batch to compare/complement cron. Unfortunately, I couldn't decipher the syntax completely. I could put in a job, but then I couldn't see the results of the job (so it's not much use to me, eh?)

Package management in Ubuntu

September 2, 2008: Package tools and command line tips.

What's the difference between dpkg, apt-get, aptitude, synaptic, and add/remove? Which one should I use?

  1. dpkg: In the beginning, there were packages. And the community divided the stable packages from the testing, and the dev team cried out, 'Lo, these things need a manager, for how else are we to install and remove them?', and the milestone begat 'dpkg'.

    For dpkg was the first, before all others, and is still the heart of package maintenance. dpkg still does the real work, though the usurpers get the credit.

  2. apt-get: But the people were troubled, and they cried out, 'O mighty devs, dpkg is too hard to use. It heeds only our words, and knows not our heart. It is complicated, and we suffer because it is too easy to bork our system. And forth, into the light, came the Advanced Package Manager, and they called it apt. And the most favored among the apt was 'apt-get'

    And apt-get was a glorious step forward, for now the people's package manager heeded their hearts instead of long lists of pakage names when they wanted to update, and it gave them easier choices to understand. And obeisance to apt-get was still obeisance to dpkg, for apt-get was merely an eaier-to-use front end for dpkg.

  3. aptitude: But the people, who are just *never* satisfied, lamented 'O mighty devs, this command-line nonsense has gone far enough. We want a graphical front end tool, because these endless lists of packages are breaking our spirit and making us confused in our oblutions. And the devs heaved a mighty sigh. And their voice brought forth 'aptitude'

    And aptitude had the same commands as apt, but in an easier-to use graphical format. It was called ugly, for it was descended of the command-line, and was despised by the pretty x-server-window adherents, yet it was strong and easy to use. Obeisance to aptitude was strong among the servers and headless devices, for they never listened to the words of the followers of the x-windows-server. And aptitude simply converted the commands into apt commands, and apt converted the commands into dpkg commands, and dpkg still did the real work. And when anyone pointed out that this was a bit convoluted, they were cast out.

  4. synaptic: But the majority of the people, the whiny people, cried out once more 'O mighty devs, who have given us packages and dpkg to rule them, and apt-get to prevent us borking our systems, and aptitude so we don't need to think very hard. And that's nice, for now we can download a lot more pornography. But also, we have heeded the words of the x-windows-server, and we want pretty windows that we control with mice and touchpads and drag around the screen. So chop chop, and get us a version of aptitude for these cool x-windows. And thus was born 'synaptic'.

    And synaptic was strong and robust, and popular among the adherents of GUI were very pleased and multiplied like locusts into legions of users. And like aptitude before it, synaptic simply converted the commands into apt commands, and apt converted the commands into dpkg commands, and dpkg still did the real work.

  5. add/remove: And still the people lamented 'O mighty and wonderful devs, who have found a way to automate updates, who have given me a powerful operating system for free, who have duplicated the miracles of Microsoft and Apple, we think that package management is still too hard, particularly for new users. Your newest followers should be able to install and remove applications with just one click, even more easily than they can now. And the devs considered casting those chuckleheads into the lake of fire, but instead raised their hand and brought forth 'Add/Remove'.

    And add/remove worked with a limited number of applications, a one-click installer/uninstaller of windowed apps, with a lot of advanced details hidden. Add/remove was a front-end for synaptic, which was itself a front-end for apt-get, which was itself a front-end for dpkg.
  6. but still, the people wailed for more and easier, but the tired devs closed their ears and hardened their hearts and went off to work instead on torrent-downloaders to speed up pornography.

Fast File Sharing from Ubuntu

August 29, 2008: Command line Python tool to set up an instant FTP server:

  1. Open a terminal
  2. cd to the directory you want to share
  3. ifconfig gives the network address of the computer
  4. python -m SimpleHTTPServer starts the server on port 8000. (minimize the terminal window so you don't accidentally close it)
  5. Other computers access the server by web: http://xxx.xxx.xxx.xxx:8000, where the xxx (the IP address) came from ifconfig.
  6. Your computer accesses the internal server by web: http://xxx.xxx.xxx.xxx:8000 or http://localhost:8000.
  7. Kill the process CTRL+C to stop the server

Sharing is one-way; no uploading. No security. The entire contents of the directory are shared; no blocking. No logging (that I know of). Just pure download server. Cool.

More fun with cron and terminal

August 22, 2008: Fast experiment to use cron to open a terminal window and execute a command. Success!

The crontab item is:
* * * * * DISPLAY=:0.0 /usr/bin/xfce4-terminal -x top
* * * * * - the crontab time codes. Substitute your own.
DISPLAY=:0.0 - sends the subsequent commands to the screen
/usr/bin/xfce4-terminal - the application (opens a terminal window)
-x top - execute the top command in the application

Once a minute, a new terminal window spawns running 'top', just as intended.

AbiWord Spell Checker on Xubuntu 8.04

August 16, 2008: AbiWord doesn't come with a spell checker installed...but it doesn't tell you that.

Here are the packages to install the spell checker: (source)

Use Synaptic or the command line to install. Restart AbiWord, and Presto...now it works.

Trouble connecting to wi-fi

August 3, 2008: Trying to connect to our old 801.b Linksys router. Under Xubuntu 7.10, it connected fine...but this is the first time trying under 8.04. No luck. Just keeps refusing to associate with the ESSID, and returning error 812 (whatever that is).

But the hardware works - boot into XP, and it connects just fine.

Installing an SCR201 CAC Card Reader in Xubuntu 8.04

July 28, 2008: Trying to get an SCR201 PCMCIA smart card reader to work.

Packages to install: libccid libpcsc-perl pcsc-tools pcscd libpcsclite1 libckyapplet1 coolkey

Just can't get it to work. Oh well, we'll see if it works on Windows boot.

Creating an Evolution e-mail from shell and python

July 24, 2008: Evolution is intended as a GUI e-mail client, so the scripting options are limited. Also, it looks like you cannot autosend e-mail from a script, there's so script command equivalent to the 'send' button.

But that's a good thing; I can autosend from nail when I want to. So a script will compose the e-mail, which will sit on my desktop until I review it and click to send it. Nice.

shell command source
This command creates a window with To:, From:, Subject:, and Body filled in. 'From:' is already known by Evolution, the rest is parsed from the following command: 

evolution mailto:person@example.com?subject=Test\&body=Some%20crazy%20story%20stuff%0D%0Acolumn1%09%09column2%09%09column3%0D%0A%0D%0ASincerely,%0D%0A%0D%0AMe

Another (easier) way: 
evolution mailto:person@example.com?cc="second_person@example.com"\&subject="This Is The Subject"\&body="This is the body of the message"\&attach=Desktop/test_file.xml

python command
Python has it's own smtp module for creating e-mail, it's far more useful in most circumstances. But in this case, we want the composed evolution window.

import os
body_string = 'This is the body of the e-mail message'
body_string = body_string.replace(' ','%20')
os.popen('evolution mailto:person@example.com?subject=Test\&body=' + body_string)

>>> import os
>>> to = 'person@example.com'
>>> cc = '"second_person@example.com"'
>>> subject = '"This is the subject"'
>>> body = '"This is the body"'
>>> attachment = 'Desktop/rss_test.xml'
>>> os.popen('evolution mailto:'+to+'?cc='+cc+'\&subject='+subject+'\&body='+body+'\&attachment='+attachment)

Fixing a broken cron job

July 22, 2008: I am afflicted by Ubuntu Bug #189462, and so I'm getting occasional e-mails from cron that tell me: 

/etc/cron.daily/slocate:
slocate: fatal error: load_file: Could not open file: /etc/updatedb.conf: No such file or directory

It turns out to have a trivial solution: touch /etc/updatedb.conf

What are all these processes?

July 19, 2008: Trying to figure out all the processes I have running, to see what I can kill, and which packages I can remove.

According to gnome-system-monitor:

System or Root processes I uninstalled, 

avahi-daemon - the avahi local discovery system.
System or Root processes I don't use but can't uninstall, 
atd                - the at command daemon.
console-kit-daemon - a tracking app for multiuser systems. Many GNOME apps depend on it for some reason.
getty              - console, 6 of them open.
System or Root processes I should keep, 
bonobo-activation-server - GNOME component tracker. Safer to leave in place.
gam_server               - gamin tells the window manager about changes in the filesystem.
thunar-tpa               - Thunar trash can applet
evolution-data-server    - database including addressbook and other functions to integrate Gnome app data (breaks Evolution)
evolution-alarm-notify   - alarm clock (breaks Evolution)
system-tools-backends    - DBUS

Configuring Wine in Xubuntu

July 17, 2008: Wine 1.0 bugs and fixes.

Using DOD-essential software: PureEdge and ApproveIt and under Wine (Xubuntu)

July 16, 2008: The Army requires leaders to use PureEdge Viewer 6.5 for all forms, and ApproveIt 5.7.3 to digitally sign the forms...and they are only available for windows.

A Google search 'linux xfdl' and a Synaptic package search turned up no likely substitutes in Linux.

Under WINE 1.0, PureEdge installs cleanly (right-click on the file, 'Open with Wine Windows Program Loader'), but firefox won't send forms to it, and it chokes on a form...see wine bug #11625 .

Under WINE 1.0, ApproveIt fails to recognize it's serial number, and installation fails.

Using aria2c for ftp, http, and torrent downloads

July 16, 2008: Trying aria2c as a commpan-line replacement for wget, curl, and torrents.

Installing DOD CLASS 3 CA-7 security certificate into Firefox 3.0

July 14, 2008: The US Army has a plethora of websites to keep it's mighty bureaucracy chugging along. Unfortunately, the security certificate they all reauire is not included in Firefox 3.0 (under Ubuntu 8.04). Here's how to get it and install it.

NOTE: The certificate is worthless to non-DOD people. It doesn't give you access, you still need an account. It's really boring, anyway, and none of the cool secret stuff is in these websites. All the certificate really does for most people is prevent the annoying message: This website has a certificate that I don't trust.

  1. Download the following three files to the desktop:
    • http://dodpki.c3pki.chamb.disa.mil/rel3_dodroot_1024.p7b
    • http://dodpki.c3pki.chamb.disa.mil/rel3_dodroot_2048.p7b
    • http://dodpki.c3pki.chamb.disa.mil/dodeca.p7b

    The easy way in linux is to use curl -O http://dodpki.c3pki.chamb.disa.mil/rel_dodroot_1024.p7b -O http://dodpki.c3pki.chamb.disa.mil/rel3_dodroot_1024.p7b -O http://dodpki.c3pki.chamb.disa.mil/dodeca.p7b

  2. Go to the Firefox Certificate Manager
    • Open Firefox
    • Edit Menu --> Preferences
    • Advanced settings
    • Encryption tab
    • View Certificates button

  3. Import the three new certificates
    (Repeat for each certificate)
    • Authorities tab
    • Click the 'Import' button
    • Show firefox where the downloaded certificate is and click 'OK'

  4. Fix a bug with the CLASS 3 CA-7 Certificate
    • In the Certificate Manager, Authorities Tab, scroll down to the new 'US Government' entries
    • Select DOD CLASS 3 CA-7, and click the 'Edit' button
    • Two of the certificate boxes should be checked. Check them if they are not:
      This certificate can identify web sites
      This certificate can identify mail users

  5. Whew. You're done. Close the windows, restart Firefox and test it.

Importing the same certificates to Evolution is a similar method.

Messing with the history file in Linux

July 10, 2008: I set the following bash history to make the history file more useful to me. 

HISTCONTROL=ignoreboth     #Prevent duplicate history entries
HISTIGNORE=ls:history      #Don't remeber these commands in history

Using cron and nail/zenity to set up an e-mail reminder bot

September 9, 2008: I could do this in evolution, but it's more fun to use cron and nail or zenity

This crontab entry:
0 6 28 * * /usr/bin/nail -s "Reminder bot: Pay The Store Rent!" me@example.com </dev/null
will send the following e-mail at 6:00 am on the 28th of each month, 

From:    me <my computer>
To:      me@example.com
Subject: Reminder bot: Pay The Store Rent!
Date:    Thu, 28 Jul 2008 06:00:01 -0500 
Text:    None

So I won't forget to pay the rent.... It's important.

This sends an e-mail to a known (to nail) e-mail server, so all machines that check the account get the mail.

Alternately, if I want a a pop-up window:
0 6 28 * * DISPLAY=:0.0 /usr/bin/zenity --info --title "Reminder Bot" --text "Pay The Store Rent"
will give me a pop-up window instead, and
0 6 28 * * DISPLAY=:0.0 /usr/bin/zenity --notification --text "Reminder Bot: Pay The Store Rent"
will give me a tiny notification-area icon.

DVD ISO to AVI file

June 18, 2008: I have an old DVD ISO image (I make DVDs of my kids), and I want to make an AVI copy to upload the funniest bits to YouTube. This is on an Ubuntu 8.04 system, your mileage may vary.

  1. Install dvd::rip using Synaptic
  2. Mount the .iso image (source):
    1. Create a mount point with sudo mkdir /mnt/iso
    2. Mount the image with sudo mount -o loop /path/and/file.iso /mnt/iso
      Alternate: Somewhere along the line, my Thunar magically gained this ability when I right-click on the .iso icon, but I cannot remember which package made this possible...
  3. Rip the DVD. Open dvd::rip.
    1. Preferences: I created no special folder for ripped projects; just saved to the home directory. I checked the dependencies, and had to download a mountain of them (xine, mplayer, etc) using Synaptic.
    2. Storage: Click 'Choose DVD image directory', and select /mnt/iso. Select the button for 'Encode DVD on the fly' since it's already on the hard drive.
    3. Rip Title: Click 'Read DVD table of contents' and the chapters appear. I highlighted (CTRL + Click) the chapters I wanted to convert.
    4. Clip and Zoom: Click the preset menu, select 'Autoadjust - Medium Frame Size, HQ Resize'.
    5. Transcode: Video bitrate calculation is the size of the final file, so I can keep it small (~10 MB per minute, or ~200 MB per half hour). Finally, click 'Transcode' and wait an hour or two or five for the machine to work.
    6. Test the finished file to ensure it's really what you want. I had to do it a couple times, twiddling with various controls.
  4. Cleanup.
    1. Unmount the iso: sudo umount /mnt/iso
    2. Remove the empty mount point: sudo rmdir /mnt/iso
    3. Delete the .iso file (optional, obviously)
    4. Open Synaptic and get rid of all the packages you won't use again. Or use: 
      sudo apt-get remove dvdrip mplayer xine
sudo apt-get autoremove
sudo apt-get autoclean

The package ripmake didn't work - tcprobe failure.

Python List Tricks

June 17, 2008: Poking around looking for ways to clean up my Python code for MWC. It's probably time for me to let go of my 25-year-old BASIC programming experience, and stop making my Python look like BASIC:

Eliminating duplicates from a list using Sets. Currently I convert a list to a dictionary and back to do this: 

>>> basket = ['apple', 'orange', 'apple', 'pear', 'orange', 'banana']
>>> fruit = set(basket)              # create a set from a list, removing duplicates
>>> fruit
set(['orange', 'pear', 'apple', 'banana'])

>>> li = []                          # create a list from a set
>>> for f in fruit:
...   li.append(f)
... 
>>> li
['orange', 'pear', 'apple', 'banana']

>>> fruit = set([])                   # create an empty set
>>> fruit.add('fred')                 # adding to set
>>> fruit.remove('fred')              # removing from set

>>> fruit.clear()                     # clear all data from a set


>>> 'orange' in fruit                 # fast membership testing
True
>>> 'crabgrass' in fruit
False
Source: Dive into Python.
To do: Convert MWC keyword (tag) lists to sets - the duplicates annoy me.

Using List Comprehensions as a building block for filters and mapping. What else are they good for?

Filter data using Filters instead of for loops

Add data using Map instead of other crazy structures

Ubuntu Fonts

June 13, 2008: Looking for cool fonts for violin labels (the paper tag inside the f-hole of the instrument).

I used the free font 'Adorable' from abstractfonts.com.

Fonts in Ubuntu are much more straightforward than I expected:

Legacy Printing

June 8, 2008: Thirteen years ago, I purchased a laser printer that still works great despite a few wrinkles and liver spots. Here's how I keep it alive currently.

Hardware:

The printer doesn't have an ethernet port, only a localtalk (serial) port. Long ago, our last serial-printer Mac died, so the only way to talk to the printer is through the network via the EtherWrite adapter.

The EtherWrite is not detectable on the network if it's directly connected (by ethernet) to a mac or a router. It only works if the connection passes through a hub first. I don't know why. The EtherWrite's measly documentation is unhelpful, and the company that made it died long long ago.

The printer itself still has clean and sharp output. It's outlasted a couple serial, USB, and multifunction inkjets. Toner is still available. The case is yellowing, and the rear feeder spring has been lost, so it feeds out the top - I taped the rear feed selector level in the position I wanted.

Since I print at home only rarely, I leave the printer/adapter/hub assembly powered off for months at a time. When I need to print, I turn them all on, then alternate unplugging/replugging the printer and adapter until they show up on the network. I've never figured out what the proper power-up sequence should be - and I've tried.

Software:

The printer only speaks Apple's localtalk. It's built-in on the MacBook, and available as an add-on on Ubuntu.

Printing from the MacBook is simple. First, it needs to be plugged into the router, since my router's wireless doesn't support Appletalk. Next, print the document. If the printing errors out "cannot connect to printer," then cycle the power on the printer or the adapter. Since the printed document is in queue, when the printer appears on the network, the printer will begin working despite the original error message.

Printing from linux is a bit more tricky, and Ubuntu doesn't make it easy. But it can be done. These instructions are for Ubuntu 8.04, and your results may vary.

  1. Install the netatalk package with sudo apt-get install netatalk. Due to a bug in the package, you might get a bunch of errors: 
    dpkg - trying script from the new package instead ...
hostname: Unknown host
invoke-rc.d: initscript netatalk, action "stop" failed.
dpkg: error processing /var/cache/apt/archives/netatalk_2.0.3-6ubuntu1_i386.deb (--unpack):
 subprocess new pre-removal script returned error exit status 1
hostname: Unknown host
invoke-rc.d: initscript netatalk, action "start" failed.
dpkg: error while cleaning up:
 subprocess post-installation script returned error exit status 1
Errors were encountered while processing:
 /var/cache/apt/archives/netatalk_2.0.3-6ubuntu1_i386.deb
E: Sub-process /usr/bin/dpkg returned an error code (1)
A package failed to install. Trying to recover:
dpkg: error processing netatalk (--configure):
 Package is in a very bad inconsistent state - you should
 reinstall it before attempting configuration.
Errors were encountered while processing:
 netatalk
    Don't sweat if the errors look like these - you'll fix them in the next step. What's happening is that the bug prevents netatalk from figuring out the name of it's host.

  2. The netatalk package is broken. Edit these two files to fix them - then netatalk will work Edit the files even if you didn't get the errors in the last step. 
    /etc/default/netatalk
FROM: ATALK_NAME=`/bin/hostname --short`
TO: ATALK_NAME=`/bin/hostname`

/etc/init.d/netatalk
FROM: ATALK_NAME=`/bin/hostname --short`
TO: ATALK_NAME=`/bin/hostname`
  3. Test the printer connection.
      nbplkup PrinterName - Find the printer on the network.

    If the printer's not on the network, there are three possible reasons:

    1. Most wireless routers talk Appletalk only on the wired connections - if you're using wireless, that may be the problem. Obviously, this doesn't apply to Apple wireless routers. For example, our Macbook can't talk wirelessly to the Apple printer because the wireless router can't understand Appletalk.
    2. The netatalk daemon in being flaky. There's a second serious bug in netatalk: The daemon looks for Appletalk devices on the network at startup; if it can't find any, then it shuts down instead of sleeping. For example, if I boot my computer into eth1 (wireless) connection -and we already know my router can't handle wireless Appletalk- netatalk finds no Appletalk network and the daemon shuts down. If I plug in eth0 (wired) connection, and nbplkup PrinterName, it will still fail - netatalk never checked the new connection. Use sudo /etc/init.d/netatalk restart (or stop and start to reset the daemon.
    3. The printer or adapter is being flaky. I make sure the printer is visible to the Macbook (always best to start with a known point, and the Macbook speaks Appletalk like a native). Otherwise, cycle the power on the printer or adapter and try again. With such old hardware, I don't know the correct order to power them up. It can be annoying.

  4. Test command-line printing. Before setting up CUPS, install 'pap' (actually just a script to make pap executable from CUPS) and test it on the command line. CUPS needs pap in the next step.
    • Download pap and save it to your desktop.
    • Make pap executable with sudo chmod +x Desktop/pap
    • Install pap in the correct directory with sudo mv pap /usr/lib/cups/backend
    • Restart CUPS with sudo /etc/init.d/cupsys restart
    • Prepare a test file for printing. The printer seems to only speak Postscript, so your testfile must be in Postscript format. I used this one. In Ubuntu, the a2ps package (included with default install) will create .ps files. Usage: a2ps input_text_file -o output_file.ps
    • Test print using pap -p Printername Testfile.ps

  5. Add the printer to CUPS
    6. Most interesting. I can't get CUPs to print, though the rest is great. Hmmmm.

Custom Browser URL bar icons

June 7, 2008: Long ago, I made a favicon.ico custom icon for the Milwaukee Without A Car website, but I don't remember how.... I think I created a .bmp image and simply renamed it...

So I found a new SVG of the same graphic, but I'm not using it - instead I'll stick with the old favicon until inkscape can export to .ico files.

Another example showing how SVGs and their tools have great promise, but aren't quite there yet.

More SVG Shortcomings

June 6, 2008:

So I'm back to .png and .gif and .jpg for most HTML images. As before, SVG original for easy changes, but export raster images. Well, that's what experiments are for.

Hey, this is my first red box in over a month!

Advantages and shortcomings of SVG images

June 5, 2008: After a bit of experimenting with SVG graphics, I've reached the following conclusions:

The upshot is that I'll keep file archives in SVG for future manipulation, resized SVGs for web use, and exported ODGs for print media.

Wishlist:

  1. SVG can be scaled by HTML tags
  2. OpenOffice imports SVG natively

Sending e-mail from the command line

June 1, 2008: I want to send automatic e-mail as a cron job from my Xubuntu 8.04 laptop. Originally, I tried routing it through the legacy mail spools - it is, after all, on the same laptop as my e-mail - but gave up in despair. Thanks to Ubuntu Forums for the info.

  1. sudo apt-get install ssmtp nail
    or install ssmtp and nail using Synaptic.

  2. sudo mousepad /etc/ssmtp/ssmtp.conf to edit the ssmtp config file. Modify or add the following lines to the config file: 
    mailhub=mail.mymailserver.com
AuthUser=me@mymailserver.com
AuthPass=my_mail_password
    The nail config file does not need to be changed at all.

  3. Using the nail command to send e-mail:
    Tip: Pay close attention to the <RETURN>s and <CTRL+D>s
    First line: nail recipient@address.com<RETURN>
    Second line: Subject: subject<RETURN>
    Third line: E-mail body text<CTRL+D>

    Example:
    nail ian@virusmagnet.com
    Subject: Lotta viruses coming into this account
    Ian, you need to stop responding to the spam.

  4. Generating e-mail as part of a script or cron job:
    Script nail -s "Test 3" recipient@email.com < /tmp/test_email will send the following e-mail.

    From: You (automatic)
    To: recipient@email.com
    Subject: Test 3
    Message Body: (read from file /tmp/test_email)

    nail -s "Test 3" recipient@email.com < /dev/null will send an e-mail with a blank body - good for quick reminders using the subject line only.

    To generate e-mail from cron events, use the command crontab -e to edit your crontab file:
    26 21 * * * nail -s "Test 3" recipient@email.com < /tmp/test_email
    This crontab entry will send the same "Test 3" e-mail every day at 9:26 p.m.

Many elements can be added or cusomized - this is simple enough to generate automatic e-mails to myself.

Replacing deprecated HTML tags

May 31, 2008: HTML has changed since I learned it. For example, <style> tags weren't useful, and CSS was just starting out.

Ew, I'm old.

So I used to write an image tag as <img src="General_Lew_Wallace.jpg" width="108" height="144" align="left" border="1">.

But, an alt text is now required, and the border has been deprecated. So after some poking around with CSS, I came up with:

<img src="General_Lew_Wallace.jpg" alt="General Lew Wallace of the U.S. Civil War. I don't look like him." width="108" height="144" align="left" style="border-style: ridge; margin-right: 10px">

It does look better. Thanks to the tutorials at www.w3schools.com

Using Bash aliases to simplify the universe

May 30, 2008: I update this blog using the curl command to upload to the web server (see my entry back on March 8, 2008, "The Joy of cURL").

The command, curl -v -T TechBlog.html -u website:password ftp://ftp.freeservers.com/ian/ lives in my Bash history; each time I want to upload, I simply up-arrow through the history at the command prompt until the list time appears, and then hit -enter-. Presto! Terribly easy.

But sometimes I've done a lot of command line work in the meantime (for example, python programming), so I up-arrow seventy or ninety times before the old curl command reappears.

I'm much too lazy to do that regularly.

So I created an alias in my .bashrc file: alias blog='curl -v -T TechBlog.html -u website:password ftp://ftp.freeservers.com/ian/'. Now I can simply type blog instead of looking for the old curl command.

Linux converter for Microsoft .lit files

May 29, 2008: Ubuntu doesn't have a convenient reader or converter for .lit (e-book) files, because .lit is a proprietary format.

To install the C-Lit converter on Ubuntu, use the command line. Do the following three commands in order:

  1. wget http://ace-host.stuart.id.au/russell/files/debian/sarge/libtommath/libtommath_0.37-1_i386.deb)
  2. wget http://ace-host.stuart.id.au/russell/files/debian/sarge/clit/clit_1.8-1_i386.deb
  3. sudo dpkg -i libtommath_0.37-1_i386.deb clit_1.8-1_i386.deb

To use clit (oh, what name!): clit inputfile.lit outputdirectory/

inputfile.lit is the path to the lit file, /home/me/ebooks/mybook.lit (of course, the /home/me/ can be left off if you're not root)

outputdirectory/ is the path and name of the new folder clit will create. The trailing slash (/ or \) means that the directory does not already exist. Each .lit file should be exploded to it's own directory. For example ebooks/mybook/

So the final command would be clit ebooks/mybook.lit ebooks/mybook/, and the html version of the book will appear in the newly-created mybook folder

Adding python event notifications to the Desktop

May 28, 2008: Figured out how to get a Python script pop up a notification bubble on the desktop:

It uses the pynotify frontend to libnotify. It's not part of the python 2.5 base, but is included with the Ubuntu default install. Here's a test script for my Ubuntu system. It pops up a little bubble from the system tray

import pygtk
import pynotify

pynotify.init( "Some Application or Title" )
n = pynotify.Notification("Title", "body", "dialog-warning")
n.set_urgency(pynotify.URGENCY_NORMAL)
n.show()

Tip: The .init() call is neccessary, or you'll get a lot of ugly DBUS and GTK errors.

Changing Stuff: Simply replace the elements("Title", "body", "dialog-warning") with your desired title, body, and image path. The image display will take .jpg, .png, .svg, and likely others. For example...

n = pynotify.Notification("Milwaukee Without A Car", "The Python script MWC_Webcrawler has completed a scheduled run. The logfile has been added to your desktop", "/usr/share/icons/Rodent/48x48/apps/gnome-info.png")

Adding RSS Icon to the browser URL bar

May 19, 2008: Found out how to add an RSS icon to the URL bar:

Add the following to the <head> section of the page: <link rel="alternate" type="application/rss+xml" title="Your News Feed" href="http://somewhere.com/news.xml">

Xubuntu 8.04 Sessions

May 15, 2008: Had terrible problems with applications autostarting at login that I didn't want. I looked everywhere in /etc and /home/me/.config, but OpenOffice and Thunar just kept appearing.

Then I figured it out. One lousy time I checked the Save session for future logins option on logout. So it restarted that way every time.

So the fix was to shut down all applications, logout (saving the session), log back in. Voila! All fixed.

Upgrading to Ubuntu 8.04

April 27, 2008: Here's how I upgraded my 7.10 laptop to 8.04. Ubuntu Forums source.

  1. Download the alternate install .iso file using torrent (to avoid the overloaded servers) - all downloads - http://releases.ubuntu.com/8.04/ubuntu-8.04-alternate-i386.iso.torrent (i386 alternate torrent)
  2. Mounted the .iso file and ran the installer from Terminal: gksu "su /mnt/xubuntu-8.04-alternate-i386.iso/cdromupgrade"

But it didn't work. So I upgraded over the network instead.

Spawning a new Terminal window

April 9, 2008: For the timetable project, I want to occasionally create (or perhaps spawn) a new terminal window that will track the python script(s).

Created the following item in my /etc/cron.hourly folder: 

python ian/test_script

Created the following script as test_script.py: 

#! /usr/bin/env python
import os
os.system("xfce4-terminal")

It works on the command line, but not from an xfce4-terminal window. So I removed the code, and I'll log to a file for now instead.

Converting bitmap images to vector graphics

April 5, 2008: Our sign contractor for the store created some beautiful graphics - but the disk they gave us was all .jpg and bitmap .pdf and even a bitmap .ai file. Limited usefulness for reuse unless we can convert them to vector graphics - bitmaps are big and pixellated, vectors are small and infinitely scalable smoothly.

Used Inkscape. Turned out to be incredibly easy. Import the bitmap, then Path -> Trace Bitmap. Next, File -> Document Properties to crop the page area (so it doesn't save as one logo in the corner of an empty sheet of paper). Save the converted file.

Next problem: Inkscape saves as .svg, but OpenOffice can't open it. Easy to fix - instead of .svg, have Inkscape save as .odg, an OpenOffice drawing file. I love when things work out.

Monochrome and color laser prints of the converted graphics are great - no pixels, smooth and clean edges. Interesting: The bitmaps look better on screen, but the vector graphics are superior on paper.

DVD copying

April 4, 2008: Needed to backup a DVD. On Linux, Brasero failed to finish reading the disk, and never said why. The MacBook Disk Utility errored out with -39. Back to Linux dvdcopy - error reading Title VOB at block 159. No luck - giving up.

NTFS woes

April 3, 2008: The 320GB external NTFS drive is faring poorly, but it's Linux's fault. There's a bug in the kernel. As an attempted fix, I enabled the backport repository and updated the driver (from 6 months old to four months old). Minor improvement only. Since the next update of Ubuntu is due in three weeks, I'll wait to see it it's fixed then.

First python script

March 31, 2008: Did my first successful python scripts! The first simply imports the second, the second checks the existence of a network connection.

NTFS external drive

March 20, 2008: For fun, I checked my external drive while booted in Windows. Oh no! It needed to be defragmented! Defrag recommended CHKDSK as well.

Big mistake: One or both of them scattered a lot of my backup files (including the backups of this blog - quite a few entries gone until they turn up again). I'm afraid to look at my music...or my collection of Futurama episodes. Only fix is to hunt through the 'found' folder and put stuff away manually. Big mess.

Tesseract OCR

March 13, 2008: Tesseract is a very powerful OCR package that works only from the command line. Imagemagick is a very powerful image conversion toolkit.

So to OCR a PDF, first convert the PDF to a TIFF with convert inputfile.pdf covertedfile.tiff, then OCR with tesseract covertedfile.tiff textfile. That's amazingly easy.

Deluge Woes

March 13, 2008: I just figured out that Deluge saves a quick-resume file in the same folder as the torrent (/home/me/.config/deluge/torrentfiles). I wondered how often it updates - now I suspect it updates every time Deulge successfully quits without an error.

If so, then a good idea is to quit and restart Deluge every 6 or 12 hours or so, thereby limiting the loss from a crash.

My external drive is NTFS, and likes to crash on occasion...so I might start setting my timer.

New MacBook - File Sharing

March 13, 2008: File sharing is finally easy. System Preferences -> Sharing -> Turn sharing on. Oprion to log in as the user for total access. At last. The MacBook and iBook share wonderfully. Haven't been able to get the Linux box or the XO laptop on board yet - they see the file shares in Avahi, but can't connect to them.

New MacBook - Migrating

March 9, 2008: The new MacBook arrived a couple days ago, and I finally touched it tonight. The new MacBook had OS 10.5, the old iBook has 10.3.9. The iBook wouldn't run the Migration Assistant, and I don't have a Firewire cable anyway...so we used the house wireless, I logged into the iBook through file sharing on the MacBook, and moved most of Library by hand. Great web page detailed which files to move...but can't find it amy more.

Sharing Files Using NFS

March 9, 2008: Setting up an NFS server for local file sharing is much easier than I expected. But getting it to work is much harder:

  1. Install package nfs-kernel-server for the service.
  2. Tell NFS which folders to expose to the world with the 'export' file, sudo mousepad /etc/exports (I use Mousepad under xfce). The examples are a bit sparse, but there are a couple good examples here.

No mucking about with .config files, no ports or services to modify. The Ubuntu packages are configured properly already.

On the OLPC (great info here), it's just as easy to set up an NFS client:

  1. In Terminal, do su, then yum install nfs-utils
  2. Reboot, then in Terminal do su, then mount ipaddress:/ServerPath /LocalMountpoint, for example mount 192.168.1.38:/media /mnt

Only big sticking point so far is Permissions. The mount works, but the files are not accessable due to permissions (gid, uid, etc...). Very helpful overview here, and the quick list of essential terminal commands, in proper order:

We'll see if it works after a reboot. Still need to figure out permissions, NFS client and server on Mac, NFS server on OLPC.

Sharing Music Files Using Avahi

March 8, 2008: Success! I successfully shared my Linux (Ubuntu 7.10) music files to the Mac using Avahi on a wireless network. Now, if the Mac and Linux boxes are on the same network, the Mac can see and play any of the Linux songs.

How I set it up (surprisingly easy):

  1. Install package mt-daapd for the service, and package avahi-discover as a diagnostic tool.
  2. Open a web browser to localhost:3689, Username: (blank), Password: mt-daapd. The project is called Firefly Server.
  3. On the Configuration tab, put in the directory to music files. On the Server Status tab, click Start Scan.
  4. Open the Mac and launch iTunes. Look in the Shared section for the Firefly Server, and test play a file.

No mucking about with .config files, no command line work, no ports or services to modify.

We'll see if it works after a reboot. Still need to figure out how to share iTunes music with Rhythmbox, and how to share both wth the OLPC. Still need to figure out how to share video files, and other non-media files.

Mounting a USB stick drive on an OLPC Laptop

March 8, 2008: Success! How do you mount a USB stick on an OLPC laptop and get a window of the contents?

Tip #1: Try several of the USB ports. I was successful in the lower right side port.

Tip #2: Once inserted, the USB stick will automount under /media/UsbStickName. Just do an ls /media/ to see your stick there.

Tip #3: For the GUI, look in the Journal. The stick shows as an icon on the bottom. Click it, and the window of contents opens!

Tip #4: To unmount and eject the drive, use the Journal's USB Drive icon. Trying to use the umount command in Terminal fails, claiming the drive is busy.

The joy of cURL

March 8, 2008: curl is a wonderfully useful and scriptable upload/download tool. For example, instead up firing up FTP to upload this page to my website, I figured out that I can open a Terminal and enter: $curl -v -T localpath/TechBlog.html -u username:password ftp://ftp.freeservers.com/ian/. And it does the whole thing for me!

Cleaned out orphaned packages

March 8, 2008: Periodically, I use the terminal command deborphan to clean out orphaned packages. Found eight or nine.

It's easy to use. Deborphan merely lists the items. I use Synaptic to remove them.

Fixed Toshiba Sound!

March 7, 2008: Since the reinstall, there's been no sound in Linux. Xubuntu has output no sound. Found the following solution on the Ubuntu Forums.

Edit the alsa config file: sudo mousepad /etc/modprobe.d/alsa-base

Added the following lines at the bottom of the file:
# Fix for Toshiba sound. From Ubuntu Forums
options snd-hda-intel model=auto

We'll see how it works at the next reboot.

New MacBook on the way!

March 5, 2008: My wife's old iMac G4 started making an ugly sound last night.

So I backed it up immediately. We looked at the loose power connector, the buckling casing, and the noise...and decided it had become unreliable. So I ordered a replacement today.

Looks like the kids get a spiffy toy.

Reinstallation fallout

March 5, 2008: So far, only three problems with the reinstallation.

  1. Some data from my shared partition was lost - my backup wasn't as thorough as I thought. I also had to reenter all my e-mail accounts to Evolution (backup copy on the Mac, whew). Nothing critical, just minor annoyances...and a couple Doctor Who episodes.
  2. The trackpad was fine for the first day, then suddenly became jumpy and unusable today. Looking through various forums, it seems I'm not alone. May or may not be related to the reinstall. On Windows, it was intermittent. On Xubuntu, it's always bad.
  3. Sound works great on Windows, no sound at all in Xubuntu. Well, I kind of expected it...

Dual Boot (Win XP / Linux) Reconstruction

March 4, 2008: I decided not to wait until April...

Downloaded and burned an Ubuntu 7.10 Alternate CD. Reinstalled Windows (snore, 1 hour of installing, 2 hours of updating and faddle). Installed Ubuntu 7.10.

Uh-oh. No I didn't. The Ubuntu CD was corrupt, and instead hosed everything again.

Reinstall Windows while using the Mac to download and burn an Ubuntu 7.10 Live CD.

The live CD passes the self-test, but then is no longer recognized as bootable by the laptop.

Use the installed Windows to download and burn Xubuntu 7.10 live CD. At the same time, installed Quickbooks and Openoffice, and de-bloated the tiny (10GB) Win partition. Even used the 'compression' feature to squeeze 2.5 GB free space.

Install Xubuntu 7.10 - Success! Lots of adding and updating, but it's done. The laptop is usable again.

Quickbooks Repair on the store PC

March 3, 2008: I located the store's .qbw file on my laptop (the Linux partition is still fine), moved it to a stick, and took it to the store. Goal: Install the latest company data on the store PC, since I had been updating at home.

QuickBooks has encountered a problem and needs to close. Microsoft Visual C++ Runtime Library Runtime Error! Program: C:\PROGRAM FILES\INTUIT\QUICKBOOKS\QBW32.EXE

Okay, go to Add/Remove Control Panel and repair it...

The following error occurs when trying to register the QuickBooks items: Internal Error 2908 {7D4B5591-4C80-42BB-B0E5-F2C0CEE02C1A}

Argh. Turns out I had weeded unused programs last week. I hate bloat. I had removed Microsoft .NET Framework 2.0. Who knew it was installed by Quickbooks?

Put in the Quickbooks CD. It recognized that the correct component was missing, and installed it. Ejected the CD and put it away.

Went back to Add/Remove and the Repair option. It chugged a while (20-30 minutes) and reported success. Tried it with the moved company file....success! We're back in business.

I Hosed My Windows Partition!

March 2, 2008: I made a severe mistake in Wine - I set the virtual C: Drive on Wine to match the REAL Windows C: Drive (/media/sda1). BIG MISTAKE. I installed a program, and of course Wine's virtual registry probably overwrote the real Win registry. So now Windows refuses to boot at all...even into safe mode.

Happily, the Ubuntu partition is just fine. I have a huge drive to backup to. My linux box can read/write to NTFS, and I have plenty of time to backup. So I won't lose any data.

But it sucks. Maybe I'll wait until April (Ubuntu 8.04), and then fix it. Maybe not:

  1. List the Windows apps and Ubuntu Packages, registrations, usernames, and passwords
  2. Backup the data from the Ubuntu /home and Windows partitions.
  3. Download and burn an Ubuntu 8.04 disc. [7.10 alternate downloaded]
  4. Use the laptop's System Restore disk to reinstall Windows.
  5. Update Windows online.
  6. Reinstall my windows apps.
  7. Use the Ubuntu disk to partition the drive and intall Ubuntu and the Grub bootloader.
  8. Reinstall my Ubuntu apps (much easier than the Win apps).
  9. Rebuild mail, browser shortcuts, and more.

Scripting and CURL

March 2, 2008: Learning to use curl, to download schedule information. Using a tutorial to guide me through the minutae of faking forms and another to guide through scripting it into BASH.

Milwaukee Without A Car timetable project

March 2, 2008: Beginning a major new project today. I want to bring intercity timetable information to my website, Milwaukee Without A Car.

The goal is a set of pages, purely for my enjoyment:

  1. A news RSS feed and news.html page of local transit and company news, service advisories, media stories, and press releases.
  2. European-style yellow & white Arrival and Departure tables for the Milwaukee Intermodal Terminal.
  3. A table of the companies, when their timetables last changed, and when they next change.

I want to develop a set of scripts that run on my Linux laptop to: Regularly download the information from the various companies (Python), organize it (XML), output it to several HTML pages (Python, XSL), and upload it to the website (cURL)...all automatically (cron).

March 2, 2008: Ambitious, considering how awful I am at scripting, and how bad most of the timetables and websites are. In the end, I should be a *lot* better at curl and awk!

March 13, 2008: There's no good way to extract the data from PDF timetables online.

Next step is to hand-collect some sample data, hand-create some XML from the data, nail down the XML format, and write the XSL to convert it to the three HTML table formats I want: Yellow sheet (departures), White sheet (arrivals), and timetable format.

The final HTML on the website does not need to be dynamic - the data changes only every two weeks or month.

March 28, 2008: My utterly brilliant brother-in-law, Arno has had a big infulence on the plan.

May 18, 2008: Success! The RSS Feed is up. It's only the first few companies, but they system works!

July 2, 2008: The feed had a fatal bug on June 11, but I rewrote it and resumed today. It's cleaner, suceptible to fewer fatal bugs, easier to trace breake, and includes more companies.

March 3, 2009 The news feeds for ground transport and airlines are solid. Haven't started the others yet. Parsing (scraping) web pages turns out to be a big pain, especially when some designer changes the format - but parsing RSS feeds is much easier. Haven't figured out yet how to scrape timetable data, but I still want to.

Tarnation, Chapter 6

March 2, 2008: I need to try restarting the Mac to retest Skype.

Ubuntu Brainstorm, Chapter 3

March 2, 2008: Installed wine-doors a package manager for Wine. Nothing useful in it, but worth keeping an eye on.

More fun with Ubuntu Brainstorm

March 1, 2008: Learned a lot at Ubuntu Brainstorm...mostly that people are not much smarter than cockroaches. Great fun!

I tried two packages based on comments in Brainstorm:

mc (midnight commander) is a command-line window manager

gnome-do is an adaptive workflow application launcher.

Both are pretty cool to know about, but they're just not something I'll be using. I'm in more of a learn-to-script-it mood.

Back to Tracker?

March 1, 2008: Tracker is a powerful indexing package - that was too buggy in October 07 and killed the system. It hogged the CPU and filled(!) the /home directory disk with a corrupt index file. Bad. Bad. Bad. Uninstalled it back in October after a day of frustration and high dudgeon.

In April 2008, Ubuntu 8.04 will include the 'fixed' and improved Tracker. It is cool, and promising. The bug reports seem promising - all the big problems fixes last fall....

Okay, I'll try it again in April. I'm a fool for love. But not sooner - the current version on Synaptic is the 7.10 buggy one.

Update: I never got around to Tracker - I installed Xubuntu instead, which has no great need for it.

apt-rdepends

March 1, 2008: Cool! Package apt-rdepends installs a shell command (also apt-rdepends) that gives a clean list of all dependencies. Reverse searching is also possible. I needed something like this in Iraq, when I was upgrading by hand due to a lack of internet connection for my laptop.

Interestingly, Synaptic has grown a script-maker for offline package changes. It generates a script that you use on a connected machine to download the right packages. I could have used that, too...

Now they are merely of academic interest - unless I go back.

Instead of using the apt-rdepends package, you can get the same result using the apt-cache rdepends packagename command.

Ubuntu Brainstorm is addictive

February 29, 2008: Trying to limit myself to 30 minutes per session at Ubuntu Brainstorm. Reading the silly ideas (and the few really good ones), and responding to many. So many are already implemented! Many people want to bloat up the distro, others to pare it down to the bone. Very geeky fun.

Update April 10, 2009: Today I became an idea reviewer for Brainstorm.

Tarnation Chapter 5: Gizmo and Skype

February 29, 2008: Friday afternoons I go to the store with the kids. Can't get much done, but I try.

Test call from store Windows PC to the store ATA next to it using Gizmo Project. Call connected. Inconclusive - no mic on the PC, and it's inbound calls that are the problem.

Test all from the ATA to an 800 number (free). Successful two-way conversation. Since it's inbound calls that are the problem, it's likely a router setting issue. I didn't get time to poke at the router (darn).

Test calls from PC to Mac and Mac to PC using Skype. Mac never acknowledged inbound calls, but PC did. Mac never updated PC's online status. Successful connection from Mac to PC only after restarting the Mac's Skype. Couldn't test voice (no mic on PC), but both were behind the same router. Result: Mac's Skype seems to have a problem. Need to check Skype support for Mac.

Tarnation Chapter 4: Gizmo and Skype

February 28, 2008: Two failed test calls.

Test call from laptop to store ATA using Gizmo Project. Phone rang, picked up. She couldn't hear be, but I could hear her beautifully. Likely firewall issue at store.

Test call from laptop to her Mac using Skype. Her Skype never rang. She didn't see my status as 'online'. Hmmm....

Ubuntu Bug #158706 Verified

February 28, 2008: Success! A couple months ago I submitted a bug. Today they confirmed it. (I made a good report)

When installing the package netatalk 2.0.3-6ubuntu1 on Ubuntu 7.10, the following errors occur: 

hostname: Unknown host
invoke-rc.d: initscript netatalk, action "stop" failed.
dpkg: warning - old pre-removal script returned error exit status 1
dpkg - trying script from the new package instead ...
hostname: Unknown host
invoke-rc.d: initscript netatalk, action "stop" failed.
dpkg: error processing /var/cache/apt/archives/netatalk_2.0.3-6ubuntu1_i386.deb (--unpack):
 subprocess new pre-removal script returned error exit status 1
hostname: Unknown host
invoke-rc.d: initscript netatalk, action "start" failed.
dpkg: error while cleaning up:
 subprocess post-installation script returned error exit status 1
Errors were encountered while processing:
 /var/cache/apt/archives/netatalk_2.0.3-6ubuntu1_i386.deb
E: Sub-process /usr/bin/dpkg returned an error code (1)
A package failed to install. Trying to recover:
dpkg: error processing netatalk (--configure):
 Package is in a very bad inconsistent state - you should
 reinstall it before attempting configuration.
Errors were encountered while processing:
 netatalk
Press return to continue.

How to fix it:

  1. edit /etc/default/netatalk
    FROM: ATALK_NAME=`/bin/hostname --short`
    TO: ATALK_NAME=`/bin/hostname`
  2. edit /etc/init.d/netatalk
    FROM: ATALK_NAME=`/bin/hostname --short`
    TO: ATALK_NAME=`/bin/hostname`

Ubuntu Startup: services-admin prompts for password

February 27, 2008: Every time I login to my Ubuntu 7.10 laptop, I get the 'happy login' music and the screen begins to populate with my desktop.

Then it vanishes (before I can click on anything), and services-admin prompts me for my password. Whether I enter the password or simply cancel, the system thinks for a moment, then brings up the network and brings back the desktop.

The only difference I've noticed is that CUPS seems to be active only when I've entered my password, and isn't if I cancelled instead. If all it's starting is CUPS, then I don't care if services-admin starts or not; I can start it manually. I'm annoyed by services-admin prompting me.

Result: Nothing in bugs, nothing in forums. Left a post at www.ubuntu-forums.com to see if anyone can help.

Update April 2009: Upgrading to 8.04 eliminated the problem. The ubuntuforums thread was never answered.

Tarnation Chapter 3 - Ekiga and Gizmo Project

February 27, 2008:

  1. Skype: Successful echo test. But Skype on Mac at the store errors out "unknown error" when I try to Skype it. When I was there, I restarted Skype, and it seemed to work normally.
  2. Gizmo: Successful echo test after playing with the router firewall. I turned it off and then on again. But the garble is gone, and calls complete. Next time I have problems, just cycle the router? But calling the ATA at the store failed - after 'Hello?' the call dropped.
  3. Ekiga: Turning the firewall off made no difference in an Ekiga echo test. Still garbled outbound.

Next step: Check the router at the store.

Linux Startup: intel_rng: FWH not detected

February 27, 2008: Success! Every time I boot Linux, one of the startup messages is intel_rng: FWH not detected. Today I fixed it!

The item shows up in /var/log/dmesg as Feb 27 06:28:03 ian's_computer kernel: [ 15.128000] intel_rng: FWH not detected

It turns out to have a simple explanation, and suggested solution is to 'turn off' the kernel module.

I used sudo gedit /etc/modprobe.d/blacklist to edit the blacklist file as root. I added the following at the bottom of the file: 

# prevents minor boot error 'intel_rng: FWH not detected'
blacklist intel_rng

Reboot, and success! The error message is gone.

Tarnation Chapter 2 - Ekiga and Gizmo Project

February 26, 2008: Ekiga Bad - I double checked the Port Triggering on the home router - it's fine. But Ekiga still garbles and echoes my voice. Inbound audio is great. Not usable. Ekiga mailing list had a fellow with a similar problem in Oct 07 using Win XP, probably a driver issue. Might be here, too - audio, though it works, has always been a bit iffy on the Linux Toshiba.

Ekiga Good - Make calls from the command line using ekiga -c recipient@proxy. For example, the Gizmo echo service is ekiga -c sip:17474743246@proxy01.sipphone.com or ekiga -c sip:echo@proxy01.sipphone.com

Gizmo Bad - Several test calls, all disconnected instantly. Not usable. Opened SIPPhone ticket #YBE-375340.

Skype good - Test call to my awesome Brother-In-Law, Barrett, in Texas. He heard me clearly. Didn't have his mic so he couldn't talk back, but audio in has never been a problem...

Tarnation - Ekiga and Gizmo Project

February 25, 2008: Failure! We have an Analog Telephone Adapter (ATA) hooked up to the store phone. That way, it can receive both PSTN and Internet calls. The store SIP URL is sip:17476010543@proxy01.sipphone.com. Er, that's the store, and I'm usually not there, so please don't call it to chat...you'll just annoy my wife.

In the past, I've used this ATA (attached to Gizmo Project account 'kiwkak5678') to dial out from home and the store, so I know it works.

Yesterday I phoned it from home using Ekiga 2.0.11 for Linux (Ubuntu 7.10 on Toshiba A105-S4054) on my laptop (connection test only - no mic) - Success! The phone rang four times, the answering machine picked up, and I heard the answering machine message.

Today I tried a couple Ekiga tests on my laptop:

  1. Ekiga voice tests
    • Gizmo Project's echo service sip:echo@proxy01.sipphone.com:5060. Succesful connection using my Gizmo account on Ekiga. Too much jittery echo, and too much recurring echo (echo, echo, echo, echo).
    • Gizmo Project's TellMe service at sip:tellme@proxy01.sipphone.com. Successful connection from both Ekiga and Gizmo accounts on Ekiga. Voice response system came through clearly, but it couldn't hear my voice commands properly.
    • Ekiga'e echo service at sip:500@ekiga.net. Created an Ekiga account, and successfully connected. From the Gizmo account, failed with "Security Check Failed". Same sound quality result as the Gizmo echo test.
  2. Ekiga video test. Success! Ekiga detected my camera using the standard V4L driver, and put a video picture in the correct Ekiga window.
  3. Gizmo Project 3.1.0.77 for Linux voice tests
    • Gizmo Project's echo service. Connected, but too much recurring echo (echo, echo, echo, echo).
    • Store phone via SIP/ATA. Three test calls rang the other end, but neither side could hear the other. Caller ID picked up the SIP call as 'Out Of Area'.
    • Gizmo Project's TellMe service at. Successful connection. Voice response system came through worse than Ekiga, but understandable. It could correctly hear an interpret most of my voice commands. It tried to interpret other signals (background? echo or garble?) at times.
  4. Tried a voice test of Skype. Success! Good sound quality both ways on their echo service. Tried connecting to my wife's Mac. Success!

Support forum results: Ekiga has nothing at all about the echo echo echo echo. Gizmo claims most problems are really NAT/Firewall issues (never theirs).

Next Steps:

Created Ian's Tech Blog

February 25, 2008: Success! Today I created and tested this page, plus the feedback webform.

Success stories are so rare....

Converting a .tiff to a .pdf in Ubuntu

February 25, 2008: For some reason, my CUPS under Ubuntu 7.10 refuses to print-to-pdf. I've tried everything I know (not much)...and probably broke a lot of stuff along the way.

But I found an easy way to do it from the command line.

  1. Scan the item to a .tiff file in a known location (for example, save it to the Desktop)
  2. Open a terminal and navigate to that location (for example cd Desktop)
  3. convert -density 300 -units PixelsPerInch scanned_file.tiff converted_file.ps
  4. ps2pdf13 -sPAPERSIZE=letter converted_file.ps final_output_file.pdf

  5. rm scanned_file.tiff converted_file.ps
© 2008-2009 Ian Weisser -