James Gardner

One niggle I’ve had with Gnome for a while is that when I click a link in Icedove (Debian’s name for the Thunderbird mail program) it loads the page in Epiphany, the Gnome browser, instead of Iceweasel (Debian’s version of Firefox). Also I use gVim as my text editor but text files load in gEdit by default.

I initially thought the “Preferred Applications” program would be the place to make the change, then I thought there might be a file associations program to handle my preferences but there didn’t appear to be. The reason for my problem was that I didn’t understand how Debian handles preferred applications.

Program associations are handled via the update-alternatives program which must be run as root. In Debian, programs register themselves as alternatives to perform the same actions. Debain will choose an alternative by default but if you want to change it you can use the command line program.

For example, to make Iceweasel my default browser I run this command:

james@dirac:~$ sudo update-alternatives --config x-www-browser

Here is the output:

There are 2 alternatives which provide `x-www-browser'.

Selection Alternative
———————————————–
*+ 1 /usr/bin/epiphany
2 /usr/bin/iceweasel

Press enter to keep the default[*], or type selection number: 2
Using `/usr/bin/iceweasel’ to provide `x-www-browser’.

Incidentally, the current choice is marked with a ’*’ and the choice with the highest priority with a ’+’.

Iceweasel is now the default alternative to handle any browser related actions throughout Gnome and indeed Debain. Handy eh?

Rather pleased with this I decided to change some other associations but needed to know what group name to use. You can see a list of all the alternatives with this command but you should never manually edit files in this directory. Use the update-alternatives program to do that.

ls -l /etc/alternatives/

Part of the output looks like this:

lrwxrwxrwx 1 root root 18 2007-07-21 11:57 x-www-browser -> /usr/bin/iceweasel
lrwxrwxrwx 1 root root 34 2007-07-21 11:57 x-www-browser.1.gz -> /usr/share/man /man1/iceweasel.1.gz

As you can see x-www-browser now points to /usr/bin/iceweasel but what is interesting is that the man page has also been updated as a dependency so that you don’t get a man page for a different browser when trying to access the help.

Well, that’s how to choose preferred applications, now for changing which applications are associated with which file types. I haven’t worked out how to do this on the command line so if anyone knows I’d like to hear but you can do it via the Gnome GUI.

Right click on a text file, choose Properties then click the Open With tab. The tab presents a list of alternatives for the particular file type. Simply choose the application you prefer and click OK. Those file types will now be associated with the application you’ve chosen.

If you want to scan a drive for viruses you need ClamAV:

sudo apt-get install clamav

Configure your mirror by updating /etc/clamav/freshclam.conf like this:

# DatabaseMirror db.local.clamav.net
DatabaseMirror db.au.clamav.net

(I’ve chosen au instead of uk because the UK mirror seems to be down)

Then update the virus definitions:

sudo freshclam

Recursively scan a directory showing only infected files:

clamscan -ri /media/usbdisk/Sync

So, I reinstalled Debian today and everything looked fine but it wasn’t. I couldn’t access my removable drive and the sound wasn’t working. All very strange because they worked perfectly last time I installed it.

Eventually I tracked down the problem. When I created the user account james I didn’t add any groups to it. Debian uses groups to grant permission to various essential things. To fix all my problems here are the commands I ran as root:

adduser james plugdev
adduser james src
adduser james dialout
adduser james adm
adduser james floppy
adduser james disk
adduser james video
adduser james cdrom
adduser james audio

This sorted everything out and made all the experimenting I’d been doing with alsa and my fstab rather pointless!

I installed the release version of Debian Etch today. Here’s what I did.

Get the CD

First download CD1 of the set and run
md5sum debian-40r0-i386-CD-1.iso to check that the md5 checksum is correct. It should be b1ee06232da140b0e4ac675c034bb95d. If it is OK then burn the image to a CD. Probably best to choose a write speed below the maximum the CD drive can do to minimise the risk of an error occurring during the burn which might cause problems later.

Once the CD is burned, put it into the drive and reboot. The CD will boot and at the first screen press ENTER to boot the installer. If you see a message similar to “Kernel panic - not syncing: VFS: Unable to mount root fs on unknown-block(104,7)” is likely there was an error burning the CD. Check the ISO image and burn it again at a slower speed.

Install the Base System

I followed the screens and choose the following options:

English
United Kingdom
British English
en_GB.UTF-8
Keyboard: British English
Hostname: dirac
Domain name: 3aims.com

I then let Debian setup the paritioning choosing to use an encrypted LVM volume and one partition for all data. When setting up the encrypted volume Debain takes quite a long time erasing previous data which it doesn’t seem to do for other parition types. After it is setup you need to choose an encryption passphrase, the longer the better. I chose the hostname as the volume group for LVM. After confirming you are happy, the installer finishes setting up the partitions. It installs the base system and I chose the default kernel.

Choose a root password
Setup a user

Then I chose ftp.uk.debian.org as the apt mirror and I don’t have a proxy server. Configuring apt crashed the first time but worked the second.

After that I opted out of the popularity contest and installed Desktop Environment, Laptop and Standard System from the options. I always find it is useful to install a very standard setup even if that isn’t what you want eventually. The reason is that most documentation will assume your setup is similar to everyone else’s and if it isn’t you are likely to find you are missing parts of the system other people will expect you to have. Following this approach might use slightly more disk space in the short term but it will save a lot of effort in the long term.

Anyway, after a restart there was the system. Here are the changes I made next…

Configure the New Desktop

Install some extra software:

apt-get install vim-gnome vim-python nautilus-open-terminal python-dev python-imaging vim-scripts vim-runtime subversion unzip openssh-server thunderbird tomboy apache2-utils clamav msttcorefonts rsync build-essential

Remove some I won’t use:

apt-get remove --purge epiphany-browser nano bc dc

Remove CD from /etc/apt/sources.list otherwise apt will sometimes try to install software from it and that can be a pain if you don’t always have the CD with you. It should look like this:

deb-src http://ftp.uk.debian.org/debian/ etch main
deb http://security.debian.org/ etch/updates main contrib
deb-src http://security.debian.org/ etch/updates main contrib

Set the editor to vim:

gvim ~/.bashrc

Then uncomment the line starting PS1 and set the editor to be vim by adding this:

export EDITOR=vim

Sound doesn’t work out of the box. Run this:

dirac:/home/james# cat /proc/asound/cards
0 [I82801DBICH4 ]: ICH4 - Intel 82801DB-ICH4
Intel 82801DB-ICH4 with AD1981B at 0xd0100c00, irq 11
1 [Modem ]: ICH-MODEM - Intel 82801DB-ICH4 Modem
Intel 82801DB-ICH4 Modem at 0×2400, irq 11

Then install and configure alsa:

dirac:/home/james# apt-get install alsa-utils alsa-oss alsa-base

Actually this doesn’t seem to help although sound plays fine as root. Update: None of this sound configuration should have been necessary. The solution is documented here.

Gnome Setup

Web Browser

From the menu I selected Desktop->Preferences->Preferred Applications and chose Iceweasel instead of Ephiphany. I then removed the Ephinany icon from the top panel and added Iceweasel and Terminal ones by right-clicking and selecting Add to Panel.

Nautilis

Edit->Preferences->Behaviour
Tick Always Open in Browser Windows
Tick Always use text-entry location bar

Samba Mounts

Choose Places->Connect to server from the menus then select a Windows Share and in Location enter smb://fermi/james/file.

Thunderbird:

copy from smb://james@fermi/james/files/Backup/Thunderbird/Profiles to ./mozilla-thunderbird
Start thunderbird then rename the profile to whatever the current one is!

Edit profiles.ini to look like this:

[General]
StartWithLastProfile=1

[Profile0]
Name=default
IsRelative=1
Path=j0ves973.default

Flash Support

Get Adobe 9. Then copy the file to the ~/.mozilla/plugin
http://www.mail-archive.com/debian-user@lists.debian.org/msg466736.html

Shortcut Icons

Add the gnome terminal and gvim icons to the panel.

Fat32 Support

sudo apt-get install dosfstools

VMWare Setup

Get the server software from the VMWare site and generate a serial number from the same page then run the following commands, accepting the defaults most of the time. A new icon will appear in the Applications->System Tools menu called VMWare Server Console.

apt-get update
apt-get install linux-headers-`uname -r` libx11-6 libx11-dev x-window-system-core x-window-system xspecs libxtst6 psmisc build-essential
tar xvfz VMware-server-*.tar.gz
cd vmware-server-distrib
./vmware-install.pl
cd /usr/lib/vmware/lib/libgcc_s.so.1/
mv libgcc_s.so.1 libgcc_s.so.1.old
ln -sf /lib/libgcc_s.so.1
cd /usr/lib/vmware/lib/libpng12.so.0
mv libpng12.so.0 libpng12.so.0.old
ln -sf /usr/lib/libpng12.so.0

The last statements are to fix a bug where you can’t click the Browse button. Then run this to load vmware:

vmware &

Other useful notes

Setup a new user and home directory:

useradd -c "Joe Blogs" -m jblogs

Setup Pylons

mkdir ~/Desktop/SVN/pylonshq.com
cd ~/Desktop/SVN/pylonshq.com
svn co http://pylonshq.com/svn/Pylons

Install virtual Python

apt-get install python python-dev subversion
wget http://peak.telecommunity.com/dist/virtual-python.py
python virtual-python.py
wget http://peak.telecommunity.com/dist/ez_setup.py
~/bin/python ez_setup.py

Install AuthKit:

~/bin/easy_install "AuthKit==dev"

Fixing the uswsusp problem which occurred in the release candidate of the installer but which doesn’t affect the final version.

sudo dpkg-reconfigure uswsusp
update-initramfs: Generating /boot/initrd.img-2.6.18-4-686

Then I configure sudo, setup wireless networking and configure vim and add a printer.

I’ve managed to avoid a Windows Virus the entire time I’ve been using Windows but on Friday I finally made a mistake. I was copying files from my brother’s digital camera and inside the usual fuji_100 directory was another fuji_100 directory which I double clicked on. In fact it as an .exe file with an icon to look like a folder and the reason I didn’t spot it was that Windows was hiding the extension (even though I’ve told it not to).

I knew it was a virus the second the folder didn’t open but by then it was too late. The virus checker didn’t pick it up and it had already infected key Windows executables for logging in and shutting down. Running msconfig just turned off the computer so there was no hope of a simple repair. The really irritating thing was that this virus is probably fairly harmless since it must have been on my brother’s computer for at least 4 months (the last time he used his camera) but because I use my computer on other people’s netowrks I had no choice but to wipe the entire hard disk and restore Windows. To make matters worse, the restore disk also required me to wipe my Debian installation!

So, I’ve now reinstalled Windows, created a VMWare image from it, wiped the drive again and installed Debian over the top which will be my only operating system I use from now on with occasional use of a VMWare Windows image when it is really needed. Wish me luck with the switch!

I spent most of days 3 and 4 catching up with people I knew and exploring Vilnius itself. I spent some more time discussing the request response integration with Pylons and Django and went to the Death to Zope Instances sprint where I was delighted to see other developers (including the SchoolTool people) also implementing variations on the Zope on Paste theme and keen to agree a standard solution.

The lightning talks were as interesting as ever with my particular interest being taken by PyDoctor (a source code documentation tool) and a really impressive 3D modelling engine. The keynote on MySQL was interesting (My should really be pronounced “me” because it is the name of the daughter of the other co-founder of the project).

Python + .NET = IronPython and Silverlight and Python

The morning started with two talks about Microsoft technologies, the first was Python for the .NET platform and the second was Silverlight (a technology I’m very interested in) which is kind of like Microsoft’s answer to Flash. Both talks were very disappointing because I didn’t learn anything about either technology which I hadn’t already picked up after a brief read of the project sites and a quick play with the tools. The talks seemed to be more about demonstrating the Visual Studio IDE then getting down to the nitty gritty of what the technologies can be used for or how they work. Shame, because I’m really keen on both technologies.

The Essentials of Stackless Python

I have to admit this went well over my head. So much so that I spent most of the time reading the stackless website and after a while I did more or less understand what what was going on. After the talk was a quick break so I decided to install stackless and have a go at writing a WSGI server for Pylons using it which I eventually got working. Although the stackless server was a lot faster than the paste httpserver with default thread options for consequtive requests, it was a lot worse for concurrent requests so it isn’t a clear-cut thing as to whether a stackless apprach is always better. Either that or my implementation contained bugs!

There wasn’t anything I was particularly interested in for the next session so inspired by Simon’s mention of AuthKit yesterday I pressed on and fixed a few bugs instead and spent some time talking to Graham Stratton and Jeroen Vloothius.

If you can’t beat them … Pythonic explorations of Microsoft Sharepoint Portal Services

This was a great talk which basically reminded me why I should avoid proprietory technologies at all costs. Although Sharepoint has a WebDAV interface it can’t be used to access most documents. As a workaround you can store documents in lists or install IronPython on the Sharepoint server and write code to interface between .NET and your own custom web services API. Neithter seem a great solution but I really appreciated the information and tips in the talk, very useful!

Again, there was nothing too interesting for me in the afternoon so I decided to explore Buildout, which is a tool the Zope community use to fulfil a similar role to the likes of a virtual Python install or a working env only much more sophisticated. I’m currently writing a buildout tutorial for Pylons users which I hope to release fairly soon because it is a great product. During the lightning talks I also had a discussion with Philipp about the Zope and Paste integration. Guido talked about Python 3000 again but I felt I’d already heard most of what he had to say. Personally I’m not too worried about compatibility issues - I will recode anything which needs changing.

In the evening was the conference dinner and because I spent a bit long chatting with Simon Willison and Nat Downe over the wine we couldn’t get a table! This turned out to be quite handy because we sat slightly further away from everyone else and had a chance to discuss some further collaboration including the integration of the Pylons request and response Simon mentioned in his keynote earlier. A few others joined us and before long we were debating all sorts of issues from use of the work “geek” to the implications of religion. Great fun and a long night out in a Vilnius bar followed.

Well, here I am in Vilnius in Lithuania at EuroPython 2007. I’ve already bumped into a load of people I already know and have been pleased to see Pylons used in three of the talks already!

Here are some notes on the talks I’ve been to:

Extending Python with EasyExtend

I was actually planning on going to Object databases for Python but I ended up in the wrong room but was pleased I did. EasyExtend is a Python package which allows you to create new language syntax. This can be used to implement Python 2.5 language features in Python 2.4 or create a new programming language completely. I’m particularly interested in it because it could be used to restrict access to certain Python features so could potentially be used to provide scripting for users using a cut-down and safe version of Python. This might be handy in say a web-based spreadsheet application where a custom language could be implemented for the formulae.

Seamless object persistence with ZODB

I hear a lot about ZODB because Zope is used extensively on other projects at the Institute of Child Health where I work but I’ve never actually used it myself. This talk was a good introduction. Basically you have to ensure you use special persistant classes for lists and the like in order for ZODB to keep track of changes you make to them. As long as all your objects are derived from the special persistent classes all works well. One interesting method you can use on transactions is doomed() which will prevent any further commits but will still allow you to access all the data as it currently stands for use in debugging code.

The Storm Object-Relational Mapper

This talk outlined Storm, a new object relational mapper created by Canonical which hadn’t been widely announced to the community before. Key features are that it works with normal Python classes, is very simple and has more lines of unit tests than it does actual code so is likely to be very stable. It has also been used in production by Canonical for over a year I think this would be my object relational mapper of choice if I were to use an ORM. It has similar functionality to SQLObject or Elixir, is much simpler and more stable than SQLAlchemy and is well supported by Canonical. The speaker also claimed it was very very simple to implement new drivers and had a very nice SQL query layer. Definitely worth a look if you are interested in ORMs.

Case study of a Pylons project

In this session Max Ischenko discussed his experiences of converting a Ukranian developer’s site written in PHP to use Pylons. Overall Max seemed very pleased with the how Pylons performed although he found he had to do quite a lot of the groundwork himself. View the slides if you are interested in his thoughts.

KSS, Ajax development with style

KSS effectively lets designers and HTML writers handle JavaScript functionality in the same way they would use CSS and provides a clean interface to allow developers to provide the functionality those designers can use. This was one of the most interesting projects I saw at EuroPython and I spent a couple of hours discussing it afterwards with Jeroen Vloothius, a Plone developer who works at Pareto.

KSS lets you apply JavaScript to elements via CSS-style stylesheets called KSS files. This means you abstract away all the JavaScript used by your appliaction into simple rules that even a non-technical user can apply to their HTML in the same way they would apply CSS. It also means that you can use any JavaScript library you like to implement the KSS which means if you swap JavaScript libraries later you have a clean interface to implement.

The second important feature if that KSS lets you determine the actions to perform after an AJAX request on the server side via an HTML document returned to the browser rather than needing to do it client side. This is tremendously useful because after all it is the developers who need control over what happens after an AJAX request, not the designers. If you’ve ever come across a problem where someone has used some nice bit of JavaScript to update a status but failed to notice that the status is also displayed on other parts of the page which need updating too then you’ll understand why this is useful.

KSS is already used in Plone and Jeroen applied it to the Pylons QuickWiki tutorial code to demonstrate that it was cross-framework too. While demoing it one of the Python developers in the audience asked how he managed to get it running so fast (it was just running the standard Pylons server though) so it just goes to show that if you can use AJAX effectively it can really impress people and KSS represents a great tool for effective AJAX use.

One feature which would be nice are the ability to update the URL on each click (a la Dojo) to provide an AJAX history in such a way that all the links can use an AJAX replace but that if the user ever hit refresh on the current URL they would see exactly the same page. I’m having a think about how this paradigm could be implemented in Pylons.

Using FormEncode for web data validation

This talk was given by a friend of mine, Graham Stratton and gave a good insight into how to use FormEncode. The key points were that FormEncode itself, despite its name, is simply a tool for converting data from a particular format to Python and back again. In the context of web forms, FormEncode converts form data to Python objects and can then covert Python objects back from Python to web form values again. How the form data is then displayed is up to the developer but FormEncode also comes with a tool called htmlfill which can parse an HTML fragment, extract field names and generate a new HTML fragment with all the values and errors set, ready to redisplay. There is no requirement to use htmlfill, but it is a useful approach for populating fields in a very generic way.

Zope on a Paste

This talk by Philipp von Weitershausen really interested me. What he has been trying to do is take Zope 3’s WSGI support and see how far he can integrate Zope 3 with Paste in an effort to end relience on Zope 3 instances and instead encourage developers to use Zope 3 more like a library where they import, setup and use only those bits they need. Philipp produced some paster create templates to create a sample project which can then be served by paster serve. This also meant that he could add all the Pylons middleware to Zope 3 and at one point he demonstrated a TurboGears app and a Zope app mounted using Paste to a WSGI app and then served together on the same server. I hadn’t realised just how far Zope had come in terms of its refactoring and as a result much of Zope 3 may very well become accessible to projects such as Pylons in the very near future, particularly with this sort of work going on. If any members of the Zope community want WSGI, Paste or Pylons advice related to this sort of opening up, please get in touch, I’m keen to help.

Other Talks

In the afternoon I saw py.test: towards interactive, distributed and rapid testing and unittest is Broken. There seem to be lots of testing tools available at the moment and nosetest and py.test in particular seem very similar. My instinct is to wait a bit longer to see how the various alternatives settle down.

Keynote

After the talks Simon Willison delivered his keynote on OpenID and as well as pointing out my AuthKit package as a Python OpenID implementation he announced that he would be working with the Pylons community to help unify the Django request and response with the Pylons one. If things go well we might have a chance to sprint on this on Wednesday or Thursday. I’ve heard Simon speak about OpenID quite a few times now but every time his arguments get more refined and his vision for decentralised social networks based on OpenID and web services becomes more focussed. Simon encouraged developers to work towards an open alternative to the Facebook platform. As always, I agree with virtually everything he said.

After the keynote and a quick chat to Nat and Simon I headed out into Vilnius with Graham to explore the city based on some of Natalie’s recommendations and Graham and I eventually ended up in the traditional-style bar/restaurant which Guido had chosen for the evening. It was made mainly of wood and had a tree growing inside. Beer was on Google and we met two developers who had travelled all the way from South Africa.