Xen on Thinkpad R50e Debian 4.0 Etch

Posted:	2007-04-30 19:47
Tags:	Debian

I'm a great fan of Xen but the installation is still a little tricky, even using the precompiled binaries.

First of all you need to find out if your CPU supports PAE. You can do this with the following command:

grep pae /proc/cpuinfo

If the output contains pae then you know your CPU has PAE support, if there isn't any output it doesn't. The software you need to install depends on whether you have PAE support. If it doesn't you run this command to install Xen:

sudo apt-get install linux-image-2.6.18-4-xen-686 xen-utils xen-hypervisor-3.0.3-1-i386 libc6-xen

My CPU is a Celeron which does support PAE so I installed these:

sudo apt-get install xen-linux-system-2.6.18-4-xen-686 libc6-xen

Depending on when you read this the correct packages to install may have slightly different version numbers so you can use sudo apt-cache search xen-linux-system to see the versions available.

Also you don't have to install libc6-xen but I've read it saves you having to manually disable threadlocal storage.

What you do at this stage depends on the filesystem on /boot. If you have an ext2 filesystem you should be able to reboot your machine and find that the Xen kernel boots successfully into your usual login screen. If you don't have an ext2 filesystem or you have a complicated disk setup you might find that when you try to boot you get an error similar to this:

Cannot open root device "hda2" or unknown-block(0,0)

To fix this you must create an initial RAMdisk containing all the kernel's modules. Initrd images created in the usual Debian mkinitrd manner won't work with the Xen kernel. Instead, you must use mkinitramfs.

First find out the modules available:

ls /lib/modules

and then install initramfs-tools if you don't already have it:

sudo apt-get install initramfs-tools

Then create a new RAMdisk using the correct version of the modules if the output file doesn't already exist:

sudo mkinitramfs -o /boot/initrd.img-2.6.18-4-xen-686 2.6.18-4-xen-686

At this stage you need to edit your Grub menu in /boot/grub/menu.lst so that when the Xen kernel boots it uses your new initrd.img. Find the Xen options and add the following line if it isn't already there:

module    /boot/initrd.img-2.6.18-4-xen-686

to the end of that section so it looks something like this: (yours may be slightly different depending on your boot partition and the versions of Xen you are using):

title           Xen 3.0.3 / XenLinux 2.6.18-4
root            (hd0,1)
kernel          /boot/xen-3.0.3-1-i386-pae.gz
module          /boot/vmlinuz-2.6.18-4-xen-686 root=/dev/hda2 ro console=tty0
module          /boot/initrd.img-2.6.18-4-xen-686

Now you should be able to reboot into Debian with your Xen kernel!

First of all configure the networking to use NAT rather than bridging or routing. This effectively means the DomX guests run on their own network and use the DomU as a gateway. This is very handy because they can then ping each other and the DomU so you can use them to test a network of machines. The disadvantage is that they aren't publically accessible on the main network, but that suits me for the moment.

Make a backup of your Xen configuration:

sudo cp /etc/xen/xend-config.sxp /etc/xen/xend-config.sxp.bak

Edit your /etc/xen/xend-config.sxp file to look like this after making a back up:

# -*- sh -*-
(dom0-min-mem 196)
(network-script    network-nat)
(vif-script        vif-nat)</code>

The above is all you need and if you add extra entries you might confuse things! At this point you will need to restart xend for the changes to take effect.

sudo /etc/init.d/xend restart

If you already have some Xen images and config files you shoud now be able to run them, otherwise lets create one from scratch. You do this with the xen-create-image command. You'll need to install the xen-tools package to get this command:

sudo apt-get install xen-tools

We are going to create an image with a hostname rms with an IP address 10.0.0.1:

sudo mkdir /mnt/xen
sudo xen-create-image --debootstrap --dir=/mnt/xen --size=2Gb --memory=512Mb --fs=ext3 \
   --cache=yes --dist=etch --hostname=rms --ip 10.0.0.1 --netmask 255.0.0.0 \
   --gateway 10.0.0.254 --initrd=/boot/initrd.img-2.6.18-4-xen-686 \
   --kernel=/boot/vmlinuz-2.6.18-4-xen-686 --mirror=http://ftp.uk.debian.org/debian/

Output looks something like this:

General Infomation
--------------------
Hostname       :  rms
Distribution   :  etch
Fileystem Type :  ext3

Size Information
----------------
Image size     :  2Gb
Swap size      :  128Mb
Image type     :  sparse
Memory size    :  512Mb
Kernel path    :  /boot/vmlinuz-2.6.18-4-xen-686
Initrd path    :  /boot/initrd.img-2.6.18-4-xen-686

Networking Information
----------------------
IP Address 1   : 10.0.0.1
Netmask        : 255.0.0.0
Gateway        : 10.0.0.254

Creating swap image: /mnt/rms/xen/domains/rms/swap.img
Done

Creating disk image: /mnt/rms/xen/domains/rms/disk.img
Done

Creating ext3 filesystem on /mnt/rms/xen/domains/rms/disk.img
Done

Installing your system with debootstrap mirror http://mirror.ox.ac.uk/debian/
Done

Running hooks
Done

No role script specified.  Skipping

Creating Xen configuration file
Done
All done

Logfile produced at:
/var/log/xen-tools/rms.log

At this point it is worth backing up your setup so that you can use it later as a base for any future domains you set up:

cd /mnt/xen/domains/rms
tar -czpf swap.tgz swap.img
tar -czpf disk.tgz disk.img

Change the networking portion of the /etc/xen/rms.cfg file generated so that it looks like this:

#
#  Networking
#
vif  = [ 'ip=10.0.0.1' ]
dhcp="off"
hostname="rms.dev"
ip="10.0.0.1"
netmask="255.0.0.0"
gateway="10.0.0.129"</code>

It doesn't seem to matter too much what you choose the gateway to be for some reason.

Make sure Xen is running:

/etc/init.d/xend start
/etc/init.d/xendomains start

Then start it up (the -c option connects the current terminal to that of the booting Xen domain):

sudo xm create -c /etc/xen/rms.cfg

Creating a new Xen partition without needing the download again:

sudo mkdir /mnt/disk
sudo mount -o loop /mnt/xen/domains/rms/disk.img /mnt/disk

sudo xen-create-image  --dir=/mnt/xen --size=2Gb --memory=256Mb --fs=ext3 \
   --cache=yes --dist=etch --hostname=new --ip 10.0.0.3 --netmask 255.0.0.0 \
   --gateway 10.0.0.254 --initrd=/boot/initrd.img-2.6.18-4-xen-686 \
   --kernel=/boot/vmlinuz-2.6.18-4-xen-686 --copy /mnt/disk \
   --image=sparse

sudo umount /mnt/disk

The --image=sparse option I included this time builds the image in a special type of file that doesn't actually use the full amount of space on the filesystem. If you use ls you will see it is reported as 2Gb but using df will show it is really a lot smaller. When files are deleted the space isn't recovered so the real amount of space used will grow over time.

After booting the new image with:

sudo xm create -c /etc/xen/new.cfg

and signing on as root you will want to update the system though to ensure you have all the packages:

apt-get update
apt-get upgrade
apt-get dist-upgrade

You should also be able to ping both google and the IP of the guest machine (in my case 192.168.1.6 - find out with the ifconfig command):

new:~# ping google.com
PING google.com (64.233.187.99) 56(84) bytes of data.
64 bytes from google.com (64.233.187.99): icmp_seq=1 ttl=239 time=120 ms
64 bytes from google.com (64.233.187.99): icmp_seq=2 ttl=239 time=110 ms
...

new:~# ping 192.168.1.6
PING 192.168.1.6 (192.168.1.6) 56(84) bytes of data.
64 bytes from 192.168.1.6: icmp_seq=1 ttl=64 time=0.461 ms
64 bytes from 192.168.1.6: icmp_seq=2 ttl=64 time=0.173 ms</code>

You should also be able to ping the guest from DomU:

james@dirac:~$ ping 10.0.0.3
PING 10.0.0.3 (10.0.0.3) 56(84) bytes of data.
64 bytes from 10.0.0.3: icmp_seq=1 ttl=64 time=5.06 ms
64 bytes from 10.0.0.3: icmp_seq=2 ttl=64 time=0.218 ms

At this point you have everything working nicely. You can exit the guest console by pressing CTRL+].

You can list running guests with:

sudo xm list

and you'll see something like this:

Name                                      ID Mem(MiB) VCPUs State   Time(s)
Domain-0                                   0      727     1 r-----    135.4
new.cfg                                     1      512     1 -b----      4.2

Then you can and reconnect to the console with:

sudo xm console 1

Replacing 1 with the guest to connect to.

Finally you can shutdown a guest with:

sudo xm shutdown 1

That should be it. You could now install things like openssh-server, apache2 or postgres8.1 on the guest and could then even SSH in rather than using the xm console tool.

Next we want to setup VNC so we can run a real desktop. Note: This will allow anyone to sign in over the network and hence is not secure.

apt-get install xserver-xfree86 vncserver  xfonts-100dpi xfonts-75dpi xfonts-base rxvt
apt-get install icewm-experimental
apt-get install gdm

cat <<EOF >> /etc/gdm/gdm.conf
[server-VNC]
name=VNC server
command=/usr/bin/Xvnc -geometry 800x600 -depth 24
flexible=true
EOF

perl -pi.bak -e 's/^0=Standard/0=VNC/g' /etc/gdm/gdm.conf

/etc/init.d/gdm stop
/etc/init.d/gdm start

Start vncserver:

vncserver

You will be asked for a password.

Then on the guest:

sudo apt-get install xvncviewer
xvncviewer 10.0.0.3:1

Enter the password and a window will popup with the desktop of the xen server.

• http://www.debian-administration.org/articles/322
• http://alum.wpi.edu/~tfraser/Stories/t60.html
• http://www.debianhelp.org/node/2235

Update 2007-07-15: Here is a sample config file produced by Xen:

#
#  Configuration file for the Xen instance rms, created on
# Tue Jul 17 09:45:27 2007.
#
#
#  Kernel + memory size
#
kernel  = '/boot/vmlinuz-2.6.18-4-xen-686'
ramdisk = '/boot/initrd.img-2.6.18-4-xen-686'
memory  = '512'
#
#  Disk device(s).
#
root    = '/dev/sda1 ro'
disk    = [ 'file:/home/james/xen/domains/rms/disk.img,sda1,w', 'file:/home/james/xen/domains/rms/swap.img,sda2,w' ]
#
#  Hostname
#
name    = 'rms'
#
#  Networking
#
vif  = [ 'ip=10.0.0.1' ]
#
#  Behaviour
#
on_poweroff = 'destroy'
on_reboot   = 'restart'
on_crash    = 'restart'

(view source)