Custom Debian EC2 AMIs From Xen Images

Creating the Xen Image

Now you need to create the base image using the xen-create-image tool (you will have installed it if you followed the Xen tutorial linked to above). Obviously since the EC2 hardware is i686 hardware this needs to be done on a PC platform:

sudo mkdir /mnt/xen sudo xen-create-image --debootstrap --verbose
--dir=/mnt/xen --size=1Gb --memory=512Mb --fs=ext3 --cache=yes --dist=etch
--mirror=http://ftp.uk.debian.org/debian/ --hostname=debian_etch
--initrd=/boot/initrd.img-2.6.18-4-xen-686
--kernel=/boot/vmlinuz-2.6.18-4-xen-686 --dhcp

These settings will create an ext3 filesystem of a Debian Etch install in an image which will get a network address using DHCP.

This will spit out a 1.0GB Debian Etch minimal image in /mnt/xen/domain/debian/disk.img don't worry about the memory size being different from the amount Xen uses. Even the kernel version doesn't matter too much since EC2 will replace it with its own.

At this point you have two choices:

1. Use xm create -c /etc/xen/debian.cfg to start the Xen virtual machine and configure it from there
2. Mount the image as a filesystem and make changes directly

If you are used to using Xen you'll have no problem making the changes directly to the booted image so I'll show you how to do it the other way.

Incidentally, you can also create everything entirely from scrath as described here: http://docs.amazonwebservices.com/AWSEC2/2007-03-01/DeveloperGuide/public-ami-guidelines.html

To create an AMI from the Xen image first mount the image:

sudo mkdir /mnt/image
sudo mount -t ext3 -o loop /mnt/xen/domains/debian/disk.img /mnt/image

Then set up the fstab which EC2 will expect:

cd /mnt/image/etc
sudo mv fstab fstab.xen
sudo vi /etc/fstab.ec2

Add the following content:

/dev/sda1 /     ext3  defaults             1 1
/dev/sda2 /mnt  ext3  defaults,user_xattr  1 2
/dev/sda3 swap  swap  defaults             0 0
none      /proc proc  defaults             0 0
none      /sys  sysfs defaults             0 0

Then:

sudo ln -sf fstab.ec2 fstab

At this point you need to set a root password or follow the instructions in the next section to make a public AMI.

Extra Steps For Public AMIs

If you want to make your AMIs available to the publis there are some extra steps you need to take described here: http://docs.amazonwebservices.com/AWSEC2/2007-03-01/DeveloperGuide/AESDG-chapter-sharingamis.html

Basically you need to perform three steps:

1. Randomise the root password the first time an instance boots
2. Change the default SSH setup
3. Set up the public part of the SSH key pair so that the user can sign in

Here is how you can do this on Debian Etch.

First tweak the SSH config as recommended by Amazon:

sed -i -e 's/PermitRootLogin no/#PermitRootLogin no/g' /mnt/image/etc/ssh/sshd_config

cat <<EOL >> /mnt/image/etc/ssh/sshd_config
UseDNS  no
PermitRootLogin without-password
EOL

Then alter the boot process to set up the public key. Edit this file:

vim mnt/image/etc/rc.local

So it looks like this:

#!/bin/sh -e
#
# rc.local
#
# This script is executed at the end of each multiuser runlevel.
# Make sure that the script will "exit 0" on success or any other
# value on error.
#
# In order to enable or disable this script just change the execution
# bits.
#
# By default this script does nothing.

if [ ! -d /root/.ssh ] ; then
        mkdir -p /root/.ssh
        chmod 700 /root/.ssh
fi
# Fetch public key using HTTP
wget -q -P /tmp http://169.254.169.254/2007-03-01//meta-data/public-keys/0/openssh-key
cat /tmp/openssh-key > /tmp/my-key
rm /tmp/openssh-key
if [ $? -eq 0 ] ; then
        cat /tmp/my-key >> /root/.ssh/authorized_keys
        chmod 600 /root/.ssh/authorized_keys
        rm /tmp/my-key
fi
if [ -f "/.firstrun" ] ; then
  dd if=/dev/urandom count=50|md5sum > /tmp/p.out
  POUT=`cat /tmp/p.out | cut -f1 -d' '`
  rm -f /tmp/p.out
  /usr/sbin/usermod -p $POUT root
  rm -f /.firstrun
fi
exit 0

Then create the .firstrun file so that the password changing is triggered on boot:

touch /mnt/image/.firstrun

Appending the public key to the authorized keys file on every boot might seem overkill but this is how Amazon does it and it is unlikely you'll be rebooting your image too many times anyway.

Now you can unmount the image:

cd /
sudo umount /mnt/image

Bundling

This is very similar to before but rather than using ec2-bundle-vol you use ec2-bundle-image. You'll need the X.509 certificate and private key to bundle the image as before.

The name given by the option -p determines the name of the manifest file. In this case we chose debian_etch:

ec2-bundle-image -p debian_etch -i /mnt/xen/domains/debian/disk.img -k ~/Desktop/pk-HKZYKTAIG2ECMXYIBH3HXV4ZBZQ55CLO.pem -c ~/Desktop/cert-HKZYKTAIG2ECMXYIBH3HXV4ZBZQ55CLO.pem -u 495219933132
Splitting /tmp/debian_etch.tar.gz.enc...
Created debian_etch.part.00
Created debian_etch.part.01
Created debian_etch.part.02
...
Created debian_etch.part.51
Created debian_etch.part.52
Generating digests for each part...
Digests generated.
Creating bundle manifest...
ec2-bundle-image complete.

By default images are written to /tmp unless you use the -d option. Let's check they are there:

james@dirac:/$  ls -lh /tmp/debian_etch.*
-rw-r--r-- 1 james james 9.0K 2007-08-27 21:48 /tmp/debian_etch.manifest.xml
-rw-r--r-- 1 james james  10M 2007-08-27 21:47 /tmp/debian_etch.part.00
-rw-r--r-- 1 james james  10M 2007-08-27 21:47 /tmp/debian_etch.part.01
-rw-r--r-- 1 james james  10M 2007-08-27 21:47 /tmp/debian_etch.part.02
...
-rw-r--r-- 1 james james  10M 2007-08-27 21:48 /tmp/debian_etch.part.51
-rw-r--r-- 1 james james 8.9M 2007-08-27 21:48 /tmp/debian_etch.part.52

As you can see, the Debian Etch install has 53 parts which is quite a lot more than the 23 my previous tutorial had.

You can now upload it to S3 but unless your network connection is as fast as the 250Mb/s each EC2 instance has this will take a lot longer than it took to to upload the parts from EC2 which is what we did in the last tutorial:

ec2-upload-bundle -b james-music -m /tmp/debian_etch.manifest.xml -a <AWS Access Key>  -s <AWS shared key>

Once the image is uploaded we need to register it. Load up a Python prompt again and use boto to create a connection before registering the image:

>>> from boto.ec2.connection import EC2Connection
>>> conn = EC2Connection('<AWS Access Key>', '<AWS shared key>')
>>> conn.register_image('james-music/debian_etch.manifest.xml')
u'ami-ddec09b4'

You should now be able to start the image:

>>> for i, image in enumerate(images):
...     if image.location.startswith('james-music'):
...         print "%s, %s, %s"%(i, image.id, image.location)

Once you have found the image number of the image (in this case 40) you can start it:

>>> image = images[40]
>>> reservation = image.run(key_name='gsg-keypair', security_groups=['web_test'])

And check progress:

>>> reservation.instances[0].update()
>>> reservation.instances[0].state
u'pending'

Custom Modules

When you boot an instance, EC2 replaces the kernel with its own which is a Xen 2.6.16 kernel. This means you can't use a modified kernel but you can use kernel modules. Although I haven't tried it yet I've read you do this by booting the Fedora Development image, compiling the modules you want and copying them to the /lib/modules/ directory of the Debian image. You need to use Fedora because the modules need compiling with GCC 4.0 but Etch uses 4.1

http://elastic8.com/?q=node/2 http://tim.dysinger.net/2007/07/28/compiling-fuse-kernel-modules-for-debian-40-on-ec2/ http://info.rightscale.com/2007/8/7/recompiling-kernel-modules-for-ec2-instances

You can also get the default ones directly from Amazon but they seem to get set up anyway so you don't need to:

wget http://s3.amazonaws.com/ec2-downloads/modules-2.6.16-ec2.tgz
tar zxfv modules-2.6.16-ec2.tgz
mv the files to /lib/modules
depmod -a