Friday, January 21, 2011

LXC(Linux Containers) on Ubuntu 10.04

I was able to get LXC(Linux Containers) to work on an Ubuntu 10.04 host system. Linux containers provide lightweight virtualization that lets you isolate processes and resources without the need to provide instruction interpretation mechanisms and other complexities of full virtualization. So if you need to quickly spin-up a virtual server to test a new version of rails or any other software then this post might help you do that.

For the impatient ones, I wrote a bash script which initializes a file system as well as create and runs a container instance. It can be found here and here. You still have to make changes to the host system yourself though before using the scripts.

To install the required tools do:
sudo apt-get install lxc bridge-utils debootstrap

Next is to setup cgroups with:
sudo mkdir /cgroup
sudo echo "none /cgroup cgroup defaults 0 0" >> /etc/fstab
sudo mount /cgroup

Then proceed to setup the network for bridge networking with:
sudo apt-get remove –purge network-manager network-manager-gnome

and then modify /etc/network/interfaces so it contains:
auto lo
iface lo inet loopback

auto br0
iface br0 inet static
bridge_ports eth0
bridge_stp off
bridge_maxwait 5
post-up /usr/sbin/brctl setfd br0 0

then we setup the nameserver
sudo echo "nameserver" > /etc/resolv.conf

then reboot the host machine with:
sudo reboot

Log-in again and check that the network is still working with:

and also

Next we setup a root file system template that will be used by the containers. The following steps can be done easily using this script.

To start, we build a lucid root file system
sudo mkdir /lxc
sudo cd /lxc
sudo debootstrap --variant=minbase --arch i386 lucid rootfs_template

The next commands below will refer to placeholders so make sure to adjust the commands and replace them with the following:
$ROOTFS --> /lxc/rootfs_template

to continue, we fix the device files
rm -rf $ROOTFS/dev
mkdir $ROOTFS/dev
mknod -m 666 $ROOTFS/dev/null c 1 3
mknod -m 666 $ROOTFS/dev/zero c 1 5
mknod -m 666 $ROOTFS/dev/random c 1 8
mknod -m 666 $ROOTFS/dev/urandom c 1 9
mkdir -m 755 $ROOTFS/dev/pts
mkdir -m 1777 $ROOTFS/dev/shm
mknod -m 666 $ROOTFS/dev/tty c 5 0
mknod -m 666 $ROOTFS/dev/tty0 c 4 0
mknod -m 666 $ROOTFS/dev/tty1 c 4 1
mknod -m 666 $ROOTFS/dev/tty2 c 4 2
mknod -m 666 $ROOTFS/dev/tty3 c 4 3
mknod -m 666 $ROOTFS/dev/tty4 c 4 4
mknod -m 600 $ROOTFS/dev/console c 5 1
mknod -m 666 $ROOTFS/dev/full c 1 7
mknod -m 600 $ROOTFS/dev/initctl p
mknod -m 666 $ROOTFS/dev/ptmx c 5 2

next we modify /etc/apt/sources.list so that it contains:
deb $MIRROR lucid main universe multiverse
deb $MIRROR lucid-security main universe multiverse

next we install and update the packages, run the following as root:
chroot $ROOTFS apt-get install --force-yes -y gpgv
chroot $ROOTFS apt-get update
chroot $ROOTFS apt-get install -y language-pack-en
chroot $ROOTFS locale-gen en_US.UTF-8
chroot $ROOTFS /usr/sbin/update-locale LANG="en_US.UTF-8" LANGUAGE="en_US.UTF-8" LC_ALL="en_US.UTF-8" LC_CTYPE="C"
chroot $ROOTFS apt-get install -y \
    adduser \
    apt-utils \
    console-setup \
    iproute \
    iptables \
    nano \
    netbase \
    openssh-blacklist \
    openssh-blacklist-extra \
    openssh-server \
    iputils-ping \
    rsyslog \
    sudo \
    ufw \
chroot $ROOTFS apt-get clean

as root user, setup mtab with:
chroot $ROOTFS ln -s /proc/mounts /etc/mtab

as root user, next we append the following to $ROOTFS/etc/environment:

as root user, next remove unneeded files from $ROOTFS/etc/init of the template system with:
cd $ROOTFS/etc/init
rm -f rc-sysinit.conf console* control* hwclock* module* mount* network-interface* plymouth* procps* tty{4,5,6}.conf udev* upstart*

as root user, create a $ROOTFS/etc/init/rc-sysinit file with the following content:
rm -f $(find /var/run -name '*pid')
rm -f /var/lock/apache/*
exit 0

as root user, make it executable with:
chmod a+x $ROOTFS/etc/init/rc-sysinit

as root user, modify ssh.conf to contain:
# ssh - OpenBSD Secure Shell server
# The OpenSSH server provides secure shell access to the system.

description     "OpenSSH server"

start on startup
stop on runlevel S

expect fork
respawn limit 10 5
umask 022
# replaces SSHD_OOM_ADJUST in /etc/default/ssh
oom never

pre-start script
    test -x /usr/sbin/sshd || { stop; exit 0; }
    test -e /etc/ssh/sshd_not_to_be_run && { stop; exit 0; }
    test -c /dev/null || { stop; exit 0; }

    mkdir -p -m0755 /var/run/sshd
end script

# if you used to set SSHD_OPTS in /etc/default/ssh, you can change the
# 'exec' line here instead
exec /usr/sbin/sshd

as root user, we create an archive of the template so we can reuse it:
cd /lxc
tar zcvf rootfs_template.tgz rootfs_template

To make things a lot easier, you can just use the bash script I wrote which can be downloaded from here. Save it in /lxc and make it executable and run it with:
sudo chmod +x /lxc/
sudo cd /lxc
sudo ./

To configure and run a container, download another bash script I wrote here and save it in /lxc.
Make it executable and run it with:
sudo chmod +x /lxc/
cd /lxc
sudo ./

I hope you found the post useful. You can subscribe via email or subscribe via a feed reader to get relevant updates from this blog. Have a nice day.


  1. To install sun-java on the container, I had to modify /etc/apt/sources.list so that it contains

    deb lucid partner

    then do:

    apt-get install sun-java6-jre

  2. Excellent write up. Thank you!

    Minor typo found in "systinit":

    as root user, create a $ROOTFS/etc/init/rc-systinit

  3. Thanks for the correction

  4. Thanks for the scripts, they are a dead easy way to make a lucid guest.

    I am having a small problem with Lucid guests made using your script and manually.

    The guest starts fine with no errors displayed when run without the -d option, but for some reason none of the init.d scripts run.

    ssh server starts fine, presumably due to your script that starts on startup, but none of the apps installed using apt-get startup automatically, and more frustratingly, no default gateway is assigned.

    Logging in through ssh and adding the gateway and starting the services manually works fine, and the system otherwise works perfectly once they are started, I just can't get them to start on their own.

    Any ideas?

  5. glad to be of some help

    I believe the init system is currently broken for lxc so for now you have to start the services/servers that you add manually

    please post back if you do figure it out and get init scripts to work

  6. every thing went fine but while running "" script, in the end where the script calls lxc-create...

    i am getting an error configuration path '/usr/var/lib/lxc' not found

    It seems the script /usr/bin/lxc-create calls tries to access this path '/usr/var/lib/lxc' which is not present.

    Could you please let me know why this would be the the case. Thanks

  7. Have a bustling way of life? Set up your dinners or snacks the night prior to your exercises begin and simply get them directly from its food containers and you're in a hurry. storage bins