Xen Virtualization under Debian GNU/Linux Lenny

Jean Baptiste FAVRE

juin 2009

Introduction

Xen is one of the most used opensource virtualization system. Firts releases were only able to virtualize open operating systems because some kernel modifications were needed.

Nowadays, recent processors embbed features which allow to virtualize more operating system without any kernel modifications.

Let's see how to install, configure and get started with Xen under Debian GNU/Linux Lenny with amazing features like DRBD and LVM.

Glossary

Xen domain
One Xen instance. It can be privileged (dom0) or not (domU)
Dom0
Privileged Xen domain. Is used as Xen host manager. Has full access to hardware devices.
DomU
Unprivileged Xen domain. Also called "guest" or "VM" for Virtual Machine. By default, does not have any direct access to hardware devices.
DRBD
Distributed Redundant Block Device. Used to replicate Hard disks through network. Heavily used in High Availability environments.
LVM
Logical Volume manager. Abstraction level for disks and partitions. Makes disk managment much more easy. Can also provide snapshots features.

Storage choice

You have multiple choice for storage

Image file
Hard disk is "virtualized" as file, just like VMWare does. Quite simple to setup and use, it sometimes suffer some performances issues and does not allow live migration, except with centralized storage like NFS.
NFS
Usefull with image files, you must have a strong network to get decent performances. If you do not plan to use live migration, NFS is not the best solution to use.
LVM
Much better solution, especially with DRBD or LVM mirorring features. You'll get local storage performances with much more flexibility.
LVM + DRBD
LVM flexibility with DRBD replication feature. A must have ! Of course, to use this, you'll need at least 2 servers.

As I have 2 servers for my tests, I choose to use LVM+DRBD. Configured in dual primary mode, DRBD gives you the ability to perform live migration on a per VM basis. Usefull when you want to spread your VMs accross you Xen hosts.

Our system will behave as follow:

LVM configuration

If you do not know LVM, you can read LVM: Logical Volume Manager

You first have to create a partition and prepare it as Pfysical Volume. For example, on my dom0, I create 2 PVs:

DRBD configuration

If you don't know DRBD, you can read: DRBD: Distributed Replicated Block Device

Xen installation

Debian GNU/Linux Lenny ships Xen in verison 3.2. Xen installation is as easy as executing:

Xen installation
apt-get install xen-hypervisor xen-utils
Dom0 xenified kernel installation 
apt-get install linux-image-2.6.18-6-xen-amd64

Once kernel is installed, you can reboot your node. After reboot, just check that Xen hypervisor is started:

Running Xen domains list
xm list
Name                                        ID   Mem VCPUs      State   Time(s)
Domain-0                                     0  3756     4     r-----     35.3

Network configuration

Like storage, Xen can deal with various network configuration:

Bridged network
Easiest solution, each VM has its own IP address. Sometimes forbidden by some hosting companies because having multiple MAC adresses per interface is not supported for security reasons (that avoid MAC adress spoofing).
Routed network
Dom0 acts as router. Each VM still has its own IP address, but MAC adresses won't be broadcasted any more outside of Xen host.
Natted network
Dom0 will act as gateway for domU and will translate IP addresses. Reserved for very specific use because domU won't have direct access to the network.

We'll use bridged network solution. Let's activate it:

/etc/xen/xend-config.sxp
##
# To bridge network traffic, like this:
#
# dom0: ----------------- bridge -> real eth0 -> the network
#                            |
# domU: fake eth0 -> vifN.0 -+

...

# Configuration par défaut:
#(network-script network-dummy)
# Notre configuration:
(network-script network-bridge)

Then:

Xendi service restart
/etc/init.d/xend restart
Restarting XEN control daemon: xend.

Finally:

Check network configuration
ifconfig
eth0      Link encap:Ethernet  HWaddr 00:XX:XX:XX:XX:XX
          inet adr:192.168.0.1  Bcast:192.168.0.255  Masque:255.255.255.0
          adr inet6: fe80::2XX:XXXX:XXXX:XXXX/64 Scope:Lien
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
          RX packets:10466 errors:0 dropped:0 overruns:0 frame:0
          TX packets:6558 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 lg file transmission:0
          RX bytes:9560501 (9.1 MiB)  TX bytes:900269 (879.1 KiB)

lo        Link encap:Boucle locale
          inet adr:127.0.0.1  Masque:255.0.0.0
          adr inet6: ::1/128 Scope:Hôte
          UP LOOPBACK RUNNING  MTU:16436  Metric:1
          RX packets:0 errors:0 dropped:0 overruns:0 frame:0
          TX packets:0 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 lg file transmission:0
          RX bytes:0 (0.0 B)  TX bytes:0 (0.0 B)

peth0     Link encap:Ethernet  HWaddr 00:XX:XX:XX:XX:XX
          adr inet6: fe80::2XX:XXXX:XXXX:XXXX/64 Scope:Lien
          UP BROADCAST RUNNING PROMISC MULTICAST  MTU:1500  Metric:1
          RX packets:10604 errors:0 dropped:0 overruns:0 frame:0
          TX packets:6710 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 lg file transmission:1000
          RX bytes:9575961 (9.1 MiB)  TX bytes:919563 (898.0 KiB)
          Interruption:18

If you can see peth0 interface, you're done. It's a veray classic bridged network, there's no surprise in there.

HVM domU configuration

I'll only cover fully virtualized VM (or HVM) confiugration.

This allows you to get "real" independent servers. As an example, this configuration is required to install a Windows based DomU.

Each VM must have its own configuration file.

Generic configuration file: /etc/xen/xps-101.cfg
#
# HVM configuration file
# Linux
#
################################

kernel       = '/usr/lib/xen-3.2-1/boot/hvmloader'
device_model = '/usr/lib/xen-3.2-1/bin/qemu-dm'
builder      = 'hvm'
memory       = '1024'
vcpus        = '1'
cpus         = '1'
localtime    = 0
serial       = 'pty'

#
# Parametrage disque
# boot on floppy (a), hard disk (c) or CD-ROM (d) or Network (n)
boot         = 'dcn'
disk         = [
		'phy:/dev/drbd1,ioemu:hda,w'
	       ]

#
# Parametrage reseau
#
vif          = [ 
		  'bridge=eth0,type=ioemu,mac=00:16:3E:01:01:65,ip=192.168.0.101'
	       ]

#
# Comportement
#
on_poweroff  = 'destroy'
on_reboot    = 'restart'
on_crash     = 'restart'

extra = "xencons=tty clocksource=jiffies notsc pci=noapci"

#
# Parametrage VNC pour l'installation et/ou la recup
#
#vfb = [ 'type=vnc,vnclisten=127.0.0.1,vncdisplay=1' ]
vfb = [ 'type=vnc,vnclisten=0.0.0.0,vncdisplay=1' ]
keymap       = "fr"

#
# Parametrage de la machine virtuelle
#
name         = 'xps-101'
hostname     = 'xps-101.mydomain.com'

Once you're done with confiiguration, you can start your domU and install it as you would do with a physical server.

HVM DomU control

Controlling a domU is quite simple. There's not so much operation available:

domU creation

Virtual machine creation
xm create xps-101.cfg
Using config file "/etc/xen/xps-101.cfg".
Started domain xps-101

Power off

Restart
xm restart xps-101
Soft stop
xm shutdown xps-101
Hard stop
xm destroy xps-101

Migration

For live migration to work, you need more informations. As an exemple, you will need remote dom0 peer.

Xen must also be configured to communicate with its peer. For this, you need to adapt /etc/xen/xend-config-xenapi.sxp file.

Activate live migration for xend service
(xend-relocation-server yes)

You also have to restart xend service. Then, use following command to live migrate your domU:

domU live migration
xm migrate xps-101 192.168.0.2 --live

Where 192.168.0.2 is remote Xen dom0 peer.

Maintenance

Sometime, you could want to access you domU without using network (if for example you just made a mistake with firewall :) ).

Connect to domU console
xm console xps-101
Debian GNU/Linux 5.0 xps-101 ttyS0
xps-101 login:

To leave console mode, just use CTRL+ALTGr+]

Sources and references

Official websites

Xen
  • http://www.xen.org/
DRBD
  • http://www.drbd.org/
GNU/Debian Linux
  • http://www.debian.org/

Documentation and Howto

Xen
  • http://www.howtoforge.com/debian_etch_xen_3.1
  • http://www.labo-linux.org/articles-fr/premiere-approche-de-xen/installation-de-xen
  • http://www.xen-tools.org/software/xen-tools/examples.html
  • http://wiki.debian.org/Xen
  • https://asi.insa-rouen.fr/asipedia/index.php/Xen
  • https://help.ubuntu.com/community/Xen
  • http://www.howtoforge.com/debian_etch_xen_from_debian_repository
  • http://howto.landure.fr/gnu-linux/debian-4-0-etch/installer-et-configurer-xen-sur-debian-4-0-etch
  • http://www.hostinggeek.com/
  • http://www.projet-plume.org/fiche/xen
  • http://virtualization4xceptn.wordpress.com/2008/09/15/xen-how-to-define-the-boot-order-for-hvm-in-vmcfg/
  • http://etbe.coker.com.au/2007/07/27/a-support-guide-to-xen/
LVM
  • http://www.morot.fr/spip.php?article35
  • http://linux.developpez.com/lvm/#L4.2
  • http://clx.anet.fr/spip/article.php3?id_article=237
DRBD
  • http://www.drbd.org/users-guide/

Some tips

Dom0 Lenny kernel
  • http://lists.debian.org/debian-devel/2008/07/msg00312.html
  • http://kernel-archive.buildserver.net/debian-kernel/waldi/xen-extra/pool/main/l/linux-2.6/
UDEV inside DomU
  • http://codewut.de/xen-on-32bit-os-with-8gb-ram
  • http://lists.xensource.com/archives/html/xen-users/2006-09/msg00284.html
HVM support
  • http://wiki.xensource.com/xenwiki/HVM_Compatible_Processors

Troubleshooting

Access domU LVM partition from within dom0
  • http://blogs.simc.be/simc/index.php/post/2009/01/22/Kpartx-pour-aller-un-peut-plus-loin-avec-LVM
  • http://cedric-tintanet.developpez.com/tutoriels/linux/sauvegarde-virt/
DomU brutal test
  • http://www.cyberciti.biz/faq/understanding-bash-fork-bomb/
Mirror creation
  • http://smhteam.info/wiki/index.linux.php5?wiki=CreerUnMirroirDebianLocal
  • http://www.debian.org/mirror/ftpmirror.fr.html
  • http://www.debian.org/mirror/list-full#FR
  • http://howto.landure.fr/gnu-linux/ubuntu-edgy-eft/developpement-et-serveurs/creation-dun-mirroir-ubuntu-local-a-laide-de-apt-mirror
  • http://doc.ubuntu-fr.org/tutoriel/creer_un_miroir_de_depot_local
Keymap setting for VNC
  • http://linux.derkeiler.com/Mailing-Lists/Debian/2007-11/msg01128.html

Windows as Xen domU

About Jean Baptiste FAVRE

I spend most of my free time on the Internet working on GNU/Linux with Debian or CentOS, virtualization with Xen and KVM technology, as well as cluster stacks with corosync and OpenAIS. Particularly interested in Linux, Netfilter, virtualization, monitoring and clusters, most of my personal works are published on this website and others should not delay.

By way professional, I manage servers running RedHat or CentOS and VMware ESXi farm.

From time to time, I manage to drop my keyboard and read a book while listening to music, but it never lasts long.

License

Creative Commons License This document is published under Creative Common by-nc-sa licence

Valid XHTML 1.0 Strict |  Valid CSS |  Creative Common by-nc-sa licence

Index

  1. Introduction
  2. Storage choice
  3. LVM configuration
  4. DRBD configuration
  5. Xen installation
  6. Network configuration
  7. HVM domU configuration
  8. HVM domU control
  9. Sources and references
  10. About
  11. License