|Download Linux On Linux||Virtual Machine Monitor Documentation|
Linux-on-Linux is Copyright 2005-2007 to National ICT Australia, the University of NSW, Matthew Chapman and Peter Chubb. It is licensed under the GNU Public Licence, version 2.0.
gcc(version >= 4.1)
$ tar xjvf LinuxOnLinux-1.0.tar.bz2
Create two copies of Linux kernel source.
The patches from the CVS repository are usually against a recent
One tree will be for the host, and one for the guest.
Call them, e.g.,
I usually do:
$ cg-clone git://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux-2.6.git linux-2.6-import
$ cg-clone -l linux-2.6-import linux-2.6-guest
$ cg-clone -l linux-2.6-import linux-2.6-host
.../LinuxOnLinux/scripts/prepare.sh on each source
tree to patch it and to create the symbolic links.
$ sh LinuxOnLinux/scripts/prepare.sh linux-2.6-host host
$ sh LinuxOnLinux/scripts/prepare.sh linux-2.6-guest guest
prepare.sh carries out the following steps:
.../LinuxOnLinux/userspace/vmm-offsets.hinto the guest Linux
include/asm-ia64/vmm-offsets.h, and from
CONFIG_VMMHOSTin the configuration, and exclude
iptablesto set up NAT if you want your virtual machines to talk the the rest of the internet.
Build the virtual machine monitor. The build process needs to be able to get at the output directory of the patched host kernel.
$ cd LinuxOnLinux/userspace
$ KERNEL=/lib/modules/`uname -r`/build sh autogen.sh
(to rebuild, just type
Configure the guest OS as if for the SKI simulator, but also
CONFIG_AFTERBURNER, and exclude
CONFIG_HUGETLBFS (again, there are sample
configurations in the LinuxOnLinux source tree).
I prefer not to configure modules, because it's painful to get the modules into an appropriate place for the virtual machine to use.
Build with a command line that looks like this:
Run the resulting system with
If you prefer, there is a script
runvmm in the source
tree that you can start with to create and configure virtual
To use it, set the environment variable
to the base of the LinuxOnLinux source tree. Or read the manual page
and create a
.vmmrc file in your home directory.
The VMM uses
/dev/tap? as an ethernet
interface; you can bridge these with
assign an IP address and use IP forwarding.
-koption gives old behaviour.
straceon the hypervisor.
runvmmscript; it now works better in environments other than mine, and is more configurable.
Troubleshooting is harder than it should be at present; this is one of the things we're working on.
The first thing to do when troubleshooting is to run with
-k. If you're using
it'll print stuff to standard error; if you're using
runvmm the trace will be in
/tmp/xNNNNN where NNNNN is the
process number of
You should get an error message of some sort in that trace.
If it's not enough, edit
change the definition of the
DEBUG symbol to
(~0UL), and rebuild.
You'll now get a very verbose trace from the virtual machine monitor: every page fault, every privileged instruction, etc., will be logged.
The commonest problems are because the guest kernel hasn't been
built correctly. Check that all the patches applied cleanly, that
CONFIG_USRVMM is defined and that the afterburner was used
when building. I've had problems using distcc where the
afterburner wasn't installed in the same place on every machine;
also you should disable
ccache as it cannot see the
afterburner. gcc-4.0 sometimes miscompiles, as well. I use
gcc-4.1 without problems.
If you see a message like:
Cannot map from 0x8000000100000000,0x8000000103ffffff to 0x4000000--- mmap: Invalid argument PANIC in VMM: tlbop23 failed
it's because you've configured
in the host kernel. Note that the address is in region 4, which on a
standard kernel is used for hugetlb pages, but for the virtual machine
is used for the guest kernel's virtual memory area.