Some months ago I wished for a tool to create virtual disk images, something similar to debootstrap except it would create a disk image instead of a directory. It would be used like this:

sudo ./vmdebootstrap --image test.img --size 1G \
--log test.log --mirror http://mirror.lan/debian/

debootstrap is wonderful for creating a basic Debian system for use with chroot. In the modern world, something similar but for virtual machines (such as KVM) would be nice to have, I thought. Particularly if it is something that is not tied to a particular virtualization technology, and does not require running in an actual virtual machine (for speed and simplicity).

I could imagine using a tool like this for automatically building appliance-like system images, for example for FreedomBox. Being able to do that in a fully automatic manner would make development simpler and more easy for others to reproduce, and would allow for things like automatic integration testing: build image, boot it in a virtual machine, run automatic tests, report results.

There are a bunch of tools that almost do this, but not quite. For example, live-build builds a disk image, but it's one that is aimed for live CDs and such. This brings in some limitations. By default, the image is not persistent. You can make a persistent image, but that requires you to provide a second partition (or disk) that holds the modified parts, using copy-on-write. It also means that if the bootloader or initramfs needs to be updated, then the read-only image needs to be updated. That's fine for live CDs, but not so good if you want a real installed system that you can upgrade to all future releases of Debian. (Ben Armstrong kindly helped me use live-build the right way. Thanks!)

There's also grml-debootstrap which seems to work on a real system, but not installing onto a virtual disk image (and the difference is crucial to me).

Based on feedback I received back then, and some experimentation I've done the past couple of days, I wrote a very rudimentary version of the tool I want. The code is on gitorius. (You need my cliapp library to run it.)

Be warned: it really is a very rudimentary version. For example, it provides not flexibility in choice of partitioning, filesystem type, what packages get installed, etc. It does not configure networking, and the root password is disabled. Some of that flexibility would be quite easy to add.

It also uses extlinux as the bootloader, since I utterly failed to get grub2 to install onto the virtual disk. (I did, however, manage to overwrite my develpment machine's bootloader.)

The command above works, and results in a RAW format disk image that boots under KVM. It might boot off a hard disk or USB flashdrive too, but I haven't tried that.

I now have several questions:

  • Does something like this already exist?
  • Does anyone else find this useful?
  • Is another approach better? Perhaps running a fully preseeded debian-installer in a virtual machine might be better.
  • If no other tool exists for the same job, what should I do next with vmdebootstrap?
I always thought ubuntu-vm-builder was a tool doing something similar. Though never have checked it out so don't know in detail what it does.
Comment by Floris Fri May 13 12:16:33 2011

This is very cool.

Some time ago I tried to package vm-builder in Debian, but I failed precisely because I was unable to get grub/grub2 working with an image file.

I'll keep an eye on your project.

Comment by miguel.cc Fri May 13 16:57:45 2011
vm-builder and ubuntu-vm-builder rely on, as far as I see, on having a virtualization setup working (specifically, libvirt), which I think is a mistake. A tool like vmdebootstrap should be able to work without it. I admit I haven't argued why this is so particularly well, but it's simpler, more efficient, and my gut feeling.
Comment by Lars Wirzenius Sat May 14 09:18:34 2011
I'd like to make GRUB work properly here. Could you mail me details of what you tried that failed?
Comment by cjwatson [livejournal.com] Sun May 15 15:11:36 2011

I've sent an email.

For anyone else: to get grub to work with vmdebootstrap, the right way to invoke the grub tools needs to be found to replace the install_extlinux in vmdebootstrap. The code is in https://gitorious.org/vmdebootstrap for anyone wants to have a go. (Be warned, though, that wrong use of grub-install can replace the bootloader on the host machine, instead of the one in the disk image file.)

Comment by Lars Wirzenius Sun May 15 16:38:00 2011

An example of how to install Grub legacy into an image file can be found here - scroll down to 'KVM the hard way':

http://bit.ly/lLqX1Q

or the long version if you prefer:

http://webcache.googleusercontent.com/search?q=cache:vk5O2hDKf7MJ:www.wand.net.nz/~smr26/wordpress/+kvm-the-hard-way&cd=7&hl=en&ct=clnk&gl=ca&client=firefox-a&source=www.google.ca

Comment by andmalc Sun May 15 18:56:16 2011
Thank you. However, I don't think I'm interested in grub-legacy. If I'm not using the bootloader Debian uses by default, then extlinux is just as good as grub, I'd say.
Comment by Lars Wirzenius Sun May 15 19:27:51 2011

Hi.

I run everytime in this error:

Installing extlinux EEEK! Something bad happened... Cleaning up ERROR: command failed: ['extlinux', '--install', '/tmp/tmp0IIOI0']

/tmp/tmp0IIOI0 is device /dev/mapper/loop0p1

my os is a debian sid and I have installed all the dependencies

here are the command I am using to create the VM's: ./vmdebootstrap --image test.img --size 2G --verbose --root-password=xxx --distribution=sid --log test.log

Comment by Angelo Sat Jul 23 18:51:39 2011
i have solved the problem with liw's help :) vmdebootstrap only works with extlinux from squeeze. the extlinux on sid and wheezy is broken.
Comment by Angelo Sat Jul 23 19:21:29 2011