Install and Use Unionfs ( Merging Directories )

Howto: How Install and Use Unionfs (Merging Directories)

Unionfs, developed at Stony Brook university since 2004, is a stackable unification file system, which can merge the contents of several directories (so called branches) while keeping their physical content separate. It allows any mix of read-only and read-write branches, as well as insertion and deletion of branches on the fly. Unionfs can be used in several ways, for example to unify home directories from multiple filesystems on different disk partitions, or to merge several CDs to create a unified view of a photo archive. In a similar view, Unionfs, with copy-on-write functionality, can be used to merge read-only and read-write filesystems together and to virtually allow modification of read-only filesystems saving changes to the writable ones.

SLAX is a 177 MB Linux Live distribution which aims at compacting full featured Linux operating system to a portable medium (like usb flash drive or mini-cd) and allows everyone to boot Linux on any machine without the need to install it. It works even on computers with no harddisk at all. Unionfs is the most important part of a SLAX, it allows SLAX to seem and act as a real Linux OS with full-writable root directory tree. So let's speak about unionfs first.

Getting started
To get unionfs working, you need to create a Linux kernel module by compiling its source codes. Unionfs is available as a module extension for Linux Kernel 2.4.20 / 2.6.9 and higher. Download the latest version from FTP and extract the content of the archive by using

$ tar -xzf unionfs-x-y-z.tar.gz

Then cd to its directory and read README and INSTALL files which are part of the archive. There are many instructions how to avoid problems. Before the compilation itself, you might find it useful to know that it's possible to disable compiling debug information together with the module. Debug info is useful for reporting bugs, but significantly increases the size of kernel module. Two parameters must be set to disable debug at all:

* create a file called fistdev.mk in the directory with sources
* add this text to it:

EXTRACFLAGS=-DUNIONFS_NDEBUG
UNIONFS_DEBUG_CFLAG=

The compiled kernel module will be about 90 KB big without debug info, compared to 5 MB with it (you can download fistdev.mk used to compile unionfs for SLAX)

Another important thing to make the compilation work properly is to download and extract sources for your running kernel and to modify LINUXSRC variable in unionfs' Makefile, adding path where you actually extracted it (this can be autodetected in some cases).

Finally, use the following commands to build and install unionfs module into /lib/modules/$(KernelVersion)/kernel/fs/unionfs:

$ make
$ make install
$ depmod -a

Using unionfs

In the following example, we will merge contents of two directories into a single directory /mnt/union. We assume that all directories already exist.

$ modprobe unionfs
$ mount -t unionfs -o dirs=/mnt/cdrom1=ro:/mnt/cdrom2=ro unionfs /mnt/union

From now, the directory /mnt/union will contain all files and directories from /mnt/cdrom1 and /mnt/cdrom2, merged together and both read only. If the same filename is used in both cdrom directories, the one from cdrom1 has precedence (because it was specified leftmost in the list).
Using unionctl

Unionctl is a tool which is created (together with uniondbg) during unionfs compilation and is installed to /usr/local/sbin. Unionctl is intended to manage the existing union, to list, add, modify or delete existing branches. Some simple example follows, use unionctl command without any argument to see all available options.

To list branches in existing union, use

$ unionctl /mnt/union --list

which will produce the following output

/mnt/cdrom1 (r-)
/mnt/cdrom2 (r-)

To add another directory (/mnt/cdrom3) into existing union, use

$ unionctl /mnt/union --add --after /mnt/cdrom2 --mode ro /mnt/cdrom3

and unionctl --list will now produce

/mnt/cdrom1 (r-)
/mnt/cdrom2 (r-)
/mnt/cdrom3 (r-)

In the case when you change the content of branches themselves, execute the following command to force revalidation of the union:

uniondbg -g /mnt/union


Writing to union

Merging read-only directories is useful in many cases, but the union itself remains read-only too, until a read-write branch is added to it. In that case, all changes are stored in leftmost branch (using copy-up method, see below) and file deletions are done by using one of the two methods available:

- WHITEOUT mode, inserts a .wh (whiteout) file to mask out a real file
- DELETE_ALL mode, tries to delete all instances of a file from all branches

WHITEOUT mode is used as default. Copy-up is a special method used to handle file modifications in union. A file from ro branch can't be modified, so it is copied to upper (left) read-write branch at the time when the modification should begin. Then the modification is possible and modified file remains in rw branch.

To add a rw branch at the top of union in our example, type

$ unionctl /mnt/union --add --before /mnt/cdrom1 --mode rw /mnt/changes

All the changes will be stored in /mnt/changes and the union will look like this:

/mnt/changes (rw)
/mnt/cdrom1 (r-)
/mnt/cdrom2 (r-)
/mnt/cdrom3 (r-)


Practical unionfs application - SLAX

Data stored on a read-only medium like CD-ROM can't be modified. A Live CD Linux distribution, which is offering full write support to all directories, needs to use special techniques to allow virtual modifications and to save all changes in memory. SLAX is using these techniques for very long time, starting at the end of 2003 with ovlfs and implementing unionfs at the end of 2004. SLAX 5, released in April 2005, can give you an impression of what miracles could be, thanks to unionfs, created.


Links
- Stony Brook university: http://www.fsl.cs.sunysb.edu/
- UnionFS: http://www.fsl.cs.sunysb.edu/project-unionfs.html
- SLAX: http://www.slax.org
- Linux Live scripts: http://www.linux-live.org
- Linux kernel: http://www.kernel.org

Installing Ubuntu Linux (and others) on a Usb PenDrive

How-to:

This tutorial will show how-to install Ubuntu on a usb stick. Even though this tutorial uses Ubuntu as its base distribution, you could Installing Ubuntu Linux on a usb pendrivevirtually use any type of Linux liveCD distribution.

Being able to run Linux out of a usb bar is a great way to enjoy the live CD experience (being able to use Linux on any computer you might get by) and the big advantage of being easier to carry around than a CD.

1. Requirements

In order to reproduce this tutorial, you will need a few items such as:

• a ubuntu liveCD
• a usb bar of at least 1G
• a running Linux operating system

Now that you have all this, it is time to prepare you USB bar do host the Ubuntu liveCD files.

2. Setting up the USB disk

2.1. Finding the device

In the first place, you need to plug your usb drive and check under which device it is associated. To find out the device, run:

$ sudo fdisk -l

On my system, the device appears as being /dev/sdb, I will therefore use /dev/sdb as a reference for this tutorial, please replace it accordingly to your system (might be sda, sdc ...).
Once you found your device, you are going to create the partitions.

Using the wrong device name might destroy your system partition, please double check

2.2. Making the partitions

Make sure every of your already mounted partition are unmounted:

$sudo umount /dev/sdb1

and then launch fdisk, a tool to edit partition under linux:

sudo fdisk /dev/sdb

We are going delete all the partition and then create 2 new partition: one fat partition of 750M which will host the files from the live CD iso, and the rest on another partition.

At fdisk prompt type d x where x is the partition number (you can simply type d if you only have one partition), then:

• n to create a new partition
• p to make it primary
• 1 so it is the first primary partition
• Accept the default or type 1 to start from the first cylinder
• +750M to make it 750 Meg big
• a to toggle the partition active for boot
• 1 to choose the 1 partition
• t to change the partition type
• 6 to set it to FAT16

Now we have out first partition set up, let's create the second one:

• n to create yet again a new partition
• p to make it primary
• 2 to be the second partition
• Accept the default by typing Enter
• Accept the default to make your partition as big as possible
• Finally, type w to write the change to your usb pendrive

Partitions are now created, let's format them.

2.3. Formatting the partitions

The first partition is going to be formated as a FAT filesystem of size 16 and we are going to attribute it the label "liveusb".

$ sudo mkfs.vfat -F 16 -n liveusb /dev/sdb1

The second partition is going to be of type ext2 with a blocksize of 4096 bytes and the label casper-rw. Mind that it has to be labeled as casper-rw otherwise the tutorial won't work!.

$ sudo mkfs.ext2 -b 4096 -L casper-rw /dev/sdb2

At this stage, our usb pendrive is ready to host the liveCD image. Now, let's copy the files to the usb bar.

How-to: Installing Ubuntu Linux on a usb pendrive -- page 2

3. Installing Ubuntu on the USB stick

3.1. Mounting Ubuntu liveCd image

In the first place we need to mount our ubuntu iso. Depending if you have the .iso file or the CD, there is 2 different ways of mounting it.

3.1.1. Mounting from the CD

People using Ubuntu or any other user-friendly distro, might just have to insert the cd and it will be mounted automatically. If this is not the case:

$ sudo mount /media/cdrom

should mount it.

3.1.2. Mounting from an .iso image file

We will need to create a temporary directory, let say /tmp/ubuntu-livecd and then mount our iso (I will be using a feisty fawn iso).

$ mkdir /tmp/ubuntu-livecd
$ sudo mount -o loop /path/to/feisty-desktop-i386.iso /tmp/ubuntu-livecd

Once the cd image is ready, it is time to mount the newly created usb bar partitions:

3.2. Mounting the usb bar partitions

Same here, you might be able to get both your partition by simply replugging the usb pendrive, partition might appears as: /media/liveusb and /media/casper-rw. If this is not the case, then you will need to mount them manually:

$ mkdir /tmp/liveusb
$ sudo mount /dev/sdb1 /tmp/liveusb

All the partitions we need are now mounted, let's copy the files.

3.3. Copying the files to the usb bar

Let positionned yourself on the CD image directory (in my case: /tmp/ubuntu-livecd , but it might be /media/cdrom , and copy at the root of your usb first partition:

• the directories: 'casper', 'disctree', 'dists', 'install', 'pics', 'pool', 'preseed', '.disk'
• The content of directory 'isolinux'
• and files 'md5sum.txt', 'README.diskdefines', 'ubuntu.ico'
• as well as files: 'casper/vmlinuz', 'casper/initrd.gz' and 'install/mt86plus'

$ cd /tmp/ubuntu-livecd
$ sudo cp -rf casper disctree dists install pics pool preseed .disk isolinux/* md5sum.txt README.diskdefines ubuntu.ico casper/vmlinuz casper/initrd.gz install/mt86plus /tmp/liveusb/

It might complain about symbolic links not being able to create, you can ignore this.

Now let's go to the first partition of your usb disk and rename isolinux.cfg to syslinux.cfg:

$ cd /tmp/liveusb
$ sudo mv isolinux.cfg syslinux.cfg

change /tmp/liveusb according to your settings

Edit syslinux.cfg so it looks like:

DEFAULT persistent
GFXBOOT bootlogo
GFXBOOT-BACKGROUND 0xB6875A
APPEND  file=preseed/ubuntu.seed boot=casper initrd=initrd.gz ramdisk_size=1048576 root=/dev/ram rw quiet splash --
LABEL persistent
menu label ^Start Ubuntu in persistent mode
kernel vmlinuz
append  file=preseed/ubuntu.seed boot=casper persistent initrd=initrd.gz ramdisk_size=1048576 root=/dev/ram rw quiet splash --
LABEL live
menu label ^Start or install Ubuntu
kernel vmlinuz
append  file=preseed/ubuntu.seed boot=casper initrd=initrd.gz ramdisk_size=1048576 root=/dev/ram rw quiet splash --
LABEL xforcevesa
menu label Start Ubuntu in safe ^graphics mode
kernel vmlinuz
append  file=preseed/ubuntu.seed boot=casper xforcevesa initrd=initrd.gz ramdisk_size=1048576 root=/dev/ram rw quiet splash --
LABEL check
menu label ^Check CD for defects
kernel vmlinuz
append  boot=casper integrity-check initrd=initrd.gz ramdisk_size=1048576 root=/dev/ram rw quiet splash --
LABEL memtest
menu label ^Memory test
kernel mt86plus
append -
LABEL hd
menu label ^Boot from first hard disk
localboot 0x80
append -
DISPLAY isolinux.txt
TIMEOUT 300
PROMPT 1
F1 f1.txt
F2 f2.txt
F3 f3.txt
F4 f4.txt
F5 f5.txt
F6 f6.txt
F7 f7.txt
F8 f8.txt
F9 f9.txt
F0 f10.txt

Woof, finally we have our usb disk almost usuable. We have a last thing to do: make the usb bootable.

3.4. Making the usb bar bootable.

in order to make our usb disk bootable, we need to install syslinux and mtools:

$ sudo apt-get install syslinux mtools

And finally unmount /dev/sdb1 and make it bootable:

$ cd
$ sudo umount /tmp/liveusb
$ sudo syslinux -f /dev/sdb1

Here we are :D , reboot, set your BIOS to boot from the usb bar and enjoy Ubuntu linux from a pendrive

4. Troubleshooting

If you are having trouble booting on the usb bar, this might be due to your MBR being corrupted. In order to fix it up, you can use lilo (I installed lilo on my box only for thid purpose).

$ lilo -M /dev/sdb

will fix the MBR on device /dev/sdb

Taken From: http://duncanbrown.org/linux/system_administration/usb_flash_system/