Live CD From Your In Installation / Running System - Ubuntu - Hard Way

Transforming your Installation into a Live DVD/CD


José Oliveira: I tested this tutorial on Ubuntu 8.04 Hardy Heron using the method without using de debootstrap, and it worked like a charm, my only work was copying the commands from tutorial and paste them in the bash shell :).


This HOWTO is about making a live CD/DVD from the main system on your hard drive. This might be desired if you have customized your system and want to have it on CD.

Another approach that will be discussed here is building your live CD/DVD from scratch. This will be done by building a new system using debootstrap. This is usefull if you want to build a clean live CD, or if you want to build a minimal rescue cd. (Consult Appendix.2 for more details about building a CD from scratch).

The live CD is usually created with a filesystem called squashfs. Squashfs is read only compressed filesystem that allow us to squeeze our system into a single CD. Note that your system has to be about 2GB (this might need some trial an error) to produce a compressed image that fits on the CD.




Background on live CD/DVD

Note: This section is a clarification of how live CD works. You don't have to read it. You can skip it if you want.

A live CD/DVD is basically a normal linux installation just like an ordinary harddrive installation. However, simply copying the harddirve installation over to a CD/DVD is not enough to produce a working system. Why? because there are still minor differences between a live CD/DVD and on ordinary harddrive installation. So in addition to copying our harddirve installation to the CD/DVD we must address those differences as well.

So what are these differences?

1. The CD or DVD is read only media. Linux needs to have write access to certain parts of the system to be able to operate properly (like "/dev" "/proc" "/var" "/tmp"). There are a lot of approaches to address this problem. All of which utilize the system RAM. Some of these approaches enable write access only to essential directories and files, and hence, they do not allow the user to modify the system or install new packages while in the live CD. Other approaches, like unionfs which is what is used in this guide, allows the user to write to any part of the system. This is done by merging part of the RAM with the read-only filesystem of the live CD/DVD and making the look like one filesystem that have read-write access. Unionfs has to be mounted at boot in a certain manner.
   
   
   
2. With the harddrive installation the location of the root filesystem is fixed. So it is passed to the kernel at boot time using the root=/dev/... parameter. With a live CD/DVD, the location of the root device is not fixed as the user might have multiple cdrom drives, these drives can be ide, scsi ... etc. So for the root filesystem to be mounted, there must be a way to identify the root device, and then we have to load the suitable kernel modules (to be able to access the cdrom controller as well as the CD filesystem). All this has to be done even before we have a root filesystem mounted.
   
   
   
   
3. To fit on a CD, the filesystem is usually compressed using squashfs. So we need to autodetect the filesystem type. We also need to have the proper modules for mounting it.

These considerations require special preparation at boot time, some of which must be performed even before mounting the actual filesystem. How can we do this?

Linux introduced a mechanism that allow for such preparations at boot time before the actual root filesystem is mounted. It is called the initial root filesystem or initramfs. This mechanism is used also in mounting normal harddirve installations, as it adds flexibilty to the boot process.


initramfs is virtual filesystem. It is a compressed cpio (cpio is an archive format similar to tar) archive that contains a minimal shell, kernel modules necessary for mounting the root filesystem and number of scripts that perform some tasks at boot time. The most important of these scripts is a script called init located at the root of the initramfs.

How does initramfs work?

The boot loader loads both the kernel and the initramfs into memory and starts the kernel. The kernel then unpacks the initramfs and mount it as initial root filesystem, and then looks for the init program within the initial filesystem, and once it finds it, it executes it and hand the boot process over to it. This init scirpt is responsible for finding the real root filesystem and mounting it. It is also responsible for any special preparations required at boot time.

So any special operations required for booting the system from live media can be coded into the initramfs boot scripts.

How is initramfs is created?

We do not have to create initramfs manually (although it can be done). There are tools for creating and updating initramfs like the command update-initramfs. Moreover, these tools can include custom scripts into the initramfs if they are placed in a certain preset locations (/usr/share/initramfs/scripts). So all we have to do is dump our custom scripts (which will do all the required preparation for booting the live CD/DVD) into these preset locations, and then create a custom initramfs by running update-initramfs.

We don't even have to write these scripts. Why? becuase there are packages that have scripts tailored for booting form live CDs. One of these packages is called casper (this is the package used in this howto). By installing casper into the system, it places the scripts in there proper locations (where they can be spotted by update-initrfamfs). The only thing we need to do after installing casper is running update-initramfs to create an initramfs suitable for live CD/DVD.



The live CD/DVD structure:

The directory tree of the live CD/DVD we are going to create is going to look like this:

Code:

(CD ROOT)
|-------+casper
| |-------filesystem.${FORMAT}
| |-------filesystem.manifest
| |-------filesystem.manifest-desktop
|
|-------+boot
| |--------+grub
| | |--------menu.lst
| | |--------stage2_eltorito
| |
| |-------vmlinuz
| |-------initrd.gz
| |-------memtest86+
|
|--------md5sum.txt

• /casper/filesystem.${FORMAT}: This is the container of the linux filesystem we are going to copy from our harddisk. It is usually a compressed filesystem like squahsfs.

• /casper/filesystem.manifest: This file is optional. You only need it if you decide to include the Ubuntu installer in the CD. The purpose of this file will be explained later.

• /casper/filesystem.manifest-desktop: This file is optional. You only need it if you decide to include the Ubuntu installer in the CD. The purpose of this file will be explained later.

• /boot/grub/menu.lst: File containing boot options for the live CD/DVD.

• /boot/grub/stage2_eltorito: The boot loader of the CD. (stage2 of grub).

• /boot/vmlinuz: The linux kernel. This is copied form the linux filesystem.

• /boot/initrd.gz: the initramfs that contain the customizations necessary for the live CD/DVD.

• /boot/memtest86+: Optional file used to test the RAM of the machine form the live CD/DVD.

• /md5sum.txt: Optional file containing checksums for all the files in the CD.

What you need:

• A working Debian or Ubuntu machine with internet access.

• CD/DVD Writer.

• Enough space in your harddirve. At least two times as much space as your installation size.

Outline of the steps:

A. Prepare Our work environment.

B. Copy the Source system to the target directory.
Note: People building a live CD/DVD from scratch using debootstrap: This step and replaced with the instructions listed here.

C. Chroot into the new system and make some modifications.

D. Prepare The CD directory tree.

E. Build the CD/DVD

Appendix 1. Adapting this guide to Debian.
Appendix 2. Building the live media form scratch using Debootstrap.


Conventions used in this HOWTO:

• Text highlighted in Magenta is meant to be replaced by the user's custom value.

• Commands performed within a chroot will be in Blue.

• Optional arguments or steps will be highlighted in Gray.

• Special notes and instructions for people building a live CD/DVD from scratch using debootstrap are highlighted in Green

• I will use gedit as my default text editor. Replace gedit with your favorite text editor.

A. Preparing the environment

1. Set some variables

Code:

export WORK=~/work
export CD=~/cd
export FORMAT=squashfs
export FS_DIR=casper

The WORK Directory is where our temporary files and mount point will reside.
The CD is the location of the CD tree.
FORMAT is the filesystem type. We you are going to use a compressed squashfs
FS_DIR is the location of the actual filesystem image within the cd tree.

Replace only the values highlighted in Magenta.


2. Create the CD and the WORK directory structure:

Code:

sudo mkdir -p ${CD}/{${FS_DIR},boot/grub} ${WORK}/rootfs



3. Install some packages on your current system:

Code:

sudo apt-get update


Code:

sudo apt-get install mkisofs grub squashfs-tools linux-ubuntu-modules-$(uname -r) qemu


qemu is optional. It is only needed for testing the cd before burning it. It can be substituted with any other virtualization software like virtualbox.

linux-ubuntu-modules-$(uname -r) is only needed for Ubuntu Gutsy and later. If using an Ubuntu version prior to Gutsy omit this package as it is part of the main kerenl package.


B. Copy your installation into the new filesystem.

Note: People building a live CD/DVD from scratch using debootstrap: skip this step and replace it with the instructions listed here.

Code:

sudo rsync -av --one-file-system --exclude=/proc/* --exclude=/dev/*\
--exclude=/sys/* --exclude=/tmp/* --exclude=/home/*\
--exclude=/lost+found / ${WORK}/rootfs


Note: rsync is used instead of cp to take advantage of the --one-file-system and the --exclude options.


If you have a separate boot partition you will have to copy it using the following command:

Code:

sudo cp -av /boot/* ${WORK}/rootfs/boot



(Optional) Copy settings in your home dir:


If you want to preseve your user account settings which are stored in your home directory, you can copy them to ${WORK}/rootfs/etc/skel/.

But first we have to define what files we want to copy. For example I am using xcfe4 as my DE, and it stores all it settings in a directory called .config in my home directory, so I am going to add .config to the variable $CONFIG:

Code:

CONFIG='.config .bashrc'

Now, Copy the CONFIG files using the following command:

Code:

cd ~ && for i in $CONFIG
do
sudo cp -rpv --parents $i ${WORK}/rootfs/etc/skel
done


C. Chroot into the new system and modify it:

1. Chroot into the copied system after mounting proc and dev:

Note: People building a live CD/DVD from scratch using debootstrap: skip this step.

Code:

sudo mount -o bind /dev/ ${WORK}/rootfs/dev


Code:

sudo mount -t proc proc ${WORK}/rootfs/proc


Code:

sudo chroot ${WORK}/rootfs /bin/bash

N.B: All commands in Blue are done within a chroot.

Now you are within chroot environment, type the following command:

Code:

LANG=



Note: People building a live CD/DVD from scratch using debootstrap: Resume the steps of the guide here.


2. Install Packages Essential for live CD:

Code:

apt-get update


Code:

apt-get install casper discover1 xresprobe


casper contain the live scirpts.
discover1 & xresprobe are used for autodetectin hardware at startup.


3. (Optional) If you want your live cd to have an installer, install the Ubuntu installer:

Code:

apt-get install ubiquity


Note: People using kde replace replace the previous command with

Code:

apt-get install ubiquity ubiquity-frontend-kde


Credit for this goes note to Fragadelic author of remastersys. Remastersys.



(Optional Step)Install any packages you want to be in the CD. Some of the following packages are useful in emergency situations:

Code:

sudo apt-get install gparted ms-sys testdisk wipe partimage xfsprogs reiserfsprogs jfsutils ntfs-3g ntfsprogs dosfstools mtools


gparted: patitioning tool. It is automatically installed as a dependecy of ubiquity.
ms-sys: writing a Microsoft compatible boot record (MBR).
testdisk: Partition scanner and disk recovery tool.
wipe: Secure file deletion.
partimage: backup partitions into a compressed image file (like norton ghost).
xfsprogs reiserfsprogs jfsutils: Tools for handling different filesystems.
mtools: Tools for manipulating MSDOS files


Note: People building a live CD/DVD from scratch using debootstrap: Additional step right here. Look in appendix.2 for details.



4. Update the initramfs:

Note: People building a live CD/DVD from scratch using debootstrap: Commands in this step are to be modified. Look in appendix.2 for details.


First update modules.dep:

Code:

depmod -a $(uname -r)

Code:

update-initramfs -u -k $(uname -r)


As already metioned above, the initramfs is reponsible for much of the preparation required at the boot time of the CD/DVD. The updated initramfs now contain the live scirpts installed with casper.


5. Delete these files.

Code:

for i in "/etc/hosts /etc/hostname /etc/resolv.conf /etc/timezone /etc/fstab /etc/mtab /etc/shadow /etc/shadow- /etc/gshadow /etc/gshadow- /etc/gdm/gdm-cdd.conf /etc/gdm/gdm.conf-custom /etc/X11/xorg.conf /boot/grub/menu.lst /boot/grub/device.map"
do
rm $i
done 2>/dev/null


These files are not needed in the CD/DVD. some of them are could interfer with the CD/DVD boot process. (e.g. shadow and gdm.conf-custom can interfere with autologin).


6. Clean apt cache

Code:

apt-get clean


7. Clean some dirs and files:

Code:

rm -r /tmp/* /root/* 2>/dev/null


Code:

rm /boot/*.bak 2>/dev/null


8. Remove non system users

Note: People building a live CD/DVD from scratch using debootstrap: skip this step.

Code:

for i in `cat /etc/passwd | awk -F":" '{print $1}'`
do
uid=`cat /etc/passwd | grep "^${i}:" | awk -F":" '{print $3}'`
[ "$uid" -gt "999" -a "$uid" -ne "65534" ] && userdel --force ${i} 2>/dev/null
done

Non-system users are users created by you that have user id more than 999.


9. Clean the chroot environment form unnecessary files:

Note: People building a live CD/DVD from scratch using debootstrap: skip this step.

Code:

find /var/run /var/log /var/mail /var/spool /var/lock /var/backups /var/tmp -type f -exec rm {} \;



10. If you are using GDM recreate it's config file:

Note: People building a live CD/DVD from scratch using debootstrap: skip this step.

Code:

[ -f "/etc/gdm/factory-gdm.conf" ] && cp -f /etc/gdm/factory-gdm.conf /etc/gdm/gdm.conf 2>/dev/null


Sometimes a customized /etc/gdm/gdm.conf can interfere with the live CD/DVD autologin.



11. Create some files in /var/log:

Note: People building a live CD/DVD from scratch using debootstrap: skip this step.

Code:

for i in dpkg.log lastlog mail.log syslog auth.log daemon.log faillog lpr.log mail.warn user.log boot debug mail.err messages wtmp bootstrap.log dmesg kern.log mail.info
do
touch /var/log/${i}
done


Most of these files are log files that have been cleaned in step 7. We created an empty files in their place to prevent the system from complaining at boot.


12. Exit chroot

Code:

exit

D. Prepare The CD directory tree:

1. Copy the kernel, the updated initrd and memtest prepared in the chroot:

Note: People building a live CD/DVD from scratch using debootstrap: Commands in this step are to be modified. Look in appendix.2 for details.

Code:

sudo cp -vp ${WORK}/rootfs/boot/vmlinuz-$(uname -r) ${CD}/boot/vmlinuz


Code:

sudo cp -vp ${WORK}/rootfs/boot/initrd.img-$(uname -r) ${CD}/boot/initrd.gz


Code:

sudo cp -vp ${WORK}/rootfs/boot/memtest86+.bin ${CD}/boot


2. Generate manifest:

Note: This step is only needed if you installed the Ubuntu installer ubiquity. This step generates two files (filesystem.manifest & filesystem.manifest-desktop).


Code:

sudo chroot ${WORK}/rootfs dpkg-query -W --showformat='${Package} ${Version}\n' | sudo tee ${CD}/${FS_DIR}/filesystem.manifest


Code:

sudo cp -v ${CD}/${FS_DIR}/filesystem.manifest{,-desktop}


Code:

REMOVE='ubiquity casper user-setup discover1 xresprobe os-prober libdebian-installer4'


Code;

for i in $REMOVE
do
sudo sed -i "/${i}/d" ${CD}/${FS_DIR}/filesystem.manifest-desktop
done


These two files are used by the ubiquity installer when installing to harddisk. These two files are just lists of packages. Ubiquity compares these two files and removes packages unique to filesystem.manifest. This way when installing to harddisk, packages like casper which is only useful in a live CD/DVD are removed. These packages that will be removed at install are defined in the variable $REMOVE


3. Unmount bind mounted dirs:


Code:

sudo umount ${WORK}/rootfs/proc


Code:

sudo umount ${WORK}/rootfs/sys


Code:

sudo umount ${WORK}/rootfs/dev




4. Convert the directory tree into a squashfs:

Code:

sudo mksquashfs ${WORK}/rootfs ${CD}/${FS_DIR}/filesystem.${FORMAT}


Note: Make sure the resulting file size can fit into your live media.


Note: Compression might incur a slight performace penalty. If you are using DVD and your overall system size is less than 4GB you don't have to use compression, instead you can use squashfs without compression by adding the -noI -noD -noF switches to mksquashfs.



5. Make Grub the bootloader of the CD

Copy grub file:

Code:

sudo find /boot /usr/lib/grub/ -iname 'stage2_eltorito' -exec cp -v {} ${CD}/boot/grub \;



Make the menu.lst

Code:

sudo gedit ${CD}/boot/grub/menu.lst

Copy the following text into it and save it.

Code:

# By default, boot the first entry.
default 0

# Boot automatically after 30 secs.
timeout 30

color cyan/blue white/blue


title Start Linux in Graphical Mode
kernel /boot/vmlinuz BOOT=casper boot=casper nopersistent rw quiet splash
initrd /boot/initrd.gz

title Start Linux in Safe Graphical Mode
kernel /boot/vmlinuz BOOT=casper boot=casper xforcevesa rw quiet splash
initrd /boot/initrd.gz

title Start Linux in Text Mode
kernel /boot/vmlinuz BOOT=casper boot=casper nopersistent textonly rw quiet
initrd /boot/initrd.gz

title Start Presistent Live CD
kernel /boot/vmlinuz BOOT=casper boot=casper persistent rw quiet splash
initrd /boot/initrd.gz

title Start Linux Graphical Mode from RAM
kernel /boot/vmlinuz BOOT=casper boot=casper toram nopersistent rw quiet splash
initrd /boot/initrd.gz

title Memory Test
kernel /boot/memtest86+.bin

title Boot the First Hard Disk
root (hd0)
chainloader +1



6. Calculate MD5

Code:

cd $CD && find . -type f -print0 | xargs -0 sudo md5sum | sudo tee ${CD}/md5sum.txt




E. Build the CD/DVD

1. Make the ISO file

Code:

sudo mkisofs -b boot/grub/stage2_eltorito \
-no-emul-boot -boot-load-size 4 -boot-info-table \
-V "Custom Live CD" -cache-inodes -r -J -l \
-o ~/live-cd.iso $CD



2. Test the CD

Test using qemu emulator

Code:

qemu -cdrom ~/live-cd.iso -boot d


Or use any other virtualization program you like.


Update: As noted by az in this post, while testing the iso with qemu sometimes it drops you to an initramfs shell because of a problem with qemu. This behaviour has been confirmed by other users. In this case it is advisable to retest the iso with another virtualization software like virtualbox or to burn the iso to REWRITABLE cd and test directly on your pc.


3. (Optional) Clean our workspace

Code:

[ -d "$WORK" ] && rm -r $WORK $CD




Final Notes:

• You can add a splash image to the grub menu.

• If you are using a custom kernel make sure it has support for the following:
  
  
  1. Support of loopback device.
  2. Support for the filesystem format you are using (e.g. squashfs ).
  3. Support for unionfs.
  4. Support for initramfs.

• There are some extra options I put in the grub menu. The ones that I have not tried are highlighted in Red below:
  
  
  
  1. Start linux form RAM. This option is only possible if your ram is larger than data on the live media. This option can be useful if you are building a minimal command line rescue disc as it would enhance performance to start it from RAM.
     
     
  2. Start in presistent mode. To learn about it more look here.
     
     
  3. Start Linux in Text Mode. This will not start X. The user will be autologged into a virtual terminal (the kind of terminal you get when you press Alt+Ctrl+F1). Note that this option will not work in all Ubuntu versions prior to Gutsy.

Appendix 1. Adapting this guide to Debian

This guide can be modified to apply to Debian systems as well. There are number of differeces between Ubuntu and Debian that we must take into account:

1. As of Debian lenny, casper is deprecated and replaced with live-initramfs. live-initramfs is a fork of casper and it has the same options, with one difference in the CD directory tree sturcute. This can be solved by setting the variable FS_DIR=live instead of FS_DIR=casper.
   
2. Sqaushfs modules and unionfs modules are in two separate packages.
   
3. Ubiquity installer is not present in the Debain repositories. Ubiquity is only needed if you intend to install Linux from the live CD/DVD to the harddisk. I have not tried using ubiquity on Debian so I am not sure if it will work. To install it on Debian you have to add Ubuntu main repository to your sources.list.

So in light of the points mentioned above we have to make the following modifications to adapt the guide to Debian:
In step A.1 replace FS_DIR=casper with


Code:

FS_DIR=live


Replace the command in Step A.3 with:

Code:

sudo apt-get install mkisofs grub squashfs-tools squashfs-modules-$(uname -r) qemu


Replace the command in Step C.2 with:

Code:

apt-get install live-initramfs unionfs-modules-$(uname -r) discover1 xresprobe


Skip step C.3 and D.2 if you do not intend to try ubiquity on Debian.

In Step D.5 Replace every occurence of BOOT=casper and boot=casper in menu.lst with BOOT=live and boot=live respectively




Appendix 2. Building the live media form scratch using debootstrap.

Instead of using your current installation to be the basis of you live CD, you can build a custom system from scratch into any directory in your system using debootstrap, and then use that as the basis of your CD. The modifications you have to make are:

• skip step B alltogether. Instead, do the instructions listed here to build your custom system from scratch using debootstarp

• after finishing the instructions of the guide mentioned above, you resume the steps in this guide, going straight to step C.2 (skip step C.1).

Before step C.4 set the following variable:

Code:

export kversion=`cd /boot && ls vmlinuz-* | sed 's@vmlinuz-@@'`


Modify the two commands in step C.4 so they look lik this:

Code:

depmod -a ${kversion}

Code:

update-initramfs -u -k ${kversion}



Skip the following steps: C8, C9, C10, C11

Modify the commands in Step D.1 as follows:

Code:

find ${WORK}/rootfs/boot -iname 'vmlinuz*' -exec sudo cp -vp {} ${CD}/boot/vmlinuz \;


Code:

find ${WORK}/rootfs/boot -iname 'initrd.img*' -exec sudo cp -vp {} ${CD}/boot/initrd.gz \;


Code:

sudo cp -vp ${WORK}/rootfs/boot/memtest86+.bin ${CD}/boot


dit (1): 20/02/08

1. Added Suggestions for packages useful in rescue CD.
2. Modified step D.5 as per RumorsOfWar suggestion.

Update (2): 14/03/08

Fragadelic kindly posted this guide on his website. Fragadelic is the author of remastersys. Remastersys is a tool that can create a live CD/DVD in an automated manner as opposed to the step by step nature of this guide. Another advantage of remastersys is that it has a GUI for those who do not want to miss with the command line.


Edit (3): 12/04/08
Correcting a typo in step E.2 as pointed out by vbgeek


Edit (4): 21/04/08
Removed live-initramfs from the variable REMOVE (in step D.2) as pointed out by gjhicks


Edit (5): 05/05/08
Removed the appendix of how to make the cd using a unix filesystem like ext2. This is now obsolete as you can use squashfs without compression by adding the -noI -noD -noF switches to mksquashfs.

---

Last edited by capink; May 13th, 2008 at 07:27 AM.

Taken From: http://ubuntuforums.org/showthread.php?t=688872