How to boot Ubuntu on a Cr-48 Part 2

[javajolt]

◄ return to part 1

Acquire an Ubuntu filesystem

First, although the Cr-48 CPU and BIOS is 64-bit, the kernel and rootfs are presently 32-bit. Since we're reusing the Chrome OS kernel, we'll download and use the 32-bit ubuntu-10.10-desktop-i386.iso from an Ubuntu CD mirror.

For example:

wget [url=http://mirror.anl.gov/pub/ubuntu-iso/CDs/10.10/ubuntu-10.10-desktop-i386.iso]http://mirror.anl.gov/pub/ubuntu-iso/CDs/10.10/ubuntu-10.10-desktop-i386.iso[/url]

Second, even in developer mode, the Cr-48 will only boot external drives that are signed by Google. That means we can't (easily) use the normal Ubuntu live installer. Instead, I'll create an empty disk image file with a 5G partition on it, and use VirtualBox to install into that. Then I'll transfer that raw partition onto the Cr-48.

To install VirtualBox (and its command line management tool, VBoxManage) on Ubuntu:

sudo aptitude install virtualbox-ose

I'll do this on my normal linux workstation, of course. I'll create the file inside the chroot so I can use the cgpt tool, but I'll run VirtualBox from outside. We could create the file using fdisk, but using cgpt ensures all the numbers match. You could also build the cgpt tool for use external to the chroot, but whatever.

We'll need our disk image to be a little larger than 5G so it has room for the GPT headers.

So, do the following in the Chromium OS chroot:

cd /tmp
rm -rf test.bin
dd if=/dev/zero bs=512 seek=10486015 count=1 of=test.bin
cgpt create test.bin
cgpt boot -p test.bin
cgpt add -b 128 -s 10485760 -t data -l ubuntu test.bin

From outside the chroot, translate the binary into a virtualbox disk:

cd /path/to/chromiumos/chroot/tmp
VBoxManage convertdd test.bin test.vdi --format VDI

Launch the VirtualBox GUI (virtualbox) and create a new virtual machine with 2G of RAM. Configure test.vdi as the existing hard disk (SATA port 0), and attach the Ubuntu .iso as the IDE secondary master. Everything else can be left as default.

Boot the virtual machine and install Ubuntu onto /dev/sda1. The Chrome OS kernel does not use swap memory, so be sure to specify partitions manually. Select the middle /dev/sda1, and format it for ext2, and mount on /. The ubuntu installer should warn you that you have no swap partition, and it will therefore disable swap.  This is what we want.

Once Ubuntu is installed, working and updated, shut the virtual machine down and convert the .vdi image back into a binary:

VBoxManage internalcommands converttoraw test.vdi test_new.bin

Now, we'll extract just the Ubuntu rootfs partition from the binary file:

dd if=test_new.bin bs=512 skip=128 count=10485760 of=rootfs.bin

Next, we copy the ubuntu rootfs directly to the Cr-48 ROOT-C partition (/dev/sda7).  To copy the file, run the following command on the Cr-48, where USER@HOST refers to the account and host from which you are copying. 

Note: the root filesystem is a huge file that will take quite a while to upload.  This is much less painful with a fast network connection, so, if possible, use a USB-to-ethernet adapter to avoid going through Wi-Fi.

ssh USER@HOST cat /path/to/rootfs.bin | dd of=/dev/sda7

You can see that it worked by mounting the new rootfs and looking around:

mkdir /tmp/urfs
mount /dev/sda7 /tmp/urfs
ls /tmp/urfs

While we're looking, let's copy the cgpt tool into Ubuntu's rootfs. Make sure it's executable. We'll need this later.

cp /usr/bin/cgpt /tmp/urfs/usr/bin/
chmod a+rx /tmp/urfs/usr/bin/cgpt

We'll need to copy all the kernel modules too.

cd /lib/modules
cp -ar * /tmp/urfs/lib/modules/

That should do it.

umount /tmp/urfs

The next step is to copy the currently running Chrome OS kernel to KERN-C (/dev/sda6).  Remember, an image consists of a kernel and a rootfs on consecutive partitions.  So, to figure out which kernel is running, we can check which partition is currently mounted as the rootfs:

rootdev -s

If this prints /dev/sda3 (ROOT-A), then our kernel is on /dev/sda2 (KERN-A).
If this prints /dev/sda5 (ROOT-B), then our kernel is on /dev/sda4 (KERN-B).
Assuming our kernel is on /dev/sda2 (KERN-A), copy it to /dev/sda6 (KERN-C):

dd if=/dev/sda2 of=/dev/sda6

Now we need to change the kernel command line to use our Ubuntu rootfs instead of a Chrome OS rootfs. For this, we'll use make_dev_ssd.sh, located in the Chromium OS source tree under src/platform/vboot_reference/scripts/image_signing/.  The latest version has options to change individual kernel command lines.

Use scp to copy it from your host to the stateful partition of the Cr-48. Note we'll also need common.sh from the same directory.

cd /mnt/stateful_partition
scp USER@HOST:/some/path/to/latest/make_dev_ssd.sh .
scp USER@HOST:/some/path/to/latest/common.sh .

Extract the existing kernel command line from KERN-C, and save it to a file named foo.6 (the .6 extension is added by the script):

sh ./make_dev_ssd.sh --partitions '6' --save_config foo

The default Chrome OS kernel command line looks something like this:

quiet console=tty2 init=/sbin/init add_efi_memmap boot=local rootwait ro noresume noswap i915.modeset=1 loglevel=1 cros_secure kern_guid=%U tpm_tis.force=1 tpm_tis.interrupts=0 root=/dev/dm-0 dm_verity.error_behavior=3 dm_verity.max_bios=-1 dm_verity.dev_wait=1 dm="vroot none ro,0 1740800 verity /dev/sd%D%P /dev/sd%D%P 1740800 1 sha1 50adbfb72bb1efda0c1a86dcd1c1d6a0b46726d1" noinitrd

The only editor on the Cr-48 is qemacs, so open foo.6 with qemacs:

qemacs foo.6

Edit the file to look like this:

console=tty1 init=/sbin/init add_efi_memmap boot=local rootwait ro noresume noswap i915.modeset=1 loglevel=7 kern_guid=%U tpm_tis.force=1 tpm_tis.interrupts=0 root=/dev/sda7 noinitrd

Then save (Ctrl-x Ctrl-s) and exit (Ctrl-x Ctrl-c). qemacs occasionally gets confused, so double-check to be sure you have the right content:

cat foo.6

Finally, use make_dev_ssd.sh to replace the kernel command line in KERN-C:

sh ./make_dev_ssd.sh --partitions '6' --set_config foo

The kernel command line is part of the kernel partition, and the entire partition must be cryptographically signed in order to work. We can't replace the command line part without affecting the entire kernel partition. The script will save a backup copy before replacing the partition content, but we shouldn't need it.

Set boot priority

At this point we should have the following situation:

Image-A is an official Google Chrome OS which can boot either in normal mode or dev-mode.
Image-B is (or will be after the first autoupdate) another official Google Chrome OS which can boot either in normal mode or dev-mode.
Image-C is Chrome OS kernel with a modified command line and an Ubuntu rootfs, which can only boot in dev-mode.

Next, we adjust the priority among the images so we can try out our Ubuntu image. The image priority is an attribute of its kernel partition.  Run cgpt show /dev/sda again, to see the kernel priorities:

localhost ~ # cgpt show /dev/sda
...
      4096     32768       2  Label: "KERN-A"
                              Type: ChromeOS kernel
                              UUID: D176DC60-81F1-654E-8953-E3D28019738C
                              Attr: priority=3 tries=0 successful=1
...
     36864     32768       4  Label: "KERN-B"
                              Type: ChromeOS kernel
                              UUID: F1A2C65C-CC22-FF4A-A8BC-67BA233F3D40
                              Attr: priority=0 tries=15 successful=0
...
  12369920     32768       6  Label: "KERN-C"
                              Type: ChromeOS kernel
                              UUID: B6954485-4295-9749-956A-C315B01FB684
                              Attr: priority=0 tries=15 successful=0

The priority determines the order in which the BIOS tries to find a valid kernel (bigger is higher, zero means don't even try). The tries field is decremented each time the BIOS tries to boot it, and if it's zero, the kernel is considered invalid (this lets us boot new images without looping forever if they don't work). The successful field overrides the tries field, but is only set by the OS once it's up and running.

Let's change the priority of KERN-C to 5:

cgpt add -i 6 -P 5 -T 1 -S 0 /dev/sda

This makes KERN-C the highest priority, but only gives us one chance to boot it. That way if it doesn't work, we're not completely stuck.

If you reboot now, you should come up in Ubuntu!  Note that Computer Science Standard Answer #1 applies: It Works For Me™

If something went wrong and Ubuntu crashes or you powered off, the tries field for KERN-C will have been decremented to 0 and you'll fall back to booting Chrome OS.

Assuming that Ubuntu booted and you could log in, go to Applications->Accessories->Terminal to get a shell, and run

sudo cgpt add -i 6 -P 5 -S 1 /dev/sda

This will mark the Ubuntu kernel as valid, so it will continue to boot next time.

Now you can switch back and forth between the official Chrome OS release and Ubuntu just by flipping the dev-mode switch. Going from dev-mode to normal mode erases STATE (/dev/sda1), but much more quickly. Going from normal to dev-mode again would normally do a slow erase of /dev/sda1, but since we're booting Ubuntu that doesn't happen.

This works because although KERN-C has the highest priority, it isn't signed by Google. In dev-mode that's okay, but in normal mode it will be rejected by the BIOS. Since we've set the successful flag to 1, the BIOS won't mark it invalid but will just skip it each time. This makes the normal-mode boot time slightly longer, but only by a second or two.

Of course you could also switch between images from within dev-mode just by manually setting the priorities with cgpt before rebooting.

Note that if the normal image autoupdates, it will probably change the kernel priorities so that Image-C is no longer the highest and the next time you switch to dev mode, you will
  a) have a long wait,
  b) still be running Chrome OS, and
  c) have to use cgpt to raise the KERN-C priority again.

But, because autoupdate only switches between Image-A and Image-B, the Ubuntu kernel and rootfs shouldn't be affected.

Footnotes

Ugh

Recompiling the Ubuntu kernel should be possible, with a few caveats:

First, you'll have to modify or reconfigure the kernel to not use initrd. initramfs should work though.

Second, official Chrome OS still has a few closed-source hardware drivers.  The pure open-source Chromium OS experience is therefore a bit sub-optimal, although still quite useable.

Third, if you use the 32-bit image, you'll also have to modify the kernel source to boot correctly from the 64-bit Chrome OS BIOS.
See http://git.chromium.org/cgi-bin/gitweb.cgi?p=kernel.git;a=commit;h=a7cfa1075c04df162d58dc2ed93df6045e9c3271 for details.

If you use a 64-bit image it shouldn't need the boot hacks, but driver support is much less robust.

Finally, you'll have to sign the kernel yourself and put it into the KERN-C partition. That is actually the easiest step, but it's not really well documented. Look in the developer pages.

Double-ugh

initrd is typically handled by the bootloader, which reads the specified image from the disk into RAM and passes the address to the kernel as it's invoked.

The Chrome OS BIOS is a modified EFI BIOS. The bootstub is a standard EFI Application, but it's embedded in the kernel image in a dedicated partition type, rather than accessible through a FAT filesystem. To decrease boot time, the BIOS does not discover or pass the standard disk drive handles to the bootstub, so the bootstub doesn't know anything about disks or filesystems. There is also no Compatibility Support Module in the BIOS. In theory elilo or grub2 could replace the bootstub, but they would have to reimplement some of the device discovery functions normally done by an EFI BIOS.

If you want to take this on, go for it. That would let us create a kernel partition that just contained an EFI bootloader, which could then chain-boot to external USB drives, etc. That might be kind of cool.

end