This is a bit of a follow up from my last post. I’ve still got my Asus Chromebox. I’m not running ESXi any more… but something I do want to try out is VyOS. Why? Well, my original intent when purchasing the Chromebox was to play around with it, but then I really did expect it to replace my home router. I’ve used pfSense for a long time, but I’m curious about VyOS because I know several people using EdgeRouter Lite which uses EdgeOS, a fork of VyOS. So I wanted to play around on existing hardware.
Note: everything below assumes your Chromebox has custom firmware for installing any OS you choose (see last post).
Turns out though, VyOS isn’t quite ready for installing on UEFI systems. And the Chromebox is… well… sort of UEFI, even though it uses SeaBIOS. Also, while most UEFI systems have a management screen where you can configure legacy mode, the Chromebox is a bit more limited in how we can work around its oddities.
Bottom line, VyOS installer would not boot directly on the Chromebox. I wasn’t thinking “UEFI is the problem” when I first hit the boot issue; I assumed it was an issue with USB 3. After much poking and prodding, I was able to get it running. But it was tricky enough, I figured I should document the process.
First things first, you’ll need another system, a BIOS system, on which you can install VyOS. This doesn’t have to be a physical computer. VyOS is very often used in virtualized environments. In my case, all initial testing was done on a physical BIOS machine, but for proving out a second round of tests, and to aid in documentation for this blog, I did another install using a VMware machine for my starting system.
Using VMware (etc), it’s super easy to install. I used a VM with 1GB of RAM and 10GB of hard disk. I removed most extra devices, as we don’t need them, but I did keep the USB controller. Note: 10GB hard drive was chosen to be sure it is smaller than the 16GB SSD in the Chromebox; it may not matter in the long run but makes me feel safer.
Once you have a VM created, and the ISO downloaded, boot and follow the VyOS install guide.
- login as vyos
- run install image
- use default config
- set password for vyos user to something
- auto partition disk sda using whole 10GB of disk
- install grub to sda
Reboot to make sure it boots. Huzzah!
We aren’t going to clone in a traditional sense, no Ghost or Clonezilla… nope. We’re just going to take the bits we need. First, we need a rescue disk to boot from. I’m a long time fan of SystemRescueCD as it has lots and lots of useful tools, can cache itself 100% into RAM, and has a decent X Windows setup available if you need it.
So, download the latest ISO of SystemRescueCD. Before we go any further, make a bootable USB thumb drive from this. The easiest way is to be running Linux already, mount the ISO image, and run the usb_inst.sh script. On Windows you can try using Rufus, which I use when on Windows. On OSX, well… dd might work.
OK, set the SystemRescueUSB aside for now…
Copy the Disk
Now we are ready to clone the disk. Configure the ISO image as your bootable drive in your VyOS VM, then boot it up. You may need to be quick on the “ESC” key to get into the boot menu. The default SystemRescueCD boot option should be fine.
Once you are sitting at a shell prompt, insert that SystemRescueUSB drive you made. You’ll want to make sure it’s connected to the VM, not your host system.
[email protected] /root % fdisk -l /dev/sdb Disk /dev/sdb: 15 GiB, 16079781888 bytes, 31405824 sectors Units: sectors of 1 * 512 = 512 bytes Sector size (logical/physical): 512 bytes / 512 bytes I/O size (minimum/optimal): 512 bytes / 512 bytes Disklabel type: dos Disk identifier: 0x0002f767 Device Boot Start End Sectors Size Id Type /dev/sdb1 * 1 31405823 31405823 15G c W95 FAT32 (LBA)
It’s connected, so mount it up, and create a place to store our VyOS stuff
mkdir /mnt/usb && \ mount /dev/sdb1 /mnt/usb && \ mkdir /mnt/usb/VYOS && \ cd /mnt/usb && \ ls boot bootprog ntpasswd syslinux sysrcd.md5 usb_inst.sh version bootdisk efi readme.txt sysrcd.dat usb_inst usbstick.htm VYOS
Now, copy out the MBR, which holds both the partition table and the bootloader.
dd if=/dev/sda of=/mnt/usb/VYOS/mbr.img bs=512 count=1 1+0 records in 1+0 records out 512 bytes (512 B) copied, 0.00109355 s, 468 kB/s
Sweet… now make a copy of the filesystem.
mkdir /mnt/disk && \ mount /dev/sda1 /mnt/disk && \ cd /mnt/disk && \ tar -zcf /mnt/usb/VYOS/1.1.7/rootfs.tar.gz boot && \ ls -lh /mnt/usb/VYOS total 233M -rwxr-xr-x 1 root root 512 Aug 4 04:47 mbr.img -rwxr-xr-x 1 root root 232M Aug 4 04:50 rootfs.tar.gz
Excellent! sync for good measure, then unmount and remove the USB drive.
sync && umount /mnt/usb
We are done with the VM for now, but it doesn’t hurt to leave it around in case you want to inspect something.
Paste the Disk
Now, lets pull out the physical Chromebox.
Insert the SystemRescueUSB drive and power it up. Again the the default SystemRescueCD boot option is fine, but if you choose the second (docache), you’ll need to manually mount the USB drive. Using the default, our USB drive is now auto-mounted to /livemnt/boot .
We can verify, since a stock SystemRescueCD won’t have our VYOS stuff.
ls -lh /livemnt/boot/VYOS total 233M -rw-r--r-- 1 root root 512 Aug 4 04:47 mbr.img -rw-r--r-- 1 root root 232M Aug 4 04:50 rootfs.tar.gz
ALERT: NEXT STEP WILL ERASE EVERYTHING ON YOUR CHROMEBOX SSD!
First, nuke existing partition maps. You’ll have to tell it you want a DOS disk label, then arrow over to “Write” to save the partition map.
cfdisk -z /dev/sda
Restore the VM’s MBR.
dd if=/livemnt/boot/VYOS/mbr.img of=/dev/sda bs=512 count=1 1+0 records in 1+0 records out 512 bytes (512 B) copied, 0.00135381 s, 378 kB/s fdisk -l /dev/sda Disk /dev/sda: 14.9 GiB, 16013942784 bytes, 31277232 sectors Units: sectors of 1 * 512 = 512 bytes Sector size (logical/physical): 512 bytes / 512 bytes I/O size (minimum/optimal): 512 bytes / 512 bytes Disklabel type: dos Disk identifier: 0x000ce790 Device Boot Start End Sectors Size Id Type /dev/sda1 * 2048 20971519 20969472 10G 83 Linux
Create a filesystem on this partition. Note: I had some trouble with SystemRescueCD’s ext4 fs utils creating a version of ext4 that was too new for the older utils/kernel of VyOS, so I went simple here and used ext3.
mkfs.ext3 -j /dev/sda1 mke2fs 1.42.13 (17-May-2015) /dev/sda1 contains a file system Proceed anyway? (y,n) y Discarding device blocks: done Creating filesystem with 2621184 4k blocks and 655360 inodes Filesystem UUID: d2e311fc-bf9e-4f40-b88a-a9047c969255 Superblock backups stored on blocks: 32768, 98304, 163840, 229376, 294912, 819200, 884736, 1605632 Allocating group tables: done Writing inode tables: done Creating journal (32768 blocks): done Writing superblocks and filesystem accounting information: done
Lay down our copy of the VM filesystem.
mkdir /mnt/disk && \ mount /dev/sda1 /mnt/disk && \ cd /mnt/disk && \ tar zxf /livemnt/boot/VYOS/rootfs.tar.gz && \ ls boot lost+found
So far this has been pretty simple way to copy any Linux file system and partition map. Now its time to do the magic required to get this to boot on the Chromebox.
If you were to reboot right now, you’d get a lovely grub rescue> console, and you wouldn’t be able to boot VyOS.
Convert to GPT Magic
First, unmount that disk again, so we are able to work with it.
cd /root && umount /mnt/disk
Use gdisk to write GPT version of the MBR data we currently have. Choose “1” to use MBR as authoritative ( you MAY not be asked this ), then “w” to write the new GPT data.
gdisk /dev/sda GPT fdisk (gdisk) version 1.0.1 Partition table scan: MBR: MBR only BSD: not present APM: not present GPT: present Found valid MBR and GPT. Which do you want to use? 1 - MBR 2 - GPT 3 - Create blank GPT Your answer: 1 Command (? for help): w Final checks complete. About to write GPT data. THIS WILL OVERWRITE EXISTING PARTITIONS!! Do you want to proceed? (Y/N): y OK; writing new GUID partition table (GPT) to /dev/sda. The operation has completed successfully.
Now we need to add an ESP (EFI System Partition) and BIOS compatibility boot/grub partition.
parted /dev/sda unit MiB print Model: ATA SanDisk SSD U110 (scsi) Disk /dev/sda: 15272MiB Sector size (logical/physical): 512B/512B Partition Table: gpt Disk Flags: Number Start End Size File system Name Flags 1 1.00MiB 10240MiB 10239MiB ext3 Linux filesystem
This shows us where we can start adding partitions, 10239MiB. So we add to next mega byte location the ESP, then one more for our BIOS boot part.
parted /dev/sda mkpart primary fat32 10240MiB 10241MiB Information: You may need to update /etc/fstab. parted /dev/sda mkpart primary fat32 10241MiB 10441MiB Information: You may need to update /etc/fstab. parted /dev/sda set 2 bios_grub on set 2 msftdata off Information: You may need to update /etc/fstab. parted /dev/sda set 3 boot on set 3 msftdata off set 3 esp on Information: You may need to update /etc/fstab. parted /dev/sda unit MiB print Model: ATA SanDisk SSD U110 (scsi) Disk /dev/sda: 15272MiB Sector size (logical/physical): 512B/512B Partition Table: gpt Disk Flags: Number Start End Size File system Name Flags 1 1.00MiB 10240MiB 10239MiB ext3 Linux filesystem 2 10240MiB 10241MiB 1.00MiB primary bios_grub 3 10241MiB 10442MiB 201MiB fat16 primary boot, esp mkfs.vfat /dev/sda2 mkfs.fat 3.0.28 (2015-05-16) mkfs.vfat /dev/sda3 mkfs.fat 3.0.28 (2015-05-16)
Now we need to unsquash the filesystem image used by VyOS so we can re-install the right version of grub.
mount /dev/sda1 /mnt/disk && \ cd /mnt/disk && \ unsquashfs boot/1.1.7/1.1.7.squashfs Parallel unsquashfs: Using 2 processors 32494 inodes (34238 blocks) to write [===============================================================================================================================-] 34238/34238 100% created 30398 files created 9824 directories created 1457 symlinks created 37 devices created 1 fifos ls squashfs-root bin boot dev etc home lib lib64 media mnt opt proc root sbin selinux srv sys tmp usr var
Let’s bind some system dirs and install grub (the version from VyOS, not SystemRescueCD)!
mount -o bind /dev /mnt/disk/squashfs-root/dev mount -o bind /proc /mnt/disk/squashfs-root/proc mount -o bind /mnt/disk/boot /mnt/disk/squashfs-root/boot chroot /mnt/disk/squashfs-root /bin/bash # grub-install /dev/sda Installation finished. No error reported.
PHEW! That wasn’t so hard, was it? 🙂
Let’s cleanup the mounts, reboot, and see what happens.
umount squashfs-root/boot umount squashfs-root/proc umount squashfs-root/dev cd .. && umount disk reboot
Don’t forget you’ll need to remove the USB drive before rebooting.
When booting the Chromebox I now get a VyOS boot and login.