Category Archives: Networks

VyOS on an Asus Chromebox M004U

This is a bit of a follow up from my last post. I’ve still got my Asus Chromebox.  I’m not running ESXi any more… but something I do want to try out is VyOS. Why? Well, my original intent when purchasing the Chromebox was to play around with it, but then I really did expect it to replace my home router. I’ve used pfSense for a long time, but I’m curious about VyOS because I know several people using EdgeRouter Lite which uses EdgeOS, a fork of VyOS. So I wanted to play around on existing hardware.

Note: everything below assumes your Chromebox has custom firmware for installing any OS you choose (see last post).

Turns out though, VyOS isn’t quite ready for installing on UEFI systems. And the Chromebox is… well… sort of UEFI, even though it uses SeaBIOS. Also, while most UEFI systems have a management screen where you can configure legacy mode, the Chromebox is a bit more limited in how we can work around its oddities.

Bottom line, VyOS installer would not boot directly on the Chromebox. I wasn’t thinking “UEFI is the problem” when I first hit the boot issue; I assumed it was an issue with USB 3. After much poking and prodding, I was able to get it running. But it was tricky enough, I figured I should document the process.

Install VyOS

First things first, you’ll need another system, a BIOS system, on which you can install VyOS. This doesn’t have to be a physical computer. VyOS is very often used in virtualized environments. In my case, all initial testing was done on a physical BIOS machine, but for proving out a second round of tests, and to aid in documentation for this blog, I did another install using a VMware machine for my starting system.

I used both the 1.1.7 stable and 1.2.0-beta1 releases for my testing, but always 64-bit ISO installations. I didn’t use any pre-made images.

Using VMware (etc), it’s super easy to install. I used a VM with 1GB of RAM and 10GB of hard disk. I removed most extra devices, as we don’t need them, but I did keep the USB controller. Note: 10GB hard drive was chosen to be sure it is smaller than the 16GB SSD in the Chromebox; it may not matter in the long run but makes me feel safer.

Once you have a VM created, and the ISO downloaded, boot and follow the VyOS install guide.

Quick summary:

  1. boot
  2. login as vyos
  3. run install image
  4. use default config
  5. set password for vyos user to something
  6. auto partition disk sda using whole 10GB of disk
  7. install grub to sda
  8. DONE

Reboot to make sure it boots. Huzzah!

SystemRescueCD

We aren’t going to clone in a traditional sense, no Ghost or Clonezilla… nope. We’re just going to take the bits we need. First, we need a rescue disk to boot from. I’m a long time fan of SystemRescueCD as it has lots and lots of useful tools, can cache itself 100% into RAM, and has a decent X Windows setup available if you need it.

So, download the latest ISO of SystemRescueCD. Before we go any further, make a bootable USB thumb drive from this. The easiest way is to be running Linux already, mount the ISO image, and run the usb_inst.sh script. On Windows you can try using Rufus, which I use when on Windows. On OSX, well… dd might work.

OK, set the SystemRescueUSB aside for now…

Copy the Disk

Now we are ready to clone the disk. Configure the ISO image as your bootable drive in your VyOS VM, then boot it up. You may need to be quick on the “ESC” key to get into the boot menu. The default SystemRescueCD boot option should be fine.

Once you are sitting at a shell prompt, insert that SystemRescueUSB drive you made. You’ll want to make sure it’s connected to the VM, not your host system.

[email protected] /root % fdisk -l /dev/sdb
Disk /dev/sdb: 15 GiB, 16079781888 bytes, 31405824 sectors
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disklabel type: dos
Disk identifier: 0x0002f767

Device Boot Start End Sectors Size Id Type
/dev/sdb1 * 1 31405823 31405823 15G c W95 FAT32 (LBA)

It’s connected, so mount it up, and create a place to store our VyOS stuff

mkdir /mnt/usb && \
mount /dev/sdb1 /mnt/usb && \
mkdir /mnt/usb/VYOS && \
cd /mnt/usb && \
ls
boot bootprog ntpasswd syslinux sysrcd.md5 usb_inst.sh version
bootdisk efi readme.txt sysrcd.dat usb_inst usbstick.htm VYOS

Now, copy out the MBR, which holds both the partition table and the bootloader.

dd if=/dev/sda of=/mnt/usb/VYOS/mbr.img bs=512 count=1
1+0 records in
1+0 records out
512 bytes (512 B) copied, 0.00109355 s, 468 kB/s

Sweet… now make a copy of the filesystem.

mkdir /mnt/disk && \
mount /dev/sda1 /mnt/disk && \
cd /mnt/disk && \
tar -zcf /mnt/usb/VYOS/1.1.7/rootfs.tar.gz boot && \
ls -lh /mnt/usb/VYOS 
total 233M
-rwxr-xr-x 1 root root 512 Aug 4 04:47 mbr.img
-rwxr-xr-x 1 root root 232M Aug 4 04:50 rootfs.tar.gz

Excellent! sync for good measure, then unmount and remove the USB drive.

sync && umount /mnt/usb

We are done with the VM for now, but it doesn’t hurt to leave it around in case you want to inspect something.

Paste the Disk

Now, lets pull out the physical Chromebox.

Insert the SystemRescueUSB drive and power it up. Again the the default SystemRescueCD boot option is fine, but if you choose the second (docache), you’ll need to manually mount the USB drive. Using the default, our USB drive is now auto-mounted to /livemnt/boot .

We can verify, since a stock SystemRescueCD won’t have our VYOS stuff.

ls -lh /livemnt/boot/VYOS
total 233M
-rw-r--r-- 1 root root 512 Aug 4 04:47 mbr.img
-rw-r--r-- 1 root root 232M Aug 4 04:50 rootfs.tar.gz

ALERT: NEXT STEP WILL ERASE EVERYTHING ON YOUR CHROMEBOX SSD!

First, nuke existing partition maps. You’ll have to tell it you want a DOS disk label, then arrow over to “Write” to save the partition map.

cfdisk -z /dev/sda

Restore the VM’s MBR.

dd if=/livemnt/boot/VYOS/mbr.img of=/dev/sda bs=512 count=1
1+0 records in
1+0 records out
512 bytes (512 B) copied, 0.00135381 s, 378 kB/s
fdisk -l /dev/sda
Disk /dev/sda: 14.9 GiB, 16013942784 bytes, 31277232 sectors
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disklabel type: dos
Disk identifier: 0x000ce790

Device Boot Start End Sectors Size Id Type
/dev/sda1 * 2048 20971519 20969472 10G 83 Linux

Create a filesystem on this partition. Note: I had some trouble with SystemRescueCD’s ext4 fs utils creating a version of ext4 that was too new for the older utils/kernel of VyOS, so I went simple here and used ext3.

mkfs.ext3 -j /dev/sda1
mke2fs 1.42.13 (17-May-2015)
/dev/sda1 contains a file system
Proceed anyway? (y,n) y
Discarding device blocks: done 
Creating filesystem with 2621184 4k blocks and 655360 inodes
Filesystem UUID: d2e311fc-bf9e-4f40-b88a-a9047c969255
Superblock backups stored on blocks: 
 32768, 98304, 163840, 229376, 294912, 819200, 884736, 1605632

Allocating group tables: done 
Writing inode tables: done 
Creating journal (32768 blocks): done
Writing superblocks and filesystem accounting information: done

Lay down our copy of the VM filesystem.

mkdir /mnt/disk && \
mount /dev/sda1 /mnt/disk && \
cd /mnt/disk && \
tar zxf /livemnt/boot/VYOS/rootfs.tar.gz && \
ls
boot lost+found

So far this has been pretty simple way to copy any Linux file system and partition map. Now its time to do the magic required to get this to boot on the Chromebox.

If you were to reboot right now, you’d get a lovely grub rescue> console, and you wouldn’t be able to boot VyOS.

Convert to GPT Magic

First, unmount that disk again, so we are able to work with it.

cd /root && umount /mnt/disk

Use gdisk to write GPT version of the MBR data we currently have. Choose “1” to use MBR as authoritative ( you MAY not be asked this ), then “w” to write the new GPT data.

gdisk /dev/sda
GPT fdisk (gdisk) version 1.0.1

Partition table scan:
 MBR: MBR only
 BSD: not present
 APM: not present
 GPT: present

Found valid MBR and GPT. Which do you want to use?
 1 - MBR
 2 - GPT
 3 - Create blank GPT

Your answer: 1

Command (? for help): w

Final checks complete. About to write GPT data. THIS WILL OVERWRITE EXISTING
PARTITIONS!!

Do you want to proceed? (Y/N): y
OK; writing new GUID partition table (GPT) to /dev/sda.
The operation has completed successfully.

Now we need to add an ESP (EFI System Partition) and BIOS compatibility boot/grub partition.

parted /dev/sda unit MiB print
Model: ATA SanDisk SSD U110 (scsi)
Disk /dev/sda: 15272MiB
Sector size (logical/physical): 512B/512B
Partition Table: gpt
Disk Flags:

Number Start End Size File system Name Flags
 1 1.00MiB 10240MiB 10239MiB ext3 Linux filesystem

This shows us where we can start adding partitions, 10239MiB.  So we add to next mega byte location the ESP, then one more for our BIOS boot part.

parted /dev/sda mkpart primary fat32 10240MiB 10241MiB 
Information: You may need to update /etc/fstab.

parted /dev/sda mkpart primary fat32 10241MiB 10441MiB 
Information: You may need to update /etc/fstab.

parted /dev/sda set 2 bios_grub on set 2 msftdata off 
Information: You may need to update /etc/fstab.

parted /dev/sda set 3 boot on set 3 msftdata off set 3 esp on
Information: You may need to update /etc/fstab.

parted /dev/sda unit MiB print 
Model: ATA SanDisk SSD U110 (scsi)
Disk /dev/sda: 15272MiB
Sector size (logical/physical): 512B/512B
Partition Table: gpt
Disk Flags: 

Number Start End Size File system Name Flags
 1 1.00MiB  10240MiB 10239MiB ext3  Linux filesystem
 2 10240MiB 10241MiB 1.00MiB        primary    bios_grub
 3 10241MiB 10442MiB 201MiB   fat16 primary    boot, esp

mkfs.vfat /dev/sda2
mkfs.fat 3.0.28 (2015-05-16)

mkfs.vfat /dev/sda3
mkfs.fat 3.0.28 (2015-05-16)

Install Grub

Now we need to unsquash the filesystem image used by VyOS so we can re-install the right version of grub.

mount /dev/sda1 /mnt/disk && \
cd /mnt/disk && \
unsquashfs boot/1.1.7/1.1.7.squashfs 
Parallel unsquashfs: Using 2 processors
32494 inodes (34238 blocks) to write

[===============================================================================================================================-] 34238/34238 100%

created 30398 files
created 9824 directories
created 1457 symlinks
created 37 devices
created 1 fifos

ls squashfs-root
bin boot dev etc home lib lib64 media mnt opt proc root sbin selinux srv sys tmp usr var

Let’s bind some system dirs and install grub (the version from VyOS, not SystemRescueCD)!

mount -o bind /dev /mnt/disk/squashfs-root/dev
mount -o bind /proc /mnt/disk/squashfs-root/proc
mount -o bind /mnt/disk/boot /mnt/disk/squashfs-root/boot 
chroot /mnt/disk/squashfs-root /bin/bash
# grub-install /dev/sda
Installation finished. No error reported.

PHEW! That wasn’t so hard, was it? 🙂

Let’s cleanup the mounts, reboot, and see what happens.

umount squashfs-root/boot
umount squashfs-root/proc
umount squashfs-root/dev 
cd .. && umount disk
reboot

SUCCESS!

Don’t forget you’ll need to remove the USB drive before rebooting.

When booting the Chromebox I now get a VyOS boot and login.

ESXi on an Asus Chromebox M004U

I recently stumbled across the ASUS CHROMEBOX-M004U. It’s a Chromebox, which is cool in its own right, but I was interested in using it’s dual-core Celeron Haswell CPU, expandable RAM (up to 16GB), and M2 Sata storage for other purposes.

First things first, you’ve got to enable the box to run things other than ChromeOS… Thankfully that’s well documented ( http://kodi.wiki/view/ASUS_Chromebox ), so I won’t go into detail at this time. But I will say, I used the standalone, custom coreboot firmware and chose the headless option, as my current planned use cases are for servers. Apparently you want non-headless if you plan to use the box as a Plex Home Theater or Kodi box. Oh, also, I set it to boot first from USB if  available.

Now the Chromebox will boot happily from it’s internal SSD or USB if attached.

Time for ESXi! I haven’t been blogging much in the past couple years, but I have been doing a lot of home work with virtualization, ESXi, vSPhere, ZFS, SmartOS, OmniOS… so at this point I’ve got a bunch of crazy tech projects going on at home.

First, you need an ESXi installer ISO image… I’m using ESXi 5.5u2  and it’s freely available from VMware…https://my.vmware.com/web/vmware/evalcenter You probably have to register to get it. 6.0 is out now, but I haven’t made that jump. However, while 6.0 DOES have USB3/xhci driver, even the latest ISO for 5.5 does not… Even though the matrix of VIB versions per release https://docs.google.com/spreadsheets/d/10Vzx4NLhx1XzhmS-hQuIO1wXa98_8EKN3bhXjFx2GgQ shows later versions which have it. So we’ll have to add that/upgrade after install. (Also, this is helpful information about that matrix: http://www.v-front.de/2012/11/are-esxi-5x-patches-cumulative.html)

Second, we need to customize our installer to have non-standard packages for this machine.

I’ve been using ESXi Customer: https://vibsdepot.v-front.de/wiki/index.php/ESXi-Customizer

And, you’ll need to get some custom VIBs (drivers/configs)… you need the following 2, and you want the offline-bundle versions

To customize, you use a source ISO and add an offline bundle… ESXi Customer only does one VIB at a time… so after it gives you one customized ISO, rename it, then use it as the source, and add the second VIB.

Now, you are ready to install…  We will install to a different, better supported machine, ensuring your install is good to go, then move to the Chromebox. 

Using your custom ISO burned to a USB flash drive (various tools to do that, on windows I use Rufus), install to a target USB flash drive at least 8GB (not the one you are installing from). Installing can take some time, so be patient. Also, make sure you only install to the desired, target flash drive, not your hard drive or something. 🙂

Once installed, remove the installer flash drive, and boot from the target ESXi flash drive. Again, it takes a while.

Now we have a few configuration steps.

First, we’ll use F2, to customize the system… navigate to troubleshooting options… press enter on “Enable SSH” ( screenshots here: http://www.thomasmaurer.ch/2012/09/activate-ssh-on-vmware-esxi-5-1/ )
For the rest of the steps, SSH into your ESXi machine, it should be displaying it’s IP on the screen.

Second, we need to fix a network setting. By default, the vmk0 (virtual NIC created on which the DHCP client runs) will do a one-time clone of the MAC HW address of your system’s NIC, so you’ll need to fix this, else when booting the USB in a different machine, the installation will boot with the original machine’s MAC… it will CONFLICT with your original machine, you’ll have network issues, including using same IP if you have DHCP static addresses, etc.

esxcfg-advcfg -s 1 /Net/FollowHardwareMac

more detail from: http://kb.vmware.com/selfservice/microsites/search.do?language=en_US&cmd=displayKC&externalId=1031111

Third, you need to install and make sure that USB3/xhci gets loaded at boot…

esxcli network firewall ruleset set -e true -r httpClient
esxcli software profile update -d https://hostupdate.vmware.com/software/VUM/PRODUCTION/main/vmw-depot-index.xml -p ESXi-5.5.0-20141004001-standard
esxcli system module set -e true -m xhci

and add following to /etc/rc.local.d/local.sh  :

vmkload_mod xhci

more detail from: http://www.v-front.de/2014/11/vmware-silently-adds-native-usb-30.html

Finally, you should be able to shutdown, remove the USB flash drive with ESXi on it,  put into Chromebox, and boot. And it should come up with it’s own HW MAC, IP, etc, have access to update its own config on its USB drive, and be able to use an attached keyboard.

SSH to new ESXi IP and/or use the vSphere Client to manage as normal.

VPN on Ubuntu Linux with Juniper Network Connect

There’s one standard document on HOWTO get Network Connect working on Ubuntu Linux. It’s mad scientist’s doc: http://mad-scientist.us/juniper.html . However, there are a few things not covered. I’ll assume that you’ve followed mad scientist’s excellent guide before going any further.

Issue #1: 64-bit Ubuntu

By default, when you install java on your 64-bit system, you get a 64-bit java. No surprise there, right? Well, Juniper’s tools don’t play nice with 64-bit java. If you attempt to start the junipernc script you’ll promptly see the “VPN has failed!” error message.

VPN has failed!
VPN has failed!

Also if you look closely in your Terminal you’ll see the text error:

Failed to load the ncui library.

This is the clue that we are dealing with the 64-bit issue.

The work around for this is to install a 32-bit java on your system. Type the following into your Terminal:

sudo apt-get install ia32-sun-java6-bin

After typing your password, a 32-bit copy of java will be installed at: /usr/lib/jvm/ia32-java-6-sun .

Now, you need to convince Juniper Network Connect to use the 32-bit java. If you don’t use java for much besides your new VPN, you may just want to make the 32-bit java your default. This can be done by typing the following into your Terminal:

update-alternatives --set java /usr/lib/jvm/ia32-java-6-sun/jre/bin/java

If you DO use java and just want to tell the VPN to use the 32-bit java, you should modify the junipernc by adding the following line right after the block of lines that start with “#”:

export JDK_HOME=/usr/lib/jvm/ia32-java-6-sun

Now, when you run junipernc, it will use 32-bit java and you should no longer have the failure due to ncui.

Issue #2: Determining Your Realm

The scripting for Network Connect asks a few questions that may not make sense to a typical user. Even a networking savvy programmer may not be certain what values to use for the “Realm” or “PIN + SecureID Code”.

Finding your realm is fairly straight forward if you don’t mind diving into some HTML. Point your web browser to your company’s VPN website: https://vpn.mycompany.com or https://connect.mycompany.com .   View the source of that page and look for a line like:

<input type="hidden" name="realm" value="REALMNAME">

The value of REALMNAME is what you’ll need to enter when prompted.  Your IT department may or may not know what this is if you ask them.

If you don’t know your “PIN + SecureID Code”, it’s simply the password you type along with your username on the VPN website to gain access. As mad scientist says, some companies use “SecurID so [they] enter a personal PIN plus the value provided by the SecurID fob,” which explains why he coded that as the prompt for the password input.

If you need help, there’s a long running thread over at the ubuntu forums where this continues to be discussed a lot: http://ubuntuforums.org/showthread.php?t=232607 . I gathered my info from both mad scientist’s page above and the thread itself.

One further note, I’ve tested this on Ubuntu 9.04 64-bit as well as 8.10 32-bit. Hope this is helpful to all you who need Juniper VPN access on 64-bit Ubuntu Linux.

Fight Back! (When VPN Clients Mis-Behave)

I have to use VPNs at work. Specifically, to access my production webservers (etc), I have to use a Cisco VPN client. Sadly, the VPN concentrator overrides my choice of allowing local LAN access. So, when I am on the VPN, I have my DNS options changed so I can’t use any local servers. This is a serious, serious pain. So painful in fact, that many times instead of fight with it, I simply would run a Windows session in VMware (on my Mac) and connect the VPN there. This has drawbacks too, but it’s better than not having local network access.

So I set out to find a solution and I found a post by loudhush which described using the scutil to modify DNS network settings after connecting to a Cisco VPN. This was great, but I needed something a bit handier.

So, I cranked out the following which goes in my /Users/username/.profile:

# .profile or .bash_profile
function myvpn {
vpnclient connect VPNPROFILENAME user MYVPNUSERNAME
myworkdns
}
function myworkdns {
printf "get State:/Network/Service/com.cisco.VPN/DNS\nd.add ServerAddresses * 192.168.1.252, 192.168.1.198\nd.add SearchDomains * example.com, other.example.com\nset State:/Network/Service/com.cisco.VPN/DNS" | sudo scutil
}

These are bash functions which i run from the command line. (I also find the Client GUI Cisco to be a pain, and prefer command line)

So, obviously, you’ll need to substitute in your Cisco VPN profile name ( found in /etc/opt/cisco-vpnclient/Profiles), your VPN username, your DNS server IP addresses, and your DNS search domains to your legitimate values.

To use, run Terminal, then type myvpn. The VPN client will prompt you for your username and password. You’ll then have to hit CTRL+Z to suspend the VPN client so the script can run the DNS updates; this part uses sudo to run the command as root, so you will probably need to type your Mac password immediately after hitting CTRL+Z. If you didn’t want to bother with the command line VPN client, you could just use your GUI Cisco VPN client, then run myworkdns from Terminal, which will still probably prompt you for your Mac password.

Hope others find this useful. If I find a cleaner way, I’ll post that too.

Investigating OpenID

Aaron (one of my co-workers), recently posted a link about OpenID. I’ve given OpenID only cursory glances over the last year, but the Coding Horror link in Aaron’s post had a comment to this Google Video where Simon Willison gives a Google Tech Talk on The Implications of OpenID. The video is nearly a year old, but to date, it’s done more to convince me to get on the OpenID bandwagon than anything else.

Advertising Linux Services via Avahi/Bonjour

Update: most of this information is still correct but an update for combining service definitions into one file and setting an icon is available here: http://holyarmy.org/2008/11/bonjour-avahi-addendum

In my last post I outlined how I followed others’ directions to enable netatalk on Linux and Time Machine backups to a shared AFP folder. Originally, I also described how to put all your shares on netatalk. I suppose if only have Mac clients or you REALLY want to use AFP, you can do so. As I worked with files over AFP shares, I started noticing that the performance seemed to be quite bad. No, I didn’t benchmark, but copying large video files to a shared folder over my gigabit network was substantially slower over AFP (netatalk) than over CIFS/SMB (samba). I use my network shares pretty heavily, so this was a concern. Also, netatalk tries very hard to replicate an HFS filesystem complete with resource fork support. This means that your shared directories end up with lots of extra folders named “.AppleDouble”(and a few others) containing Mac specific info. (Note: even on CIFS you’ll get the “.AppleDB” folders unless you disable a setting in Finder. I can deal with .AppleDB better than .AppleDouble AND .AppleDB) So, because of these two issues I decided to try using CIFS and samba again.
Continue reading Advertising Linux Services via Avahi/Bonjour

Time Machine backup to Linux via Netatalk

So, when I got the upgrade from Tiger to Leopard on my MacBook Pro, I was looking for a good backup solution. I’ve used rsync in the past, but when I saw that Apple had a new Time Machine backup tool, I was curious to give it a shot. The catch is you basically needed an external USB or Firewire drive, until they recently came out with the Time Capsule. Anyway, tonight I got the itch to really see if I could make Time Machine work without buying extra hardware. I mean, seriously, I’ve got a good hunk of mirrored disk sitting on my home server; that seems like a good place to do backups.
Some googling found me this link to a blogger who’d done it!
I’ll make my own version of this post, since I had a few differences from the original I where I found the info.

Continue reading Time Machine backup to Linux via Netatalk

Network Directory Services

Network directory services are core to Internet functionality. The Domain Name System (DNS) provides a global (and/or local) directory of hosts and services. Lightweight Directory Access Protocol (LDAP) servers can provide some of the same information as DNS (or be used to back DNS), but are more frequently used to create network user databases, store user group information, providing centralized account information and password storage.

I recently completed an upgrade of these two core services on a network I manage. We had been running outdated (but functional) BIND v8 and OpenLDAP v2.0 instances for of DNS and LDAP servers. Also, throw a Windows Server 2003 into the mix, which, as an Active Directory domain controller has to run its own DNS and LDAP (AD is tweaked LDAP) servers.

Continue reading Network Directory Services