Category Archives: Unixy

Ubuntu USB Ethernet Naming for Alias and VLAN Interfaces

In my ongoing adventures of playing with the Chromebox as a firewall, I eventually found a reason to use stock Ubuntu 16.04 instead of VyOS.

For the most part, devices just work as you’d expect under Ubuntu. Even my USB ethernet network interface (NIC) devices work fine. But I eventually noticed a problem.

By default USB NICs under Ubuntu 16.04 (which uses systemd & udev) are named “enxABCDEF123456” where ABCDEF123456 is the full MAC address of the device.

This is fine for basic usage, but problems occur when you want to use device aliasing or VLAN tagging.

As an example, a standard USB interface definition would look like this in /etc/network/interfaces:

iface enxABCDEF123456 inet static
 address 192.168.1.20
 netmask 255.255.255.0

Linux interface aliasing lets you add another IP to the same device like by adding another entry with a colon plus integer (eg, :1 or :10). The aliased device can be treated as a unique device for ifup/ifdown, etc.  It looks like this:

iface enxABCDEF123456:1 inet static
 address 192.168.1.35
 netmask 255.255.255.0

You can add VLAN tagging to an interface very simply in a similar way. It’s done by adding   a dot/period plus integer (eg, :100 or :201) and referencing the parent “raw” device. The aliased device can be treated as a unique device for ifup/ifdown, etc.  It looks like this:

iface enxABCDEF123456.201 inet static
    vlan-raw-device enxABCDEF123456
    address 192.168.2.20
    netmask 255.255.255.0

Looks good, right? Nope! Guess what, this isn’t going to work as is!

If you actually try to bring up one of these devices, you’ll see this.

$ sudo ifup enxABCDEF123456.201
RTNETLINK answers: Numerical result out of range
Failed to bring up enxABCDEF123456.201.

Turns out this is due to the (very long) length of the device name. Actually, someone already filed a bug for this with Ubuntu: https://bugs.launchpad.net/ubuntu/+source/systemd/+bug/1567744

 

So, what to do? There’s a reasonably simple solution, but it took some searching to figure it out.

 

First, if you just read docs about systemd you’ll see a seemingly “obvious” way to name devices by mapping the MAC address. This creates a file which should rename the device enxABCDEF123456 to enx0.

cat > /etc/systemd/network/10-enx0.link << EOF
[Match]
MACAddress=AB:CD:EF:12:34:56

[Link]
Name=enx0
EOF

It seems good, but doesn’t work by itself.. the trick is that udev is also at work tweaking the NIC device names.

Per the systemd man page: https://www.freedesktop.org/software/systemd/man/systemd.link.html

All link files are collectively sorted and processed in lexical order, regardless of the directories in which they live. However, files with identical filenames replace each other. Files in /etc have the highest priority, files in /run take precedence over files with the same name in /usr/lib. This can be used to override a system-supplied link file with a local file if needed. As a special case, an empty file (file size 0) or symlink with the same name pointing to /dev/null disables the configuration file entirely (it is “masked”).

This gave me the idea that masked files may also be at play with udev…. the file which directs udev to name USB devices with full MAC address is /lib/udev/rules.d/73-usb-net-by-mac.rules  so I’ll attempt to mask it in in /etc.

ln -s /dev/null /etc/udev/rules.d/73-usb-net-by-mac.rules
update-initramfs -u
reboot

Note that udev and systemd rules,  need to be in the initfamfs, so that has to be updated before rebooting for the changes to take effect.

IT WORKS!

Oh, of course, now if I was depending on that USB NIC, I just lost networking because my /etc/network/interfaces is invalid. So based on the examples above, here’s a snapshot of what the that file should now look like:

iface enx0 inet static
 address 192.168.1.20
 netmask 255.255.255.0

iface enx0:1 inet static
 address 192.168.1.35
 netmask 255.255.255.0

iface enx0.201 inet static
 vlan-raw-device enx0
 address 192.168.2.20
 netmask 255.255.255.0

Now, the base USB interface, the alias interface, and the VLAN interface should all work!

ESXi on an Asus Chromebox M004U

I recently stumbled across the ASUS CHROMEBOX-M004U. It’s a Chromebox, which is cool in its own right, but I was interested in using it’s dual-core Celeron Haswell CPU, expandable RAM (up to 16GB), and M2 Sata storage for other purposes.

First things first, you’ve got to enable the box to run things other than ChromeOS… Thankfully that’s well documented ( http://kodi.wiki/view/ASUS_Chromebox ), so I won’t go into detail at this time. But I will say, I used the standalone, custom coreboot firmware and chose the headless option, as my current planned use cases are for servers. Apparently you want non-headless if you plan to use the box as a Plex Home Theater or Kodi box. Oh, also, I set it to boot first from USB if  available.

Now the Chromebox will boot happily from it’s internal SSD or USB if attached.

Time for ESXi! I haven’t been blogging much in the past couple years, but I have been doing a lot of home work with virtualization, ESXi, vSPhere, ZFS, SmartOS, OmniOS… so at this point I’ve got a bunch of crazy tech projects going on at home.

First, you need an ESXi installer ISO image… I’m using ESXi 5.5u2  and it’s freely available from VMware…https://my.vmware.com/web/vmware/evalcenter You probably have to register to get it. 6.0 is out now, but I haven’t made that jump. However, while 6.0 DOES have USB3/xhci driver, even the latest ISO for 5.5 does not… Even though the matrix of VIB versions per release https://docs.google.com/spreadsheets/d/10Vzx4NLhx1XzhmS-hQuIO1wXa98_8EKN3bhXjFx2GgQ shows later versions which have it. So we’ll have to add that/upgrade after install. (Also, this is helpful information about that matrix: http://www.v-front.de/2012/11/are-esxi-5x-patches-cumulative.html)

Second, we need to customize our installer to have non-standard packages for this machine.

I’ve been using ESXi Customer: https://vibsdepot.v-front.de/wiki/index.php/ESXi-Customizer

And, you’ll need to get some custom VIBs (drivers/configs)… you need the following 2, and you want the offline-bundle versions

To customize, you use a source ISO and add an offline bundle… ESXi Customer only does one VIB at a time… so after it gives you one customized ISO, rename it, then use it as the source, and add the second VIB.

Now, you are ready to install…  We will install to a different, better supported machine, ensuring your install is good to go, then move to the Chromebox. 

Using your custom ISO burned to a USB flash drive (various tools to do that, on windows I use Rufus), install to a target USB flash drive at least 8GB (not the one you are installing from). Installing can take some time, so be patient. Also, make sure you only install to the desired, target flash drive, not your hard drive or something. 🙂

Once installed, remove the installer flash drive, and boot from the target ESXi flash drive. Again, it takes a while.

Now we have a few configuration steps.

First, we’ll use F2, to customize the system… navigate to troubleshooting options… press enter on “Enable SSH” ( screenshots here: http://www.thomasmaurer.ch/2012/09/activate-ssh-on-vmware-esxi-5-1/ )
For the rest of the steps, SSH into your ESXi machine, it should be displaying it’s IP on the screen.

Second, we need to fix a network setting. By default, the vmk0 (virtual NIC created on which the DHCP client runs) will do a one-time clone of the MAC HW address of your system’s NIC, so you’ll need to fix this, else when booting the USB in a different machine, the installation will boot with the original machine’s MAC… it will CONFLICT with your original machine, you’ll have network issues, including using same IP if you have DHCP static addresses, etc.

esxcfg-advcfg -s 1 /Net/FollowHardwareMac

more detail from: http://kb.vmware.com/selfservice/microsites/search.do?language=en_US&cmd=displayKC&externalId=1031111

Third, you need to install and make sure that USB3/xhci gets loaded at boot…

esxcli network firewall ruleset set -e true -r httpClient
esxcli software profile update -d https://hostupdate.vmware.com/software/VUM/PRODUCTION/main/vmw-depot-index.xml -p ESXi-5.5.0-20141004001-standard
esxcli system module set -e true -m xhci

and add following to /etc/rc.local.d/local.sh  :

vmkload_mod xhci

more detail from: http://www.v-front.de/2014/11/vmware-silently-adds-native-usb-30.html

Finally, you should be able to shutdown, remove the USB flash drive with ESXi on it,  put into Chromebox, and boot. And it should come up with it’s own HW MAC, IP, etc, have access to update its own config on its USB drive, and be able to use an attached keyboard.

SSH to new ESXi IP and/or use the vSphere Client to manage as normal.

Viagra 50mg vs levitra 20mg Killing “find” Errors

I’ve used the unix filesystem search utility find for many years. Though, like most things, until forced to learn its deeper secrets, I generally get by with only the most basic knowledge.

One of the cool things about find is that you can specify a search and then execute an action on the results, all in one command.

This example is probably my most common use of find:

find . -type d -name .svn -exec rm -fr {} \;

This means: find, in the current directory (.), a directory (-type d), named .svn (-name .svn), and for every result (-exec) remove that directory (rm -fr {}) .  The “{}” represents the matched path string, and the “\;” is required to end the “-exec” command.

Here’s a sample of what this looks like, including output:

$ find . -type d -name .svn -exec rm -fr {} \;
find: ./.svn: No such file or directory
find: ./getid3-2.0.0b4/.svn: No such file or directory
find: ./getid3-2.0.0b4/extras/.svn: No such file or directory
find: ./getid3-2.0.0b4/getid3/.svn: No such file or directory

This is great, and I’ve used this exact syntax for years. But today I ran into a problem. I wanted to run this exact command as part of a custom build script in Xcode. When this command ran I had 81 errors popup in my build results! What happened is that all thos “No such file or directory” messages are actually errors, and Xcode reported them as such.

The solution is to add one extra argument: -depth . This causes find to do a depth-first traversal of the sub-directories being searched. That is, find will check the contents of directories before acting (eg, running an -exec command) on the directory in question. The default is to act on the directory (in our case, removing it) before attempting to visit it’s contents. So after we removed the directory, find was still trying to look at it; -depth fixes that.

So, the final answer is, I now use:

find . -type d -name .svn -depth -exec rm -fr {} \;

Yes, this is a slightly verbose explanation for something so simple, but maybe it will help someone else.

Foxmarks: Bookmark Synchronization Heaven

Long have I searched for the magic bullet solution to my bookmark synchronization woes.  I’ve wanted a simple plugin that would synchronize my bookmarks between multiple installations of Firefox and Safari, thus making it simple to access said bookmarks from any computer, or between the two commonly used browsers on my Mac.  I’ve looked at many options, but always the solution only allows me to sync one browser or the other, leaving me looking for some secondary sync tool to get between Firefox and Safari on my Mac itself, not via network or a proper sync.

I’d almost given up on finding a solution, then, a few months ago I started using Delicious. It was cool because there were plugins for Safari, Firefox, and IE, and of course, it’s by default a web based bookmarking system. Its cool, I like it, but I just didn’t use it much. The plugins integrate it into the browser by giving you ANOTHER bookmarks menu, not by integrating with the browsers’ bookmark system.

Before ever using Safari or Delicious, my Firefox bookmark sync tool of choice was Foxmarks. It provided a web interface for remote access to my bookmarks, plus a nice sync interface for all my Firefox installations. I was randomly poking around today and discovered that Foxmarks now works with Safari and IE! I was excited and wasted no time installing Foxmarks for Safari. So far, it works great!

For the most part, it works just as you’d expect, bookmarks sync between all my Safari(Mac only, for now, I think) and Firefox(Mac, Linux, and Windows) installations without hassle. I’ve yet to try out the Internet Explorer functionality, but I’m guessing it works pretty well. I just don’t use IE enough to care.

One caveat to be aware of: both Firefox and Safari use some browser specific URL syntax to access internal functionality for recent bookmarks, etc. That stuff will only work on the browser where it was created, Safari on Safari, Firefox on Firefox, etc. For me, that’s a non issue, I rarely use those features. I have tested and confirmed that javascript bookmarklets (like for Tinyurl and Cornify) do seem to work after syncing. Those are about the only reason I use on the bookmark menubar.

Use vi key bindings in bash

A long time ago I used ksh with vi key bindings, and life was good.

Then I moved on to bash, but for some reason, I never investigated using vi key bindings. I simply lived with the defaults (which, for the record, are emacs-like key bindings).

So, just the other day I said to myself, “Self, I want to use vi key bindings in bash. I want to again experience the joy of traversing and editing my command line in COMMAND MODE. I want the speed and the power of my precious vi (well, I use vim) at my finger tips. And I NO LONGER want to waste time holding arrow keys or to think about using emacs-like commands.”

So, I fired up google.com; low and behold I stumbled onto this little post about using vi key bindings in bash and zsh. So sweet!

In a nutshell, the bash command to enable vi mode is:

set -o vi

This can be set in your .bashrc file, and if it doesn’t pickup when you start a new terminal session, add something like this to your .profile or .bash_profile:

if [ -f ~/.bashrc ]; then
. ~/.bashrc
fi

With vi mode enabled, you’ll start your bash session in insert mode, so things should behave as normal. But, to turn on the power, just hit the ESC key to enter COMMAND MODE. 🙂 Now all your vi commands are availble. Move to end of line with “$”, beginning of line with “^”, delete a word with “dw”, etc.

Enjoy!

Bonjour Avahi Addendum

A while back I wrote about advertising Linux services via Avahi/Bonjour. Since then I’ve made a few changes to my setup.

First, I nixed netatalk for direct AFP support. My primary reason for using it was to gain a more Mac-like network filesystem which would make Time Machine happier. Well, Time Machine uses a sparse bundle disk image on it’s target; after learning about that, using AFP seemed a bit unnecessary. Also, Samba CIFS/SMB seemed to perform better. I don’t have solid benchmarks for this, but simple file copies seemed to be consistenly faster with Samba. One of the biggest annoyances about netatalk was all the extra hidden files and folders it created. I run a hybrid network, I have more Mac machines, but also Windows, plus I browse file systems on the command line quite often; and those excess files pushed me over the edge.

Second, I nixed Time Machine. Just when I thought everything was working perfectly, it completely blew up and could no longer access its data store. Not good for a backup solution. I plan to write about my new home backup solution sometime, but it’s basically rsync with a few key points.

Continue reading Bonjour Avahi Addendum

Advertising Linux Services via Avahi/Bonjour

Update: most of this information is still correct but an update for combining service definitions into one file and setting an icon is available here: https://holyarmy.org/2008/11/bonjour-avahi-addendum

In my last post I outlined how I followed others’ directions to enable netatalk on Linux and Time Machine backups to a shared AFP folder. Originally, I also described how to put all your shares on netatalk. I suppose if only have Mac clients or you REALLY want to use AFP, you can do so. As I worked with files over AFP shares, I started noticing that the performance seemed to be quite bad. No, I didn’t benchmark, but copying large video files to a shared folder over my gigabit network was substantially slower over AFP (netatalk) than over CIFS/SMB (samba). I use my network shares pretty heavily, so this was a concern. Also, netatalk tries very hard to replicate an HFS filesystem complete with resource fork support. This means that your shared directories end up with lots of extra folders named “.AppleDouble”(and a few others) containing Mac specific info. (Note: even on CIFS you’ll get the “.AppleDB” folders unless you disable a setting in Finder. I can deal with .AppleDB better than .AppleDouble AND .AppleDB) So, because of these two issues I decided to try using CIFS and samba again.
Continue reading Advertising Linux Services via Avahi/Bonjour

Time Machine backup to Linux via Netatalk

So, when I got the upgrade from Tiger to Leopard on my MacBook Pro, I was looking for a good backup solution. I’ve used rsync in the past, but when I saw that Apple had a new Time Machine backup tool, I was curious to give it a shot. The catch is you basically needed an external USB or Firewire drive, until they recently came out with the Time Capsule. Anyway, tonight I got the itch to really see if I could make Time Machine work without buying extra hardware. I mean, seriously, I’ve got a good hunk of mirrored disk sitting on my home server; that seems like a good place to do backups.
Some googling found me this link to a blogger who’d done it!
I’ll make my own version of this post, since I had a few differences from the original I where I found the info.

Continue reading Time Machine backup to Linux via Netatalk