UniFi v5.6.x Goes Testing!

UniFi 5.6.10 Testing has dropped. I haven’t been paying much attention to the v5.6.x Unstable releases as I’d been waiting for v5.5.x to become Stable, but now that we’re here… Here’s a run down of the differences I see from v5.5.19 -> v5.6.10.

USG Properties

First and foremost, a bunch of things have been removed from the Properties pane for the USG. Rejoice! All of that stuff has been moved to various places with Settings.

The Advanced section offers lots of new goodness, including control over Offload features.

Site

Over in Site settings, we now have control over SSH access. Previously SSH was always enabled.

Wireless Networks

Fast Roaming for 802.11r devices is now available. Use at your own risk.

Networks

Networks brings much more control over DHCP. Highlights are DHCP Relay, being able to specify a gateway other than the USG, and control over various settings used for PXE and VOIP booting. It’s not everything we’ve always wanted but it’s a great first step.

Firewall Settings

Firewall has a new Settings tab with all sorts of knobs to tweak.

Port Forwarding

And Port Forwarding has been moved to the Routing & Firewall section.

admins

More control over user permissions are now available.

logging

Fine-grained control over logging.

And the next several shots show the various new tabs under Services.

DHCP Relay

DDNS

mdns

upnp

Life with a pair of USGs

The past fall I took an interest in Ubiquiti’s UniFi routing and switching product lines. It started with getting some US-16-XG switches from their Beta program and I immediately fell in love with UniFi. Aside from providing real slick management of features like VLANs and LACP, the Clients and Insights features within UniFi make it real easy to figure out what’s attached where on my network.

There’s just one glaring omission with the switching product range:

UniFi switches don’t natively support inter-VLAN routing.

The UniFi way of routing between VLANs is to use a UniFi Security Gateway. Using an external router is not ideal for a network with much inter-VLAN traffic, and for my home network I was tempted to use a virtualized pfSense router to maintain > 1GB/s inter-VLAN speeds, but I saw great potential value in deploying UniFi-managed routers for small offices. Before recommending them to anyone else I figured I should dogfood them for a while. I bought a USG-PRO-4 for my home office plus a USG 3-port for my girlfriend’s apartment, and ran them from December through June.


Swapping in the USG Pro for my EdgeRouter 8, the first problem I ran into is that there’s no way in the UniFi Controller to override the device’s WAN MAC address. Lots of ISP CPEs lock on to a router’s MAC address and won’t accept DHCP requests from a new device without jumping through some hoops. Most any router you would pick up from Best Buy supports changing the MAC address.

Now, the USG runs EdgeOS, the CLI of EdgeOS supports specifying MAC addresses for interfaces, and you absolutely can get at the CLI of a USG. But… anything you commit will not be saved back to the UniFi Controller and will be lost when the device re-provisions.

The process of persisting CLI changes on the USG is to put them in a special file on the UniFi Controller, /usr/lib/unifi/data/sites/site-code/config.gateway.json. The format of this file looks similar to the format an EdgeRouter configuration file, except that it’s JSON.

Here’s a fragment from an EdgeRouter:

interfaces {
    ethernet eth2 {
        mac 00:0D:3A:27:02:78
    }

And an equivalent from a USG Pro:

{
        "interfaces": {
                "ethernet": {
                        "eth2": {
                                "mac": "00:0D:3A:27:02:78"
                        }
                }
        }
}

Now, this JSON format isn’t what you get when you type show. You have to exit from configure, then run mca-ctrl -t dump. Figure out which fragments you need, put them in config.gateway.json, and then tear out your hair when it doesn’t work because what you made wasn’t actually valid JSON. You missed the opening or closing braces. Or that options taking multiple values need brackets and commas. Or forgot to quote something.

The kicker is that /config/config.boot on a USG is exactly as it is on an EdgeRouter. This JSON mess is needlessly exposing you to an implementation detail of how UniFi and a USG interact with each other.

Spend enough time messing with USG JSON files and you’ll eventually get the hang of it, running all of your changes through a JSON validator before saving, but it’s a giant pile of suck.


My next problem was the DHCP server — it provides virtually no control over common DHCP options. They’ve added Domain Name since my initial setup but that’s it. You’ll need JSON for anything else.

No problem, I’ll set up DHCP Relaying and keep using my existing DHCP server! Oh, that’s more JSON, too.

DNS Overrides so my internal zones get resolved by my internal DNS servers? More JSON.

L2TP remote user VPN with local accounts? More JSON. (Now it supports L2TP for remote user VPN and a UniFi-managed RADIUS server)

At its largest, my config.gateway.json file was 147 lines.


Now, one of the biggest features people tout about the USG is its DPI reports. They’re very pretty. And very useless.

dpi-usg

Whole lot of streaming media being used here, but I drill down trying to see who’s using it… and I seriously have no idea what I’m looking at here.

dpi-edgemax

This is DPI on my EdgeRouter. It isn’t nearly as pretty, but I can easily see who transferred all those GBs.


Another problem I ran into was with VPN connections. The automatic Site-to-Site VPN between USGs is real slick, but comes with a major caveat: The USGs themselves cannot communicate across the VPN tunnel. They can’t talk to each other, nor can they communication with the remote LAN.

This creates a problem for a common remote office deployment scenario: The remote end doesn’t have a local DNS server for internal zones and needs to forward those queries to the central office’s USG or DNS server.

I’ve been told this might not be a problem if Dynamic Routing is enabled, and I’m told this will be fixed in the future by using a /30 assignment for the vti interfaces instead of a /32.


The final straw in my USG experiment was when I added a second ISP at home for failover load-balancing. None of my inbound port forwards worked on the second WAN interface, whether it was the current interface or not.

This is a limitation of how EdgeOS port-forward works — they’re tied to a single interface.

I also discovered that my Dynamic DNS doesn’t failover — also tied to a single interface and UniFi does not expose any option to use a different interface.

All of this could be worked-around with more JSON hackery for DNAT rules and additional Dynamic DNS entries… but I reached the point where migrating back to the EdgeRouter 8 was more appealing than continuing to work-around the USG’s limitations.


I want to love the USG. Single-pane-of-glass remote management of an entire networking stack is a compelling value proposition. UniFi’s UI is slick and makes many somewhat complicated things very simple.

But the USG remains an immature product only suitable for very simple deployments. I would absolutely deploy a full UniFi stack w/ USG to a family member’s home, where the ease of remote management greatly outweighs the USG’s limitations.

I would not deploy a USG for a small business client. Period. UniFi switches and wireless, absolutely! But the USG is dead to me as an SMB device until the UniFi Controller offers more control of DHCP and DNS, and they resolve the VPN communication issues. All of this is likely to come in UniFi v5.6.x but that is probably 4-6 months from reaching Stable.

My home USG Pro has been sold. I’m hanging on the the USG 3-port for my lab, and will soon pick up another, so that I can continue to track its progress and run mock deployments. And I’m definitely looking forward to the USG-XG becoming available — it’ll be great for 10Gb/s inter-VLAN routing even if the platform itself is still iffy as a gateway router.

UniFi 5.5.19 has finally gone Stable!

Feels like I’ve been waiting for UniFi v5.5.x to make a Stable release for as long as I’ve been using UniFi. It has finally arrived!!

The list of changes from v5.4.x is massive. The biggest impact, IMO, is that nearly all of the deficiencies from Chris Sherwood’s (in)famous USG vs. EdgeRouter comparison have been addressed in v5.5.x. The USG remains far from perfect — for many SMB scenarios it is pretty close to unsuitable — but it has vastly improved over the past six months.

I’ll have my own write-up of where the USG shines, and falls short, in the coming weeks.