USG-XG-8 is Dead

Ubiquiti has finally admitted that the USG-XG-8 is dead. The big problem with the USG-XG-8, aside from UBNT taking a solid year to product decent firmware, is that UniFi just doesn’t play in that league. It’s forgivable that the USG and USG Pro sacrifice features for simplicity because they’re cheap and UniFi is awesome. The USG-XG-8 is not cheap and that makes it harder to ignore all the things it couldn’t do. Especially when the EdgeRouter version can do those things at a much lower price.

Good riddance!

Ring and Retry

I have a Ring Doorbell Pro on my front door which has always been problematic. At first I could get it to join the WiFi but then it would error — turns out it was trying to use an outside DNS server and I had blocked clients from using any DNS but mine.

When I replaced my temporary AmpliFi setup with UniFi, I couldn’t get it to find my SSID at all. I literally held an AP directly in front of it and it would find several neighbors WiFi but not mine.

img_3400

I read somewhere that sometimes a Ring will get confused seeing several APs broadcasting the same SSID, so I decided to give it its own AP and SSID on 2.4GHz-only. I put it in the attic slightly offset from being directly above the door. This has mostly worked ok, except that it takes a long time to re-connect after the AP reboots from firmware updates.

Today I had cause to look at my AP retry rates…

Screenshot 2018-11-07 at 12.23.20 PM

And the Ring’s AP retry rate is just ridiculously bad. I crawled into the attic and shoved the AP all the way into the soffit. Gained 6dBm but the retry rate didn’t budge. Changed from Channel 1 to Channel 11, no difference.

Then I had the thought that, since initially installing this stuff, I’ve put a great deal of effort into tuning the power levels and minRSSI values to get devices to use the right AP instead of clinging to a poor signal. Let’s try turning off an AP in the attic I don’t really need anymore, bring the one I’ve been using for the Ring back into broadcasting my normal SSID on 2.4GHz and 5GHz, and trying joining the Ring to it.

And it joined right up! On 5GHz. The signal is decent and the retry rates have dropped to a more reasonable level. Huzzah!

Blasting WiFi across the street

I have a lot of front yard to maintain.

img_0160

UMA-D_Front_Angle

It would be nice to have good WiFi signal while mowing all this lawn. There’s an AP in the attic above the front door but the signal doesn’t reach all that far, maybe 30-40′ out. I needed something with a bit more oomf and the UAP-AC-M + UMA-D antenna combination sounded like the perfect solution.

If you haven’t heard, the UMA-D is a tiny miracle antenna: dual-band, 15dBi, 45-degrees on 5GHz and 90-degrees on 2.4GHz, for $99. It transforms the otherwise unimpressive UAP-AC-M into a directional WiFi blaster that will send its signal hundreds of feet downrange in open terrain.

As an initial test, I placed the combo in the bonus room knee wall space:

img_0157

Blasting through my roof I was getting about 180Mb/s of download speeds to my iPhone XS… from across the street! That’s 140-ish feet away.

Of course, that wasn’t good enough for me, so I found a pre-existing hole to run an Ethernet cable to and mounted it outside the garage.

img_0214

The improvement is incredible.

img_0269

That’s from my phone. 140 feet away.

If you need to blast a WiFi signal far away outdoors, the UAP-AC-M + UMA-D are a powerful and affordable solution.

UniFi Protect moves away from self-installs

Much online rage has been spilled this weekend over UniFi Protect not being made available for self-installation. If you’ve not been paying attention, UniFi Protect is a new NVR platform from Ubiquiti. Presently Ubiquiti says that UniFi Video will continue to be developed and supported… but nobody expects this to continue for very long. Protect is the new hotness, and since Ubiquiti doesn’t charge for their software, it is unimaginable that they will continue putting resources towards two separate products that do the same thing.

Today, the only way to get Protect is on the UCK-G2-PLUS — which has just launched for $199 with an 8-core ARM SoC w/ 3GB RAM and an easily upgradeable 1TB 2.5″ hard drive. Support for the UAS-XG — an attractive but otherwise bog standard 1U server with a $1,999 MSRP — is coming soon.

People are miffed for a variety of reasons. The G2+ offers very limited storage options. The UAS-XG is a fair value compared to buying an equivalent server from an Enterprise vendor, but it’s incredibly expensive relative to DIY or other thrifty options. There’s presently no middle ground.

And with the UAS-XG being a standard Intel server running Ubuntu 16.04 LTS, there’s no technical reason that Protect can’t be offered for self-installs. Using Docker — as they’ve done with UNMS, and had alluded to providing for Protect during the Beta cycle — would greatly reduce the support challenges of providing self-installable software for Linux.


I’m not sure how I feel about this. I have 5 UVC-G3-Flex cameras that I have been planning to deploy on the G2+ w/ Protect, with the expectation that if I was happy with the platform over time I would replace 6 Ring Spotlights, 4 Amazon CloudCams, and probably add a couple more.

That’s all within the advertised capabilities of the G2+, but with its limitation of a single 2.5″ drive it won’t be able to meet my continuous recording retention targets. I happen to have a 2nd G2+ already so maybe this isn’t a total deal-breaker for me, if it turns out that I really love Protect, but right now I’m questioning if Ubiquiti’s camera platforms are worth the lock-in and premium pricing.

I already have an NVR server that is substantially more powerful than the UAS-XG, an unlimited camera license for Sighthound, and a bunch of Reolink cameras from my old house which are comparable in quality to the G3 Flex.

I’m Baaaack!

Personal stuff has limited my participation in the Ubiquiti community for much of this year, but I’m finally getting my head back above water. The biggest news is that I bought a new-to-me house on 1.5 acres and have been consolidating households with my girlfriend and her children.

New home means lots of upcoming technology projects:

  • Pulling Ethernet to every room in the main house.
  • 60GHz Mikrotik PtMP link between the main house, pool house and detached garage apartment.
  • Building out a fully isolated network for the apartment dwellers.
  • Radius-assigned VLANs for wireless devices.
  • UniFi Protect and 5x UVC-G3-Flex deployment using a UCK-G2-PLUS.
  • Deciding on more permanent placement of APs, which are presently strewn haphazardly across the attic. Or possibly replacing most of them with UAP-IW-AC.
  • Getting complete coverage of the front and back yards using a combination of UAP-AC-M, UMA-D, and UAP-AC-M-Pro — there’s roughly 100′ of depth in the front yard and 200′ beyond the pool house.
  • Attempting to order 2Gb/s fiber service from Comcast.

I’m also back to dogfooding the USG with plans to bring a USG-XG-8 into service as soon as a couple of show-stopping bugs are resolved.

The Lab Begins

2017-07-07 13.24.02

UPS just dropped off my first new acquisitions for the Ubiquiti routing and switching lab I am constructing — 2x ER-X, 2x ER-4, 2x ER-4 rack kits, and a Monoprice keystone patch panel. Ultimately it’s going to have about a dozen routers, several switches, an AP, and several dedicated client devices.

More to come once my rack shelves arrive and I get the 7′ rack moved to its new home. In the meantime I’m going to put the ER-Xs through their paces to see if they’re a suitable substitute for the ER-8 I’m running at home now…

UniFi v5.6.x Goes Testing!

UniFi 5.6.10 Testing has dropped. I haven’t been paying much attention to the v5.6.x Unstable releases as I’d been waiting for v5.5.x to become Stable, but now that we’re here… Here’s a run down of the differences I see from v5.5.19 -> v5.6.10.

USG Properties

First and foremost, a bunch of things have been removed from the Properties pane for the USG. Rejoice! All of that stuff has been moved to various places with Settings.

The Advanced section offers lots of new goodness, including control over Offload features.

Site

Over in Site settings, we now have control over SSH access. Previously SSH was always enabled.

Wireless Networks

Fast Roaming for 802.11r devices is now available. Use at your own risk.

Networks

Networks brings much more control over DHCP. Highlights are DHCP Relay, being able to specify a gateway other than the USG, and control over various settings used for PXE and VOIP booting. It’s not everything we’ve always wanted but it’s a great first step.

Firewall Settings

Firewall has a new Settings tab with all sorts of knobs to tweak.

Port Forwarding

And Port Forwarding has been moved to the Routing & Firewall section.

admins

More control over user permissions are now available.

logging

Fine-grained control over logging.

And the next several shots show the various new tabs under Services.

DHCP Relay

DDNS

mdns

upnp

Life with a pair of USGs

The past fall I took an interest in Ubiquiti’s UniFi routing and switching product lines. It started with getting some US-16-XG switches from their Beta program and I immediately fell in love with UniFi. Aside from providing real slick management of features like VLANs and LACP, the Clients and Insights features within UniFi make it real easy to figure out what’s attached where on my network.

There’s just one glaring omission with the switching product range:

UniFi switches don’t natively support inter-VLAN routing.

The UniFi way of routing between VLANs is to use a UniFi Security Gateway. Using an external router is not ideal for a network with much inter-VLAN traffic, and for my home network I was tempted to use a virtualized pfSense router to maintain > 1GB/s inter-VLAN speeds, but I saw great potential value in deploying UniFi-managed routers for small offices. Before recommending them to anyone else I figured I should dogfood them for a while. I bought a USG-PRO-4 for my home office plus a USG 3-port for my girlfriend’s apartment, and ran them from December through June.


Swapping in the USG Pro for my EdgeRouter 8, the first problem I ran into is that there’s no way in the UniFi Controller to override the device’s WAN MAC address. Lots of ISP CPEs lock on to a router’s MAC address and won’t accept DHCP requests from a new device without jumping through some hoops. Most any router you would pick up from Best Buy supports changing the MAC address.

Now, the USG runs EdgeOS, the CLI of EdgeOS supports specifying MAC addresses for interfaces, and you absolutely can get at the CLI of a USG. But… anything you commit will not be saved back to the UniFi Controller and will be lost when the device re-provisions.

The process of persisting CLI changes on the USG is to put them in a special file on the UniFi Controller, /usr/lib/unifi/data/sites/site-code/config.gateway.json. The format of this file looks similar to the format an EdgeRouter configuration file, except that it’s JSON.

Here’s a fragment from an EdgeRouter:

interfaces {
    ethernet eth2 {
        mac 00:0D:3A:27:02:78
    }

And an equivalent from a USG Pro:

{
        "interfaces": {
                "ethernet": {
                        "eth2": {
                                "mac": "00:0D:3A:27:02:78"
                        }
                }
        }
}

Now, this JSON format isn’t what you get when you type show. You have to exit from configure, then run mca-ctrl -t dump. Figure out which fragments you need, put them in config.gateway.json, and then tear out your hair when it doesn’t work because what you made wasn’t actually valid JSON. You missed the opening or closing braces. Or that options taking multiple values need brackets and commas. Or forgot to quote something.

The kicker is that /config/config.boot on a USG is exactly as it is on an EdgeRouter. This JSON mess is needlessly exposing you to an implementation detail of how UniFi and a USG interact with each other.

Spend enough time messing with USG JSON files and you’ll eventually get the hang of it, running all of your changes through a JSON validator before saving, but it’s a giant pile of suck.


My next problem was the DHCP server — it provides virtually no control over common DHCP options. They’ve added Domain Name since my initial setup but that’s it. You’ll need JSON for anything else.

No problem, I’ll set up DHCP Relaying and keep using my existing DHCP server! Oh, that’s more JSON, too.

DNS Overrides so my internal zones get resolved by my internal DNS servers? More JSON.

L2TP remote user VPN with local accounts? More JSON. (Now it supports L2TP for remote user VPN and a UniFi-managed RADIUS server)

At its largest, my config.gateway.json file was 147 lines.


Now, one of the biggest features people tout about the USG is its DPI reports. They’re very pretty. And very useless.

dpi-usg

Whole lot of streaming media being used here, but I drill down trying to see who’s using it… and I seriously have no idea what I’m looking at here.

dpi-edgemax

This is DPI on my EdgeRouter. It isn’t nearly as pretty, but I can easily see who transferred all those GBs.


Another problem I ran into was with VPN connections. The automatic Site-to-Site VPN between USGs is real slick, but comes with a major caveat: The USGs themselves cannot communicate across the VPN tunnel. They can’t talk to each other, nor can they communication with the remote LAN.

This creates a problem for a common remote office deployment scenario: The remote end doesn’t have a local DNS server for internal zones and needs to forward those queries to the central office’s USG or DNS server.

I’ve been told this might not be a problem if Dynamic Routing is enabled, and I’m told this will be fixed in the future by using a /30 assignment for the vti interfaces instead of a /32.


The final straw in my USG experiment was when I added a second ISP at home for failover load-balancing. None of my inbound port forwards worked on the second WAN interface, whether it was the current interface or not.

This is a limitation of how EdgeOS port-forward works — they’re tied to a single interface.

I also discovered that my Dynamic DNS doesn’t failover — also tied to a single interface and UniFi does not expose any option to use a different interface.

All of this could be worked-around with more JSON hackery for DNAT rules and additional Dynamic DNS entries… but I reached the point where migrating back to the EdgeRouter 8 was more appealing than continuing to work-around the USG’s limitations.


I want to love the USG. Single-pane-of-glass remote management of an entire networking stack is a compelling value proposition. UniFi’s UI is slick and makes many somewhat complicated things very simple.

But the USG remains an immature product only suitable for very simple deployments. I would absolutely deploy a full UniFi stack w/ USG to a family member’s home, where the ease of remote management greatly outweighs the USG’s limitations.

I would not deploy a USG for a small business client. Period. UniFi switches and wireless, absolutely! But the USG is dead to me as an SMB device until the UniFi Controller offers more control of DHCP and DNS, and they resolve the VPN communication issues. All of this is likely to come in UniFi v5.6.x but that is probably 4-6 months from reaching Stable.

My home USG Pro has been sold. I’m hanging on the the USG 3-port for my lab, and will soon pick up another, so that I can continue to track its progress and run mock deployments. And I’m definitely looking forward to the USG-XG becoming available — it’ll be great for 10Gb/s inter-VLAN routing even if the platform itself is still iffy as a gateway router.

UniFi 5.5.19 has finally gone Stable!

Feels like I’ve been waiting for UniFi v5.5.x to make a Stable release for as long as I’ve been using UniFi. It has finally arrived!!

The list of changes from v5.4.x is massive. The biggest impact, IMO, is that nearly all of the deficiencies from Chris Sherwood’s (in)famous USG vs. EdgeRouter comparison have been addressed in v5.5.x. The USG remains far from perfect — for many SMB scenarios it is pretty close to unsuitable — but it has vastly improved over the past six months.

I’ll have my own write-up of where the USG shines, and falls short, in the coming weeks.