The problem with things that work by “Magic” is that they must always “Just Work”

Posted on December 11, 2023 by ubntfan

Roughly 24 hours ago, the primary Internet at our mountain cabin went down for 90 minutes. No big deal, we’re not there, and there’s an LTE backup on WAN2. However, the “Site Magic” VPN tunnel never re-established itself.

In the past I’ve run EdgeRouters and USGs w/ IPSec tunnels so I’m familiar with Ubiquiti’s self-imposed challenges with multi-WAN, and that even with single-WAN the IPSec tunnels are not as resilient as they ought to be.

But Site Magic is supposed to be “magic.” It’s supposed to make it all better. It’s supposed to be so good that they don’t need bother giving me any tools to troubleshoot or give it a kick in the ass when it doesn’t work. A cursory Google search didn’t lead me to any “secret” CLI commands to provide more than the unifi.ui.com interface.

I’d say I’m disappointed… but, honestly, this kind of thing is typical of them and is why I want to quit their gateways.

UniFi Express

Posted on November 29, 2023 by ubntfan

Ubiquiti has launched the $149 UniFi Express, which adds to the UXG-Lite a WiFi AP, a tiny screen, the ability to host its own UniFi controller ~~or be adopted to an external one~~, and the flexibility to be used as just a ~~router or~~ AP, with or without wireless meshing.

Edit: Apparently it also subtracts IDS/IPS. Which I’ve never been shy about calling absolutely useless and of negative value.

CPU and RAM specs have not yet been disclosed and apparently none of the YouTube ~~shills~~ Influencers that Ubiquiti seeded with free devices ahead of the launch thought that was a detail worth digging into.

Edit: So without IDS/IPS, it probably limps along with the same 1GB RAM and dual-core A53 as the UXG-Lite.

Against my better judgement, I’ve ordered one for the cabin. Probably won’t find the roundtuits to actually deploy it for a while as I have no idea what I’ve done with the CK Gen2+ that I was using there previously for Protect before the UDM flaked out.

Next-Gen Gateway Lite

Posted on November 15, 2023 by ubntfan

Ubiquiti is finally launching a replacement for the decrepit USG. The UXG-Lite will be available for purchase from the Ubiquiti store on 11/20 for $129 and is roughly a de-spec’d UDR — A53 dual-core, 1GB RAM, 2x 1GbE ports, and an unknown amount of on-board storage that is hopefully eMMC but largely irrelevant.

I struggle to understand why they continue down this path of separate product lines for on-device and externally run UniFi Controllers. Yes, this device would need more RAM and possibly storage but that shouldn’t move the BOM much and they likely could have made that up with other cost optimizations that would go unnoticed.

In theory I should love this for my cabin in the mountains, where I’m wasting a UDM Pro SE on 100/4 Internet and 4 cameras and all the blinken lights are supremely annoying because there’s nowhere to tuck it away out of sight… but I think about it and dread the idea of dusting off a CloudKey Gen2+ to run the UniFi Controller and Protect to run the tiny number of UniFi/Protect devices in place. My ideal device there would have three assignable GbE interfaces, no PoE or AP, able to run the UniFi Controller and Protect, with an accessible M.2 or 2.5″ bay to be used for continuous recordings. Which, looking at this and the UDR, seems like a pretty reasonable $179 product.

But ultimately I’ve been leaning towards just replacing all of my UniFi routers with mini PCs and $129 easily buys an Intel N100-based dual-GbE system complete with enough RAM and storage to run a few VMs / Containers alongside whichever routing platform I’d choose. While I would never pretend that’s a fair comparison to what Ubiquiti offers, my mindset is that their products used to be extremely “hacking friendly” — which helped make up for their glacial pace of delivering and improving functionality — and I haven’t been feeling that love for an extremely long time. The only thing holding me back is that for my home network it’s too big of a project for a rip-and-replace cutover. It’d be so much easier if show configuration commands were still a thing.

UniFi Dream Router leaves EA

Posted on April 26, 2022 by ubntfan

I figured it would end up a $159 device but it has landed at $199. My overall opinion remains the same. As a “USG v.Next” it’s alright. As a Protect host, it’s a hot mess.

UniFi Disappointment Router?

Posted on October 16, 2021 by ubntfan

The UniFi fanbois were aflutter when Ubiquiti released this video promoting an upcoming UniFi Dream Router:

It sounded like a substantial upgrade to the UniFi Dream Machine: WiFi 6, two ports of PoE, 128GB SSD, an SD slot for storage expansion, and the ability to run Protect and other Ubiquiti controllers that haven’t been available to UDM users due to the lack of storage.

Then it hit the Early Access store for $79. Huh?

Continue reading →

UniFi Internet Security DNS Filtering

Posted on February 4, 2020 by ubntfan

Today I was prompted to figure out what exactly the DNS Filters settings in UniFi Internet Security are doing.

unifi-dns-filtering

The main thing that happens is that the DNS queries for the associated VLAN are forwarded to the cleanbrowsing.org public resolver for the chosen filtering category. This is accomplished by creating a new dnsfilter network interface, binding another instance of dnsmasq to it, and using NAT to redirect DNS queries from the associated VLAN.

The implementation is slick from a technical perspective but I have a couple major problems with it.

Firstly, cleanbrowsing.org supports DNS-over-TLS, DNS-over-HTTPS, and DNSCrypt, but Ubiquiti has chosen to forward the queries unencrypted.

Secondly, if you’re using a local DNS server, say Active Directory or PiHole, those NAT rules will prevent the local DNS from functioning when queries are coming from or destined to a filtered VLAN. This might be fixable with some config.gateway.json or scripting hackery on the USG, I have not dug into that, but UDM users are hosed since there is no json replacement nor a way to automatically run a script.

Thirdly, Ubiquiti could have licensed the cleanbrowsing.org RPZs, converted them to dnsmasq format for local blacklisting ala PiHole, and eliminated the need to forward DNS to a 3rd-party and interfere with cross-VLAN DNS.

I cannot recommend using this DNS filtering because it will cause issues if your DNS implementation isn’t “UniFi Default” and the filtering options are minimal. My personal preference for home use is a PiHole in conjunction with OpenDNS Home.

For an office, Cisco Umbrella is the way to go. Do not underestimate the value of having a commercial relationship with a security provider. I <3 PiHole, but if you try to figure out why it blocks, say, zombo.com, and how you’d get it removed, you will immediately recognize that “free” security lists aren’t worth more than you’re paying for it.

Update: Also want to point out that the dnsfilter interfaces use IPs in the 203.0.113.0/24 subnet. That is TESTNET-3 in RFC 5737. It’s unlikely this will ever cause a problem… but over the long haul, every misuse of an IPv4 subnet ends up causing a problem. The loopback range 127.0.0.0/8 or link-local 169.254.0.0/16 would have been more appropriate choices.

Isolating UniFi Devices

Posted on November 29, 2019 by ubntfan

I’ve been lax in firealling my VLANs at home, but with the recent controversy over UniFi devices phoning home without consent, this has taken on renewed importance. I’m also taking on a new tenant in my detached apartment and would like to keep all their stuff segregated from mine.

Fortunately it’s all pretty easy.

For my network, I’m keeping my Cloud Key and Pi-Hole on the main LAN. I have additional Corporate VLANs created for Management, Cameras, and the Apartment.

In the Firewall rules, for WAN_OUT I’ve created two rules to Drop all traffic from the Management and Cameras networks. They cannot reach the Internet at all.

To allow and deny particular cross-VLAN traffic, the first step is to create a group of all the Private IP address ranges:

10.0.0.0/8
192.168.0.0/16
172.16.0.0/12
100.64.0.0/10 (this is CGNAT, might be a bad idea if your ISP uses them)

Back in the Rules area, LAN_IN needs a series of rules:

Allow (Management | Cameras) networks access to the CloudKey.
Allow (Management | Cameras | Apartment) networks access to the Pi-Hole.
Drop (Management | Cameras | Apartment) networks access to the Private IP ranges group.

The Allow rules must be before the Deny rules for each Network.

The gotcha with denying devices access to the Internet is that they cannot directly obtain firmware updates. For UniFi Networking products this can be worked-around by having the UniFi Controller cache the firmware prior to upgrading — see Settings -> Maintenance -> Firmware.

I’m not sure whether Protect can distribute firmware updates to the Cameras. Guess I’ll find out the next time there’s an update available. Once my UniFi Protect NVR arrives I will place that in the Cameras VLAN so that traffic doesn’t have to cross the router and figure out the WAN_OUT / LAN_IN firewall rules needed to keep it happy.

UDM Pro thoughts

Posted on October 28, 2019 by ubntfan

UDM Pro I finally broke down and added the UDM Pro to my Ubiquiti router lab. Here are the specs as provided in the Early Access store:

8-Port gigabit switch with 10G SFP+ port
Dual WAN ports for redundancy and load balancing: 10G SFP+ and 1G RJ-45
Bluetooth connectivity for easy setup via UniFi app
Scalable UniFi Network Controller with advanced management capabilities
UniFi Protect video surveillance NVR with 3.5″ (or 2.5″) HDD support
Enterprise-class IPS/IDS and DPI capabilities
1 x 1.3″ Touchscreen display for quick status information
Powered by fast 1.7 GHz quad-core processor

Not mentioned is that the UDM / UDM Pro use a new OS that is not derived from EdgeOS / VyOS / Vyatta. This is why I’d held off on buying one, when the UDM was first made available to Early Access back in March it was FAR from having feature-parity with the EdgeOS-based USGs. The present state of UbiOS is much closer to production-ready (by UniFi standards).

From the perspective of the USG Pro, this is a pretty serious upgrade in performance with a very minor bump in the expected MSRP. 10Gb/s inter-VLAN routing and WAN. Supposedly can hit 5Gb/s of IPS/IDS throughput — I’m underwhelmed by Ubiquiti slapping a pretty face over open source Suricata with publicly-available lists, but I seem to be a minority.

It’s also quiet. With it sitting on my desk and the LCD showing the fans at 50%, I struggle to hear it. The USG Pro, USG-XG-8, and their EdgeRouter siblings are not quiet-space-friendly.

I feel like they’ve missed the boat in a couple of areas with regards to Protect.

One, those LAN ports should have POE. As a router, those ports are of marginal value — where a $379 router is justified, it’s going to be attached to a larger switch. But as an NVR with a larger storage capacity than the Cloudkey Gen2 Plus, having 8 PoE ports would cover many deployment scenarios and would be quite valuable (12-16 ports would be better).

Two, it should have more drive bays. 2x LFF would have been nice. More LAN ports w/ PoE and 4x SFF bays could be better.

Three, it needs a USB host port for offloading footage directly to removable storage. As things stand now, pulling footage out of Protect is a pile of suck, but presumably it will get better, and being able to push footage directly to removable storage would be a great feature.

Ubiquiti has teased a larger, 4x LFF device. It’s not clear if that will be a NAS that Protect can use, or if it will run Protect directly, and they haven’t shown the back yet so we don’t know if it has PoE switch ports to act as a more traditional NVR appliance.

DNS Changes in UniFi 5.11.50

Posted on October 19, 2019 by ubntfan

It’s not DNS.
There’s no way it’s DNS.
…
It was DNS.

Took a nasty hit yesterday from a change made in UniFi 5.11.50:

If you have any ‘service dns forwarding options’ configuration defined in config.gateway.json, it will overwrite the provisioning of statically defined name servers, leaving you with no DNS. Either remove the ‘service dns forwarding options’ portion of config.gateway.json, or add additional ‘options’ lines defining name servers, such as ‘server=1.1.1.1’, ‘server=8.8.8.8’, etc.

I’ve long used some config.gateway.json tweaks to add conditional forwarders for my Active Directory zones because Ubiquiti still hasn’t seen fit to put that functionality in the GUI.

So after upgrading to 5.11.50, the first full re-provision cycle of my USG Pro ended up with no catchall forwarder. The interesting bit is how this manifested itself. My client devices are pointed directly at a local PiHole so that its metrics don’t show everything as coming from the router, so they all kept humming.

What broke was failover load-balancing. By default, the USG’s WAN health check pings a target… by DNS name. So the USG thought both my WANs had failed, and due to the vagaries of how the routing tables are managed, that ends up sending all traffic out WAN2. Not a desirable condition when using WAN2 costs me money (LTE) and is significantly slower than WAN1.

This led to a couple hours of cursing at how badly failover load-balancing is messed up on Ubiquiti’s routers, but in the end, as it is so often, the problem was DNS.

More interesting is that this problem was brought about by some behind-the-scenes changes in how UniFi is managing DNS for the gateway device. Prior to this version of UniFi, the USG defaults to putting the WAN DNS value in /etc/resolv.conf. This is not ideal for two reasons:

No DNS caching takes place for things running on the router itself. If you have IPS or load-balancing enabled this can result in an excessive number of DNS queries originating from the router, and creates a ton of noise if you have DNS metrics through PiHole, OpenDNS, etc.
DNS servers are always queried one-by-one in the order they appear. This may or may not be desirable, but when the first DNS server is unavailable it will result in slow DNS responses.

I’ve worked around this on my USGs by configuring the WAN DNS servers manually and making the first entry 127.0.0.1. So the router will query the local dnsmasq first, and dnsmasq itself will ignore that entry when selecting an upstream DNS to forward to. dnsmasq is also “sticky” by default in choosing an upstream DNS server — it won’t keep sending queries to the first resolv.conf entry if it’s non-responsive.

What UniFi 5.11.50 has changed is that now they’re automatically placing just a 127.0.0.1 entry in resolv.conf and have moved the rest of the DNS configuration into /etc/dnsmasq.conf.

Another change they’ve made is setting all-servers. This causes dnsmasq to query all upstream DNS servers simultaneously and whichever answers first wins.

This is great, in that the USG’s DNS configuration is now made sensible by default. And it’s not great, in that if you have config.gateway.json changes to service dns forwarding options they will override what UniFi is doing to dnsmasq.conf and you will need to make adjustments to provide the DNS forwarders yourself.

And beware if you’re configuring multiple DNS servers with differing views of DNS! This has always been a bad idea but all-servers can make it much worse to troubleshoot.

UniFi v5.6.x Goes Testing!

Posted on July 5, 2017 by ubntfan

UniFi 5.6.10 Testing has dropped. I haven’t been paying much attention to the v5.6.x Unstable releases as I’d been waiting for v5.5.x to become Stable, but now that we’re here… Here’s a run down of the differences I see from v5.5.19 -> v5.6.10.

First and foremost, a bunch of things have been removed from the Properties pane for the USG. Rejoice! All of that stuff has been moved to various places with Settings.

The Advanced section offers lots of new goodness, including control over Offload features.

Over in Site settings, we now have control over SSH access. Previously SSH was always enabled.

Fast Roaming for 802.11r devices is now available. Use at your own risk.

Networks brings much more control over DHCP. Highlights are DHCP Relay, being able to specify a gateway other than the USG, and control over various settings used for PXE and VOIP booting. It’s not everything we’ve always wanted but it’s a great first step.