I don’t love UniFi Threat Management and neither should you

Posted on February 23, 2020 by ubntfan

When Ubiquiti put out the first Beta releases of IDS / IPS, I was surprised by the overall excitement of the enthusiast community. People were snatching up $2,000+ USG-XG-8s just to be able to use this feature without slowing down their WAN. Cheaper line-rate IDS / IPS has been a major force behind the UDM / UDM Pro hype train.

I am far less enthused, about IDS / IPS specifically and UniFi Threat Management in general.

My core issue: It’s Free.

Free is not inherently bad. I use lots of things that are free. Ubiquiti is built on lots of things that are free — VyOS, Linux, OpenWRT, hostapd. But security is different. Security is a process, not a deliverable.

Look at how Ubiquiti is offering security. Did they produce an IDS / IPS product? No, it’s just Suricata. Are they employing an army of security professionals to discover new threats, analyze alerts from their customers, produce new signatures? No, they’re just passing along the free rulesets that any Suricata user has the ability to use. Is Ubiquiti adding any value to Suricata alerts, such as making them easier to interpret or correlate? Heck no.

There’s a saying that If you’re not the customer, you’re the product. Does that apply here? Also, no. Enabling UniFi’s IDS/IPS isn’t passing any value back to the Open Information Security Foundation, the Suricata project, or any of their sponsoring organizations.

My other issue: What Ubiquiti is providing isn’t particularly good.

IDS / IPS: Suricata is purely signature based, like an antivirus program from 30 years ago… only worse. Most signatures are simplistic, prone to false positives, and the alerts they generate do not provide much context to decide whether they warrant further investigation or can be ignored. I’m not positive about this, due to the minimal information provided, but I think many of the alerts I’ve received are for traffic that the firewall was going to drop anyways.

There are many things Ubiquiti could be doing to add value to Suricata alerts — pruning rulesets and providing more context would be a great start — but that would require substantial investment.

Zeek (Bro) is generally considered better than Suricata, in that it’s focussed on flagging anomalous traffic instead of depending on signatures, but turning that into a user-friendly solution also requires significant investment.

DNS Filtering: I’ve previously written about this. In short, they’ve delivered an extremely simplistic filtering solution that depends on redirecting DNS traffic to an undisclosed 3rd-party. The manner in which they’ve implement filtering is unsuitable for a broad range of common DNS scenarios and they’ve provided zero control beyond choosing from the 3rd-party’s three filtering options.

It’s not worth using. Use PiHole and / or OpenDNS at home. Subscribe to Cisco Umbrella for business filtering.

Network Scanners: The Endpoint Scanner is basically a point-in-time nmap. No history, no correlation with UniFi’s Client History data. The Internal Honeypot is also extremely simplistic — it seems to alert simply on connection attempts to particular ports.

GeoIP Filtering: Hey, it does exactly what’s expected! I wish it did more tho. In particular, I might like to drop traffic from some countries to particular ports (VPN-related) but not others (HTTP / HTTPS).

IP Reputation: Tor blocking and Restrict Access to Malicious IP Addresses do what they say, tho again, it’s unclear what the information source is and if 3rd-party disclosures are involved.

Missing Features: Competing Unified Threat Management solutions generally have features that Ubiquiti isn’t (yet) providing: A/V scanning, HTTP / HTTPS interception, email filtering, data loss (PII / PHI exfiltration) protection, integration with Network Access Protection / Network Access Control systems, and more.

At some level it’s great that Ubiquiti is making security tooling available to users with less technical expertise or budget. What’s not great is those are the demographics who will read the marketing and believe they’re getting much more than is actually being provided.

The Real MGP

Posted on February 19, 2020 by ubntfan

UniFi Management Gateway Pro, that is. Who comes up with these names?

UMG-front UMG-back

Freshly announced, ahead of being available for Early Access purchase, we have essentially a UDM Pro… minus the switch, minus the HDD bay, half the RAM… and no local controllers! Adopt it to your Cloud Key, cloud-provider, or self-hosted UniFi install.

It also has a built-in UniFi Smart Power Plug. I can understand why — reboot your Cable / DSL “modem” if the Internet goes down — but it’s just such an odd thing to integrate into what is otherwise a plain router.

I’m happy to see this, tho disappointed that it’s not in more of an ER-4 form-factor with an optional rack mount kit. Hopefully this is a sign that a smaller-but-equally-capable desktop unit will come eventually.

I’m also hoping that this is a sign that in the future it will be possible to disable all local controllers on the UDM / UDM Pro.

Update: They’ve tweaked the original post, now it’s “UniFi Managed Gateway Pro” and the UMG Pro will be the first of the “UniFi Managed Gateway Product Line.”

Update #2: Renamed again, UXG-Pro. If Ubiquiti is anything, it’s consistently inconsistent.

UniFi Internet Security DNS Filtering

Posted on February 4, 2020 by ubntfan

Today I was prompted to figure out what exactly the DNS Filters settings in UniFi Internet Security are doing.

unifi-dns-filtering

The main thing that happens is that the DNS queries for the associated VLAN are forwarded to the cleanbrowsing.org public resolver for the chosen filtering category. This is accomplished by creating a new dnsfilter network interface, binding another instance of dnsmasq to it, and using NAT to redirect DNS queries from the associated VLAN.

The implementation is slick from a technical perspective but I have a couple major problems with it.

Firstly, cleanbrowsing.org supports DNS-over-TLS, DNS-over-HTTPS, and DNSCrypt, but Ubiquiti has chosen to forward the queries unencrypted.

Secondly, if you’re using a local DNS server, say Active Directory or PiHole, those NAT rules will prevent the local DNS from functioning when queries are coming from or destined to a filtered VLAN. This might be fixable with some config.gateway.json or scripting hackery on the USG, I have not dug into that, but UDM users are hosed since there is no json replacement nor a way to automatically run a script.

Thirdly, Ubiquiti could have licensed the cleanbrowsing.org RPZs, converted them to dnsmasq format for local blacklisting ala PiHole, and eliminated the need to forward DNS to a 3rd-party and interfere with cross-VLAN DNS.

I cannot recommend using this DNS filtering because it will cause issues if your DNS implementation isn’t “UniFi Default” and the filtering options are minimal. My personal preference for home use is a PiHole in conjunction with OpenDNS Home.

For an office, Cisco Umbrella is the way to go. Do not underestimate the value of having a commercial relationship with a security provider. I <3 PiHole, but if you try to figure out why it blocks, say, zombo.com, and how you’d get it removed, you will immediately recognize that “free” security lists aren’t worth more than you’re paying for it.

Update: Also want to point out that the dnsfilter interfaces use IPs in the 203.0.113.0/24 subnet. That is TESTNET-3 in RFC 5737. It’s unlikely this will ever cause a problem… but over the long haul, every misuse of an IPv4 subnet ends up causing a problem. The loopback range 127.0.0.0/8 or link-local 169.254.0.0/16 would have been more appropriate choices.

Isolating UniFi Devices

Posted on November 29, 2019 by ubntfan

I’ve been lax in firealling my VLANs at home, but with the recent controversy over UniFi devices phoning home without consent, this has taken on renewed importance. I’m also taking on a new tenant in my detached apartment and would like to keep all their stuff segregated from mine.

Fortunately it’s all pretty easy.

For my network, I’m keeping my Cloud Key and Pi-Hole on the main LAN. I have additional Corporate VLANs created for Management, Cameras, and the Apartment.

In the Firewall rules, for WAN_OUT I’ve created two rules to Drop all traffic from the Management and Cameras networks. They cannot reach the Internet at all.

To allow and deny particular cross-VLAN traffic, the first step is to create a group of all the Private IP address ranges:

10.0.0.0/8
192.168.0.0/16
172.16.0.0/12
100.64.0.0/10 (this is CGNAT, might be a bad idea if your ISP uses them)

Back in the Rules area, LAN_IN needs a series of rules:

Allow (Management | Cameras) networks access to the CloudKey.
Allow (Management | Cameras | Apartment) networks access to the Pi-Hole.
Drop (Management | Cameras | Apartment) networks access to the Private IP ranges group.

The Allow rules must be before the Deny rules for each Network.

The gotcha with denying devices access to the Internet is that they cannot directly obtain firmware updates. For UniFi Networking products this can be worked-around by having the UniFi Controller cache the firmware prior to upgrading — see Settings -> Maintenance -> Firmware.

I’m not sure whether Protect can distribute firmware updates to the Cameras. Guess I’ll find out the next time there’s an update available. Once my UniFi Protect NVR arrives I will place that in the Cameras VLAN so that traffic doesn’t have to cross the router and figure out the WAN_OUT / LAN_IN firewall rules needed to keep it happy.

DNS Changes in UniFi 5.11.50

Posted on October 19, 2019 by ubntfan

It’s not DNS.
There’s no way it’s DNS.
…
It was DNS.

Took a nasty hit yesterday from a change made in UniFi 5.11.50:

If you have any ‘service dns forwarding options’ configuration defined in config.gateway.json, it will overwrite the provisioning of statically defined name servers, leaving you with no DNS. Either remove the ‘service dns forwarding options’ portion of config.gateway.json, or add additional ‘options’ lines defining name servers, such as ‘server=1.1.1.1’, ‘server=8.8.8.8’, etc.

I’ve long used some config.gateway.json tweaks to add conditional forwarders for my Active Directory zones because Ubiquiti still hasn’t seen fit to put that functionality in the GUI.

So after upgrading to 5.11.50, the first full re-provision cycle of my USG Pro ended up with no catchall forwarder. The interesting bit is how this manifested itself. My client devices are pointed directly at a local PiHole so that its metrics don’t show everything as coming from the router, so they all kept humming.

What broke was failover load-balancing. By default, the USG’s WAN health check pings a target… by DNS name. So the USG thought both my WANs had failed, and due to the vagaries of how the routing tables are managed, that ends up sending all traffic out WAN2. Not a desirable condition when using WAN2 costs me money (LTE) and is significantly slower than WAN1.

This led to a couple hours of cursing at how badly failover load-balancing is messed up on Ubiquiti’s routers, but in the end, as it is so often, the problem was DNS.

More interesting is that this problem was brought about by some behind-the-scenes changes in how UniFi is managing DNS for the gateway device. Prior to this version of UniFi, the USG defaults to putting the WAN DNS value in /etc/resolv.conf. This is not ideal for two reasons:

No DNS caching takes place for things running on the router itself. If you have IPS or load-balancing enabled this can result in an excessive number of DNS queries originating from the router, and creates a ton of noise if you have DNS metrics through PiHole, OpenDNS, etc.
DNS servers are always queried one-by-one in the order they appear. This may or may not be desirable, but when the first DNS server is unavailable it will result in slow DNS responses.

I’ve worked around this on my USGs by configuring the WAN DNS servers manually and making the first entry 127.0.0.1. So the router will query the local dnsmasq first, and dnsmasq itself will ignore that entry when selecting an upstream DNS to forward to. dnsmasq is also “sticky” by default in choosing an upstream DNS server — it won’t keep sending queries to the first resolv.conf entry if it’s non-responsive.

What UniFi 5.11.50 has changed is that now they’re automatically placing just a 127.0.0.1 entry in resolv.conf and have moved the rest of the DNS configuration into /etc/dnsmasq.conf.

Another change they’ve made is setting all-servers. This causes dnsmasq to query all upstream DNS servers simultaneously and whichever answers first wins.

This is great, in that the USG’s DNS configuration is now made sensible by default. And it’s not great, in that if you have config.gateway.json changes to service dns forwarding options they will override what UniFi is doing to dnsmasq.conf and you will need to make adjustments to provide the DNS forwarders yourself.

And beware if you’re configuring multiple DNS servers with differing views of DNS! This has always been a bad idea but all-servers can make it much worse to troubleshoot.

Life with a pair of USGs

Posted on July 5, 2017 by ubntfan

The past fall I took an interest in Ubiquiti’s UniFi routing and switching product lines. It started with getting some US-16-XG switches from their Beta program and I immediately fell in love with UniFi. Aside from providing real slick management of features like VLANs and LACP, the Clients and Insights features within UniFi make it real easy to figure out what’s attached where on my network.

There’s just one glaring omission with the switching product range:

UniFi switches don’t natively support inter-VLAN routing.

The UniFi way of routing between VLANs is to use a UniFi Security Gateway. Using an external router is not ideal for a network with much inter-VLAN traffic, and for my home network I was tempted to use a virtualized pfSense router to maintain > 1GB/s inter-VLAN speeds, but I saw great potential value in deploying UniFi-managed routers for small offices. Before recommending them to anyone else I figured I should dogfood them for a while. I bought a USG-PRO-4 for my home office plus a USG 3-port for my girlfriend’s apartment, and ran them from December through June.

Swapping in the USG Pro for my EdgeRouter 8, the first problem I ran into is that there’s no way in the UniFi Controller to override the device’s WAN MAC address. Lots of ISP CPEs lock on to a router’s MAC address and won’t accept DHCP requests from a new device without jumping through some hoops. Most any router you would pick up from Best Buy supports changing the MAC address.

Now, the USG runs EdgeOS, the CLI of EdgeOS supports specifying MAC addresses for interfaces, and you absolutely can get at the CLI of a USG. But… anything you commit will not be saved back to the UniFi Controller and will be lost when the device re-provisions.

The process of persisting CLI changes on the USG is to put them in a special file on the UniFi Controller, /usr/lib/unifi/data/sites/site-code/config.gateway.json. The format of this file looks similar to the format an EdgeRouter configuration file, except that it’s JSON.

Here’s a fragment from an EdgeRouter:

interfaces {
    ethernet eth2 {
        mac 00:0D:3A:27:02:78
    }

And an equivalent from a USG Pro:

{
        "interfaces": {
                "ethernet": {
                        "eth2": {
                                "mac": "00:0D:3A:27:02:78"
                        }
                }
        }
}

Now, this JSON format isn’t what you get when you type show. You have to exit from configure, then run mca-ctrl -t dump. Figure out which fragments you need, put them in config.gateway.json, and then tear out your hair when it doesn’t work because what you made wasn’t actually valid JSON. You missed the opening or closing braces. Or that options taking multiple values need brackets and commas. Or forgot to quote something.

The kicker is that /config/config.boot on a USG is exactly as it is on an EdgeRouter. This JSON mess is needlessly exposing you to an implementation detail of how UniFi and a USG interact with each other.

Spend enough time messing with USG JSON files and you’ll eventually get the hang of it, running all of your changes through a JSON validator before saving, but it’s a giant pile of suck.

My next problem was the DHCP server — it provides virtually no control over common DHCP options. They’ve added Domain Name since my initial setup but that’s it. You’ll need JSON for anything else.

No problem, I’ll set up DHCP Relaying and keep using my existing DHCP server! Oh, that’s more JSON, too.

DNS Overrides so my internal zones get resolved by my internal DNS servers? More JSON.

L2TP remote user VPN with local accounts? More JSON. (Now it supports L2TP for remote user VPN and a UniFi-managed RADIUS server)

At its largest, my config.gateway.json file was 147 lines.

Now, one of the biggest features people tout about the USG is its DPI reports. They’re very pretty. And very useless.

Whole lot of streaming media being used here, but I drill down trying to see who’s using it… and I seriously have no idea what I’m looking at here.

This is DPI on my EdgeRouter. It isn’t nearly as pretty, but I can easily see who transferred all those GBs.

Another problem I ran into was with VPN connections. The automatic Site-to-Site VPN between USGs is real slick, but comes with a major caveat: The USGs themselves cannot communicate across the VPN tunnel. They can’t talk to each other, nor can they communication with the remote LAN.

This creates a problem for a common remote office deployment scenario: The remote end doesn’t have a local DNS server for internal zones and needs to forward those queries to the central office’s USG or DNS server.

I’ve been told this might not be a problem if Dynamic Routing is enabled, and I’m told this will be fixed in the future by using a /30 assignment for the vti interfaces instead of a /32.

The final straw in my USG experiment was when I added a second ISP at home for failover load-balancing. None of my inbound port forwards worked on the second WAN interface, whether it was the current interface or not.

This is a limitation of how EdgeOS port-forward works — they’re tied to a single interface.

I also discovered that my Dynamic DNS doesn’t failover — also tied to a single interface and UniFi does not expose any option to use a different interface.

All of this could be worked-around with more JSON hackery for DNAT rules and additional Dynamic DNS entries… but I reached the point where migrating back to the EdgeRouter 8 was more appealing than continuing to work-around the USG’s limitations.

I want to love the USG. Single-pane-of-glass remote management of an entire networking stack is a compelling value proposition. UniFi’s UI is slick and makes many somewhat complicated things very simple.

But the USG remains an immature product only suitable for very simple deployments. I would absolutely deploy a full UniFi stack w/ USG to a family member’s home, where the ease of remote management greatly outweighs the USG’s limitations.

I would not deploy a USG for a small business client. Period. UniFi switches and wireless, absolutely! But the USG is dead to me as an SMB device until the UniFi Controller offers more control of DHCP and DNS, and they resolve the VPN communication issues. All of this is likely to come in UniFi v5.6.x but that is probably 4-6 months from reaching Stable.

My home USG Pro has been sold. I’m hanging on the the USG 3-port for my lab, and will soon pick up another, so that I can continue to track its progress and run mock deployments. And I’m definitely looking forward to the USG-XG becoming available — it’ll be great for 10Gb/s inter-VLAN routing even if the platform itself is still iffy as a gateway router.

UniFi 5.5.19 has finally gone Stable!

Posted on July 3, 2017 by ubntfan

Feels like I’ve been waiting for UniFi v5.5.x to make a Stable release for as long as I’ve been using UniFi. It has finally arrived!!

The list of changes from v5.4.x is massive. The biggest impact, IMO, is that nearly all of the deficiencies from Chris Sherwood’s (in)famous USG vs. EdgeRouter comparison have been addressed in v5.5.x. The USG remains far from perfect — for many SMB scenarios it is pretty close to unsuitable — but it has vastly improved over the past six months.

I’ll have my own write-up of where the USG shines, and falls short, in the coming weeks.

set-inform.com

Adventures in Ubiquiti Routing and Switching

Category Archives: USG Pro