Unpopular Opinion: Don’t use a Raspberry Pi for that!

A Raspberry Pi is great if you have a need for which it excels. GPIO, extremely low power requirements, tight space constraints. But the Pi should not be the first thing you reach for when “Unobtrusive and Inexpensive Linux Host” are the only requirements.

Years ago I migrated my Pi-hole from an actual RPi to a NUC-sized system based on the AMD GX-415GA that I paid $5 for bare bones, roughly $45 all-in with PSU, 4GB RAM, and SATA SSD. It’s not screaming fast but it’s still overkill for something like Pi-hole. More importantly, it boots faster than a Pi and the storage is WAY more reliable than micro-SD cards and those things are kind of a big deal when DNS being down effectively means the Internet is down. At about 7w in use the difference in power consumption is about a penny per day.

More recently I wanted to build a stack of Docker servers to run a couple Frigate instances and consolidate my sprawl of containers running within VMs. I bought this stack of HP Prodesk 600 G4 micro desktops for an average of $260/ea. Two came as i5-8500T / 16GB RAM, one i5-8600T / 8GB RAM, all with 256GB NVMe drives. That’s a lot of compute in a tiny package and I’ll be upgrading them all to 32GB / 1TB NVMe.

A complete Raspberry Pi 4 Model B 8GB kit is admittedly cheaper — typically $150 these days — but you can find complete i5-6500T systems in that ballpark with 8GB RAM and a hard drive or small SSD. Lower specs, or i3-6100T systems, can get down to $100. Again, for the money a 6th-gen Intel CPU is a ton more compute than a Pi, provides faster and more reliable storage, and you don’t have to put up with the quirks of Raspian or running an alternative distro that has zero community.

Granted, these are systems that will idle at 10-15w and can hit 55-60w at 100% load. There are situations where that may be unacceptable but that’s probably not the situation when you’re building a tiny Linux server at home.

Once you start down the rabbit holes of Pi-hole and Home Assistant, you’ll probably acquire a bunch of other things to run — I’m at 10 distinct Docker-ized stacks and have a few more things to migrate — and you’ll be happier having starting with one system that’s overkill for everything you’ll want to throw at it than accumulating a bunch of limited-purpose RPis that you’ll eventually want to consolidate on something more powerful anyways.

If you’d like to learn more about tiny PC options, check out ServeTheHome’s TinyMiniMicro series. I specifically looked for HP G4 systems from the MP9 / ProDesk 600 / EliteDesk 800 lines because they have dual M.2 M key sockets plus an A+E key, which provides maximum flexibility for NVMe storage and Coral TPUs.

Repeating Old Mistakes

Early last month, my 20-month old UNVR stopped working. I pulled the drives, tried the Reset button, and thanked the Deities that I live in an area where a UNVR is something that I can buy. In a store. On a Sunday.

At the time I’d seen hints that wearing out the internal storage was not uncommon. What I hadn’t learned, yet, was that the internal storage is a USB stick. My discovery of this was accidental — I was mucking around on my new UNVR and decided to run lsbusb -tv and there it was.

With a quick search of the Googles I found fresh knowledge that it is, in fact, a generic USB stick, and that replacing it is as simple as putting in a blank drive and holding the Reset button on boot. I guess they learned some lessons from the EdgeRouter Lite USB failure debacle. Just. Not the lesson that they should never put a USB stick in a device!

Old and Busted. Blast it with a heat gun or your girlfriend’s blow dryer for a moment to release the glue.
New Hotness, Samsung Fit 32GB.

I’m tempted to hack this “extra” UNVR into a NAS, though I have concerns about what could go wrong if the USB fails again. I’ve had great luck with Samsung Fit drives but maybe an M.2 SATA SSD in a USB adapter would be a better option.

Migrating

This site has been running from my home Internet connection from Day 1 but my determination to get control over my Docker disasters finally overcame my inherent don’t fix shit that ain’t broke laziness. Now coming at you live from colo in 55 Marietta Street.

Now to work on finding some motivation to create some fresh content…

UniFi OS Hacking

With the UDM / UDM Pro I’ve been regularly expressing disappointment that Ubiquiti is transitioning to a custom Linux distro that doesn’t have a package manager and doesn’t really have any provisions for persisting anything useful across reboots — particularly configuration changes and mechanisms to launch your own scripts.

With the second-stage transition to “UniFi OS” they’ve been moving more things into containers and it has now spread from the UDM Pro to the UNVR-4, which was previously running straight Debian with no containers.

Yesterday it was pointed out to me that “UniFi OS” isn’t merely a re-branding of the “UbiOS” the UDM debuted with. The unfi-os container is a full Debian environment. A quick investigation on my UDM Pro showed that I could enter the unifi-os container, apt install software packages, and make changes which persist across reboots. It would appear that all changes within the container are persistent via an overlay for / which goes to persistent storage on the host.

This is not at all how Containers are supposed to be used, it is a gross violation of best practices… but it’s a foot in the door to using these devices in ways that Ubiquiti didn’t bless.

I’m super-disappointed that nobody seems to be exploring the unifi-os container in public. Google turns up nothing, there hasn’t been anything meaningful on /r/UniFi or ubntwiki.com. Probably all hidden on the Discord.

I don’t love UniFi Threat Management and neither should you

When Ubiquiti put out the first Beta releases of IDS / IPS, I was surprised by the overall excitement of the enthusiast community. People were snatching up $2,000+ USG-XG-8s just to be able to use this feature without slowing down their WAN. Cheaper line-rate IDS / IPS has been a major force behind the UDM / UDM Pro hype train.

I am far less enthused, about IDS / IPS specifically and UniFi Threat Management in general.

My core issue: It’s Free.

Free is not inherently bad. I use lots of things that are free. Ubiquiti is built on lots of things that are free — VyOS, Linux, OpenWRT, hostapd. But security is different. Security is a process, not a deliverable.

Look at how Ubiquiti is offering security. Did they produce an IDS / IPS product? No, it’s just Suricata. Are they employing an army of security professionals to discover new threats, analyze alerts from their customers, produce new signatures? No, they’re just passing along the free rulesets that any Suricata user has the ability to use. Is Ubiquiti adding any value to Suricata alerts, such as making them easier to interpret or correlate? Heck no.

There’s a saying that If you’re not the customer, you’re the product. Does that apply here? Also, no. Enabling UniFi’s IDS/IPS isn’t passing any value back to the Open Information Security Foundation, the Suricata project, or any of their sponsoring organizations.

My other issue: What Ubiquiti is providing isn’t particularly good.

IDS / IPS: Suricata is purely signature based, like an antivirus program from 30 years ago… only worse. Most signatures are simplistic, prone to false positives, and the alerts they generate do not provide much context to decide whether they warrant further investigation or can be ignored. I’m not positive about this, due to the minimal information provided, but I think many of the alerts I’ve received are for traffic that the firewall was going to drop anyways.

There are many things Ubiquiti could be doing to add value to Suricata alerts — pruning rulesets and providing more context would be a great start — but that would require substantial investment.

Zeek (Bro) is generally considered better than Suricata, in that it’s focussed on flagging anomalous traffic instead of depending on signatures, but turning that into a user-friendly solution also requires significant investment.

DNS Filtering: I’ve previously written about this. In short, they’ve delivered an extremely simplistic filtering solution that depends on redirecting DNS traffic to an undisclosed 3rd-party. The manner in which they’ve implement filtering is unsuitable for a broad range of common DNS scenarios and they’ve provided zero control beyond choosing from the 3rd-party’s three filtering options.

It’s not worth using. Use PiHole and / or OpenDNS at home. Subscribe to Cisco Umbrella for business filtering.

Network Scanners: The Endpoint Scanner is basically a point-in-time nmap. No history, no correlation with UniFi’s Client History data. The Internal Honeypot is also extremely simplistic — it seems to alert simply on connection attempts to particular ports.

GeoIP Filtering: Hey, it does exactly what’s expected! I wish it did more tho. In particular, I might like to drop traffic from some countries to particular ports (VPN-related) but not others (HTTP / HTTPS).

IP Reputation: Tor blocking and Restrict Access to Malicious IP Addresses do what they say, tho again, it’s unclear what the information source is and if 3rd-party disclosures are involved.

Missing Features: Competing Unified Threat Management solutions generally have features that Ubiquiti isn’t (yet) providing: A/V scanning, HTTP / HTTPS interception, email filtering, data loss (PII / PHI exfiltration) protection, integration with Network Access Protection / Network Access Control systems, and more.


 

At some level it’s great that Ubiquiti is making security tooling available to users with less technical expertise or budget. What’s not great is those are the demographics who will read the marketing and believe they’re getting much more than is actually being provided.

The Real MGP

UniFi Management Gateway Pro, that is. Who comes up with these names?

UMG-frontUMG-back

Freshly announced, ahead of being available for Early Access purchase, we have essentially a UDM Pro… minus the switch, minus the HDD bay, half the RAM… and no local controllers! Adopt it to your Cloud Key, cloud-provider, or self-hosted UniFi install.

It also has a built-in UniFi Smart Power Plug. I can understand why — reboot your Cable / DSL “modem” if the Internet goes down — but it’s just such an odd thing to integrate into what is otherwise a plain router.

I’m happy to see this, tho disappointed that it’s not in more of an ER-4 form-factor with an optional rack mount kit. Hopefully this is a sign that a smaller-but-equally-capable desktop unit will come eventually.

I’m also hoping that this is a sign that in the future it will be possible to disable all local controllers on the UDM / UDM Pro.

Update: They’ve tweaked the original post, now it’s “UniFi Managed Gateway Pro” and the UMG Pro will be the first of the “UniFi Managed Gateway Product Line.”

Update #2: Renamed again, UXG-Pro. If Ubiquiti is anything, it’s consistently inconsistent. 

UniFi Internet Security DNS Filtering

Today I was prompted to figure out what exactly the DNS Filters settings in UniFi Internet Security are doing.

unifi-dns-filtering

The main thing that happens is that the DNS queries for the associated VLAN are forwarded to the cleanbrowsing.org public resolver for the chosen filtering category. This is accomplished by creating a new dnsfilter network interface, binding another instance of dnsmasq to it, and using NAT to redirect DNS queries from the associated VLAN.

The implementation is slick from a technical perspective but I have a couple major problems with it.

Firstly, cleanbrowsing.org supports DNS-over-TLS, DNS-over-HTTPS, and DNSCrypt, but Ubiquiti has chosen to forward the queries unencrypted.

Secondly, if you’re using a local DNS server, say Active Directory or PiHole, those NAT rules will prevent the local DNS from functioning when queries are coming from or destined to a filtered VLAN. This might be fixable with some config.gateway.json or scripting hackery on the USG, I have not dug into that, but UDM users are hosed since there is no json replacement nor a way to automatically run a script.

Thirdly, Ubiquiti could have licensed the cleanbrowsing.org RPZs, converted them to dnsmasq format for local blacklisting ala PiHole, and eliminated the need to forward DNS to a 3rd-party and interfere with cross-VLAN DNS.

I cannot recommend using this DNS filtering because it will cause issues if your DNS implementation isn’t “UniFi Default” and the filtering options are minimal. My personal preference for home use is a PiHole in conjunction with OpenDNS Home.

For an office, Cisco Umbrella is the way to go. Do not underestimate the value of having a commercial relationship with a security provider. I <3 PiHole, but if you try to figure out why it blocks, say, zombo.com, and how you’d get it removed, you will immediately recognize that “free” security lists aren’t worth more than you’re paying for it.

Update: Also want to point out that the dnsfilter interfaces use IPs in the 203.0.113.0/24 subnet. That is TESTNET-3 in RFC 5737. It’s unlikely this will ever cause a problem… but over the long haul, every misuse of an IPv4 subnet ends up causing a problem. The loopback range 127.0.0.0/8 or link-local 169.254.0.0/16 would have been more appropriate choices. 

Isolating UniFi Devices

I’ve been lax in firealling my VLANs at home, but with the recent controversy over UniFi devices phoning home without consent, this has taken on renewed importance. I’m also taking on a new tenant in my detached apartment and would like to keep all their stuff segregated from mine.

Fortunately it’s all pretty easy.

For my network, I’m keeping my Cloud Key and Pi-Hole on the main LAN. I have additional Corporate VLANs created for Management, Cameras, and the Apartment.

In the Firewall rules, for WAN_OUT I’ve created two rules to Drop all traffic from the Management and Cameras networks. They cannot reach the Internet at all.

To allow and deny particular cross-VLAN traffic, the first step is to create a group of all the Private IP address ranges:

  • 10.0.0.0/8
  • 192.168.0.0/16
  • 172.16.0.0/12
  • 100.64.0.0/10 (this is CGNAT, might be a bad idea if your ISP uses them)

Back in the Rules area, LAN_IN needs a series of rules:

  • Allow (Management | Cameras) networks access to the CloudKey.
  • Allow (Management | Cameras | Apartment) networks access to the Pi-Hole.
  • Drop (Management | Cameras | Apartment) networks access to the Private IP ranges group.

The Allow rules must be before the Deny rules for each Network.

The gotcha with denying devices access to the Internet is that they cannot directly obtain firmware updates. For UniFi Networking products this can be worked-around by having the UniFi Controller cache the firmware prior to upgrading — see Settings -> Maintenance -> Firmware.

I’m not sure whether Protect can distribute firmware updates to the Cameras. Guess I’ll find out the next time there’s an update available. Once my UniFi Protect NVR arrives I will place that in the Cameras VLAN so that traffic doesn’t have to cross the router and figure out the WAN_OUT / LAN_IN firewall rules needed to keep it happy.

UniFi Protect NVR

A bigger NVR for UniFi Protect finally hit the Early Access store early this morning… and was sold out by afternoon. Four LFF/SFF bays, support for RAID1 and RAID5, 1x GbE and 1x 10GbE SFP+ ports, and a port for the UniFi SmartPower Redundant Power System (also in EA). No details on the CPU or RAM but I’d presume it’s the same platform as the Cloud Key Gen2 Plus.

At $299 in EA it seems like a decent enough value.

I’ve ordered one, as I’m just finally starting to put Protect to real use in my backyard and the 5TB max drive size on the CKG2+ will soon be a problem. Hopefully I’ll find time to get the rest of my G3 Flex cameras up before the New Year.

unifi-protect-screenshot

UDM Pro thoughts

UDM ProI finally broke down and added the UDM Pro to my Ubiquiti router lab. Here are the specs as provided in the Early Access store:

  • 8-Port gigabit switch with 10G SFP+ port
  • Dual WAN ports for redundancy and load balancing: 10G SFP+ and 1G RJ-45
  • Bluetooth connectivity for easy setup via UniFi app
  • Scalable UniFi Network Controller with advanced management capabilities
  • UniFi Protect video surveillance NVR with 3.5″ (or 2.5″) HDD support
  • Enterprise-class IPS/IDS and DPI capabilities
  • 1 x 1.3″ Touchscreen display for quick status information
  • Powered by fast 1.7 GHz quad-core processor

Not mentioned is that the UDM / UDM Pro use a new OS that is not derived from EdgeOS / VyOS / Vyatta. This is why I’d held off on buying one, when the UDM was first made available to Early Access back in March it was FAR from having feature-parity with the EdgeOS-based USGs. The present state of UbiOS is much closer to production-ready (by UniFi standards).

From the perspective of the USG Pro, this is a pretty serious upgrade in performance with a very minor bump in the expected MSRP. 10Gb/s inter-VLAN routing and WAN. Supposedly can hit 5Gb/s of IPS/IDS throughput — I’m underwhelmed by Ubiquiti slapping a pretty face over open source Suricata with publicly-available lists, but I seem to be a minority.

It’s also quiet. With it sitting on my desk and the LCD showing the fans at 50%, I struggle to hear it. The USG Pro, USG-XG-8, and their EdgeRouter siblings are not quiet-space-friendly.

I feel like they’ve missed the boat in a couple of areas with regards to Protect.

One, those LAN ports should have POE. As a router, those ports are of marginal value — where a $379 router is justified, it’s going to be attached to a larger switch. But as an NVR with a larger storage capacity than the Cloudkey Gen2 Plus, having 8 PoE ports would cover many deployment scenarios and would be quite valuable (12-16 ports would be better).

Two, it should have more drive bays. 2x LFF would have been nice. More LAN ports w/ PoE and 4x SFF bays could be better.

Three, it needs a USB host port for offloading footage directly to removable storage. As things stand now, pulling footage out of Protect is a pile of suck, but presumably it will get better, and being able to push footage directly to removable storage would be a great feature.

Ubiquiti has teased a larger, 4x LFF device. It’s not clear if that will be a NAS that Protect can use, or if it will run Protect directly, and they haven’t shown the back yet so we don’t know if it has PoE switch ports to act as a more traditional NVR appliance.