6 ms·
I've gotten into networking and very much enjoy seeing how others approach the problem space at the scale level! It's gotten more important with not just the ex
by xoa 5y ago
I've gotten into networking and very much enjoy seeing how others approach the problem space at the scale level! It's gotten more important with not just the explosion of online threats, smart home devices etc., but also just because there is more and more value to be extracted from a high bandwidth WAN link via your own private services. That said I definitely strongly recommend against using anything UniFi based for routing/gateway tasks at this point (author mentions their AIO custom hardware UDM thing). They have significant issues now in WiFi and even switching, but at least remain kind of workable there. But the company has been an absolutely depressing development dumpster fire for years now and the trajectory still appears aimed straight down. And for whatever reason routing/gateway in particular is an area that only ever briefly got much oxygen there. So for a greenfield situation I'd be wary.
There are lots of enormously better solutions for that. Personally I've dumped all my UniFi gateways (including some of the "next generation" ones I had for testing) and moved to OPNsense running on decent SuperMicro 1U edge systems (excellent value, quiet, regularly available dirt cheap on ebay and the like). But there are many great options there including VyOS, OpenWRT, or even just running straight OpenBSD if that's what you like. Router/gateway is one area where I thinking devoting real metal, preferably with higher reliability hardware and extra remote management options (like IPMI), is worth it. It's an important linchpin for a typical network, and while virtualizing it is possible and definitely desirable at large scale it's not where I want to add more moving parts without a team behind it. Everything else can go tits up while I mess with it or there are odd interactions and as long as the gateway/routing is still solid odds are I can recover without much disruption or having to physically be there.
- adrianpike 5y agoI'm about to do a big revamp of my home network and would love any specific gear recommendations to research - are these the supermicro setups you're talking about? https://www.supermicro.com/en/products/embedded/fanless-and-iot-gateway https://www.supermicro.com/en/products/embedded/fanless-and-...
- xoa 5y agoI've used one of those in a satellite location and liked it ok, but what I actually ended up getting generally were full 1U replacements for the USG-4/UXG from SuperMicro's 5018D series. I kept an eye on ebay and found a bunch of new 5018D-FN8Ts listed for ~$650 a pop. They've got dedicated IPMI (obviously this needs to be secured), and unlike HP or Dell they include virtual iKVM for free and a full license is only $30. Pricing on the others can easily add +$200 for what should be included functionality even if they've got a spiffier coat of paint, which is significant on a low end server device. The 5018Ds are not fanless, but running they drop down to a load speed that I can't hear over anything else I've got (like a POE switch, all of which except the tiniest models have fans). HP's Proliants in contrast are jet engines all the time even when they're pulling <50W. I'm not running a total silent environment and preferred to make a rack closet instead well away from any living areas, but I don't want loud noise and SMs have been fine. Of course, one can just replace the fans with 40mm Notcuas or the like too if desired. They've got Xeon processors that have a bit more oomph than the Atoms or Celerons and also means ECC memory, since again I really want to be able to rely on gateways to a reasonable degree (this has already identified one bad memory module out of the dozen I had around). I install OPNsense on ZFS on a cheap small decent NVMe drive (PNY CS1030 250GB is around $35) which still means fast boot and no concerns about all the logging or the like I could desire. I have one single site that is doing much heavier 10G+ routing and usage that I also wanted to mess with more intensive SDN and security with. For that last year I ended up picking the much beefier and much more expensive EPYC Embedded based 5019D-FTN4 and putting a Mellanox card in it. It's also extremely quiet and has been really impressive, but that's stupid overkill right now. Also, EPYC Embedded is currently still based off of Gen1, there was no Zen3 update due to not having low TDP given the way the chiplets were upgraded vs the IO chip. I expect Zen4 next year will see an upgraded Embedded platform that will essentially be a 3 gen leap forward, so at this point not the best time anwyay. There is no perfect solution IMO. Though probably nothing that'd throw off the typical HNer, OPNsense does have its warts, rough edges and missing bits (no Webauthn so no security keys for login for example). It's based off of a FreeBSD variant (soon to be directly off FreeBSD) with all that comes with that for better or worse. Like, OPNsense does have a user space plugin option for WireGuard (along with ZeroTier and so on), but WG has not yet made it to the FreeBSD kernel which in some situations could be an issue (countered by raw CPU in my case). But it's powerful, well maintained, overall fairly user friendly, has pretty solid documentation and getting started guides, and a nice community with a good mix of developers and some companies behind it. The company Deciso for example does offer a paid business edition and paid support options if it's desired. It does have DNS Blacklist options ala PiHole, stats/telemetry/IDS/IPS via built-in and 3rd party offerings like Sensei, etc. There are plugins for Let's Encrypt, FreeRADIUS and other handy functionality. Someone who is very familiar with Linux might find VyOS more worth looking at but with my background I found OPNsense reasonably pleasant to get into. The decision tree here does also depend on how much network functionality you want to have into your gateway/routing system vs how much to stick on a separate server elsewhere (maybe virtualized or as a part of a NAS). Gateways can be very minimal or can handle damn near everything on the network. There are straight forward tradeoffs there in terms of failure modes and complexity.
- pyther24 5y agoFor router hardware look at the PC Engines APU2. Sadly they appear to be out of stock until 2022, but they are great low power devices. I ran OpenWrt on mine in the past and currently running Fedora. Can do about 1Gpbs and cost around $200 built. For managed switches, look at the Aruba Instant-On 1930 series. I've just ordered two of these so I don't have any first hand experience, but the feedback online is generally positive. Do note, these switches can only be managed from HTTPS, but the interface seems clean. From my research the cheaper Tplink and Netgear don't have an isolated management interface, meaning it can be accessed from all vlans. This was a deal breaker for me. I also considered the HP Officeconnect 1820 series switches, but they've been out for a while and I worried their EOL may be coming up shortly. For Access Points, look at the Aruba Instant-On AP 22. The biggest down fall is the access point uses a cloud controller, requiring an internet connection to manage the device. There is no local management. This is the same exact hardware as the Aruba AP-505 which runs for ~$400. Given Aruba makes solid wireless hardware, the advanced features compared to other units in this price range, and the lower price point, I'm willing to give up local management control. After all, I don't modify my AP that often. My Ubiquity access point has crashed multiple times, the most recent crash appeared to be memory leak. Maybe I'm just unlucky, but this aligns to numerous complaints about firmware quality.
- gerdesj 5y ago"From my research the cheaper Tplink and Netgear don't have an isolated management interface, meaning it can be accessed from all vlans." For Netgear you want a T eg GS724T(P) in the model which implies smart switch (web managed). They do have command line managed ones too ie "managed switch". For home use, decent L1 and 2 is enough - you don't want to do L3 switching in general, unless your house is at the top of the Mall. So, GS724TPv2 and GS110TPv3 get you a 24 or 8 port PoE+ switch with L1 and L2 covered in a web interface with VLANs etc. The newer interface with the PVID section that shows tags and ports n LAGs at a glance is one of the best, regardless of price or status. For the money those switches are quite hard to beat.
- rsync 5y ago"For router hardware look at the PC Engines APU2" I use a lot of these in my personal networking setup but I am going to start moving to a raspberry pi CM4 built into this dual-ethernet board: https://www.seeedstudio.com/Dual-GbE-Carrier-Board-with-4GB-RAM-32GB-eMMC-RPi-CM-4-p-4898.html https://www.seeedstudio.com/Dual-GbE-Carrier-Board-with-4GB-... It's a smaller footprint, more CPU horsepower, etc., and I like mounting little devices onto DIN rails more than I like trying to rackmount the 7" or 9" APUs ...
- vdqtp3 5y agoWhat did you move to for wifi? Gateway/firewall/edge is easy to replace, but I haven't seen much that could compete with UBNT at their price point.
- xoa 5y ago>What did you move to for wifi? I haven't moved everything on WiFi or L2 switching yet and am still evaluating that, poking at different solutions from a variety of places across price ranges (from MikroTik to Peplink or Ruckus). Part of the true pain of all this is that there isn't any clear obvious successor to all the things Ubiquiti aimed for or else a lot more of us would have probably bailed completely already vs drawing things out. Not merely the price point, which I'd actually be fine paying somewhat more for given how important it is, but zero cloud dependency, the fairly pleasant (though getting worse) single pane of glass for managing things, and decent physical design that doesn't look like giant mutant space bugs all matter. However, even in WiFi it has started to get flakier and while the older WiFi 5 devices which benefited from what was once a lot of great engineering talent have held on my experience with the newer WiFi 6 stuff has been mediocre. And the controller has continued to go downhill as well, actively removing useful functionality and information density as they keep messing with the UI every single version in the most classic bikeshedding fashion :(. WiFi 6E/WiFi 7 and multigig will I think be the major decision point, so another 1-3 years. 6 GHz spectrum will be genuinely very useful in a variety of settings, so we're going to be looking at a point where it'll be desirable to replace a lot of hardware anyway. Once there is that kind of commitment that's going to represent not wanting to change again for another 5-7 years, well, might as well reevaluate. And I hate it but I just don't see Ubiquiti getting better. I will say that with a touch of irony, Ubiquiti's rot spiral has actually reinforced the value of Ubiquiti's approach IMO. Because such things can happen to any company, and at least with UniFi/UNMS the self-host control thing has worked out exactly the last-resort backstop we all always said it could. There isn't any required cloud tie. There isn't any forced firmware updates. The system can be quite well isolated, with routing/gateway gone the rest can go onto a dedicated management VLAN with zero ingress/egress. That leaves a lot more runway even without updates. I'm sorry it was needed but it makes me more determined then ever to avoid remote dependencies. So I'll thank Ubiquiti for that I guess :\.
- 5y ago
- ubntthrow 5y ago> That said I definitely strongly recommend against using anything UniFi based for routing/gateway tasks at this point (author mentions their AIO custom hardware UDM thing). They have significant issues now in WiFi and even switching, but at least remain kind of workable there. But the company has been an absolutely depressing development dumpster fire for years now and the trajectory still appears aimed straight down. This is depressing to read because I worked a Ubiquiti before things got bad. The company culture took a huge dive when the CEO began removing executives and managers that were getting things done. The company was always weird, but when I started they were good about finding strong engineers who cared about networking and letting us build good products. Everything changed when they tried to reoganize the company around the UDM and move development to China. The UDM reorg was so awful that employees and executives were leaving in droves. For months we couldn't even tell who was supposed to be in charge of UDM or UniFi because so many people left at once. I think we had 4 different UniFi lead developers get hired and then quit a few months later after the early employees all left. Sad situation. Ubiquiti was a cool company that paid well and let engineers do good work when I joined. I still wonder why it had to fall apart so fast. The company was never perfect but it was sad to watch the CEO drive away all of the good parts of the company.
- omreaderhn 5y agoIs there a company that these people are going to? What Ubiquiti/Unifi alternatives do you like?
- bradknowles 5y agoI’ve heard good things about MicroTik. Not as consumer-friendly as Uniquiti Unifi, but solid engineering from a company that seems to really understand networking at the 1/2.5/10/25/40 Gig Ethernet scale for anything from SOHO to small/medium size businesses. Don’t have any personal experience with them, however.
- nikomen 5y agoMicroTik is solid and I used one of their routers for a few years. However, there's no support with mDNS across VLANs. You can obviously roll your own solution, but who has time for that? I prefer to have my IoT devices on a separate VLAN. Without support from the router, you're SoL if you want your devices to be discoverable by HomeKit and other services. At the moment I'm using Unifi for my router, but I'd consider switching to something as secure as Mikrotik but with support for mDNS across VLANs.
- jiveturkey 5y ago> enormously better solutions Strongly disagree, even if you disregard price point. Unless you are limiting your criticism to the USG/UDM and routing. Which I think you are, but just want to be sure. For the wifi and switching they are very very hard to beat. For normies and techies alike. I only dislike that the APs are underpowered so you need to carpet bomb your house if you have a large footprint. For dense living it's great though. Even for large area (blocked by walls) it's ok if you don't mind filling in the gaps with mesh. By the time you buy 1 AP and 1 mesh you are at $400 though. The gateway products are absolutely abysmal, even before considering the buggy new hardware.
- devnulll 5y ago> I definitely strongly recommend against using anything UniFi based As a counter point, mine works beautifully. UDM Pro, multiple Access Points, access point roaming, 2.4 and 5ghz networks, and all the goodness. Everything works flawlessly and it's been quite reliable. They seem to really enjoy spending money moving my cheese on the web site admin interface, and a few unexpected features seem to have vanished, but overall... there's nothing as good or centrally integrated. Everything else is a collection of point solutions, which is more than many (including myself) have the time for...
- rsync 5y ago" ... moved to OPNsense running on decent SuperMicro 1U edge systems ..." I agree with what you have done here and I will echo the sentiment to run away screaming from Ubiquiti/UniFi. It's easy to be confused by all of the professional ISP gear that Ubiquiti used to produce and think that the UniFi products (like the "Dream Machine") are professional networking equipment. Try blocking ubnt domains at your network edge losing the ability to log into UniFi ... you'll see how "professional" it is right away. I will also say that it is ironic, and sad, that you mention Supermicro as an alternative. Their recent moves make it quite clear that they are trying to move in the same direction ... which is to say, making wonderfully built products that fulfill useful (but boring) use-cases is never enough. Supermicro circa 2010 was making people rich but not billionaire rich ... and that is a problem they are working hard to fix.
- xoa 5y agoNot sure if you're still here or will see this, but: >It's easy to be confused by all of the professional ISP gear that Ubiquiti used to produce and think that the UniFi products (like the "Dream Machine") are professional networking equipment. Try blocking ubnt domains at your network edge losing the ability to log into UniFi ... you'll see how "professional" it is right away. The 'dream machine'/UniFi OS stuff is real trash and maybe where the rot truly showed to have completely taken over, though there were very bad signs earlier like when they trashed their self-hosted video solution and reversed earlier promises to restrict the follow up to their proprietary hardware (CK G2). Worth clarifying though that it's still perfectly possible to have a full self-hosted controller with local accounts, multiple sites and zero greater internet dependencies (route management VLANs through wireguard back to the L3 controller). The UDMs are in no way necessary. Their problems are more fundamental than that :). >I will also say that it is ironic, and sad, that you mention Supermicro as an alternative. Their recent moves make it quite clear that they are trying to move in the same direction ... I'm genuinely curious what you mean by that? At your scale of course you have far more insight into businesses of that size, but I honestly haven't seen any signs of that on the SM side nor is it quite clear to me how they would even go about that? They just make systems don't they? Even their IPMI is based off the AST2k series BMC, and they've been extremely reasonable about what they offer there in stark contrast to players like HPE. Their systems also aren't packed with proprietary junk that punishes you the instant you try to do anything else, again in stark contrast to HPE. They're just computers. Of course I'm only in at the lower end of their spectrum of offerings! >making wonderfully built products that fulfill useful (but boring) use-cases is never enough. Supermicro circa 2010 was making people rich but not billionaire rich ... One of the particularly frustrating/outright confusing things about Ubiquiti FWIW is that this isn't actually true. Ubiquiti did in fact turn Pera into a billionaire just by making wonderful products that fulfilled important use cases, and there were clear and loudly suggested natural revenue opportunities that they never even bothered with. Looking at all of their shitty moves and self-destruction, it actually doesn't generally have any sort of revenue ties! And indeed on the contrary, they've repeatedly skipped out on (accepted even) features that would flat out encourage more hardware sales and revenue (like L2 replicants to L3 control). They've actively spent major money and development time on things that people hate yet are completely free. It's not like all those new ever worse Controller versions actually are paid. It's not like they've added subscriptions and microtransactions everywhere. Even their cloud stuff again doesn't actually have any revenue story attached. They made a big deal about offering one of the most classic super high margin boosting things of all, "Premium Business Support Contracts", and then... just kind of abandoned it and let it die out pissing off a ton of professionals and business in the process even though it could clearly print money. It's not a matter of their being some cold money grubbing business logic to their moves that, however evil, one can understand the point of. There simply isn't any logic at all, beyond maybe trying to outsource so hard and create such a toxic environment that they literally just don't have the internal capability to execute on much of anything anymore. Outside of a few remaining decent people still plugging away a bit on bug fixes, most of the activity seems directionless, meetings and throwing stuff at the wall with no strategy or follow up, and ever widening set of products that sometimes get dropped before they are even out of "early access". So yeah. If anything the Ubiquiti of old would be doing fantastically better right now just because demand for networking and everything related has accelerated so much. It's just so stupid.