5 ms·

Depending on what network gear you're using (I'm now switching to Omada, previously/still on some site son UniFi, but lots of even 'prosumer' stuff does this) t

by xoa 4y ago

Depending on what network gear you're using (I'm now switching to Omada, previously/still on some site son UniFi, but lots of even 'prosumer' stuff does this) there are specific mitigations available. All of this falls under the heading of "managing broadcast traffic", which is very important even for smaller networks. The three major categories of traffic on a network are unicast, broadcast, and multicast. Unicast is the normal case of one device talking to a single address. Broadcast involves sending a packet to every single possible recipient in the entire broadcast domain (almost always the subnet). ARP fits in here. Multicast is essentially in between, more efficient then broadcast, can still talk to multiple devices that have signed up to hear it. Obviously an actual broadcast storm can take down an entire network, but excessive broadcast traffic on WiFi specifically can also suck up a huge amount of airtime for little bandwidth. Every single device has to go to the slowest speed and stop what they're doing to listen and make sure no one is left out. Using STP/RSTP with proper values set and LACP for aggregated interfaces can help prevent inadvertent network loops. Many switches also support some kind of port isolation and explicit per port storm control restricting max numbers of packets/second for unicast/broadcast/multicast traffic. WiFi APs can use proxy ARP to cut it in their domain too. The WAP already of course knows the MACs of every device connected to it by definition, so there isn't generally any reason not to have it answer ARP requests on their behalf then forward the traffic itself.

donmcronald 4y ago
Why the switch to Omada? I’m looking at doing the same due to supply issues with the UniFi stuff. Any tips or opinions on Omada vs UniFi?
- xoa 4y ago
  Opinions hooboy do I have them ;), I was just asked that exact question on HN a couple of weeks ago in fact. Gave it a shot in a response here [0] which still applies. But basically Ubiquiti has become a toxic dumpster fire of a company and their product lines (UniFi in particular) on a downward trajectory in terms of performance, features and stability for quite a while. I had a certain amount invested in UniFi (think the final total will end up as a few hundred devices) so it's been a staged switch, with a total change of all routing/gateway/security functions to OPNsense completed first. That bought a lot more runway, it's always been the weakest and most neglected area in the ecosystem while obviously also being pretty critical. Yet the Ubiquiti debacle has served to underline for me how valuable self-host is, I've been able to have a nice slow ramp and deal with their implosion precisely because UniFi/UNMS/UISP and all the hardware is fully under my control. So I've been hoping someone would come along and see the potential of the UniFi niche of the networking market and basically copy it without all the junk. Which seems to basically be Omada to a tee. I'd actually originally (and still at many sites) intended to hold off and wait for WiFi 7 gear, because at that point a bunch of clients (and myself for that matter) will be interested in replacing WAPs anyway which is a very natural point to consider changing manufacturer as well. But a breaking point has come at a few places with a final feature which is PPSK, allowing the system to have many different passwords for an SSID that can be assigned different tags. Basically it allows having many of the benefits of WPA-Enterprise in terms of segmenting different clients onto different VLANs and revoking credentials and the like with more security and less manual work than MAB (MAC bypass) while still looking like a normal PSK scheme, which means the vast universe of brand new stuff which doesn't support 802.1x and never will works with it happily (by the same token none of that is going to play directly with using a secure virtual network or other better systems either sadly). Lower overhead and better compatibility than captive portals for non or semi-interactive devices as well. Someone hacked together a demo showing this could work on UniFi WAPs like four freaking years ago and Ubiquiti never did anything with it in favor of endless bikeshedding GUI changes to add more white space and hide important features and information (yes I'm a touch bitter). So I'm not in the position of wholeheartedly recommending Omada yet, I don't have years under my belt there and it's relatively speaking fairly new. It has its own warts and rough edges for sure, from the software to the hardware physical design. But it can be self-hosted and the trajectory looks massively better, has already had more meaningful improvement in months than UniFi has had in years, seems to perform much better so far as well. Of course the Venn diagram of self-hosting, herding lots of hardware with single pane, fully networking features, ecosystem richness and so on is pretty minimal in the overlap. Take away any one or multiple of those and options expand a lot, Aruba InstantOn for example. And welp, this didn't end up "basically" at all did it, sorry about that. I am bummed by the sheer wasted potential with Ubiquiti. So it goes in tech over and over again though, we've all seen this movie many, many times. As far as tips, I would suggest if you plan to stay on the managing-your-own-networks route to very strongly consider having the router/gateway stuff be separate and fully open source as I ended up. Doesn't have to be OPNsense, could be VyOS or plain OpenBSD or whatever else you're most comfortable with and depending on how you want to manage stuff and what needs there are for others to take over. But it's very, very pleasant to have the full spectrum of quality PC hardware available, you can get far more power for less, and you're never stuck with a critical aspect. I'd still suggest generally running that on metal rather than virtualizing it in a (semi)production network, but opinions vary there. ---- 0: https://news.ycombinator.com/item?id=32297556 https://news.ycombinator.com/item?id=32297556
  - donmcronald 4y ago
    Thanks for taking the time to reply. I've also soured on UniFi a lot over the last few years. It's crazy. The SMB / MSP market is massively underserved and instead of staying focused and capturing it Unifi is trying to compete in the enterprise market where, IMHO, they aren't going to succeed. If we try out Omada and feel like it gives us everything Unifi does without all the feature creep then we'll do the same thing as you; EoL sites will move from Unifi to Omada. I'll always take fewer features and stability over continuous new features. > I would suggest if you plan to stay on the managing-your-own-networks route to very strongly consider having the router/gateway stuff be separate and fully open source as I ended up. We've used pfSense forever and have been happy with it. However, 2.6.0 has some gateway/dpinger issues where the bug fixes are slotted for 2.7.0 on the CE side, so it feels like the inevitable neglect of the CE version might be starting to kick in there. I tried to explain to them the realities of the market we're in (small businesses) along with real numbers in terms of what kind of money they could extract from businesses like ours and all it got me was an offer to talk to someone in sales and we're not buying into subscriptions (ever). It would cost us $15k+ / year to switch to pfSense+. The software only subscription is nearly 2x the cost of buying their entry level appliance (assuming a 5 year lifecycle). IMHO, that's an indicator they don't want to support 3rd party hardware and there's a risk they'll eventually drop it. I wish VyOS had less crazy pricing because their stuff looks nice. They used to have a $600 / year professional subscription which they don't appear to offer anymore. I think the corporate subscription used to be <$4k / year and now it's $6k. I wonder why no one wants to commit to subscriptions. How can I sell anything to a customer when I can't give a predictable price for ongoing maintenance a year into the future? Our customers will literally go back to using their ISP supplied modems instead of proper firewalls before they'll pay the prices everyone is asking for. I don't blame them either. They only need a fraction of the functionality being sold and all of the fancy, expensive features everyone is using to justify sky high pricing are simply bad value in that context because they'll basically unneeded bloat.
    - xoa 4y ago
      >Thanks for taking the time to reply. No problem, glad to chat about this and commiserate a bit with someone in a similar boat. >I've also soured on UniFi a lot over the last few years. It's crazy. The SMB / MSP market is massively underserved and instead of staying focused and capturing it Unifi is trying to compete in the enterprise market where, IMHO, they aren't going to succeed. Yeah, it's such a huge waste. But the CEO is a classic trouble case with near total control (owns majority of the stock and thus effectively owns the board, a few caveats in terms of protection for minority shareholders in public companies don't really stop the trend). >If we try out Omada and feel like it gives us everything Unifi does without all the feature creep then we'll do the same thing as you; EoL sites will move from Unifi to Omada. I'll always take fewer features and stability over continuous new features. UniFi hasn't even had "continuous new features" though! Or rather, their "features" have been stuff like reskinning the GUI or doing their own entire custom silly speedtest stack for a while, not "features" like PPSK, or for that matter just having normal DHCP and DNS management! Or any sort of certificate management or customization, or, like, anything of substance. Or on hardware, they didn't even bother to upgrade their UniFi Security Gateways literally ever, not once! The ones still selling now are the exact same hardware as 2014 launch, no new updates in forever. They intro new stuff without ever sunsetting older stuff in general really. Use "EA" as a popularity poll test. The dysfunction goes on and on. I don't know if Omada will be the answer yet but worth a spin IMO. I'd definitely hate having to either go back to lots of individual management or something cloud-based. I am stick with UI for PtP/PtMP stuff for the time being though, even if some rot is creeping in there too. >We've used pfSense forever and have been happy with it. However, 2.6.0 has some gateway/dpinger issues where the bug fixes are slotted for 2.7.0 on the CE side, so it feels like the inevitable neglect of the CE version might be starting to kick in there. I'm afraid that company has a history of scummy behavior too :(. I think OPNsense is just plain better but ymmv. FWIW, while it's OSS there is at least one company (Deciso) that does a "Business Edition" at a relatively sane rate (~$150/yr, less for a 3 year) with no hardware reqs (they sell hardware too but I don't suggest getting it) and does business support contracts as well. And Sunny Valley Networks does an interesting little layer 7 inspector that can plugin to it. It conversely might not be good enough for your needs either, but it's not crazy. And the free version is still quite solid. As I said in another comment the docs are pretty decent so might be worth giving them a glance and firing up a system in a VM. At any rate my clients are happy now. And despite rough edges I'm grateful there are a number of essentially decent options available.
- kerr23 4y ago
  I did the same, originally planned on UniFi but since i couldn’t get any i tried Omada. It’s very nice, the only issue i’ve had is my mesh not connecting, but i solved that by reorienting the target AP and boosting the source APs radio. I think i have some tuning left though because my devices don’t switch APs as fast as i’d like.
zamadatix 4y ago
For home I wouldn't really bother with storm control. In the rare case something is actually misbehaving in that regard it's not much work to go fix right and I'd rather do that than mask it with storm control so it's still happening just not enough to cause an obvious signal. For home wireless another trick, assuming you have a higher end model, is setting it to a different network than the wired. If you're not in particular need for slow service at the very edge of the coverage zone you can also raise the minimum data rate and then BUM traffic and wireless management frames won't be as painful even when they do occur.
- vladvasiliu 4y ago
  > In the rare case something is actually misbehaving in that regard it's not much work to go fix right and I'd rather do that than mask it with storm control so it's still happening just not enough to cause an obvious signal. While I agree with the general idea, I think that there are only so many hours in a day. If the broken device isn't easily fixable, like most IoT junk out there, it's nice to be able to limit the damage.
  - zamadatix 4y ago
    There are better network only solutions for unfixable devices when you have that class of gear, such as moving it to its own VLAN which is a good practice for even properly functioning IoT devices. Storm control has more a "John brought a home Netgear (which ignores but doesn't forward spanning tree packets) and created a loop between two wallports despite spanning tree being enabled on them. Thanks to storm control Jane is still able use her IP phone for a meeting while the network admin investigates a monitoring alert about excessive broadcast drops on John's port which is causing slowdowns in that network" type use case. It's not meant to be the long term solution to anything it's just there for when other long term solutions fail to kick in and you don't want a full outage until you can fix them.

by xoa 4y ago

donmcronald 4y ago

Why the switch to Omada? I’m looking at doing the same due to supply issues with the UniFi stuff. Any tips or opinions on Omada vs UniFi?

xoa 4y ago
Opinions hooboy do I have them ;), I was just asked that exact question on HN a couple of weeks ago in fact. Gave it a shot in a response here [0] which still applies. But basically Ubiquiti has become a toxic dumpster fire of a company and their product lines (UniFi in particular) on a downward trajectory in terms of performance, features and stability for quite a while. I had a certain amount invested in UniFi (think the final total will end up as a few hundred devices) so it's been a staged switch, with a total change of all routing/gateway/security functions to OPNsense completed first. That bought a lot more runway, it's always been the weakest and most neglected area in the ecosystem while obviously also being pretty critical. Yet the Ubiquiti debacle has served to underline for me how valuable self-host is, I've been able to have a nice slow ramp and deal with their implosion precisely because UniFi/UNMS/UISP and all the hardware is fully under my control. So I've been hoping someone would come along and see the potential of the UniFi niche of the networking market and basically copy it without all the junk. Which seems to basically be Omada to a tee. I'd actually originally (and still at many sites) intended to hold off and wait for WiFi 7 gear, because at that point a bunch of clients (and myself for that matter) will be interested in replacing WAPs anyway which is a very natural point to consider changing manufacturer as well. But a breaking point has come at a few places with a final feature which is PPSK, allowing the system to have many different passwords for an SSID that can be assigned different tags. Basically it allows having many of the benefits of WPA-Enterprise in terms of segmenting different clients onto different VLANs and revoking credentials and the like with more security and less manual work than MAB (MAC bypass) while still looking like a normal PSK scheme, which means the vast universe of brand new stuff which doesn't support 802.1x and never will works with it happily (by the same token none of that is going to play directly with using a secure virtual network or other better systems either sadly). Lower overhead and better compatibility than captive portals for non or semi-interactive devices as well. Someone hacked together a demo showing this could work on UniFi WAPs like four freaking years ago and Ubiquiti never did anything with it in favor of endless bikeshedding GUI changes to add more white space and hide important features and information (yes I'm a touch bitter). So I'm not in the position of wholeheartedly recommending Omada yet, I don't have years under my belt there and it's relatively speaking fairly new. It has its own warts and rough edges for sure, from the software to the hardware physical design. But it can be self-hosted and the trajectory looks massively better, has already had more meaningful improvement in months than UniFi has had in years, seems to perform much better so far as well. Of course the Venn diagram of self-hosting, herding lots of hardware with single pane, fully networking features, ecosystem richness and so on is pretty minimal in the overlap. Take away any one or multiple of those and options expand a lot, Aruba InstantOn for example. And welp, this didn't end up "basically" at all did it, sorry about that. I am bummed by the sheer wasted potential with Ubiquiti. So it goes in tech over and over again though, we've all seen this movie many, many times. As far as tips, I would suggest if you plan to stay on the managing-your-own-networks route to very strongly consider having the router/gateway stuff be separate and fully open source as I ended up. Doesn't have to be OPNsense, could be VyOS or plain OpenBSD or whatever else you're most comfortable with and depending on how you want to manage stuff and what needs there are for others to take over. But it's very, very pleasant to have the full spectrum of quality PC hardware available, you can get far more power for less, and you're never stuck with a critical aspect. I'd still suggest generally running that on metal rather than virtualizing it in a (semi)production network, but opinions vary there. ---- 0: https://news.ycombinator.com/item?id=32297556 https://news.ycombinator.com/item?id=32297556
- donmcronald 4y ago
  Thanks for taking the time to reply. I've also soured on UniFi a lot over the last few years. It's crazy. The SMB / MSP market is massively underserved and instead of staying focused and capturing it Unifi is trying to compete in the enterprise market where, IMHO, they aren't going to succeed. If we try out Omada and feel like it gives us everything Unifi does without all the feature creep then we'll do the same thing as you; EoL sites will move from Unifi to Omada. I'll always take fewer features and stability over continuous new features. > I would suggest if you plan to stay on the managing-your-own-networks route to very strongly consider having the router/gateway stuff be separate and fully open source as I ended up. We've used pfSense forever and have been happy with it. However, 2.6.0 has some gateway/dpinger issues where the bug fixes are slotted for 2.7.0 on the CE side, so it feels like the inevitable neglect of the CE version might be starting to kick in there. I tried to explain to them the realities of the market we're in (small businesses) along with real numbers in terms of what kind of money they could extract from businesses like ours and all it got me was an offer to talk to someone in sales and we're not buying into subscriptions (ever). It would cost us $15k+ / year to switch to pfSense+. The software only subscription is nearly 2x the cost of buying their entry level appliance (assuming a 5 year lifecycle). IMHO, that's an indicator they don't want to support 3rd party hardware and there's a risk they'll eventually drop it. I wish VyOS had less crazy pricing because their stuff looks nice. They used to have a $600 / year professional subscription which they don't appear to offer anymore. I think the corporate subscription used to be <$4k / year and now it's $6k. I wonder why no one wants to commit to subscriptions. How can I sell anything to a customer when I can't give a predictable price for ongoing maintenance a year into the future? Our customers will literally go back to using their ISP supplied modems instead of proper firewalls before they'll pay the prices everyone is asking for. I don't blame them either. They only need a fraction of the functionality being sold and all of the fancy, expensive features everyone is using to justify sky high pricing are simply bad value in that context because they'll basically unneeded bloat.
  - xoa 4y ago
    >Thanks for taking the time to reply. No problem, glad to chat about this and commiserate a bit with someone in a similar boat. >I've also soured on UniFi a lot over the last few years. It's crazy. The SMB / MSP market is massively underserved and instead of staying focused and capturing it Unifi is trying to compete in the enterprise market where, IMHO, they aren't going to succeed. Yeah, it's such a huge waste. But the CEO is a classic trouble case with near total control (owns majority of the stock and thus effectively owns the board, a few caveats in terms of protection for minority shareholders in public companies don't really stop the trend). >If we try out Omada and feel like it gives us everything Unifi does without all the feature creep then we'll do the same thing as you; EoL sites will move from Unifi to Omada. I'll always take fewer features and stability over continuous new features. UniFi hasn't even had "continuous new features" though! Or rather, their "features" have been stuff like reskinning the GUI or doing their own entire custom silly speedtest stack for a while, not "features" like PPSK, or for that matter just having normal DHCP and DNS management! Or any sort of certificate management or customization, or, like, anything of substance. Or on hardware, they didn't even bother to upgrade their UniFi Security Gateways literally ever, not once! The ones still selling now are the exact same hardware as 2014 launch, no new updates in forever. They intro new stuff without ever sunsetting older stuff in general really. Use "EA" as a popularity poll test. The dysfunction goes on and on. I don't know if Omada will be the answer yet but worth a spin IMO. I'd definitely hate having to either go back to lots of individual management or something cloud-based. I am stick with UI for PtP/PtMP stuff for the time being though, even if some rot is creeping in there too. >We've used pfSense forever and have been happy with it. However, 2.6.0 has some gateway/dpinger issues where the bug fixes are slotted for 2.7.0 on the CE side, so it feels like the inevitable neglect of the CE version might be starting to kick in there. I'm afraid that company has a history of scummy behavior too :(. I think OPNsense is just plain better but ymmv. FWIW, while it's OSS there is at least one company (Deciso) that does a "Business Edition" at a relatively sane rate (~$150/yr, less for a 3 year) with no hardware reqs (they sell hardware too but I don't suggest getting it) and does business support contracts as well. And Sunny Valley Networks does an interesting little layer 7 inspector that can plugin to it. It conversely might not be good enough for your needs either, but it's not crazy. And the free version is still quite solid. As I said in another comment the docs are pretty decent so might be worth giving them a glance and firing up a system in a VM. At any rate my clients are happy now. And despite rough edges I'm grateful there are a number of essentially decent options available.
kerr23 4y ago
I did the same, originally planned on UniFi but since i couldn’t get any i tried Omada. It’s very nice, the only issue i’ve had is my mesh not connecting, but i solved that by reorienting the target AP and boosting the source APs radio. I think i have some tuning left though because my devices don’t switch APs as fast as i’d like.

zamadatix 4y ago

For home I wouldn't really bother with storm control. In the rare case something is actually misbehaving in that regard it's not much work to go fix right and I'd rather do that than mask it with storm control so it's still happening just not enough to cause an obvious signal. For home wireless another trick, assuming you have a higher end model, is setting it to a different network than the wired. If you're not in particular need for slow service at the very edge of the coverage zone you can also raise the minimum data rate and then BUM traffic and wireless management frames won't be as painful even when they do occur.

vladvasiliu 4y ago
> In the rare case something is actually misbehaving in that regard it's not much work to go fix right and I'd rather do that than mask it with storm control so it's still happening just not enough to cause an obvious signal. While I agree with the general idea, I think that there are only so many hours in a day. If the broken device isn't easily fixable, like most IoT junk out there, it's nice to be able to limit the damage.
- zamadatix 4y ago
  There are better network only solutions for unfixable devices when you have that class of gear, such as moving it to its own VLAN which is a good practice for even properly functioning IoT devices. Storm control has more a "John brought a home Netgear (which ignores but doesn't forward spanning tree packets) and created a loop between two wallports despite spanning tree being enabled on them. Thanks to storm control Jane is still able use her IP phone for a meeting while the network admin investigates a monitoring alert about excessive broadcast drops on John's port which is causing slowdowns in that network" type use case. It's not meant to be the long term solution to anything it's just there for when other long term solutions fail to kick in and you don't want a full outage until you can fix them.