30 ms·

> “The breach was massive, customer data was at risk, access to customers’ devices deployed in corporations and homes around the world was at risk.” > “They we

> “The breach was massive, customer data was at risk, access to customers’ devices deployed in corporations and homes around the world was at risk.” > “They were able to get cryptographic secrets for single sign-on cookies and remote access, full source code control contents, and signing keys exfiltration,” Maybe putting your network control plane in 'the cloud' isn't such a good idea after all... Edit: Just re-read the article, this part stood out: > the attacker(s) had access to privileged credentials that were previously stored in the LastPass account of a Ubiquiti IT employee, and gained root administrator access to all Ubiquiti AWS accounts, including all S3 data buckets, all application logs, all databases, all user database credentials, and secrets required to forge single sign-on (SSO) cookies. > Adam says Ubiquiti’s security team picked up signals in late December 2020 that someone with administrative access had set up several Linux virtual machines that weren’t accounted for. If this is true, and whoever breached them had full access to their AWS account, can we really trust them to clean up all their tokens and fully eradicate all forms of persistence the hackers may have gotten?

ryandrake 6y ago
> Maybe putting your network control plane in 'the cloud' isn't such a good idea after all... Isn't one of the major selling points of cloud-everything "How can you possibly secure your service better than BigRespectableCompany?" I know any time I bring up self-hosting E-mail or a web site or whatever, someone always comes out of the woodwork to remind me that I am not an expert in securing Internet services, and that BigRespectableCompanies have full-time employees dedicated to security. Surely I should be moving to the cloud for this expertise! This is sounding more and more like FUD to me.
- vkou 6y ago
  You may be smart, and have secured your systems properly, but someone with the same resume as you in another company might not be. As your manager, how can I tell the difference between someone who actually did the work right, and someone who said they did the work right (and also legitimately believes that they did)?
  - grayhatter 6y ago
    You never can be... but you should already know that being a manager. But if you're the target of an advanced persistent threat. It doesn't matter how good your guys is, they'll win eventually when the next 0day no one knew about shows up. But then your cloud provider will have been broken into dozens of times already. Hundreds of companies have to do a security audit of all of their networks now* because Ubnt got, got. The only ones who don't are idiots, or not using ubnt et al.
    - vkou 6y ago
      So what, you are suggesting a strategy of staying away from large services and hoping that you won't be targeted? I posit that it doesn't take burning a zero day, or a coordinated effort by the CIA, the FSB, and Randy Waterhouse to break the typical DIY self-hosted security implementation. (And that the manager paying someone to build it has no ability to tell between a great, a good and a bad DIY job.)
      - kortilla 6y ago
        A network controller for local WiFi shouldn’t be reachable from the Internet at all. I’ll take a vulnerability ridden controller on an isolated management VLAN over cloud shit any day.
        msh 6y ago
        But if you have multiple sites you need to reach it remotely. Maybe over vpn but still remotely.
- sofixa 6y ago
  > BigRespectableCompanies Ubiquiti really aren't in the same ballpark as AWS or Microsoft, which are the companies people use that argument for, and you can bet your ass their security is better than in most places.
  - kortilla 6y ago
    This is a fallacy. Just because these companies have great security teams doesn’t mean that things don’t fall through the cracks. Shit slips past the security team in product meetings all the time.
    - themacguffinman 6y ago
      Still better than the security competence of most individuals.
    - sellyme 6y ago
      The claim wasn't that they never have security flaws, the claim was that they almost certainly have fewer security flaws than the alternative self-hosted solution someone named MastodonFan87 comes up with.
    - res0nat0r 6y ago
      This is user error though and not any fault of AWS.
- 0xEFF 6y ago
  Managed services with state of the art IAM policies are more secure than lifting and shifting a Linux box running whatever PAM configuring was setup on it in 2005.
biktor_gj 6y ago
After the Unifi Video fiasco, I bought a UDM Pro to test Unifi Protect. Once I saw it required cloud login I got scared. After I saw an ubiquiti ssh key preinstalled in a device with unfeteted internet access I shut it down to never bring it up again
- lazyweb 6y ago
  Wow, are you serious?
- xtat 6y ago
  All of this makes me skeptical about ubnt but a few corrections 1) You dont need to turn on cloud acccess 2) My UDM pro doesn't have ssh open to the world so not sure how that would be useful externally
  - biktor_gj 6y ago
    There was no option to bypass cloud login when it got to my hands, apparently that has been "fixed" with some update, but if you buy a device and it comes with an outdated firmware, as it tends to be the case with their cameras and APs, your only choice is activate on cloud, setup, update, factory reset, setup on local. About 2... I guess when you got access to all their source and infra is just a matter of pushing an update to enable ssh and they don't even need to even push a key. My problem with the keys is that they come bundled with it and you don't know it. There's no reason for them to install a key in there without your consent. Imagine Microsoft presetting an Administrator account on every Windows Server without telling anyone... It's just a security problem, even more in a firewall
rectang 6y ago
> can we really trust them to clean up all their tokens and fully eradicate all forms of persistence the hackers may have gotten? The state of security in the tech industry is miserable. The only companies we should trust not to leak our data are those that never collected it in the first place.
- anticristi 6y ago
  We are certainly not having this conversation enough. I regularly chat with a risk office and she keeps telling me: Data minimization is your first line of defense.
- kazen44 6y ago
  Heck, most operating systems are leaky by default. Even openBSD, which has a stellar trackrecord in terms of security and "goes against the grain" on many decisions for the sake of secure by default (for instance, disabling hyperthreading altogether to prevent any kind of SPECTRE vulnerability) is under constant scrutiny for not being secure enough. Maybe connecting everything to a network and making it a high value target by collecting everyone's data is just a terrible idea in the long run.
  - cyberpunk 6y ago
    > constant scrutiny for not being secure enough. Do you have a source for this? I follow OpenBSD quite closely and this is news to me..
    - thrwaeasddsaf 6y ago
      I haven't got much sources for you but what I've picked up over the years: a lot of OpenBSD's security is just old fashioned manual code review and audits, and there are not enough eyeballs. Someone like Ilja van Sprundel can go in the source code and find a bunch of issues without too much trouble [1]. I don't see any concentrated efforts to improve the status quo (where's formal methods, where's automated fuzzing, where are initiatives to employ more safe programming languages, static analysis, etc.). And while OpenBSD pride themselves on their mitigations, they aren't exactly state of the art and some of the more recent stuff (like trying to eliminate ROP gadgets) seems just futile. The biggest thing OpenBSD did with mitigations was enabling them by default for the base system and ports. What does anyone remember OpenBSD for in 2010-2020? Pledge, probably. That's a nice thing but more for containing the damage than actually making stuff secure in the first place. https://www.csoonline.com/article/3250653/is-the-bsd-os-dying-some-security-researchers-think-so.html https://www.csoonline.com/article/3250653/is-the-bsd-os-dyin... My concern (and the concern of many others, I think) is that if OpenBSD suddenly got enough attention from the wider security community, including people who actively look for holes that can be exploited, there'd be plenty of important stuff found. Until then, these issues sit quietly waiting for a malicious party to discover them. There's quite some fanfare for OpenBSD, but how many of you are actively auditing the code? I'm subscribed to cvs@ and tech@ and I read them daily and I just don't see much contribution at all from outsiders. And when I do see it, it's mostly stuff like fixing typos or amending man pages. All the commits that change code with security implications tend to come from the core developers, and are reviewed by a handful of people at best. And I have seen some obviously broken stuff slip through.
650REDHAIR 6y ago
What a shockingly large breech. Wow.
- toomuchtodo 6y ago
  The breaches are common, the reporting/discovery of them is not. Security just isn’t a priority for a lot of Orgs, as the consequences are minimal (see: Equifax) due to a lack of regulatory or financial penalty pain when a breach occurs. “Help yourself to a free year of identify theft insurance” and all that jazz.
  - MattGaiser 6y ago
    Discovery of breaches seems to be undesirable in the current environment, if many go undetected. If you discover, you have to report. If you don’t, odds are nobody will notice/will blame someone else.
  - neuronic 6y ago
    This is correct. Worked for a fairly large corp with lots of customer data and while I haven't witnessed breaches of said data it's pretty much a matter of time. Me and my colleagues always pushed for more secure setups and configs but the common rebuttal was "no need there's a keycloak running several layers above and you need to use a VPN and need access to AWS first, go implement features instead." I hope for them that no rogue employee decides to play around a bit or that no one stores their credentials in some cloud LastPass account with a '123456qwerty' master password.
IgorPartola 6y ago
Was shopping for alternatives to my Ubiquiti last night. Seems like there is nothing good out there. Engenius has shit hardware and a cloud controller. Aruba has a cloud controller AND you have to pay for a license. Cisco makes you pay for a license. TP-Link is cloud-based. WTF. Does anyone have a decent WAP where I can use PoE, deploy like 5 of them and have them support roaming between APs, all managed locally? Is that too much to ask?
- croutonwagon 6y ago
  Ruckus R710 or R510 unleashed. I was talking about Ubnt's horrendous security in another thread just last night. https://news.ycombinator.com/item?id=26628198 https://news.ycombinator.com/item?id=26628198 Or if you just want Wave1 Hardware...R700/R500 You can get these as overstock on the cheap on amazon etc. The unleashed version means it can run the controller on the AP.
  - taddevries 6y ago
    The R700/R500 are End-of-Life[1] so be sure you're OK with not getting new firmware. 1. https://support.ruckuswireless.com/product_families/4-eol-ruckus-products https://support.ruckuswireless.com/product_families/4-eol-ru...
    - croutonwagon 6y ago
      Totally agree. And the first gen wave2 stuff (ie:710/510) is probably not too far behind. I do find myself rarely looking for firmware upgrades unless there’s a specific issue I can’t workaround. Even on my ubnt equipment. I find it best to just leave it segmented/network isolated and humming. All these cloud features just increase exposure and grant the vendor leverage to hold you hostage.
- traceroute66 6y ago
  For those people here saying "go Ruckus unleashed" ... caveat emptor my friends ! I have it on very good authority that Ruckus have started rolling out a change in their pricing model to require a Unleashed license per AP to operate, a move which obviously increases costs to the end-user. Some people might say its a deliberate move prevent cannibalisation of their main business model by nudging people away from Unleashed. I couldn't possibly comment.
dathinab 6y ago
Man I really wonder why the lack of proper 2FA is so wide spread? Is it rally cost and complexity? Or just missing awareness? Or the lack of consequences when you get hacked in a way which could easily have been prevented (through then they might have attacked in a different way, tbh.).
- neuronic 6y ago
  It's people not getting it and being plain annoyed by the second factor. YubiKey or Authenticator app on a different device... it's too inconvenient and people often only do it if forced (e.g. banks do this afaik).
  - jodrellblank 6y ago
    Every day I sit at the same desk, at the same computer, logging into the same websites, using 2FA over and over and over and over while sites time out "for my protection". It's a plague. Write a damn desktop app I can run locally, I didn't ask for people from Turkmenistan to be able to login as me, so you could sell me a halfassed web version of something. Joseph Heller predicted 2FA in Catch 22 when he wrote: "Almost overnight the Glorious Loyalty Oath Crusade was in full flower, and Captain Black was enraptured to discover himself spearheading it. He had really hit on something. All the enlisted men and officers on combat duty had to sign a loyalty oath to get their map cases from the intelligence tent, a second loyalty oath to receive their flak suits and parachutes from the parachute tent, a third loyalty oath for Lieutenant Balkington, the motor vehicle officer, to be allowed to ride from the squadron to the airfield in one of the trucks. Every time they turned around there was another loyalty oath to be signed. They signed a loyalty oath to get their pay from the finance officer, to obtain their PX supplies, to have their hair cut by the Italian barbers. To Captain Black, every officer who supported his Glorious Loyalty Oath Crusade was a competitor, and he planned and plotted twentyfour hours a day to keep one step ahead. He would stand second to none in his devotion to country. When other officers had followed his urging and introduced loyalty oaths of their own, he went them one better by making every son of a bitch who came to his intelligence tent sign two loyalty oaths, then three, then four;" Notice how 2FA turns into MFA? Keep adding FA until you're as secure as the security theater demands. "To anyone who questioned the effectiveness of the loyalty oaths, he replied that people who really did owe allegiance to their country would be proud to pledge it as often as he forced them to. The more 2factor logins a person went through in a working day, the more secure he was; to Captain Black it was as simple as that" "Captain Piltchard and Captain Wren were both too timid to raise any outcry against Captain Black, who scrupulously enforced each day the doctrine of 'Continual Reaffirmation' that he had originated, a doctrine designed to trap all those men who had become insecure since the last time they passed a 2factor authentication prompt a few minutes earlier."
Grazester 6y ago
There is Fortinet(which acquired Meru 5 years ago). Meru was pretty OK. I helped manage a setup of 2500 + access points on a campus. I left that job 6 months after Meru was acquired so I cant say how they are now.
- xvf22 6y ago
  Got 3 no brainer CVEs against them. We're an enterprise customer who is now moving away because after Fortinet acquired them support dropped off a cliff. They had some good people but it bacame rather apparent that there was a bit of a toxic culture there.
rossipedia 6y ago
> can we really trust them absolutely not
ryan29 6y ago
It's odd how the big cloud vendors have been able to escape criticism for being completely open by default. Other vendors have been taken to task and have adopted better security practices. For example, SuperMicro IPMI comes with a random password now. It's extremely difficult to lock down an AWS account when there are a bajillion services, IAM policies, roles, etc.. I've been trying for the last few days and it's so difficult that I can understand things like this. I don't think it's acceptable, but I can see how it happens. I think the expectation for AWS, Azure, GCP, etc. needs to change. Accounts should allow nothing by default and part of the tutorial / learning process should be understanding the permissions needed for each service and how to limit access to those services. As a bonus, they should show you how to configure Budget Actions to catch anomalies and runaway services. For example, I'm trying to set up my account so SMTP access to SES gets revoked for SMTP users if the message count exceeds a certain threshold. It's really, really hard because there's not a single document / guide that shows the process from start to finish.
- musingsole 6y ago
  You can use AWS Accounts like microservices. The biggest security walls in AWS are the account barriers. Those have to be specifically configured to cross. Sometimes (1%) its unavoidable, but if you have multiple services running on an account, you force yourself to weave arcane webs of IAM permissions crisscrossing all over to get what you need where. It's a terrible model that people inflict on themselves because it's how everything used to work.
- yebyen 6y ago
  The triangle says Confidentiality, Availability, Integrity. While your concerns are 100% valid, we need to remember too that setting up access in restricted ways and inviting users to understand the protection and remove the correct barriers, or implement the concerns necessary to interact with those for themselves, always runs the risk that some users will find your protections cumbersome and instead find a (totally incorrect) way to baffle them, or otherwise even route around them entirely mooting any efforts to secure a platform. And every time I hear this played out in conversation, the answer is "that's on them!" But it's clearly a balancing act, it's a trade off; tautologically, when you make the service less accessible then... it is, well, ... made less accessible. Besides facilitation of the secure access also sales conversion ratios will depend on that accessibility. The crux of your argument stands, the defaults are too open, and we need to do more to ensure that naive users aren't handed a loaded gun to aim at their own feet.
deleted 6y ago
[deleted]
cbozeman 6y ago
Really glad I decided to go with a TP-Link Omada network over Ubiquiti now. My TL-R605 router, OC300, HD660s, and 8 port 2.5 gigabit switch are going strong, and I put the whole network together for under $1000.
mutant 6y ago
> Maybe putting your network control plane in 'the cloud' isn't such a good idea after all... Uh. AWS? GCLOUD? Those have network control planes, maybe not for physical networks, but a control plane nevertheless.
fukmbas 6y ago
Lmao who allows privileged accounts without expiring passwords? And why are you idiots using LastPass
xtat 6y ago
Full ACK but also you can run a local only setup
Thorrez 6y ago
>If this is true, and whoever breached them had full access to their AWS account, can we really trust them to clean up all their tokens and fully eradicate all forms of persistence the hackers may have gotten? This is the same for any breach. At least if you're using AWS, you know that your management tools aren't lying to you (as long as you assume AWS itself isn't hacked) and you can use those tools to cleanup. If you run your own machines, you can't assume your management tools work correctly. All your machines could have rootkits, all your tools could contain backdoors, and every attempt to cleanup might just be a fake veneer. See Reflections on Trusting Trust. Full disclosure I work for a cloud computing company (but not AWS).
boonware 6y ago
Complete failure of security here. No single credentials should be able to grant that much privileged access.
- TwoBit 6y ago
  At my last company, signing keys were physically stored in a safe with limited access. No network security breach could possibly get them.
- d1str0 6y ago
  This isn't really true. If you have an AWS, you need a global god admin. That's the root user. As an IT guy, I have to store those creds somewhere. So I make the password super long and random, store it in lastpass, add 2fa, and add alerting for all logins. It's never used except in the super rare case we have to do something that requires the megagod level privs of the root account (like changing billing to a master account etc)
krageon 6y ago
> can we really trust them to Yes, if they destroy all of their backups, all of their hardware and every one of their current AWS accounts. Then start entirely from scratch. Any measure falling short of that (and let's be reasonable, it definitely will) means that they're entirely untrustworthy from now on. Of course having your home network controlled from the cloud should already have been entirely untrustworthy, so in practice it won't be an issue for their sales.
TriNetra 6y ago
When you're operating such massive services, at minimum you should protect the admin accounts not just with 2FA, but also with IP firewall. Looks like both were missing from here ...
deleted 6y ago
[deleted]
stiray 6y ago
> Maybe putting your network control plane in 'the cloud' isn't such a good idea after all... Sure it isnt. It is extremely bad idea and actually something like the ubiquiti breach is not even strange to me, once you have worked once in "enterprise(tm)" world this doesnt seem like anything strange. There is just no way to buy a router that communicates with 3rd party servers and to let it access the LAN is a complete no-go (even if I am paying ISP router as a part of the package it is running as bridge just to pass the connection to my router). I consider router as a first line of defense for inbound traffic and last line of defense for outbound and there is just no way to trust some fishy corporation for this. And if the corporation is actually promoting cloud access, like Ubiquiti or Google, they are pretty much banned from my shopping list for all times.
jbpnoy6fifty 6y ago
Why didn’t they have a honeypot alert in their pager duty?

ryandrake 6y ago

> Maybe putting your network control plane in 'the cloud' isn't such a good idea after all... Isn't one of the major selling points of cloud-everything "How can you possibly secure your service better than BigRespectableCompany?" I know any time I bring up self-hosting E-mail or a web site or whatever, someone always comes out of the woodwork to remind me that I am not an expert in securing Internet services, and that BigRespectableCompanies have full-time employees dedicated to security. Surely I should be moving to the cloud for this expertise! This is sounding more and more like FUD to me.

vkou 6y ago
You may be smart, and have secured your systems properly, but someone with the same resume as you in another company might not be. As your manager, how can I tell the difference between someone who actually did the work right, and someone who said they did the work right (and also legitimately believes that they did)?
- grayhatter 6y ago
  You never can be... but you should already know that being a manager. But if you're the target of an advanced persistent threat. It doesn't matter how good your guys is, they'll win eventually when the next 0day no one knew about shows up. But then your cloud provider will have been broken into dozens of times already. Hundreds of companies have to do a security audit of all of their networks now* because Ubnt got, got. The only ones who don't are idiots, or not using ubnt et al.
  - vkou 6y ago
    So what, you are suggesting a strategy of staying away from large services and hoping that you won't be targeted? I posit that it doesn't take burning a zero day, or a coordinated effort by the CIA, the FSB, and Randy Waterhouse to break the typical DIY self-hosted security implementation. (And that the manager paying someone to build it has no ability to tell between a great, a good and a bad DIY job.)
    - kortilla 6y ago
      A network controller for local WiFi shouldn’t be reachable from the Internet at all. I’ll take a vulnerability ridden controller on an isolated management VLAN over cloud shit any day.
      - msh 6y ago
        But if you have multiple sites you need to reach it remotely. Maybe over vpn but still remotely.
sofixa 6y ago
> BigRespectableCompanies Ubiquiti really aren't in the same ballpark as AWS or Microsoft, which are the companies people use that argument for, and you can bet your ass their security is better than in most places.
- kortilla 6y ago
  This is a fallacy. Just because these companies have great security teams doesn’t mean that things don’t fall through the cracks. Shit slips past the security team in product meetings all the time.
  - themacguffinman 6y ago
    Still better than the security competence of most individuals.
  - sellyme 6y ago
    The claim wasn't that they never have security flaws, the claim was that they almost certainly have fewer security flaws than the alternative self-hosted solution someone named MastodonFan87 comes up with.
  - res0nat0r 6y ago
    This is user error though and not any fault of AWS.
0xEFF 6y ago
Managed services with state of the art IAM policies are more secure than lifting and shifting a Linux box running whatever PAM configuring was setup on it in 2005.

biktor_gj 6y ago

After the Unifi Video fiasco, I bought a UDM Pro to test Unifi Protect. Once I saw it required cloud login I got scared. After I saw an ubiquiti ssh key preinstalled in a device with unfeteted internet access I shut it down to never bring it up again

lazyweb 6y ago
Wow, are you serious?
xtat 6y ago
All of this makes me skeptical about ubnt but a few corrections 1) You dont need to turn on cloud acccess 2) My UDM pro doesn't have ssh open to the world so not sure how that would be useful externally
- biktor_gj 6y ago
  There was no option to bypass cloud login when it got to my hands, apparently that has been "fixed" with some update, but if you buy a device and it comes with an outdated firmware, as it tends to be the case with their cameras and APs, your only choice is activate on cloud, setup, update, factory reset, setup on local. About 2... I guess when you got access to all their source and infra is just a matter of pushing an update to enable ssh and they don't even need to even push a key. My problem with the keys is that they come bundled with it and you don't know it. There's no reason for them to install a key in there without your consent. Imagine Microsoft presetting an Administrator account on every Windows Server without telling anyone... It's just a security problem, even more in a firewall

rectang 6y ago

> can we really trust them to clean up all their tokens and fully eradicate all forms of persistence the hackers may have gotten? The state of security in the tech industry is miserable. The only companies we should trust not to leak our data are those that never collected it in the first place.

anticristi 6y ago
We are certainly not having this conversation enough. I regularly chat with a risk office and she keeps telling me: Data minimization is your first line of defense.
kazen44 6y ago
Heck, most operating systems are leaky by default. Even openBSD, which has a stellar trackrecord in terms of security and "goes against the grain" on many decisions for the sake of secure by default (for instance, disabling hyperthreading altogether to prevent any kind of SPECTRE vulnerability) is under constant scrutiny for not being secure enough. Maybe connecting everything to a network and making it a high value target by collecting everyone's data is just a terrible idea in the long run.
- cyberpunk 6y ago
  > constant scrutiny for not being secure enough. Do you have a source for this? I follow OpenBSD quite closely and this is news to me..
  - thrwaeasddsaf 6y ago
    I haven't got much sources for you but what I've picked up over the years: a lot of OpenBSD's security is just old fashioned manual code review and audits, and there are not enough eyeballs. Someone like Ilja van Sprundel can go in the source code and find a bunch of issues without too much trouble [1]. I don't see any concentrated efforts to improve the status quo (where's formal methods, where's automated fuzzing, where are initiatives to employ more safe programming languages, static analysis, etc.). And while OpenBSD pride themselves on their mitigations, they aren't exactly state of the art and some of the more recent stuff (like trying to eliminate ROP gadgets) seems just futile. The biggest thing OpenBSD did with mitigations was enabling them by default for the base system and ports. What does anyone remember OpenBSD for in 2010-2020? Pledge, probably. That's a nice thing but more for containing the damage than actually making stuff secure in the first place. https://www.csoonline.com/article/3250653/is-the-bsd-os-dying-some-security-researchers-think-so.html https://www.csoonline.com/article/3250653/is-the-bsd-os-dyin... My concern (and the concern of many others, I think) is that if OpenBSD suddenly got enough attention from the wider security community, including people who actively look for holes that can be exploited, there'd be plenty of important stuff found. Until then, these issues sit quietly waiting for a malicious party to discover them. There's quite some fanfare for OpenBSD, but how many of you are actively auditing the code? I'm subscribed to cvs@ and tech@ and I read them daily and I just don't see much contribution at all from outsiders. And when I do see it, it's mostly stuff like fixing typos or amending man pages. All the commits that change code with security implications tend to come from the core developers, and are reviewed by a handful of people at best. And I have seen some obviously broken stuff slip through.

650REDHAIR 6y ago

What a shockingly large breech. Wow.

toomuchtodo 6y ago
The breaches are common, the reporting/discovery of them is not. Security just isn’t a priority for a lot of Orgs, as the consequences are minimal (see: Equifax) due to a lack of regulatory or financial penalty pain when a breach occurs. “Help yourself to a free year of identify theft insurance” and all that jazz.
- MattGaiser 6y ago
  Discovery of breaches seems to be undesirable in the current environment, if many go undetected. If you discover, you have to report. If you don’t, odds are nobody will notice/will blame someone else.
- neuronic 6y ago
  This is correct. Worked for a fairly large corp with lots of customer data and while I haven't witnessed breaches of said data it's pretty much a matter of time. Me and my colleagues always pushed for more secure setups and configs but the common rebuttal was "no need there's a keycloak running several layers above and you need to use a VPN and need access to AWS first, go implement features instead." I hope for them that no rogue employee decides to play around a bit or that no one stores their credentials in some cloud LastPass account with a '123456qwerty' master password.

IgorPartola 6y ago

Was shopping for alternatives to my Ubiquiti last night. Seems like there is nothing good out there. Engenius has shit hardware and a cloud controller. Aruba has a cloud controller AND you have to pay for a license. Cisco makes you pay for a license. TP-Link is cloud-based. WTF. Does anyone have a decent WAP where I can use PoE, deploy like 5 of them and have them support roaming between APs, all managed locally? Is that too much to ask?

croutonwagon 6y ago
Ruckus R710 or R510 unleashed. I was talking about Ubnt's horrendous security in another thread just last night. https://news.ycombinator.com/item?id=26628198 https://news.ycombinator.com/item?id=26628198 Or if you just want Wave1 Hardware...R700/R500 You can get these as overstock on the cheap on amazon etc. The unleashed version means it can run the controller on the AP.
- taddevries 6y ago
  The R700/R500 are End-of-Life[1] so be sure you're OK with not getting new firmware. 1. https://support.ruckuswireless.com/product_families/4-eol-ruckus-products https://support.ruckuswireless.com/product_families/4-eol-ru...
  - croutonwagon 6y ago
    Totally agree. And the first gen wave2 stuff (ie:710/510) is probably not too far behind. I do find myself rarely looking for firmware upgrades unless there’s a specific issue I can’t workaround. Even on my ubnt equipment. I find it best to just leave it segmented/network isolated and humming. All these cloud features just increase exposure and grant the vendor leverage to hold you hostage.
traceroute66 6y ago
For those people here saying "go Ruckus unleashed" ... caveat emptor my friends ! I have it on very good authority that Ruckus have started rolling out a change in their pricing model to require a Unleashed license per AP to operate, a move which obviously increases costs to the end-user. Some people might say its a deliberate move prevent cannibalisation of their main business model by nudging people away from Unleashed. I couldn't possibly comment.

dathinab 6y ago

Man I really wonder why the lack of proper 2FA is so wide spread? Is it rally cost and complexity? Or just missing awareness? Or the lack of consequences when you get hacked in a way which could easily have been prevented (through then they might have attacked in a different way, tbh.).

neuronic 6y ago
It's people not getting it and being plain annoyed by the second factor. YubiKey or Authenticator app on a different device... it's too inconvenient and people often only do it if forced (e.g. banks do this afaik).
- jodrellblank 6y ago
  Every day I sit at the same desk, at the same computer, logging into the same websites, using 2FA over and over and over and over while sites time out "for my protection". It's a plague. Write a damn desktop app I can run locally, I didn't ask for people from Turkmenistan to be able to login as me, so you could sell me a halfassed web version of something. Joseph Heller predicted 2FA in Catch 22 when he wrote: "Almost overnight the Glorious Loyalty Oath Crusade was in full flower, and Captain Black was enraptured to discover himself spearheading it. He had really hit on something. All the enlisted men and officers on combat duty had to sign a loyalty oath to get their map cases from the intelligence tent, a second loyalty oath to receive their flak suits and parachutes from the parachute tent, a third loyalty oath for Lieutenant Balkington, the motor vehicle officer, to be allowed to ride from the squadron to the airfield in one of the trucks. Every time they turned around there was another loyalty oath to be signed. They signed a loyalty oath to get their pay from the finance officer, to obtain their PX supplies, to have their hair cut by the Italian barbers. To Captain Black, every officer who supported his Glorious Loyalty Oath Crusade was a competitor, and he planned and plotted twentyfour hours a day to keep one step ahead. He would stand second to none in his devotion to country. When other officers had followed his urging and introduced loyalty oaths of their own, he went them one better by making every son of a bitch who came to his intelligence tent sign two loyalty oaths, then three, then four;" Notice how 2FA turns into MFA? Keep adding FA until you're as secure as the security theater demands. "To anyone who questioned the effectiveness of the loyalty oaths, he replied that people who really did owe allegiance to their country would be proud to pledge it as often as he forced them to. The more 2factor logins a person went through in a working day, the more secure he was; to Captain Black it was as simple as that" "Captain Piltchard and Captain Wren were both too timid to raise any outcry against Captain Black, who scrupulously enforced each day the doctrine of 'Continual Reaffirmation' that he had originated, a doctrine designed to trap all those men who had become insecure since the last time they passed a 2factor authentication prompt a few minutes earlier."

Grazester 6y ago

There is Fortinet(which acquired Meru 5 years ago). Meru was pretty OK. I helped manage a setup of 2500 + access points on a campus. I left that job 6 months after Meru was acquired so I cant say how they are now.

xvf22 6y ago
Got 3 no brainer CVEs against them. We're an enterprise customer who is now moving away because after Fortinet acquired them support dropped off a cliff. They had some good people but it bacame rather apparent that there was a bit of a toxic culture there.

rossipedia 6y ago

> can we really trust them absolutely not

ryan29 6y ago

It's odd how the big cloud vendors have been able to escape criticism for being completely open by default. Other vendors have been taken to task and have adopted better security practices. For example, SuperMicro IPMI comes with a random password now. It's extremely difficult to lock down an AWS account when there are a bajillion services, IAM policies, roles, etc.. I've been trying for the last few days and it's so difficult that I can understand things like this. I don't think it's acceptable, but I can see how it happens. I think the expectation for AWS, Azure, GCP, etc. needs to change. Accounts should allow nothing by default and part of the tutorial / learning process should be understanding the permissions needed for each service and how to limit access to those services. As a bonus, they should show you how to configure Budget Actions to catch anomalies and runaway services. For example, I'm trying to set up my account so SMTP access to SES gets revoked for SMTP users if the message count exceeds a certain threshold. It's really, really hard because there's not a single document / guide that shows the process from start to finish.

musingsole 6y ago
You can use AWS Accounts like microservices. The biggest security walls in AWS are the account barriers. Those have to be specifically configured to cross. Sometimes (1%) its unavoidable, but if you have multiple services running on an account, you force yourself to weave arcane webs of IAM permissions crisscrossing all over to get what you need where. It's a terrible model that people inflict on themselves because it's how everything used to work.
yebyen 6y ago
The triangle says Confidentiality, Availability, Integrity. While your concerns are 100% valid, we need to remember too that setting up access in restricted ways and inviting users to understand the protection and remove the correct barriers, or implement the concerns necessary to interact with those for themselves, always runs the risk that some users will find your protections cumbersome and instead find a (totally incorrect) way to baffle them, or otherwise even route around them entirely mooting any efforts to secure a platform. And every time I hear this played out in conversation, the answer is "that's on them!" But it's clearly a balancing act, it's a trade off; tautologically, when you make the service less accessible then... it is, well, ... made less accessible. Besides facilitation of the secure access also sales conversion ratios will depend on that accessibility. The crux of your argument stands, the defaults are too open, and we need to do more to ensure that naive users aren't handed a loaded gun to aim at their own feet.

deleted 6y ago

[deleted]

cbozeman 6y ago

Really glad I decided to go with a TP-Link Omada network over Ubiquiti now. My TL-R605 router, OC300, HD660s, and 8 port 2.5 gigabit switch are going strong, and I put the whole network together for under $1000.

mutant 6y ago

> Maybe putting your network control plane in 'the cloud' isn't such a good idea after all... Uh. AWS? GCLOUD? Those have network control planes, maybe not for physical networks, but a control plane nevertheless.

fukmbas 6y ago

Lmao who allows privileged accounts without expiring passwords? And why are you idiots using LastPass

xtat 6y ago

Full ACK but also you can run a local only setup

Thorrez 6y ago

>If this is true, and whoever breached them had full access to their AWS account, can we really trust them to clean up all their tokens and fully eradicate all forms of persistence the hackers may have gotten? This is the same for any breach. At least if you're using AWS, you know that your management tools aren't lying to you (as long as you assume AWS itself isn't hacked) and you can use those tools to cleanup. If you run your own machines, you can't assume your management tools work correctly. All your machines could have rootkits, all your tools could contain backdoors, and every attempt to cleanup might just be a fake veneer. See Reflections on Trusting Trust. Full disclosure I work for a cloud computing company (but not AWS).

boonware 6y ago

Complete failure of security here. No single credentials should be able to grant that much privileged access.

TwoBit 6y ago
At my last company, signing keys were physically stored in a safe with limited access. No network security breach could possibly get them.
d1str0 6y ago
This isn't really true. If you have an AWS, you need a global god admin. That's the root user. As an IT guy, I have to store those creds somewhere. So I make the password super long and random, store it in lastpass, add 2fa, and add alerting for all logins. It's never used except in the super rare case we have to do something that requires the megagod level privs of the root account (like changing billing to a master account etc)

krageon 6y ago

> can we really trust them to Yes, if they destroy all of their backups, all of their hardware and every one of their current AWS accounts. Then start entirely from scratch. Any measure falling short of that (and let's be reasonable, it definitely will) means that they're entirely untrustworthy from now on. Of course having your home network controlled from the cloud should already have been entirely untrustworthy, so in practice it won't be an issue for their sales.

TriNetra 6y ago

When you're operating such massive services, at minimum you should protect the admin accounts not just with 2FA, but also with IP firewall. Looks like both were missing from here ...

deleted 6y ago

[deleted]

stiray 6y ago

> Maybe putting your network control plane in 'the cloud' isn't such a good idea after all... Sure it isnt. It is extremely bad idea and actually something like the ubiquiti breach is not even strange to me, once you have worked once in "enterprise(tm)" world this doesnt seem like anything strange. There is just no way to buy a router that communicates with 3rd party servers and to let it access the LAN is a complete no-go (even if I am paying ISP router as a part of the package it is running as bridge just to pass the connection to my router). I consider router as a first line of defense for inbound traffic and last line of defense for outbound and there is just no way to trust some fishy corporation for this. And if the corporation is actually promoting cloud access, like Ubiquiti or Google, they are pretty much banned from my shopping list for all times.

jbpnoy6fifty 6y ago

Why didn’t they have a honeypot alert in their pager duty?