9 ms·
How we discovered a database leak in one of the biggest Swiss hosting providers
- TheRealPomax 9y agoFor the longest time a "clean" MySQL install would set up an no-password superuser for presumably dev convenience. I don't know if they changed that (it's been a while since I last installed MySQL) but if not, this could simply be a security hole by design, with the maintainers simply not paying attention to their install script flags.
- strictnein 9y agoI thought it was no password, but only available via localhost?
- TheRealPomax 9y agoAll that requires to be exploited "on" localhost is some PHP script interpreting unsanitized user uploads (uploading a php script that has an image file extension's a pretty famous example) on any of a thousand customer sites. You don't want that MySQL user to exist, ever =/
- deleted 9y ago[deleted]
- parliament32 9y agoI'm getting a "Something went wrong. Please retry in a moment." error when trying to submit a domain to Security Guardian (tried different domains and email addresses). I assume you're being hugged to death.
- raducu 9y agoThe moral of the story should have been -- change your hosting provider the minute they commit such a blunder.
- bringtheaction 9y agoIndeed. Like they say, fool me once, shame on you; fool me twice, shame on me. However how do you go about picking a new provider? How do you know that anyone else is any better?
- twhiston 9y agonine.ch is better (disclaimer - I work for them ;) )
- aepiepaey 9y agoThe first time might have been forgivable, but with the second incident it was definitely time to change provider. The conclusion for what happened is obvious: the host "patched" what the attacker had done, but not the vulnerability that gave them access. Thus, the attackers re-used the same attack later to re-gain access.
- willvarfar 9y agoThis is a nice anecdote to promote their new Guardian service.
- rurban 9y agoThey probably use a homebuilt admin panel and sw mgmt, and an update brought back in the old root vuln. They don't use cPanel or Plesk. Or the Chinese hacked it again. Interestingly they - https://security.infoteam.ch/ https://security.infoteam.ch/ - offer the very same security service, automatic security audits for their customers. Which explains their angry response the 2nd time.
- bipson 9y agoMaybe I got you wrong, but infoteam.ch are the guys who wrote the blog-post, so your link is to their product they are describing, and not one by the provider? How does that again explain the angry response? AFAIK there is no mention or link to the provider, I guess since they don't want to make enemies just yet (and lose hosting while doing so). But I am willing to speculate that they are looking for another provider just now :)
- rurban 9y agoI got it wrong, not you. Thanks for the heads-up. They didn't mention who leaked the data. I got confused. The symptom they described looks like an automatically updated system with a leak. One of our swiss customers, a big hosting provider, uses such a system and would fit.
- Mashimo 9y agoCan't test the product they try to promote because emails with a `+` in them are not valid.
- ScottBurson 9y ago> emails with a `+` in them are not valid Wrong: https://en.wikipedia.org/wiki/Email_address#Syntax https://en.wikipedia.org/wiki/Email_address#Syntax (I know this is of little help if your email app doesn't allow them. I just want to point out that the standard does allow them.)
- coldacid 9y agoI think what Mashimo means is that Infoteam.ch thinks that `+` is invalid when it isn't. Sadly they're far from the only ones whose email validation code won't accept the plus character, or many other legal characters in the local part of an email address.
- codetrotter 9y agoEmail validation regexes are so annoying. Everyone ought to just use .+@.+ as their validation regex and not be more strict than that. Beyond that just queue and try to deliver the email. Tell the user than an email should arrive shortly and that if it doesn’t they should check their spam folder and that they should check that they gave the correct email address. When you say this you repeat the email address that the user gave you (escaped for XSS of course). I think some people “validate” against a strict pattern to keep their users from mistyping, but really there are so many ways to make a typo and still match those regexes that IMO it’s pointless to use a complicated regex and 80% of the times those regexes end up rejecting actually valid (though unusual) email addresses. I think for a lot of developers the reason they do this is that they’ve learned that they should validate data and so they decide to validate email and to do so they either copy-paste some random-ass regex off the internet or they write their own broken regexes. All your regex should do is to ensure that there is an @ in the address and that there is something before and something after. This keeps people from mistakenly entering say for example their phone number because they didn’t read what the field was for. To prevent people from making your machine send your emails where it should not, such as to root@localhost of your server or elsewhere on your local network (don’t know why anyone would and also it wouldn’t be a big issue, just a tiny bit annoying), is a server configuration concern. Specifically, a concern of configuration of the email server software and of your firewalls. User presses sign up -> Send then to registration form, they fill in their details which you validate lightly client side, they submit -> You validate lightly server-side and either send them back to the form or on to the next step -> You tell them “Thank you, your registration is now complete. An email should arrive in your inbox shortly. If it does not, please check your spam folder and also control that you entered your email address correctly. The email address you gave us was somebody@example.com.”
- sneak 9y agoImportant to remember that these “one of the biggest $x in $y” where $y is a country with a population under 10 million means that the statement encompasses many entities which are just a half-dozen people in a small office somewhere. I know nothing about the particular hosting provider in question.
- bipson 9y agoFor whoever was wondering who this provider is: according to whois-nslookup-mxtoolbox_arin_lookup, the server hosting infoteam.ch is provided by metanet (metanet.ch) Not trying to ruin their business, but they should consider handling issues like this one properly.
- madshiva 9y agothey don't care to run a business only in german at first, we are in Swiss they should provide three languages. If they fail this, they fail everything. The only serious provider in Swiss is Infomaniak.
- logronoide 9y agoMaybe I’m wrong but the PTR of the IP seems to be in a spam blacklist: https://apility.io/search?q=infoteam.ch https://apility.io/search?q=infoteam.ch https://apility.io/search?q=80.74.143.113 https://apility.io/search?q=80.74.143.113 https://apility.io/search?q=dynco.ch https://apility.io/search?q=dynco.ch This thing happens sometimes, specially if you use a shared hosting. Recently using a well known cloud provider the Public IP address assigned was in a spam blacklist! It’s a good idea to have a look at your IP and domains frequently. (Disclaimer: I’m the founder of the tool used for the lookup in the blacklists)
- deleted 9y ago[deleted]
- redsec 9y agoSorry, but to avoid problem for now, we prefer to keep the provider anonymous, however we can assure you that it is not metanet.ch.
- patte 9y agocurrently infoteam.ch seems to be hosted on METANET (metanet.ch). Is there anyone who can deny or confirm that this is the provider they don't want to mention? source: nslookup infoteam.ch; whois 80.74.143.113
- ar0 9y agoJust in case anyone is reading here first and because this could really hurt the business: The infoteam.ch people have stated above[1] that the provider was not Metanet. (I do think, however, that it would be important to know.) [1]: https://news.ycombinator.com/item?id=16529273 https://news.ycombinator.com/item?id=16529273
- lonesome-rider 9y agoAngry DB Admin who left the company and a way to revenge later.
- orf 9y agotl;dr, portscanned a server, found an open MySQL port with a weak password.
- LeonM 9y agoNot just weak, but no password at all, for the root user. And the problem returned a week after the 'fixed' it.
- nordras 9y agoTheir vulnerability scanner is basically an on-demand DOS attack. Tried it on my site and almost brought it down
- TheRealPomax 9y agoGood. That also tells you that your server probably needs some DOS mitigation. Because if their service almost takes it down, so can a trivial nmap sweep.
- nordras 9y agoYes, my site does - but that's not the point - it's a text field on a website anyone can use to hit any site without effort. It's also supposed to be a helpful service, which generally implies it's not gonna behave in a way some would consider to be malicious, but it has no rate limits. If I saw that hit my logs i'd consider it to be an attack, not a friendly vulnerability scan.
- ryanlol 9y agoPerhaps your site is just really slow?
- nordras 9y agoIt's not super fast, but it's not really slow. A single web server doing 50 req/s, and they pushed that up to 100 req/s. The limit is somewhere around 75.
- vectorEQ 9y ago:s i can't even get my mysql to get me to be allowed to login root without password >.< that takes a special kind of negligence.... and really, how long was it there before they developed a new product and tested it on themselves? :/ seems logical, especially for a security service provider that with the lack of such product still this would be noticed? that besides pitching their own product for an issue any similar natured scan would pick up i'd say it smells like marketing department at work more than chinese hackers or shitty service provider.... >.> i doubt they would have left a passwordless root on their mysql, or didnt they check the initial setup they were given by the provider before taking it in use?
- barkingcat 9y agoIf you read the story you would see that the database host is a shared host (as in hundreds of other clients of the Webhost have accounts on it) and that the error is likely the result of a persistent hack. As in there is a vulnerability where someone can get access to the server and create a passwordless root account, so that they can siphon the data out. Once that account is deleted, a new passwordless root account is created by the attacker in order to continue access.
- kennydude 9y agoSounds like they may have had a deploy script which ran again a week later or something like that :/ (Also overriding scrolling is not cool)
- sdoering 9y agoI second the scrolling part. Between the massive fixed header and the overridden scrolling the page is unusable to me. I am able to get rid of the first with a little bookmarklet - the second one I haven't figured out yet.
- unixhero 9y agoMoral of the story. For my sake: Check the access logs. Regularly.
- peterwwillis 9y agoWhy, are you bored? They can just access the db from an internal host that already has connections and exfiltrate from there. That's part of why people retrieve data using holes in web apps - it looks like a spike in normal traffic.
- unixhero 9y agoI'm sharing my takeaway after reading this. Access logs are fun to some people ;)
- smoyer 9y agoSecurity Guardian is not a he/him. It may be a translation issue ... or maybe you've achieved human-level AI and it's become self-aware? In any case, I find it interesting that your first response is that the tool might have a bug ... and the link also on HN at this moment is about the Apollo 13 mission control engineers thinking their telemetry might be at fault. This is an excellent first response and it's important to provide a way to distinguish between the two.
- coldacid 9y agoShips are not she/her and yet we use that kind of language with them all the time. Silly thing to quibble about.
- dredmorbius 9y agoGiven the number of failures I've seen as a direct result of security and/or availability systems, the instinct to first question your own readings is a valid one. It's a good check particularly to make sure that your tests are valid, and that you aren't going off half-cocked with an alert (alarm fatigue is a Real Thing, and attention is a scarce resource). I've also seen, in general, far more false alerts than real ones. (Yes, Nagios, I am in fact looking at you.) The gender-casing is a quirk of the article, and would reflect at least three of the languages official within Switzerland (German, French, and Italian are all gendered languages, I'm fairly certain Romanche is as well). It's fair to note that, though perhaps not quite to the extent you did.
- redsec 9y agoA little update on the service Security Guardian after the publication of this post. Thanks to Hacker News and its incredible community, there have been a massive number of new users. We are working on adding more resources to the infrastructure to make the scans quicker. For now, it is possible that some of you have to wait some hours before receiving the first results. Thanks for trying our new product, we hope to improve it with your feedbacks.
- deleted 9y ago[deleted]
- doorbellguy 9y ago> Then, they blocked the IP address of our scanner so we could not scan their server anymore. Long time lurker, made an account just to ask this: What are your comments on this unprofessional reaction from the Swiss?
- iforgotpassword 9y agoThey should release the name of the hoster. Not so much for internet points but to warn the customers. Who knows what information could have leaked from all those DBs, and I doubt that the hosted sent out a report on that incident.
- doorbellguy 9y ago100% agreed.
- peterwwillis 9y ago"If you use this network security appliance, you can stop all traffic that doesn't match the profile of your normal traffic from leaving your network." "That's expensive, and complicated! We'll just do regular audits and be fine." [some time later] "Someone exfiltrated all our data using mysqldump!" = /
- stareatgoats 9y ago> "Hopefully, we had ‘only’ read access and could not write or delete anything" Sounds a lot like feigned ignorance about the nature of the root user. Not entirely sure if it would help them in a court of law. They should probably anonymized the whole thing better to be completely on the safe side (not a lawyer though).
- dredmorbius 9y agoThat reads more like an attempt to make a light-hearted quip suggesting the exposure was limited. Say, more like: "OK, so we could directly access the database, but surely nobody would provide a full read/write administrative account publicly accessible and passwordless.... Oh." Given a few other hints at English not being the first tongue (or possibly even the fourth, CH having four official co-equal languages) of the writer(s) in question, that could just be awkwardness from a non-native speaker. Their English is vastly superior to my German, French, Italian, or Romanche.