13 ms·
Works great. You can already find questions on Stack Overflow from people getting their database deleted https://stackoverflow.com/questions/63067062/elastic-s
by user5994461 6y ago
Works great. You can already find questions on Stack Overflow from people getting their database deleted
https://stackoverflow.com/questions/63067062/elastic-search-indexes-gets-deleted-frequently https://stackoverflow.com/questions/63067062/elastic-search-...
Edit: The person raising that question is working for Atlassian (Jira), looks like Atlassian got their database deleted lol
- philshem 6y agoThe top-voted answer links to this HN page. I'm stuck in an infinite loop.
- aledalgrande 6y agoYou should configure a timeout.
- inetknght 6y agoGood tree^H^H^H^Hgraph traversal algorithms have a history stack specifically to detect and deal with loops.
- red0point 6y ago1) If it‘s a tree, it ain‘t got no loops 2) The stack isn‘t to deal with loops, the „visited“ flag at each edge is there for that. The stack (for DFS, BFS would be a queue) is there to keep track of which nodes have been visited such that you can construct a path from the starting node to the one you‘re looking for. Obviously there are variants to this, depending on what you‘re actually trying to achieve with it. My point is that a stack would be a very inefficient way to deal with loops.
- inetknght 6y ago1) you're right, I edited my message to reflect that I meant a graph traversal algorithm. 2) a visited flag on an edge? That won't support simultaneous traversals. Keeping a stack is a lot more efficient than permitting only one traversal at a time.
- red0point 6y agoI‘m not sure why you‘re bringing concurrency to the table. My point still is that looking something up in a stack (did I visit this node?) costs O(n) time, so the BFS will degrade from O(m+n) to O(m*n+n). To come back to the concurrency, if you can index your edges in some way, you can also store the visited flag in a separate datastracture to support concurrent access (one „flag store“ for each access).
- inetknght 6y ago> I‘m not sure why you‘re bringing concurrency to the table. Not using data structures that enable concurrency prevents performance improvements since modern hardware is, in general, more parallel than vertical.
- hinkley 6y agoModifying the graph turns it into shared (mutable) state. Your code is still re-entrant, but it's no longer concurrent.
- throwaway8941 6y ago>tree^H^H^H^Hgraph If we pretend we're using readline here, ^W (yank previous word) and ^U (yank to the start of the line) should save you some key presses. Some recommended bedtime reading: https://catonmat.net/ftp/readline-emacs-editing-mode-cheat-sheet.pdf https://catonmat.net/ftp/readline-emacs-editing-mode-cheat-s... https://en.wikipedia.org/wiki/GNU_Readline#Emacs_keyboard_shortcuts https://en.wikipedia.org/wiki/GNU_Readline#Emacs_keyboard_sh...
- hinkley 6y agoThese are not emacs commands. They aren't even unix shell commands. They are TTY commands, some of them dating back to the dot matrix teletype terminals. My favorite is ^U, which 90% of the time lets you start over on a password prompt when you are sure you just fat fingered but not sure how badly.
- djeiasbsbo 6y agoThank you, that one is really useful! The one I had always remembered was ^H for whenever no other way to delete characters works. I need it in some SQL REPLs which aren't configurable.
- dialamac 6y agoI think you might be conflating these (or maybe I should say the gp is). Nevertheless. Do you have a reference to the history of key combos like ctrl f, b, n, p and a and e? Those are typically referred to as emacs style navigation and I am genuinely unaware of history of those as common tty control codes outside of emacs for cursor movement. They weren’t dec vt control codes. Ctrl-U was though and even has ASCII assignment as “NAK”. Ctrl-H and C are similar.. but people don’t typically refer to those as “emacs” keys.
- inetknght 6y ago> Do you have a reference to the history of key combos like ctrl f, b, n, p and a and e? Those are typically referred to as emacs style navigation and I am genuinely unaware of history of those as common tty control codes outside of emacs for cursor movement. I've always heard of it as an ASCII control character and gets its history from Unix interpretations of really old IBM keyboards which got its history from typewriters. I've literally never used emacs for anything other than ^X to exit; I'd rather use cat than emacs. I use vim. But ^H has worked for me as intended on a serial console and on telnet. Some diving through wikipedia: [0] says: Pressing the backspace key on a computer terminal would generate the ASCII code 08, BS or Backspace, a control code which would delete the preceding character. That control code could also be accessed by pressing Control-H, as H is the eighth letter of the Latin alphabet. [1] says: In some typewriters, a typist would, for example, type a lowercase letter A with acute accent (á) by typing a lowercase letter A, backspace, and then the acute accent key. This technique (also known as overstrike) is the basis for such spacing modifiers in computer character sets such as the ASCII caret (^, for the circumflex accent). [2] says: Unix (command line and programs using readline): Ctrl+H = Delete previous character [3] supports [0] and says: Caret notation is a notation for control characters in ASCII. The notation assigns ^A to control-code 1, sequentially through the alphabet to ^Z assigned to control-code 26 (0x1A). Often a control character can be typed on a keyboard by holding down the Ctrl and typing the character shown after the caret. However, it's worth noting that it also says The meaning or interpretation of, or response to the individual control-codes is not prescribed by the caret notation. But, despite that, ASCII describes control character 8 as backspace [4] [5]. According to this Wikipedia article, work on ASCII began in 1960 and its first release in 1963 [6]. Emacs' first release was in 1985 [7]. I suggest that perhaps in your bubble control sequences are referred to as emacs style navigation even if it's not necessarily the most historically accurate. I'm glad you're working with *nix enough to be familiar with emacs and there's always new old things to learn. There's a lot of history to learn and understand. [0] https://en.wikipedia.org/wiki/Backspace#%5EH https://en.wikipedia.org/wiki/Backspace#%5EH [1] https://en.wikipedia.org/wiki/%5EH https://en.wikipedia.org/wiki/%5EH [2] https://en.wikipedia.org/wiki/Control_key#Table_of_examples https://en.wikipedia.org/wiki/Control_key#Table_of_examples [3] https://en.wikipedia.org/wiki/Caret_notation https://en.wikipedia.org/wiki/Caret_notation [4] https://en.wikipedia.org/wiki/Control_character#In_ASCII https://en.wikipedia.org/wiki/Control_character#In_ASCII [5] https://en.wikipedia.org/wiki/ASCII_control_characters#Delete_&_Backspace https://en.wikipedia.org/wiki/ASCII_control_characters#Delet... [6] https://en.wikipedia.org/wiki/ASCII https://en.wikipedia.org/wiki/ASCII [7] https://en.wikipedia.org/wiki/Emacs https://en.wikipedia.org/wiki/Emacs
- hinkley 6y agoYou can cheese infinite loop detection by setting a max number of edge traversals (that way you don't just loop longer when someone speeds up the happy path). For real code, you wouldn't generate a web page with 5 million entries in it, so you can be pretty sure that the data is bad even if it's not cyclical (but it probably is)
- m12k 6y agoNah, you're just in an unbounded recursion - don't worry, I can already now tell you that it ends with stack overflow.
- IncRnd 6y agoHa! Clever!
- oconnor663 6y agoI dunno, does Chrome implement tail call optimization? :)
- smabie 6y agoNah stack overflow learned their lesson and switched over to free monads.
- pietroglyph 6y agoSounds like a good public service. I’d much rather have my data deleted until it’s secured than have it stolen by someone else.
- cryptoz 6y agoVandalism is not a good public service. > I’d much rather have my data deleted until it’s secured than have it stolen by someone else There are multiple logical fallacies in this sentence. First is the use of the world 'until' which is ambiguous here; it suggests that your data can be 'undeleted' after the DB has been secured or you would rather not have any data stored anywhere that is not secured. Either option to me seems like an incorrect read of your comment but I'm not sure. And "than have it stolen by someone else" seems to imply that you know that this data was never copied and cannot be stolen still. I think that seems incorrect, unless there is something I missed that assures everyone that the data could not have been stolen during these hacks. Lastly, your personally preferred outcome for your personal data is not a measure for all of society, but you grant it that "public service" label as if your preference matters above everyone else's. You don't know what other people think about their data. You don't know what the data even is. What if some of it was just a hobby project for someone, with no financial implications of unsecured data or of data loss, but with emotional attachment to their data? Do they not matter to you? A blind deletion of unknown data belonging to unknown people is not a public service.
- p1necone 6y agoI assume the comment was partially in jest. But this would actually work well if it was consistent and fast. If databases get wiped before you have time to put anything important in them then noone gets hurt.
- deleted 6y ago[deleted]
- dmurray 6y agoYeah, it's bad for the industry right now, but this is just a transition period! Once we get through the pain of losing a few databases, the new steady state where nobody's data is stored in world-writable databases will be better for everyone, and that will be worth the cost. Consider if this happened five years ago, it would have had a smaller cost than happening today. And it was probably going to happen at some point, so better that it happened five years ago than today. By the same argument, better that it happened now than at any point in the future. I'm not sure how serious I am about this argument but...at least a little bit? I guess the alternative argument is that any day now software vendors would have all moved to secure-by-default platforms where deploying a world-writable Redis in production would have been so difficult that it rarely happened.
- pramodhs 6y agoI'm working on a personal project and not at all related to my work. I accidentally kept ports open :facepalm, sorting things out now :)
- user5994461 6y agoRecommend to setup two subnets in your project. One public and one private. This prevents this sort of issues, instances in the private subnet simply don't get a public IP, they can't be reached over the internet. For reference, the standard practice in a company is to have a (third) separate subnet for databases, with zero internet access (no NAT gateway). Connection must be explicitly opened from/to database clients. It's a nightmare to manage on premise but it works really well in the cloud with firewalls allowing traffic based on instance tags.
- moooo99 6y ago> Recommend to setup two subnets in your project. One public and one private. This is very good advice. We recently had a uni project where we had to use a MongoDB database. Somebody just apt-get installed a mongodb onto a DO droplet called it a day. Two days later the only remaining records prompted us to transfer x amount of BTC to a adress that was store in our DB. It just contained dummy data, but it is worrying that something like this apparently happens to lots of companies as well. The only thing I find weird is that ElasticSearch itself does not offer a way to handle authentication, it was just enabled by a plugin that was paid (it seems like its free now).
- rovr138 6y agoRegarding elasticsearch, that’s actually fine. Just block access to it on your firewall to the public ports and require people SSH or VPN for access if needed. It’s not
- KMag 6y ago> The only thing I find weird is that ElasticSearch itself does not offer a way to handle authentication, it was just enabled by a plugin that was paid (it seems like its free now). "Wierd" is an interesting euphemism for "irresponsible." Defaults are very important. Insecure by default is insecure for 90+% of deployments.
- rodgerd 6y agoI wonder how much customer info they were leaking before then.
- Izkata 6y agoThe question was posted early Friday morning. If this drove their services, I suspect a lot of us would've known about it a lot sooner than now...
- mdoms 6y agoAtlassian is not on Google Cloud, they are an AWS shop. I suspect this is an unrelated personal project.
- bigiain 6y agoHow much would you bet against that guy having fairly highly privileged AWS IAM access in Atlassian's account?
- nomel 6y agoThis edit is speculation. > I'm running an elastic search for a personal project on google-cloud and I use as a search index for my application. He very clearly says it’s a personal project. Trying to learn new topics outside of your direct responsibilities, while employed, is very common in the software industry. Not everyone that works at a company is involved in databases at that company.
- creato 6y agoI wonder how many of the deleted databases are just people learning with databases of dummy data?
- nix23 6y agoNice, now they learn never expose a DB directly to the net additionally...bonus points ;)
- AnIdiotOnTheNet 6y agoExcept of course they probably already knew that, they just accepted to risk to their toy database as a trade-off for the convenience of being able to directly access it over the internet.
- nix23 6y agoHave fun to re-setup it again then...
- stedaniels 6y agoThat was edited in afterwards. https://stackoverflow.com/posts/63067062/revisions https://stackoverflow.com/posts/63067062/revisions whilst it very well may be a personal project, it certainly wasn't "very clear".
- JosephRedfern 6y agoMaybe he edited it to clarify after strangers on the internet got him in trouble at work having implied that his employer had suffered a data breach?
- KMag 6y agoIf meant as a public service, it would have been much less destructive to use the change passwords API [0] to set random passwords for all of the users. [0] https://www.elastic.co/guide/en/elasticsearch/reference/current/security-api-change-password.html https://www.elastic.co/guide/en/elasticsearch/reference/curr...
- dredmorbius 6y agoGiven that "unsecured" means "data are accessible and modifiable by anyone", creating tremendous externalities for all referenced in the data, , I'm happy with deletion. FTA: One of the first publicly known examples of a Meow attack is an Elasticsearch database belonging to a VPN provider that claimed not to keep any logs.
- KMag 6y agoIn some cases, I might be tempted to agree with you, but this is blindly being applied by an automated attack. What if some of that deleted data is volunteer-canvassed anonymized survey data of homeless people, and its loss sets back a homeless relief program by months, resulting in several people freezing to death this winter?
- dredmorbius 6y agoThe data may be modified at any time without a trace, rendering it void. Secure your damned database. The fault and responsibility lie with the deploying organisation and tools vendor. Meow is just the messenger.
- KMag 6y agoBut if they had used the password changes API to assign random passwords to all accounts, as suggested, then the data couldn't be modified. Am I missing something?
- dredmorbius 6y ago
- trevor-e 6y agoI hate to laugh when people's hard work is being destroyed, but this is some impressive trolling by the attacker: > An interesting theory as to why the attacker used the term "meow" is because cats like to drop (or knock) items from tables.
- LilBytes 6y agoI hadn't considered this. I'm enjoying this even more.