3 ms·

HN runs on plain files. He wasn't modifying database, but calling functions(I believe) in the repl to change the parent id of the thread. But that apart, even

by irahul 14y ago

HN runs on plain files. He wasn't modifying database, but calling functions(I believe) in the repl to change the parent id of the thread. But that apart, even if there were an admin button to change the parent id of a thread, he would still have made the same mistake. Unless the code in question was checking for loops. In that case, repl would have worked the same.

lucb1e 14y ago
I sort of meant that you shouldn't modify things like that directly. Be it a filesystem, database, or any other place that makes it possible to mess things up to bring a rather strong server down.
- triplepoint217 14y ago
  Yes, I was trying to make posts editable on the HN instance I run, so I got clever and started messing with the files in emacs. Then I learned that the HN code does not like files in the story directory with ~ on the end of their name (emacs backup files), oops ;).
- irahul 14y ago
  I see where you are coming from. But I am saying this didn't happen because he did things live. This happened because he entered incorrect id making a thread its own parent(or grandparent; doesn't matter). This is the kind of mistake one would make even if you were writing proper migrations. He was doing things live isn't an issue; neither is an incorrect id. The issue is the code doesn't check for loops.
  - chernevik 14y ago
    I am but an egg, I have two questions. One, if the data were held in a database, should a change like this be captured in the database logs? I am seeing more and more situations where I want these, I notice that they are by default turned off for mysql and wonder if this reflects a de facto judgment that logging slows performance more than is usually worthwhile. Two, if the data were kept in a database, wouldn't something like this be prevented by a constraint preventing a comment from making itself an ancestor? But I suppose there is a slight performance hit in checking such constraints, and the case arises so rarely that this hit isn't generally worthwhile.
    - lmm 14y ago
      Databases, at least the SQL kind, really aren't good at dealing with hierarchical data, and I don't know how you'd even begin to express that kind of constraint. I don't think a traditional database is the answer here. (If it were me, once I'd done it more than twice I'd write a "move thread" admin tool in the UI, and after I screwed it up like this I'd have a place to add such a check to).
      - mr_luc 14y ago
        If you were using some kind of representation for Nested Sets -- left-to-right depth-first numbering, or a human-readable id.id.id chain -- then it's really easy to write a constraint for that: parent left < myleft, right > myright, or dotted_id.split('.').filter{|first, rest| return false if rest.contains first} (yeah, yeah, that second pseudocode would be unrealistically PITA for some DBs). More generally: I'm not a big SQL wonk anymore, but I find a lot of people have the intuition that relational databases are ill-suited for trees. An intuition that is much closer to the truth is that almost all databases can handle trees pretty well, because there's still an unambiguous concept of ordering and containment, and you can usually arrange things so as to do range/ancestor/inclusion queries efficiently. It's graphs with loops/without unambiguous concept of ordering/containment that are really hard.
        EEGuy 14y ago
        Found this: The excellent Postgres documentation includes an SQL graph search with two different ways of graph cycle checking, here: http://www.postgresql.org/docs/9.0/static/queries-with.html http://www.postgresql.org/docs/9.0/static/queries-with.html One way involves accumulating an array of nodes already visited as the tree gets walked, checking each node as-visited for membership in the array-to-date. The other method, a bit more of a hack, is just adding a LIMIT clause. I think the 'WITH' clause is a great addition to the SQL standard, very much worth the learning the weirdness of its syntax and its optional 'RECURSIVE' term (which, as the Postgres documentation points out, isn't really recursion, it's iteration).
      - irahul 14y ago
gummadi 14y ago
Glad that you bought this up. If you have more knowledge regarding this, can you please explain how exactly the posts & nested comments are stored directly using flat files. How are concurrency issues handled?

by irahul 14y ago

lucb1e 14y ago

I sort of meant that you shouldn't modify things like that directly. Be it a filesystem, database, or any other place that makes it possible to mess things up to bring a rather strong server down.

triplepoint217 14y ago
Yes, I was trying to make posts editable on the HN instance I run, so I got clever and started messing with the files in emacs. Then I learned that the HN code does not like files in the story directory with ~ on the end of their name (emacs backup files), oops ;).
irahul 14y ago
I see where you are coming from. But I am saying this didn't happen because he did things live. This happened because he entered incorrect id making a thread its own parent(or grandparent; doesn't matter). This is the kind of mistake one would make even if you were writing proper migrations. He was doing things live isn't an issue; neither is an incorrect id. The issue is the code doesn't check for loops.
- chernevik 14y ago
  I am but an egg, I have two questions. One, if the data were held in a database, should a change like this be captured in the database logs? I am seeing more and more situations where I want these, I notice that they are by default turned off for mysql and wonder if this reflects a de facto judgment that logging slows performance more than is usually worthwhile. Two, if the data were kept in a database, wouldn't something like this be prevented by a constraint preventing a comment from making itself an ancestor? But I suppose there is a slight performance hit in checking such constraints, and the case arises so rarely that this hit isn't generally worthwhile.
  - lmm 14y ago
    Databases, at least the SQL kind, really aren't good at dealing with hierarchical data, and I don't know how you'd even begin to express that kind of constraint. I don't think a traditional database is the answer here. (If it were me, once I'd done it more than twice I'd write a "move thread" admin tool in the UI, and after I screwed it up like this I'd have a place to add such a check to).
    - mr_luc 14y ago
      If you were using some kind of representation for Nested Sets -- left-to-right depth-first numbering, or a human-readable id.id.id chain -- then it's really easy to write a constraint for that: parent left < myleft, right > myright, or dotted_id.split('.').filter{|first, rest| return false if rest.contains first} (yeah, yeah, that second pseudocode would be unrealistically PITA for some DBs). More generally: I'm not a big SQL wonk anymore, but I find a lot of people have the intuition that relational databases are ill-suited for trees. An intuition that is much closer to the truth is that almost all databases can handle trees pretty well, because there's still an unambiguous concept of ordering and containment, and you can usually arrange things so as to do range/ancestor/inclusion queries efficiently. It's graphs with loops/without unambiguous concept of ordering/containment that are really hard.
      - EEGuy 14y ago
        Found this: The excellent Postgres documentation includes an SQL graph search with two different ways of graph cycle checking, here: http://www.postgresql.org/docs/9.0/static/queries-with.html http://www.postgresql.org/docs/9.0/static/queries-with.html One way involves accumulating an array of nodes already visited as the tree gets walked, checking each node as-visited for membership in the array-to-date. The other method, a bit more of a hack, is just adding a LIMIT clause. I think the 'WITH' clause is a great addition to the SQL standard, very much worth the learning the weirdness of its syntax and its optional 'RECURSIVE' term (which, as the Postgres documentation points out, isn't really recursion, it's iteration).
    - irahul 14y ago

gummadi 14y ago

Glad that you bought this up. If you have more knowledge regarding this, can you please explain how exactly the posts & nested comments are stored directly using flat files. How are concurrency issues handled?