4 ms·

I was just thinking, what could be the problem in Elon Musk's thread. It shows just one comment. I didn't say there is a dynamic viewport optimization. The iss

I was just thinking, what could be the problem in Elon Musk's thread. It shows just one comment. I didn't say there is a dynamic viewport optimization. The issue is not in DOM structure. I'm creator of "arguman.org", maybe you've heard about it (it's down now). I had the same issues there as well. I'm also happy with HN's plain html interface. Also, how would you render a page with 800 nodes with user content, it is still a question. I would like to hear and learn. Btw, yes, Bulletin board style forums is a strait forward solution, there's always some ways to do that, but there's also ways to do it like in Twitter's way. That's the point I was talking about.

krapp 5y ago
>Also, how would you render a page with 800 nodes with user content, it is still a question. I would like to hear and learn. It would be trivial with any site using a relational database. The two basic models used are nested sets[0], which optimize rendering over insertion and deletion, versus the much simpler relational model (rows with id and parent id, sometimes root id), which is much easier to insert and delete from, but requires recursive queries to render an entire tree from any branch other than the root. The vast majority of threaded forums use the former model, many larger sites use the latter, and rendering large threads isn't an issue either way unless maybe you're at Reddit scale, which most forums including HN aren't. Hacker News doesn't use a relational database, it uses flat text files (limiting it to file I/O speed for data not cached in RAM), and it also runs under a single thread with limited RAM. It likely wasn't designed to operate efficiently at the scale that it currently does, and the increasing size and velocity of threads is pushing it to its limit. Edit: looks like the stone passed. [0]https://en.wikipedia.org/wiki/Nested_set_model https://en.wikipedia.org/wiki/Nested_set_model
- fatih-erikli 5y ago
  Thanks for your comments. Optimizing the render part in insertion and deletion is also difficult. Some nodes might have user-level views, so some of them won't be visible to some other users. So you can't just label the nodes like this is "left" or "right" of that record. My experience with a similar problem: Instead of putting the rendering part to the insertion and deletion (nested set model), I was using a simple nginx file based cache. This is a trade off and I chose the application layer to be "consistent" and "reliable", over "being fast". Keeping the data integrity is difficult when you have a nested set model and it's also expensive to doing that calculation during the insertion. Sometimes you need a message broker to handle such an expensive query, because it's not safe to do it within a request-response lifecycle, even if you do have a transactional database, there's always a barrier of "request time out".

krapp 5y ago

>Also, how would you render a page with 800 nodes with user content, it is still a question. I would like to hear and learn. It would be trivial with any site using a relational database. The two basic models used are nested sets[0], which optimize rendering over insertion and deletion, versus the much simpler relational model (rows with id and parent id, sometimes root id), which is much easier to insert and delete from, but requires recursive queries to render an entire tree from any branch other than the root. The vast majority of threaded forums use the former model, many larger sites use the latter, and rendering large threads isn't an issue either way unless maybe you're at Reddit scale, which most forums including HN aren't. Hacker News doesn't use a relational database, it uses flat text files (limiting it to file I/O speed for data not cached in RAM), and it also runs under a single thread with limited RAM. It likely wasn't designed to operate efficiently at the scale that it currently does, and the increasing size and velocity of threads is pushing it to its limit. Edit: looks like the stone passed. [0]https://en.wikipedia.org/wiki/Nested_set_model https://en.wikipedia.org/wiki/Nested_set_model

fatih-erikli 5y ago
Thanks for your comments. Optimizing the render part in insertion and deletion is also difficult. Some nodes might have user-level views, so some of them won't be visible to some other users. So you can't just label the nodes like this is "left" or "right" of that record. My experience with a similar problem: Instead of putting the rendering part to the insertion and deletion (nested set model), I was using a simple nginx file based cache. This is a trade off and I chose the application layer to be "consistent" and "reliable", over "being fast". Keeping the data integrity is difficult when you have a nested set model and it's also expensive to doing that calculation during the insertion. Sometimes you need a message broker to handle such an expensive query, because it's not safe to do it within a request-response lifecycle, even if you do have a transactional database, there's always a barrier of "request time out".