3 ms·

It seems simple enough to just generate the conversation as one big page, then break it up. As you traverse the tree, save a list of indices of top-level comme

by mkl 5y ago

It seems simple enough to just generate the conversation as one big page, then break it up. As you traverse the tree, save a list of indices of top-level comments and store the (recursive) sum of children sizes, then split appropriately. This might be slow with millions of nodes, but I don't understand how that process alone would be slow with hundreds. What would be slow (and may be happening), is generating that dynamically for every page refresh on a busy story by every visitor (i.e. no caching).

fatih-erikli 5y ago
Generating one big page is not the scalable way of doing that, I think, It will eventually reach to a point that it's no longer a web-page but a downloadable document with couple of megabytes. I'm interested to brain-storm about how to model and implement "Breaking-up" part of your comment. For the backend part, I think Splay-tree kind of a data structure will help to serve it faster.
- mkl 5y ago
  I misspoke a little. I wouldn't generate the whole thing as a blob of text and then split that. I would traverse the tree depth-first making HTML blobs for new comments that that hasn't been done for already, generate a linear list of comment items that each know the size of their own subtree (set that when you traverse back up to the node), and make a list of the indices of the top-level comments as I go. Then you can go through the top-level comments, adding them (and their subtrees - everything up to the next top-level index in the list) one by one into your webpage and adding up the total size so far until you reach some threshold of "too big", at which point you finish off that page and start the next one. I think caching would be important to make it fast, and you could cache the HTML of each comment or subtree. Actually, if I was making my own forum and not copying HN I would probably have the comment tree structure represented in the DOM tree, as that seems better for navigation and accessibility. All of this is totally theoretical of course! I haven't actually done it, so may be wrong.
  - fatih-erikli 5y ago
    "Comment tree structure represented in the DOM tree, as that seems better for navigation and accessibility." I agree. It's also good for screen readers, and you can also have keyboard shortcuts to navigate on tree.

by mkl 5y ago

fatih-erikli 5y ago

Generating one big page is not the scalable way of doing that, I think, It will eventually reach to a point that it's no longer a web-page but a downloadable document with couple of megabytes. I'm interested to brain-storm about how to model and implement "Breaking-up" part of your comment. For the backend part, I think Splay-tree kind of a data structure will help to serve it faster.

mkl 5y ago
I misspoke a little. I wouldn't generate the whole thing as a blob of text and then split that. I would traverse the tree depth-first making HTML blobs for new comments that that hasn't been done for already, generate a linear list of comment items that each know the size of their own subtree (set that when you traverse back up to the node), and make a list of the indices of the top-level comments as I go. Then you can go through the top-level comments, adding them (and their subtrees - everything up to the next top-level index in the list) one by one into your webpage and adding up the total size so far until you reach some threshold of "too big", at which point you finish off that page and start the next one. I think caching would be important to make it fast, and you could cache the HTML of each comment or subtree. Actually, if I was making my own forum and not copying HN I would probably have the comment tree structure represented in the DOM tree, as that seems better for navigation and accessibility. All of this is totally theoretical of course! I haven't actually done it, so may be wrong.
- fatih-erikli 5y ago
  "Comment tree structure represented in the DOM tree, as that seems better for navigation and accessibility." I agree. It's also good for screen readers, and you can also have keyboard shortcuts to navigate on tree.