7 ms·

All forums with large threads have to paginate for performance, including HN. Discourse's infini-scroll feature is best understood as "automatic page turning."

All forums with large threads have to paginate for performance, including HN. Discourse's infini-scroll feature is best understood as "automatic page turning." In a paginated thread, you can only Ctrl-F for stuff on the current page, and Discourse is no different; it's no more "broken" than a traditional paginated forum with manual page turning. Similarly, you can only Ctrl-S to save the current page, not the whole thread. Many forums have a "search in this thread" feature, and Discourse does, too. In fact, if you click in the page and press Ctrl-F, it intercepts the keystroke and shows Discourse's "search in thread" search box; some forums with manual page turning do the same thing. Some people seem to get really pissed off at automatic page turning; I kinda understand why, but there are clear arguments for automatic page turning. Why make the user click "Next Page" when the software could do it for you? You'll notice that all of the most popular feed-based sites (Facebook, Twitter, Pinterest, Instagram) use automatic page turning, too, and that's because experiments show that people engage more when automatic page turning is enabled. None of this will make you any happier with APT, but hopefully it will annoy you slightly less to know that this is a data-driven decision and not done specifically to annoy you.

mattmanser 10y ago
HN doesn't paginate comments any more. Here's a thread with 970 comments for example: https://news.ycombinator.com/item?id=14023198 https://news.ycombinator.com/item?id=14023198
- ihuman 10y ago
  When did they make that change? A month ago or so, I remember having to click a link at the bottom of a page to load more comments.
  - cookiecaper 10y ago
    Same. I think that they enable it on a case-by-case basis as load gets high. Also, the "Who's hiring?" threads do not typically spawn active subthreads, so that may make them more cacheable/easier to load.
  - eriknstr 10y ago
    I've been on HN for a long time (this is not my first account even) but I had completely forgotten about pagination on HN. Even when you talked about it I only get a vague feeling that I've seen it ever. Probably I have seen it but still it feels like the sort of memory that one has of something that didn't actually happen but which you have a "memory" of because someone told you it happened. I guess I usually simply stop scrolling before I get to the very bottom of HN. Just to be clear though, are we talking about a "next" link at the bottom of the page, or Reddit-style "continue" links for subtrees of comments beyond a certain depth?
- 1wd 10y ago
  And the comments on that page (840kB) are smaller than the Javascript on a discourse site (4MB).
  - codinghorror 10y ago
    yes, but the minute you click a _second_ page, the Javascript app is already loaded, and we request only JSON for the new data. Also this 4MB number isn't correct; I count 700kb. Feel free to double check my work: http://imgur.com/a/gPiEb http://imgur.com/a/gPiEb
    - aleyan 10y ago
      Are you really arguing that Discourse's superior linear complexity outweighs the huge constant cost in any meaningfull scenario? At 4mb/840kb * 970 comments, disregarding any additional JSON data that Discourse loads, a Discourse thread would need to have 4974 comments before its strategy is more efficient than Hackernews. Jeff, I respect you, but I don't understand where you are going with that comeback.
      - accountyaccount 10y ago
        I'm sure they could get that 4mb down if they removed all text formatting, media, quotes, replies, editing tools, user icons, and pagination.
      - hashhar 10y ago
        There's a lot of other things Discourse threads have (media, collapsing, proper markdown, editing toolbars etc.).
        mattmanser 10y ago
        HN has collapsable threads and uses tables for layout, which aren't exactly efficient markdown, so makes the opposite point you're trying to make. Namely, even with bloody awful markdown, HN still delivers 1000 comments in 1/4 of the bytes. Another interesting idea, as almost everyone doesn't actually comment, wouldn't it make more sense to load the editor javascript dynamically? As far as I know, the silent majority read comments but never reply. In fact, I vaguely remember Jeff even having a blog post that most people never participate, they just consume (could be wrong!). Also, I haven't kept up with it, but mobile browsers used to have fairly aggressive cache invalidation to save space on the phone, so serving big chunks of JavaScript in the hope it might get reused one day wasn't actually a great idea.
finchisko 10y ago
>Similarly, you can only Ctrl-S to save the current page, not the whole thread. This brought me to thinking why browsers still save page as source code and not as serialised actual DOM? I mean developers still could use curl to save source code, but regular users will save what they see on the screen, similarly to printing to PDF.
- xfalcox 10y ago
  We made a feature, where we capture a CTRL+P and open a new window with the full topic and start the print, so it's easier to save a big topic to PDF and read it offline.
- thristian 10y ago
  What do you mean? Browsers do serialise the actual DOM. I know this because I once saved a JS-heavy page and when I loaded it back into the browser, I discovered the JS-generated menus were already present in the source and were re-generated by the JS each time the page loaded.
  - finchisko 10y ago
    >re-generated by the JS each time the page loaded. yes, regenerated because js files are also stored. So the menu generating functions and all constants needed to generate menus were probably present in downloaded js files. But imagine, site that is also fetching data via ajax and render that data somehow. When you save and then reopen saved page, you might not get exact state as you were in time of saving. There are many reason why state would be different (you could be logged off, etc...) So if you send saved page to somebody, he/she will see different state of page than you. All of this could be eliminated if browsers saved state of DOM instead of page source code (and js,css resources).
gue5t 10y ago
The tone of your message feels rather condescending, which is frustrating in the face of software usability issues. I think that in general it's a good idea to treat people's complaints as well-founded, even if they don't know exactly how to describe the underlying problem. I don't think it's the case that I and other users don't know what we want. I do believe that "people engage more" on infinite-scroll-style pages, but I have no reason to believe this makes them happier or more productive. On the other hand, the jerkiness/glitches, breakage of common navigational tools, and slowness of Discourse pages are very annoying. I should have been more descriptive than simply saying "broken", but I figured I was probably preaching to the choir and so I didn't go into details. In particular, it becomes much harder to use tools like ctrl+F and ctrl+S because the page content is dynamic and after scrolling down a little there's no way to easily tell which bits of the page are still loaded and which aren't. ctrl+F is useful not just for initially locating a key phrase, but also for jumping between bits of the page you've already read for comparison. In a manually-paginated thread this works fine, and I know for sure whether posts are on the same page (where I can use ctrl+F) or different pages (which I can open in separate tabs). Basically, it becomes very difficult for a user to maintain an accurate mental model of the page content that they're using a browser to interact with. This is quite different from static pagination systems, where scrolling never changes page contents. The scrollbar really is broken, because users expect its size to show the length of the page, which no longer holds for Discourse or other infinite-scroll pages. The fact that Discourse unloads earlier content makes this even worse, as you can't even remember an old scrollbar position to scroll to something you just saw or scroll halfway to the top to go halfway to the start of the page. Yes, Discourse does provide an alternative way to see some of this information, but user interface changes (having to move from the native scrollbar to the Discourse summary box) are frustrating for users: they break muscle memory and require learning a new interface, since they never work quite the same as the other one. It makes me no less upset to hear that data was involved in the decision to ignore users' needs. I don't think it's intentional to annoy users, but hearing that there was some level of analysis involved rather than mere accident is upsetting because it shows the slight to be willful. I don't think websites should push worsened usability on their users in the name of driving up engagement metrics. I think the arguments about automatic vs. manual page turning mostly come down to questions of control and implementation quality: users want to be in control (because the computer is a tool, and tools must be predictable), and it's difficult to avoid bugs when implementing automatic page turning on the Web. I'm not sure exactly why the latter is the case, but I've never seen a system without glitches in the history list, visual glitches, or slowness.
maxerickson 10y ago
I don't care that it does it, but I do wish they would develop it on underpowered computers.
deleted 10y ago
[deleted]
gabemart 10y ago
> All forums with large threads have to paginate for performance, including HN. I can read "War and Peace" [1] as an HTML document on my 7 year old cheapo Android phone. The browser even "streams" the data, displaying each chunk as it loads over a slow connection. As far as I know, you're correct that all forum software I've used paginates large threads, but I'm curious if they "have" to - as in, I think a team could probably build forum software that didn't paginate large threads (for reasonable definitions of "large") and still have good performance on a wide range of devices, if they were willing to make the necessary trade-offs. Whether those trade-offs are worth making, I don't know. [1] http://www.gutenberg.org/files/2600/2600-h/2600-h.htm http://www.gutenberg.org/files/2600/2600-h/2600-h.htm
- emodendroket 10y ago
  Well, for one thing, there is nothing relational about a giant HTML document. But for another, is a page that long really the ideal experience for a discussion forum?
  - dredmorbius 10y ago
    Why not allow the user or client define the pagination experience? You can get and read War and Peace (or other similarly large books -- I frequently reference Adam Smith's Wealth of Nations) at Project Gutenberg. As a single HTML file. The ability to search for any particular text instance within that document through browser capabilities is useful. As is, I agree at times, the ability to present the information as some sort of consistently paginated context. I'm thinking that somewhere between the 21st and 42nd centuries, computers might advance to a state where this is in fact possible.
    - emodendroket 10y ago
      If I wanted to search a discussion thread I'd rather use a search function.
      - deleted 10y ago
        [deleted]
deleted 10y ago
[deleted]

mattmanser 10y ago

HN doesn't paginate comments any more. Here's a thread with 970 comments for example: https://news.ycombinator.com/item?id=14023198 https://news.ycombinator.com/item?id=14023198

ihuman 10y ago
When did they make that change? A month ago or so, I remember having to click a link at the bottom of a page to load more comments.
- cookiecaper 10y ago
  Same. I think that they enable it on a case-by-case basis as load gets high. Also, the "Who's hiring?" threads do not typically spawn active subthreads, so that may make them more cacheable/easier to load.
- eriknstr 10y ago
  I've been on HN for a long time (this is not my first account even) but I had completely forgotten about pagination on HN. Even when you talked about it I only get a vague feeling that I've seen it ever. Probably I have seen it but still it feels like the sort of memory that one has of something that didn't actually happen but which you have a "memory" of because someone told you it happened. I guess I usually simply stop scrolling before I get to the very bottom of HN. Just to be clear though, are we talking about a "next" link at the bottom of the page, or Reddit-style "continue" links for subtrees of comments beyond a certain depth?
1wd 10y ago
And the comments on that page (840kB) are smaller than the Javascript on a discourse site (4MB).
- codinghorror 10y ago
  yes, but the minute you click a _second_ page, the Javascript app is already loaded, and we request only JSON for the new data. Also this 4MB number isn't correct; I count 700kb. Feel free to double check my work: http://imgur.com/a/gPiEb http://imgur.com/a/gPiEb
  - aleyan 10y ago
    Are you really arguing that Discourse's superior linear complexity outweighs the huge constant cost in any meaningfull scenario? At 4mb/840kb * 970 comments, disregarding any additional JSON data that Discourse loads, a Discourse thread would need to have 4974 comments before its strategy is more efficient than Hackernews. Jeff, I respect you, but I don't understand where you are going with that comeback.
    - accountyaccount 10y ago
      I'm sure they could get that 4mb down if they removed all text formatting, media, quotes, replies, editing tools, user icons, and pagination.
    - hashhar 10y ago
      There's a lot of other things Discourse threads have (media, collapsing, proper markdown, editing toolbars etc.).
      - mattmanser 10y ago
        HN has collapsable threads and uses tables for layout, which aren't exactly efficient markdown, so makes the opposite point you're trying to make. Namely, even with bloody awful markdown, HN still delivers 1000 comments in 1/4 of the bytes. Another interesting idea, as almost everyone doesn't actually comment, wouldn't it make more sense to load the editor javascript dynamically? As far as I know, the silent majority read comments but never reply. In fact, I vaguely remember Jeff even having a blog post that most people never participate, they just consume (could be wrong!). Also, I haven't kept up with it, but mobile browsers used to have fairly aggressive cache invalidation to save space on the phone, so serving big chunks of JavaScript in the hope it might get reused one day wasn't actually a great idea.

finchisko 10y ago

>Similarly, you can only Ctrl-S to save the current page, not the whole thread. This brought me to thinking why browsers still save page as source code and not as serialised actual DOM? I mean developers still could use curl to save source code, but regular users will save what they see on the screen, similarly to printing to PDF.

xfalcox 10y ago
We made a feature, where we capture a CTRL+P and open a new window with the full topic and start the print, so it's easier to save a big topic to PDF and read it offline.
thristian 10y ago
What do you mean? Browsers do serialise the actual DOM. I know this because I once saved a JS-heavy page and when I loaded it back into the browser, I discovered the JS-generated menus were already present in the source and were re-generated by the JS each time the page loaded.
- finchisko 10y ago
  >re-generated by the JS each time the page loaded. yes, regenerated because js files are also stored. So the menu generating functions and all constants needed to generate menus were probably present in downloaded js files. But imagine, site that is also fetching data via ajax and render that data somehow. When you save and then reopen saved page, you might not get exact state as you were in time of saving. There are many reason why state would be different (you could be logged off, etc...) So if you send saved page to somebody, he/she will see different state of page than you. All of this could be eliminated if browsers saved state of DOM instead of page source code (and js,css resources).

gue5t 10y ago

The tone of your message feels rather condescending, which is frustrating in the face of software usability issues. I think that in general it's a good idea to treat people's complaints as well-founded, even if they don't know exactly how to describe the underlying problem. I don't think it's the case that I and other users don't know what we want. I do believe that "people engage more" on infinite-scroll-style pages, but I have no reason to believe this makes them happier or more productive. On the other hand, the jerkiness/glitches, breakage of common navigational tools, and slowness of Discourse pages are very annoying. I should have been more descriptive than simply saying "broken", but I figured I was probably preaching to the choir and so I didn't go into details. In particular, it becomes much harder to use tools like ctrl+F and ctrl+S because the page content is dynamic and after scrolling down a little there's no way to easily tell which bits of the page are still loaded and which aren't. ctrl+F is useful not just for initially locating a key phrase, but also for jumping between bits of the page you've already read for comparison. In a manually-paginated thread this works fine, and I know for sure whether posts are on the same page (where I can use ctrl+F) or different pages (which I can open in separate tabs). Basically, it becomes very difficult for a user to maintain an accurate mental model of the page content that they're using a browser to interact with. This is quite different from static pagination systems, where scrolling never changes page contents. The scrollbar really is broken, because users expect its size to show the length of the page, which no longer holds for Discourse or other infinite-scroll pages. The fact that Discourse unloads earlier content makes this even worse, as you can't even remember an old scrollbar position to scroll to something you just saw or scroll halfway to the top to go halfway to the start of the page. Yes, Discourse does provide an alternative way to see some of this information, but user interface changes (having to move from the native scrollbar to the Discourse summary box) are frustrating for users: they break muscle memory and require learning a new interface, since they never work quite the same as the other one. It makes me no less upset to hear that data was involved in the decision to ignore users' needs. I don't think it's intentional to annoy users, but hearing that there was some level of analysis involved rather than mere accident is upsetting because it shows the slight to be willful. I don't think websites should push worsened usability on their users in the name of driving up engagement metrics. I think the arguments about automatic vs. manual page turning mostly come down to questions of control and implementation quality: users want to be in control (because the computer is a tool, and tools must be predictable), and it's difficult to avoid bugs when implementing automatic page turning on the Web. I'm not sure exactly why the latter is the case, but I've never seen a system without glitches in the history list, visual glitches, or slowness.

maxerickson 10y ago

I don't care that it does it, but I do wish they would develop it on underpowered computers.

deleted 10y ago

[deleted]

gabemart 10y ago

> All forums with large threads have to paginate for performance, including HN. I can read "War and Peace" [1] as an HTML document on my 7 year old cheapo Android phone. The browser even "streams" the data, displaying each chunk as it loads over a slow connection. As far as I know, you're correct that all forum software I've used paginates large threads, but I'm curious if they "have" to - as in, I think a team could probably build forum software that didn't paginate large threads (for reasonable definitions of "large") and still have good performance on a wide range of devices, if they were willing to make the necessary trade-offs. Whether those trade-offs are worth making, I don't know. [1] http://www.gutenberg.org/files/2600/2600-h/2600-h.htm http://www.gutenberg.org/files/2600/2600-h/2600-h.htm

emodendroket 10y ago
Well, for one thing, there is nothing relational about a giant HTML document. But for another, is a page that long really the ideal experience for a discussion forum?
- dredmorbius 10y ago
  Why not allow the user or client define the pagination experience? You can get and read War and Peace (or other similarly large books -- I frequently reference Adam Smith's Wealth of Nations) at Project Gutenberg. As a single HTML file. The ability to search for any particular text instance within that document through browser capabilities is useful. As is, I agree at times, the ability to present the information as some sort of consistently paginated context. I'm thinking that somewhere between the 21st and 42nd centuries, computers might advance to a state where this is in fact possible.
  - emodendroket 10y ago
    If I wanted to search a discussion thread I'd rather use a search function.
    - deleted 10y ago
      [deleted]

deleted 10y ago

[deleted]