5 ms·

Improving the New York Times’ line wrap balancer

60 points by finite_jest 5y ago

twhb 5y ago
I wonder if the nascent CSS Layout API enables a proper solution to this. A quick skim of the spec seems to indicate that it could run the author’s algorithm in a single layout pass and it does reveal the browser’s chosen line break points, but it doesn’t give word by word metrics. But, maybe you could use custom elements and JS-inserted children to get word by word metrics with performance still good since there’s still only one pass.
SeanLuke 5y ago
Such a long discussion and no mention the Knuth algorithm?
- gwern 5y ago
  "The method is suitable for headlines and a single short paragraph of text. It’s not suitable for long paragraphs of texts or multiple paragraphs. It yields no benefit over the browser’s default layout engine when the text exceeds a couple of lines. You’ll also end up with paragraphs of uneven widths, which breaks up the cohesion of the right-hand text edge of your design. It works great for two–four lines of text, though." Knuth-Plass is for setting all text to a specified width and balancing as best as possible within that, not having varying widths.
- d2wa 5y ago
  It wasn’t really relevant in this context. It’s similar to what the browsers do by default, however.
nyanpasu64 5y ago
I'm disappointed to see websites stack increasing amounts of JS to micro-optimize page appearance, when New York Times won't even load images below the first one per article with JS disabled.
- JadeNB 5y ago
  > I'm disappointed to see websites stack increasing amounts of JS to micro-optimize page appearance, when New York Times won't even load images below the first one per article with JS disabled. I think this isn't a failure of technical will, but scorn for non-JS users. At least on WP—I've never tried NYT without Javascript—disabling Javascript gets you past the paywall, and I assume otherwise cripples their tracking, so they don't want you there.
  - Wowfunhappy 5y ago
    > disabling Javascript gets you past the paywall Well, this could also tell you just how few users disable Javascript. The numbers are so small, the Washington Post would rather let them read for free than update their paywall code and potentially convert them to subscribers.
    - micromacrofoot 5y ago
      most people don’t even know what javascript is
hoten 5y ago
All this effort for what is certain to be an imprecise implementation and slow (forcing the page to redo layout multiple times is not worth solving the widow problem, IMO). Why not spend the effort exploring the problem at the browser implementation level? Is there opportunity to get NYT eng time dedicated to it?
- promblom 5y ago
  From the article: > In 2013, Adobe proposed a new CSS text-wrap: balance property to make the browser do (and optimize) the heavy lifting. TextBalancer is a JavaScript implementation of this CSS property. The property has yet to be implemented into any web browsers. (Come on, guys! It’s in CSS Text Level 4.)
  - hoten 5y ago
    Yes, I read it. if they are able to make the investment they can work on the implementation themselves instead of waiting for someone else to take it up. They are the most likely to benefit from it.
- Wowfunhappy 5y ago
  > forcing the page to redo layout multiple times is not worth solving the widow problem, IMO I'm not sure I agree, if it enhances readability. The users are there to read articles, so this is exactly the right place to spend spare CPU cycles. Of course, it should be optimized as much as possible, and it sounds like the NYT's implementation was not very optimized. Then again, it also sounds like the NYT only briefly experimented with their version, and does not use it today. > Why not spend the effort exploring the problem at the browser implementation level? Well, but the NYT is not fundamentally a technology company, so pushing for browser changes seems beyond their scope IMO. Adobe did try, for their part.
- d2wa 5y ago
  Adobe has made a proposal for a new CSS attribute, but no browser has implemented it. This is discussed in the article.
bastawhiz 5y ago
I'm trying to read the code linked on Codeberg, which I've never heard of before. It's astounding how bad it is. The file list truncates file names when there's plenty of space. The code view has a margin and huge line numbers, but then tries to wrap lines. But it wraps the code mid-word, which makes it so much harder to follow. Simply putting the code in a PRE tag and calling it a day would have been preferable; how is it possible to get displaying code so wrong in the year of the Linux desktop 2021? This is table stakes for a git service.
loevborg 5y ago
In my experience, browsers are not good at word-wrapping a div with, say, width=100px in a consistent way. First, different browsers break text at different points. Second, if the div is scaled (e.g. to 120px) and the font-size is scaled accordingly, this may again cause line breaks to occur at different points, potentially causing the number of lines to change. This was surprising to me. What are my options if I want to enforce consistent line breaks? I've thought about "freezing" line breaks by inserting <br/> tags, but to do that I would have to find out where the browser actually breaks the text, and I'm not sure how to go about that task.
- d2wa 5y ago
  > First, different browsers break text at different points. If the screen resolution and scaling is the same, and the operating system is the same, then you should get very consistent results across operating systems. They use the same algorithm, but subpixel hinting may differ slightly. If you stick to metrically compatible fonts for the different operating systems, you’ll get almost identical results across OSes too (assuming same screen resolution and scaling). https://www.ctrl.blog/entry/font-stack-text.html https://www.ctrl.blog/entry/font-stack-text.html
  - loevborg 5y ago
    I agree that font rendering is fairly consistent. The problem is that a single pixel difference in vertical rendering can affect word-wrapping in such a way that a new line is added, making the div taller, potentially breaking the page layout. One related thing I know for a fact is different between browsers is how they treat fractional line-height. While Firefox accurately scales the line-height along with a fractional font-height, Chrome notoriously snaps the line-height to an integer number.
    - nyanpasu64 5y ago
      And both Firefox and Chrome quantize the height of text divs integer physical pixels, meaning the size of a div changes based on zoom level and DPI scaling, and can even take on fractional levels on non-100% scaling!
Wowfunhappy 5y ago
Random thought—would it perhaps make sense to do this server side? You could pre-calculate the optimal width at a bunch of different viewports, and send that to the client. You could generate a stylesheet with a stupid amount of generated breakpoints, or you could assume the user isn't going to resize the browser after loading the page. (Most won't, and if they do, the only consequence is non-ideal text wrapping).
- somebodythere 5y ago
  Does the server get the viewport size as part of the request? The second problem is solvable though I think, by loading the JS text balancer asynchronously.
  - Wowfunhappy 5y ago
    > Does the server get the viewport size as part of the request? ...oh, uh, no, that was dumb of me. I do still wonder though if auto-generating a css file with lots of media queries would be feasible.
    - somebodythere 5y ago
      Sure, you only need tens of breakpoints setting one style rule each. Gzipped I bet it would weigh about as much as the JS text-wrapper implementation.
      - Wowfunhappy 5y ago
        It should be lighter on CPU though, and avoid a FOUC.
  - d2wa 5y ago
    > Does the server get the viewport size as part of the request? It can, yes. The response header `Accept-CH: Viewport-Width` suggest to the client that future request should transmit a `Viewport-Width: 320` header. Chrome only as the other browsers consider it harmful. It doesn’t solve the problem, though. Font availability and subpixel rendering varies from OS to OS and browser to browser.

60 points by finite_jest 5y ago

twhb 5y ago

I wonder if the nascent CSS Layout API enables a proper solution to this. A quick skim of the spec seems to indicate that it could run the author’s algorithm in a single layout pass and it does reveal the browser’s chosen line break points, but it doesn’t give word by word metrics. But, maybe you could use custom elements and JS-inserted children to get word by word metrics with performance still good since there’s still only one pass.

SeanLuke 5y ago

Such a long discussion and no mention the Knuth algorithm?

gwern 5y ago
"The method is suitable for headlines and a single short paragraph of text. It’s not suitable for long paragraphs of texts or multiple paragraphs. It yields no benefit over the browser’s default layout engine when the text exceeds a couple of lines. You’ll also end up with paragraphs of uneven widths, which breaks up the cohesion of the right-hand text edge of your design. It works great for two–four lines of text, though." Knuth-Plass is for setting all text to a specified width and balancing as best as possible within that, not having varying widths.
d2wa 5y ago
It wasn’t really relevant in this context. It’s similar to what the browsers do by default, however.

nyanpasu64 5y ago

I'm disappointed to see websites stack increasing amounts of JS to micro-optimize page appearance, when New York Times won't even load images below the first one per article with JS disabled.

JadeNB 5y ago
> I'm disappointed to see websites stack increasing amounts of JS to micro-optimize page appearance, when New York Times won't even load images below the first one per article with JS disabled. I think this isn't a failure of technical will, but scorn for non-JS users. At least on WP—I've never tried NYT without Javascript—disabling Javascript gets you past the paywall, and I assume otherwise cripples their tracking, so they don't want you there.
- Wowfunhappy 5y ago
  > disabling Javascript gets you past the paywall Well, this could also tell you just how few users disable Javascript. The numbers are so small, the Washington Post would rather let them read for free than update their paywall code and potentially convert them to subscribers.
  - micromacrofoot 5y ago
    most people don’t even know what javascript is

hoten 5y ago

All this effort for what is certain to be an imprecise implementation and slow (forcing the page to redo layout multiple times is not worth solving the widow problem, IMO). Why not spend the effort exploring the problem at the browser implementation level? Is there opportunity to get NYT eng time dedicated to it?

promblom 5y ago
From the article: > In 2013, Adobe proposed a new CSS text-wrap: balance property to make the browser do (and optimize) the heavy lifting. TextBalancer is a JavaScript implementation of this CSS property. The property has yet to be implemented into any web browsers. (Come on, guys! It’s in CSS Text Level 4.)
- hoten 5y ago
  Yes, I read it. if they are able to make the investment they can work on the implementation themselves instead of waiting for someone else to take it up. They are the most likely to benefit from it.
Wowfunhappy 5y ago
> forcing the page to redo layout multiple times is not worth solving the widow problem, IMO I'm not sure I agree, if it enhances readability. The users are there to read articles, so this is exactly the right place to spend spare CPU cycles. Of course, it should be optimized as much as possible, and it sounds like the NYT's implementation was not very optimized. Then again, it also sounds like the NYT only briefly experimented with their version, and does not use it today. > Why not spend the effort exploring the problem at the browser implementation level? Well, but the NYT is not fundamentally a technology company, so pushing for browser changes seems beyond their scope IMO. Adobe did try, for their part.
d2wa 5y ago
Adobe has made a proposal for a new CSS attribute, but no browser has implemented it. This is discussed in the article.

bastawhiz 5y ago

I'm trying to read the code linked on Codeberg, which I've never heard of before. It's astounding how bad it is. The file list truncates file names when there's plenty of space. The code view has a margin and huge line numbers, but then tries to wrap lines. But it wraps the code mid-word, which makes it so much harder to follow. Simply putting the code in a PRE tag and calling it a day would have been preferable; how is it possible to get displaying code so wrong in the year of the Linux desktop 2021? This is table stakes for a git service.

loevborg 5y ago

In my experience, browsers are not good at word-wrapping a div with, say, width=100px in a consistent way. First, different browsers break text at different points. Second, if the div is scaled (e.g. to 120px) and the font-size is scaled accordingly, this may again cause line breaks to occur at different points, potentially causing the number of lines to change. This was surprising to me. What are my options if I want to enforce consistent line breaks? I've thought about "freezing" line breaks by inserting <br/> tags, but to do that I would have to find out where the browser actually breaks the text, and I'm not sure how to go about that task.

d2wa 5y ago
> First, different browsers break text at different points. If the screen resolution and scaling is the same, and the operating system is the same, then you should get very consistent results across operating systems. They use the same algorithm, but subpixel hinting may differ slightly. If you stick to metrically compatible fonts for the different operating systems, you’ll get almost identical results across OSes too (assuming same screen resolution and scaling). https://www.ctrl.blog/entry/font-stack-text.html https://www.ctrl.blog/entry/font-stack-text.html
- loevborg 5y ago
  I agree that font rendering is fairly consistent. The problem is that a single pixel difference in vertical rendering can affect word-wrapping in such a way that a new line is added, making the div taller, potentially breaking the page layout. One related thing I know for a fact is different between browsers is how they treat fractional line-height. While Firefox accurately scales the line-height along with a fractional font-height, Chrome notoriously snaps the line-height to an integer number.
  - nyanpasu64 5y ago
    And both Firefox and Chrome quantize the height of text divs integer physical pixels, meaning the size of a div changes based on zoom level and DPI scaling, and can even take on fractional levels on non-100% scaling!

Wowfunhappy 5y ago

Random thought—would it perhaps make sense to do this server side? You could pre-calculate the optimal width at a bunch of different viewports, and send that to the client. You could generate a stylesheet with a stupid amount of generated breakpoints, or you could assume the user isn't going to resize the browser after loading the page. (Most won't, and if they do, the only consequence is non-ideal text wrapping).

somebodythere 5y ago
Does the server get the viewport size as part of the request? The second problem is solvable though I think, by loading the JS text balancer asynchronously.
- Wowfunhappy 5y ago
  > Does the server get the viewport size as part of the request? ...oh, uh, no, that was dumb of me. I do still wonder though if auto-generating a css file with lots of media queries would be feasible.
  - somebodythere 5y ago
    Sure, you only need tens of breakpoints setting one style rule each. Gzipped I bet it would weigh about as much as the JS text-wrapper implementation.
    - Wowfunhappy 5y ago
      It should be lighter on CPU though, and avoid a FOUC.
- d2wa 5y ago
  > Does the server get the viewport size as part of the request? It can, yes. The response header `Accept-CH: Viewport-Width` suggest to the client that future request should transmit a `Viewport-Width: 320` header. Chrome only as the other browsers consider it harmful. It doesn’t solve the problem, though. Font availability and subpixel rendering varies from OS to OS and browser to browser.