3 ms·

I would not consider myself super deep in web technology, but rendering to canvas allows programmers to have pixel perfect control over the look of their applic

by jerry1979 5y ago

I would not consider myself super deep in web technology, but rendering to canvas allows programmers to have pixel perfect control over the look of their applications across all devices. Currently, web developers need to "reset" lots of default rendering behaviors in every major browser to ensure that their applications look the same. After building lots of specialized UI components within the HTML standard, a programmer may ask themself if they might as well write their own UI library. Specifically, lots of specialized applications have UI components which do not have a corresponding HTML standard. For example, in a spreadsheet, a cell may have a clickable triangle in its upper right corner that should display a comment bubble. Should a programmer create that in css or write a specialized library?

remram 5y ago
Do your users care about your app looking the same or do they care about their browser looking and acting like a browser? Moving to canvas is sure to break many features such as text selection, adblocking, and accessibility. All in the name of more controls over the pixels? Are you truly doing it for the users?
- kroltan 5y ago
  Very good point, users do not care that a padding is rounded up or down when they switch from their laptop to their desktop, as long as the application is usable, understandable and visually competent. But mindspace is much more important for interaction, so if their laptop is a Mac, their brain will be in "Mac mode", and "Linux" or "Windows" mode when on their other device. Respecting the platform's conventions will allow them to keep their cognitive load due to "fiddling" to a minimum.
  - remram 5y ago
    I don't think the GP was talking about the app looking consistent with the rest of the platform, but the exact opposite: the app looking the same whatever the platform. Using Flutter allows them to have the app look the same on the browser as on mobile. This means that at best it will look "Mac mode" for everyone, including Windows users; at worst it will look foreign to everyone.
    - kroltan 5y ago
      Yes, I understand and agree. My second paragraph was maybe a bit unclear, I meant that the user would expect platform conventions to be respected over application conventions. (especially if we consider platforms with different primary interactions)
    - BlueTemplar 5y ago
      Having a program look the same on a big screen with mouse+keyboard input as well as on a small touchscreen is a recipe for having a bad user experience on both.
- tshaddox 5y ago
  I might be wrong, but I'm pretty sure Google Docs is already using a completely custom implementation of word wrapping, text layout, text selection, cursor placement, etc. It's not like it's just a <textarea /> with some CSS styles. Likewise, they seem to already have completely separate DOM elements that are invisible to normal browsers but can be read by screen readers. Based on the DOM on the new read-only preview document they link to, it looks like they will continue to use traditional DOM elements for some of the editing UI (just not the actual WYSIWYG editing area) and for screen readers.
kroltan 5y ago
> [...] rendering to canvas allows programmers to have pixel perfect control over the look of their applications across all devices. Canvas-based fingerprinting due to rendering differences is a thing, so using the canvas is not pixel-perfect either. Creating an UI library atop of that is a lot of work, though to be fair certainly manageable by Google. Remember, a UI library is not just about putting things on the screen, but sanely defining layouts, interactions, accessibility... > Specifically, lots of specialized applications have UI components which do not have a corresponding HTML standard I think that is the entire point behind Web Components [0], and if one really really don't want more DOM elements for their visuals, then the CSS Paint API and in fact the whole Houdini initiative [1] should be pursued instead, at least at the Google scale. Besides these technical points, interoperability should be considered. Web browsers do a lot of work to match user expectations in behaviour to their native operating systems, as well as web conventions. (The classic example infractions being: links that cannot be control-clicked or middle-clicked to open on new tabs because they are not actual links but elements with click handlers; or not being able to scroll with page up/down, arrows, or middle-click, because the page reimplements scrolling in an unsemantic way) Making your own UI toolkit is bound to all those problems, and those are user-facing problems that will affect the often-ignored long tail of users with unconventional setups. Look at Flutter for Web, for example [2], it definitely feels entirely different from a regular website, even if it were to look the same. The scrolling does not respect my system settings, interaction is limited to only the most-common method, image scaling is subtly different. And as Google observed themselves, extensibility and the user-agent should be considered before making such a decision, but it appears they consider it a liability in detriment of the user. [0]: https://developer.mozilla.org/en-US/docs/Web/Web_Components https://developer.mozilla.org/en-US/docs/Web/Web_Components [1]: https://developer.mozilla.org/en-US/docs/Web/Houdini https://developer.mozilla.org/en-US/docs/Web/Houdini [2]: https://gallery.flutter.dev/ https://gallery.flutter.dev/
- ______- 5y ago
  > Canvas-based fingerprinting due to rendering differences is a thing, so using the canvas is not pixel-perfect either Luckily Firefox asks you if you want to use the HTML5 canvas API. You can specifically whitelist some pages to use the canvas API and stop it from running by default on all your other browsing. Also: you may want a dedicated browser just for Google's whole ecosystem so they can't track you across the web. I have a Chromebook just for that, which has its own unique canvas fingerprint completely separated from my main workstation PC's fingerprint.
deckard1 5y ago
Canvas still sucks for rendering fonts. Getting accurate font metrics is still hacky[1] and more advanced methods are still experimental[2]. Though I'm sure with Google Docs moving to canvas, some of this will be expedited. But it's yet another one of those things about web tech that anyone would half a brain would tell you should have been there day one. [1] https://stackoverflow.com/questions/1134586/how-can-you-find-the-height-of-text-on-an-html-canvas https://stackoverflow.com/questions/1134586/how-can-you-find... [2] https://developer.mozilla.org/en-US/docs/Web/API/TextMetrics https://developer.mozilla.org/en-US/docs/Web/API/TextMetrics

remram 5y ago

Do your users care about your app looking the same or do they care about their browser looking and acting like a browser? Moving to canvas is sure to break many features such as text selection, adblocking, and accessibility. All in the name of more controls over the pixels? Are you truly doing it for the users?

kroltan 5y ago
Very good point, users do not care that a padding is rounded up or down when they switch from their laptop to their desktop, as long as the application is usable, understandable and visually competent. But mindspace is much more important for interaction, so if their laptop is a Mac, their brain will be in "Mac mode", and "Linux" or "Windows" mode when on their other device. Respecting the platform's conventions will allow them to keep their cognitive load due to "fiddling" to a minimum.
- remram 5y ago
  I don't think the GP was talking about the app looking consistent with the rest of the platform, but the exact opposite: the app looking the same whatever the platform. Using Flutter allows them to have the app look the same on the browser as on mobile. This means that at best it will look "Mac mode" for everyone, including Windows users; at worst it will look foreign to everyone.
  - kroltan 5y ago
    Yes, I understand and agree. My second paragraph was maybe a bit unclear, I meant that the user would expect platform conventions to be respected over application conventions. (especially if we consider platforms with different primary interactions)
  - BlueTemplar 5y ago
    Having a program look the same on a big screen with mouse+keyboard input as well as on a small touchscreen is a recipe for having a bad user experience on both.
tshaddox 5y ago
I might be wrong, but I'm pretty sure Google Docs is already using a completely custom implementation of word wrapping, text layout, text selection, cursor placement, etc. It's not like it's just a <textarea /> with some CSS styles. Likewise, they seem to already have completely separate DOM elements that are invisible to normal browsers but can be read by screen readers. Based on the DOM on the new read-only preview document they link to, it looks like they will continue to use traditional DOM elements for some of the editing UI (just not the actual WYSIWYG editing area) and for screen readers.

kroltan 5y ago

> [...] rendering to canvas allows programmers to have pixel perfect control over the look of their applications across all devices. Canvas-based fingerprinting due to rendering differences is a thing, so using the canvas is not pixel-perfect either. Creating an UI library atop of that is a lot of work, though to be fair certainly manageable by Google. Remember, a UI library is not just about putting things on the screen, but sanely defining layouts, interactions, accessibility... > Specifically, lots of specialized applications have UI components which do not have a corresponding HTML standard I think that is the entire point behind Web Components [0], and if one really really don't want more DOM elements for their visuals, then the CSS Paint API and in fact the whole Houdini initiative [1] should be pursued instead, at least at the Google scale. Besides these technical points, interoperability should be considered. Web browsers do a lot of work to match user expectations in behaviour to their native operating systems, as well as web conventions. (The classic example infractions being: links that cannot be control-clicked or middle-clicked to open on new tabs because they are not actual links but elements with click handlers; or not being able to scroll with page up/down, arrows, or middle-click, because the page reimplements scrolling in an unsemantic way) Making your own UI toolkit is bound to all those problems, and those are user-facing problems that will affect the often-ignored long tail of users with unconventional setups. Look at Flutter for Web, for example [2], it definitely feels entirely different from a regular website, even if it were to look the same. The scrolling does not respect my system settings, interaction is limited to only the most-common method, image scaling is subtly different. And as Google observed themselves, extensibility and the user-agent should be considered before making such a decision, but it appears they consider it a liability in detriment of the user. [0]: https://developer.mozilla.org/en-US/docs/Web/Web_Components https://developer.mozilla.org/en-US/docs/Web/Web_Components [1]: https://developer.mozilla.org/en-US/docs/Web/Houdini https://developer.mozilla.org/en-US/docs/Web/Houdini [2]: https://gallery.flutter.dev/ https://gallery.flutter.dev/

______- 5y ago
> Canvas-based fingerprinting due to rendering differences is a thing, so using the canvas is not pixel-perfect either Luckily Firefox asks you if you want to use the HTML5 canvas API. You can specifically whitelist some pages to use the canvas API and stop it from running by default on all your other browsing. Also: you may want a dedicated browser just for Google's whole ecosystem so they can't track you across the web. I have a Chromebook just for that, which has its own unique canvas fingerprint completely separated from my main workstation PC's fingerprint.

deckard1 5y ago

Canvas still sucks for rendering fonts. Getting accurate font metrics is still hacky[1] and more advanced methods are still experimental[2]. Though I'm sure with Google Docs moving to canvas, some of this will be expedited. But it's yet another one of those things about web tech that anyone would half a brain would tell you should have been there day one. [1] https://stackoverflow.com/questions/1134586/how-can-you-find-the-height-of-text-on-an-html-canvas https://stackoverflow.com/questions/1134586/how-can-you-find... [2] https://developer.mozilla.org/en-US/docs/Web/API/TextMetrics https://developer.mozilla.org/en-US/docs/Web/API/TextMetrics