4 ms·

HTML, the dom, css and JS in browsers are far from simple. IMHO we’ve been tacking on more and more “application stuff” on top of a document viewer since the mi

HTML, the dom, css and JS in browsers are far from simple. IMHO we’ve been tacking on more and more “application stuff” on top of a document viewer since the mid 90s. The whole reason React and other virtual dom frameworks exist is to wrangle the original document viewer into an application platform. Yes, after all those years browsers are top notch when it comes to accessibility, but it comes with lots of additional complexity. I am hopeful that with things like WASM and Canvas we are finally starting to move away from this document-based paradigm towards more app-focused paradigms. Although there is still lots of work to be done to make approaches like this work with accessibility tools. Sometimes i think that we are deliberately not adding dom support to wasm in order to not make the same mistakes we did when trying to make html/css/js suitable for apps.

tenaciousDaniel 5y ago
That framing makes sense. I've always thought the document model was poorly suited for modern applications. I'm still not sold on canvas itself being the solution - I'd rather open up the underlying accessibility/i18n tech so that developers don't have to reinvent the wheel. But yes, otherwise agree with you.
- vbsteven 5y ago
  My ideal solution would be that we can add accessibility apis to canvas. That means we can write GUI toolkits in any language compiling to wasm using canvas apis. Those toolkits gain accessibility support through canvas. edit: some toolkits already have wasm/canvas support (like Flutter and Gtk I think). The missing link is the ability to integrate accessibility.
  - dathinab 5y ago
    Not just that. There is quite a bunch of thinks you simple can't do (in reasonable high quality) with canvas rendering without it undermining the users privacy. E.g. you can do a lot to reduce the fingerprint-ability of canvas (Google doesn't care) but doing so will lead to potentially slightly slower rendering and in turn potentially less fluid animations. You can't have links as in a website without getting access the the whole browser history. You can't take advantage of the GPU in the same way the browsers can. You always have quite a bit of size and (potentially) load time overhead (I mean just look a Pangolin UI for a good example for this). And "everyone" potentially ships their own copy of a whole UI framework (not just some layers on top of it like it's common in the web). So if there is a rendering but normally it get's fixed by you browser but now it needs to be fixed by library authors and then the fix needs to be used by the website which had a problem. I think a non document based base would be nice, but it needs to be browser native, standardized, not vendor specific and with first class accessibility. At least at the core. And let's be honest flutter is non of this. And given some decisions like making text non selecteable by default I don't see it ever coming to that point tbh.
dathinab 5y ago
I can understand you, but IMHO for most websites a document focused paradigm is just the right fit. I mean e.g. a news site is just a collection of documents in the end, blogs are just documents, so is wikipedia. Search machines are basically like external indices, which fit well as documents and even things like booking sites are in the end just fancy formulars. Sure there are a lot of more "app" like usage, and for them having a different base is very important. But I'm not the biggest fan of "turning" document use cases into apps (as many sites already try to do today), as this in my experience adds a lot of overhead and includes a massive bucket of behavioral differences which can make it very hard for some people to use the internet. If everyone would agree of how roughly apps should behave for the roughly same UX aspect it would be fine. But people don't I mean look at the massive UX divergence which slowly creeped in between Android(Stock) and iOs. Worst this is a very subtitle creep at many places and it often seems as many designer doesn't realize that what they think is innovative (and might be innovative for e.g. a iOs) user isn't necessarily innovative for a lot of other people (e.g. all the Android users). Or in other words the whole concept of what is innovative diverged between different grubs of people even if they belong in the same "culture"/country/etc.. Anyway still for the places where it does make sense having a alternative to HTML/CSS is nice but I believe it must be browser native, non vendor specific, standardized and have first class accessibility support. Non of which applies to Flutter (web) in the way it currently is.
- vbsteven 5y ago
  Yes, we definitely want similar things. Maybe it was not clear from my original reply but I’m not saying the document paradigm is bad and we should move away from it. The document paradigm has evolved for 25 years and it is the right fit for most “websites”. The problem is that we’re trying to build apps on the same base as well and that is where the mismatch is. If you look at cross platform GUI toolkits like Gtk, Qt, Flutter and Druid, what most of them have in common is that they are typically abstractions on top of a graphics rendering library (cairo, skia, canvas) with integration into platform accessibility and window APIs. That makes them easily portable to Linux, Windows, Android, iOS etc. The problem with the web browser as a target platform for these toolkits is that the accessibility support is currently coupled too tightly with dom/html/css. The graphics drawing toolkit is there (canvas) and most of the above toolkits have abstracted over it already. The missing link is standardised accessibility APIs in browsers that can be used with canvas instead of dom.

tenaciousDaniel 5y ago

That framing makes sense. I've always thought the document model was poorly suited for modern applications. I'm still not sold on canvas itself being the solution - I'd rather open up the underlying accessibility/i18n tech so that developers don't have to reinvent the wheel. But yes, otherwise agree with you.

vbsteven 5y ago
My ideal solution would be that we can add accessibility apis to canvas. That means we can write GUI toolkits in any language compiling to wasm using canvas apis. Those toolkits gain accessibility support through canvas. edit: some toolkits already have wasm/canvas support (like Flutter and Gtk I think). The missing link is the ability to integrate accessibility.
- dathinab 5y ago
  Not just that. There is quite a bunch of thinks you simple can't do (in reasonable high quality) with canvas rendering without it undermining the users privacy. E.g. you can do a lot to reduce the fingerprint-ability of canvas (Google doesn't care) but doing so will lead to potentially slightly slower rendering and in turn potentially less fluid animations. You can't have links as in a website without getting access the the whole browser history. You can't take advantage of the GPU in the same way the browsers can. You always have quite a bit of size and (potentially) load time overhead (I mean just look a Pangolin UI for a good example for this). And "everyone" potentially ships their own copy of a whole UI framework (not just some layers on top of it like it's common in the web). So if there is a rendering but normally it get's fixed by you browser but now it needs to be fixed by library authors and then the fix needs to be used by the website which had a problem. I think a non document based base would be nice, but it needs to be browser native, standardized, not vendor specific and with first class accessibility. At least at the core. And let's be honest flutter is non of this. And given some decisions like making text non selecteable by default I don't see it ever coming to that point tbh.

dathinab 5y ago

I can understand you, but IMHO for most websites a document focused paradigm is just the right fit. I mean e.g. a news site is just a collection of documents in the end, blogs are just documents, so is wikipedia. Search machines are basically like external indices, which fit well as documents and even things like booking sites are in the end just fancy formulars. Sure there are a lot of more "app" like usage, and for them having a different base is very important. But I'm not the biggest fan of "turning" document use cases into apps (as many sites already try to do today), as this in my experience adds a lot of overhead and includes a massive bucket of behavioral differences which can make it very hard for some people to use the internet. If everyone would agree of how roughly apps should behave for the roughly same UX aspect it would be fine. But people don't I mean look at the massive UX divergence which slowly creeped in between Android(Stock) and iOs. Worst this is a very subtitle creep at many places and it often seems as many designer doesn't realize that what they think is innovative (and might be innovative for e.g. a iOs) user isn't necessarily innovative for a lot of other people (e.g. all the Android users). Or in other words the whole concept of what is innovative diverged between different grubs of people even if they belong in the same "culture"/country/etc.. Anyway still for the places where it does make sense having a alternative to HTML/CSS is nice but I believe it must be browser native, non vendor specific, standardized and have first class accessibility support. Non of which applies to Flutter (web) in the way it currently is.

vbsteven 5y ago
Yes, we definitely want similar things. Maybe it was not clear from my original reply but I’m not saying the document paradigm is bad and we should move away from it. The document paradigm has evolved for 25 years and it is the right fit for most “websites”. The problem is that we’re trying to build apps on the same base as well and that is where the mismatch is. If you look at cross platform GUI toolkits like Gtk, Qt, Flutter and Druid, what most of them have in common is that they are typically abstractions on top of a graphics rendering library (cairo, skia, canvas) with integration into platform accessibility and window APIs. That makes them easily portable to Linux, Windows, Android, iOS etc. The problem with the web browser as a target platform for these toolkits is that the accessibility support is currently coupled too tightly with dom/html/css. The graphics drawing toolkit is there (canvas) and most of the above toolkits have abstracted over it already. The missing link is standardised accessibility APIs in browsers that can be used with canvas instead of dom.