10 ms·

From SVG to Canvas – A new way of building interactions

139 points by kretaceous 3y ago

marekhozak10 3y ago
There are of course arguments for and against SVG and Canvas. Yesterday, we posted a detailed comparison of these technologies and outlined why we think SVG is the better choice: https://www.jointjs.com/blog/svg-versus-canvas https://www.jointjs.com/blog/svg-versus-canvas
deterministic 3y ago
Frameworks are great if your application fits perfectly into the subset of applications that the framework was designed for. However the moment your applications grows beyond the design boundaries of the framework, you might end up in a situation where you have to drastically rewrite the app or even do a complete rewrite. Which is why I have always been more productive long term using my own frameworks. It takes longer to get started, but once you get the train going, nothing stops it.
zagrebian 3y ago
What’s with all the <p-inline> tags in the text?
- lelandfe 3y ago
  I just see span.p-inline elements, which appear to be their way of representing inline code (the <code> element would be a better choice, but meh, not a big deal)
nawgz 3y ago
I'm interested in Canvas - I write a lot of data viz in SVG. I find it to have lots of warts, but provide lots of value. I'm interested by this article and wish to challenge it too, here's my thoughts. Maybe I'm an idiot though - where is this library? I can't find it on their GitHub or linked from the articles > In the SVG world, we use CSS z-index to place the DragBox at the back and the SelectionFrame at the front. Not a great start IMO - SVGs don't support z-index. The front-to-back ordering z-index influences in HTML is decided in SVG directly by the order in the HTML within the <svg> element (in reverse - later elements are "on top"). What they could mean to say, if this isn't entirely inaccurate, is that "we render multiple SVGs and arrange them with z-index", which... I also find strange. I don't want to read their source code but I also don't find this to be giving them huge credibility. > Performance > Having every element produce a whole load of DOM elements for React to manage becomes a problem for performance when every element attaches event handlers, has to manage their lifecycles, observe state, etc., etc. My concerns are mounting. This is fundamentally the problem React solves, not one it creates. The amount of layers they picture in their example is not huge. I had the target to get 60fps on a Graph with 1000 nodes + smart edge routing, and you can do lots of interactions in the graph - each node can be moved independently or in a group, selected, you can drag a background box to select objects geometrically intersecting, and you can create new connections with drag, you know - basics. Not sure how performance is impacting them at an order of magnitude less, unless they are struggling with React itself. > Interactivity Limitations > with the text taking up so much space it’s very easy to accidentally drag the text instead > This was a very longstanding bug I interpret this as them saying "the text background was not displayed to users, so while it appeared the mouse was over the polygon, it was actually over the text background, therefore the text was dragged". Seems to me hover state would give the user the hints they need here, but I look forward to the Canvas solution > Maintainability Once again I really can't help but interpret this as a weak handle on React. Onto the solution. > The new system: per-state InteractionHandlers Honestly, the fact they had to leave React to manage this once again speaks to bad React practices. Have they never heard of Context/Provider? > The general idea is this: there are a lot of different things you can do in Felt, but at any given time, the number of things you can do is limited. If we can therefore define which interactions are active at any given time, and also manage the flow of events through the handlers we should be in a good place Yes, this is good architecting. However, the only thing they describe here that piques my interest is this: > Here is the full sequence of events that an InteractionHandler can process, with the ability to break the chain at any point, and prevent other handlers in the stack from being called. > <graph showing how mouse events are managed manually, enabling mouse movement to correlate to available mouse events> This is a big pain point in React - very difficult to manage onMouseDown separately from onClick. Usually I accomplished this by overloading onMouseDown and onMouseUp to call the actual dev-user-available action handlers, but their described result is more graceful. > Goal 1: Interactions should be decoupled from each other > Each interaction handler has no knowledge of any other. The knowledge about how to prevent “collisions” in interactions resides with the map of handlers and the manager. Right, but why couldn't this be done in React? > Goal 2: Interactions should be performant > Because there is only one handler per feature as opposed to a bunch of handlers per element, there is a lot less allocation of resources. Sounds and looks cool, is the implication that there is only O(feature count) event listeners for the Canvas? I'll have to profile my library for this > Goal 3: Interactions should be decoupled from access control > Access control is now simply a case of writing a different list of interaction handlers for each access level. There’s no longer any isEditor code littered throughout > Goal 4: Interactions should be decoupled from rendering Yikes. I think they really conflated bad architecture with React problems a lot in this article. Alright though, I see some clear wins at the end: > Goal 5: Complex interactions should be achievable > <video shows that text is interacted with not as a bounding rect but character polygons> This is a win, and one they obviously wanted. It's probably true this isn't possible with the DOM, unless you render each character individually as a polygon or something that would undoubtedly be insanely painful > Doing this with an element-centric approach is nigh-on impossible. There’s no way (as far as we know) to say to a DOM element: “receive clicks, but pass-through mousedowns and drags.” I agree with this claim, but wish they would elaborate what these use cases look like. > OK, there are some hidden details there such as ... > we maintain a spatial index of our elements using rbush which we query for coarse intersections with the cursor before performing more accurate hit tests using Turf.js. Right, this all sounds quite graceful, indeed it sounds a bit like implementing a graphics system. It sounds like it met some of their goals too, and enabled some long-desired features the DOM didn't make possible. But their criticisms of React & SVG are really weak here, and it sounds to me like they never properly decided how the system should behave and what parts should interact in the DOM world, and their issues fell out of that. I also think that you should really not talk so much about z-index with SVG unless you make it more clear how those things could even be related. Overall, I think they had a great team that implemented a great solution that aligned with their end-goals, but the work they replaced doesn't sound like it was even well-aligned with the framework it was implemented in, let alone their end goals. I would not agree with their recommendation to move to Canvas if SVG doesn't "feel right" until you do more analysis of why your solution is failing and what options are available.
- mch82 3y ago
  Nothing wrong with a long comment, but it might facilitate discussion to break your long comment up into a few top level comments.
  - nawgz 3y ago
    Possibly true; I think it would also be reasonable to pick some part of this if it interests you and reply to just that, nothing wrong with that.
- hicksyfern 3y ago
  Author here. It might be worth reading part 1, also. Aside from that, yes z-index was because there are multiple SVGs rendered. We also had non-SVG stuff for each element like inputs and stuff, plus it was just like that when I got here, and it doesn’t seem unreasonable to me. Another thing was you said it’s a problem React solves rather than creates. This is very clearly not true. If you add event listeners to every component which is big-standard React, you end up with 1000s or tens of 1000s of event handlers often doing almost nothing. And sure, JavaScript can do 10_000 lots of almost nothing very quickly, but it’s noticeable when you’ve got a strict frame budget.
thomasfromcdnjs 3y ago
I've built diagram like tools several times, constantly umming and ahing between Canvas and SVG. SVG seems to get you pretty far, but I always end up with Canvas in the long run. Fun fact: Google docs is actually a canvas.
- thrashh 3y ago
  Same. Working around the issues with SVG sometimes takes as long as rewriting parts of SVG from scratch. It’s not an SVG problem so much as Canvas and SVG are not reconciled into one bigger rendering system, so when you hit an edge case that requires breaking outside out of SVG, you have no path.
- alixanderwang 3y ago
  i'm building a diagramming tool (https://terrastruct.com https://terrastruct.com) and still umm and ah b/t it. I'm kind of waiting for webgpu to reach 95+% browser support before switching though. If I do all the work of switching, SVG better not get gpu/hardware-acceleration support the next day
  - Solvency 3y ago
    I'm confused... Figma can seemingly do all of this already, from a rendering perspective. And it's outrageously performant. It's one of the most performant web apps I've ever seen. Why don't you do exactly what Figma is doing?
    - avmich 3y ago
      A long time ago a small startup used unusual tooling in the industry. The startup was small, but the tooling had some great qualities, which allowed the startup to compete with bigger teams feature for feature, spending less efforts to produce the same functionality as their industry peers. Eventually the startup had a successful exit, the founders become rich, and one of them even started sharing his experience in a rather systematic manner. Happy end. Is it so that SVG requires to use all the features of it, or is it possible that you can mix and match, choosing what's important now and what's not, while benefiting from already existing SVG features? Is it possible that SVG followers don't do something which SVG does for them? Aren't there features, sufficiently aligned with SVG, which outperform directly working with other layers?
marcus_holmes 3y ago
> These InteractionHandlers are just objects satisfying a specific TypeScript interface. They’re not coupled to React or hooks or anything like that, which feels quite freeing, honestly. There’s no worrying about stale closures or dependencies or hooks rules - it really is just TypeScript. This is a great example of "fighting the framework". React + SVG got them from nothing to a working product. But then they had to rewrite completely to solve problems in the product because the framework no longer fit the product. I've seen this so many times. But it's hard for people to believe when they start out and choose the "perfect" framework for their product: "you will eventually have to rewrite your entire product because that's easier than fighting the framework to make it do what you need" "no, we won't, this $framework is flexible enough to do whatever we need" "OK, good luck" 2 years later: "we're rewriting the product completely because $framework wouldn't let us do $thing" Not that I'm advocating for not using frameworks, just for people to realise that eventually (almost) everyone fights the framework and rewrites. And the more the framework does, the earlier the fight begins.
- djbusby 3y ago
  Or, your stuck doing version upgrades of the framework before even getting to the fight.
- klysm 3y ago
  It's unfortunate that coevolution like this usually lands in a very suboptimal start. If we tore it all down and started from what we knew we'd have much better things. Occasionally stuff like that happens, but it's really hard to fight against the momentum of decades of shit
- dirkc 3y ago
  I've also seen the other side where the argument is: "we should be doing things correctly from the start" 2 years later: "the maintenance of custom doesn't deliver business value and makes recruitment and on-boarding increasingly difficult. We should rewrite in $framework since it's well known, maintained and flexible enough to do whatever they need" ...
  - 3y ago
dekhn 3y ago
I gave up on interactive applications like this using web tech- Qt has the QGraphicsView framework which is far and away the best implementation I've seen so far in terms of programmability. There is a (significant) learning curve, but basically you're just learning how modern windowing systems work (and once you do, the browser model starts to seem awfully clumsy). One of the crazy bits about QGraphicsView is that it provides not only simple primitives and the ability to make arbitrarily nested primitives, you can also render SVG, *and any Qt widget including a web browser) into the space.
talkingtab 3y ago
React three fiber and React three drei? Not sure what you are trying to do, but I have been super impressed by what you can do with react-three-etc. Video textures, images, click through works well. And when you need web thingies - input boxes for example - adding in a html is simple. Built on top of the outstanding threejs library. Examples of what you can do: https://docs.pmnd.rs/react-three-fiber/getting-started/examples https://docs.pmnd.rs/react-three-fiber/getting-started/examp... of what you can do. This one: https://codesandbox.io/s/drei-reflector-bfplr https://codesandbox.io/s/drei-reflector-bfplr is pretty cool and not much code. After a few weeks with r3f I am trying to figure out whether I can just use a very small amount of css and react. I integrated webrtc video chat as a texture pretty easily. Surely there must be some gotcha, but I have not found it yet.
simlevesque 3y ago
ctrl+f "accessibility" in part 2: zero results ctrl+f "accessibility" in part 1: zero results Does anyone care about digital access to content for disabled people ? Or about the ADA ? Svg to canvas without any answer for accessibility is backwardness, plain and simple. You are letting down anyone who's deaf, blind or has trouble hearing or seeing, anyone who does not know how to read, has motor impairments, temporary or not and the older folks. Also screen translaters will not work on canvas. Please follow the WCAG guidelines.
- Rapzid 3y ago
  A somewhat common approach is to maintain a parallel DOM structure for accessibility and keep it in sync with the data structures driving the canvas view.
- tadfisher 3y ago
  Genuinely asking, how do you make something like their map view accessible with so many objects and layers? I'd imagine you end up with a solution that exposes various content rectangles and ignores everything else, kind of how Google Maps does it.
  - mhoad 3y ago
    This is a problem that turns up in Flutter when they compile for the web. The long term answer is the Accessibility Object Model spec currently under development. The current answer is to build a replica DOM structure for accessibility purposes.
  - ygra 3y ago
    Not all of accessibility is for those who cannot see (well). WCAG also has a bunch of guidelines for loss of dexterity and other ailments: • keyboard support (being able to navigate to all controls as needed, not being stuck in a navigation island from which there is no escape with the keyboard, etc.) • pointing device considerations (not requiring drag unless absolutely essential, no action on mouse-down, etc.) • not requiring the simultaneous use of mouse and keyboard (e.g. via modifier keys) • sufficient size for buttons and other pointer targets • the ability to suppress animations, flashing images, etc. • the ability to control colors and contrast of things on the screen ... Of course, something like Photoshop, Illustrator, or the visual part of Google Maps is hard to use when you cannot see, but there are so many different disabilities that some or even most accessibility guidelines can apply to pretty much any software.
jgalt212 3y ago
Has anyone tried using canvas to stymie scrapers?
- mkl 3y ago
  Won't stop anyone determined - OCR is pretty easy. Better would be access control and rate limiting, but it's still a fundamentally unsolvable problem, as if it's visible it's scrapable.
wly_cdgr 3y ago
Canvas is a really nice vector API, very underappreciated precisely because of its elegant simplicity. Its problem is that, because it's so straightforward and well-designed, people can't easily use having had to work with it to pretend they are Big Brain.
robocat 3y ago
It sounds like most of their problems were with using the browser SVG event handlers (or perhaps the framework). They could have written their own event system for SVG (e.g. using event handlers on document), which would have fixed the gripes they blamed on SVG events. They wrote their own event system for canvas, so probably not much more difficulty in coding work? Performance is the remaining issue that isn’t clear that SVG would meet. I assume SVG painting could be as fast as canvas painting. I am assuming if you attach no events to SVG elements, you remove the event performance issues (well, replace them with performance issues in your own custom event handlers, but no worse than canvas). Surely removing all the now unneeded SVG-grab-handle-elements (“DragBox”) that were only there to capture events would also improve performance (painting and event capture). Canvas is using imperative drawing primitives, and SVG can use composable primitives - both can be rendered by GPU. Canvas updates (eg. during dragging) require heavier imperative updates, while SVG primitives can theoretically be offloaded to the GPU. I can’t guess performance differences and performance delta probably strongly depends on browser, driver, and GPU. Assuming you write your own event handling stack: > the mouse only hits SVG elements inside their painted region Fixed (as good as canvas) > we could use things like pointer-events to disable certain interactions at certain times Unnecessary (plus lowers stress on rendering?) > Having every element produce a whole load of DOM elements for React to manage becomes a problem for performance when every element attaches event handlers, has to manage their lifecycles, observe state, etc., etc. Fixed with document events > Interactivity limitations. With our SVG renderer, each element managed its own interactions, which made it impossible to drag elements that were underneath other elements. This is a surprisingly common when making complex maps. Fixed with document events > Maintainability. However, with every element attaching its own interactions, there is a tendency for every interaction to know about every other, checking if it should be enabled or not. Fixed with document events > Another maintainability issue is the proliferation of event.stopPropagation() calls in the code. These are to stop events bubbling up through the complex DOM structure and allowing other components to handle events. Fixed with document events (equivalent difficulty to canvas). When writing complex code you learn that stopPropagation() is evil. Writing code to avoid it requires skills. IMHO stopPropagation should never be in any complex codebase (or perhaps with rules to prevent errors). > We also have a few places where we have awkward setTimeout() calls Oh my god, no! Terrible hack with terrible consequences, as they discovered. setTimeout() introduces very evil race-conditions, and testability issues. An alternative which isn’t much better is to use async to get microtasks, so at least you are not reordering events/tasks. IMHO in any modern codebase setTimeout should be banned or extremely restricted when it is absolutely needed. SUMMARY Basically, it sounds like the team needs some more experienced GUI developers. I would bet good money they are trading one set of problems for another, and ending up with a codebase that is less maintainable. I can only hope the article is not written by the lead developer - heaven help them if it were. Opinions are my 10c as a custom framework developer (admittedly only one product that was not as complex as a GUI editor - that is why the inexpert mistakes sound so painful to me - although I could also avoid and detect problems because it was my own framework: depending on a third party framework makes it a heap harder to fix systematic faults).
_virtu 3y ago
I’ve been building a kinopio.club clone in Phoenix LiveView and the problems the author faces align with my experience from working with uis that require explicitly setting element position for the document. Having to manage all of the imperative handler logic was a huge pain on the JS side. I had also given myself the limitation to not use React for more complex Phoenix Hooks. It’s all native document rendering with the only dependency being x-state. Moving the mouse event handler logic out into a state machine was where I found myself going in comparison to this author’s approach. I was pretty tickled by how much more maintainable it had been than the first time I had implemented it. x-state was by no means a requirement but it had a handful of features I didn’t feel like building for the current project. I can absolutely believe that this is worth the effort, especially considering the fact that the team works with maps so much. The mental model presented in the article would be far easier to grok when working in the codebase than having to deal with the somewhat tricky to remember Event api provided by browsers for these types of use cases. Others are mentioning fighting the framework but this is still a pain without React. But of course adding in React on top of all of this would make it harder.
- hicksyfern 3y ago
  Author here. Absolutely right what you said about being easier to grok. Shortly after this was merged to main (though under feature flag) another dev had to fix something and commented something like “I just fixed X in two minutes which would have taken me hours before”
  - hicksyfern 3y ago
    And since you mention state machines: most of the handlers are done as state machines. Plus the overall app state is done as a kind of reverse state machine where we derive the status from various app states. I would like to change this to not be backwards one day but it was too big of a lift to change everything at once!
deleted 3y ago
[deleted]
hollow-moe 3y ago
Ah yes, people blocking canvas fingerprinting will also love your blank pages, but as they're a minority of users it's not a concern
- dingledork69 3y ago
  If you choose to disable features in your browser then they won't work. Is this surprising to you?
mohsen1 3y ago
6 years ago I came to the same conclusion https://azimi.me/2016/12/27/canvas-rendering-vs-react-and-svg.html https://azimi.me/2016/12/27/canvas-rendering-vs-react-and-sv...
CornCobs 3y ago
> These InteractionHandlers are just objects satisfying a specific TypeScript interface. They’re not coupled to React or hooks or anything like that, which feels quite freeing, honestly. There’s no worrying about stale closures or dependencies or hooks rules - it really is just TypeScript. > None of the interactions depend on a particular rendering system. We can actually use the DOM as a render target with all its pointer events disabled, and use this interaction system on top, which is quite neat! In general, I think this is the most important takeaway here. Too often, we couple all our frontend logic to the rendering framework (especially state) when it has no business living in React and being tied to the component lifecycles. I've had really good experience with MobX for React - separating out all our state into independent observable objects that React simply reacts to. This really cleans up a lot of logic and makes the presentation layer very clean. In particular, a pattern that I believe in is simply exposing singleton, global stores for various parts of my application - effectively treating them as APIs that the rest of the app can subscribe to.
rektide 3y ago
Super late to this but they mention element overlap in SVG making thinks like drag & drop hard. That's their "interactivity" gripe, and the one that for me carried the most weight. I'd be very interested to get more discussion on this specific issue, of handling mouse events well in SVG. The particular example felt contrived, in that I might just apply a class HasPointerEvents to every element and change the pointer events off for everyone except the drag & the dropped element. But I suspect there may be some other real mouse handling topics in SVG that really are interesting & challenging, & would love some discussion on that. Kind of iwld that the internet doesn't have a ton of obvious forums to go hash out tech topics like this... Post a random question/challenge to stackoverflow & cross your fingers.
- hicksyfern 3y ago
  Author here. The example given is slightly contrived. A more realistic example involves large polygons that you zoom “inside” or locked elements overlaying active ones. But I had to try to give an example without a ton of context. There are some more interesting but less easy to explain to a broad audience examples. One is having “nearest the pointer” hovering logic rather than rectangles or circles overlaying each other. Another is hovering line segments and the projecting the nearest hovered point onto the line. That was approximated with SVGs before and was just kind of a bug. Another example is allowing some “wiggle room” for touching elements without having to render SVGs multiple times with transparent elements and stuff like that.
  - rektide 3y ago
    Circling back just to say this was really fun & interesting stuff. Appreciate you responding with these cases!

139 points by kretaceous 3y ago

There are of course arguments for and against SVG and Canvas. Yesterday, we posted a detailed comparison of these technologies and outlined why we think SVG is the better choice: https://www.jointjs.com/blog/svg-versus-canvas https://www.jointjs.com/blog/svg-versus-canvas

deterministic 3y ago

Frameworks are great if your application fits perfectly into the subset of applications that the framework was designed for. However the moment your applications grows beyond the design boundaries of the framework, you might end up in a situation where you have to drastically rewrite the app or even do a complete rewrite. Which is why I have always been more productive long term using my own frameworks. It takes longer to get started, but once you get the train going, nothing stops it.

zagrebian 3y ago

What’s with all the <p-inline> tags in the text?

lelandfe 3y ago
I just see span.p-inline elements, which appear to be their way of representing inline code (the <code> element would be a better choice, but meh, not a big deal)

nawgz 3y ago

I'm interested in Canvas - I write a lot of data viz in SVG. I find it to have lots of warts, but provide lots of value. I'm interested by this article and wish to challenge it too, here's my thoughts. Maybe I'm an idiot though - where is this library? I can't find it on their GitHub or linked from the articles > In the SVG world, we use CSS z-index to place the DragBox at the back and the SelectionFrame at the front. Not a great start IMO - SVGs don't support z-index. The front-to-back ordering z-index influences in HTML is decided in SVG directly by the order in the HTML within the <svg> element (in reverse - later elements are "on top"). What they could mean to say, if this isn't entirely inaccurate, is that "we render multiple SVGs and arrange them with z-index", which... I also find strange. I don't want to read their source code but I also don't find this to be giving them huge credibility. > Performance > Having every element produce a whole load of DOM elements for React to manage becomes a problem for performance when every element attaches event handlers, has to manage their lifecycles, observe state, etc., etc. My concerns are mounting. This is fundamentally the problem React solves, not one it creates. The amount of layers they picture in their example is not huge. I had the target to get 60fps on a Graph with 1000 nodes + smart edge routing, and you can do lots of interactions in the graph - each node can be moved independently or in a group, selected, you can drag a background box to select objects geometrically intersecting, and you can create new connections with drag, you know - basics. Not sure how performance is impacting them at an order of magnitude less, unless they are struggling with React itself. > Interactivity Limitations > with the text taking up so much space it’s very easy to accidentally drag the text instead > This was a very longstanding bug I interpret this as them saying "the text background was not displayed to users, so while it appeared the mouse was over the polygon, it was actually over the text background, therefore the text was dragged". Seems to me hover state would give the user the hints they need here, but I look forward to the Canvas solution > Maintainability Once again I really can't help but interpret this as a weak handle on React. Onto the solution. > The new system: per-state InteractionHandlers Honestly, the fact they had to leave React to manage this once again speaks to bad React practices. Have they never heard of Context/Provider? > The general idea is this: there are a lot of different things you can do in Felt, but at any given time, the number of things you can do is limited. If we can therefore define which interactions are active at any given time, and also manage the flow of events through the handlers we should be in a good place Yes, this is good architecting. However, the only thing they describe here that piques my interest is this: > Here is the full sequence of events that an InteractionHandler can process, with the ability to break the chain at any point, and prevent other handlers in the stack from being called. > <graph showing how mouse events are managed manually, enabling mouse movement to correlate to available mouse events> This is a big pain point in React - very difficult to manage onMouseDown separately from onClick. Usually I accomplished this by overloading onMouseDown and onMouseUp to call the actual dev-user-available action handlers, but their described result is more graceful. > Goal 1: Interactions should be decoupled from each other > Each interaction handler has no knowledge of any other. The knowledge about how to prevent “collisions” in interactions resides with the map of handlers and the manager. Right, but why couldn't this be done in React? > Goal 2: Interactions should be performant > Because there is only one handler per feature as opposed to a bunch of handlers per element, there is a lot less allocation of resources. Sounds and looks cool, is the implication that there is only O(feature count) event listeners for the Canvas? I'll have to profile my library for this > Goal 3: Interactions should be decoupled from access control > Access control is now simply a case of writing a different list of interaction handlers for each access level. There’s no longer any isEditor code littered throughout > Goal 4: Interactions should be decoupled from rendering Yikes. I think they really conflated bad architecture with React problems a lot in this article. Alright though, I see some clear wins at the end: > Goal 5: Complex interactions should be achievable > <video shows that text is interacted with not as a bounding rect but character polygons> This is a win, and one they obviously wanted. It's probably true this isn't possible with the DOM, unless you render each character individually as a polygon or something that would undoubtedly be insanely painful > Doing this with an element-centric approach is nigh-on impossible. There’s no way (as far as we know) to say to a DOM element: “receive clicks, but pass-through mousedowns and drags.” I agree with this claim, but wish they would elaborate what these use cases look like. > OK, there are some hidden details there such as ... > we maintain a spatial index of our elements using rbush which we query for coarse intersections with the cursor before performing more accurate hit tests using Turf.js. Right, this all sounds quite graceful, indeed it sounds a bit like implementing a graphics system. It sounds like it met some of their goals too, and enabled some long-desired features the DOM didn't make possible. But their criticisms of React & SVG are really weak here, and it sounds to me like they never properly decided how the system should behave and what parts should interact in the DOM world, and their issues fell out of that. I also think that you should really not talk so much about z-index with SVG unless you make it more clear how those things could even be related. Overall, I think they had a great team that implemented a great solution that aligned with their end-goals, but the work they replaced doesn't sound like it was even well-aligned with the framework it was implemented in, let alone their end goals. I would not agree with their recommendation to move to Canvas if SVG doesn't "feel right" until you do more analysis of why your solution is failing and what options are available.

mch82 3y ago
Nothing wrong with a long comment, but it might facilitate discussion to break your long comment up into a few top level comments.
- nawgz 3y ago
  Possibly true; I think it would also be reasonable to pick some part of this if it interests you and reply to just that, nothing wrong with that.
hicksyfern 3y ago
Author here. It might be worth reading part 1, also. Aside from that, yes z-index was because there are multiple SVGs rendered. We also had non-SVG stuff for each element like inputs and stuff, plus it was just like that when I got here, and it doesn’t seem unreasonable to me. Another thing was you said it’s a problem React solves rather than creates. This is very clearly not true. If you add event listeners to every component which is big-standard React, you end up with 1000s or tens of 1000s of event handlers often doing almost nothing. And sure, JavaScript can do 10_000 lots of almost nothing very quickly, but it’s noticeable when you’ve got a strict frame budget.

thomasfromcdnjs 3y ago

I've built diagram like tools several times, constantly umming and ahing between Canvas and SVG. SVG seems to get you pretty far, but I always end up with Canvas in the long run. Fun fact: Google docs is actually a canvas.

thrashh 3y ago
Same. Working around the issues with SVG sometimes takes as long as rewriting parts of SVG from scratch. It’s not an SVG problem so much as Canvas and SVG are not reconciled into one bigger rendering system, so when you hit an edge case that requires breaking outside out of SVG, you have no path.
alixanderwang 3y ago
i'm building a diagramming tool (https://terrastruct.com https://terrastruct.com) and still umm and ah b/t it. I'm kind of waiting for webgpu to reach 95+% browser support before switching though. If I do all the work of switching, SVG better not get gpu/hardware-acceleration support the next day
- Solvency 3y ago
  I'm confused... Figma can seemingly do all of this already, from a rendering perspective. And it's outrageously performant. It's one of the most performant web apps I've ever seen. Why don't you do exactly what Figma is doing?
  - avmich 3y ago
    A long time ago a small startup used unusual tooling in the industry. The startup was small, but the tooling had some great qualities, which allowed the startup to compete with bigger teams feature for feature, spending less efforts to produce the same functionality as their industry peers. Eventually the startup had a successful exit, the founders become rich, and one of them even started sharing his experience in a rather systematic manner. Happy end. Is it so that SVG requires to use all the features of it, or is it possible that you can mix and match, choosing what's important now and what's not, while benefiting from already existing SVG features? Is it possible that SVG followers don't do something which SVG does for them? Aren't there features, sufficiently aligned with SVG, which outperform directly working with other layers?

marcus_holmes 3y ago

> These InteractionHandlers are just objects satisfying a specific TypeScript interface. They’re not coupled to React or hooks or anything like that, which feels quite freeing, honestly. There’s no worrying about stale closures or dependencies or hooks rules - it really is just TypeScript. This is a great example of "fighting the framework". React + SVG got them from nothing to a working product. But then they had to rewrite completely to solve problems in the product because the framework no longer fit the product. I've seen this so many times. But it's hard for people to believe when they start out and choose the "perfect" framework for their product: "you will eventually have to rewrite your entire product because that's easier than fighting the framework to make it do what you need" "no, we won't, this $framework is flexible enough to do whatever we need" "OK, good luck" 2 years later: "we're rewriting the product completely because $framework wouldn't let us do $thing" Not that I'm advocating for not using frameworks, just for people to realise that eventually (almost) everyone fights the framework and rewrites. And the more the framework does, the earlier the fight begins.

djbusby 3y ago
Or, your stuck doing version upgrades of the framework before even getting to the fight.
klysm 3y ago
It's unfortunate that coevolution like this usually lands in a very suboptimal start. If we tore it all down and started from what we knew we'd have much better things. Occasionally stuff like that happens, but it's really hard to fight against the momentum of decades of shit
dirkc 3y ago
I've also seen the other side where the argument is: "we should be doing things correctly from the start" 2 years later: "the maintenance of custom doesn't deliver business value and makes recruitment and on-boarding increasingly difficult. We should rewrite in $framework since it's well known, maintained and flexible enough to do whatever they need" ...
- 3y ago

dekhn 3y ago

I gave up on interactive applications like this using web tech- Qt has the QGraphicsView framework which is far and away the best implementation I've seen so far in terms of programmability. There is a (significant) learning curve, but basically you're just learning how modern windowing systems work (and once you do, the browser model starts to seem awfully clumsy). One of the crazy bits about QGraphicsView is that it provides not only simple primitives and the ability to make arbitrarily nested primitives, you can also render SVG, *and any Qt widget including a web browser) into the space.

talkingtab 3y ago

React three fiber and React three drei? Not sure what you are trying to do, but I have been super impressed by what you can do with react-three-etc. Video textures, images, click through works well. And when you need web thingies - input boxes for example - adding in a html is simple. Built on top of the outstanding threejs library. Examples of what you can do: https://docs.pmnd.rs/react-three-fiber/getting-started/examples https://docs.pmnd.rs/react-three-fiber/getting-started/examp... of what you can do. This one: https://codesandbox.io/s/drei-reflector-bfplr https://codesandbox.io/s/drei-reflector-bfplr is pretty cool and not much code. After a few weeks with r3f I am trying to figure out whether I can just use a very small amount of css and react. I integrated webrtc video chat as a texture pretty easily. Surely there must be some gotcha, but I have not found it yet.

simlevesque 3y ago

ctrl+f "accessibility" in part 2: zero results ctrl+f "accessibility" in part 1: zero results Does anyone care about digital access to content for disabled people ? Or about the ADA ? Svg to canvas without any answer for accessibility is backwardness, plain and simple. You are letting down anyone who's deaf, blind or has trouble hearing or seeing, anyone who does not know how to read, has motor impairments, temporary or not and the older folks. Also screen translaters will not work on canvas. Please follow the WCAG guidelines.

Rapzid 3y ago
A somewhat common approach is to maintain a parallel DOM structure for accessibility and keep it in sync with the data structures driving the canvas view.
tadfisher 3y ago
Genuinely asking, how do you make something like their map view accessible with so many objects and layers? I'd imagine you end up with a solution that exposes various content rectangles and ignores everything else, kind of how Google Maps does it.
- mhoad 3y ago
  This is a problem that turns up in Flutter when they compile for the web. The long term answer is the Accessibility Object Model spec currently under development. The current answer is to build a replica DOM structure for accessibility purposes.
- ygra 3y ago
  Not all of accessibility is for those who cannot see (well). WCAG also has a bunch of guidelines for loss of dexterity and other ailments: • keyboard support (being able to navigate to all controls as needed, not being stuck in a navigation island from which there is no escape with the keyboard, etc.) • pointing device considerations (not requiring drag unless absolutely essential, no action on mouse-down, etc.) • not requiring the simultaneous use of mouse and keyboard (e.g. via modifier keys) • sufficient size for buttons and other pointer targets • the ability to suppress animations, flashing images, etc. • the ability to control colors and contrast of things on the screen ... Of course, something like Photoshop, Illustrator, or the visual part of Google Maps is hard to use when you cannot see, but there are so many different disabilities that some or even most accessibility guidelines can apply to pretty much any software.

jgalt212 3y ago

Has anyone tried using canvas to stymie scrapers?

mkl 3y ago
Won't stop anyone determined - OCR is pretty easy. Better would be access control and rate limiting, but it's still a fundamentally unsolvable problem, as if it's visible it's scrapable.

wly_cdgr 3y ago

Canvas is a really nice vector API, very underappreciated precisely because of its elegant simplicity. Its problem is that, because it's so straightforward and well-designed, people can't easily use having had to work with it to pretend they are Big Brain.

robocat 3y ago

It sounds like most of their problems were with using the browser SVG event handlers (or perhaps the framework). They could have written their own event system for SVG (e.g. using event handlers on document), which would have fixed the gripes they blamed on SVG events. They wrote their own event system for canvas, so probably not much more difficulty in coding work? Performance is the remaining issue that isn’t clear that SVG would meet. I assume SVG painting could be as fast as canvas painting. I am assuming if you attach no events to SVG elements, you remove the event performance issues (well, replace them with performance issues in your own custom event handlers, but no worse than canvas). Surely removing all the now unneeded SVG-grab-handle-elements (“DragBox”) that were only there to capture events would also improve performance (painting and event capture). Canvas is using imperative drawing primitives, and SVG can use composable primitives - both can be rendered by GPU. Canvas updates (eg. during dragging) require heavier imperative updates, while SVG primitives can theoretically be offloaded to the GPU. I can’t guess performance differences and performance delta probably strongly depends on browser, driver, and GPU. Assuming you write your own event handling stack: > the mouse only hits SVG elements inside their painted region Fixed (as good as canvas) > we could use things like pointer-events to disable certain interactions at certain times Unnecessary (plus lowers stress on rendering?) > Having every element produce a whole load of DOM elements for React to manage becomes a problem for performance when every element attaches event handlers, has to manage their lifecycles, observe state, etc., etc. Fixed with document events > Interactivity limitations. With our SVG renderer, each element managed its own interactions, which made it impossible to drag elements that were underneath other elements. This is a surprisingly common when making complex maps. Fixed with document events > Maintainability. However, with every element attaching its own interactions, there is a tendency for every interaction to know about every other, checking if it should be enabled or not. Fixed with document events > Another maintainability issue is the proliferation of event.stopPropagation() calls in the code. These are to stop events bubbling up through the complex DOM structure and allowing other components to handle events. Fixed with document events (equivalent difficulty to canvas). When writing complex code you learn that stopPropagation() is evil. Writing code to avoid it requires skills. IMHO stopPropagation should never be in any complex codebase (or perhaps with rules to prevent errors). > We also have a few places where we have awkward setTimeout() calls Oh my god, no! Terrible hack with terrible consequences, as they discovered. setTimeout() introduces very evil race-conditions, and testability issues. An alternative which isn’t much better is to use async to get microtasks, so at least you are not reordering events/tasks. IMHO in any modern codebase setTimeout should be banned or extremely restricted when it is absolutely needed. SUMMARY Basically, it sounds like the team needs some more experienced GUI developers. I would bet good money they are trading one set of problems for another, and ending up with a codebase that is less maintainable. I can only hope the article is not written by the lead developer - heaven help them if it were. Opinions are my 10c as a custom framework developer (admittedly only one product that was not as complex as a GUI editor - that is why the inexpert mistakes sound so painful to me - although I could also avoid and detect problems because it was my own framework: depending on a third party framework makes it a heap harder to fix systematic faults).

_virtu 3y ago

I’ve been building a kinopio.club clone in Phoenix LiveView and the problems the author faces align with my experience from working with uis that require explicitly setting element position for the document. Having to manage all of the imperative handler logic was a huge pain on the JS side. I had also given myself the limitation to not use React for more complex Phoenix Hooks. It’s all native document rendering with the only dependency being x-state. Moving the mouse event handler logic out into a state machine was where I found myself going in comparison to this author’s approach. I was pretty tickled by how much more maintainable it had been than the first time I had implemented it. x-state was by no means a requirement but it had a handful of features I didn’t feel like building for the current project. I can absolutely believe that this is worth the effort, especially considering the fact that the team works with maps so much. The mental model presented in the article would be far easier to grok when working in the codebase than having to deal with the somewhat tricky to remember Event api provided by browsers for these types of use cases. Others are mentioning fighting the framework but this is still a pain without React. But of course adding in React on top of all of this would make it harder.

hicksyfern 3y ago
Author here. Absolutely right what you said about being easier to grok. Shortly after this was merged to main (though under feature flag) another dev had to fix something and commented something like “I just fixed X in two minutes which would have taken me hours before”
- hicksyfern 3y ago
  And since you mention state machines: most of the handlers are done as state machines. Plus the overall app state is done as a kind of reverse state machine where we derive the status from various app states. I would like to change this to not be backwards one day but it was too big of a lift to change everything at once!

deleted 3y ago

[deleted]

hollow-moe 3y ago

Ah yes, people blocking canvas fingerprinting will also love your blank pages, but as they're a minority of users it's not a concern

dingledork69 3y ago
If you choose to disable features in your browser then they won't work. Is this surprising to you?

mohsen1 3y ago

6 years ago I came to the same conclusion https://azimi.me/2016/12/27/canvas-rendering-vs-react-and-svg.html https://azimi.me/2016/12/27/canvas-rendering-vs-react-and-sv...

CornCobs 3y ago

> These InteractionHandlers are just objects satisfying a specific TypeScript interface. They’re not coupled to React or hooks or anything like that, which feels quite freeing, honestly. There’s no worrying about stale closures or dependencies or hooks rules - it really is just TypeScript. > None of the interactions depend on a particular rendering system. We can actually use the DOM as a render target with all its pointer events disabled, and use this interaction system on top, which is quite neat! In general, I think this is the most important takeaway here. Too often, we couple all our frontend logic to the rendering framework (especially state) when it has no business living in React and being tied to the component lifecycles. I've had really good experience with MobX for React - separating out all our state into independent observable objects that React simply reacts to. This really cleans up a lot of logic and makes the presentation layer very clean. In particular, a pattern that I believe in is simply exposing singleton, global stores for various parts of my application - effectively treating them as APIs that the rest of the app can subscribe to.

rektide 3y ago

Super late to this but they mention element overlap in SVG making thinks like drag & drop hard. That's their "interactivity" gripe, and the one that for me carried the most weight. I'd be very interested to get more discussion on this specific issue, of handling mouse events well in SVG. The particular example felt contrived, in that I might just apply a class HasPointerEvents to every element and change the pointer events off for everyone except the drag & the dropped element. But I suspect there may be some other real mouse handling topics in SVG that really are interesting & challenging, & would love some discussion on that. Kind of iwld that the internet doesn't have a ton of obvious forums to go hash out tech topics like this... Post a random question/challenge to stackoverflow & cross your fingers.

hicksyfern 3y ago
Author here. The example given is slightly contrived. A more realistic example involves large polygons that you zoom “inside” or locked elements overlaying active ones. But I had to try to give an example without a ton of context. There are some more interesting but less easy to explain to a broad audience examples. One is having “nearest the pointer” hovering logic rather than rectangles or circles overlaying each other. Another is hovering line segments and the projecting the nearest hovered point onto the line. That was approximated with SVGs before and was just kind of a bug. Another example is allowing some “wiggle room” for touching elements without having to render SVGs multiple times with transparent elements and stuff like that.
- rektide 3y ago
  Circling back just to say this was really fun & interesting stuff. Appreciate you responding with these cases!