5 ms·

Use.GPU Goes Trad

SeanAnderson 4y ago
The author's entire section on "Real Reactivity" hit quite close to home. I recently played around with using https://reactpixi.org/ https://reactpixi.org/ to build a simple ant farm simulation (https://meomix.github.io/antfarm/ https://meomix.github.io/antfarm/) and was left disappointed regarding rendering performance, but not surprised. It's a real challenge to get strong numbers when stacking a declarative wrapper on an imperative base. I found myself fighting React's reconciler for performance almost immediately. I was talking to my coworkers about the issue and one of them suggested trying to use react-three-fiber and just force it to render 2D, but, as the author notes, the problem feels intractable with competing layers of abstraction. I'm really excited to learn about this library. I feel that I was between a rock and a place with my web-first, declarative graphics tooling. I was pushing myself to learn Rust, to use Bevvy, to have a well-supported, declarative framework, but I felt I would prototype quicker if I stuck with JavaScript. I considered A-Frame, but it's really not about 2D rendering at all with its VR-first approach. There's really quite a desert of modern, active, declarative, web-first graphics rendering frameworks. Stoked for UseGPU to hit 1.0!
- unconed 4y ago
  it won't reach 1.0 if nobody helps tho
- mlsarecmg 4y ago
  React acknowledges that nothing can ever be purely declarative, it allows you to declare a view while giving you an outlet for imperative computations and side-effects (hooks). There is no contradiction in that. In Fiber to animate is to use useFrame, which happens to be outside of React and bears no extra cost or overhead, nor does it conflict with reactive prop updates. useFrame allows any individual component to tie its mutations into the render loop. The reactivity debate is next to irrelevant in WebGL because that is a loop driven system, it is not event-driven like the DOM. If you doubt that observe Svelte/Threlte, which can update the view directly, yet relies on useFrame modeled after Fibers. That is because you need deltas (to be refresh-rate independent), flags (needsUpdate), imperative functions (updateProjectionMatrix etc), it is never just a prop update. That said, i don't know much about React-Pixi/Konva and so on, and if these do not have a loop outlet then yes i agree. But the whole premise of that article just falls flat.
bioemerl 4y ago
I can't comment on the framework here, but that demo of the 3d pipes being drawn on the top of the page is genuinely very very neat. And you can scroll up to see it in full 3d!
- ChintanGhate 4y ago
  I remember seeing it years ago, still amazed by it. What's even cooler is that it was created almost 10 years ago and I believe it hasn't changed much since then. You can read about it [here](https://acko.net/blog/zero-to-sixty-in-one-second/ https://acko.net/blog/zero-to-sixty-in-one-second/)
- obert 4y ago
  Uh, I didn’t notice it because on iOS it’s a static image at the top of the page
  - wlesieutre 4y ago
    Works on iPad, so maybe a screen size thing?
    - funcantor 4y ago
      From the blog that another reply linked (https://acko.net/blog/zero-to-sixty-in-one-second/ https://acko.net/blog/zero-to-sixty-in-one-second/) 'Rather than try and emulate some of the bling for CSS 3D-only environments, it's all or nothing. Without WebGL, you get plain images.' Although this blog post is from 2013, so who knows if this is the actual reason. Thankfully, the blog post also links to a video so you can still view it: https://www.youtube.com/watch?v=zjwA1VmuPnw https://www.youtube.com/watch?v=zjwA1VmuPnw
      - unconed 4y ago
        10 years ago, having a showy webgl demo on a blog post, on mobile phones, would result in complaints on HN. Nowadays people complain they can't see the WebGL on their phone. I guess that's progress?
thewebcount 4y ago
> …certain properties have an invisible setter, which immediately triggers a "change" event when you assign a new value to it. I’ve hit this with SwiftUI. Isn’t this a general problem of the entire “reactive” style of programming? If you have anything that requires a computation step after model changes, you need to gate when you take into account a set of model changes before you kick off your calculations or you end up with this problem of constant updates that kill performance. I don’t do a lot of front end work, but that’s been my experience with this method of programming in general.
- esperent 4y ago
  Is it a problem with the reactive style in general, or only when you try to connect it to systems not designed with this style in mind?
  - munchbunny 4y ago
    If I’m someone who just wants to get an app out the door, then I’d argue that not being able to gracefully bridge to other paradigms when there is a gap in what is available under a reactive pattern is by itself a problem (an ecosystem problem). That’s why I use React. Despite the issues with bridging to non-reactive abstractions, it remains a useful pattern. Others may be more concerned with the applicability of the pattern itself.
    - pygy_ 4y ago
      This is not unlike integrated automatically GC'd and non-GC'd code. The gap can be bridged for granular reactive systems as it can be for React, provided the reactive system is well designed.
- jeroenhd 4y ago
  You can have reactive programming with batch state updates. For the longest time, most React code used setState to update the entirety of a component's state at once rather than set each property individually and hope the framework batches up changes fast (or you use manual state update limitation logic like debounces). These days, I see a lot of React code having switched to functional components that suffer from rapid state updates, possibly without the authors even knowing. In theory, these state updates shouldn't be a problem as your rendering logic and your state logic should be separate, at most rendering one or two frames wrong if the state updates are inconsistent. Video games have done this for over a decade and every "modern" GUI framework uses a GPU surface to paint its own controls rather than use existing system windowing, so you may as well take a page out of games' book. Sadly, I don't think many frameworks do this stuff right and a lot of time is spend doing unnecessary state recalculations. It's certainly easier to write bug free GUI code through reactive programming, as state conflict bugs now get gracefully handled by the frameworks, but I think because these issues are now no longer clearly visible they also don't get recognised as bugs anymore. I think this is just one of the many ways development has shifted towards developer comfort over user experience; changing a number in a region in memory and redrawing a string doesn't need to go through 50 chained method calls to update application state, but they make life for the devs easier so they're often tolerated. You get more features in return, but I'm not happy with the price we paid for those (mostly useless) features.
blondin 4y ago
i dislike reactjs's way of building applications. it left a bad taste in my mouth and seeing it applied to 3D rendering makes me sad. but i do appreciate author for his previous work. especially the real time math in the browser stuff. wishing him luck with this new project.
- SeanAnderson 4y ago
  What do you dislike about it?
  - recursive 4y ago
    Can't speak for the grandparent, but for me, my beef basically boils down to one thing. When you use a stateful hook, there has to be some concept of identity of the calling component. You don't want to get someone else's state back when you call useState(). However, I've never seen it documented what actual process is used to determine the identity of a calling component. In fact, it is something that I've had problem with before. Related, I don't feel comfortable giving up control of when a real DOM element is replace by a new instance. For some things, like canvas, file inputs, audio tags, this replacement needs to be carefully controlled or prevented. This seems to violate the react philosophy.
    - acemarke 4y ago
      > "not documented what the process is" It's fairly straightforward. React uses a module-scoped variable, assigns to it when it starts rendering a component, and nulls it out when it's done rendering the component. Any other function in the hooks implementation file can then update that current "Fiber" object when the hook gets called by the component. Shawn Swyx Wang has an excellent talk called "Getting Closure with React Hooks" where he builds a miniature version of this in about 20 minutes: https://www.swyx.io/hooks/ https://www.swyx.io/hooks/ You can see the actual implementation in this file - I've highlighted that "current Fiber" variable: https://github.com/facebook/react/blob/v18.0.0/packages/react-reconciler/src/ReactFiberHooks.new.js#L181-L187 https://github.com/facebook/react/blob/v18.0.0/packages/reac... As for the "when a real DOM element is replaced" bit, _you_ have control over that. React will keep existing DOM nodes in place as long as you continue to tell it "render a node of this type in this spot in the tree". It only removes that DOM node when your render logic stops telling React it should exist (ie, a component is unmounted, or the component's render output changes and no longer includes that `<div>` or `<canvas>` or whatever type). See my post "Guide to React Rendering Behavior" for some more details on this: https://blog.isquaredsoftware.com/2020/05/blogged-answers-a-mostly-complete-guide-to-react-rendering-behavior/#component-types-and-reconciliation https://blog.isquaredsoftware.com/2020/05/blogged-answers-a-...
Cixelyn 4y ago
The Live reconciler[1] included in this project looks quite interesting as a reconciler for building a canvas-based image editing app. Since WebGPU is still quite experimental and not usable without install canary browser builds, I'm really hoping that Live can be used separately. Like the author, I'm pretty sick of doing the data retained/declarative dance with libraries like pixijs many times now. At least from this article, it seems that the author is using their @use-gpu/state[2] package independently. Would love to know if anyone might have any more details or experience? [1] https://usegpu.live/docs/guides-live-vs-react https://usegpu.live/docs/guides-live-vs-react [2] https://usegpu.live/docs/reference-live-@use-gpu-state https://usegpu.live/docs/reference-live-@use-gpu-state
- unconed 4y ago
  Yes it's entirely stand alone. No dependencies.
pygy_ 4y ago
> What reactivity really does is take cache invalidation, said to be the hardest problem, and turn the problem itself into the solution. You never invalidate a cache without immediately refreshing it, and you make that the sole way to cause anything to happen at all. Crazy, and yet it works. > When I tell people this, they often say "well, it might work well for your domain, but it couldn't possibly work for mine." And then I show them how to do it. Signals-based reactivity is garbage collection on steroids. It does solve cache invalidation, and extends it to effects as well. It really should be a language feature at this point. I wish the model could be extended to distributed systems, but adding network unreliability to the mix is non-trivial.
AgentME 4y ago
This post is great. Having helped move a large codebase over to React, I love React's emphasis of one-way data flow which I've seen greatly simplify many things and help avoid whole classes of bugs. I've been looking into graphics libraries (including three.js) lately, and the retained-mode style APIs in many that end up encouraging two-way data flow feel like a large step back. I've been wanting for graphics libraries that emphasize a React-style one-way data flow. I had been looking into react-three-fiber, but I've suspected that a wrapper like it doesn't take the fullest advantage of one-way data flow. This post addresses what I've been hoping for while being a good introduction to the issue for others. >Three.js might seem like a great choice for the job: it has a 3D scene, editing controls and so on. But, my scene is not the source of truth, it's the output of a process. The actual source of truth being live-edited is another tree that sits before it. So I need to solve a two-way synchronization problem between both. This requires careful reasoning about state changes.

The author's entire section on "Real Reactivity" hit quite close to home. I recently played around with using https://reactpixi.org/ https://reactpixi.org/ to build a simple ant farm simulation (https://meomix.github.io/antfarm/ https://meomix.github.io/antfarm/) and was left disappointed regarding rendering performance, but not surprised. It's a real challenge to get strong numbers when stacking a declarative wrapper on an imperative base. I found myself fighting React's reconciler for performance almost immediately. I was talking to my coworkers about the issue and one of them suggested trying to use react-three-fiber and just force it to render 2D, but, as the author notes, the problem feels intractable with competing layers of abstraction. I'm really excited to learn about this library. I feel that I was between a rock and a place with my web-first, declarative graphics tooling. I was pushing myself to learn Rust, to use Bevvy, to have a well-supported, declarative framework, but I felt I would prototype quicker if I stuck with JavaScript. I considered A-Frame, but it's really not about 2D rendering at all with its VR-first approach. There's really quite a desert of modern, active, declarative, web-first graphics rendering frameworks. Stoked for UseGPU to hit 1.0!

unconed 4y ago
it won't reach 1.0 if nobody helps tho
mlsarecmg 4y ago
React acknowledges that nothing can ever be purely declarative, it allows you to declare a view while giving you an outlet for imperative computations and side-effects (hooks). There is no contradiction in that. In Fiber to animate is to use useFrame, which happens to be outside of React and bears no extra cost or overhead, nor does it conflict with reactive prop updates. useFrame allows any individual component to tie its mutations into the render loop. The reactivity debate is next to irrelevant in WebGL because that is a loop driven system, it is not event-driven like the DOM. If you doubt that observe Svelte/Threlte, which can update the view directly, yet relies on useFrame modeled after Fibers. That is because you need deltas (to be refresh-rate independent), flags (needsUpdate), imperative functions (updateProjectionMatrix etc), it is never just a prop update. That said, i don't know much about React-Pixi/Konva and so on, and if these do not have a loop outlet then yes i agree. But the whole premise of that article just falls flat.

bioemerl 4y ago

I can't comment on the framework here, but that demo of the 3d pipes being drawn on the top of the page is genuinely very very neat. And you can scroll up to see it in full 3d!

ChintanGhate 4y ago
I remember seeing it years ago, still amazed by it. What's even cooler is that it was created almost 10 years ago and I believe it hasn't changed much since then. You can read about it [here](https://acko.net/blog/zero-to-sixty-in-one-second/ https://acko.net/blog/zero-to-sixty-in-one-second/)
obert 4y ago
Uh, I didn’t notice it because on iOS it’s a static image at the top of the page
- wlesieutre 4y ago
  Works on iPad, so maybe a screen size thing?
  - funcantor 4y ago
    From the blog that another reply linked (https://acko.net/blog/zero-to-sixty-in-one-second/ https://acko.net/blog/zero-to-sixty-in-one-second/) 'Rather than try and emulate some of the bling for CSS 3D-only environments, it's all or nothing. Without WebGL, you get plain images.' Although this blog post is from 2013, so who knows if this is the actual reason. Thankfully, the blog post also links to a video so you can still view it: https://www.youtube.com/watch?v=zjwA1VmuPnw https://www.youtube.com/watch?v=zjwA1VmuPnw
    - unconed 4y ago
      10 years ago, having a showy webgl demo on a blog post, on mobile phones, would result in complaints on HN. Nowadays people complain they can't see the WebGL on their phone. I guess that's progress?

thewebcount 4y ago

> …certain properties have an invisible setter, which immediately triggers a "change" event when you assign a new value to it. I’ve hit this with SwiftUI. Isn’t this a general problem of the entire “reactive” style of programming? If you have anything that requires a computation step after model changes, you need to gate when you take into account a set of model changes before you kick off your calculations or you end up with this problem of constant updates that kill performance. I don’t do a lot of front end work, but that’s been my experience with this method of programming in general.

esperent 4y ago
Is it a problem with the reactive style in general, or only when you try to connect it to systems not designed with this style in mind?
- munchbunny 4y ago
  If I’m someone who just wants to get an app out the door, then I’d argue that not being able to gracefully bridge to other paradigms when there is a gap in what is available under a reactive pattern is by itself a problem (an ecosystem problem). That’s why I use React. Despite the issues with bridging to non-reactive abstractions, it remains a useful pattern. Others may be more concerned with the applicability of the pattern itself.
  - pygy_ 4y ago
    This is not unlike integrated automatically GC'd and non-GC'd code. The gap can be bridged for granular reactive systems as it can be for React, provided the reactive system is well designed.
jeroenhd 4y ago
You can have reactive programming with batch state updates. For the longest time, most React code used setState to update the entirety of a component's state at once rather than set each property individually and hope the framework batches up changes fast (or you use manual state update limitation logic like debounces). These days, I see a lot of React code having switched to functional components that suffer from rapid state updates, possibly without the authors even knowing. In theory, these state updates shouldn't be a problem as your rendering logic and your state logic should be separate, at most rendering one or two frames wrong if the state updates are inconsistent. Video games have done this for over a decade and every "modern" GUI framework uses a GPU surface to paint its own controls rather than use existing system windowing, so you may as well take a page out of games' book. Sadly, I don't think many frameworks do this stuff right and a lot of time is spend doing unnecessary state recalculations. It's certainly easier to write bug free GUI code through reactive programming, as state conflict bugs now get gracefully handled by the frameworks, but I think because these issues are now no longer clearly visible they also don't get recognised as bugs anymore. I think this is just one of the many ways development has shifted towards developer comfort over user experience; changing a number in a region in memory and redrawing a string doesn't need to go through 50 chained method calls to update application state, but they make life for the devs easier so they're often tolerated. You get more features in return, but I'm not happy with the price we paid for those (mostly useless) features.

blondin 4y ago

i dislike reactjs's way of building applications. it left a bad taste in my mouth and seeing it applied to 3D rendering makes me sad. but i do appreciate author for his previous work. especially the real time math in the browser stuff. wishing him luck with this new project.

SeanAnderson 4y ago
What do you dislike about it?
- recursive 4y ago
  Can't speak for the grandparent, but for me, my beef basically boils down to one thing. When you use a stateful hook, there has to be some concept of identity of the calling component. You don't want to get someone else's state back when you call useState(). However, I've never seen it documented what actual process is used to determine the identity of a calling component. In fact, it is something that I've had problem with before. Related, I don't feel comfortable giving up control of when a real DOM element is replace by a new instance. For some things, like canvas, file inputs, audio tags, this replacement needs to be carefully controlled or prevented. This seems to violate the react philosophy.
  - acemarke 4y ago
    > "not documented what the process is" It's fairly straightforward. React uses a module-scoped variable, assigns to it when it starts rendering a component, and nulls it out when it's done rendering the component. Any other function in the hooks implementation file can then update that current "Fiber" object when the hook gets called by the component. Shawn Swyx Wang has an excellent talk called "Getting Closure with React Hooks" where he builds a miniature version of this in about 20 minutes: https://www.swyx.io/hooks/ https://www.swyx.io/hooks/ You can see the actual implementation in this file - I've highlighted that "current Fiber" variable: https://github.com/facebook/react/blob/v18.0.0/packages/react-reconciler/src/ReactFiberHooks.new.js#L181-L187 https://github.com/facebook/react/blob/v18.0.0/packages/reac... As for the "when a real DOM element is replaced" bit, _you_ have control over that. React will keep existing DOM nodes in place as long as you continue to tell it "render a node of this type in this spot in the tree". It only removes that DOM node when your render logic stops telling React it should exist (ie, a component is unmounted, or the component's render output changes and no longer includes that `<div>` or `<canvas>` or whatever type). See my post "Guide to React Rendering Behavior" for some more details on this: https://blog.isquaredsoftware.com/2020/05/blogged-answers-a-mostly-complete-guide-to-react-rendering-behavior/#component-types-and-reconciliation https://blog.isquaredsoftware.com/2020/05/blogged-answers-a-...

Cixelyn 4y ago

The Live reconciler[1] included in this project looks quite interesting as a reconciler for building a canvas-based image editing app. Since WebGPU is still quite experimental and not usable without install canary browser builds, I'm really hoping that Live can be used separately. Like the author, I'm pretty sick of doing the data retained/declarative dance with libraries like pixijs many times now. At least from this article, it seems that the author is using their @use-gpu/state[2] package independently. Would love to know if anyone might have any more details or experience? [1] https://usegpu.live/docs/guides-live-vs-react https://usegpu.live/docs/guides-live-vs-react [2] https://usegpu.live/docs/reference-live-@use-gpu-state https://usegpu.live/docs/reference-live-@use-gpu-state

unconed 4y ago
Yes it's entirely stand alone. No dependencies.

pygy_ 4y ago

> What reactivity really does is take cache invalidation, said to be the hardest problem, and turn the problem itself into the solution. You never invalidate a cache without immediately refreshing it, and you make that the sole way to cause anything to happen at all. Crazy, and yet it works. > When I tell people this, they often say "well, it might work well for your domain, but it couldn't possibly work for mine." And then I show them how to do it. Signals-based reactivity is garbage collection on steroids. It does solve cache invalidation, and extends it to effects as well. It really should be a language feature at this point. I wish the model could be extended to distributed systems, but adding network unreliability to the mix is non-trivial.

AgentME 4y ago

This post is great. Having helped move a large codebase over to React, I love React's emphasis of one-way data flow which I've seen greatly simplify many things and help avoid whole classes of bugs. I've been looking into graphics libraries (including three.js) lately, and the retained-mode style APIs in many that end up encouraging two-way data flow feel like a large step back. I've been wanting for graphics libraries that emphasize a React-style one-way data flow. I had been looking into react-three-fiber, but I've suspected that a wrapper like it doesn't take the fullest advantage of one-way data flow. This post addresses what I've been hoping for while being a good introduction to the issue for others. >Three.js might seem like a great choice for the job: it has a 3D scene, editing controls and so on. But, my scene is not the source of truth, it's the output of a process. The actual source of truth being live-edited is another tree that sits before it. So I need to solve a two-way synchronization problem between both. This requires careful reasoning about state changes.