6 ms·

I'd really really like to see 3d content start making it's way into the DOM. (Ideally in manners far far far far beyond showing one lone hovering item, as per A

I'd really really like to see 3d content start making it's way into the DOM. (Ideally in manners far far far far beyond showing one lone hovering item, as per Apple's <model> proposal.) Seeing Steven use his Live library to provide such an implementation should be utterly unsurprising. His past works have been so exemplary & leading-edge: mathbox, TermKit, Live itself, & others. Live was an interesting exploration of what grounds really lay underfoot the latest emerged React patterns (hooks), with lots of new capabilities, and seeing it applied to such a holy grail of an idea (3d via dom) is thrilling to see. > Live goes far beyond the usual React semantics, introducing continuations, tree reductions, captures, and more There's a ton of incredible wonkery here. As usual for Steven, he's chased incredibly far reaching frontiers, applying this hyper-advanced computing toolkit to 3d, then hunting down a series of hard problems/demos to make expident work out of. The article talks to a lot of specific frontiers, but the "general" work of the underlying Live library (and it's tour-de-force proof-of-value by creating Use.GPU out of it) is quite the distillation of computing into a consolidated, clear form: > The result is a tree of functions which is simultaneously: 1. an execution trace 2. the application state 3. a dependency graph of that state Hardly the point, but one thing I'd like a little more affirmative on, to see more clearly- where my desires & Steven's intents might be mis-aligned is- what would it look like rendering environments & spaces here? Can this "dsl for dsls" help us craft landscapes, houses, & actors inside? What do situated spaces look like in this environment, if we can pull that off? Can we represent space in the DOM?

cercatrova 4y ago
Have you seen what react-three-fiber is doing? https://docs.pmnd.rs/react-three-fiber/getting-started/examples https://docs.pmnd.rs/react-three-fiber/getting-started/examp... https://twitter.com/0xca0a https://twitter.com/0xca0a (the creator, who has some examples as well)
- rektide 4y ago
  Good example! I was trying to use A-frame a bit back, another contender. https://aframe.io/ https://aframe.io/ Somewhat ashamed to admit, I fell on my face a couple times trying to get the legacy cjs modules building with a couple toolchains & put my 3d web projects back in deep-freeze. https://github.com/aframevr/aframe/issues/3732 https://github.com/aframevr/aframe/issues/3732 Briefly, it does make me a bit sad that React isn't the real dom. I doubt very much any of these solutions make the content actually available in view-source. Which, like, 10 years ago would get you 20 lashings for bad behavior. Now-a-days? Normal, react & elsewhere. Un-page content is anti-user.
rambojazz 4y ago
Browsers are resource hogs, I'm not sure I'd want any 3D content more than is necessary. CSS alone is capable of hogging my CPU at 100%
- mdp2021 4y ago
  Not just that: few people are Steve Wittens or Farbrausch vetted, most are bewilderingly contented with the ugly and deformed. Even among professionals, and with decades long evidence. See for example the adoption of 3d in online maps, that brought from "good as expected" to "utterly unwatchable, unusable" (and it is fortunate that you can still access the past versions). And that is the big 'G', with nobody able to note, "look - and do look -, this is an unacceptable result". Normally you need good illustrators and artists: with some approaches you need much better illustrators, artists and taste. Related: a few weeks ago StackOverflow came out with a visual filter that made the page psychedelic, in the wrongest way. ...As a feature.
  - rambojazz 4y ago
    I didn't notice any psychedelic filter? Where can I see it?
- exodontist 4y ago
  Most consumer machines have a literal supercomputer sitting mostly idle as you browse the web. Css is slow because it was designed for cpus and css doesn't map to gpus very cleanly. But Webgl/webgpu enable some incredible things.
dakom 4y ago
The problem imho is bandwidth. Unless you are iq or similar, creating geometry and textures from thin-air with pure math is just not possible, so you need to load 3d models, which are much heavier than their html/png counterparts. For individual models, like if you browse sketchfab, it's not such a big deal if the artist optimized things to be "game-ready", but for full environments, it's a problem, especially as audience expectations are influenced by what they see with multi-gigabyte assets on standalone games and simulations. I don't think this is a complete showstopper, like 10-20 years ago we had the same problem with 2d games and Flash, but as long as the experience was compelling enough and the preloader wasn't completely boring, people waited, but there has been a shift towards wanting things to be instantaneous on the web (e.g. video with adaptive bitrate switching, blurry but instant > sharp but buffered) Anyway, I guess my point is I don't think it's exactly a technical limitation of programming, as much as a conflict between audience expectations and wait time
- MrLeap 4y ago
  Signed distance fields _are not that bad_ (thank you Inigo Quilez) Once you have your primitives, it becomes basic math and union, subtraction, intersection for the most part. I made this shader a few days after learning the basics -> https://twitter.com/LeapJosh/status/1494417750395179012 https://twitter.com/LeapJosh/status/1494417750395179012 If glsl ever got function pointers or similar, you'd see simple SDF DSLs and frameworks bloom overnight and many of them would be easy and awesome.
  - bluescrn 4y ago
    Raymarching ever-more-complex mathematical functions is neat if you're creating a demo out of curiosity or for art's sake. But it's a very expensive and not-very-scalable method of rendering things, certainly not a replacement for meshes of triangles.
    - wwwtyro 4y ago
      Shameless plug - I wrote sdf-csg[1] as an attempt to have that cake and eat it too. Basically, build things out of SDFs, then create meshes out of them for efficient rendering. [1] https://github.com/wwwtyro/sdf-csg https://github.com/wwwtyro/sdf-csg
baybal2 4y ago
dmitriid 4y ago
> Can we represent space in the DOM? Why would you want to represent this in DOM, of all places? It's uniquely unsuited for anything that's not a few paragraphs of text with a few images.
- rektide 4y ago
  > Why would you want to represent this in DOM, of all places? It's uniquely unsuited for anything that's not a few paragraphs of text with a few images. Your negative bias is showing. That's, just, like, your opinion man. One built around disdain & dislike, and not generally shared. There's no books that publish that opinion. You don't find that limited view abundant in trade publications. Use.GPU is here to enable 3D via the DOM, and is proof positive that this declarative setup is highly powerful. The heirarchical/acyclic-graph structure more than passingly resembles a scene-graph, which is ultra-common in 3d. Since the hierarchy is apparent, bindings all just work: > The shader snippets will end up inlined in the right places with all the right bindings, so you can just go nuts. This is just Use.GPU. It's a huge step up the vast amounts of rote work required to set up bindings in modern vulkan & 3d in general. Barely the tip of the ice-berg. As I said, Use.GPU is a tour-de-force demonstration of the value of Live, which is a much more general & competent way of representing compute on the DOM. Steven calls Live a DSL for DSLs; a small language which can keep representing more, all with common structure & form of the DOM language underneath, a much higher level base for computing that escapes much reinvention along the way. Steven touches numerous times across Live write-ups & this article about how the shape of the DOM is ultra-close to that of Lisp, another blissfully homoiconic computing representation known for it's flexible means of assembling compute. > Along the way Live has gained actual bona-fide <Quote> and <Unquote> operators, to drive this recursive <Reconcile>. This means Use.GPU now neatly sidesteps Greenspun's law by containing a complete and well-specified version of a Lisp. Score I think it's under-ambitious to stop at just Space, and that the DOM's S-exp like nature is a powerful & advanced tool for representing all manners of things. Outside of Steven's work, there's plenty of other material. Work like React-Router are extremely clear & direct means of expressing routing/addressability concerns, far cleaner than any other toolkit I've seen. I invite everyone to keep an open mind, and try to use their brain to escape the limited preconceptions they already know, & to explore openly & with active regard, looking for potential. I think the potential for the DOM is high.
czottmann 4y ago
> I'd really really like to see 3d content start making it's way into the DOM. There's a lot of overlap w/ HTML5-native game development. If you're interested in that, pay https://github.com/hmans https://github.com/hmans a visit and take a look at his projects (RenderComposer, ShaderComposer, VFXComposer etc.). Nothing short of amazing!

cercatrova 4y ago

Have you seen what react-three-fiber is doing? https://docs.pmnd.rs/react-three-fiber/getting-started/examples https://docs.pmnd.rs/react-three-fiber/getting-started/examp... https://twitter.com/0xca0a https://twitter.com/0xca0a (the creator, who has some examples as well)

rektide 4y ago
Good example! I was trying to use A-frame a bit back, another contender. https://aframe.io/ https://aframe.io/ Somewhat ashamed to admit, I fell on my face a couple times trying to get the legacy cjs modules building with a couple toolchains & put my 3d web projects back in deep-freeze. https://github.com/aframevr/aframe/issues/3732 https://github.com/aframevr/aframe/issues/3732 Briefly, it does make me a bit sad that React isn't the real dom. I doubt very much any of these solutions make the content actually available in view-source. Which, like, 10 years ago would get you 20 lashings for bad behavior. Now-a-days? Normal, react & elsewhere. Un-page content is anti-user.

rambojazz 4y ago

Browsers are resource hogs, I'm not sure I'd want any 3D content more than is necessary. CSS alone is capable of hogging my CPU at 100%

mdp2021 4y ago
Not just that: few people are Steve Wittens or Farbrausch vetted, most are bewilderingly contented with the ugly and deformed. Even among professionals, and with decades long evidence. See for example the adoption of 3d in online maps, that brought from "good as expected" to "utterly unwatchable, unusable" (and it is fortunate that you can still access the past versions). And that is the big 'G', with nobody able to note, "look - and do look -, this is an unacceptable result". Normally you need good illustrators and artists: with some approaches you need much better illustrators, artists and taste. Related: a few weeks ago StackOverflow came out with a visual filter that made the page psychedelic, in the wrongest way. ...As a feature.
- rambojazz 4y ago
  I didn't notice any psychedelic filter? Where can I see it?
exodontist 4y ago
Most consumer machines have a literal supercomputer sitting mostly idle as you browse the web. Css is slow because it was designed for cpus and css doesn't map to gpus very cleanly. But Webgl/webgpu enable some incredible things.

dakom 4y ago

The problem imho is bandwidth. Unless you are iq or similar, creating geometry and textures from thin-air with pure math is just not possible, so you need to load 3d models, which are much heavier than their html/png counterparts. For individual models, like if you browse sketchfab, it's not such a big deal if the artist optimized things to be "game-ready", but for full environments, it's a problem, especially as audience expectations are influenced by what they see with multi-gigabyte assets on standalone games and simulations. I don't think this is a complete showstopper, like 10-20 years ago we had the same problem with 2d games and Flash, but as long as the experience was compelling enough and the preloader wasn't completely boring, people waited, but there has been a shift towards wanting things to be instantaneous on the web (e.g. video with adaptive bitrate switching, blurry but instant > sharp but buffered) Anyway, I guess my point is I don't think it's exactly a technical limitation of programming, as much as a conflict between audience expectations and wait time

MrLeap 4y ago
Signed distance fields _are not that bad_ (thank you Inigo Quilez) Once you have your primitives, it becomes basic math and union, subtraction, intersection for the most part. I made this shader a few days after learning the basics -> https://twitter.com/LeapJosh/status/1494417750395179012 https://twitter.com/LeapJosh/status/1494417750395179012 If glsl ever got function pointers or similar, you'd see simple SDF DSLs and frameworks bloom overnight and many of them would be easy and awesome.
- bluescrn 4y ago
  Raymarching ever-more-complex mathematical functions is neat if you're creating a demo out of curiosity or for art's sake. But it's a very expensive and not-very-scalable method of rendering things, certainly not a replacement for meshes of triangles.
  - wwwtyro 4y ago
    Shameless plug - I wrote sdf-csg[1] as an attempt to have that cake and eat it too. Basically, build things out of SDFs, then create meshes out of them for efficient rendering. [1] https://github.com/wwwtyro/sdf-csg https://github.com/wwwtyro/sdf-csg

baybal2 4y ago

dmitriid 4y ago

> Can we represent space in the DOM? Why would you want to represent this in DOM, of all places? It's uniquely unsuited for anything that's not a few paragraphs of text with a few images.

rektide 4y ago
> Why would you want to represent this in DOM, of all places? It's uniquely unsuited for anything that's not a few paragraphs of text with a few images. Your negative bias is showing. That's, just, like, your opinion man. One built around disdain & dislike, and not generally shared. There's no books that publish that opinion. You don't find that limited view abundant in trade publications. Use.GPU is here to enable 3D via the DOM, and is proof positive that this declarative setup is highly powerful. The heirarchical/acyclic-graph structure more than passingly resembles a scene-graph, which is ultra-common in 3d. Since the hierarchy is apparent, bindings all just work: > The shader snippets will end up inlined in the right places with all the right bindings, so you can just go nuts. This is just Use.GPU. It's a huge step up the vast amounts of rote work required to set up bindings in modern vulkan & 3d in general. Barely the tip of the ice-berg. As I said, Use.GPU is a tour-de-force demonstration of the value of Live, which is a much more general & competent way of representing compute on the DOM. Steven calls Live a DSL for DSLs; a small language which can keep representing more, all with common structure & form of the DOM language underneath, a much higher level base for computing that escapes much reinvention along the way. Steven touches numerous times across Live write-ups & this article about how the shape of the DOM is ultra-close to that of Lisp, another blissfully homoiconic computing representation known for it's flexible means of assembling compute. > Along the way Live has gained actual bona-fide <Quote> and <Unquote> operators, to drive this recursive <Reconcile>. This means Use.GPU now neatly sidesteps Greenspun's law by containing a complete and well-specified version of a Lisp. Score I think it's under-ambitious to stop at just Space, and that the DOM's S-exp like nature is a powerful & advanced tool for representing all manners of things. Outside of Steven's work, there's plenty of other material. Work like React-Router are extremely clear & direct means of expressing routing/addressability concerns, far cleaner than any other toolkit I've seen. I invite everyone to keep an open mind, and try to use their brain to escape the limited preconceptions they already know, & to explore openly & with active regard, looking for potential. I think the potential for the DOM is high.

czottmann 4y ago

> I'd really really like to see 3d content start making it's way into the DOM. There's a lot of overlap w/ HTML5-native game development. If you're interested in that, pay https://github.com/hmans https://github.com/hmans a visit and take a look at his projects (RenderComposer, ShaderComposer, VFXComposer etc.). Nothing short of amazing!