4 ms·

It seems pretty clear that Anthropic models have been specifically trained to be good at generating three.js (JavaScript 3-D Graphics) code, so given current st

It seems pretty clear that Anthropic models have been specifically trained to be good at generating three.js (JavaScript 3-D Graphics) code, so given current state of AI code generation in general, I don't find three.js models/animations as indicative of anything other than the model's ability to write three.js code. When Fable was first released the day-1 demos of it on Twitter (presumably from people who were given early access, and/or Anthropic employees) were pretty much 100% three.js stuff. Yes, it looks nice, but it doesn't tell me any better than an Erdos proof whether the LLM will be able to run my vending machine.

fasterik 3mo ago
Taking a single paragraph of literary text, which is abstract and ambiguous, and converting it into a 3D animation requires an enormous amount of implicit knowledge about spatial relationships, intuitive physics, everyday objects, and so forth. Not to mention the mathematics of 3D transformations and computer graphics more generally. Saying that it's indicative of no more than three.js coding ability is absurd.
- beepbooptheory 3mo ago
  Why does it require knowledge about spatial relationships?
  - fasterik 3mo ago
    Because the degree of realism in a video is determined by implicit knowledge of concepts like "in front of", "behind", "next to", "inside", "outside", "between", "occluded by", etc. as well as distances, angles, and the relative size of objects when viewed from a perspective.
    - beepbooptheory 3mo ago
      I guess I just don't have an intuition for why that is different from what it does for non-"spatial" things. Like the fact that it works is still because it produced the code it did token by token. Whether its dealing with, e.g., "beside the rock" or "in the array", it's doing the same kind of inferential activity.
      - fasterik 3mo ago
        >Whether its dealing with, e.g., "beside the rock" or "in the array", it's doing the same kind of inferential activity. Are you sure about that? Presumably the spatial concept of "inside" and the computer memory concept of "inside" have different contextual embeddings in an LLM, not so different from how our brains have different neural activation patterns when we use different concepts. Unless you think that "the same kind of inferential activity" also applies to human neurons firing.
      - emp17344 3mo ago
        You are correct. It’s important not to let hype peddlers imply AI possesses consciousness.
        fasterik 3mo ago
        [dead]
      - RugnirViking 3mo ago
        > I just don't have an intuition for why that is different from what it does for non-"spatial" things A big reason people separate this out is because it was only a short time ago that AI models were noticably and uniquely bad at this. I don't say this to be like "oh so imagine where theyll be in x amount of time", rather that this thing that was once a serious limitation of the technology is slowly being compensated for by larger and better trained models
      - hombre_fatal 3mo ago
        That’s becoming as illuminating as saying you produced that comment word by word.
        beepbooptheory 3mo ago
        What is becoming illuminating?
      - quietbritishjim 3mo ago
        If you're at (-2, 3, 5), pointing in the (2, 1, -1) direction (as a direction vector), then which is in front of the other from your point of view: the object at (4, 5, -8) or the one at (6, 4, 3)? (or neither) You can't just defer this decision to the three.js program because you need to understand this sort of relationship yourself if you're going to suitably place objects (and your own view) in a three.js scene in the first place. Building a complex scene involves making hundreds, or more, of decisions about where to place coordinates in 3d space. This example is underspecified (e.g. what are the shapes and sizes of the objects and what is the angular width of your view) but it illustrates the problem. Even with a good intuitive understanding of 3d space, we would struggle with this. LLMs are not calculators, so it's surprising if they manage it.
    - jeffybefffy519 3mo ago
      The problem i have with this argument is they still struggle to generate images of so many basic things that should be explained in their "world model" (text, fingers, reflections, faces) - so clearly they dont have a "world model" in the way that we have one. And if you use image generation capabilities, you clearly see that LLM's suck as understanding "next" to a thing.
  - yeoyeo42 3mo ago
    quite loaded words there. "knowledge", "relationships". the llm must be able to output the correct code when the input says something like 'place object x to the left of object y' vs when it says 'to the right'. there is an absurd combinatorial space of the possible inputs vs outputs it must generate -> the whole set of these, interpreted from a human point of view, you could call knowledge. the [x] in input x output you could call the relationships. And the fact that the LLM doesn't have to brute force represent all of them (impossible in the limited embedding vector x internal representation state) you could interpret as 'understanding'. But again, these are loaded words that are somewhat meaningless when looking at what an LLM does in a literal way. They have plenty graphics code to train from, so this structure will be directly or indirectly available in the training data in a very plentiful way. In literal llm transformer terms, the embeddings must have some of their components statistically represent spatial structure in some way that later in the internal layers of the LLM give some statistics of how likely it is to occur for certain code tokens to appear relative to the input of spatial wording in the token stream. it is likely that their internal representations encode something more general than specific input x output stream combinations (because we already know this is the case for regular words and concepts). if not in the embeddings, then a couple layers into the network for sure.
  - whywhywhywhy 3mo ago
    See 3D examples from earlier models and how they try to do it but things are stacked on top of the same x/y/z point or mushed together or parented wrong or disconnected. It’s something that specifically stopped happening in the smarter models. Same way the “pelican test” became worthless beyond a certain point and just some idea of “I like this one better” not technically more competent in any way.
- Voultapher 3mo ago
  *Of text it has read the full book of, read a huge corpus of text discussing it and watched the highly successful adaption of. We should feed a snippet of an unreleased book in a novel universe.
lowbloodsugar 3mo ago
Don’t worry. I’m sure they’re not training it to be good at things like writing database backends, financial services, logistics systems, user interfaces, or anything of economic value. As long as you’re not working on three.js specifically, I’m sure Anthropic isn’t making any progress you should be worried about.
- HarHarVeryFunny 3mo ago
  Do you feel that rendering a 3-D picture of Bilbo Baggins' house is a good indicator of generating economic value (i.e "AGI" in OpenAI parlance)?
abletonlive 3mo ago
[flagged]
- peterleiser 3mo ago
  But LLM's passed the Turing test, so we're done, right? ;-) Seriously, though, I agree with you that people keep moving the ball.
- Applejinx 3mo ago
  Oh, they still are, it's just that it's a rare parrot that can reproduce 'all of the written corpus of three.js' or stochastically align all that with arbitrary and weird conditions for what to squawk out. This is what you get. It's like the more general concept of starting with the written word (or 1000 words) and then replacing it with a picture. You've done something strikingly different, but is it serving the same function? It's fascinating to see this stuff combine such disparate sources in unexpected ways. But it is parrot, just not in the way you're expecting.
- tomhow 3mo ago
  The guidelines ask: Please don't fulminate. Please don't sneer, including at the rest of the community. https://news.ycombinator.com/newsguidelines.html https://news.ycombinator.com/newsguidelines.html Many people from all sections of HN, the tech industry and broader society have been surprised and wrong in all directions about how the emergence of AI is playing out. I certainly have. I can't think of a single person whose predictions have been precisely accurate. So, please don't use terms like “wall of shame” and “cope”. It doesn't help anyone make better predictions and only makes you, and HN generally, seem mean.
onion2k 3mo ago
I don't find three.js models/animations as indicative of anything other than the model's ability to write three.js code. This weekend I've been converting a game from three.js to ogl.js in order to see if I can optimise the time-to-interactive loading time. I took the three.js driven page weight from about 600KB (500KB being three.js) to about 50KB, and reduced the loading time from multiple seconds on a 4G mobile connection to around 0.5s. This has mostly been a combination of Opus 5 and Sonnet 5 in Claude Code. It very clearly has a good grasp of WebGl, and of what impacts page loading times and rendering speed. It was able to drive Claude Code's integrated browser to measure the impact of changes, and as I spiked out a test of ogl.js it could test the differences changes made. It's not the best game (https://tinyslots.ooer.com https://tinyslots.ooer.com) but that's on me. As an exercise in building 3D in a browser, and in page speed optimization, with Claude models I am really impressed.
- redox99 3mo ago
  The UI is the equivalent of "You're absolutely right!". It's a big turnoff at least to me. That toast is such an AI thing.
levocardia 3mo ago
I don't think so. I've used Claude to generate 3D animations in python purely by making and manipulating raw meshes, and I've had great results. A more likely explanation is that 3D graphics is just pretty straightforward matrix algebra, and models (or Claude specifically) has that down cold.
killerstorm 3mo ago
Yeah it's also specifically trained to disprove Jacobian conjectures, and so on...
- emp17344 3mo ago
  Yes, it quite literally is. That’s what RLVR does.
dannyw 3mo ago
Why do you think it's pretty clear? Anthropic models are excellent at working with Blender APIs, other industry standard 3D modelling programs and tasks, and game libraries that have nothing to do with Three.js. Three.js is a quite popular library; and browser-based apps are more easily sharable and more portable. So models having a preference for using it when unprompted doesn't suggest anything, just like how models using React or Tailwind by default doesn't suggest anything.
- HarHarVeryFunny 3mo ago
  It was very notable that most of the day-1 Fable demos were all three.js based, which seems too much to be a coincidence, especially since I expect anyone giving day-1 demos are basically insiders (early access or employees). It seems they trained it to be good at it, then requested everyone to demo it.

fasterik 3mo ago

Taking a single paragraph of literary text, which is abstract and ambiguous, and converting it into a 3D animation requires an enormous amount of implicit knowledge about spatial relationships, intuitive physics, everyday objects, and so forth. Not to mention the mathematics of 3D transformations and computer graphics more generally. Saying that it's indicative of no more than three.js coding ability is absurd.

beepbooptheory 3mo ago
Why does it require knowledge about spatial relationships?
- fasterik 3mo ago
  Because the degree of realism in a video is determined by implicit knowledge of concepts like "in front of", "behind", "next to", "inside", "outside", "between", "occluded by", etc. as well as distances, angles, and the relative size of objects when viewed from a perspective.
  - beepbooptheory 3mo ago
    I guess I just don't have an intuition for why that is different from what it does for non-"spatial" things. Like the fact that it works is still because it produced the code it did token by token. Whether its dealing with, e.g., "beside the rock" or "in the array", it's doing the same kind of inferential activity.
    - fasterik 3mo ago
      >Whether its dealing with, e.g., "beside the rock" or "in the array", it's doing the same kind of inferential activity. Are you sure about that? Presumably the spatial concept of "inside" and the computer memory concept of "inside" have different contextual embeddings in an LLM, not so different from how our brains have different neural activation patterns when we use different concepts. Unless you think that "the same kind of inferential activity" also applies to human neurons firing.
    - emp17344 3mo ago
      You are correct. It’s important not to let hype peddlers imply AI possesses consciousness.
      - fasterik 3mo ago
        [dead]
    - RugnirViking 3mo ago
      > I just don't have an intuition for why that is different from what it does for non-"spatial" things A big reason people separate this out is because it was only a short time ago that AI models were noticably and uniquely bad at this. I don't say this to be like "oh so imagine where theyll be in x amount of time", rather that this thing that was once a serious limitation of the technology is slowly being compensated for by larger and better trained models
    - hombre_fatal 3mo ago
      That’s becoming as illuminating as saying you produced that comment word by word.
      - beepbooptheory 3mo ago
        What is becoming illuminating?
    - quietbritishjim 3mo ago
      If you're at (-2, 3, 5), pointing in the (2, 1, -1) direction (as a direction vector), then which is in front of the other from your point of view: the object at (4, 5, -8) or the one at (6, 4, 3)? (or neither) You can't just defer this decision to the three.js program because you need to understand this sort of relationship yourself if you're going to suitably place objects (and your own view) in a three.js scene in the first place. Building a complex scene involves making hundreds, or more, of decisions about where to place coordinates in 3d space. This example is underspecified (e.g. what are the shapes and sizes of the objects and what is the angular width of your view) but it illustrates the problem. Even with a good intuitive understanding of 3d space, we would struggle with this. LLMs are not calculators, so it's surprising if they manage it.
  - jeffybefffy519 3mo ago
    The problem i have with this argument is they still struggle to generate images of so many basic things that should be explained in their "world model" (text, fingers, reflections, faces) - so clearly they dont have a "world model" in the way that we have one. And if you use image generation capabilities, you clearly see that LLM's suck as understanding "next" to a thing.
- yeoyeo42 3mo ago
  quite loaded words there. "knowledge", "relationships". the llm must be able to output the correct code when the input says something like 'place object x to the left of object y' vs when it says 'to the right'. there is an absurd combinatorial space of the possible inputs vs outputs it must generate -> the whole set of these, interpreted from a human point of view, you could call knowledge. the [x] in input x output you could call the relationships. And the fact that the LLM doesn't have to brute force represent all of them (impossible in the limited embedding vector x internal representation state) you could interpret as 'understanding'. But again, these are loaded words that are somewhat meaningless when looking at what an LLM does in a literal way. They have plenty graphics code to train from, so this structure will be directly or indirectly available in the training data in a very plentiful way. In literal llm transformer terms, the embeddings must have some of their components statistically represent spatial structure in some way that later in the internal layers of the LLM give some statistics of how likely it is to occur for certain code tokens to appear relative to the input of spatial wording in the token stream. it is likely that their internal representations encode something more general than specific input x output stream combinations (because we already know this is the case for regular words and concepts). if not in the embeddings, then a couple layers into the network for sure.
- whywhywhywhy 3mo ago
  See 3D examples from earlier models and how they try to do it but things are stacked on top of the same x/y/z point or mushed together or parented wrong or disconnected. It’s something that specifically stopped happening in the smarter models. Same way the “pelican test” became worthless beyond a certain point and just some idea of “I like this one better” not technically more competent in any way.
Voultapher 3mo ago
*Of text it has read the full book of, read a huge corpus of text discussing it and watched the highly successful adaption of. We should feed a snippet of an unreleased book in a novel universe.

lowbloodsugar 3mo ago

Don’t worry. I’m sure they’re not training it to be good at things like writing database backends, financial services, logistics systems, user interfaces, or anything of economic value. As long as you’re not working on three.js specifically, I’m sure Anthropic isn’t making any progress you should be worried about.

HarHarVeryFunny 3mo ago
Do you feel that rendering a 3-D picture of Bilbo Baggins' house is a good indicator of generating economic value (i.e "AGI" in OpenAI parlance)?

abletonlive 3mo ago

[flagged]

peterleiser 3mo ago
But LLM's passed the Turing test, so we're done, right? ;-) Seriously, though, I agree with you that people keep moving the ball.
Applejinx 3mo ago
Oh, they still are, it's just that it's a rare parrot that can reproduce 'all of the written corpus of three.js' or stochastically align all that with arbitrary and weird conditions for what to squawk out. This is what you get. It's like the more general concept of starting with the written word (or 1000 words) and then replacing it with a picture. You've done something strikingly different, but is it serving the same function? It's fascinating to see this stuff combine such disparate sources in unexpected ways. But it is parrot, just not in the way you're expecting.
tomhow 3mo ago
The guidelines ask: Please don't fulminate. Please don't sneer, including at the rest of the community. https://news.ycombinator.com/newsguidelines.html https://news.ycombinator.com/newsguidelines.html Many people from all sections of HN, the tech industry and broader society have been surprised and wrong in all directions about how the emergence of AI is playing out. I certainly have. I can't think of a single person whose predictions have been precisely accurate. So, please don't use terms like “wall of shame” and “cope”. It doesn't help anyone make better predictions and only makes you, and HN generally, seem mean.

onion2k 3mo ago

I don't find three.js models/animations as indicative of anything other than the model's ability to write three.js code. This weekend I've been converting a game from three.js to ogl.js in order to see if I can optimise the time-to-interactive loading time. I took the three.js driven page weight from about 600KB (500KB being three.js) to about 50KB, and reduced the loading time from multiple seconds on a 4G mobile connection to around 0.5s. This has mostly been a combination of Opus 5 and Sonnet 5 in Claude Code. It very clearly has a good grasp of WebGl, and of what impacts page loading times and rendering speed. It was able to drive Claude Code's integrated browser to measure the impact of changes, and as I spiked out a test of ogl.js it could test the differences changes made. It's not the best game (https://tinyslots.ooer.com https://tinyslots.ooer.com) but that's on me. As an exercise in building 3D in a browser, and in page speed optimization, with Claude models I am really impressed.

redox99 3mo ago
The UI is the equivalent of "You're absolutely right!". It's a big turnoff at least to me. That toast is such an AI thing.

levocardia 3mo ago

I don't think so. I've used Claude to generate 3D animations in python purely by making and manipulating raw meshes, and I've had great results. A more likely explanation is that 3D graphics is just pretty straightforward matrix algebra, and models (or Claude specifically) has that down cold.

killerstorm 3mo ago

Yeah it's also specifically trained to disprove Jacobian conjectures, and so on...

emp17344 3mo ago
Yes, it quite literally is. That’s what RLVR does.

dannyw 3mo ago

Why do you think it's pretty clear? Anthropic models are excellent at working with Blender APIs, other industry standard 3D modelling programs and tasks, and game libraries that have nothing to do with Three.js. Three.js is a quite popular library; and browser-based apps are more easily sharable and more portable. So models having a preference for using it when unprompted doesn't suggest anything, just like how models using React or Tailwind by default doesn't suggest anything.

HarHarVeryFunny 3mo ago
It was very notable that most of the day-1 Fable demos were all three.js based, which seems too much to be a coincidence, especially since I expect anyone giving day-1 demos are basically insiders (early access or employees). It seems they trained it to be good at it, then requested everyone to demo it.