2 ms·

I guess I just don't have an intuition for why that is different from what it does for non-"spatial" things. Like the fact that it works is still because it pro

I guess I just don't have an intuition for why that is different from what it does for non-"spatial" things. Like the fact that it works is still because it produced the code it did token by token. Whether its dealing with, e.g., "beside the rock" or "in the array", it's doing the same kind of inferential activity.

fasterik 3mo ago
>Whether its dealing with, e.g., "beside the rock" or "in the array", it's doing the same kind of inferential activity. Are you sure about that? Presumably the spatial concept of "inside" and the computer memory concept of "inside" have different contextual embeddings in an LLM, not so different from how our brains have different neural activation patterns when we use different concepts. Unless you think that "the same kind of inferential activity" also applies to human neurons firing.
emp17344 3mo ago
You are correct. It’s important not to let hype peddlers imply AI possesses consciousness.
- fasterik 3mo ago
  [dead]
RugnirViking 3mo ago
> I just don't have an intuition for why that is different from what it does for non-"spatial" things A big reason people separate this out is because it was only a short time ago that AI models were noticably and uniquely bad at this. I don't say this to be like "oh so imagine where theyll be in x amount of time", rather that this thing that was once a serious limitation of the technology is slowly being compensated for by larger and better trained models
hombre_fatal 3mo ago
That’s becoming as illuminating as saying you produced that comment word by word.
- beepbooptheory 3mo ago
  What is becoming illuminating?
quietbritishjim 3mo ago
If you're at (-2, 3, 5), pointing in the (2, 1, -1) direction (as a direction vector), then which is in front of the other from your point of view: the object at (4, 5, -8) or the one at (6, 4, 3)? (or neither) You can't just defer this decision to the three.js program because you need to understand this sort of relationship yourself if you're going to suitably place objects (and your own view) in a three.js scene in the first place. Building a complex scene involves making hundreds, or more, of decisions about where to place coordinates in 3d space. This example is underspecified (e.g. what are the shapes and sizes of the objects and what is the angular width of your view) but it illustrates the problem. Even with a good intuitive understanding of 3d space, we would struggle with this. LLMs are not calculators, so it's surprising if they manage it.
- beepbooptheory 3mo ago
  I think of all these responses this articulates my point the most. If humans would struggle with it, why is it still supposed to be "spatial reasoning"? And underspecified or not, you frame this exactly in the way an LLM would tackle this: if I am making a game, I don't place objects in arbitrary positions in a space and try to keep it all in my head, but in relations to one another. Then, the point in front of another is the one where A-B>0.

fasterik 3mo ago

>Whether its dealing with, e.g., "beside the rock" or "in the array", it's doing the same kind of inferential activity. Are you sure about that? Presumably the spatial concept of "inside" and the computer memory concept of "inside" have different contextual embeddings in an LLM, not so different from how our brains have different neural activation patterns when we use different concepts. Unless you think that "the same kind of inferential activity" also applies to human neurons firing.

emp17344 3mo ago

You are correct. It’s important not to let hype peddlers imply AI possesses consciousness.

fasterik 3mo ago
[dead]

RugnirViking 3mo ago

> I just don't have an intuition for why that is different from what it does for non-"spatial" things A big reason people separate this out is because it was only a short time ago that AI models were noticably and uniquely bad at this. I don't say this to be like "oh so imagine where theyll be in x amount of time", rather that this thing that was once a serious limitation of the technology is slowly being compensated for by larger and better trained models

hombre_fatal 3mo ago

That’s becoming as illuminating as saying you produced that comment word by word.

beepbooptheory 3mo ago
What is becoming illuminating?

quietbritishjim 3mo ago

If you're at (-2, 3, 5), pointing in the (2, 1, -1) direction (as a direction vector), then which is in front of the other from your point of view: the object at (4, 5, -8) or the one at (6, 4, 3)? (or neither) You can't just defer this decision to the three.js program because you need to understand this sort of relationship yourself if you're going to suitably place objects (and your own view) in a three.js scene in the first place. Building a complex scene involves making hundreds, or more, of decisions about where to place coordinates in 3d space. This example is underspecified (e.g. what are the shapes and sizes of the objects and what is the angular width of your view) but it illustrates the problem. Even with a good intuitive understanding of 3d space, we would struggle with this. LLMs are not calculators, so it's surprising if they manage it.

beepbooptheory 3mo ago
I think of all these responses this articulates my point the most. If humans would struggle with it, why is it still supposed to be "spatial reasoning"? And underspecified or not, you frame this exactly in the way an LLM would tackle this: if I am making a game, I don't place objects in arbitrary positions in a space and try to keep it all in my head, but in relations to one another. Then, the point in front of another is the one where A-B>0.