4 ms·

Seems like depending on your field these days, the hot thing to do is build your own private benchmarks. In my own testing, no frontier model knows how to repl

Seems like depending on your field these days, the hot thing to do is build your own private benchmarks. In my own testing, no frontier model knows how to replicate an original 1990s Super Soaker prototype design, which for the most part, should be almost completely possible with Home Depot parts. They just don't understand PVC parts, triggers, etc.

softwaredoug 3mo ago
Or defensively expect models to be stupid. Seems the smart thing to do is not assume an agent will do the right thing. But to create the scaffold / harness that enforces constraints to steer them towards a good result. Then you can swap out the really smart model for maybe something cheaper.
thierrydamiba 3mo ago
Or you’re getting steered into la la land because of your prompt
- bellowsgulch 3mo ago
  Certainly, but deconstructing the problem, none of the models seem to appreciate the staggering difference between a ball valve and a button release. Of course, there's also no super soaker engineer jobs to take, so I'm sure training sophisticated models to do well in that area is not a high priority for any firms.
  - cyanydeez 3mo ago
    I assume you prep them with a proper manual of smaller part combos so they atleast have some chase of stumbling into the correct configurations. I wonder if a more generic lego-manual like task would be more representative. It kind seems like you're testing for AGI.
    - bellowsgulch 3mo ago
      Yeah, I start with smaller tasks, like matching dimensional parts. Tasks like these you can't one shot, or they end up producing diagrams of plausible looking Super Soakers, but maybe ones that look more like an enthusiast making their own designs from scratch. Fair approach! But replication is different. For me, it's not knowing whether or not it understands there's a big difference between a ball valve and a button release, or that once you start talking about depressing mechanisms for pressure release, you're activating some sort of signals that are too close to triggers (which, is what you want, after all!) and "triggers" are embedded with a very short distance to "guns" in any well-trained model.
ACCount37 3mo ago
It's a "commonsense spatial reasoning/problem solving" kind of problem. LLMs fail at spatial reasoning forever. What humans "easily" solve in seconds with raw spatial reasoning LLMs often find easier to solve by invoking A* or a constraint solver. Might be that text data is particularly bad at teaching that to LLMs. Or that being good at spatial reasoning requires true recurrence, and autoregressive chain of thought is a poor substitute. Or it might be that human brain was optimized by evolution for solving spatial problems in open ended 3D environments for hundreds of millions of years, optimized for language for mere hundreds of thousands of years, and only optimized for writing computer code for a few decades at most. The current frontier is halfway competent at benign closed 2D work, but still completely fumbles anything remotely close to open ended real world 3D work. It's getting better, but very slowly.
- therobots927 3mo ago
  A literal bird brain would outperform an LLM on most spatial reasoning tasks. Extrapolating the core theory of LLMs - that we can reverse engineer reasoning through language - does that imply that if we train a bird song LLM to predict next “token” (pitch) of a birdsong, that the LLM could excel in a bird flight simulator? I think it’s pretty clear that this is a dead end.
  - ACCount37 3mo ago
    Do birds expose enough of their cognition through birdsong? Do birds expose locomotion-relevant functions specifically through birdsong? Do we have enough birdsong data available to start solving the inverse problem? If "yes" on all, then we might be able to do it. I expect "no" on most of that, for birds. But humans treat language as an interface to their higher cognitive functions, and stockpile language data. That looks an awful lot like a set of two "yes". The last open question is: is there enough spatial reasoning reflected in the language data we have? It's plausible that spatial reasoning is too evolutionary old and too low-level, too far removed from higher cognition, to leak into language heavily. And it's also plausible that existing LLM architecture is uniquely poorly suited to learning spatial reasoning - higher cognitive functions involved in things like writing code or even composing poetry are a better fit for the architecture. And it's plausible that we're underestimating just how complex spatial reasoning truly is - Moravec's paradox strikes again. We know that LLMs perform poorly and improve slowly on spatial reasoning tasks, but not exactly why. And progress on things like ARC-AGI series shows that they're not completely inept.

softwaredoug 3mo ago

Or defensively expect models to be stupid. Seems the smart thing to do is not assume an agent will do the right thing. But to create the scaffold / harness that enforces constraints to steer them towards a good result. Then you can swap out the really smart model for maybe something cheaper.

thierrydamiba 3mo ago

Or you’re getting steered into la la land because of your prompt

bellowsgulch 3mo ago
Certainly, but deconstructing the problem, none of the models seem to appreciate the staggering difference between a ball valve and a button release. Of course, there's also no super soaker engineer jobs to take, so I'm sure training sophisticated models to do well in that area is not a high priority for any firms.
- cyanydeez 3mo ago
  I assume you prep them with a proper manual of smaller part combos so they atleast have some chase of stumbling into the correct configurations. I wonder if a more generic lego-manual like task would be more representative. It kind seems like you're testing for AGI.
  - bellowsgulch 3mo ago
    Yeah, I start with smaller tasks, like matching dimensional parts. Tasks like these you can't one shot, or they end up producing diagrams of plausible looking Super Soakers, but maybe ones that look more like an enthusiast making their own designs from scratch. Fair approach! But replication is different. For me, it's not knowing whether or not it understands there's a big difference between a ball valve and a button release, or that once you start talking about depressing mechanisms for pressure release, you're activating some sort of signals that are too close to triggers (which, is what you want, after all!) and "triggers" are embedded with a very short distance to "guns" in any well-trained model.

ACCount37 3mo ago

It's a "commonsense spatial reasoning/problem solving" kind of problem. LLMs fail at spatial reasoning forever. What humans "easily" solve in seconds with raw spatial reasoning LLMs often find easier to solve by invoking A* or a constraint solver. Might be that text data is particularly bad at teaching that to LLMs. Or that being good at spatial reasoning requires true recurrence, and autoregressive chain of thought is a poor substitute. Or it might be that human brain was optimized by evolution for solving spatial problems in open ended 3D environments for hundreds of millions of years, optimized for language for mere hundreds of thousands of years, and only optimized for writing computer code for a few decades at most. The current frontier is halfway competent at benign closed 2D work, but still completely fumbles anything remotely close to open ended real world 3D work. It's getting better, but very slowly.

therobots927 3mo ago
A literal bird brain would outperform an LLM on most spatial reasoning tasks. Extrapolating the core theory of LLMs - that we can reverse engineer reasoning through language - does that imply that if we train a bird song LLM to predict next “token” (pitch) of a birdsong, that the LLM could excel in a bird flight simulator? I think it’s pretty clear that this is a dead end.
- ACCount37 3mo ago
  Do birds expose enough of their cognition through birdsong? Do birds expose locomotion-relevant functions specifically through birdsong? Do we have enough birdsong data available to start solving the inverse problem? If "yes" on all, then we might be able to do it. I expect "no" on most of that, for birds. But humans treat language as an interface to their higher cognitive functions, and stockpile language data. That looks an awful lot like a set of two "yes". The last open question is: is there enough spatial reasoning reflected in the language data we have? It's plausible that spatial reasoning is too evolutionary old and too low-level, too far removed from higher cognition, to leak into language heavily. And it's also plausible that existing LLM architecture is uniquely poorly suited to learning spatial reasoning - higher cognitive functions involved in things like writing code or even composing poetry are a better fit for the architecture. And it's plausible that we're underestimating just how complex spatial reasoning truly is - Moravec's paradox strikes again. We know that LLMs perform poorly and improve slowly on spatial reasoning tasks, but not exactly why. And progress on things like ARC-AGI series shows that they're not completely inept.