4 ms·
Reads well if you don't think about it too much... For example: Where does the succulent go if the light bulbs are nestled into the lens of the DLSR? Balanced o
by RandomLensman 3y ago
Reads well if you don't think about it too much... For example: Where does the succulent go if the light bulbs are nestled into the lens of the DLSR? Balanced on the light bulbs? Why would the gummy worm package need to be in the center of the book to maintain balance?
- tornato7 3y agohttps://i.imgur.com/feEiiZA.png https://i.imgur.com/feEiiZA.png I tried to stack all of these objects myself and couldn't really. I think GPT-4's approach is actually really good. It correctly points out that the gummy worms make a flexible base for the DSLR (otherwise the protruding buttons/viewfinder make it wobbly on the hard book), and the light bulbs are able to nestle into the front of the lens. If they were smaller light bulbs I could probably use the four of them as a small base on top of the lens to host the succulent.
- RandomLensman 3y agoMight also put the light bulbs as a base (especially if in a box). They are pretty sturdy and can hold a book.
- tornato7 3y agoThe point is that ChatGPT undeniably built a world model good enough to understand the physical and three-dimensional properties of these items pretty well, and it gives me a somewhat workable way to stack them, despite never having seen that in its training data.
- RandomLensman 3y agoYou cannot conclude that from the output - the training data will likely contain a lot stacking things. Everyday objects also might have some stacking properties that make these questions easy to answer even with semi-random answers. Plus, some stuff clearly makes no sense or is ignored (like the gummy worms in the center, forgetting about the succulent in some cases). If you want to test world modeling, give it objects it will have never encountered, describe them and then ask to stack etc. For example, a bunch of 7 dimensional objects that can only be stacked a certain way.
- biorach 3y ago> For example, a bunch of 7 dimensional objects that can only be stacked a certain way. That's a ridiculous example.
- RandomLensman 3y agoWhy? You need make sure that a solution requires true understanding and isn't in the training set. If it can reason properly, it shouldn't have a problem with such a problem.
- DiogenesKynikos 3y agoHow well do humans reason about 7-dimensional objects? I'm already impressed if a computer can reason flexibly about 3-dimensional objects.
- RandomLensman 3y agoHumans having the right mathematical tooling do ok.
- DiogenesKynikos 3y agoWhat percentage of humans have that mathematical tooling? The fact that people are even raising these sorts of obscure tests shows just how far AI has advanced.
- moffkalast 3y ago> If you want to test world modeling, give it objects it will have never encountered, describe them and then ask to stack etc. For example, a bunch of 7 dimensional objects that can only be stacked a certain way. And when it does that perfectly, I assume you'll say that was also in the training data? All examples I've seen or tried point to LLMs being able to do some kind of reasoning that is completely dynamic, even when presented with the most outlandish cases.