4 ms·
You're imagining LLMs don't just regurgitate and recombine things they already know from things they have seen before. A new variant would not be in the dataset
by andy_ppp 2y ago
You're imagining LLMs don't just regurgitate and recombine things they already know from things they have seen before. A new variant would not be in the dataset so would not be understood. In fact this is quite a good way to show LLMs are NOT thinking or understanding anything in the way we understand it.
- shaky-carrousel 2y agoYes, that's how you can really tell if the model is doing real thinking and not recombinating things. If it can correctly play a novel game, then it's doing more than that.
- jahnu 2y agoI wonder what the minimal amount of change qualifies as novel? "Chess but white and black swap their knights" for example?
- the_af 2y agoI wonder what would happen with a game that is mostly chess (or chess with truly minimal variations) but with all the names changed (pieces, moves, "check", etc, all changed). The algebraic notation is also replaced with something else so it cannot be pattern matched against the training data. Then you list the rules (which are mostly the same as chess). None of these changes are explained to the LLM, so if it can tell it's still chess, it must deduce this on its own. Would any LLM be able to play at a decent level?
- jahnu 2y agoNice. Even the tiniest rule, I strongly suspect, would throw off pattern matching. “Every second move, swap the name of the piece you move to the last piece you moved.”
- dwighttk 2y agoNo LLM model is doing any thinking.
- selestify 2y agoHow do you define thinking?
- antononcube 2y agoBeing fast at doing linear algebra computations. (Is there any other kind?!)
- landryraccoon 2y agoMaking the OP feel threatened/emotionally attached/both enough to call the language model a rival / companion / peer instead of a tool.
- Jerrrrrrry 2y agoLolol. It's a chess thread, say it. We are pawns, hoping to be maybe a Rook to the King by endgame. Some think we can promote our pawns to Queens to match. Luckily, the Jester muses!
- timdiggerm 2y agoBy that standard (and it is a good standard), none of these "AI" things are doing any thinking
- Jerrrrrrry 2y agomusical goalposts, gotta love it. These LLM's just exhibited agency. Swallow your pride.
- samatman 2y ago"Does it generalize past the training data" has been a pre-registered goalpost since before the attention transformer architecture came on the scene.
- Jerrrrrrry 2y ago>'thinking' vs 'just recombinating things If there is a difference, and LLM's can do one but not the other... >By that standard (and it is a good standard), none of these "AI" things are doing any thinking >"Does it generalize past the training data" has been a pre-registered goalpost since before the attention transformer architecture came on the scene. Then what the fuck are they doing. Learning is thinking, reasoning, what have you. Move goalposts, re-define words, it won't matter.
- empath75 2y agoYou say this quite confidently, but LLMs do generalize somewhat.