4 ms·
Tricks are nothing but patterns in the logical formulae we reduce. Ergo these are latent vectors in our brain. We use analogies like geometry in order to use A
by vatsachak 7mo ago
Tricks are nothing but patterns in the logical formulae we reduce.
Ergo these are latent vectors in our brain. We use analogies like geometry in order to use Algebraic Geometry to solve problems in Number Theory.
An AI trained on Lean Syntax trees might develop it's own weird versions of intuition that might actually properly contain ours.
If this sounds far fetched, look at Chess. I wonder if anyone has dug into StockFish using mechanistic interpretability
- slopinthebag 7mo agoStockfish's power comes from mostly search, and the ML techniques it uses are mainly about better search, i.e. pruning branches more efficiently.
- vatsachak 7mo agoThe weights must still have some understanding of the chess board. Though there is always the chance that it makes no sense to us
- slopinthebag 7mo agoEven that is probably too much. It has no understanding of what "chess" is, or what a chess board is, or even what a game is. And yet it crushes every human with ease. It's pretty nuts haha.
- anematode 7mo agoActually, the neural net itself is fairly imprecise. Search is required for it to achieve good play. Here's an example of me beating Stockfish 18 at depth 1: https://lichess.org/XmITiqmi https://lichess.org/XmITiqmi
- Sopel 7mo agochess is just a simple mathematical construct so that's not surprising
- hollerith 7mo agoDoes Stockfish have weights or use a neural net? I know older versions did not.
- Sopel 7mo agoyes
- emp17344 7mo agoWhy must it involve understanding? I feel like you’re operating under the assumption that functionalism is the “correct” philosophical framework without considering alternative views.
- PowerElectronix 7mo agoThere is no understanding, the weights are selected based on better fit. Our cells have no understanding of optics just because they have the eyes coded in their DNA.
- Sopel 7mo agoThe ML techniques it uses are only about evaluation, but you were close
- myffical 7mo agoSome DeepMind researchers used mechanistic interpretability techniques to find concepts in AlphaZero and teach them to human chess Grandmasters: https://www.pnas.org/doi/10.1073/pnas.2406675122 https://www.pnas.org/doi/10.1073/pnas.2406675122
- hodgehog11 7mo agoThis argument, that LLMs can develop new crazy strategies using RLVR on math problems (like what happened with Chess), turns out to be false without a serious paradigm shift. Essentially, the search space is far too large, and the model will need help to explore better, probably with human feedback. https://arxiv.org/abs/2504.13837 https://arxiv.org/abs/2504.13837
- narrator 7mo agoThe search space for the game of Go was also thought to be too large for computers to manage.
- sealeck 7mo agoYes and making a horse drawn cart drive itself was thought to be impossible so why don't we have faster than light travel yet...
- Finbel 7mo agoYes but "the search space is too large" is something that has been said about innumerable AI-problems that were then solved. So it's not unreasonable that one doubts the merit of the statement when it's said for the umpteenth time.
- hodgehog11 7mo agoI should have been more specific then. The problem isn't that the search space is too large to explore. The problem is that the search space is so large that the training procedure actively prefers to restrict the search space to maximise short term rewards, regardless of hyperparameter selection. There is a tradeoff here that could be ignored in the case of chess, but not for general math problems. This is far from unsolvable. It just means that the "apply RL like AlphaGo" attitude is laughably naive. We need at least one more trick.
- 7mo ago