3 ms·
This is how I envision LLMs working to some extent. As in that the "logic paths" follow something like this where the markov-chain-esque probabilities jump arou
by elcritch 1y ago
This is how I envision LLMs working to some extent. As in that the "logic paths" follow something like this where the markov-chain-esque probabilities jump around the vector space. It reminds me that to get the answer I want that I need to setup the prompt to get me near the right "attractor logic" pathway. Once in a close enough ballpark then they'll bounce to the right path.
As a counter, I found that if you add an incorrect statement or fact that lies completely outside the realm of the logic-attractor for a given topic that the output is severally degraded. Well more like a statement or fact that's "orthogonal" to the logic-attractor for a topic. Very much as if it's struggling to stay on the logic-attractor path but the outlier fact causes it to stray.
Sometimes less is more.
- cs702 1y agoInteresting. Nothing prohibits us from thinking of pretrained LLMs as dynamical systems that take a token state and compute an updated token state: x_{n+1} = LLM(x_n), starting from an initial token state x_0. Surely we can compute trajectories (without sampling, for determinism) and study whether LLMs exhibit chaotic behavior. I don't think I've seen research along those lines before. Has anyone here?
- elcritch 1y agoLooks like @cs702 [1] posted a related link where a NN follows an attractor pattern! I've only skimmed it but it very much looks like what I've been imagining. It'd be cool to see more research into this area. 1: https://news.ycombinator.com/item?id=45427778 https://news.ycombinator.com/item?id=45427778 2: https://towardsdatascience.com/attractors-in-neural-network-circuits-beauty-and-chaos/ https://towardsdatascience.com/attractors-in-neural-network-...