4 ms·

> What is this idea of AGI that there exists a significant gap still? Because an LLM doesn't simulate the brain. It's a completely different model that simula

by stale2002 3y ago

> What is this idea of AGI that there exists a significant gap still? Because an LLM doesn't simulate the brain. It's a completely different model that simulates something much different. > There is nothing special about fatigue or frustration that make it any less modellable than any other implicit structure Fatigue isn't being modeled. Instead, the predictive text as the result of that fatigue is being models. You are confusing output to input. To make this more clear, imagine a coin flip modeler. A model that just picked a random seed, is completely different from a model that is doing physics calculations based on a coin flipping in the air. Even if both models only output "heads" or "tails" . Same argument applies to language models.

famouswaffles 3y ago
>Because an LLM doesn't simulate the brain. That's pretty moot. You don't need a human brain anyone than a plane needs feathers and to flap wings to fly. >Fatigue isn't being modeled. Instead, the predictive text as the result of that fatigue is being models. Emotion is definitely being modelled. There's nothing random about anything that's happened. Train on protein sequences alone and biological structure and function emerge in the inner layers alone. It doesn't matter that those things are not explicitly stated in the data. Because they implicitly structure it, it gets learnt. https://www.pnas.org/doi/full/10.1073/pnas.2016239118 https://www.pnas.org/doi/full/10.1073/pnas.2016239118 Similarity, emotion is evidently being modelled to a high degree. https://arxiv.org/abs/2307.11760 https://arxiv.org/abs/2307.11760
- stale2002 3y ago
  I don't think you addressed the main point. Do you agree that a simulation that uses a random number generator to flip a coin is different from a physics simulator that measures the exact physics of a coin flipping? And then after you answer this question, do to understand the parallels to other AI models?
  - famouswaffles 3y ago
    You can simulate things with different models. Sure. I don't think there's any point to address here though. It's not like you can ascertain what kind of model is being used for LLM predictions other than a hunch. Meanwhile I've given several examples. I can show another one of an Othello LLM constructing a state of the board of the game of Othello to aid predictions. https://thegradient.pub/othello/ https://thegradient.pub/othello/

famouswaffles 3y ago

>Because an LLM doesn't simulate the brain. That's pretty moot. You don't need a human brain anyone than a plane needs feathers and to flap wings to fly. >Fatigue isn't being modeled. Instead, the predictive text as the result of that fatigue is being models. Emotion is definitely being modelled. There's nothing random about anything that's happened. Train on protein sequences alone and biological structure and function emerge in the inner layers alone. It doesn't matter that those things are not explicitly stated in the data. Because they implicitly structure it, it gets learnt. https://www.pnas.org/doi/full/10.1073/pnas.2016239118 https://www.pnas.org/doi/full/10.1073/pnas.2016239118 Similarity, emotion is evidently being modelled to a high degree. https://arxiv.org/abs/2307.11760 https://arxiv.org/abs/2307.11760

stale2002 3y ago
I don't think you addressed the main point. Do you agree that a simulation that uses a random number generator to flip a coin is different from a physics simulator that measures the exact physics of a coin flipping? And then after you answer this question, do to understand the parallels to other AI models?
- famouswaffles 3y ago
  You can simulate things with different models. Sure. I don't think there's any point to address here though. It's not like you can ascertain what kind of model is being used for LLM predictions other than a hunch. Meanwhile I've given several examples. I can show another one of an Othello LLM constructing a state of the board of the game of Othello to aid predictions. https://thegradient.pub/othello/ https://thegradient.pub/othello/