2 ms·
There's more of a model inside large language models than was previously thought. How much of a model? Nobody seems to know. There was that one result where som
by Animats 4y ago
There's more of a model inside large language models than was previously thought. How much of a model? Nobody seems to know. There was that one result where someone found what looked like an Othello board in the neuron state.
Someone wrote, below:
> We know the basic architecture of large language models, but hardly anything about how they calculate anything specific. That’s the mystery. It will take research, not casual tinkering.
Yes. This is an unexpected situation. Understanding how these things work is way behind making them work. Which is a big problem, since they make up plausible stuff when they don't understand.
- anonyfox 4y agoIn quantum physics, we also don’t really understand anything („shut up and calculate“) still people build awesome stuff that works. Humans learned how to use and create fire looooong before understanding what fire actually is! Just a few centuries ago, people believed that fire is its own element! Feels kinda similar to people searching for „consciousness“ that „understands“ things as if it would be something special/magic… when it’s probably more like naturally emerging behaviors when scaling up neural networks?