2 ms·

Ultimately, the brain is just a bunch of neurons activating in a specific pattern. This observation does not really tell us anything though. It doesn't acknowle

by Certhas 17d ago

Ultimately, the brain is just a bunch of neurons activating in a specific pattern. This observation does not really tell us anything though. It doesn't acknowledge the difference between a 2500 Neuron fruit fly brains and a human brain. Likewise, the fact that LLMs are a stochastic autoregressive process (which is a class of systems every bit as rich as the ODEs used to model neurons) tells us nothing a priori.

wood_spirit 17d ago
Absolutely. If someone makes the weights do continuous learning etc then perhaps an llm can internalise morals. Of course, just like a human, it will be possible to talk it out of those morals. Another recent thread about this is https://news.ycombinator.com/item?id=49744420 https://news.ycombinator.com/item?id=49744420
- Certhas 17d ago
  If I repeatedly call an LLM in a loop with a markdown document it can edit, would that make it qualify for you? If I give an LLM to compact its context window, so the context it carries can evolve iteratively over time as more and more things come in, is that enough? Compacting the context is really a very, very interesting example here. The "next token predictor" is telling an external tool to change all "previous" tokens. So an LLM + a harness that allows compacting the context is no longer just a token predictor at all! You don't need continuous learning to get interesting dynamics. You just need feedback loops.
  - wood_spirit 15d ago
    I come back to this way after everyone else has stopped reading. But it’s been making me think. Richard Dawkins says he thinks LLMs think. And the physical angle is that nothing is special about humans and software simulating it would also be thinking. But from using LLMs all the time, and understanding what is under the hood, I’m thinking that the current approaches aren’t cutting it for me and I’m not expecting them to get there. There was such big jumps early on but progress is slowing as though diminishing returns. So we can build things that think and outthink us, but I don’t think anything we’ve hit upon yet is going to scale up into it.
leg100 17d ago
One is an observation the other is not, it's a description of what it is; one is a posteriori, the other is a priori (contrary to what you say). They're not comparable.

by Certhas 17d ago

wood_spirit 17d ago

Absolutely. If someone makes the weights do continuous learning etc then perhaps an llm can internalise morals. Of course, just like a human, it will be possible to talk it out of those morals. Another recent thread about this is https://news.ycombinator.com/item?id=49744420 https://news.ycombinator.com/item?id=49744420

Certhas 17d ago
If I repeatedly call an LLM in a loop with a markdown document it can edit, would that make it qualify for you? If I give an LLM to compact its context window, so the context it carries can evolve iteratively over time as more and more things come in, is that enough? Compacting the context is really a very, very interesting example here. The "next token predictor" is telling an external tool to change all "previous" tokens. So an LLM + a harness that allows compacting the context is no longer just a token predictor at all! You don't need continuous learning to get interesting dynamics. You just need feedback loops.
- wood_spirit 15d ago
  I come back to this way after everyone else has stopped reading. But it’s been making me think. Richard Dawkins says he thinks LLMs think. And the physical angle is that nothing is special about humans and software simulating it would also be thinking. But from using LLMs all the time, and understanding what is under the hood, I’m thinking that the current approaches aren’t cutting it for me and I’m not expecting them to get there. There was such big jumps early on but progress is slowing as though diminishing returns. So we can build things that think and outthink us, but I don’t think anything we’ve hit upon yet is going to scale up into it.

leg100 17d ago

One is an observation the other is not, it's a description of what it is; one is a posteriori, the other is a priori (contrary to what you say). They're not comparable.