3 ms·
It also can’t learn. Once the training is done, the network is set in stone.
by Philadelphia 3y ago
It also can’t learn. Once the training is done, the network is set in stone.
- adamisom 3y agoMakes me wonder why we don’t see deployed models that keep learning during inference.
- LegitShady 3y agoMicrosoft tay has entered the chat
- Der_Einzige 3y agoThe curse of dimensionality and exploding/vanishing gradients are why incremental learning is still so rare.
- optimalsolver 3y agohttps://en.wikipedia.org/wiki/Catastrophic_interference https://en.wikipedia.org/wiki/Catastrophic_interference
- danielbln 3y agoTechnically it can do in-context learning (and really well, too), but that's not persisted into the network.
- kenjackson 3y agoAnd that just seems like an engineering problem. Not something that is considered intractable.
- version_five 3y agoIt's easy to say that, but "surely it must be possible to connect an llm in such a way that it becomes intelligent" (tell me if I'm misinterpreting) is not a demonstration of anything. It's basically restating the view from the 50s that with computers having been invented, an intelligent computer is a short way off.
- deleted 3y ago[deleted]
- ryanjshaw 3y agoWhat do you mean by "learn"? The network has learned human patterns of language, knowledge and information processing. If you want to update that, you can re-train it on a regular basis, and re-play its sensory/action history to "restore" its state. If you mean "learn from experience", (1) a lot of that is pointless because it's already learned from the experiences of millions of humans through their writing and (2) LLMs can "learn" when you explain consequences.
- pixl97 3y agoIn theory they could learn by having their discussions fed back to them in the future, and it does seem that this occurs. Now, there is no continuous learning in the human/animal sense. Of course it is thought that even humans have to sleep and re-weight their networks so short term knowledge is converted to long term knowledge.