3 ms·
Neural plasticity is real, and something LLMs are incapable of. So sorry.
by daveguy 3mo ago
Neural plasticity is real, and something LLMs are incapable of. So sorry.
- dahart 3mo agoTrue for today’s static models during inference. Not true for self-supervised learning, not true during training or fine-tuning, of course. Ignores that LLMs might start continuous training in the future - there’s no fundamental or technical constraint that prevents LLM ‘plasticity’. And ignores that accumulating context/memories/skills/etc affects performance and might count as a valid analogy to what many people loosely call ‘neural plasticity’, which is sometimes casually mistaking knowledge for network modification.
- daveguy 3mo agoBoth of those things only happen once by the LLM model provider and not every time a prompt is issued.
- dahart 3mo agoToday, depending on which model you use. You’re making unstated assumptions. And that’s not a fundamental property of LLMs, it’s happenstance. LLMs are capable of ‘plasticity’, by design.
- daveguy 3mo agoYou are incorrect. "memory.md" and other context manipulations do not change the underlying model.
- dahart 2mo agoYou misunderstood me. I was referring to training, not memories. It’s you who is wrong, my friend. It is possible to train on every prompt. ChatGPT is not currently doing that, but it is in fact possible to do.