4 ms·
> future models (which admittedly may not look much like today's LLMs) will blur the lines between mutable context and immutable weights. "may not look much li
by daveguy 2y ago
> future models (which admittedly may not look much like today's LLMs) will blur the lines between mutable context and immutable weights.
"may not look much like today's LLMs" is really sweeping the whole point under the rug. The difference between what we are doing now with static LLM models and dynamic models will require algorithmic changes that have not yet been invented. That's in addition to the fact that the processing of a single neuron is completely parallel with respect to inputs. GPUs make them more parallel, but the analog devices we are trying to mimic are vastly more complicated than what we are using as a substitute.
- CamperBob2 2y agoWhatever. This ( https://eqbench.com/results/creative-writing-v2/deepseek-ai__DeepSeek-R1.txt https://eqbench.com/results/creative-writing-v2/deepseek-ai_... ) is intelligence in action, from an entity with an internal world model that's at least as valid as mine. Either that, or it's witchcraft. (Either that, or somebody with both time and talent to waste is taking the piss and pretending to be an AI.) If anyone ever does succeed in updating weights dynamically on a continuous basis, things are going to get really interesting really quickly. Even now, arguments based on relative complexity are completely invalid. A few fast neurons are at least as good at 'thinking' as a lot of slow ones are.
- daveguy 2y agoHah, "Whatever." Seriously? The writing is still derivative meandering pulp. No need to invoke magic. It's cool that it can make complete paragraphs though. LLMs are a huge breakthrough. They're just not intelligent. Updating weights on a continual basis requires processing weights productively on a regular basis. That's many flops per weight away from where we are in processing and bandwidth. The computation limitations still apply.
- mrbungie 2y agoMetacommentary: I stopped arguing because this thread is reeking of AI bro non-sense, they can't argue with facts so they continue with vague statements and deflection (e.g like the "Whatever" you've got above). The GenAI bubble can't burst soon enough. As I said, neuroscience be damned, we've got transformers now. (/s)
- sgt101 2y ago"My king, it's amazing these things can contain and regurgitate knowledge on demand. The next step is an army of Gollems that will destroy your enemies, all I need is three bags of gold." - Sargon the scribe, near Baghdad first month, 2024 BC "My king, the Hittites have created scrolls that are wide and long despite that we burned their caravan taking palm leaves to the north. They have found a way to flense lambs to make pages themselves, and their ink is pure silver. They will soon have an army of Gollems and will destroy your empire. To stop this all I need is ten bags of gold." - Sargon the scribe, near Baghdad second month, 2024 BC "My king, alas all the gold has gone, and yet neither will the gollems walk, nor have the Hittities come. My dancing girls grow cold, and I have no figs for my table all I need is 20 bags of gold" - Sargon the scribe, near Baghdad third month, 2024 BC What happened to Sargon next is not recorded...
- ninetyninenine 2y agoThere’s agents now. It’s not just paragraphs. And LLMs can beat you on intellectual tasks. I don’t see how you can throw that word around as if you know your more intelligent when it clearly can be superior to you on many many tasks.