3 ms·

I'm no neuroscientist, but I hung out with a few, so here are some bits I picked up from them. In biological brians: - neurons have signal surpression (-ive ac

by sgt101 2y ago

I'm no neuroscientist, but I hung out with a few, so here are some bits I picked up from them. In biological brians: - neurons have signal surpression (-ive activation) as well as propagation (+activation) - neurons appear to process and propagation due to unknown internal mechanisms. - there are complex sub structures within biological neural nets where the architecture is radically different from in other sections of the network, strongly in contrast to the homogenous structure of ANN's - many different types of neurons with different properties and behaviors in terms of network formation and network activity are present in BNN's in contrast to ANN's - BNN's learn during processing. ANN's are static after training.

qrsjutsu 2y ago
[dead]
CamperBob2 2y ago
Those are mostly structure-versus-function arguments, centered on what a brain is as opposed to what it does. Only the static nature of the current ML models seems like a valid point... and it's changing too. I'd wager that future models (which admittedly may not look much like today's LLMs) will blur the lines between mutable context and immutable weights. When that happens, all bets are truly off.
- daveguy 2y ago
  > future models (which admittedly may not look much like today's LLMs) will blur the lines between mutable context and immutable weights. "may not look much like today's LLMs" is really sweeping the whole point under the rug. The difference between what we are doing now with static LLM models and dynamic models will require algorithmic changes that have not yet been invented. That's in addition to the fact that the processing of a single neuron is completely parallel with respect to inputs. GPUs make them more parallel, but the analog devices we are trying to mimic are vastly more complicated than what we are using as a substitute.
  - CamperBob2 2y ago
    Whatever. This ( https://eqbench.com/results/creative-writing-v2/deepseek-ai__DeepSeek-R1.txt https://eqbench.com/results/creative-writing-v2/deepseek-ai_... ) is intelligence in action, from an entity with an internal world model that's at least as valid as mine. Either that, or it's witchcraft. (Either that, or somebody with both time and talent to waste is taking the piss and pretending to be an AI.) If anyone ever does succeed in updating weights dynamically on a continuous basis, things are going to get really interesting really quickly. Even now, arguments based on relative complexity are completely invalid. A few fast neurons are at least as good at 'thinking' as a lot of slow ones are.
    - daveguy 2y ago
      Hah, "Whatever." Seriously? The writing is still derivative meandering pulp. No need to invoke magic. It's cool that it can make complete paragraphs though. LLMs are a huge breakthrough. They're just not intelligent. Updating weights on a continual basis requires processing weights productively on a regular basis. That's many flops per weight away from where we are in processing and bandwidth. The computation limitations still apply.
      - mrbungie 2y ago
        Metacommentary: I stopped arguing because this thread is reeking of AI bro non-sense, they can't argue with facts so they continue with vague statements and deflection (e.g like the "Whatever" you've got above). The GenAI bubble can't burst soon enough. As I said, neuroscience be damned, we've got transformers now. (/s)
        sgt101 2y ago
        "My king, it's amazing these things can contain and regurgitate knowledge on demand. The next step is an army of Gollems that will destroy your enemies, all I need is three bags of gold." - Sargon the scribe, near Baghdad first month, 2024 BC "My king, the Hittites have created scrolls that are wide and long despite that we burned their caravan taking palm leaves to the north. They have found a way to flense lambs to make pages themselves, and their ink is pure silver. They will soon have an army of Gollems and will destroy your empire. To stop this all I need is ten bags of gold." - Sargon the scribe, near Baghdad second month, 2024 BC "My king, alas all the gold has gone, and yet neither will the gollems walk, nor have the Hittities come. My dancing girls grow cold, and I have no figs for my table all I need is 20 bags of gold" - Sargon the scribe, near Baghdad third month, 2024 BC What happened to Sargon next is not recorded...

by sgt101 2y ago

qrsjutsu 2y ago

[dead]

CamperBob2 2y ago

Those are mostly structure-versus-function arguments, centered on what a brain is as opposed to what it does. Only the static nature of the current ML models seems like a valid point... and it's changing too. I'd wager that future models (which admittedly may not look much like today's LLMs) will blur the lines between mutable context and immutable weights. When that happens, all bets are truly off.

daveguy 2y ago
> future models (which admittedly may not look much like today's LLMs) will blur the lines between mutable context and immutable weights. "may not look much like today's LLMs" is really sweeping the whole point under the rug. The difference between what we are doing now with static LLM models and dynamic models will require algorithmic changes that have not yet been invented. That's in addition to the fact that the processing of a single neuron is completely parallel with respect to inputs. GPUs make them more parallel, but the analog devices we are trying to mimic are vastly more complicated than what we are using as a substitute.
- CamperBob2 2y ago
  Whatever. This ( https://eqbench.com/results/creative-writing-v2/deepseek-ai__DeepSeek-R1.txt https://eqbench.com/results/creative-writing-v2/deepseek-ai_... ) is intelligence in action, from an entity with an internal world model that's at least as valid as mine. Either that, or it's witchcraft. (Either that, or somebody with both time and talent to waste is taking the piss and pretending to be an AI.) If anyone ever does succeed in updating weights dynamically on a continuous basis, things are going to get really interesting really quickly. Even now, arguments based on relative complexity are completely invalid. A few fast neurons are at least as good at 'thinking' as a lot of slow ones are.
  - daveguy 2y ago
    Hah, "Whatever." Seriously? The writing is still derivative meandering pulp. No need to invoke magic. It's cool that it can make complete paragraphs though. LLMs are a huge breakthrough. They're just not intelligent. Updating weights on a continual basis requires processing weights productively on a regular basis. That's many flops per weight away from where we are in processing and bandwidth. The computation limitations still apply.
    - mrbungie 2y ago
      Metacommentary: I stopped arguing because this thread is reeking of AI bro non-sense, they can't argue with facts so they continue with vague statements and deflection (e.g like the "Whatever" you've got above). The GenAI bubble can't burst soon enough. As I said, neuroscience be damned, we've got transformers now. (/s)
      - sgt101 2y ago
        "My king, it's amazing these things can contain and regurgitate knowledge on demand. The next step is an army of Gollems that will destroy your enemies, all I need is three bags of gold." - Sargon the scribe, near Baghdad first month, 2024 BC "My king, the Hittites have created scrolls that are wide and long despite that we burned their caravan taking palm leaves to the north. They have found a way to flense lambs to make pages themselves, and their ink is pure silver. They will soon have an army of Gollems and will destroy your empire. To stop this all I need is ten bags of gold." - Sargon the scribe, near Baghdad second month, 2024 BC "My king, alas all the gold has gone, and yet neither will the gollems walk, nor have the Hittities come. My dancing girls grow cold, and I have no figs for my table all I need is 20 bags of gold" - Sargon the scribe, near Baghdad third month, 2024 BC What happened to Sargon next is not recorded...