2 ms·

> when they observe a human confidently says something wrong For the nth+1: well wired humans have built their systems trained on double checking and distrusti

by mdp2021 2y ago

> when they observe a human confidently says something wrong For the nth+1: well wired humans have built their systems trained on double checking and distrusting their own ideas.

viraptor 2y ago
And unsurprisingly, the reasoning models do better by introducing self-doubt/reflection and you can improve the response quality by getting multiple answers and auto-judging them. Maps pretty well onto the concept you describe.
- mdp2021 2y ago
  Sure! But we have even more work to do. E.g. -- Natural intelligences learn - after reflection, the mind is updated (hopefully upgraded): we need dynamic Neural Networks in the systems. -- Ideas honing, and the perfectioning of the World Model and cognitive systems behind them, should be iterative and continuous...
  - viraptor 2y ago
    https://arxiv.org/abs/2502.06049 https://arxiv.org/abs/2502.06049 But for most practical use cases we don't want the persistent memory. We already have opportunistic fact extraction like chatgpt profile building if we want that. We also have graph builders. This mostly covers the private usage. Public version was attempted and just ends up like this: https://en.wikipedia.org/wiki/Tay_(chatbot) https://en.wikipedia.org/wiki/Tay_(chatbot)
    - mdp2021 2y ago
      Thank you! This (LM2: Large Memory Models) is one article I will find the time to read properly, analytically. If you like the metaphors (which this article seems to come close to), you can have a sort of "unconscious" and a "conscious" - where the "unconscious" may be somehow similar to Kant's "trascendental" (interpreted as the mind's Operating System before experience). > Tay Well, if the bot becomes "opinionated", it's probably just immature - just like a human. You will tell it to ponder well texts like Adam Rutherford's How to Argue With a Racist... But surely we want an Intelligence that forms its own opinions: we want to understand and manage reality, so we should hope that a super-Chomsky and a super-Ferguson will at some point converge and at some point put their "identity"-related positions into perspective.
  - lostmsu 2y ago
    -- this is already how the LLMs are trained -- same
    - mdp2021 2y ago
      To my information, and following searches only days ago, the NNs encoding LLMs are said to be static. Whereas (for clarity) everytime one thinks he learns something and hones some skill.
      - lostmsu 2y ago
        The ones you can download, yes. But when they are trained the weights are obviously not static.
        mdp2021 2y ago
        Are you saying that some LLMs are in continuous training? E.g. "to read the news"? (I have not yet checked how the process works.) Were it so, anyway, it still does not seem like the process of "pondering" (that seems to be more like "absorbing information").
renox 2y ago
[flagged]

> when they observe a human confidently says something wrong For the nth+1: well wired humans have built their systems trained on double checking and distrusting their own ideas.

viraptor 2y ago

And unsurprisingly, the reasoning models do better by introducing self-doubt/reflection and you can improve the response quality by getting multiple answers and auto-judging them. Maps pretty well onto the concept you describe.

mdp2021 2y ago
Sure! But we have even more work to do. E.g. -- Natural intelligences learn - after reflection, the mind is updated (hopefully upgraded): we need dynamic Neural Networks in the systems. -- Ideas honing, and the perfectioning of the World Model and cognitive systems behind them, should be iterative and continuous...
- viraptor 2y ago
  https://arxiv.org/abs/2502.06049 https://arxiv.org/abs/2502.06049 But for most practical use cases we don't want the persistent memory. We already have opportunistic fact extraction like chatgpt profile building if we want that. We also have graph builders. This mostly covers the private usage. Public version was attempted and just ends up like this: https://en.wikipedia.org/wiki/Tay_(chatbot) https://en.wikipedia.org/wiki/Tay_(chatbot)
  - mdp2021 2y ago
    Thank you! This (LM2: Large Memory Models) is one article I will find the time to read properly, analytically. If you like the metaphors (which this article seems to come close to), you can have a sort of "unconscious" and a "conscious" - where the "unconscious" may be somehow similar to Kant's "trascendental" (interpreted as the mind's Operating System before experience). > Tay Well, if the bot becomes "opinionated", it's probably just immature - just like a human. You will tell it to ponder well texts like Adam Rutherford's How to Argue With a Racist... But surely we want an Intelligence that forms its own opinions: we want to understand and manage reality, so we should hope that a super-Chomsky and a super-Ferguson will at some point converge and at some point put their "identity"-related positions into perspective.
- lostmsu 2y ago
  -- this is already how the LLMs are trained -- same
  - mdp2021 2y ago
    To my information, and following searches only days ago, the NNs encoding LLMs are said to be static. Whereas (for clarity) everytime one thinks he learns something and hones some skill.
    - lostmsu 2y ago
      The ones you can download, yes. But when they are trained the weights are obviously not static.
      - mdp2021 2y ago
        Are you saying that some LLMs are in continuous training? E.g. "to read the news"? (I have not yet checked how the process works.) Were it so, anyway, it still does not seem like the process of "pondering" (that seems to be more like "absorbing information").

renox 2y ago

[flagged]