3 ms·

BlueSky version: https://bsky.app/profile/thomwolf.bsky.social/post/3ljpkl6c6ss24 https://bsky.app/profile/thomwolf.bsky.social/post/3ljpkl6c6... --- Quite in

BlueSky version: https://bsky.app/profile/thomwolf.bsky.social/post/3ljpkl6c6ss24 https://bsky.app/profile/thomwolf.bsky.social/post/3ljpkl6c6... --- Quite interesting post that asks the right question about "asking the right questions". Yet one aspect I felt missing (which might automatically solve this) is first-principles-based causal reasoning. A truly intelligent system — one that reasons from first principles by running its own simulations and physical experiments — would notice if something doesn't align with the "textbook version". It would recognize when reality deviates from expectations and ask follow-up questions, naturally leading to deeper insights and the right questions - and answers. Fascinating in this space is the new "Reasoning-Prior" approach (MIT Lab & Harvard), which trains reasoning capabilities learned from the physical world as a foundation for new models (before evening learning about text). Relevant paper: "General Reasoning Requires Learning to Reason from the Get-go."

mentalgear 2y ago
PS: great explainer video https://www.youtube.com/watch?v=seTdudcs-ws&t=180s https://www.youtube.com/watch?v=seTdudcs-ws&t=180s
zombot 2y ago
"Reality" is an alien concept to an LLM. All they have is sequences of words that they can complete.
- reverius42 2y ago
  "Reality" is an alien concept to a Human. All they have is sequences of motions and noises they can complete.
  - neom 2y ago
    I explained to someone mum is mum because of lips, air and sounds and they didn't believe me, so that's what we're contending with....
- herculity275 2y ago
  I used to think that but newer models' increasing coherence coupled with the astonishing success of RLHF makes me think that LLMs that can effectively intuit "reality" will emerge fairly soon.
- alex77456 2y ago
  Maybe we're overestimating average human brain capabilities
  - chezelenkoooo 2y ago
    I think the evidence of all human progress disagrees with this statement fairly strongly.
    - alex77456 2y ago
      Human progress is driven by a small percentage of the population
      - sage92 2y ago
        The human body is driven by a small percentage of the overall genome. It remains to be seen if that small percentage really doesn’t play a part… we tend to remember those who scored the goal, but often forget about what it took for the scorer to have a shot in the first place…
antonkar 2y ago
Interesting, I think the guy who’ll make the GUI for LLMs is the next Jobs/Gates/Musk and Nobel Prize Winner (I think it’ll solve alignment by having millions of eyes on the internals of LLMs), because computers became popular only after the OS with a GUI appeared. I just started ASK HN to let people share their AI safety ideas, both crazy and not: https://news.ycombinator.com/item?id=43332593 https://news.ycombinator.com/item?id=43332593

mentalgear 2y ago

PS: great explainer video https://www.youtube.com/watch?v=seTdudcs-ws&t=180s https://www.youtube.com/watch?v=seTdudcs-ws&t=180s

zombot 2y ago

"Reality" is an alien concept to an LLM. All they have is sequences of words that they can complete.

reverius42 2y ago
"Reality" is an alien concept to a Human. All they have is sequences of motions and noises they can complete.
- neom 2y ago
  I explained to someone mum is mum because of lips, air and sounds and they didn't believe me, so that's what we're contending with....
herculity275 2y ago
I used to think that but newer models' increasing coherence coupled with the astonishing success of RLHF makes me think that LLMs that can effectively intuit "reality" will emerge fairly soon.
alex77456 2y ago
Maybe we're overestimating average human brain capabilities
- chezelenkoooo 2y ago
  I think the evidence of all human progress disagrees with this statement fairly strongly.
  - alex77456 2y ago
    Human progress is driven by a small percentage of the population
    - sage92 2y ago
      The human body is driven by a small percentage of the overall genome. It remains to be seen if that small percentage really doesn’t play a part… we tend to remember those who scored the goal, but often forget about what it took for the scorer to have a shot in the first place…

antonkar 2y ago

Interesting, I think the guy who’ll make the GUI for LLMs is the next Jobs/Gates/Musk and Nobel Prize Winner (I think it’ll solve alignment by having millions of eyes on the internals of LLMs), because computers became popular only after the OS with a GUI appeared. I just started ASK HN to let people share their AI safety ideas, both crazy and not: https://news.ycombinator.com/item?id=43332593 https://news.ycombinator.com/item?id=43332593