3 ms·

> You've been fooled by a next-token predictor. I also have a so called "pocket calculator" left over from when I went to school. Is this false? Have I been f

> You've been fooled by a next-token predictor. I also have a so called "pocket calculator" left over from when I went to school. Is this false? Have I been fooled by a little box of logic gates? That half-adder circuit in there is especially suss. It's really just manipulating 1s and 0s, but -and I've been explicitly told this- no one cares how it actually does it; so long as the truth table matches up. There is no understanding of mathematics going on. There is no single transistor in the whole thing that knows how to do so much as add 1+1. If I put it in the chinese room, I still wouldn't know how it did it. Clearly the entire premise must be false! ;-)

Kim_Bruning 2mo ago
I'm going to add this as a separate comment: These kinds of stories probably read very differently for someone who uses Opus and Fable agents all day and goes "ohhh, I saw this in miniature last week; this and this and this must have happened" , vs someone who tried free-tier Gemini flash one rainy Sunday, got hallucinated at, and concludes it must all be a scam.
- chrisjj 2mo ago
  Probably everything reads very differently to someone who talks to chatbots all day.
  - Kim_Bruning 2mo ago
    > Probably everything reads very differently to someone who talks to chatbots all day. A chatbot is a particular kind of harness. Typically an LLM driving a chatbot won't be able to hack very much. So we agree, someone who talks to bad chatbots all day probably has a very different view of SOTA agents. :-P
  - infinite_spin 2mo ago
    Please stop dropping backhanded insults to other people here. It's not productive and violates guidelines.
chrisjj 2mo ago
> That half-adder circuit in there is especially suss. It's really just manipulating 1s and 0s, but -and I've been explicitly told this- no one cares how it actually does it; so long as the truth table matches up. "so long as the truth table matches up." Yup. Now try getting your chatbot's output to match up. Your calculator was designed to tell truth. Your chatbot was designed to tell a mash up of whatever its creators managed to scrape from the internet.
- deleted 2mo ago
  [deleted]
- wongarsu 2mo ago
  LLMs are very explicitly designed to "understand, and make judgments or have opinions that are based on reason". The learning part is debatable, as is the level of success achieved The mash-up of the entire internet is the mechanism by which they attempt to achieve the goal, not the goal itself. And it's only the first training step
  - chrisjj 2mo ago
    > LLMs are very explicitly designed to "understand, and make judgments or have opinions that are based on reason". I think you've mistaken the sales pitch for the design. Not even the enclopedia anyone can edit comes remotely near that: "A large language model (LLM) is an AI model (typically a neural network) trained on a vast amount of text for natural language processing tasks, especially language generation. LLMs can typically generate, summarize, translate, and analyze text in many contexts.[1] They are the basis for many modern chatbots, such as ChatGPT, Claude, Gemini, Grok, and DeepSeek. LLMs are typically based on transformer architecture.[2] Generative pre-trained transformers (GPTs) are a type of LLM that is pre-trained to predict the next word.[3] GPTs are then often fine-tuned to follow instructions and to behave as assistants.[4] Biased or inaccurate training data can make an LLM's output less reliable. Benchmark evaluations for LLMs attempt to measure model reasoning, factual accuracy, alignment, and safety."
    - Kim_Bruning 2mo ago

Kim_Bruning 2mo ago

I'm going to add this as a separate comment: These kinds of stories probably read very differently for someone who uses Opus and Fable agents all day and goes "ohhh, I saw this in miniature last week; this and this and this must have happened" , vs someone who tried free-tier Gemini flash one rainy Sunday, got hallucinated at, and concludes it must all be a scam.

chrisjj 2mo ago
Probably everything reads very differently to someone who talks to chatbots all day.
- Kim_Bruning 2mo ago
  > Probably everything reads very differently to someone who talks to chatbots all day. A chatbot is a particular kind of harness. Typically an LLM driving a chatbot won't be able to hack very much. So we agree, someone who talks to bad chatbots all day probably has a very different view of SOTA agents. :-P
- infinite_spin 2mo ago
  Please stop dropping backhanded insults to other people here. It's not productive and violates guidelines.

chrisjj 2mo ago

> That half-adder circuit in there is especially suss. It's really just manipulating 1s and 0s, but -and I've been explicitly told this- no one cares how it actually does it; so long as the truth table matches up. "so long as the truth table matches up." Yup. Now try getting your chatbot's output to match up. Your calculator was designed to tell truth. Your chatbot was designed to tell a mash up of whatever its creators managed to scrape from the internet.

deleted 2mo ago
[deleted]
wongarsu 2mo ago
LLMs are very explicitly designed to "understand, and make judgments or have opinions that are based on reason". The learning part is debatable, as is the level of success achieved The mash-up of the entire internet is the mechanism by which they attempt to achieve the goal, not the goal itself. And it's only the first training step
- chrisjj 2mo ago
  > LLMs are very explicitly designed to "understand, and make judgments or have opinions that are based on reason". I think you've mistaken the sales pitch for the design. Not even the enclopedia anyone can edit comes remotely near that: "A large language model (LLM) is an AI model (typically a neural network) trained on a vast amount of text for natural language processing tasks, especially language generation. LLMs can typically generate, summarize, translate, and analyze text in many contexts.[1] They are the basis for many modern chatbots, such as ChatGPT, Claude, Gemini, Grok, and DeepSeek. LLMs are typically based on transformer architecture.[2] Generative pre-trained transformers (GPTs) are a type of LLM that is pre-trained to predict the next word.[3] GPTs are then often fine-tuned to follow instructions and to behave as assistants.[4] Biased or inaccurate training data can make an LLM's output less reliable. Benchmark evaluations for LLMs attempt to measure model reasoning, factual accuracy, alignment, and safety."
  - Kim_Bruning 2mo ago