11 ms·

Starting to feel more and more like chinese room experiment The models are confidently answering physics questions, treating it as a math problem, but they don

Starting to feel more and more like chinese room experiment The models are confidently answering physics questions, treating it as a math problem, but they don't fundamentally "get it" and even recently failed simple "should i drive to car wash" test The sample efficiency is just crazy low Still surprising that even with this they managed to saturate the benchmarks

red75prime 12d ago
> they don't fundamentally "get it" There's no clear decision criteria for this. Do trick questions demonstrate that most people don't "get it"? And, well, older model saying dumb things doesn't establish a general principle that LLMs don't "get it" in general. > The sample efficiency is just crazy low Autoregressive pretraining requires huge amount of data to go from a blank state to a somewhat functional model. Fine-tuning, LORA, reinforcement learning of foundation models and in-context learning are much more sample efficient. > Chinese room ...creates a wrong intuition that by cranking a Leibniz's mill you are somehow responsible for whether it understands something or not.
- rogerrogerr 12d ago
  > Do trick questions demonstrate that most people don't "get it"? "Should I walk to the car wash" is hardly a trick question. If a human told me to walk to the car wash because it's so close, I would say that demonstrates they don't "get it".
  - red75prime 12d ago
    I don't see that much difference with "A plane crashes on the border of the United States and Canada. Where do they bury the survivors?" Fast thinking fails to activate slow thinking.
webern777 11d ago
Searle's thought experiment just kicks the can down the road. Since the room as a whole produces fluent Chinese it becomes a language philosophy debate about where meaning comes from. Brandom's inferentialism gets into this more than any normal person would ever care to read about. LLMs really show the absurdity of the Chinese Room rulebook. The whole thought experiment should really be left to history at this point. Searle’s social ontology is much more interesting also but this outdated thought experiment completely overshadows his more interesting stuff.

red75prime 12d ago

> they don't fundamentally "get it" There's no clear decision criteria for this. Do trick questions demonstrate that most people don't "get it"? And, well, older model saying dumb things doesn't establish a general principle that LLMs don't "get it" in general. > The sample efficiency is just crazy low Autoregressive pretraining requires huge amount of data to go from a blank state to a somewhat functional model. Fine-tuning, LORA, reinforcement learning of foundation models and in-context learning are much more sample efficient. > Chinese room ...creates a wrong intuition that by cranking a Leibniz's mill you are somehow responsible for whether it understands something or not.

rogerrogerr 12d ago
> Do trick questions demonstrate that most people don't "get it"? "Should I walk to the car wash" is hardly a trick question. If a human told me to walk to the car wash because it's so close, I would say that demonstrates they don't "get it".
- red75prime 12d ago
  I don't see that much difference with "A plane crashes on the border of the United States and Canada. Where do they bury the survivors?" Fast thinking fails to activate slow thinking.

webern777 11d ago

Searle's thought experiment just kicks the can down the road. Since the room as a whole produces fluent Chinese it becomes a language philosophy debate about where meaning comes from. Brandom's inferentialism gets into this more than any normal person would ever care to read about. LLMs really show the absurdity of the Chinese Room rulebook. The whole thought experiment should really be left to history at this point. Searle’s social ontology is much more interesting also but this outdated thought experiment completely overshadows his more interesting stuff.