10 ms·

Large models of what? Mistaking engineering achievements for linguistic agency

184 points by Anon84 2y ago

mnkv 2y ago
Good summary of some of the main "theoretical" criticism of LLMs but I feel that it's a bit dated and ignores the recent trend of iterative post-training, especially with human feedback. Major chatbots are no doubt being iteratively refined on the feedback from users i.e. interaction feedback, RLHF, RLAIF. So ChatGPT could fall within the sort of "enactive" perspective on language and definitely goes beyond the issues of static datasets and data completeness. Sidenote: the authors make a mistake when citing Wittgenstein to find similarity between humans and LLMs. Language modelling on a static dataset is mostly not a language game (see Bender and Koller's section on distributional semantics and caveats on learning meaning from "control codes")
- dartos 2y ago
  FWIW even more recently, models have been tuned using a method called DPO instead of RLHF. IIRC DPO doesn’t have human feedback in the loop
  - valec 2y ago
    it does. that's what the "direct preference" part of DPO means. you just avoid training an explicit reward model on it like in rlhf and instead directly optimize for log probability of preferred vs dispreferred responses
    - meroes 2y ago
      What is it called when humans interact with a model through lengthy exchanges (mostly humans correcting the model’s responses to a posed question to the model, mostly through chat and labeling each statement by the model as correct or not), and then all of that text (possibly with some editing) is fed to another model to train that higher model? Does this have a specific name?
      - dartos 2y ago
        I don’t think that process has a specific name. It’s just how training these models works. Conversations you have with like chatgpt are likely stored, then sorted through somehow, then added to an ever growing dataset of conversations that would be used to train entirely new models.
  - hackernewds 2y ago
    DPO most essentially has human feedback, depends on what the preference optimizations are
mistrial9 2y ago
oh what a kettle of worms here... Now the mind must consider "repetitive speech under pressure and in formal situations" in contrast and comparison to "limited mechanical ability to produce grammatic sequences of well-known words" .. where is the boundary there? I am a fan of this paper, warts and all ! (and the paper summary paragraph contained some atrocious grammar btw)
Animats 2y ago
Full paper: [1]. Not much new here. The basic criticism is that LLMs are not embodied; they have no interaction with the real world. The same criticism can be applied to most office work. Useful insight: "We (humans) are always doing more than one thing." This is in the sense of language output having goals for the speaker, not just delivering information. This is related to the problem of LLMs losing the thread of a conversation. Probably the only reasonably new concept in this paper. Standard rant: "Humans are not brains that exist in a vat..." "LLMs ... have nothing at stake." Arguable, in that some LLMs are trained using punishment. Which seems to have strong side effects. The undesirable behavior is suppressed, but so is much other behavior. That's rather human-like. "LLMs Don’t Algospeak". The author means using word choices to get past dumb censorship algorithms. That's probably do-able, if anybody cares. [1] https://arxiv.org/pdf/2407.08790 https://arxiv.org/pdf/2407.08790
- ainoobler 2y ago
  The optimization process adjusts the weights of a computational graph until the numeric outputs align with some baseline statistics of a large data set. There is no "punishment" or "reward", gradient descent isn't even necessary as there are methods for modifying the weights in other ways and the optimization still converges to a desired distribution which people claim is "intelligent". The converse is that people are "just" statistical distributions of the signals produced by them but I don't know if there are people who claim they are nothing more than statistical distributions. I think people are confused because they do not really understand how software and computers work. I'd say they should learn some computability theory to gain some clarity but I doubt they'd listen.
  - bubblyworld 2y ago
    If you really want to phrase it that way, organisms like us are "just" distributions of genes that have been pushed this way and that by natural selection until they converged to something we consider intelligent (humans). It's pretty clear that these optimisation processes lead to emergent behaviour, both in ML and in the natural sciences. Computability theory isn't really relevant here.
KHRZ 2y ago
That's a lot of thinking they've done about LLMs, but how much did they actually try LLMs? I have long threads where ChatGPT refine solutions to coding problems. Their example of losing the thread after printing a tiny list of 10 philosophers seems really outdated. Also it seems LLMs utilize nested contexts as well, for example when it can break it' own rules while telling a story or speaking hypothetically.
- tkgally 2y ago
  For a paper submitted on July 11, 2024, and with several references to other 2024 publications, it is indeed strange that it gives ChatGPT output from April 2023 to demonstrate that “LLMs lose the thread of a conversation with inhuman ease, as outputs are generated in response to prompts rather than a consistent, shared dialogue” (Figure 1). I have had many consistent, shared dialogues with recent versions of ChatGPT and Claude without any loss of conversation thread even after many back-and-forths.
- Der_Einzige 2y ago
  Most LLM critics (and singularity-is-near influencers) don't actually use the systems enough to have relevant opinions about them. The only really good sources of truth is the chatbot-arena from lmsys and the comment section of r/localllama (I'm quoting Karpathy), both are "wisdom of the crowd" and often the crowd on r/localllama is getting that wisdom by spending hours with one hand on the keyboard and another under their clothes.
GeneralMayhem 2y ago
I am highly skeptical of LLMs as a mechanism to achieve AGI, but I also find this paper fairly unconvincing, bordering on tautological. I feel similarly about this as to what I've read of Chalmers - I agree with pretty much all of the conclusions, but I don't feel like the text would convince me of those conclusions if I disagreed; it's more like it's showing me ways of explaining or illustrating what I already believed. On embodiment - yes, LLMs do not have corporeal experience. But it's not obvious that this means that they cannot, a priori, have an "internal" concept of reality, or that it's impossible to gain such an understanding from text. The argument feels circular: LLMs are similar to a fake "video game" world because they aren't real people - therefore, it's wrong to think that they could be real people? And the other half of the argument is that because LLMs can only see text, they're missing out on the wider world of non-textual communication; but then, does that mean that human writing is not "real" language? This argument feels especially weak in the face of multi-modal models that are in fact able to "see" and "hear". The other flavor of argument here is that LLM behavior is empirically non-human - e.g., the argument about not asking for clarification. But that only means that they aren't currently matching humans, not that they couldn't. Basically all of these arguments feel like they fall down to the strongest counterargument I see proposed by LLM-believers, which is that sufficiently advanced mimicry is not only indistinguishable from the real thing, but at the limit in fact is the real thing. If we say that it's impossible to have true language skills without implicitly having a representation of self and environment, and then we see an entity with what appears to be true language skills, we should conclude that that entity must contain within it a representation of self and environment. That argument doesn't rely on any assumptions about the mechanism of representation other than a reliance on physicalism. Looking at it from the other direction, if you assume that all that it means to "be human" is encapsulated in the entropy of a human body, then that concept is necessarily describable with finite entropy. Therefore, by extension, there must be some number of parameters and some model architecture that completely encode that entropy. Questions like whether LLMs are the perfect architecture or whether the number of parameters required is a number that can be practically stored on human-manufacturable media are engineering questions, not philosophical ones: finite problems admit finite solutions, full stop. Again, that conclusion feels wrong to me... but if I'm being honest with myself, I can't point to why, other than to point at some form of dualism or spirituality as the escape hatch.
dboreham 2y ago
The first stage is denial.
- nativeit 2y ago
  Well, I suppose that's rather convenient.
nativeit 2y ago
I'm more or less a layperson when it comes to LLMs and this nascent concept of AI, but there's one argument that I keep seeing that I feel like I understand, even without a thorough fluency with the underlying technology. I know that neural nets, and the mechanisms LLMs employ to train and form relational connections, can plausibly be compared to how synapses form signal paths between neurons. I can see how that makes intuitive sense. I'm struggling to articulate my cognitive dissonance here, but is there any empirical evidence that LLMs, or their underlying machine learning technology, share anything at all with biological consciousness beyond a convenient metaphor for describing "neural networks" using terms borrowed from neuroscience? I don't know that it necessarily follows that just because something was inspired by, or is somehow mimicking, the structure of the brain and its basic elements, that it should necessarily relate to its modeled reality in any literal way, let alone provide a sufficient basis for instantiating a phenomena we frankly know very little about. Not for nothing, but our models naturally cannot replicate any biological functions we do not fully understand. We haven't managed to reproduce biological tissues that are exponentially less complex than the brain, are we really claiming that we're just jumping straight past lab-grown t-bones to intelligent minds? I'm sure most of the people reading this will have seen Matt Parker's videos where they "teach" matchbooks to win a game against humans. Is anyone suggesting those matchbooks, given infinite time and repetition, would eventually spark emergent consciousness? > The argument would be that that conceptual model is encoded in the intermediate-layer parameters of the model, in a different but analogous way to how it's encoded in the graph and chemical structure of your neurons. Sorry if I have misinterpreted anyone. I honestly thought all the "neuron" and "synapse" references were handy metaphors to explain otherwise complex computations that resemble this conceptual idea of how our brains work. But it reads a lot like some of the folks in this thread believe it's much more than metaphors, but rather a literal analog.
- obirunda 2y ago
  I don't think anyone in research actually believes this. Note that the whole idea behind claiming "scaling laws" will infinitely improve these models is a funding strategy rather than a research one. None of these folks think human-like consciousness will "rise" from this effort, even though they veil it to continue the hype-cycle. I guarantee all these firms are desperately looking for architectural breakthroughs, even while they wax poetic about scaling laws, they know there is a bottleneck ahead. Notice how LeCun is the only researcher being honest about this in a public fashion. Meta is committed to AI already and will at least match the spend of competitors anyway, so he doesn't have as much pressure to try and convince investors that this rabbit whole is deeper. Don't get me wrong, LLMs are a tremendous improvement on knowledge compression and distillation, but it's still unreliable enough that old school search is likely a superior method nonetheless.
kazinator 2y ago
The authors of this paper are just another instance of the AI hype being used by people who have no connection to it, to attract some kind of attention. "Here is what we think about this current hot topic; please read our stuff and cite generously ..." > Language completeness assumes that a distinct and complete thing such as `a natural language' exists, the essential characteristics of which can be effectively and comprehensively modelled by an LLM Replace "LLM" by "linguistics". Same thing. > The assumption of data completeness relies on the belief that a language can be quantified and wholly captured by data. That's all that a baby has, who becomes a native speaker of their surrounding language. Language acquisition does not imply totality of data. Not every native speaker recognizes exactly the same vocabulary and exactly the same set of grammar rules.
- IshKebab 2y ago
  Babies have feedback and interaction with someone speaking to them. Would they learn to speak if you just dumped them in front of a TV and never spoke to them? I'm not sure. But anyway I agree with you. This is just a confused HN comment in paper form.
  - xpe 2y ago
    I personally don’t get much value out of the paper, but it is orders of magnitude more substantive and thoughtful than a median “confused Hacker News comment”.
  - keybored 2y ago
    > Babies have feedback and interaction with someone speaking to them. Would they learn to speak if you just dumped them in front of a TV and never spoke to them? I'm not sure. Feedback and interaction is not vital for acquisition for secondary language learning at least according to one theory. And if that’s good enough for adults it might be good enough for sponge-brain babies. https://en.wikipedia.org/wiki/Input_hypothesis https://en.wikipedia.org/wiki/Input_hypothesis
- JohnKemeny 2y ago
  They are two researchers/assistant professors working with cognitive science, psychology, and trustworthy AI. The paper is peer reviewed and has been accepted for publication in the Journal of Language Sciences. You should publish your critique of their research in that same journal. P.s. if you find any grave mistakes, you can contact the editor in chief, who happens to be a linguist.
amne 2y ago
tl:dr; we're duck-typing LLMs as AGI
Simon_ORourke 2y ago
Where I work, there's a somewhat haphazardly divided org structure, where my team has some responsibility to answer the executives demands for "use AI to help our core business". So we applied off-the-shelf models to extract structured context from mostly unstructured text - effectively a data engineering job - and thereby support analytics and create more dashboards for the execs to mull over. Another team, with a similar role in a different part of the org has jumped (feet first) into optimizing large language models to turn them into agents, without consulting the business about whether they need such things. RAG, LoRA and all this optimization is well and good, but this engineering focus has found no actual application, expect wasting several million bucks hiring staff to do something nobody wants.
rramadass 2y ago
See also Beyond the Hype: A Realistic Look at Large Language Models • Jodie Burchell • GOTO 2024 - https://www.youtube.com/watch?v=Pv0cfsastFs https://www.youtube.com/watch?v=Pv0cfsastFs
flimflamm 2y ago
How would the authors consider a paralyzed individual who can only move their eyes since birth? That person can learn the same concepts as other humans and communicate as richly (using only their eyes) as other humans. Clearly, the paper is viewing the problem very narrowly.
- fairthomas 2y ago
  > ...a paralyzed individual who can only move their eyes since birth... I don't think such an individual is possible.
  - throwthrowuknow 2y ago
    I didn’t want to Google it for you because it always makes me sad but things like spina bifida and moebius syndrome exist. Not everyone gets to begin life healthy.
throwthrowuknow 2y ago
“Enactivism” really? I wonder if these complaints will continue as LLMs see wider adoption, the old first they ignore you, then they ridicule you, then they fight you… trope that is halfways accurate. Any field that focuses on building theories on top of theories is in for a bad time. https://en.m.wikipedia.org/wiki/Enactivism https://en.m.wikipedia.org/wiki/Enactivism
- beepbooptheory 2y ago
  What is the thing the LLMs are fighting for?
beepbooptheory 2y ago
There is a lot of frustration here over what appears to be essentially this claim: > ...we argue that it is possible to offer generous interpretations of some aspects of LLM engineering to find parallels with human language learning. However, in the majority of key aspects of language learning and use, most specifically in the various kinds of linguistic agency exhibited by human beings, these small apparent comparisons do little to balance what are much more deep-rooted contrasts. Now, why is this so hard to stomach? This is the argument of this paper. To feel like this extremely general claim is something you have to argue against means you believe in a fundamental similarity between what our linguistic agency and the model. But is embodied human agency something that you really need the LLMs to have right now? Why? What are the stakes here? The ones actually related to the argument at hand? This ultimately not that strong of a claim! To the point that its almost vacuous... Of course the LLM will never learn the stove is "hot" like you did when you were a curious child. How can this still be too much to admit for someone? What is lost? It makes me feel little crazy here that people constantly jump over the text at hand whenever something gets a little too philosophical, and the arguments become long pseudo-theories that aren't relevant to argument.
Royshiloh 2y ago
Why assume you "know" what language is? Like there is a study backed insight on the ultimate definition of language? it's the same as saying "oh, it's not 'a,b,c' its 'x,y,z'", which makes you as dogmatic as the one you critique. This is absurd.

184 points by Anon84 2y ago

mnkv 2y ago

Good summary of some of the main "theoretical" criticism of LLMs but I feel that it's a bit dated and ignores the recent trend of iterative post-training, especially with human feedback. Major chatbots are no doubt being iteratively refined on the feedback from users i.e. interaction feedback, RLHF, RLAIF. So ChatGPT could fall within the sort of "enactive" perspective on language and definitely goes beyond the issues of static datasets and data completeness. Sidenote: the authors make a mistake when citing Wittgenstein to find similarity between humans and LLMs. Language modelling on a static dataset is mostly not a language game (see Bender and Koller's section on distributional semantics and caveats on learning meaning from "control codes")

dartos 2y ago
FWIW even more recently, models have been tuned using a method called DPO instead of RLHF. IIRC DPO doesn’t have human feedback in the loop
- valec 2y ago
  it does. that's what the "direct preference" part of DPO means. you just avoid training an explicit reward model on it like in rlhf and instead directly optimize for log probability of preferred vs dispreferred responses
  - meroes 2y ago
    What is it called when humans interact with a model through lengthy exchanges (mostly humans correcting the model’s responses to a posed question to the model, mostly through chat and labeling each statement by the model as correct or not), and then all of that text (possibly with some editing) is fed to another model to train that higher model? Does this have a specific name?
    - dartos 2y ago
      I don’t think that process has a specific name. It’s just how training these models works. Conversations you have with like chatgpt are likely stored, then sorted through somehow, then added to an ever growing dataset of conversations that would be used to train entirely new models.
- hackernewds 2y ago
  DPO most essentially has human feedback, depends on what the preference optimizations are

mistrial9 2y ago

oh what a kettle of worms here... Now the mind must consider "repetitive speech under pressure and in formal situations" in contrast and comparison to "limited mechanical ability to produce grammatic sequences of well-known words" .. where is the boundary there? I am a fan of this paper, warts and all ! (and the paper summary paragraph contained some atrocious grammar btw)

Animats 2y ago

Full paper: [1]. Not much new here. The basic criticism is that LLMs are not embodied; they have no interaction with the real world. The same criticism can be applied to most office work. Useful insight: "We (humans) are always doing more than one thing." This is in the sense of language output having goals for the speaker, not just delivering information. This is related to the problem of LLMs losing the thread of a conversation. Probably the only reasonably new concept in this paper. Standard rant: "Humans are not brains that exist in a vat..." "LLMs ... have nothing at stake." Arguable, in that some LLMs are trained using punishment. Which seems to have strong side effects. The undesirable behavior is suppressed, but so is much other behavior. That's rather human-like. "LLMs Don’t Algospeak". The author means using word choices to get past dumb censorship algorithms. That's probably do-able, if anybody cares. [1] https://arxiv.org/pdf/2407.08790 https://arxiv.org/pdf/2407.08790

ainoobler 2y ago
The optimization process adjusts the weights of a computational graph until the numeric outputs align with some baseline statistics of a large data set. There is no "punishment" or "reward", gradient descent isn't even necessary as there are methods for modifying the weights in other ways and the optimization still converges to a desired distribution which people claim is "intelligent". The converse is that people are "just" statistical distributions of the signals produced by them but I don't know if there are people who claim they are nothing more than statistical distributions. I think people are confused because they do not really understand how software and computers work. I'd say they should learn some computability theory to gain some clarity but I doubt they'd listen.
- bubblyworld 2y ago
  If you really want to phrase it that way, organisms like us are "just" distributions of genes that have been pushed this way and that by natural selection until they converged to something we consider intelligent (humans). It's pretty clear that these optimisation processes lead to emergent behaviour, both in ML and in the natural sciences. Computability theory isn't really relevant here.

KHRZ 2y ago

That's a lot of thinking they've done about LLMs, but how much did they actually try LLMs? I have long threads where ChatGPT refine solutions to coding problems. Their example of losing the thread after printing a tiny list of 10 philosophers seems really outdated. Also it seems LLMs utilize nested contexts as well, for example when it can break it' own rules while telling a story or speaking hypothetically.

tkgally 2y ago
For a paper submitted on July 11, 2024, and with several references to other 2024 publications, it is indeed strange that it gives ChatGPT output from April 2023 to demonstrate that “LLMs lose the thread of a conversation with inhuman ease, as outputs are generated in response to prompts rather than a consistent, shared dialogue” (Figure 1). I have had many consistent, shared dialogues with recent versions of ChatGPT and Claude without any loss of conversation thread even after many back-and-forths.
Der_Einzige 2y ago
Most LLM critics (and singularity-is-near influencers) don't actually use the systems enough to have relevant opinions about them. The only really good sources of truth is the chatbot-arena from lmsys and the comment section of r/localllama (I'm quoting Karpathy), both are "wisdom of the crowd" and often the crowd on r/localllama is getting that wisdom by spending hours with one hand on the keyboard and another under their clothes.

GeneralMayhem 2y ago

I am highly skeptical of LLMs as a mechanism to achieve AGI, but I also find this paper fairly unconvincing, bordering on tautological. I feel similarly about this as to what I've read of Chalmers - I agree with pretty much all of the conclusions, but I don't feel like the text would convince me of those conclusions if I disagreed; it's more like it's showing me ways of explaining or illustrating what I already believed. On embodiment - yes, LLMs do not have corporeal experience. But it's not obvious that this means that they cannot, a priori, have an "internal" concept of reality, or that it's impossible to gain such an understanding from text. The argument feels circular: LLMs are similar to a fake "video game" world because they aren't real people - therefore, it's wrong to think that they could be real people? And the other half of the argument is that because LLMs can only see text, they're missing out on the wider world of non-textual communication; but then, does that mean that human writing is not "real" language? This argument feels especially weak in the face of multi-modal models that are in fact able to "see" and "hear". The other flavor of argument here is that LLM behavior is empirically non-human - e.g., the argument about not asking for clarification. But that only means that they aren't currently matching humans, not that they couldn't. Basically all of these arguments feel like they fall down to the strongest counterargument I see proposed by LLM-believers, which is that sufficiently advanced mimicry is not only indistinguishable from the real thing, but at the limit in fact is the real thing. If we say that it's impossible to have true language skills without implicitly having a representation of self and environment, and then we see an entity with what appears to be true language skills, we should conclude that that entity must contain within it a representation of self and environment. That argument doesn't rely on any assumptions about the mechanism of representation other than a reliance on physicalism. Looking at it from the other direction, if you assume that all that it means to "be human" is encapsulated in the entropy of a human body, then that concept is necessarily describable with finite entropy. Therefore, by extension, there must be some number of parameters and some model architecture that completely encode that entropy. Questions like whether LLMs are the perfect architecture or whether the number of parameters required is a number that can be practically stored on human-manufacturable media are engineering questions, not philosophical ones: finite problems admit finite solutions, full stop. Again, that conclusion feels wrong to me... but if I'm being honest with myself, I can't point to why, other than to point at some form of dualism or spirituality as the escape hatch.

dboreham 2y ago

The first stage is denial.

nativeit 2y ago
Well, I suppose that's rather convenient.

nativeit 2y ago

I'm more or less a layperson when it comes to LLMs and this nascent concept of AI, but there's one argument that I keep seeing that I feel like I understand, even without a thorough fluency with the underlying technology. I know that neural nets, and the mechanisms LLMs employ to train and form relational connections, can plausibly be compared to how synapses form signal paths between neurons. I can see how that makes intuitive sense. I'm struggling to articulate my cognitive dissonance here, but is there any empirical evidence that LLMs, or their underlying machine learning technology, share anything at all with biological consciousness beyond a convenient metaphor for describing "neural networks" using terms borrowed from neuroscience? I don't know that it necessarily follows that just because something was inspired by, or is somehow mimicking, the structure of the brain and its basic elements, that it should necessarily relate to its modeled reality in any literal way, let alone provide a sufficient basis for instantiating a phenomena we frankly know very little about. Not for nothing, but our models naturally cannot replicate any biological functions we do not fully understand. We haven't managed to reproduce biological tissues that are exponentially less complex than the brain, are we really claiming that we're just jumping straight past lab-grown t-bones to intelligent minds? I'm sure most of the people reading this will have seen Matt Parker's videos where they "teach" matchbooks to win a game against humans. Is anyone suggesting those matchbooks, given infinite time and repetition, would eventually spark emergent consciousness? > The argument would be that that conceptual model is encoded in the intermediate-layer parameters of the model, in a different but analogous way to how it's encoded in the graph and chemical structure of your neurons. Sorry if I have misinterpreted anyone. I honestly thought all the "neuron" and "synapse" references were handy metaphors to explain otherwise complex computations that resemble this conceptual idea of how our brains work. But it reads a lot like some of the folks in this thread believe it's much more than metaphors, but rather a literal analog.

obirunda 2y ago
I don't think anyone in research actually believes this. Note that the whole idea behind claiming "scaling laws" will infinitely improve these models is a funding strategy rather than a research one. None of these folks think human-like consciousness will "rise" from this effort, even though they veil it to continue the hype-cycle. I guarantee all these firms are desperately looking for architectural breakthroughs, even while they wax poetic about scaling laws, they know there is a bottleneck ahead. Notice how LeCun is the only researcher being honest about this in a public fashion. Meta is committed to AI already and will at least match the spend of competitors anyway, so he doesn't have as much pressure to try and convince investors that this rabbit whole is deeper. Don't get me wrong, LLMs are a tremendous improvement on knowledge compression and distillation, but it's still unreliable enough that old school search is likely a superior method nonetheless.

kazinator 2y ago

The authors of this paper are just another instance of the AI hype being used by people who have no connection to it, to attract some kind of attention. "Here is what we think about this current hot topic; please read our stuff and cite generously ..." > Language completeness assumes that a distinct and complete thing such as `a natural language' exists, the essential characteristics of which can be effectively and comprehensively modelled by an LLM Replace "LLM" by "linguistics". Same thing. > The assumption of data completeness relies on the belief that a language can be quantified and wholly captured by data. That's all that a baby has, who becomes a native speaker of their surrounding language. Language acquisition does not imply totality of data. Not every native speaker recognizes exactly the same vocabulary and exactly the same set of grammar rules.

IshKebab 2y ago
Babies have feedback and interaction with someone speaking to them. Would they learn to speak if you just dumped them in front of a TV and never spoke to them? I'm not sure. But anyway I agree with you. This is just a confused HN comment in paper form.
- xpe 2y ago
  I personally don’t get much value out of the paper, but it is orders of magnitude more substantive and thoughtful than a median “confused Hacker News comment”.
- keybored 2y ago
  > Babies have feedback and interaction with someone speaking to them. Would they learn to speak if you just dumped them in front of a TV and never spoke to them? I'm not sure. Feedback and interaction is not vital for acquisition for secondary language learning at least according to one theory. And if that’s good enough for adults it might be good enough for sponge-brain babies. https://en.wikipedia.org/wiki/Input_hypothesis https://en.wikipedia.org/wiki/Input_hypothesis
JohnKemeny 2y ago
They are two researchers/assistant professors working with cognitive science, psychology, and trustworthy AI. The paper is peer reviewed and has been accepted for publication in the Journal of Language Sciences. You should publish your critique of their research in that same journal. P.s. if you find any grave mistakes, you can contact the editor in chief, who happens to be a linguist.

amne 2y ago

tl:dr; we're duck-typing LLMs as AGI

Simon_ORourke 2y ago

Where I work, there's a somewhat haphazardly divided org structure, where my team has some responsibility to answer the executives demands for "use AI to help our core business". So we applied off-the-shelf models to extract structured context from mostly unstructured text - effectively a data engineering job - and thereby support analytics and create more dashboards for the execs to mull over. Another team, with a similar role in a different part of the org has jumped (feet first) into optimizing large language models to turn them into agents, without consulting the business about whether they need such things. RAG, LoRA and all this optimization is well and good, but this engineering focus has found no actual application, expect wasting several million bucks hiring staff to do something nobody wants.

rramadass 2y ago

See also Beyond the Hype: A Realistic Look at Large Language Models • Jodie Burchell • GOTO 2024 - https://www.youtube.com/watch?v=Pv0cfsastFs https://www.youtube.com/watch?v=Pv0cfsastFs

flimflamm 2y ago

How would the authors consider a paralyzed individual who can only move their eyes since birth? That person can learn the same concepts as other humans and communicate as richly (using only their eyes) as other humans. Clearly, the paper is viewing the problem very narrowly.

fairthomas 2y ago
> ...a paralyzed individual who can only move their eyes since birth... I don't think such an individual is possible.
- throwthrowuknow 2y ago
  I didn’t want to Google it for you because it always makes me sad but things like spina bifida and moebius syndrome exist. Not everyone gets to begin life healthy.

throwthrowuknow 2y ago

“Enactivism” really? I wonder if these complaints will continue as LLMs see wider adoption, the old first they ignore you, then they ridicule you, then they fight you… trope that is halfways accurate. Any field that focuses on building theories on top of theories is in for a bad time. https://en.m.wikipedia.org/wiki/Enactivism https://en.m.wikipedia.org/wiki/Enactivism

beepbooptheory 2y ago
What is the thing the LLMs are fighting for?

beepbooptheory 2y ago

There is a lot of frustration here over what appears to be essentially this claim: > ...we argue that it is possible to offer generous interpretations of some aspects of LLM engineering to find parallels with human language learning. However, in the majority of key aspects of language learning and use, most specifically in the various kinds of linguistic agency exhibited by human beings, these small apparent comparisons do little to balance what are much more deep-rooted contrasts. Now, why is this so hard to stomach? This is the argument of this paper. To feel like this extremely general claim is something you have to argue against means you believe in a fundamental similarity between what our linguistic agency and the model. But is embodied human agency something that you really need the LLMs to have right now? Why? What are the stakes here? The ones actually related to the argument at hand? This ultimately not that strong of a claim! To the point that its almost vacuous... Of course the LLM will never learn the stove is "hot" like you did when you were a curious child. How can this still be too much to admit for someone? What is lost? It makes me feel little crazy here that people constantly jump over the text at hand whenever something gets a little too philosophical, and the arguments become long pseudo-theories that aren't relevant to argument.

Royshiloh 2y ago

Why assume you "know" what language is? Like there is a study backed insight on the ultimate definition of language? it's the same as saying "oh, it's not 'a,b,c' its 'x,y,z'", which makes you as dogmatic as the one you critique. This is absurd.