7 ms·
>Also, the LLMs are cheating! No...they aren't. Humans aren't learning from thin air by any stretch of the imagination.
by famouswaffles 3y ago
>Also, the LLMs are cheating!
No...they aren't. Humans aren't learning from thin air by any stretch of the imagination.
- enord 3y agoWe did, at least once.
- meepmorp 3y agoNo, humans learn their languages through observation and generation - watching older people speak, and then imitating it. They get corrections, too, when they mispronounce or misuse words.
- squeaky-clean 3y agoThey mean at one point there were no older people to listen to in order to learn language because language did not exist yet. We somehow transitioned from grunts to words.
- enord 3y agoI didn’t say everyone learns it from scratch at least one, I said we did.
- meepmorp 3y agoThat's not learning, though, is it? Inventing, maybe, or developing or evolving.
- enord 3y agoWe either have the capacity to make boats and traverse the oceans or we don’t. These days almost nobody makes boats, but we sure used to and still can. What’s your point?
- meepmorp 3y agoI have no idea what you're talking about and it doesn't really matter to me anyway. Enjoy your day.
- CydeWeys 3y agoWhatever semantic game you want to play here isn't relevant; the point is it's something humans can do that AIs cannot.
- CydeWeys 3y agoYou've got a huge chicken and egg problem here that you haven't resolved.
- famouswaffles 3y agoThe brain is obviously pre-disposed to language. But evolution fueled learning is by no means from "thin air".
- enord 3y agoThat’s a red herring. Ex post facto the activity of language use as a construction must supersede some enaction by capable actors. This is a complete tautology in terms, and any dispute must therefore be over the ontology in which it is expressed. To elaborate how something came to be is not required to express that something is.
- famouswaffles 3y agoWhat makes capable actors capable in the first place is not derived from thin air. There's a reason a capable actor came to be where there was none generations prior. That reason did not come about without reaction to some outside vector. Evolution optimized for language learning abilities before the full capabilities came. Evolution is not thin air.
- enord 3y agoThe «air» is here an allegory for «large language corpus». It is self-evidently true that as we move backwards through history we eventually pass the first language utterance for any given definition of language or utterance. From here on out, the «air» is indeed «thin». There are still other things, but no «air» (i.e. «language corpora»).
- jvanderbot 3y agoHumans are, in the sense that we're working off the checkpoint passed to us from our parents: the bare minimum language centers our brain is evolved to produce regardless. The sequence of checkpoints evolves effectively from scratch, but individual humans adapt the checkpoint for their environment from birth. I like to think of the brain as the general model we've trained with evolution, and the person's experiences as the specialization.
- mjburgess 3y agoHumans learn from the structure of the world -- not the structure of language. LLMs cheat at generating text because they do so via a model of the statistical structure of text. We're in the world, it is us who stipulate the meaning of words and the structure of text. And we stipulate new meanings to novel parts of the world daily. What else is an 'iPhone' etc. ? There's nothing in `i P h o n e` which is at all like an iphone. We have just stipulated this connection. The machine replays these stipulations to us -- it does not make them, as we do.
- deleted 3y ago[deleted]
- somewhereoutth 3y agoExactly. A very uncomfortable truth for those heavily invested (time/money/credence) in this latest AI wave.
- refulgentis 3y agoIt’s odd to see people doomwaving two general reasoning engines. It’s especially hard to parse a dark sweeping condemnation based on…people are investing in it? It doesn’t have the right to assign names to things? Idk what the argument is. My most charitable interpretation is “it cant reason abour anything unless we already said it” which is obviously false.
- root_axis 3y ago> one of which is an average 14 year old, the other an honors student college freshman The point is that they're not those things. Yes, language models can produce solutions to language tests that a 14 year old could also produce solutions for, but a calculator can do the same thing in the dimension of math - that doesn't make a calculator a 14 year old.
- refulgentis 3y agoYes, the AI isn’t literally a 14 year old, and we should do an anthromorphization acknowledgement. Thank you for pointing it out, it can waste a lot of time when you get sloppy with language in AI discussions. I removed the reference, in retrospect, it’s unnecessary. No need to indicate the strong performance, we’re all aware.
- chubot 3y agoChildren raised in isolation will spontaneously develop language I don’t have a reference handy now (someone can probably do better) but I believe one way to see this is via the hearing impaired or hearing and sight impaired
- justrealist 3y agoThere's really no good data to support this claim.
- romwell 3y ago>There's really no good data to support this claim. Fine. We know that languages did not exist at some point. We made them exist.
- justrealist 3y agoYes, I acknowledge that we have evidence that groups of tribal humans will evolve language constructs over tens of thousands of years. Whether that happens in 1 generation or 10,000 generations is completely unknown.
- sudosysgen 3y agoAs others have mentioned in this thread, the Nicaraguan Sign Language shows it can happen in years, not even generations.
- dr-detroit 3y ago[dead]
- akprasad 3y agoThe best example I'm aware of is Nicaraguan Sign Language [1]. > Steven Pinker, author of The Language Instinct, claims that "The Nicaraguan case is absolutely unique in history ... We've been able to see how it is that children—not adults—generate language, and we have been able to record it happening in great scientific detail. And it's the only time that we've actually seen a language being created out of thin air." [1] https://en.wikipedia.org/wiki/Nicaraguan_Sign_Language https://en.wikipedia.org/wiki/Nicaraguan_Sign_Language
- CydeWeys 3y agoHumans invented English, and every other human language. Not one single human, and not in a single lifetime, but yes, we did it. LLMs are not doing this.
- throwawaymaths 3y agoI mean have you seen the whole "encode a poem in made up emoji" followed by "decode the poem from emoji" stuff? I think it's not unreasonable to think with the right prompts LLMs could do this.
- CydeWeys 3y agoThe LLMs that are doing this are trained on trillions of examples of human language. This is not remotely a counterexample. Now if an LLM can invent a new full language from scratch, without any training data of existing languages (like how AlphaZero learned to play Go), then that will be impressive, and a difference in kind.
- tialaramex 3y agoUm but you have the example of English. Modern English was based on Middle English, which in turn is based on Old English, but greatly influenced by Norman on account of the invasion, as well as by Norse
- CydeWeys 3y agoKeep playing it back through history and you'll find the first language invented by people. There is no equivalent accomplishment for AI.
- throwawaymaths 3y agohttps://bair.berkeley.edu/blog/2019/09/19/bit-swap/ https://bair.berkeley.edu/blog/2019/09/19/bit-swap/
- 3y ago
- QuadmasterXLII 3y agoIndividual humans aren’t, but we’re talking about the emergent properties of a swarm of humans. 6 gigabytes of dna describes an entity that, if copied 60 times and dropped on an island, will produce a viable language in inly a few decades. We haven’t found a NN architecture with this property for language, but we have for , eg, chess, starcraft, and go
- datameta 3y agoThis might be too overarching of a statement but "kind" learning environments like games that have set rules and win conditions are very far from the turing completeness of human language.
- deleted 3y ago[deleted]