9 ms·
Natural language instructions induce generalization in networks of neurons
- kevindamm 3y agoIt's as if language is itself the latent space for these psychophysical tasks, especially compositional instruction. Their description of it as a scaffolding also seems apt.
- zer00eyz 3y agoI hate the reductive nature of the concept of "latent spaces". A good enough formula for a task isn't a solution for every task. Yes Newtonian mechanics work, but Einstein is a better reflection of reality.
- bongodongobob 3y agoI'm not sure I understand the analogy. The very idea of NNs is that it's not perfect, it is messy and not optimal, but is very generalizable.
- zer00eyz 3y ago>> The very idea of NNs is that it's not perfect, it is messy and not optimal, but is very generalizable. Newton: Do you need more than that to describe the speed of a thrown baseball on a train? No. DO you you need more than newton to get to the moon? No. Is it going to be accurate at high speed in a large scale system (anything traveling near C)? NO, it fails spectacularly. NN's are great at simulation, language, weather... But what people using them for weather seem to understand and the ML folks (screaming about AI and AGI) dont is that simulation is not a path to emulation. Lorenz showed that there were limits in weather, that most other disciplines have embraced these limits.
- nerdponx 3y agoThe entire innovation (discovery?) of LLMs is that a good formula for the task of sequence completion turns out to also be a good formula for a wide range of AI tasks. That emergent property is why language models are called language models.
- error_logic 3y agoThe usefulness is why the term is so widespread in familiarity but I think the term would have existed to describe the linguistic mapping even if they hadn't proven to have direct problem-solving capabilities.
- hackerlight 3y agoThat's an inductive bias, not an emergent property. The inductive bias of transformers is that they're good at integrating global context from different parts of a sequence without a particular bias towards recent time steps or localized regularities. And that happens to be a good fit for many (but not all) real-world sequence learning tasks. The "emergent property" aspect is when LLMs are good at a task at scale X*3 but were incompetent at scale X.
- nerdponx 3y agoMy point was that this particular inductive bias doesn't inherently beget a "language model".
- hackerlight 3y agoIt does beget a Transformer which we choose to call a language model when it's applied to language data
- nothis 3y agoI'm not pretending to understand half the words uttered in this discussion but I'm constantly reminded of how much it helps me to articulate things (explain them to others, write them down, etc) to understand them. Maybe that thinking indeed happens almost entirely on a linguistic level and I'm not doing half as much other thinking (visualization, abstract logic, etc.) in the process as I thought. That feels weird.
- robwwilliams 3y agoOr is the real thinking sub-linguistic and “you” and those you talk to are the target audience of language? Sentences emerge from a pre-linguistic space we do not understand.
- error_logic 3y agoI do find it funny that this discussion thread has tried to represent language as a universal form of thought when it would be messy to encode the inner workings of a LLM (the weightings/relationships) themselves as natural language. You could sort of represent the deterministic contents of an LLM by compiling all the algorithms and training data in some form, or maybe a visual mosaic of the weights and tokens, or what have you...but that still doesn't really explain the outcome when a model is presented with novel strings. The patterns are emergent properties that converge on familiar language--they're something deeper than the individual words that result.
- mrblah 3y agoi've always assumed that language was required to give your brain the abstractions needed to reference things in the past compared to your current perception (aka now), like an index. if you think about your earliest memories, they almost certainly came after language. i'd be interested to know if any of the documented 'wild child' cases (infants 'raised by wolves') ever delved into what the children remembered before, after being taught language as an adolescent.
- tsimionescu 3y agoThere were efforts to teach them language as adolescents, but they didn't acquire it - as far as we know, it's not possible to acquire language if you don't do it as an infant. This is similar to other brain functions that aren't present at birth and require stimulation, such as sight. That is, if your eyes are forced closed for the first few months of your life, you will never be able to see, even if later they are uncovered, and keep working perfectly. The brain functions responsible for interpreting visual signals can only develop if they get visual signals in a (quite short) developmental window - and we know this with quite a bit of certainty from quite cruel animal studies. Language acquisition is not proven to be the same, as the required studies would be deeply unethical, but the few experiences with feral children are highly suggestive that the same applies.
- mrblah 3y agoi just had the uncomfortable thought that it's possible a disease could kill off everyone older than 1, sterilizing the species to language at some scale. for the few that survive, their world would be feral.
- mrblah 3y agoWent down the rabbit hole and found the case of Danish Bear Boy, he was reportedly taught to speak but he claimed to have no memory of his time living with the bears. Fascinating stuff https://books.google.com/books?id=k2MRHJuQiVEC&dq=hesse+wolf+children+1304+1341+1344&pg=PA44#v=snippet&q=Danish%20Bear%20Boy&f=false https://books.google.com/books?id=k2MRHJuQiVEC&dq=hesse+wolf...
- jameshart 3y ago> We found that language scaffolds sensorimotor representations such that activity for interrelated tasks shares a common geometry with the semantic representations of instructions, allowing language to cue the proper composition of practiced skills in unseen settings. Sapir-Whorf with the surprise comeback?
- sisyphus_coding 3y ago> comeback Did it fall out of favour?
- jameshart 3y agoStrong Sapir-Whorf (linguistic determinism - language constrains thought) became pretty much seen as a joke by the 1980s. Linguistic relativism (weak Sapir-Whorf - language shapes thought) is still respectable (because, I mean, of course it does). Actually, this research might just as well be evidence for linguistic universalism (Chomsky - language enables thought). In general linguistic philosophers have been coming out with either laughably obvious or utterly untestable hypotheses for a century and it’s amusing to see how these AI studies shake up the hornets.
- a_gnostic 3y agoOftentimes I find myself understanding complex concepts before I can describe them, even internally. I am sure everyone has this, as I often read comments praising others' submissions for formulating their thoughts efficiently. So thoughts occur independent of language, but need it to be expressed and shared, even if through pictures and sounds.
- lo_zamoyski 3y agoExpression is not language. What you're having trouble doing is expressing what you understand.
- 3y ago
- nyrikki 3y ago>Tasks that are instructed using conditional clauses also require a simple form of deductive reasoning (if p then q else s) > Our models ofer several experimentally testable predictions outlining how linguistic information must be represented to facilitate flexible and general cognition in the human brain. Aren't those claims falsified by more recent studies that show that even in flys, preferred direction to a moving stimulus uses the timing of spikes. And that fear conditioning in even mice uses Dendritic Compartmentalization? Or that humans can even do xor with a single neuron. If "must be represented" was "may be modeled by" I would have less of an issue and obviously spikey artificial NNs have had problems with riddled basins and make autograd problematic in general. So ANNs need to be binary and it is best to model biological neurons as such for practical models... but can someone please clarify why 'must' can apply when using what we know now is an oversimplified artificial neuron models? Here are a couple of recent papers but I think dendritic compartmentalization and spike timing sensitivity has been established for over a decade. https://pubmed.ncbi.nlm.nih.gov/35701166/ https://pubmed.ncbi.nlm.nih.gov/35701166/ https://www.sciencedirect.com/science/article/pii/S0092867418311061 https://www.sciencedirect.com/science/article/pii/S009286741...
- ben_w 3y ago> Or that humans can even do xor with a single neuron. That's news to me. I'm not hugely surprised given I've heard a biological neuron is supposed to be equivalent to a small ANN network, but still, first I've heard of that claim.
- cs702 3y agoTL;DR: The authors embed task instructions in a vector space with a language model, and train a sensorimotor-controlling model on top to perform tasks given the instruction embeddings. The authors find that the models generalize to previously unseen tasks, specified in natural language. Moreover, the authors show that the hidden states learn to represent task subcomponents, which helps explains why the model is able to generalize.
- ShamelessC 3y agoWhat is the language model pretrained on? Wouldn’t the far more robust LLM’s priors impact ability to generalize to “unseen” tasks?
- deleted 3y ago[deleted]
- deleted 3y ago[deleted]
- worstspotgain 3y agoBirds inspired planes. Later, aerodynamics fed back into ornithology. I've been waiting for LLMs to be evaluated as a model of human thought. Complexity and scale have held neuroscience back. It's nowhere close to building a high-level brain model up from biological primitives. Like ornithology, it could use some feedback. All arguments about AGI aside, a machine was built that writes like a human. Its design is very un-biological in places, so it's tempting to dismiss it. Why not see how deep the rabbit hole goes? Like all conjectures it just might surprise us. For one, the intuition that language is central to thought predates LLMs, but it's certainly consistent with it.
- smokel 3y ago> Later, aerodynamics fed back into ornithology. You mean to imply that birds have learned from jet fighter designs? I fail to understand the point you're making.
- deleted 3y ago[deleted]
- worstspotgain 3y agoConcepts from the mechanics of flight (e.g. lift and drag) have helped ornithologists understand birds better. Birds themselves did not learn much.
- vaidhy 3y agoI believe the point is that we understand the birds' flight better applying the principles we learnt designing and flying airplanes. Similarly, we can learn more about human brain by applying things we learn from building the ANN back to the study of humans (anthropology in general, maybe neuroscience and psychology )
- Sardtok 3y agoHe didn't say it fed back into birds, but into the study of birds.
- throwaway_6142 3y ago
- retskrad 3y agoIt's clear that both biological sentient beings and sentient being made in factories in the future will essentially be two sides of the same coin, differing only in their physical composition. Humans today operate as biological AI, powered by cells, and the sentient beings instead operate on transistors. As we progress toward a future where both sentient beings exhibit comparable intelligence, emotions, and learned experiences, the distinction between the two becomes increasingly blurred. It wouldn't be crazy to think that we'll program a person made out of transistors to go through the same life as a biological human. In such a scenario, why should we consider the sentient being made of cells inherently superior to its transistor-based counterpart?
- deleted 3y ago[deleted]
- arijun 3y agoI don’t think it’s a given that people do. Also what does that have to do with the article?
- spr-alex 3y agoproof left as exercise to the reader
- smokel 3y agoHm, no. To start, cells are able to replicate themselves, whereas most silicon used today is not even close to doing so. The story you suggest seems to be built on a limited understanding of the processes involved. It's pretty hard to predict the future, especially given incorrect assumptions.
- dmead 3y agoI hate that we just turned out to be stochastic machines another not something more interesting.
- KoolKat23 3y agoDelusions of grandeur
- error_logic 3y agoThis reminds me of how 'interesting' it is when people defend their use of language by downplaying the effects on other people. We really do have so many ripple effects from everything we broadcast to others, and tacking a negative, or a double/triple negative into a sentence doesn't change the fact that mentioning pink elephants will color an arbitrary portion of someone's day. In other words, this is why tone absolutely matters, and I love how this awareness feeds into support for genuine expression. Sarcasm is chaotic with an unknown audience who would be less likely to recognize the intended subversion and toying with meaning, instead taking it in whatever direction the listener feels like without being sure that the speaker is understood.
- alexfromapex 3y agoI think the title of this is wrong...it should say "Structured Language Patterns Facilitate Structured Generalization in Neural Networks".
- vassilis-uk 3y agohere is a video of the author explaining the work https://www.youtube.com/watch?v=miEwuSz7Pts https://www.youtube.com/watch?v=miEwuSz7Pts