5 ms·
Birds inspired planes. Later, aerodynamics fed back into ornithology. I've been waiting for LLMs to be evaluated as a model of human thought. Complexity and sca
by worstspotgain 3y ago
Birds inspired planes. Later, aerodynamics fed back into ornithology. I've been waiting for LLMs to be evaluated as a model of human thought. Complexity and scale have held neuroscience back. It's nowhere close to building a high-level brain model up from biological primitives. Like ornithology, it could use some feedback.
All arguments about AGI aside, a machine was built that writes like a human. Its design is very un-biological in places, so it's tempting to dismiss it. Why not see how deep the rabbit hole goes?
Like all conjectures it just might surprise us. For one, the intuition that language is central to thought predates LLMs, but it's certainly consistent with it.
- smokel 3y ago> Later, aerodynamics fed back into ornithology. You mean to imply that birds have learned from jet fighter designs? I fail to understand the point you're making.
- deleted 3y ago[deleted]
- worstspotgain 3y agoConcepts from the mechanics of flight (e.g. lift and drag) have helped ornithologists understand birds better. Birds themselves did not learn much.
- vaidhy 3y agoI believe the point is that we understand the birds' flight better applying the principles we learnt designing and flying airplanes. Similarly, we can learn more about human brain by applying things we learn from building the ANN back to the study of humans (anthropology in general, maybe neuroscience and psychology )
- Sardtok 3y agoHe didn't say it fed back into birds, but into the study of birds.
- throwaway_6142 3y agoKnowledge learned from airplane propeller development, as well as from jet plane wing aerodynamics, has had a great impact on the design of wind turbine blades, which are now affecting the evolution of birds.
- lo_zamoyski 3y agoWhy assume this is sensible? Aerodynamics can help discover general principles that apply to both birds and planes or whatever else. I don't see how this holds for LLMs and brains. The similarity between LLMs and brains is superficial. Besides, with human beings, we have a host of philosophical problems that undermine the neuroscientific presumption that a mechanistic and closed view of the brain can account for mental activity entirely, like the problem of intentionality.
- famouswaffles 3y ago>I don't see how this holds for LLMs and brains. The similarity between LLMs and brains is superficial. It's really not any more superficial than planes and bird flight.
- visarga 3y ago> a machine was built that writes like a human But the real hero here is not the LLM, but the training set. It took ages to collect all the knowledge, ideas and methods we put in books. It cost a lot of human effort to provide the data. Without the data we would have nothing. Without GPT we could use RWKV, Mamba, S4, etc and still get similar results. It's the data not the model. > the intuition that language is central to thought predates LLMs Language carries AI and humans. The same distribution of language can be the software running in our brains and in LLMs. I think humans act like conditional language models with multi modality and actions. We use language to plan and solve our problem, work together and learn (a lot) from others. Language itself is an evolutionary system and a self replicator. Its speed is much faster than biology. We've been on the language exponential for millennia, but just now hit the critical mass for LLMs to be possible. It's not so important that GPT-4 is a 2T weights model, what matters is that it was trained on 13T tokens of human experience and it now "writes like a human". Does that mean humans also learn the same skills GPT-4 has learned from its training set mostly by language as well?
- AlecSchueler 3y ago> But the real hero here is not the LLM, but the training set. And in the case of windmills the hero is the wind. But the mill is still a fantastic achievement.
- deleted 3y ago[deleted]