4 ms·
Started good by mentioning the data wall, but finished bad. It's not the DNA, we share most of it with other species. DNA can't contain human culture, and if it
by visarga 2y ago
Started good by mentioning the data wall, but finished bad. It's not the DNA, we share most of it with other species. DNA can't contain human culture, and if it could, we would have been as capable 200K years ago as today.
It's the one thing we have and they don't - language. During our 200K years our species has accumulated concepts, models, methods and values. We put them in language form and transmitted them across generations.
The main problem of course is search. We search for understanding, and do it collectively. A single lifetime would not suffice, it takes humanity across many generations to travel the road from caves to AI. That is why language is key, allows for iterative improvement. It also articulates search space, goal space and action space. It is symbolic and discrete, that allows for exact replication. DNA shares the same qualities with language - isn't it interesting?
Imagine you make a discovery, but having no language you can't reproduce the steps again, because it's all fuzzy. Or you can't teach anyone else. Language, with its symbolic structure is necessary to reproduce complex sequences of steps. Without it our exploration would not yield the fruits of exploitation. We would not benefit for unreproducible discovery.
I am against all kinds of essentialism. Chomsky thinks we have innate grammar, but forgets about co-adaptation of language for learnability in children, and about learning from our five senses - they provide a better explanation than innateness.
Searle explains our special status by biology, we have biological brains that's why we genuinely understand, he rejects distributed understanding but can't explain how come no single neuron understands on its own anything.
Chalmers thinks there is an inexplainable experience (qualia), but doesn't consider relational embeddings that can model qualities in experience. Relational embeddings are created by relating experiences against other experiences, creating their own high dimensional semantic space. No, it's not panpsychism, the whole universe doesn't need to be conscious.
And this time, in the article the magic is attributed to DNA. It's not that, it is search. We search and learn, learn and transmit, cooperate and reuse. Its not even a brain thing. It's social. We need more than one brain to cover this path of cultural evolution. Progress and language are not based in individuals but in societies.
My point is that from now on we hit the data wall. Imitation is thousands or millions of times easier than real innovation and discovery. AI will have to pay the same exploration price, it will have to learn from the world. Of course, new discoveries are not written in any books. They have to be searched. And search is an activity dependent on the environment we search in. Not in brains alone, or even DNA. AI will search and create new data, but it will be a slow grind.
- cma 2y ago> and about learning from our five senses You can be born without 5 and still learn very well. If sound and vision are both missing before a critical age (e.g. Helen Keller lost hers at 19mo) it can affect cognitive development.
- dboreham 2y ago> DNA can't contain human culture Furthermore: My conjecture is that DNA doesn't contain any "training data". There's no data path for information to get from an organism's learnings about its surroundings in generation N, into the DNA for generation N+1. DNA is just plans for how to wire up the I/O devices. Everything we think we see that seems like "instinct" will turn out to be explained by a combination of the wiring to I/O devices, and early training.
- overtomanu 2y agoDNA does not contain the training data, but it can be transmitted to next generation in other ways, like alteration of gene expression. Reference: Transgenerational epigenetic inheritance - Wikipedia https://en.wikipedia.org/wiki/Transgenerational_epigenetic_inheritance#In_Humans https://en.wikipedia.org/wiki/Transgenerational_epigenetic_i...
- ccozan 2y agoIn a thought experiment, I would really like to see chatGPT, Claude, Gemini, etc, talk to each other in a way that the prompts are primed with something like "you are not alone in this AI room, formulate questions and answers that help exchanging ideas". Also , an observation to language: even bacteria have a lauguage, the chemical language ( quorum sensing ). Meaning, maybe we just need to create the necessary medium for current LLMs , they will start talking to each other and creating their own language.
- mistermann 2y agoThis is interesting: https://x.com/repligate https://x.com/repligate