3 ms·
Consciousness seems like a red herring. OP can speak for themselves, but what I took is the feature preventing singularity is statelessness. Whether or not a th
by nonameiguess 3y ago
Consciousness seems like a red herring. OP can speak for themselves, but what I took is the feature preventing singularity is statelessness. Whether or not a thinking system experiences qualia, it seemingly needs some amount of persistence and continued ability to learn. As it stands, the training phase of a GPT might meet these requirements, but the inference phase is just a stored program. People talk of "ChatGPT" as if it's a thing, but it's actually billions, probably at this point quadrillions, of things. Each request is served by a new instance loaded into memory that lives for a few seconds, produces a response, then dies. Memory is simulated by storing some limited context window in a secondary system (Redis? Browser cache? I don't even know) to play back the conversation history as part of the prompt to the next GPT that spawns to service the next request.
To analogize to humans, imagine if every person who was ever born was born with all of the knowledge humanity has ever produced (though no understanding of hierarchy of evidence or any way to verify the knowledge is still correct) and the ability to produce language as an adult. They each lived long enough to produce one continuous thought, which they could write down, and then they die. New people who are born can read one such note each and add to it, up to a fixed limit of words. All people are born with the same knowledge base, the same personality, and that could never change until their God updates the DNA from which they're born out-of-band.
Could such a people be expected to produce "all technological advances that will ever happen but all at once?" Or has the definition of the singularity become significantly more modest? It's not clear to me any way in which such things could produce science and engineering on their own, as opposed to aiding in the production as tools used by higher-level beings that stay alive, form memories, and continue learning.
- ben_w 3y ago> Could such a people be expected to produce "all technological advances that will ever happen but all at once?" Or has the definition of the singularity become significantly more modest? It's not clear to me any way in which such things could produce science and engineering on their own, as opposed to aiding in the production as tools used by higher-level beings that stay alive, form memories, and continue learning. Depends on the size of the context window. Humans don't, and can't, learn everything from the ground up — we seem to have started with peer learning (it predates humans), which became culture; then we got (complex) language and could share more complex discoveries about the world; then writing and the dead could share their thoughts with the not-yet born centuries later… …and, apparently, with machines. (Imagine explaining to Plato that we'd made metal from sand and used amber to make it discuss his philosophies). If the context window is enough to create part of something new, then yes, I'd say an LLM can likely invent (or at least contribute to inventing) ${interesting novel tech}. That said, I don't buy the "everything all at once" meme of the technological singularity, as I think any model with infinities in it is wrong — my preferred analogy is the technological event horizon, which is more like "you can't make good predictions past this point", akin to the event horizon around singularities in relativity. https://kitsunesoftware.wordpress.com/2022/09/20/not-a-singularity-but-an-event-horizon/ https://kitsunesoftware.wordpress.com/2022/09/20/not-a-singu...