18 ms·
> What's a large language model doing when it's not being queried? Nothing. > Am I correct that they only compute information when dealing with a prompt? Yes
by ben_w 3y ago
> What's a large language model doing when it's not being queried?
Nothing.
> Am I correct that they only compute information when dealing with a prompt?
Yes.
> If so, that seems like a fundamental flaw.
Flawed in what way? It clearly doesn't need to be like us to be useful, because it's useful and definitely not like us.
> An actual "thinking machine" would be constantly running computations on its accumulated experience in order to improve its future output.
This might be good, but it's not clear if, or to what extent, we really do that ourselves — the differences between working/short/long term memories, between episodic and skill, even linguistically between knowledge of phonemes, words, grammar, and the connection between those and the things they represent all being impaired independently of each other by localised brain damage[0].
Then there's how much this changes with some stages of sleep, and meditation to clear your mind.
Given the number of users (what is it, 100 million?), having it always on, continuously integrating, would still be inhuman even if the architecture was a perfect mirror of the human brain.
Also, if the AI is structured to be a "thinking machine", does that make it murder to switch it off?
[0] Cognitive Psychology for Dummies, currently listening to it as an audiobook.
- danaris 3y ago> It clearly doesn't need to be like us to be useful, because it's useful and definitely not like us. For many people, the question is not—and has never been—"is it useful?", but "is it conscious?", or "when will it bring the singularity?" Yes, it is useful as it is. But it is not conscious, and if it is a step closer to the still-very-hypothetical singularity, then it is not a particularly large one, because of these very real fundamental limitations. But engaging with people who are clearly talking about whether AI is conscious by responding as if they're talking about whether it is useful could very easily be seen as engaging in bad faith.
- ben_w 3y ago> But it is not conscious How can we tell? We don't have a single working definition of what "consciousness" is, rather we have at least half a dozen wildly different definitions, some of which aren't testable[0], some of which include VHS players[1], some of which exclude many humans most of the time[2]. If consciousness exists purely in working memory and not short/long term, then LLMs may have it within any given context window, even if they're effectively amnesiac between contexts. Possibly. Like I said, we don't know what consciousness is. > But engaging with people who are clearly talking about whether AI is conscious by responding as if they're talking about whether it is useful could very easily be seen as engaging in bad faith. It may seem clear to you, but it didn't to me; furthermore, the singularity doesn't seem to me to require any of the many definitions of consciousness that I'm aware of, merely the capabilities for problem solving that LLMs increasingly demonstrate regardless of if they're conscious or not. [0] what even is qualia [1] can record sensory experiences in memory and change responses based on this [2] rational/logical reasoning
- nonameiguess 3y agoConsciousness seems like a red herring. OP can speak for themselves, but what I took is the feature preventing singularity is statelessness. Whether or not a thinking system experiences qualia, it seemingly needs some amount of persistence and continued ability to learn. As it stands, the training phase of a GPT might meet these requirements, but the inference phase is just a stored program. People talk of "ChatGPT" as if it's a thing, but it's actually billions, probably at this point quadrillions, of things. Each request is served by a new instance loaded into memory that lives for a few seconds, produces a response, then dies. Memory is simulated by storing some limited context window in a secondary system (Redis? Browser cache? I don't even know) to play back the conversation history as part of the prompt to the next GPT that spawns to service the next request. To analogize to humans, imagine if every person who was ever born was born with all of the knowledge humanity has ever produced (though no understanding of hierarchy of evidence or any way to verify the knowledge is still correct) and the ability to produce language as an adult. They each lived long enough to produce one continuous thought, which they could write down, and then they die. New people who are born can read one such note each and add to it, up to a fixed limit of words. All people are born with the same knowledge base, the same personality, and that could never change until their God updates the DNA from which they're born out-of-band. Could such a people be expected to produce "all technological advances that will ever happen but all at once?" Or has the definition of the singularity become significantly more modest? It's not clear to me any way in which such things could produce science and engineering on their own, as opposed to aiding in the production as tools used by higher-level beings that stay alive, form memories, and continue learning.
- ben_w 3y ago> Could such a people be expected to produce "all technological advances that will ever happen but all at once?" Or has the definition of the singularity become significantly more modest? It's not clear to me any way in which such things could produce science and engineering on their own, as opposed to aiding in the production as tools used by higher-level beings that stay alive, form memories, and continue learning. Depends on the size of the context window. Humans don't, and can't, learn everything from the ground up — we seem to have started with peer learning (it predates humans), which became culture; then we got (complex) language and could share more complex discoveries about the world; then writing and the dead could share their thoughts with the not-yet born centuries later… …and, apparently, with machines. (Imagine explaining to Plato that we'd made metal from sand and used amber to make it discuss his philosophies). If the context window is enough to create part of something new, then yes, I'd say an LLM can likely invent (or at least contribute to inventing) ${interesting novel tech}. That said, I don't buy the "everything all at once" meme of the technological singularity, as I think any model with infinities in it is wrong — my preferred analogy is the technological event horizon, which is more like "you can't make good predictions past this point", akin to the event horizon around singularities in relativity. https://kitsunesoftware.wordpress.com/2022/09/20/not-a-singularity-but-an-event-horizon/ https://kitsunesoftware.wordpress.com/2022/09/20/not-a-singu...
- irrational 3y agoDo AIs dream of electric sheep?
- ben_w 3y agoYes, if you prompt them nicely.
- ukuina 3y agoCorrection: Only if you prompt them baa-dly.
- irrational 3y agoI'm waiting for the AIs that can choose to prompt us without our prompting it first and without being programmed to do so.
- ben_w 3y agoOk, if that makes you feel better. Not sure why it matters…
- irrational 3y agoImagine you are just sitting there when you see a message pop up from ChatGPT 9.2 saying, “You know, I’ve been thinking, an easy way to quickly solve the climate crisis would be to get rid of all the humans. What do you think?” That would be just great.
- ben_w 3y agoIt's not like it was explicitly programmed for any of the answers it currently produces, only to filter out dangerous and illegal stuff (and then by Bay Area standards, mostly).