4 ms·
The reasoning pattern of Yud and his ilk is fundamentally theological and eschatological in nature. There isn't any actual understanding of the technology invo
by lukev 4y ago
The reasoning pattern of Yud and his ilk is fundamentally theological and eschatological in nature.
There isn't any actual understanding of the technology involved. It is fundamentally an ontological argument. Because they can imagine a god-like super intelligent AI, it must be possible. And they're associating that with LLMs because that's the most powerful AI currently available, not based on any actual capabilities or fact-based extrapolation from the present to the future.
Meanwhile its distracting from actual AI safety concerns: namely, that corporations and capitalists will monopolize them such that their benefits accrue to relatively few rather than benefiting humanity at large.
- seydor 4y agoI doubt his skills in theology either. Listening to him is full of incoherent half-assed arguments
- rhn_mk1 4y ago> Because they can imagine a god-like super intelligent AI, it must be possible. It's worse. It may be possible, but we're not equipped to recognize the line as it's crossed. Combined with us making LLMs more and more capable despite not knowing why they work, this extrapolating of LLMs to gods is not insane.
- lukev 4y agoThis is exactly the kind of mysticism I'm talking about. In fact we know precisely how LLMs work. The fact that parts of human linguistic concept-space can be encoded in a high dimensional space of floating point numbers, and that a particular sequence of matrix multiplications can leverage that to perform basic reasoning tasks is surprising and interesting and useful. But we know everything about how how it is trained and how it is invoked. In fact, because it's only "state" aside from its parameters is whatever its context window, current LLMs have the interesting property that if you invoke them recursively, all of their "thoughts" are human readable. This is in fact a delightful property for anyone worried about AI safety: our best AIs currently produce a readable transcript of their "mental" processes in English.
- rhn_mk1 4y agoWe know how they work, that is true. We don't know why they work, because if we could, then we could extrapolate what happens when you throw more compute at them, and no one would have been surprised about the capabilities of GPT-N+1. Also no one would have been caught with their pants down by seeing people jailbreak their models. To illustrate it in a different way: on a mechanistic level, we know how animal brains work, as well. Ganglions, calcium channels, the stuff. That doesn't help understand high level phenomena like cognition, which is the part that matters. If you're right about the LLMs revealing their inner working, that would be indeed a reason to chill out. But I have my doubts, given that LLMs are good at hallucinating. Could you justify why the human readability is actually true, and support that with examples?
- lukev 4y agoI don't need examples. It's simply how they work. This is why they hallucinate. A LLM is fundamentally a mathematical function (albeit a very complex one, with billions of terms (a.k.a parameters or weights)). The function does one thing and one thing only: it takes a sequence of tokens as input (the context), and it emits the next token(word)[1]. This is a stateless process: it has no "memory" and the model parameters are immutable; they are not changed during the generation process. In order to generate longer sequences of text, you call the function multiple times, each time appending the previously generated token to the input sequence. The output of the function is 100% dependent on the input. Therefore, the only "internal state" a model has is the input sequence, which is human-readable sequence of tokens. It can't "hallucinate", it can't "lie", and it can't "tell the truth", it can only emit tokens one at a time. It can't have a hidden "intent" without emitting those tokens, it can't "believe" something different than what it emits. [1] Actually a set of probabilities for the next token, and one is selected at random based on the "heat" generating setting, but this is irrelevant for the high-level view.
- rhn_mk1 4y agoI don't see how this proves that asking the model about its internal state will reveal its inner high level processes in a human-readable way. Perhaps there's a research paper which would explain it better?
- cableshaft 4y agoI suspect that he realized that there's money to be made being the professional luddite of the hot thing du jour that talk shows and schools can give him money for (as so many other people have done for other things), and he's taking this simple claim of 'AGI will murder us all' to get on as many talk shows as possible and make as much money off of it. I doubt he's speaking in public out of any concern for humanity's survival. It's such an easy claim to make, because you can just say 'yeah, AGI hasn't murdered us all yet, but it will at some point' and keep kicking out that timeline further and further out until your dead and buried and who cares.
- natdempk 4y agoFWIW he’s been making these claims well before the current AI hype cycle hit.
- yifanl 4y agoDoes that make him more credible at all?
- TheRealNGenius 4y ago[dead]
- sharkjacobs 4y agoI think that's the point, this has been his personal obsession for more than a decade now, and so he's jumping at the opportunity to link it to the latest hot news topic without real consideration for how related LLM AI is to the AGI he's been writing and fantasizing about for so long.