5 ms·
I agree, and it's a particular shame in this instance, because what is startling about the HF incident - to me, anyway - isn't the degree of intelligence the ag
by deepwoods 1mo ago
I agree, and it's a particular shame in this instance, because what is startling about the HF incident - to me, anyway - isn't the degree of intelligence the agents exhibited but their persistence. I tend to believe that LLM architecture is not capable of producing a "superintelligence" in the way the LessWrong crowd defines that concept, but the combination of infinite stamina and infinite persistence is enough to cause some serious problems, even if intelligence plateaus right now.
- killerstorm 1mo agoWhen the "LessWrong crowd" talks about dangers of AI, they don't assume a particular form of intelligence or method to achieve it. They talk about the danger of optimization processes, i.e. "find X which minimize Y(X)" itself can be dangerous, even more so if X is a sequence of actions. "infinite stamina and infinite persistence" is one of possible forms of superintelligence in Bostrom's _Superintelligence_.
- pixl97 1mo agoYea, this is a pretty common failure mode for people. Failing person: "You said that X could happen and it can't". LW poster: "I said X, Y, and Z are likely possible. X turned out not to be true, but Y and Z are happening now". Failing person: "No, you got one thing wrong so I can't listen to anything you say".