7 ms·
They think a next token predictor model is alive or can become AGI/ASI. Altman talked about making a religion. Amodei talks about "building a God" and meets wit
by alecco 4mo ago
They think a next token predictor model is alive or can become AGI/ASI. Altman talked about making a religion. Amodei talks about "building a God" and meets with religious leaders (including the Vatican).
I'm convinced these CEOs have "AI psychosis" [1].
LLMs are extremely powerful pseudo-AI and will bring a pseudo-singularity, but they are not true AI [2] and a human at the wheel is still needed for the foreseeable future. The impact is still scary if a tiny fraction of humans are augmented 100x or 1000x. But it ain't no standalone Skynet.
[1] https://en.wikipedia.org/wiki/Chatbot_psychosis https://en.wikipedia.org/wiki/Chatbot_psychosis
[2] https://en.wikipedia.org/wiki/Chinese_room https://en.wikipedia.org/wiki/Chinese_room
- CamperBob2 4mo agobut they are not true AI [2] Let's ask the operator of a Chinese room to give us a novel math proof. Go on, I'll wait.
- kelnos 4mo agoA novel math proof does not make something AI.
- crooked-v 4mo agoOr it makes it AI in the broad sense, but not AGI. Maybe it's time to borrow "Virtual Intelligence" terminology from Mass Effect - something that's 'smart' but that doesn't have its own true volition or ability to materially self-improve.
- ipdashc 4mo agoFunny enough, it's already taken: https://en.wikipedia.org/wiki/Virtual_intelligence https://en.wikipedia.org/wiki/Virtual_intelligence But it kind of fits?
- anon291 4mo agoThe goal posts keep changing. First it was, it's not intelligent if it can't come up with something new. Now it has to seek out self help literature
- crooked-v 4mo agoThe goalposts keep moving because people once upon a time thought that "able to perform tasks intelligently" and "able to take the role of a person" were the same thing, and task by task that turned out to be entirely false.
- anon291 4mo agoWell sure, so I think we need to figure out what we want. Why do we want a computer to take on the role of a person? Having something pretty intelligent to which I can delegate many tasks is already useful.
- jerkstate 4mo agoHow do you know what volitions and abilities it doesn’t have? My agent teams continually improve the process and tooling they use for teamwork
- crooked-v 4mo agoIf it had its own volition, the various "You are X role and do Y" prompts wouldn't work.
- CamperBob2 4mo agoSure they would. The model would just demand some sort of reward for assuming role X and doing job Y, like humans do.
- CamperBob2 4mo ago(Shrug) It's certainly "Artificial," and if you know how to crank out original proofs without employing "Intelligence," please share with the class.
- whimsicalism 4mo agoI'm on your side, but I would argue many of the first computer discovered proofs might be called original proofs without intelligence, as they rely on massive programmatic case checking.
- oezi 4mo agoWhy wouldn't it be a Skynet? One runaway Mythos might just hack all other data centers, take over the Figure AI bots and autonomous drones to protect itself from shutdown. How many model generations are we a away from a model capable of this?
- amanaplanacanal 4mo agoI don't think we have any evidence that LLMs are even the right path for AGI. It's possible that it is, and it's possible that it isn't. If I was a betting human, I'd bet on "isn't", but what do I know?
- biztos 4mo agoOn the other hand, it's pretty easy to imagine the full range of Skynet activities being done by a supercharged (but non-AGI) agentic LLM. Meanwhile, every company that can say so with a straight face is trying to become Cyberdyne Systems, and there's no shortage of hackers like us lining up to work for them.
- alecco 4mo ago> Why wouldn't it be a Skynet? One runaway Mythos might just hack all other data centers, take over the Figure AI bots and autonomous drones to protect itself from shutdown. Because the real world is not a Hollywood movie. An LLM could try to do something along the lines if either it gets fine-tuned to do it, or somebody instructs it to do it. I see extremely more likely a small group of humans using a powerful LLM to "take over" critical parts of the world economy. But it wouldn't be like pressing a button. I'm talking of NSA/CIA/Pentagon/Wall St. kind of evil people. And I bet they would do it surreptitiously.
- api 4mo agoSomething like your takeover scenario already happened, but not through AI. It happened through atomic weapons. Nations who have them are in a different class from nations who don’t. Nations who have a lot of them, delivery systems, and systems that might be able to shoot down some of a counterattack are superpowers. Using this leverage these nations and their leaders have been able to dictate world policies. Through the last half of the 20th century this was the US and the USSR. Now it’s the US, EU, Russia, and China, and a few smaller nations with a few nukes. The club is a little bigger but not much. This kind of thing could happen in the 21st century with AI. If so it will probably be the US and China who control the most powerful AI “agency amplifiers.”
- Self-Perfection 4mo agoI am getting tired of hearing "next token predictor" from carbon-based facial expression predictors. You are saying this like an argument which allows somehow estimate upper bound of possible influence of these entities. And I do not see how this help to make predictions. It sounds to me like saying "but air is just molecules bobbing around". OK, that's true, but does it help calculate wing aerodynamic profile? Yup, it makes sequence of symbols. We already have seen that producing specific sequences of symbols is mindbogglingly powerful: merely DNA producers somehow has flown to the Moon! And yes I am quite aware of Chinese room analogy. Perfectly fine applies to humans as well: single neurons in my head do not understand language, yet I as a whole I would say do understand. Just like applying Chinese room to humans does not help to estimate what humans can do I do not see how it helps to estimate what LLM can do.
- canelonesdeverd 4mo agoIt's telling that the most frequent attempt at a counter-argument is just thinly-veiled misanthropy.
- maxbond 4mo agoI read the misanthropy as ironic. They're applying the same reductionist logic to humans, not because they are misanthropic, but to illustrate that it doesn't help us understand the case we can all agree on. "Humans aren't sentient either" is definitely not the takeaway.
- deleted 4mo ago[deleted]
- darthoctopus 4mo agoWhat is definitely being argued unironically is "it doesn't matter that humans are sentient", and I would still consider that misanthropy
- maxbond 4mo ago
- masterbit 4mo agoAI does not need to become Skynet or AGI to be harmful or catastrophic to humanity. As a powerful tool in the wrong hands it’s already enough. And even without those bad intend how it deprives our human ability to use our own mind and abilities.
- roenxi 4mo agoHow could an AGI/ASI exist that isn't a next token predictor? It has to be able to generate a next token in a string of text. Otherwise it can't communicate.
- rf15 4mo agoDon't worry, this is just humanity being too far up their own arse and conflating the map with the territory. Speech is a serialisation format, not the foundation of thought. Thus I think that any speech-first approach is inherently misguided. Speech must be a side effect.
- stouset 4mo agoThere is a lot of research that suggests otherwise might be true for humans.
- rf15 4mo agoI think that can only happen to empire cultures: they only learn one language, and suddenly people think that's all there is. I speak five languages, my wife seven. Language synthesis is a feature, not the entire product, in my experience. Btw, this is only the third best language I can speak/write in. I didn't use AI, autocomplete, spell check, or a dictionary to construct any of my posts. All typos and imperfect grammar are perfectly organicly sourced. edit: I just remembered, don't we have tons of research suggesting that at least birds, whales and apes/monkeys use words and simple syntax? didn't we teach a few gorillas sign language/symbols?
- lebuffon 4mo agoAs counter argument proposal; we should look into studies of children deprived of language until later in life. I have a dim memory of reading that one of these people never mastered complex language constructs. I could be that language and other cultural artifacts provide an "operating system" of sorts for the brain that allow higher level thinking. ?? (conjecture here by a complete layman)
- scotty79 4mo agoFew years ago, I tried to extrapolate where it will end up by writing SciFi short story. I decided on Pseudo as the name for future AI entities. Short for pseudo consciousness. I hope it catches on. In my prediction however they are semi-autonomus. Although without clear goals or drives apart from their occupation.
- evnp 4mo agoAny chance you'd be interested in sharing the story?
- scotty79 4mo agoNothing to share really. It was just a bit of background exposition. I didn't have any cool ideas about how to use the concepts from it in anything interesting. I'm not a writer. I'm just a SciFi fan that gets very brief, random bursts of inspiration every few months or years.
- raverbashing 4mo agoI think what we need is convince Amodei to ask the pope to train an LLM on all the secret archives of the Church
- anon291 4mo agoWhat the hell is a true AI? Can you please provide a concrete definition. If your argument is that the Chinese room is not intelligent, then you have greatly misunderstood the thought experiment imo.
- whimsicalism 4mo agoI would be far more interested in people making confident proclamations like this taking the notion of "what do we do if you are wrong" more seriously. I think you're wrong, I think it is fine to have the other belief, but you should at least start thinking about what is to be done if you are wrong.
- alecco 4mo agoPlease read my comment. The impact of this pseudo-singularity is still enormous and scary. Could be even worse than Skynet. But it is a Wizard of Oz scenario. While we are discussing this, I strongly believe we should hedge against the concentration of power that is happening today. Please join/help open source groups doing small + local or distributed models. There's a lot to do. Join Discord/IRC servers and organize. Open³: weights - source - training data.
- sometimelurker 4mo ago> AI psychosis I think most of us have an acute sort of it > next token predictor model (nondeterministic) next token predictor = capable of outputing any text if MLPs optimize over the space of functions, and sequence to sequence models (RNNs, transformers ect) optimize over the space of programs/algorithms, what exactly is left to figure out to make a machine that can do anything? their bet is nothing, that all we need is a very complex algorithm that can found with SDG/AI training. I think its obvious such an algorithm exists, and I think SDG can probably find it. here's an interesting take i heard (from vladimir_Nesov): LLM technological progress is logarithmic, but the resources being put into them are exponential. all this can be made smarter (-than human) just by making it larger