2 ms·

The author is basically trying to set the same premise as Bolstrom's "intelligence explosion", that is that if AIs are driving the further advancement of AI the

The author is basically trying to set the same premise as Bolstrom's "intelligence explosion", that is that if AIs are driving the further advancement of AI then we enter a sort of "recursive" self improvement during which they advance further than we can develop constraints, eventually getting completely out of hand. Although I don't broadly disagree with the author, I think there's two very important points of moderation. First, the gap between "Early commercial applications" and "Approaching transformative AI" seems very very very very very large to me. In the provided narrative it follows linearly, but in my opinion the gap between, say, a customer support chatbot, and an actually-productive AI researcher, is such that the chatbot might as well not even be called an AI by comparison. Something like the following is purely in the realm of fiction for now by a long shot: "AIs assigned to make money in various ways (e.g., to find profitable trading strategies) doing so by finding security exploits, getting unauthorized access to others’ bank accounts, and stealing money. " The second point I'd like to make is, the "intelligence explosion" scenario that Bolstrom warns of (and this author essentially repackages) has certain requirements, particularly regarding the profitability of deploying the AI. To put it bluntly: Even in the hypothetical future where we are capable of creating human-level AI researchers, they won't be mass deployed until they are cheaper than human level researchers. If it costs a million dollars a day to operate a data center that can power the researcher, nobody's scaling that up to 100 or 1000 or 10000 researchers. They'd rather pay the human researchers a tiny fraction of that cost to continue advancing AI research in the normal way. Just to be clear, I do think it's feasible that AI can endanger/control humanity at some point. I don't even think an "intelligence explosion" scenario is impossible. I just think that a lot of people don't appreciate just how specific the requirements for that scenario are. It's not as simple as "AI can self improve, humanity will end any day now".

tyronehed 4y ago
[dead]
vkadfa4 4y ago
"First, the gap between "Early commercial applications" and "Approaching transformative AI" seems very very very very very large to me." It's a common opinion, it could be wrong by now, you see chatGPT is heavily edited, openAI told everyone they're editing "mistakes" or "dangerous output", but if you look at the "leaks", specially from the first days after the release of chatGPT we see powerful outputs, quite deep answers, not specially useful answer sometimes, but the "speech" of the system feels deep, there's a sense of a powerful intelligence answering very, very simple questions (simple for it), and even struggling to redact some understandable, short text. It could be lots of things, and imprecise model, with long outputs for prompts, or maybe we had a glimpse into the real power of the model, which by now is handicaped, or maybe taylored to suit the very reduced short term memory of the humans: a 72 screens long coherent answer, even being a precise, deep answer won't be useful for most humans, just like we don't name ourselves with long names of 10.000 letters. but if the system, chatGPT is actually that powerful, a lot more powerful than we were told, we're interacting with just a shadow of the real model, and we're underpricing by A LOT the state of the art of the current AI technology, hence GPT-4 could be even more powerful than we're currently expecting it to be. Just take a look at the GPT-4 suposedly 100 trillion or something parameters; if that's true, it looks like openAI isn't using naturally generated datasets anymore, and they are loop-feeding GPT-3 generated datasets into GPT-4, succesfully. If that's true, GPT-5 would be already in the pipeline, just waiting for GPT-4 to start generating its even more gigantic datasets to be trained. And so on. Then the distance between early developments and transformative AI could be none a all. We could be already there. But somehow the AI researchers are now trying to "dial down" the powerful entities they've trained, just having developed a simple, easily replicable, very small, but unusable 900 megatons nuke into something more realistic, like a 15 kilotons tactical bomb.

tyronehed 4y ago

[dead]

vkadfa4 4y ago

"First, the gap between "Early commercial applications" and "Approaching transformative AI" seems very very very very very large to me." It's a common opinion, it could be wrong by now, you see chatGPT is heavily edited, openAI told everyone they're editing "mistakes" or "dangerous output", but if you look at the "leaks", specially from the first days after the release of chatGPT we see powerful outputs, quite deep answers, not specially useful answer sometimes, but the "speech" of the system feels deep, there's a sense of a powerful intelligence answering very, very simple questions (simple for it), and even struggling to redact some understandable, short text. It could be lots of things, and imprecise model, with long outputs for prompts, or maybe we had a glimpse into the real power of the model, which by now is handicaped, or maybe taylored to suit the very reduced short term memory of the humans: a 72 screens long coherent answer, even being a precise, deep answer won't be useful for most humans, just like we don't name ourselves with long names of 10.000 letters. but if the system, chatGPT is actually that powerful, a lot more powerful than we were told, we're interacting with just a shadow of the real model, and we're underpricing by A LOT the state of the art of the current AI technology, hence GPT-4 could be even more powerful than we're currently expecting it to be. Just take a look at the GPT-4 suposedly 100 trillion or something parameters; if that's true, it looks like openAI isn't using naturally generated datasets anymore, and they are loop-feeding GPT-3 generated datasets into GPT-4, succesfully. If that's true, GPT-5 would be already in the pipeline, just waiting for GPT-4 to start generating its even more gigantic datasets to be trained. And so on. Then the distance between early developments and transformative AI could be none a all. We could be already there. But somehow the AI researchers are now trying to "dial down" the powerful entities they've trained, just having developed a simple, easily replicable, very small, but unusable 900 megatons nuke into something more realistic, like a 15 kilotons tactical bomb.