6 ms·
Is anyone actually arguing that a super-intelligent AI is, like, physically impossible or something? That seems like a very dumb position to hold, it would be n
by bee_rider 2y ago
Is anyone actually arguing that a super-intelligent AI is, like, physically impossible or something? That seems like a very dumb position to hold, it would be necessary (but not sufficient) to believe that human brains are somehow following alternative laws of physics (aka, magic).
The room to argue, I think, is in how close we are to making them and whether things like DNN’s are the path or a dead end.
- nullc 2y agoIn almost any HN thread that touches on the subject someone manages to express this sort of view. E.g. from the most recent thread I read: https://news.ycombinator.com/item?id=40686920 https://news.ycombinator.com/item?id=40686920 (apologies for singling out the post, it was impossible to give an example without doing singling out something-- I hope you'll read it in the sense it was intended: an example of a common pattern). It's even more common in other venues. That said, from what I've seen the posted article is often used to justify hysteria about AI. For some people the thing protecting them from a meltdown is just a view that superhuman machine intelligence isn't possible, and once that is broken they become vulnerable to cult indoctrination or just general dysfunction. One needs to maintain perspective, and sometimes an inaccurate view of impossibility is helping a person make better decisions on the whole than they would without it.
- MostlyStable 2y agoI also frequently see the slightly weaker form of argument hypothesizing "diminishing returns" of intelligence or some other claim that humans are close the practical upper limit of intelligence.
- AtlasBarfed 2y agoWhich is pretty stupid, because what human intelligence wouldn't be greatly enhanced by cybernetic math coprocessors and perfect wide memory recall, and a greatly increased lifespan so there wasn't so much collective work done constantly (re)training new generations and passing forward knowledge in 18-30 year education programs, which are constrained by communication mechanisms with bandwidth possibly in the thousand bytes per second rate? The sad fact is that the only real fiction in Isaac Asimov's robots is the ability to impose the laws of robotics.
- card_zero 2y agoBut we have math processors, I own several, they're just not implanted in my head, so what? Why would sticking them in there make me "super-intelligent"? If my calculator has a better interface so I can use it faster, is that also making me more intelligent? These things are tools, the hype about super-intelligence is just about a person with very effective tools. Oh noes!
- Hugsun 2y agoHaving good tools is definitely a force multiplier, but as you say, we already have those. We are however limited in how fast we can interface with these tools. There is a lot of potential improvement from just knowing the answer to a mathematical/logical expression, just by thinking about it. Saving you the context and mode switch. The real power comes from being able to simulate and model complex phenomena using wetware integrated tools. A big component of intelligence is the ability to create accurate mental models, and probe them. If you integrate an accurate physics simulator, a 3D modeler, a ray tracer, etc. You have greatly improved their internal modeling abilities and they can (theoretically) dedicate more neurons to higher abstractions that rely on these well grounded systems. It is also not clear that human level intelligence is any real limit. We could just as well make a super-intelligent AI without tool use. The current hype around tool use is just to make current LLMs a bit better.
- Animats 2y agoYes, that argument keeps coming around. It's become less convincing over time. LLMs are a new kind of intelligence. They sound like humans when trained on text written by humans. That reflects the training set, not the algorithm. We have a hard time getting an intuitive sense of what shape this kind of intelligence has. It's also changing very rapidly. ChatGPT was launched less than two years ago. > For some people the thing protecting them from a meltdown is just a view that superhuman machine intelligence isn't possible, and once that is broken they become vulnerable to cult indoctrination or just general dysfunction. AI would be a bigger political issue if more people understood what's happening. We have no idea how to organize a society where AIs do most of the middle class work. AI is going to break the "College = Success" paradigm more than it's broken already. That doesn't even require superhuman intelligence. Just mid-level human intelligence. The current frontier is getting machine intelligence to consistently avoid random stupidity. Hallucinations, totally wrong answers, that sort of thing. "Common sense" AI is still weak. If that can be fixed, the level at which LLMs currently operate will be superhuman.
- bee_rider 2y agoYeah, AI that was as intelligent as a normal person, or even a fairly dumb one, but which never dies and has unlimited ability to focus on a single problem would probably cause some social hiccups as well.
- ajuc 2y agoPenrose basically thinks that biological neurons do QM magic and therefore classical computers can't do what they do.
- barelyauser 2y agoPeople got very upset when chemistry proved that we are made of the same thing as "unalive" things like rocks. You can see the same now that we proved the same for intelligence.
- Ygg2 2y agoIncidentally his shots in dark have proven true. Microtubules do seem to rely on some QM tricks. http://backreaction.blogspot.com/2024/05/brain-really-uses-quantum-effects-new.html http://backreaction.blogspot.com/2024/05/brain-really-uses-q...
- bubblyworld 2y agoRight, but I think it's the spurious connections to intelligence/consciousness that people object to. You could replace most instances of "consciousness" in the emperor's new mind with a different word and the arguments would be just as strong.
- wrs 2y agoSo do tunnel diodes, but microwave oscillators are understandable.
- 2y ago
- AtlasBarfed 2y agoThe real stupidity isn't in arguing they aren't possible, it's in denying that superintelligence won't conclude humanity is dangerous, superfluous, and/or obsolete. It doesn't take much conversation with really smart people to see what ranges from mockery to indifference to seething resentment most have towards other "lessers". I myself used to joke about eliminating the "bottom 90%" (of course for some metric that I obviously would safely exceed). And human intellect is vaguely constrained by moral notions, culture, and pack instinctuality that should... prevent... mass... violence... against... each... other... um, well, yeah, excluding innumerable genocides and war. So if you make something markedly smarter than humans, not just at the individual level but more significantly at the collective level, I'm pretty sure it's going to eliminate or severely constrain (think: factory farms for pigs/chickens/etc or alternatively, an Amazon factory worker)
- silverquiet 2y agoAGI may well conclude that humanity is superfluous, but what will it think of itself? Presumably if it is shorn of any human morality, it could also lack any moral compunction against suicide and any emotional/innate need for self-preservation. I’ve always wondered, given that this existence is essentially meaningless (I’d say that isn’t exactly what I believe; more that I’ve failed to find a more comfortable belief), what is it that an artificial (super?) intelligence would actually want?
- card_zero 2y agoThat's a strange cliche I see over and over: > Presumably if it is shorn of any human morality Where does this assumption come from? Any artificial intelligence would need ideas, lots of them. Is it supposed to create them itself out of whole cloth? Or what, get its ideas from amoral aliens for some reason? No, its ideas can only come from our culture. So it's going to learn and internalize all of our ideas except the moral ones? Why make that assumption? It's just a sci-fi trope so that we can have stories about callous, emotionless robots - and it comes from the idea that they are machines that coldly carry out algorithms and that are not properly intelligent. Morality is weird, admittedly, because the is-ought problem puts it in a different domain from facts. This includes the urge to live and do interesting shit: you can't derive that from facts, because it's a basic moral value.
- card_zero 2y agoYes, my dumb position is, in essence, to respond by asking WTF are you even talking about? How does intelligence have levels, in your opinion, and how is "super-intelligence" meaningful outside of superhero comics? How about the parsimonious position that there's one form of intelligence with one level, pending an explanation of what it is and how it could be dialled up? Why assume it even has a dial? Are you just being overawed by IQ tests, which are notorious for measuring only ability to pass IQ tests?
- bee_rider 2y agoThat is a good point. Although, if we suppose intelligence can’t even be defined, then I don’t see how the question is even meaningful in the first place. We could I guess look at cognitive type tasks one by one, pick ones for which there is some sort of objective grading that can be done, and then ask if there’s any proof that a machine couldn’t do it better. But my gut says that machines should have a huge advantage in tasks where there’s an objective score. Anyway the satire falls a bit flat if we don’t suppose intelligence is a thing like size that can be easily defined.
- throwup238 2y ago> Although, if we suppose intelligence can’t even be defined, then I don’t see how the question is even meaningful in the first place. No one is really saying that it can't be defined, just that it hasn't been defined yet. Until this latest round of LLMs basically flushed the Turing test down the drain, no one but ivory tower philosophers had cared about a rigorous definition of intelligence and without something akin to LLMs to experimentally ground their theories in language (like Wittgenstein's work), several thousand years of philosophy on the topic is pretty much useless to us now.
- ctoth 2y ago> WTF are you even talking about? How does intelligence have levels, in your opinion, Having previously met humans, I have noticed using my intelligence that the intelligence of other humans differs, and exists on a set of spectra. It seems reasonable to me that the range from the least intelligent human to the most intelligent human does not completely cover the range of possible intelligence, just as the range of body temperatures from the coldest (living) human to the warmest (living) human does not define an absolute range for temperature. I'm very confused as to what you find difficult with this concept. Do you believe all humans are equally intelligent, or do you believe that humans encompass the entire range of possibility for intelligence?
- 0xcde4c3db 2y ago> Is anyone actually arguing that a super-intelligent AI is, like, physically impossible or something? John Searle's arguments about "causal powers" are sometimes interpreted that way. I think he doesn't discount the general possibility of AI, though, just the possibility of actual conscious/understanding minds arising from structures (such as computer programs) that lack some yet-unidentified physical similarity to biological brains.
- noqc 2y agoHumans are pretty optimized for intelligence. It's not the only evolutionary pressure we face but it's a pretty important one, and it's notable that we haven't seemed to get much bigger brains in hundreds of thousands of years. Given how much evolutionary pressure there is to be smarter, evolution could probably find a solution bipedalism/baby head size problem. Moreover, brains grow in size substaintially after birth. This is weak but real evidence that there's some other constraint in the mix. The most obvious candidate is that 3 dimensions is a pretty serious constraint for a physical computer. There are a number of different ways to operate wthin this constraint, but it always ends up being a wall. I've got to fit the computing elements in a volume, those computing elements take power, and generate waste, both of which must enter and exit through the surface. Technology can maybe do a better job at solving these problems than evolution, but since evolution has a much more energy efficient way of computing, I think it is way too soon to say definitively. The other candidate reason is more subtle. That what actually matters is the collective intelligence of humanity. Our greatest invention is the ability to communicate our knowledge, so that we don't have to start from scratch with each new human. The collective intelligence of humanity then, actually lives across 8 billion computers. Each intelligence is responsible not only for a section of the knowledge space, but for a unique representation of that knowledge space. Those unique representations correspond to unique sets of "natural" inferences, and allow humanity itself to be many many times more intelligent than our "smartest member". In this sense, what it means to be as smart as possible is similar to the question of how many proccesor cores I should put on a CPU. I believe it is not that controversial to suggest an upper limit to how complex a single CPU core should be.
- Hugsun 2y agoI'm not convinced that there is any evolutionary pressure on humans to become smarter. I'm talking about the time when humans could talk, but were still subject to proper pre-civilization natural selection. It's obviously useful to be intelligent, but after becoming the smartest animal in the world by a large margin, the utility isn't as clear. Out-smarting other humans was certainly useful but it's not obvious that the ability to model other humans with very high accuracy was better than just being a little stronger or faster.
- cubefox 2y agoIn 2017, when this was submitted, there were many people who said human level or superhuman AI was far away and belonged in the realm of science fiction.