9 ms·
John Carmack on AI
- glonq 3y agoAs is often the case, John is right. The peasants deserve crossbows ;)
- pixl97 3y agoAnd yet we are not given nukes. Going to be interesting where the lines end up in this new world.
- api 3y agoLanguage models are not nukes and there is no real world evidence that they will become anywhere near this dangerous, or dangerous at all really. Meanwhile you can buy an AR-15 in many states if you can fog glass, or order the supplies to do CRISPR gene editing yourself. We have people calling for bans on code and math based on nothing more than sci-fi. I too agree with Carmack that AI doomerism has a definite subtext of concern for changes in power dynamics.
- mike_ivanov 3y ago"no real world evidence that they will" what is that mean, anyways?
- smegsicle 3y agolet's break it down no: "there isn't any" real world evidence: "reason to believe" that they will become dangerous i think he's referencing how the only reason anyone would think that there's any kind of existential threat is from sci-fi doomerism however i feel like perhaps i missed the point of your comment, if so could you explain?
- mike_ivanov 3y agoEvidence necessarily belongs to the past. It is something that already exists - a known fact, an event, etc. How on earth there could be evidence of things that have never been? "There is no evidence that a black swan will be ever found". Hmm. OK.
- deleted 3y ago[deleted]
- deleted 3y ago[deleted]
- goatlover 3y agoSo far, there hasn't been evidence that AI poses an actual danger to human society. There are lots of predictions that it could, or fictional scenarios where it does. But no actual real-world cases.
- mike_ivanov 3y agoLet me rephrase. There hasn't been evidence that AI doesn't pose an actual danger to human society. There are lots of predictions that it could not, or fictional scenarios where it does not. But no actual real-world cases. See my point?
- api 3y agoThere is no evidence you will become a serial killer, but there is no evidence you won’t. I could certainly construct plausible scenarios where you do and even write a book about it. Therefore we should lock you up now. Banning things on the basis of evidence free speculation is a really dangerous road to go down.
- pixl97 3y agoProbabilistically there is some evidence you'll become a serial killer. For example, if you were being hired by law enforcement and you had written a number of books that stated that being a serial killer was a good thing, then you wouldn't get hired. There appears to be a particular type of black and white response in this thread that seemingly ignores that the world is black and white.
- mike_ivanov 3y agoI'm not asking anybody to ban anything. I'm just pointing at a flawed piece of logic. Again - there is no evidence (and cannot be! - by definition) of things that never happened before. Forecasts and opinions are not evidence, that's it.
- Difwif 3y agoThis is like someone saying there's no proof of God and then responding with there's no proof that there isn't a God. It's safe to say that the burden of proof falls on the one making outrageous claims which is you.
- hackernewds 3y ago> or order the supplies to do CRISPR gene editing yourself. woah what and why
- chrisco255 3y agoThe destructive power of nukes is not the stuff of fantasy and fiction.
- pixl97 3y agoThe destructive power of human intelligence isn't the stuff of fantasy and fiction either.
- mrtksn 3y agoNukes are not that useful for stuff other than extremely destructive explosions. Besides, we have knives. I'm afraid this mindset will soon demand everything to be pre-cut in licensed facilities and make the knives illegal. This desire for censorship and restrictions is infinitely ridiculous. Instead of addressing the issues of why would someone want to hurt someone or themselves, they try to eliminate the tools. This is going to fail, maybe people in UK or in the USA might end up not knowing how things work without formal licensing but people in Russia, Ukraine, Bulgaria, China, Vietnam will know. This attitude can actually lead to the downfall of the western societies. WTF the computer wouldn't tell me how to cook meth? That's something that junkies do and it's not the lack of knowing how to cook it that keeps me from doing it. I'm just curious, not looking to harm myself or anybody but I'm being lectured by a computer.
- BudaDude 3y ago> Nukes are not that useful for stuff other than extremely destructive explosions. I would like to mention that in the Fallout series, a game that depicts a different version of history where nuclear power was widely accessible, cars and robots have their own nuclear generators. The reality is that if nuclear power was common, we would not have any problems with energy or a power grid.
- mrtksn 3y agoNukes and nuclear energy are different things. It’s also not a secret how to do these things, they teach it to kids. The complexities come from the engineering the device and that’s actually the secret part.
- taeric 3y agoThey aren't that different, all told. There is a reason IEDs are a huge safety concern. I'm failing to see how that wouldn't be worse with nuclear power sources. Or am I mistaken and you could make it so that IEDs from nuclear energy couldn't be done?
- hackernewds 3y agowhat about giving all humans guns? the crossbow seems a loaded and limited example
- ed_balls 3y agoWhat about a crossbow with a warhead that can kill 100 people?
- SoftTalker 3y agoSounds like something good to have if you need to make your government behave.
- Cthulhu_ 3y agoCarmack says in one of the subthreads on there that he's pro-second-amendment, so that tracks I suppose. But it's kinda futile, it's just the illusion of having power and self-governance, when the others have machine guns and worse.
- SilverBirch 3y agoDoes this idea work at all though? Like for one, did peasants ever in history actually have crossbows and if they did, did it ever actually help them? Like to me the crossbow sounds like an expensive weapon that would likely be issued to soldiers in an army and the idea that a peasant would have one of their own seems unlikely. I don't know the history but it just doesn't sound right at all to me. Also the closest modern analogy I guess is America where everyone has a gun and the idea that those guns actually pose a threat to the US government or the hegemony of large corporations is... fanciful. The practical effect is inner city gun violence and school shootings, not some balancing effect against corporate greed. I'm just saying... if the analogy you're using to justify your position seems to be entirely wrong doesn't that kind of undermine your position? It's not obvious to me that "peasants with crossbows" is helping here.
- thelastparadise 3y agoLook up the history of guerilla warfare.
- gary_0 3y agohttps://nitter.net/ID_AA_Carmack/status/1708905454544282083 https://nitter.net/ID_AA_Carmack/status/1708905454544282083 (full thread for non-Twitter-users)
- dvdgsng 3y agoappreciated, as always
- skyyler 3y agoThere is an extension that redirects all twitter links to nitter: https://chrome.google.com/webstore/detail/nitter-redirect/mohaicophfnifehkkkdbcejkflmgfkof https://chrome.google.com/webstore/detail/nitter-redirect/mo... https://addons.mozilla.org/en-US/firefox/addon/nitter-redirect/ https://addons.mozilla.org/en-US/firefox/addon/nitter-redire... https://github.com/SimonBrazell/nitter-redirect https://github.com/SimonBrazell/nitter-redirect
- jeffrallen 3y agoI wonder why HN doesn't do this on its own, server side.
- hackernewds 3y agothat is a very ideological opt-out position to take. I would hope they would not
- baal80spam 3y agoOr, more general, LibRedirect: https://libredirect.github.io/ https://libredirect.github.io/
- 38 3y agohttp://farside.link/twitter.com/ID_AA_Carmack/status/1708905454544282083 http://farside.link/twitter.com/ID_AA_Carmack/status/1708905...
- RecycledEle 3y agoAfter reading that, I now identify as an "Unaligned Human."
- mysterydip 3y ago"You might be a force to be reckoned with, but there are a lot of forces to reckon with." Younger me needed to hear that. Hopefully it saves someone else some time and energy.
- mansamusamoney 3y agoIn life some people are a force to be reckoned with, others: a farce.
- Cthulhu_ 3y agoIt's a really nice and succinct quote, I should remember it. Makes me think of the second amendment and John's stance on that; a "well-armed militia" might be a force to be reckoned with, but if it comes to an actual confrontation they're just peasants with crossbows vs the US military. Not that I believe the US military would ever go up against their own people, mind you.
- thelastparadise 3y agoWarfare (as someone who worked in the industry of war) has a lot more to do with things other than weapons.
- Sargos 3y ago>if it comes to an actual confrontation they're just peasants with crossbows vs the US military History is littered with armies losing to peasants. The US military lost to a bunch of poor people in the jungle then lost to a bunch of poor people in the desert. US citizens, even just Texas, would most likely prevail easily over the US military especially as the soldiers would be wary of attacking their neighbors.
- haxiomic 3y agoI thought the discussion was getting to the important points once hyperbole was resolved but Carmack ducks out Carmack: > To answer the question directly: there is no way, even with an infinitely smart (but not supernatural) oracle, that a teenager can destroy all of humanity in six months. Even the most devastating disasters people imagine will not kill every human being even over the space of years. This is a common failure of imaginative scope, not understanding all the diverse situations that people are already in, and how they will respond to disasters. You could walk it back to “kill a million people” and get into the realm of an actual discussion, but it isn’t one I am very interested in having. --- I do think that's a discussion very worth having, asymmetrical advantage that AI gives is the concern here. A virus is a special arrangement of a few thousand atoms that we can now control and design on a computer. The defence is global vaccination programs costing billions of dollars and many lives in the process We will no doubt be able to use AI to develop vaccines, that isn't the bottleneck, the bottleneck is in testing and distribution. It's a fundamentally asymmetrical situation To be clear this is a risk without AI, but AI brings down the threshold of effort to make this more likely. I would love to be convinced otherwise
- Salgat 3y agoThe defense in this case would be everyone else's infinitely smart oracles.
- chrisco255 3y agoCan we actually control viruses? No. 2020 was evidence enough of that. We were comically bad at it. Thankfully, immune systems have co-evolved with viruses for millions of years and there's limits to what viruses can do. After all, the most successful viruses don't kill off their hosts (since they would also die off). It quickly became apparent that Covid could mutate quicker than we could possibly keep up with. Like the common cold and the flu, it iterates too quickly to eliminate entirely like smallpox. At any rate, it seems all you need for AI is attention (transformers + processing power). If AI ever becomes a real threat outside of generating believable fake images and videos, it's likely the counter is a diversity of AI models that can compete with the "evil one".
- lakomen 3y agoI wonder, if I had a child, would it be possible to raise it to not outsource the thinking process to a program? With the internet, I ask search engines (usually just the one) for answers other people might give to a problem I also have. Pre ~y2k that yielded great results, because the internet was populated with sensible information but smart and educated people. When the masses came, the results became gradually worse, but I still use the internet for answers instead of thinking on my own, but in many cases I'm forced to think on my own anyway. With AI the source information is already polluted and the program is fantasizing its answers. But there will be a point where it delivers accurate answers. My child, should I ever have one, would then use AI like any other tool. Will it have more resources at its disposal or would it be better that it comes up with its own answers? I am a single child. I spent a lot of time alone, thinking, making sense of the world, figuring things out on my own. I played C64 games, with terrible graphics, which trained my imagination. I learned English by playing those computer games. There came a time where I outgrew those simplistic computer games, turned to having a social life, going to parties, getting into music creation, DJing. Would that also be something my child would do? Use AI, grow up with values the AI suggests to it, abandon it later, search for its own way in life? Would it even learn another language, when the AI conveniently translates everything on the fly? I think, before adding yet another thing on our plate most people, including me, don't fully understand, maybe we should focus on the energy problem and having a sustainable place to live. Amy technological advance is based on having enough energy to fuel it, and in the state this planet, our habitat is right now because of years of uncontrolled energy use, energy must be a priority before anything else. Unless you believe that AI can solve that riddle. Recently in the news Microsoft wanted to build small nuclear reactors to power its AI construct, adding a long term health and waste problem. It's October, it's been over 25°C the recent days, the sea is still 21-23°C. Usually it's much colder. The amount of fires and floods this year is over the top. Well sorry if that went all over the place.
- ge96 3y agoI am a GPT luddite myself. My coworkers/friends keep using it as a base of thought. I ask them "how would you do it yourself", don't ask this program to make your program. I'm not arguing it's a fast way to learn but still... it's like will they be able to imagine on their own how to make something new.
- Dowwie 3y agoThis isn't his take on AI. It's his opinion about the arguments presented in Mustafa's fear mongering book about how dangerous AI is and what government should do about it.
- mianos 3y agoSomewhat against the rules, sorry dang, but half the topics today are very badly titled. It's a clickbait invasion. Sorry again dang. :(
- hackernewds 3y agoIs this an AI comment? Why are you apologizing to Dang
- noisem4ker 3y ago...for littering the comment section with an off-topic, meta remark about an ongoing situation with submission titles?
- drt5b7j 3y agoWhy are you asking that? In fact, it's unclear what you're asking.
- Kudos 3y agohttps://en.wikipedia.org/wiki/Rhetorical_question https://en.wikipedia.org/wiki/Rhetorical_question
- drt5b7j 3y ago[flagged]
- conkeisterdoor 3y ago
- siliconc0w 3y agoFor me the more pressing concern is more societal unrest, economic and monopolistic - e.g workers being displaced without enough new jobs being created. Or every communication channel being spammed with no ability to discern trustworthy/authentic content. Or too few large companies hoarding the GPUs and owning the biggest most effective models which allows them not only to consolidate power in the AI industry but own/decide winners in every other industry. You can kinda see this now with Nvidia basically trading GPUs for equity. The 'regulate it like nukes' thing because teenagers might ask it to end the world thing seems pretty far down the line.
- ChatGTP 3y agoI hope the uber captialists decide they can use AI to build a space utopa for their families in another solar system and go there :) I honestly don't give a shit about jobs, or having a job, I have one because the system says I need one, if I don't need a job, I'd be so so so much better off. I'm certain we have enough technology and nohow to feed, house and provide electricity for everyone on earth without them having a 9-5.
- trident5000 3y agoThis view is hilarious. That you think you can sit around and produce nothing while everyone else around you produces everything you need. Or that everyone can stop what they're doing and still provide you with that. You're just lazy with dreams of contributing nothing and thats not admirable. Until the day where robots are literally walking around wiping our ass, people need to do things.
- FirmwareBurner 3y agoCheck his username.
- gmerc 3y agoThe simple fix is to scale the maximum work week based on available work, partially socializing the efficient gain from AI to society which provided much of the training data. How much can be up for debate, but giving it all to corporates is going to be a replay of the run up to WW2
- SoKamil 3y agoI don't the term "Artificial Intelligence" because it inherently implies anthropomorphism which has a negative impact on the discussion.
- carabiner 3y agoCarmack is an Ayn Rand-loving Texan so he will ALWAYS oppose anything hinting of regulation or top-down control.
- paulddraper 3y agoYes, politically he is a libertarian. Bitcoin, Ron Paul, etc.
- fnordpiglet 3y agoI think the most likely way to achieve “AI alignment” is the peasants with crossbows. Everyone having a powerful AI at their fingertips provides a balancing factor that is only overcome if those tools are in the hands of very few. Trusting Google and various governments to “do the right thing” is absurd - even assuming good behavior and intention on their part, they just aren’t nearly as competent as people seem to assume. Everyone having access everywhere all at once provides the transparency and space to innovate, but also to counterbalance the use of AI in dark ways. I fundamentally don’t like the crossbow and guns and second amendment argument. AI isn’t a tool to murder with, having no other function than the death of your neighbor. I feel like it’s more like the “should you be required to license your seeds from a megacorp” type thing. Food is existentially important, and it could be weaponized. But it’s not fundamentally a weapon, it’s fundamentally a thing of utility.
- hackernewds 3y agoIf you replace "peasants with arrows", with "good guy with a gun" I think the whole argument rings a different tone..
- iudqnolq 3y agoIt's less "good guy with a gun" and more "some random country with a small to medium army". Access to violence doesn't democratize on the individual level, but as the parent comment alludes to weapons changes allowed peasant armies to defeat trained knights, changing the structure of society. You personally might not be able to exploit open AI fully, but there are lots of countries and orgs that could. Remember millions of dollars is loose change in the defense context.
- coffeebeqn 3y agoA finite resource fairly allocated to all? Which civilization are we talking about again? Certainly not humans. Also a billion or trillion or infinite ants can’t outsmart a single human. So unless some hard cap is enforced (how?) then mini AIs will probably not do anything
- RcouF1uZ4gsC 3y agoJohn Carmack is one of the of the people responsible for AI. Until GPU's became powerful enough, there really wasn't a way to do neural networks at scale, which was one of the reason it stagnated in the 80's. One of the main reasons, GPU got powerful is because of pc gaming. One of the main drivers of PC gaming was John Carmack and id.
- SoftTalker 3y agoSo is he going to be John Connor's target when he's sent back from the future? I guess not, or we wouldn't know of him.
- qup 3y agoI'm one of the people responsible for AI. Until I existed to be part of the large market Carmack and the GPU makers were appealing to, Carmack wasn't working on PC gaming and id.
- mellosouls 3y agoI find it somewhat depressing how there are already two starkly divided camps on the safety subject. It's like the culture wars again; there also seems quite a consistent mapping with the left being more on the safety/control side; the right reckoning it will all sort itself out if we let the market get on with it, contemptuous dismissals marking lines in the sand. And it's only really just started. :(
- bsaul 3y agobecause the dilemma between personal freedom vs living in group is a fundamental one. You'll find it everywhere, and that's also the reason why political groups have been historically structured around those kind of concerns. Just like once you start reading greek philosophers, you realize all the important questions have been asked 2K years ago. Since then, we've only been trying to find the best ways to accommodate with our human nature.
- 0xbadc0de5 3y agoI agree with Carmack's take.
- nunez 3y agoI agree with Carmack, but let's be real here. Asking the average Joe to crank out their own personal LLM is like asking the average Joe to stand up their own Mastodon (i.e. unrealistic). When given the choice between owning your data and spending two minutes to do so vs. opening $MEGA_LLM in your browser and telling it to make you Instagram for Guinea Pigs, $MEGA_LLM will win every time. Doubly so now that $MEGA_LLM is starting to appear in _everything_. Case in point: OpenAI is already well-past 100M users. many of them are _paid_ users (which is unprecedented). I don't think running your own LLM should be illegal (under what grounds would that happen anyway?), but I'm also concerned about the world depending on, like, five huge entities for all knowledge.
- hackernewds 3y agowhy is this bad? we already rely on 1 entity (Google) for 80% of query searches and another (Amazon) for 45% of product searches. the world hasn't crumbled, yet retrieving information has gotten magnitudes easier than trying to find an entertaining stream (Netflix, Prime, Peacock, Pluto, Tubi, Disney, Max, Paramount+, YouTube and so on)
- brokenmachine 3y ago>the world hasn't crumbled yet
- nunez 3y agoBecause those platforms provide an illusion of choice, and LLMs do not (by default). Having a single entity that tells you what it thinks is the right answer short circuits needing to think critically about the answer that was provided. This is especially bad when, in the case of ChatGPT, people are using it as a therapist, doctor, teacher, lawyer, pair programmer, and more. There is also the possibility of a central authority using the platform to tell you how to think. When you control the model, you control the narrative. I think this could be extremely dangerous in the wrong hands.
- strikelaserclaw 3y agoGive humans a way to be lazy and they will take it every time, this is just a fundamental law of nature.
- reducesuffering 3y agoHow can you take seriously the philosophical opinions who right after responds to: "What are your thoughts on gun control?" with "Yes, I am pro 2nd amendment." Like yes, of course, but are you implying carte blanch RPG's, Howitzers, and fighter jets for everyone? Because if so, then I absolutely don't trust you to make sure AGI doesn't screw up the world. And if not, then maybe you're not so "pro 2nd amendment" by interpretation.
- Cthulhu_ 3y agoGuns aren't AI, and I've yet to find a good explanation as to why AI is dangerous outside of science fiction where it becomes self-aware and takes over the military.
- SilverBirch 3y agoThis is the core irony of the AI enthusiasts - by massively over-stating the capabilities of AI they provide the perfect ammunition to AI doomsters. If the AI enthusiasts focused on what can reasonably done with AI today they would have a lot less to worry about from regulators, but instead they keep making claims that are so ambitious that actually.. yes, we probably should heavily regulate it if the claims are true.
- janussunaj 3y agoMy cynical take is that this is actually the PR strategy at OpenAI (and others that had to jump on the train): by talking up vague "singularity" scenarios, they build up the hype with investors. Meanwhile, they're distracting the public away from the real issues, which are the same as always: filling the web with more garbage, getting people to buy stuff, decreasing attention spans and critical thinking. The threat of job loss is also real, though we should admit that many of the writing jobs eliminated by LLMs are part of the same attention economy.
- didibus 3y agoI haven't read the book, so I feel I can't give it an honest criticism, but I'm assuming the book isn't so much about what the "peasant" or "teenager" would do with AI, but more what smaller unstable nations with rogue dictators, or terrorist groups, or cartels, etc. would do with it? At least this is what I find more worrisome personally. But realistically I'm not sure we can keep it away from them, similar to how North Korea managed to acquire nukes, except it seems way easier to acquire powerful AI in this case.
- matthewfelgate 3y agoThat sounds reasonable. I share concerns about government regulations and control over AI, just as much as the implications of AI itself.
- shortrounddev2 3y agoHas Carmack always been so outspokenly right wing?
- TerrifiedMouse 3y agoFrankly, hearing him speak about various things, he just sticks to the position he thinks is right and doesn’t really give a rat’s bottom what the “left” and “right” thinks.
- shortrounddev2 3y agoNot caring whether or not you're right wing doesn't make you any less right wing
- TerrifiedMouse 3y agoOften when you say someone is right or left wing, there is an implicit assumption that that person buys into this and that. Carmack IMHO can best be described as an independent - some of his opinions will just so happen to align with one side or the other.
- shortrounddev2 3y agoNo, when you say you're right or left wing it means you have the same opinions as people on one end of the spectrum or the other. Nobody is "independent" in that sense, only in party affiliation. Being someone who complains about statists and avows their support for the 2nd amendment sounds like a textbook conservative
- huytersd 3y agoWhich part is right wing?
- norwalkbear 3y agoI agree with Carmack