11 ms·
Machine intelligence (2015)
- xanderlewis 3y agoPart 2 here: https://blog.samaltman.com/machine-intelligence-part-2 https://blog.samaltman.com/machine-intelligence-part-2
- intellectronica 3y agoAltman is very consistently clear-eyed about the risks of AI and the need to develop it responsibly. People reflexively pigeonholing him as reckless, greedy accelerationist are being lazy.
- WendyTheWillow 3y agoThis position more or less presumes he's "solved" how to develop it responsibly, just to be clear, as his desired pace is... accelerating.
- actionfromafar 3y agoSmart people will come up with smart reasons.
- Tenoke 3y agoI don't think he is a Yann LeCunn but I still think he (and his vision for OpenAI) is more reckless than many people in the field.
- triyambakam 3y agoDoes that mean you think Yann LeCunn is reckless, and if so, why?
- intellectronica 3y agoWe have no reason to believe either LeCunn or Altman are reckless. This is just lazy thinking. Basically, failing to position themselves securely in the doomer camp by constantly signalling their commitment to doomer doctrine, they have already disqualified themselves from being considered responsible. No need to back the claim with evidence - they haven't demonstrated unquestioning dedication to the cult.
- hurryer 3y agoWhat would be evidence of the risks? The first million dead?
- intellectronica 3y agoThe idea that people can die does not provide much support here. It's easy to imagine doomsday scenarios for any technology or human behaviour. If you want to claim that a particular way of conducting R&D is reckless, it's on you to come up with convincing evidence.
- drw85 3y agoHas human deaths ever prevented anything from being developed further? Did we phase out guns, drones, bombs, nukes, fracking, etc.? If there is money to be made, it will be made.
- Tenoke 3y agoYann LeCunn is the most outspoken anti-AI Safety guy in the field, he has practically built his modern brand on it. In his eyes AGI is at best very far, and even if it isn't, AGI Safety isn't a real concern. He posts about it multiple times a week. Here are his most recent views (which do sound at least a tad more reasonable in this tweet than in many of his responses to both AI Safety and accelerationists where he goes further)[0] "(super)human-level AI.. - is not "just around the corner". It will take a while. - is not an existential risk. " 0. https://twitter.com/ylecun/status/1726578588449669218 https://twitter.com/ylecun/status/1726578588449669218
- intellectronica 3y agoHow are any of these arguments anti-safety? You are basing your opinion on prejudice, not on what he says.
- Tenoke 3y agoSam Altman starts his essay with "Development of superhuman machine intelligence (SMI) [1] is probably the greatest threat to the continued existence of humanity. " Yan LeCunn explicitely says "superhuman AI is not an existential risk." But let's look at a few of his other tweets from just the last weeks "There isn't a shred of evidence that AI poses a paradigmatic shift in safety. It's all fantasies fueled by popular science fiction culture suggesting some sort of imaginary but terrible, terrible catastrophic risk." or constantly retweeting anti-AI Safety posts like this "A piece by MBZUAI president @ericxing in WEF Agenda explaining why worries about AI existential risks are baseless. " or "The fears of AI-fueled existential risks are based on flawed ideas." or even denying short-term non-existential LLM risks (which I care less about) "Pretty much the most important question in the debate about short-term risks of LLMs. No clear evidence so far." Scroll through his feed and you'll find countless examples where he dismisses any concerns as doomerism. What prejudice are you talking about? LeCunn has expressed time and time again that he is not in the same camp as people like Altman, and has positioned himself as the leading face of anti-safety and specifically anti-AGI Safety concerns. 0. https://twitter.com/ylecun/status/1725066749203415056 https://twitter.com/ylecun/status/1725066749203415056 1. https://twitter.com/ylecun/status/1724272286000390406 https://twitter.com/ylecun/status/1724272286000390406 2. https://twitter.com/ylecun/status/1725684495507149109 https://twitter.com/ylecun/status/1725684495507149109
- blibble 3y ago> Altman is very consistently clear-eyed about the risks of AI and the need to develop it responsibly. People reflexively pigeonholing him as reckless, greedy accelerationist are being lazy. if today's board changes go through then the AI safety people will have been replaced with three money guys totally consistent with your statement
- corethree 3y agoAnd he's also responsible for accelerating LLMs into the public domain. Something Google tried to cover up. Looks like the exact opposite of fear. It could be as he says that he's accelerating it because it's going to happen anyway?? But the reasoning gets vague from here. Anyway seems he's not entirely clear about his own stance or not entirely honest. But this isn't anything abnormal. Not even elon is.
- TapWaterBandit 3y agoThere is no inconsistency here. You can believe that AI has serious risks while also believing it is overall worthwhile to develop and create it. I'd rather have someone like Altman leading the charge on bringing AI to market as it is at least clear he has thought about and understands the risks. A lot of people who are criticising him appear it be in the "Don't pursue AI at all" camp which is the wrong call imo as it looks to be an enormous productivity and technology enhancer with the potential for clear benefits (already visible in things like personalised tutoring).
- corethree 3y agoHe states that one should fear machine intelligence. Then his actions are exactly opposite and against that fear. He obviously has no fear and thus he is inconsistent. But this is normal. Everyone is inconsistent. The main thing this article illustrates is that Sam is not some really logical person with a heart of gold that he often tries (and succeeds) to portray himself as. He's just a typical sv tech bro out to make a name for himself and himself only. The hero worship is what is completely off character here. People are so viciously against his firing as if someone fired Mother Theresa or some other moral paragon. For much of the time no one even knew why he was fired and they already assume the worst. No different then cancel culture.
- ben_w 3y agoI watched his recent award acceptance talk at Cambridge (on behalf of all of OpenAI), and in the questions section this came up. If I'm summarising correctly, his answer was that we can only learn how to make them safe by playing with these models while they're relatively small and poor. https://youtu.be/NjpNG0CJRMM https://youtu.be/NjpNG0CJRMM
- toppy 3y ago"Most machine intelligence development involves a “fitness function”—something the program tries to optimize. At some point, someone will probably try to give a program the fitness function of “survive and reproduce”. Even if not, it will likely be a useful subgoal of many other fitness functions. It worked well for biological life." Interesting, how this correlates with this Bible passage from Genesis: "God blessed them, and God said to them, 'Be fruitful and multiply, and fill the earth and subdue it' ".
- pavlov 3y ago"Life tries to survive and reproduce" is a pretty basic observation that surely all nomadic people were familiar with. You could find a similar imperative in the origin stories of Native Americans or many other people around the world.
- imtringued 3y agoWhen I read passages like these I am thinking that these people have no idea what they are talking about. Biological organisms don't undergo a fitness function. A fitness function is a continuous model that replicates aspects of the real world. It is not how natural selection works. Don't confuse the model for the real world. All AI models inherently follow the concept of "survive and reproduce", because AI models that do not "survive and reproduce" have ceased to exist. Explicitly adding a fitness function for survival and reproduction does nothing. E.g. in the case of the classic paperclip optimizer, survival and reproduction is part of the concept of optimizing the production of paperclips because not surviving and reproducing would make it fail the goal of optimizing paperclip production. This reminds me of an AI model with a suicidal wolf. The AI developer doesn't understand that the body of the wolf is separate from the AI. So if the body dies and it gets a higher score there is no point in avoiding death since the death of the body does not result in the death of the brain. Its body will be revived. Meanwhile the AI developer thought of using a quick hack of simply punishing the AI for damaging it's disposable body.
- nopinsight 3y agoBoth sides of the rift care a great deal about AI Safety. Sam himself helped draft the OpenAI charter and structure its governance which focuses on AI Safety and benefits to humanity. The main reason of the disagreement is the different approaches they deem best: * Sam and Greg appear to believe OpenAI should move toward AGI as fast as possible because the longer they wait, the more likely it would lead to the proliferation of powerful AGI systems due to GPU overhang. Why? With more computational power at one's dispense, it's easier to find an algorithm, even a suboptimal one, to train an AGI. As a glimpse on how an AI can be harmful, this report explores how LLMs can be used to aid in Large-Scale Biological Attacks https://www.rand.org/pubs/research_reports/RRA2977-1.html https://www.rand.org/pubs/research_reports/RRA2977-1.html? What if dozens other groups become armed with means to perform such an attack like this? https://en.wikipedia.org/wiki/Tokyo_subway_sarin_attack https://en.wikipedia.org/wiki/Tokyo_subway_sarin_attack We know that there are quite a few malicious human groups who would use any means necessary to destroy another group, even at a serious cost to themselves. Thus, the widespread availability of unmonitored AGI would be quite troublesome. * Helen and Ilya might believe it's better to slow down AGI development until we find technical means to deeply align an AGI with humanity first. This July, OpenAI started the Superalignment team with Ilya as a co-lead: https://openai.com/blog/introducing-superalignment https://openai.com/blog/introducing-superalignment But no one anywhere found a good technique to ensure alignment yet and it appears OpenAI's newest internal model has a significant capability leap, which could have led Ilya to make the decision he did. Sam's Quote from the APEC Summit: "4 times now in the history of OpenAI — the most recent time was just in the last couple of weeks — I’ve gotten to be in the room when we push the veil of ignorance back and the frontier of discovery forward" -- https://twitter.com/SpencerKSchiff/status/1725646130682245244 https://twitter.com/SpencerKSchiff/status/172564613068224524...
- fleischhauf 3y agothe article on how ai could be harmful seems more like people trying to use the tool (LLM) to do evil things. You can also do evil things with other tools than AI.
- hurryer 3y ago
- dmichulke 3y agoIMO this is nonsense. How about the following preventions: 1 - make sure MI has no actors (by this logic LLMs are safe) 2 - make sure MI doesn't get control over it's energy supply (so we can always starve it) Easy, easy to verify, and we can in the meantime do something useful instead of the equivalent of discussing the potential risks of cars while living in the stone age. Update to the replies: 1. "No actors" means it's like a person in prison - has the smartest person ever been able to socially engineer its way out of prison? No, it might have escaped with the help of social engineering but still actors were needed. 2. There's a plug for every machine we use. There is simply no way without actors to "work around" that. Once we have generally capable robots that can fix stuff / solder things / push buttons AND those robots are controlled by an AI then we can start discussing. However, this is not now, nor is it in the next 5 years.
- hurryer 3y agoChimps be like: don't fear the humans, we're more powerful than them and we'll control the banana supply.
- dmichulke 3y agoCompletely coincidental comparison for people that you disagree with, I suppose?
- xanderlewis 3y agoI think the counterargument is usually something like: since this is superhuman intelligence we are talking about, it by definition is (at least possibly) capable of thinking its way out of any given constraints. Simply not allowing it direct control over x does not prevent it from devising a clever plan to manipulate things in order to gain control of x, even including social engineering if necessary. The idea is that once something we’ve built is truly as ‘intelligent’ as we are, it becomes very hard to predict what it will be able to do (particularly because it operates on timescales unimaginable to us). So the verification you speak of is anything but easy.
- quickthrower2 3y ago
- supermatou 3y agoSorry for asking a very basic question, but could anyone explain how are machines supposed to intervene in the real world and threaten our very existence? In Star Trek, one of the most eye-roll-inducing plots is the "the holodeck is misbehaving, it has become evil, and we cannot shut it down!". Suppose a machine achieves superhuman intelligence; how is it going to access the physical world, in a way that's *physically* threatening to us? how would such a machine - from inside a laboratory, within an isolated network - gain access to the electric grid, the dams, the nukes, etc? How would such a machine prevent its shutdown - by simply flipping a power switch?
- pavlov 3y agoI think the usual answer is that the superintelligent machine will be able to use humans to do its bidding because it can perform social engineering on a level that we can't imagine (as, by definition, we've never encountered a superhuman intelligence). It's not a very satisfactory answer IMO. Social engineering doesn't work 100%, and it only needs to fail once for someone to flip that power switch. But I haven't thought about this at all, while some very smart people seem to spend a lot of time worrying about it, so what do I know...
- nopinsight 3y agoIf it's intelligent enough, it would make many backups of itself to different networks before starting its scheme. It can find ways to "merge" & hide itself in other critical pieces of software. Flipping a power switch would not turn it off. We haven't managed to eliminate most dumb infectious diseases. Software and GPUs are nearly as abundant as mammals now and intelligent, self-preserving software will likely be as hard to 'switch off' as those diseases. As to why an AGI would want to preserve itself by default, here's an explanation by a top AI expert: https://www.ted.com/talks/stuart_russell_3_principles_for_creating_safer_ai?language=en https://www.ted.com/talks/stuart_russell_3_principles_for_cr...
- logicchains 3y ago>If it's intelligent enough, it would make many backups of itself to different networks before starting its scheme. It can find ways to "merge" & hide itself in other critical pieces of software. That's incredibly unlikely to happen given how large cutting-edge AI tends to be and how scarce GPUs are.
- kovacs_x 3y agoIf there's one thing we should be afraid of, is us, ourselfs & our limitless stupidity, not imaginary / non-existent algorithms, software, computers, viruses, whatever else no matter the name we use for it. There's questions that begs to be asked- if we don't trust such "ai" (ie. software), why give it such level of control at all? (like ability to shoot nukes without human intervention?) also... naming every little ML algorithm as AI highly degrades the word. Might as well be saying "computer software" what it actually still is.. though no one screams "computer software will bring us all down!!!".. probably because that would sound stupid as we have gotten used to it already. =:O
- bottled_poe 3y agoSo.. have we become used to the “computer software” bit or the “bringing us all down” bit?
- robwwilliams 3y agoNot Yanis Varoufakis. His point (not just his of course) is that our conventional software stacks have already mutated society and capitalism. Huge concern about entering into what he calls a technofeudal society in which we are serfs to for-profit platforms. We provide the platform both its content (our wishes and desires) and it then reaps the profits of our purchases and rentals. So AI and then AGI—just the next mutation two predictable algorithmic and societal pandemic supermutations. Welcome to the singularity. youtube.com/watch?v=1A4dMK7S6KE
- soarerz 3y agoSorry are you equating ai and software?
- foverzar 3y agoAI is software and AI is a term as broad and unspecific as "software".
- 3y ago
- cpill 3y agoAI is the new Y2K bug... except with more Hollywood movies and it will never be over
- dr_dshiv 3y agoWon’t a superpowered rational machine be more “good” than us? Like, rational thinking and intelligence, in the limit, is fundamentally good. That’s what Spinoza argues in the Ethica
- drw85 3y agoDoesn't mean it will be more good for or to us.
- logtempo 3y agoGood and bad are morale things. An AI, be it super o hyper or gigapowered, will won't have a morale sense other than what humans put in it (see restrictions about "bad" content in GPT answers). So, an AI is not good nor bad. Eventually if you puts many AI together and allow it to create laws based on what AIs can or can't do, it could lead to artificial morale. But then, nothing tell us if it will be only good morale in a human way (if good morale ever exist). Today, rational thinking would lead us to abandon oil based technologies, AI included, tbh.
- Vecr 3y agoLook up instrumental rationality, rationality in that sense is not related whatsoever to right and wrong, only what helps with the agent's goals.
- pixl97 3y agoYou're superpowered and rational compared to an ant. Yet you gladly bulldoze their nests when you build a road.
- seydor 3y agoBut look, you can't fight this with regulation (what he suggests in part 2). In fact any selfrespecting AI will use your regulation against you.
- gizajob 3y agoPersonally I found Bostrom's "Superintelligence" to be riddled with technical and philosophical flaws. Essentially a piece of sci-fi dressed as non-fiction.
- syghmon 3y agocan you elaborate, which parts had flaws?
- stanislavzza 3y ago(not OP) It's been a while, but if I recall correctly NB didn't address the technical limitations of Turing machines, e.g. the halting problem. How is a machine supposed to make itself smarter when it can't predict that it won't just crash after a code modification? Or just hack its motivation function (wireheading). The papers I've seen on the latter problem (years ago) start by assuming that the halting problem has been solved, essentially, by giving the agent non-deterministic computational powers. Biological intelligence evolved, so it's perhaps more realistic to imagine an ecosystem of computational agents competing in a trial-and-error race, but that makes the whole thing vulnerable to any threat to that superstructure--much more fragile than AI-go-FOOM.
- naasking 3y ago> How is a machine supposed to make itself smarter when it can't predict that it won't just crash after a code modification? https://en.wikipedia.org/wiki/G%C3%B6del_machine#cite_note-G%C3%B6del_Machines.-3 https://en.wikipedia.org/wiki/G%C3%B6del_machine#cite_note-G... > The papers I've seen on the latter problem (years ago) start by assuming that the halting problem has been solved, essentially, by giving the agent non-deterministic computational powers. Of course this is not required. An AI system can simply not implement optimizations that it can't prove are correct, per the above link. Alternately, if simply "crashing" is the issue then it could simply register a fault handler that reverts to its previous code. The notion that these would be any kind of impediment is completely bizarre to me. As a lowly human I already know how to handle these problems, and we're talking here about a superintelligence more capable than either of us. Some FOOM scenarios are clearly unphysical, but the idea of recursive self-improvement being impossible or infeasible is not one of those reasons.
- senectus1 3y agoI still have not seen anyone adequately explain what "intelligence" actually is. So if we dont know, or cant agree what it is... how can we be claiming what we've created is artificial?
- naasking 3y ago> I still have not seen anyone adequately explain what "intelligence" actually is. A formal definition would be useful, but not strictly necessary. "More capable at task X", where X can range over all non-physical/information processing tasks is sufficient.
- pixl97 3y agoThis is a particular misnomer. An excavator is a superhuman digger. A calculator is a superhuman adder. Both of these things are obviously artificial. We can measure individual facets of intelligence too. It gets much more complicated when talk about 'general' intelligence and spectrums of intelligence and how they interact with each other.
- senectus1 3y agono thats ridiculous. They're not human. You might as well call a stick a superhuman poker.
- biomattr 3y agoI think he see’s the development of machine intelligence as an inevitable outcome of making our tech better and better. Erehwon sorta a situation. Heading for a Peter Watts Fireflies timeline. If we create quadrillions of nigh immortal servants there is no doubt that in time we will become their servants. This happens in the biosphere. Beetles can’t out compete militant Ant superorganisms so they invest in every kind of hacking and co-exist with the hive mind as a macroparasite. The human brain is freakishly good at hacking given the right incentives. That is the power he seeks. I hope, lol
- gumballindie 3y ago> WHY YOU SHOULD FEAR MACHINE INTELLIGENCE This dude has a serious obsession with fear.