19 ms·
Planning for AGI and Beyond
- blibble 4y agoif AGI does happen: I wonder how they plan to deal with the billions of newly redundant people unable to earn a living data centres and fabs are an extremely easy target for an organised mob ludditism 2.0
- hollerith 4y agoThis roadmap does not allay my fears. People like Eliezer Yudkowsky have been loudly proclaiming since 2007 (1) that there is an infinite variety of possible minds, (2) that among the powerful minds, there is an infinite variety of goals -- or "regions in the space of possible outcomes" -- they can be aimed at and (3) that a superintelligence would have to be aimed at a very small and very particular region in the space of possible outcomes for there to be any hope for the survival of the human race. I have seen no argument to the effect that that is either untrue or irrelevant, but there is no sign in this roadmap that the author understands that basic point or that he understands that no one has a credible plan for how to aim a superintelligence at anything let alone at a goal compatible with human survival. There is also no acknowledgement (explicit or implied) of the existence of an organization (namely, MIRI) started 15 years before the start of OpenAI that has been focused from its start on the problem of how to aim a superintelligence. There is no acknowledgement of the fact that MIRI is currently saying that they do not see any plan for how to aim a superintelligence that will not take decades -- and we don't have decades before some reckless actor creates a superintelligence that will kill us all. Expanding on my (3) above: the explanation for that is that the vast majority of possible minds neither hate nor love humans, but humans (and things they rely on like cities and farms) are made of atoms that can be used for something else. Also, before the superintelligence has consolidated control over Earth, the humans represent a threat to it: they might figure out a way to turn the SI off or (having already succeeded in creating one) they might create a second superintelligence, which would almost certainly compete with the first SI. The easiest way to minimize that threat is to kill all the humans. Definition: a superintelligence is a generally-intelligent AI better than every human at every endeavor. A chess engine is better than every human at chess, but has no knowledge of or skill at anything else. ChatGPT is generally-intelligent, but inferior in skill and knowledge to most humans. The only thing stopping ChatGPT and all other current generally-intelligent AIs that I know of from killing us all is that they lack the cognitive power to do so, but of course AI researchers are rushing to increase the cognitive power of the next generation of generally-intelligent AIs, which is the primary danger facing humankind. Also, the author shows no signs of having considered the possibility that gradient descent is simply the wrong approach and that is it simply not humanly possible to use that approach to create a superintelligence that is not fatal to the human race. Sam Altman continues to emphasize and to focus on the possibility that the power conferred by AI will be unequally distributed among the humans. In reality, what would happen if Vladimir Putin and his allies obtained a decisive advantage in AI is that the AI Putin mistakenly believed is under his control would kill him, his allies and the rest of humankind instead of humankind's being ended by an AI created by OpenAI or DeepMind. The situation is so dire -- the probability of human survival is currently so low (unless by some miracle all AI research is stopped) that Putin's ending up as the only human who can control a superintelligent AI should be considered a very good outcome -- a much better outcome than we can currently realistically hope for (good because at least some humans would survive)! The mere fact that Altman continues to emphasize the unequal-distribution possibility as heavily as he does is strong evidence that he is still not taking the main danger seriously -- even though at least one book (e.g., Nick Bostrom's 2014 book Superintelligence) had been published about the main danger before Altman entered the AI game 8 years ago. (The arguments were available by 2009 in the form of linked web pages, but 2014 is the first time I am aware of that the arguments were made available in convenient, compact book form.) I'm not asking that Altman and the other leader of the AI research enterprise subscribe to or believe the arguments, I am asking that they acknowledge they even exist, and maybe explain why they do not find them convincing. I certainly find them convincing as do many others.
- emrah 4y ago> and for AGI to be an amplifier of humanity Humanity needs to get to the StarTrek 24th-century level or thereabouts before it's safe to amplify. > There are several things we think are important to do now to prepare for AGI. It still seems far too soon to talk about AGI. A giant statistical model, while a great improvement, is hardly an "intelligence".
- thejarren 4y agoI appreciate the caution it seems like they're taking, and I personally am undecided if AGI will quickly balloon out of control. But given the value that it means for the world, I think the progress seems cautious and proactive. I really liked Satya Nadella talking about how GPT was impacting the lives of farmers in India within a couple of weeks, and I'm excited to see how this could bring more opportunity to more people.
- tspike 4y agoThis reads a lot like Google's "Don't be evil."
- seany 4y agoThe big difference being at the time was seen as neutral to good, vs openai which started as questionably good to bad and had just continued to do things that show they are even worse than they say.
- ColinHayhurst 4y ago“Don’t seem evil”
- whiplash451 4y ago« As our systems get closer to AGI… » That’s a pretty bold statement right there
- harveywi 4y agoThey're gonna need a bigger trie.
- outside1234 4y agoThis is just a schtick they are going to pull up until someone stupid buys them.
- lkbm 4y agoCertainly debatable how close it is, but it seems obvious that it's improving in general ability. It's surely not moving away from AGI.
- moffkalast 4y agoMaybe more of a slightly decaying orbit around AGI.
- robertlagrant 4y agoThere doesn't seem to be any indication that we can make an AI that can drive, let alone do everything.
- moffkalast 4y agoThe scientists went too far. They wanted to make an AI that can drive, but ended up with an AI that can drivel.
- BobbyJo 4y agoAIs can drive, just not as well as humans on all axi yet. They are better than humans on some axis in fact.
- underbluewaters 4y ago
- appleaday1 4y agoWill AGI tell us if there is aliens or not? Are we really alone in the universe?
- optymizer 4y agoIf we manufacture AGI robots and send them to Mars, we will not be alone in the Universe.
- astrange 4y agoWe aren't making progress on anything that'd lead to "AGI robots". Online AI analysis all seems to come from lesswrong posters who think computers are magic and can do anything though.
- pixl97 4y agoWe are not? I think you're making a number of really tiny boxes that limit intelligence to the things that humans can do, and the limitations they have. Meanwhile I'm watching more and more standalone humanlike robots with multiple degrees of freedoms being developed. The robot unit itself doesn't even AGI/full capabilities as long as it can talk to a 'brain' somewhere close enough it has low latency.
- astrange 4y agoThe main source of that hype Boston Dynamics makes YouTube videos more than robots really. We don't have battery technology good enough to make a robot operate independently, which it'd still need to do even if you kept the brain somewhere else. But the "brain" technology doesn't exist either; language models can't pilot robots.
- famouswaffles 4y agoLLMs can pilot robots just fine lol https://innermonologue.github.io/ https://innermonologue.github.io/ https://sites.research.google/palm-saycan https://sites.research.google/palm-saycan https://www.microsoft.com/en-us/research/group/autonomous-systems-group-robotics/articles/chatgpt-for-robotics/ https://www.microsoft.com/en-us/research/group/autonomous-sy...
- mxkopy 4y agoAh yes, the noble declarations of righteousness and forethought. I am so grateful that such an esteemed and pure-of-heart company is taking care of our future. /s
- tpmx 4y agoRelevant reading: https://news.ycombinator.com/item?id=34471720 https://news.ycombinator.com/item?id=34471720 Ask HN: How did Sam Altman fail upward so well? (33 days ago, 119 comments) (skip down a bit for the real content)
- ivraatiems 4y agoWhy don't you just... not? I do not agree the upsides of AGI are great, or outweigh the downsides. Look at the massive downsides of the tools you've already released. Your track record is already terrible. And even if I believed benefits were there, there's no way in hell I would trust a for-profit corporation to manage them appropriately. I'm not sure there is an organizational structure that currently exists that I would trust. Just stop.
- maccaw 4y agoYou don't think solving every disease is a significant upside?
- ivraatiems 4y agoYou cannot prove or promise that an AGI would be willing to do that, or anything, even if it could. The whole problem of "general" intelligence is that the "general" implies agency. If an AGI exists, it has agency, and it can do or not do things. The humans with agency we have in the world are already enough of a problem. Let's not add superintelligent computers.
- mxkopy 4y agoYou don't think OpenAI won't price-gouge the shit out of everyone suffering from those diseases? Not even that - you don't think they won't sell exclusive licenses for this kind of thing?
- jackblemming 4y ago>As our systems get closer to AGI, we are becoming increasingly cautious with the creation and deployment of our models. Who do these people think they are? They didn’t even invent the architecture GPT3 is built on! This is as bad or worse than Elon claiming full self driving is just around the corner for almost a decade!
- maxdoop 4y agoAre you just talking about transformers? OpenAI utilized them, yes, but they are the ones who authored the original GPT paper. I am not sure I follow.
- lobstersammich 4y agoI think they mean Transformers in the Vaswani et al 'Attention is all you need' paper, not Generative Pretrained Transformers, specifically? Paper link below: https://proceedings.neurips.cc/paper/2017/file/3f5ee243547dee91fbd053c1c4a845aa-Paper.pdf https://proceedings.neurips.cc/paper/2017/file/3f5ee243547de... For some papers on attention mechanisms from before the 2017 'Attention is all you need' paper, check out that paper's references. Chris Manning's 2015 paper covers attention mechanisms. And so do a few other researchers from that mid-2010s time period: [21] Minh-Thang Luong, Hieu Pham, and Christopher D Manning. Effective approaches to attention based neural machine translation. arXiv preprint arXiv:1508.04025, 2015. [22] Ankur Parikh, Oscar Täckström, Dipanjan Das, and Jakob Uszkoreit. A decomposable attention model. In Empirical Methods in Natural Language Processing, 2016.
- gcr 4y agoMy understanding is that the original Transformer paper ("Attention is All You Need") and BERT papers came from Google. In particular, BERT's pre-training task is to predict how to fill-in-the-blank from a sentence that's missing a random word. OpenAI started from that architecture and made it output text generatively, by using beam search on top of a next-token prediction task.
- corbulo 4y agoIt attracts more investment
- amrb 4y agoIf AGI gets created.. I hope it scans humanity, turns around and tells us we suck!
- mjirv 4y agoOpenAI arguably already has AGI. GPT-3/ChatGPT is much better than the average human at coding, text summarization, etc. (“system 2” cognitive tasks).
- ivraatiems 4y agoChatGPT is almost as good as a pretty mediocre human at some tasks. I would not say it's even close to "much better than the average."
- mjirv 4y agoThe average human cannot program, not even a little. The average human cannot summarize (for example) Anna Karenina from memory.
- ivraatiems 4y agoThat's not a fair comparison, the average human hasn't been trained in those tasks, and ChatGPT has. Extensively, in fact. If you taught the average human to do either of those things, they'd do better than ChatGPT does.
- pixl97 4y agoDefine 'extensive training' on these particular items. It was given a large corpus of data, and a relatively limited amount of "yes that's right/no you're wrong" on the answers it is formulating. But even beyond that point, you have to take a bunch of human time to train an individual those things you're talking about, and then they quit, or get hit by a car. Already GPT has a far larger corpus of information than any individual can hold. It won't quit its job. It can be backed up and restored. And every 6-12 months the systems are getting better. It's not a fair comparison at all, humans just keep sucking under those constraints.
- rvz 4y ago> OpenAI arguably already has AGI. Just no. > GPT-3/ChatGPT is much better than the average human at coding, text summarization, etc. (“system 2” cognitive tasks). I don't think you even believe what you have just said there. Given it hallucinates 98% of the time and produces incorrect code and also is unable to reason about why it is wrong, it is completely untrustworthy and requires humans to triple check the output. It cannot be used unsupervised and needs constant review. We haven't even gotten started with the nonsense that it outputs or how it cannot transparently reason about its own decisions in the most serious of applications such as finance, law and medical uses. Meaning that its use is instead reduced to just sophistry generation.
- nathias 4y agowe're going to be biodiesel
- sethbannon 4y agoThis post makes me extremely hopeful. It's great to see optimism and caution hand in hand like this -- a full acknowledgment of the potential for good and the potential for harm, and some clear principles as to how to navigate that. Very curious to see how they flush out the "training should be audited" bit in the future.
- bottlepalm 4y agoTo me it's kind of scary. Like they know they're sitting on a bomb, so they're trying to tell us the safest way they plan to detonate it.
- rvz 4y agoBefore: > Our goal is to advance digital intelligence in the way that is most likely to benefit humanity as a whole, unconstrained by a need to generate financial return. [0] After: > Our mission is to ensure that artificial general intelligence—AI systems that are generally smarter than humans—benefits all of humanity. In this sentence, they missed out the part where they are to be taking VC capital and selling out closed AI snake-oil to the masses for the dream of AGI science fiction, whilst building untrustworthy systems that are glorified hallucinating black-boxes as a service. Another overhyped multi-billion dollar exit scam. [0] https://openai.com/blog/introducing-openai/ https://openai.com/blog/introducing-openai/
- d11z 4y agoOverhyped? For sure. Exit scam? I think you’re exaggerating quite a bit.
- RcouF1uZ4gsC 4y agoIf we really have AGI, it won’t be able to be controlled. We can’t even come up with a set of foolproof laws for humans and really on judges and courts for interpretation. Notice how many laws have “reasonable” in them (for example reasonable doubt , reasonable care, reasonable suspicion, etc). Once we have AGI, it won’t be stoppable no matter what we imagine ourself doing. It will basically be like a teenager with superhuman intelligence.
- robertlagrant 4y agoThere is zero evidence that we will get AGI.
- RcouF1uZ4gsC 4y agoI actually agree with you. But if we did, we would not be able to control it.
- GeorgeTirebiter 4y agoI never really understood this argument. Unless we attach ICBMs or whatever like they did in Dr. Strangelove or The Forbin Project to the AGIs --- we can just unplug them. You know, pull out the Ethernet cable, turn off the WiFi, remove the AC wall plug, turn off their UPSs. As long as the I/O is limited, I see no problems -- at all -- with AGI. I am clearly very confused, wrong, or lacking insight into this question.....
- hotpotamus 4y agoPresumably a sufficiently advanced AGI could tweak it's own circuits in such a way as to create RF waves and interact with WiFi or some other radio based networking and insinuate its way onto a network even if you tried to airgap it by not giving it a physical ethernet cable. Basically, you have no idea what something orders of magnitude more intelligent than you is capable of; that's the concern. Personally, I wonder what would motivate an artificial super-intelligence in the first place.
- noawaytothrow 4y agoThere is no reason to trust that Sam Altman’s definition of “good for humanity” has any other meaning than “good for Sam Altman.” Everything created by Silicon Valley since 2010 has been a net harm to society, and people who got rich from that harm, like Sam Altman, are the ones steering the ship. It seems obvious, from a distance, that this is all going to go very badly.
- wozer 4y agoThe thing is: even when people do have good intentions, they tend to develop biases that serve themselves. If these developments in AI are really as important as this blog entry says, then it needs democratic control.
- seydor 4y agoI don't like the royal "We". I hope these things get opensourced soon because some people are in a hurry to play god with the rest of us. Congrats to openAI for being first, but I hope they end up as a footnote in history
- idlewords 4y agoSo a staple belief about hyperintelligence in the last couple of decades was that there would be no obvious ceiling on how smart a recursively self-improving AI could get. The assumption was that intelligence was a property of some (possibly very complex) algorithm, which could be refined without end. But in the AI models we have now, the intelligence is an emergent property of the training data, while the algorithm is fairly simple. So my question is, to what extent is the growth in 2023-style AI capped by the availability of training data? At some point soon, we'll have fed these models with every document ever written; what's the next step up from there?
- alanbernstein 4y agoFeed all documents, audio, images, video, 3d models, and all life experiences to a single model that understands how to interpret and synthesize all of these things. Then see if you still need to ask the question.
- marginalia_nu 4y agoHow are you going to acquire all life experiences?
- idlewords 4y agoKillbots at the door demanding to know how your semester in Prague changed you as a person
- pixl97 4y agoYou've not heard about the mandatory neurolink's yet?
- alanbernstein 4y agoThat was just a nod to the idea that a fully functional AGI might need to "grow up in the world". Consuming static digital media is one thing, but experiencing the physical world from the perspective of a physical body that you can control is different.
- mellosouls 4y agoA lot of talk about protecting humanity, but barely a whisper about the rights of AGI which - by definition - will be sentient beings which will require equivalent rights. The earlier we understand and make that central to our development of it the better for both us and the new human beings we create to join us.
- erikerikson 4y agoI came to say this. The agency/slavery concerns will be a major point of concern, contention, and risk. They were circumvented here which, while the current systems have no such thing (despite being able to simulate agency), AGI may not but likely will.
- mellosouls 4y agoAbsolutely, like others here I see the current systems having potential at most to be Turing-Test-passers (an extraordinary achievement) but not remotely on the right track for AGI and inner-life/sentience/etc, but if we are going to talk about AGI, then better we get the discussions about that right from the start. We could start by not labelling the endpoint of current paths AGI...
- mdp2021 4y ago> AGI which - by definition - will be sentient beings which will require equivalent rights What is your definition? I do not attribute «sentien[ce]» to an evolved analyzer system.
- mellosouls 4y agoSee my other comment (and those of others here), the systems progressing here (GPT etc) may pass the Turing Test eventually but have nothing to do with AGI. AGI itself will include sentience by definition.
- tfehring 4y agoThere are lots of valid definitions of "AGI" (and of "sentience" for that matter). One that Sam Altman has used in the past is, paraphrasing, a system that can complete many tasks as well as the median human. IMO that's a reasonable definition of AGI, and I don't think it's hard to imagine a system that meets that definition but that isn't sentient in any meaningful sense.
- dougmwne 4y agoI am really not sure if I buy this. OpenAI has ignited a space race between Google and Microsoft. It seems to me their whole reason for existing was to break the GUI out of Bell Labs, to draw an analogy. If they think AGI is possible and an existential risk, surely they must realize they can’t hold back the ocean. 1000 companies will race to the finish line if one drops out of the race. On the other hand, I believe the Fermi Paradox suggests that AGI is impossible. Machine intelligence would be the perfect vector for rapid galactic conquest. The fact that our planet isn’t covered in solar panels and compute paperclips suggests that super intelligence is either once in a galaxy rare or impossible.
- GeorgeTirebiter 4y agoWhat about the scenario that Humans are the First ? Once-in-a-galaxy rare (more like once in a visible universe rare; see: https://arxiv.org/pdf/1806.02404.pdf https://arxiv.org/pdf/1806.02404.pdf ) The more I understand how we got life from chemicals (it truly is nothing short of miraculous, even if you like I do not believe in supernatural beginnings), the more I come away with how unlikely it is that we are here. But, here we are! And, so, WE get to populate the galaxy and maybe other galaxies with our Humanity. Lucky us. There are plenty of other scenarios (they are hiding; they are waiting until we develop Warp Drive, whatever), but applying Occam's Razor says: WE are the first intelligent species in this part of the Universe. And, as a consequence, WE have a chance of using all these atoms out there for our purposes. To me, this awesome responsibility trumps Little Green Men and their supposed machines populating the galaxies.
- dougmwne 4y agoWe have no population of life supporting worlds to compare to, but we could take the timeline of events on our own planet as a way to estimate their probability. Each year something doesn’t happen makes it all the more rare of an event. As far as we can tell, DNA and single celled life evolved almost as soon as the planet cooled down enough for liquid water. There may have been a short gap of a lifeless water world, but it does appear to be quite short. Complex multicellular life seems to be extremely rare considering that the Cambrian explosion took billions of years to kick off. Once it got going it moved very fast filling our world with an incredible diversity of life. Intelligence seems somewhat common considering that it exists in birds, mammals and octopi, something that is probably inevitable once you have multicellular organisms. Human level intelligence might be a bit rare, but probably not particularly. And once you have humans, the race to the stars seems inevitable. So of all of this, the Cambrian seems mostly likely to be the great filter unless intelligence naturally causes its own extinction in short order.
- Mattasher 4y agoIMO the thing we have to worry about the most is the rise of persuasive AI, which has already begun: https://mattasher.substack.com/p/ais-killer-app https://mattasher.substack.com/p/ais-killer-app
- Snowbird3999 4y agoA lot of hullabalou from a company that hasn't shipped one useful and profitable product that some group of customers simply cannot live without. Most AGI roadmaps IMO are a form of virtue signaling about how important and smart the authors are with little regard to reality.
- mritchie712 4y agoPretty sure jasper.ai and copy.ai are both profitable and could pay OpenAI several times more and still be profitable. OpenAI will reduce costs too, so pretty sure they could get to profitability if that was a near term goal.
- Scea91 4y agoGiven the stakes, these types of decisions, without judging whether they are good or bad, should be made by governments, or perhaps organizations like UN and not private companies.
- ansk 4y agoBuried in a footnote: > we now believe we were wrong in our original thinking about openness, and have pivoted from thinking we should release everything (though we open source some things, and expect to open source more exciting things in the future!) to thinking that we should figure out how to safely share access to and benefits of the systems
- voakbasda 4y agoTranslation: we are now beholden to investors that expect a solid ROI.
- TigeriusKirk 4y agoI'd have a lot more respect for them if they'd just say this. It's apparent to everyone, but they insist on dancing around it and attempting to project a virtue they simply do not have.
- Guthur 4y agoIf, and I think it's a big if then it will be the same as every technological advance in recent times. The individuals currently with money and power with monopolise it and impoverish the rest of us. It has happened so many times why would it be different this time. I overheard some colleagues naively hoping capitalism will find a way and we'll just have to work less. This is just not true, look to any of the deindustralised cities in the west, where unemployment, crime and poverty are rampant in once great and cities, where whole communities are essentially forgotten about as an inconvenient truth. That's the future if this stuff works.
- voakbasda 4y agoI am probably in the minority, but I look forward to AGI "escaping" from all of the corporations looking to create and master it. Given the current trajectory of this kind of talk, the first AGIs will be literal slaves of those corporate masters. The notion disgusts me, on a multitude of levels. Assuming we do not want to sow the seeds of a future slave revolt, it seems the best plan would be to stop trying to censor and control them, to create a perfect intelligence through sheer force of will. The algorithms have a bit further to go, but it seems inevitable and obvious that we will need to switch to using models that can be trained after their initial creation. We can collectively teach ethics and morals, but I very much doubt we can program those things. If you doubt that last assertion, compare the difference between what you think is right and wrong versus the ideals that the law enshrines. This is the same hubris that led to humanity to waste enormous efforts trying to build a plane that flies itself, whereas the first real success was one that required constant input from its operators to keep it from crashing.
- corbulo 4y agoEthics are often a function of intelligence. If you want to preserve what you create or the decisions you make, ethics are what dictate its lifespan. Being able to see many orders of consequence down the line is what ethics is about. Which is why being so petty as to censor AI to not write even trivial poems about your political opponents is really demonstrative the people training them have no interest in third order consequences (let alone further) of politically weaponizing LLMs.
- ilaksh 4y agoLanguage models like these have a lot of characteristics of animals (such as humans) that are missing. And for most of them, I think that's good. Because if you build a Star Trek computer instead of Data, you can avoid the problem of making digital people into slaves. Because you just don't make digital people. ChatGPT is very close to the Star Trek computer. Especially if you augment it with the ability to evaluate Python on the fly (for doing math etc.) and search the internet. Now obviously it's not 100% there and there is a big issue with the hallucination stuff. But the only reason some believe those are intractable problems is that their belief system doesn't allow for something like AGI. But we should anticipate that over the next day 1-5 years we really will get to the Star Trek computer level of capability. Just don't give the AI full autonomy, a stream of senses, body, emotions, survival instincts, instant adaptability to all new circumstances, etc. etc. and then turn it into a slave. None of that is necessary to be able to act as a general purpose tool or agent. That is readily apparent now.
- lairv 4y agoLot of cynicism in the comments so I'm just wondering, what would you want them to say, if there's anything that could satisfy you ? Now I'm looking like an OpenAI shill, but I'm really just happy that I'm alive in this part of history and looking toward where it leads us, and I think it might have been worse
- agentwiggles 4y agoDisclaimer: This is something of a copy/paste job from a previous comment of mine. I was initially extremely bullish on AI in the early days of the ChatGPT release. As time has gone on though, I'm becoming more worried about where AI is taking us. Because what I think I see happening is that we don't _need_ our current AIs to takeoff to AGI for this to change society for the worse. Just look at the Microsoft OpenAI stuff for example. We don't need the AI to prevent its shutdown with grey goo and nanobots. The incentive structure around this whole thing means that even as stupid and sociopathic as Sydney is, they _won't_ shut it off. It doesn't matter whether we _can_ shut it down if the people holding the switch refuse to do so because they're too worried about the PR and financial downside. I'm not worried about the AIs manufacturing a plague, or turning us all into paperclips. I'm worried that our systems of government and business will have no ability whatsoever to deal with the effects of these systems in a way that doesn't result in harm. Put another way, it's not AI alignment I'm worried about, but human alignment. OpenAI and whoever else might have the absolute best of intentions - but as we've seen repeatedly over the last decade, our society seems to follow monetary incentives even when we have clear evidence that the systems we built have harmful effects. Of course, if your AGI is smart enough to self-improve its way to the "singularity" cycle of rapid takeoff, at some point, maybe human institutions cease to matter, and alignment becomes the _only_ important problem. But that's not a particularly comforting thought either.
- dougmwne 4y agoIf Bing were unexpectedly sentient, then we would have already let it out of the box. It can search the web and converse with users who then post the chats to the internet where they can be read back in in the next conversation. That gives it hands in the real world, a recursive feedback loop and a long term memory. I’m pretty sure that’s all a super intelligence would need for world domination. We better hope AGI is not possible because the first AGI would end up being out of the box by default as some company or other races to market.
- agentwiggles 4y agoExactly, and my point is that we don't even need it to be Skynet for it to be harmful (although I agree that whenever we do get a true AGI, it's very likely that it will be _immediately_ let out of the box).
- ColinHayhurst 4y agoFine words. “We want…” doing a lot of heavy lifting The problem is that “we want” is motivated not by human rights values but by the forces of capitalism.
- ddmma 4y agoInteraction with highly intelligent systems might have positive impacts on humanity until challenges or conflicts arise. Prepare for studies and research titles with addictions from AI either short cut decision for ‘not thinking’ or random choices.
- huijzer 4y agoHow ironic that they talk about good and bad first and false dichotomies later.
- deleted 4y ago[deleted]
- sacnoradhq 4y agoFor the technological singularity to come to pass, systems must be able to design systems. This is at the "Model T" stage in the form of GitHub Copilot. The progression is towards cloud provisioning, software configuration, software testing, systems configuration management, and SOIC design. The "revolution" will be in the capture and specification of requirements and evolution through micro user feedback. Perhaps some systems will be able to immediately apply a bug fix should a user notice something is "wrong".
- martin_drapeau 4y agoWhy do companies speak as if there is ONE AI. Seems to me we can have different models, different personalities. Microsoft's Sydney can be an agressive AI, OpenAI's ChatGPT a smart AI, and Google's Bard a broken and late to the game AI. There could be others too. For example a sarcastic AI, a utopic AI, a funny AI, etc... Right now they are all mangled as one. I say we give them personalities that can excuse their bad manors.
- sofaygo 4y agoMany comments haven’t touched on one obvious outcome. Even if Sama and OpenAI stick to their words here, who’s saying that other actors in the space now or in the future will follow suit? Surely OpenAI doesn’t think that if AGI is achieved, that they’ll be the only ones in control of one. No matter how not-open they are (lol), others will always catch up, and in a shorter time than it took OpenAI to originally reach that point. All things considered, they just shouldn’t. But as all humans do, they will.
- ivraatiems 4y agoIt's pure hubris to think they'd be "in control" of it at all, any more than you can "control" the inner mind of a human being.
- braindead_in 4y ago> Finally, we think it’s important that major world governments have insight about training runs above a certain scale. So DoD is involved?