11 ms·

From a layman's perspective when it comes to cutting edge AI, I can't help but be a bit turned off by some of the copy. It seems it goes out of its way to use p

• JimtheCoder 3y ago
  "I'm trying to understand why it rubs me particularly the wrong way here" Would I be correct to assume that superintelligence might have a negative effect on your earning potential in the future? When I originally had my responses to this, this is one of the reasons I came up with. But, now that I see through most of the BS, I am OK...
  • visarga 3y ago
    It might not necessarily have a bad effect. It would create new capabilities and those will be followed by new products. AI is amazing at demand induction. https://en.wikipedia.org/wiki/Induced_demand https://en.wikipedia.org/wiki/Induced_demand
    • JimtheCoder 3y ago
      Yes, but the immediate emotional response to this stuff is not rational or well informed. At least in my case, when this stuff just came out and I didn't really understand it...
• famouswaffles 3y ago
  Open AI spent at least hundreds of millions on GPT-4 compute. Assuming they aren't lying, a fifth of compute budget (billions) is an awful lot of money to put on an issue they don't think is as pertinent as they are presenting. Not that I think Super Intelligence can be aligned anyway. Point is, whether they are right or wrong, I believe they genuinely think this to be an issue.
  • rmilejczz 3y ago
    Just curious, why might we not be able to align super intelligence? I’m extremely ignorant in this space so forgive me if it’s a dumb question but I am definitely curious to learn more
    • famouswaffles 3y ago
      1. Models aren't "programmed" so much as "grown". We know how GPT is trained but we don't know what it is learning exactly to predict the next token. What do the weights do ? We don't know. This is obviously problematic because it makes interpretability not much better than for humans. How can you ascertain to control something you don't even understand ? 2. Hundreds of thousands of years on earth and we can't even align ourselves. 3. SuperIntelligence would be by definition unpredictable. If we could predict its answers to our problems, it wouldn't be necessary. You can't control what you can't predict.
  • anotherman554 3y ago
    A more cynical take would be they'll be spending the compute on more mundane engineering problems like making sure the AI doesn't say any naughty words, while calling it "Super Intelligence Alignment Research."
    • skinner_ 3y ago
      This effort is led by Ilya Sutskever. Listening to a bunch of interviews with him, and talking to a bunch of people who know him personally, I don't think he cares at all about AIs saying naughty words.
      • anotherman554 3y ago
        Openai certainly does, they are sending out emails to people using the AI for NSFW roleplay and warning them they'll be banned if they continue. They've also recently updated their API to make it harder to generate NSFW content.
  • 3y ago
• zzzeek 3y ago
  I think because it's horseshit is the main reason it's rubbing you the wrong way
  • SomewhatLikely 3y ago
    Strong words. Care to elaborate?
    • zzzeek 3y ago
      not really. let's wait ten years, then come back and see how it went.
• omeze 3y ago
  yes I also have that impression. If you consider the concrete objectives, this is a good announcement: - they want to make benchmarking easier by using AI systems - they want to automate red-teaming and safety-checking ("problematic behavior" i.e. cursing at customers) - they want to automate the understanding of model outputs ("interpretability") Notice how absolutely none of these things require "superintelligence" to exist to be useful? They're all just bog standard Good Things that you'd want for any class of automated system, i.e. a great customer service bot. The superintelligence meme is tiring but we're getting cool things out of it I guess...
  • gooseus 3y ago
    We'll get these cool things either way, no need to bundle them with the supernatural mumbo-jumbo, imo. My take is that every advancement in these highly complex and expensive fields is dependent on our ability to maintain global social, political, and economic stability. This insistence on the importance of Super-Intelligence and AGI as the path to Paradise or Hell is one of the many brain-worms going around that have this "Revelation" structure that makes pragmatic discussions very difficult, and in turn actually makes it harder to maintain social, political, and economic stability.
    • DennisP 3y ago
      There's nothing "supernatural" about thinking that an AGI could be smarter than humans, and therefore behave in ways that we dumb humans can't predict. There's more mumbo-jumbo in thinking human intelligence has some secret sauce that can't be replicated by a computer.
      • gooseus 3y ago
        Not if the "secret sauce" is actually a natural limit to what levels of intelligence can be reached with the current architectures we're exploring. It could be theoretically possible to build an AGI smarter than a human, but is it really plausible if it turns out to need a data center the size of the Hadron Collider and the energy of a small country to maintain itself? It could be that it turns out the only architecture we can find that is equal to the task (and feasibly produced) is the human brain, and instead the hard part of making super-intelligence is bootstrapping that human brain and training it to be more intel? Maybe the best way to solve the "alignment problem", and other issues of creating super-intelligence, is to solve the problem of how best to raise and educate intelligent and well-adjusted humans?
• majormajor 3y ago
  There's a weird implicit set of assumptions in this post. They're taking for granted the fact that they'll create AI systems much smarter than humans. They're taking for granted the fact that by default they wouldn't be able to control these systems. They're saying the solution will be creating a new, separate team. That feels weird, organizationally. Of all the unknowns about creating "much smarter than human" systems, safety seems like one that you might have to bake in through and through. Not spin off to the side with a separate team. There's also some minor vibes of "lol creating superintelligence is super dangerous but hey it might as well be us that does it idk look how smart we are!" Or "we're taking the risks so seriously that we're gonna do it anyway."
  • jq-r 3y ago
    Good explanation. It sounds like they wanted to do some organizational change (like every company does), and in this case create a new team. But they also wanted to get some positive PR for it hence the announcement. As a bonus, they also wanted to blow their own trumpet and brag that they are creating some sort of a superweapon (which is false). So a lot of hot air there.
  • niam 3y ago
    >They're taking for granted the fact that they'll create AI systems much smarter than humans. They're taking for granted that superintelligence is achievable within the next decade (regardless of who achieves it). >They're taking for granted the fact that by default they wouldn't be able to control these systems. That's reasonable though. You wouldn't need guardrails on anything if manufacturers built everything to spec without error, and users used everything 100% perfectly. But you can't make those presumptions in the real world. You can't just say "make a good hacksaw and people won't cut their arm off". And you can't presume the people tasked with making a mechanically desirable and marketable hacksaw are also proficient in creating a safe one. >They're saying the solution will be creating a new, separate team. The team isn't the solution. The solution may be borne of that team. >There's also some minor vibes of [...] "we're taking the risks so seriously that we're gonna do it anyway." The alternative is to throw the baby out with the bathwater. The goal here is to keep the useful bits of AGI and protect against the dangerous bits.
• gonehome 3y ago
  The extinction risk from unaligned supterintelligent AGI is real, it's just often dismissed (imo) because it's outside the window of risks that are acceptable and high status to take seriously. People often have an initial knee-jerk negative reaction to it (for not crazy reasons, lots of stuff is often overhyped), but that doesn't make it wrong. It's uncool to look like an alarmist nut, but sometimes there's no socially acceptable alarm and the risks are real: https://intelligence.org/2017/10/13/fire-alarm/ https://intelligence.org/2017/10/13/fire-alarm/ It's worth looking at the underlying arguments earnestly, you can with an initial skepticism but I was persuaded. Alignment is also been something MIRI and others have been worried about since as early as 2007 (maybe earlier?) so it's also a case of a called shot, not a recent reaction to hype/new LLM capability. Others have also changed their mind when they looked, for example: - https://twitter.com/repligate/status/1676507258954416128?s=20 https://twitter.com/repligate/status/1676507258954416128?s=2... - Longer form: https://www.lesswrong.com/posts/kAmgdEjq2eYQkB5PP/douglas-hofstadter-changes-his-mind-on-deep-learning-and-ai https://www.lesswrong.com/posts/kAmgdEjq2eYQkB5PP/douglas-ho... For a longer podcast introduction to the ideas: https://www.samharris.org/podcasts/making-sense-episodes/116-ai-racing-toward-brink https://www.samharris.org/podcasts/making-sense-episodes/116...
  • atlasunshrugged 3y ago
    This is an interesting comment because lately it feels like its very cool to be an alarmist! Lots of positive press for people warning about the dangers of AI, Altman and others being taken very seriously, VC and other funders obviously leaning into the space in part because of the related hype And in other fields, being alarmist has paid off too with little recourse for bad predictions -- how many times have we heard that there will be huge climate disasters ending humanity, the extinction of bees, mass starvation, etc. (not to diminish the dangers of climate change which is obviously very real)? I think alarmism is generally rewarded, at least in media.
    • gonehome 3y ago
      Some types of alarm yeah, if within the window of things it's statusy to be alarmed about. Most of the AI concern that's high status to believe has been the bias, misinformation, safety, stuff. Until very recently talk about e-risk was dismissed and mocked without really engaging with the underlying arguments. That may be changing now, but on net I still mostly see people mocked and dismissed for it. The set of people alarmed by AGI e-risk are also pretty different than the set alarmed about a lot of these other issues that aren't really e-risks (though still might have bad outcomes). At least EY, Bostrom, Toby Ord are not also as worried about about all these other things to nearly the same extent - the extinction risk of unaligned AGI is different in severity.
• jillesvangurp 3y ago
  It's easy to dismiss the future when it seems far away but right now, there's a rather significant risk of people ending up with egg on their face. People are talking years, not decades at this point when it comes to AGI. Never mind self driving cars. FSD when it starts working, (there is no if IMHO), will be a pretty significant but minor milestone in comparison. Most people aren't particularly good drivers. Indeed the vast majority of lethal accidents (the statistics are quite brutal for this) are caused by people driving poorly and could be close to 100% preventable with a properly engineered FSD system. Something that drives better on average than a human driver is not that ambitious of a goal, honestly. That's why you can already book self driving taxis in a small but growing number of places in the US and China (which isn't waiting for the US to figure this out) and probably soon a few other places. Scaling that up takes time. Most of the remaining issues are increasingly of a legislative nature. Safety is important of course. Stopping humans from killing each other using cars will be a major improvement over the status quo. It's one of the major causes of death in many countries. Insurers will drive the transition once they figure out they can charge people more if they still choose to drive themselves. That's not going to take 20 years. Once there is a choice, the liability law suits over human caused traffic deaths are not going to be pretty.
  • deleted 3y ago
    [deleted]
  • EamonnMR 3y ago
    > Most people aren't particularly good drivers. Indeed the vast majority of lethal accidents (the statistics are quite brutal for this) are caused by people driving poorly and could be close to 100% preventable with a properly engineered FSD system I'm gonna take issue with this. A properly engineered FSD system will refuse to proceed into a dangerous situation where a human driver will often push their luck. Would a full self driving car just... decline to drive you somewhere if the conditions were unsafe? Would this be acceptable to customers? Similar story for driving over the speed limit.
    • ElevenLathe 3y ago
      I think it's very clear that we /could/ engineer an automotive system that is much safe, even without self-driving tech, by modeling it on the aviation system: much more rigorous licensing requirements, certifications based on vehicle type, third-party traffic control, filing "drive plans", obsessive focus on reliability and safety. It would look a lot different from the current system, and there is no political will to get there, but the thought experiment shows that we /could/ prevent most car accidents.
• mhh__ 3y ago
  For open ai specifically I think they genuinely do believe in their own brand of pronoun-adjacent-hedonism style of liberalism.
• arisAlexis 3y ago
  Because people don't like face value statements they don't like so they try to find a conspiracy that fits their narrative better.
• jgalt212 3y ago
  It's Hucksterism 101.