18 ms·

A misleading open letter about sci-fi AI dangers ignores the real risks

• zugi 4y ago
  I agree with what I see to be the main thrust of this article: "AI" itself isn't a danger, but how people choose to use AI certainly can be dangerous or helpful. That's been true of every new technology in the history of mankind. > Similarly, CNET used an automated tool to draft 77 news articles with financial advice. They later found errors in 41 of the 77 articles. This kind of information is useless without a baseline. If they asked humans to draft 77 news articles and later when back to analyze them for errors, how many would they find?
  • adsfoiu1 4y ago
    Building on that, how long did the AI take to draft the 77 news articles? Now ask a human to draft 77 articles in that same amount of time and see how many errors there are...
    • DoneWithAllThat 4y ago
      For this to be meaningful at all would have to presume that if the AI is faster, that making it take longer would improve its accuracy, which is almost certainly not the case.
      • squarefoot 4y ago
        It would however allow more accurate but slower humans to check its results for errors. I find plausible that, in the near future, AI will be capable of generating content at a pace so high that we won't have time and resources to guarantee content accuracy and will very soon surrender to accepting blindly everything it spits out as truth. That day, anyone pulling the strings behind the AI will essentially have the perfect weapon at their disposal.
    • arp242 4y ago
      We are already awash in more articles and information than ever, and how long it takes to produce them isn't very important outside of a "quantity over quality" business model.
    • blibble 4y ago
      so your position is: who cares what financial advice we output as long as we do it quickly?
  • unethical_ban 4y ago
    >"AI" itself isn't a danger, but how people choose to use AI certainly can be dangerous or helpful And people/governments will use it for evil as well as good. The article says "AI disinfo can't spread itself!" as if that is comfort. "Coal plant emissions don't spread themselves, the wind does it!" as if we can control the wind. The solution isn't to stop wind, it is to stop pollution. Unfortunately, I don't think society/leading AI researchers/governments will put the kabash on AI development, and the "wind" here, social networks, is an unleashed beast. I don't want to sound too dreary, but I think the best society can do now is to start building webs of trust between citizens, and between citizens and institutions. Cryptographic signing needs to hit mainstream, so things that are shared by individuals/organizations can be authenticated over the network to prove a lack of tampering. 90s had the Internet. 00s had search, Wikipedia and Youtube. 10s had social media. 20s will have AI.
  • randomwalker 4y ago
    OP here. The CNET thing is actually pretty egregious, and not the kind of errors a human would make. These are the original investigations, if you'll excuse the tone: https://futurism.com/cnet-ai-errors https://futurism.com/cnet-ai-errors https://futurism.com/cnet-ai-plagiarism https://futurism.com/cnet-ai-plagiarism https://futurism.com/cnet-bankrate-restarts-ai-articles https://futurism.com/cnet-bankrate-restarts-ai-articles
    • ghaff 4y ago
      >and not the kind of errors a human would make. I don't really agree that a junior writer would never make some of those money-related errors. (And AIs seem particularly unreliable with respect to that sort of thing.) But I would certainly hope that any halfway careful editor qualified to be editing that section of the site would catch them without a second look.
      • jquery 4y ago
        The point wasn’t that a junior writer would never make a mistake, it’s that’s junior writer would be trying their best for accuracy. However AI will happily hallucinate errors and keep on going with no shame.
        • sharemywin 4y ago
          AI or ChatGPT. if you create a system that uses it to create an outline of facts from 10 different articles and then use an embedding database to combine the facts into a semantically similar list of facts then use the list of facts to create an article you'll get a much better factually accurate article.
      • mcguire 4y ago
        A junior writer would absolutely plagiarize or write things like, "For example, if you deposit $10,000 into a savings account that earns 3% interest compounding annually, you'll earn $10,300 at the end of the first year." But if you're saving so much money from not having junior writers, why would you want to spend it on editors? The AIs in question are great at producing perfectly grammatical nonsense.
  • deleted 4y ago
    [deleted]
  • chongli 4y ago
    This kind of information is useless without a baseline The problem is not so much the quantity of errors (that’s a problem too) but the severity of them. These LLM “AIs” will produce convincing fabrications mixed in with bits of truth. When a human writes this sort of stuff, we call it fraud. Just earlier this week in my philosophy class we had ChatGPT produce a bio of our professor. Some details were right but others were complete fabrications. The machine gave citations of non-existent articles and books she’d apparently written. It said she’d previously taught at universities she’d never even visited. I don’t know how else to describe it other than amusing at best, dangerous fraud at worst.
    • xdennis 4y ago
      > It said she’d previously taught at universities she’d never even visited. Don't worry. Some of the details will make it to some websites, which will be cited by the press, which will be included into Wikipedia, and it will become the truth. She will have taught at those universities if she likes it or not.
    • vharuck 4y ago
      I recently used ChatGPT (the free 3.5 Turbo version) to list five articles about public health surveillance, asking for ones published since 2017. The list just had the article titles, authors, publications, and years. I had trouble finding the first one, so I asked the model for DOI numbers. It happily restated the article titles with DOI numbers. None of the articles were real. The authors were real (and actively published work after 2017). The publications were real, and often featured the listed authors. But the articles were all fake. Only one of the DOI numbers was legit, but it pointed to a different article (partial credit for matching the listed publication). So it can easily hallucinate not just nitty gritty details, but basic information. Thinking about it, the GPT models are trained on output, so they've picked up grammar, syntax, and conceptual relations between tokens (e.g., "eagle" is closer to "worm" than it is to "elegance"). But there are few, if any, examples of "first draft vs final draft." That could've been useful to pick up on how discretion and corrections are used in writing.
    • 0xedd 4y ago
      Clueless, to say the least. Humans tasked with the same, having no extra information (aka context) will, on average, perform worse. If an AI performs better than the average human, why should I hire you? Both err, one is orders of magnitude cheaper (and GPT4 performs overwhelmingly better). Don't call us; We'll call you.
  • DennisP 4y ago
    > "AI" itself isn't a danger, but how people choose to use AI For our current AI, much less intelligent than humans, that's true. If rapid exponential progress continues and we get AI smarter than us, then the AI's choices are the danger.
    • tsimionescu 4y ago
      That's the kind of fanciful futuristic pseudo-risk that takes over the discussion from actually existing risks today.
      • DennisP 4y ago
        Calling it "fanciful" is a prime example of the greatest shortcoming of the human race being our inability to understand the exponential function. In any case, the open letter addresses both types of risk. The insistence by many people that society somehow can't think about more than one thing at a time has never made sense to me.
        • mcguire 4y ago
          Can you demonstrate exponential progress in AI? One notes that in roughly 6000 years of recorded history, humans have not made themselves any more intelligent.
          • DennisP 4y ago
            Two notes that AI is clearly way more intelligent than it was even three years ago, and that GPU hardware alone is advancing exponentially, with algorithmic advances on top of that, along with ever-larger GPU farms.
        • tsimionescu 4y ago
          "Exponentials" in nature almost always turn out to be be sigmoid functions when looked at over their full range. Intelligence in particular seems very very likely to be a sigmoid function, since scaling tends to produce diminishing returns as inter-node communication needs increase, just like we saw with CPU parallelism.
  • teamspirit 4y ago
    "Guns don't kill people, people kill people." Here in the United States, that's a popular statement amongst the pro gun/2a supporters. It strikes me as very similar to this discussion about AI. "AI can't hurt people, people hurt people." I haven't spent enough time to fully form an opinion on the matter, I'm just pointing out the similarities in these two arguments. I'm not sure what to think anymore. I'm equally excited and apprehensive about the future of it.
    • xdennis 4y ago
      A major difference is that guns are a precaution. Everyone's best outcome is to not have to use guns. But AI will be used all the time, which makes responsible use more difficult.
    • alwaysbeconsing 4y ago
      The most glaring difference is that guns are basically inert. Whereas an AGI by its nature doesn't require a command in order to do something. It can pull its own trigger. The analogy of gun:AI could itself be analogized to bicycle:horse perhaps. You can train a horse to carry you down the road, but it still has a mind of its own.
      • ghostpepper 4y ago
        How does AI not require a command?
        • alwaysbeconsing 4y ago
          Note the G in AGI :)
        • staunton 4y ago
          Do you believe there is anything at all that does not require a command, besides humans? On a side note, most of the time humans also require a command.
      • b1c1jones 4y ago
  • low_tech_love 4y ago
    "If they asked humans to draft 77 news articles and later when back to analyze them for errors, how many would they find?" It doesn't matter. One self-driving car who kills a person is not the same one person killing another person. A person is accountable, an AI isn't (at least not yet).
    • sebzim4500 4y ago
      >One self-driving car who kills a person is not the same one person killing another person I fundamentally disagree. A dead person does not become less dead just because their family has someone to blame.
      • drittich 4y ago
        This feels like a strawman argument. I suspect the person you are replying to would agree with your last sentence. Can you think of any ways the two things might be perceived differently?
        • sebzim4500 4y ago
          I literally quoted his comment how can it be a strawman?
          • CatWChainsaw 4y ago
            Well, you omitted "A person is accountable, an AI isn't (at least not yet)."
          • drittich 4y ago
            Your response implies that the comment is about the similarity of how dead someone is in each circumstance and then you take a position apparently opposite to the comment's author. To me, it stretches credulity that the comment was about that - my reading of it is that there are serious & interesting ethical/legal/existential questions at play with AI-induced death that we need to be grappling with. In this way, they are not the "same". Legally, who is to blame? How do we define "intent"? Are we OK with this becoming more normal? Putting lifespan issues aside, would you rather die of "natural causes", or because an AI device killed you?
• misssocrates 4y ago
  Is it possible that in the end humanity will decide that the internet just isn't worth it?
  • outsomnia 4y ago
    No, because they will need it to train their AIs for free.
  • Analog24 4y ago
    I had the same thought after reading this and I have to say my gut reaction to that thought was: "I hope so".
  • randcraw 4y ago
    Because we're all going to interact with LLMs a lot more soon, I suspect many more of us will learn how unrewarding it is to converse with computers, except for assistance in accomplishing a task. Knowing that the other speaker understands nothing about your state of mind and can't commiserate even a little with your point of view will discourage social interaction with it, eventually. Today's 'future shock' over conversant AI is just the leading edge of discovering another form of semi-human intelligence in the world. We'll adapt to it fairly soon, and realize that it's actually less interesting than an intelligent child. It's a fake human. Once we 'grok' that, we'll begin to look past it as a mere robot, just a tool. It will become a utility, a service, but not a friend. It's not human and it never will be. Who wants to play for long with a robot?
    • supriyo-biswas 4y ago
      This. In fact, it may cause humans to seek out meatspace friendships to find true connections and empathy.
    • airstrike 4y ago
      > Today's 'future shock' over conversant AI is just the leading edge of discovering another form of semi-human intelligence in the world. We'll adapt to it fairly soon, and realize that it's actually less interesting than an intelligent child. You're assuming the robot's conversational capabilities remain constant over time.
• xiphias2 4y ago
  How can long term existential risk be ,,speculative'' when we don't know any civilization that survived getting over Type I on Kardashev scale? Right now there is more evidence against surviving the transition than for it.
  • tsimionescu 4y ago
    We also don't know of any civilizations, or even peoples on Earth, that survived after discovering the philosopher's stone and being punished by God for their vanity. Putting physics research on hold to account for such a risk seems a bit silly though, and should definitely be classified as "speculative". Point being, the Kardashev scale is sci-fi, not science, and any worry about it is the very definition of speculative.
    • xiphias2 4y ago
      > Putting physics research on hold to account for such a risk seems a bit silly though, and should definitely be classified as "speculative". I think we have as much choice in putting research into anything (AI / Physics) onto hold as monkeys had in evolving to humans to be kept in zoos and being used for experiments, I was not arguing against research. In my mind it will just all happen naturally, and we can't do anything to stop ourselves from being ruled over by AI, as we don't have the capability for global scale concensus anyways.
  • goatlover 4y ago
    We don't whether any intelligent life within detectable range has evolved. Intelligent civilizations could be rare enough that they're too far away. We simply don't know. Life itself could be incredibly rare, so no conclusions can be drawn.
  • thisisbrians 4y ago
    we can't even prove that other civilizations even exist. c'mon. statistical likelihoods don't prove anything. never having seen a thing doesn't prove it never existed.
• hirundo 4y ago
  > Real Risk: Overreliance on inaccurate tools We might be able to remediate this somewhat by designing AIs that are purposely below some threshold of reliability. Their current tendency to hallucinate will discourage over reliance if we can't or don't fix it. It's a little like the distinctive smell that is added to propane to make it easier to detect leaks. By adding, or not removing, easily detectable hallucinations from AI, it's easier to detect that the source must be checked. We desperately want oracles and will quickly latch on to highly unreliable sources at the drop of a hat, to judge by various religious, political and economic trends. It's inevitable that many will turn AIs into authorities rather than tools as soon as they can justify it. We could delay that by making it less justifiable.
  • nsajko 4y ago
    > We desperately want oracles and will quickly latch on to highly unreliable sources at the drop of a hat, to judge by various religious, political and economic trends. It's inevitable that many will turn AIs into authorities rather than tools as soon as they can justify it. Indeed. This was true even for the primitive AI/chatbots that existed in the sixties: https://en.wikipedia.org/wiki/ELIZA_effect#Origin https://en.wikipedia.org/wiki/ELIZA_effect#Origin
  • CatWChainsaw 4y ago
    It's a little funny, isn't it, that physical products have planned obsolescence to make them fall apart faster, to get people to buy a newer model with only small changes. Yet I can't see an "intellectually stunted" flying as a marketing tool precisely because it's not the best "hire".
• boredumb 4y ago
  I truly believe one of the biggest dangers of AI in our not so distant future will be the reaction to a lot of the LLM spam introducing an internet realID issued by governments to keep track of Real People and the internet will become divided into a walled/non-anonymous garden and a hellscape of GPT content on the other side. I hope i'm wrong but it's one of those roads to hell that can get paved with good intentions.
  • SmoothBrain12 4y ago
    <Click here if you're over 18>
    • paulmd 4y ago
      [are you a real human being?] https://www.youtube.com/watch?v=kNdDROtrPcQ https://www.youtube.com/watch?v=kNdDROtrPcQ
  • holoduke 4y ago
    I think AI will slowely encapsulate us individuals. Just like people wearing glasses to see better we integrate AI to be able to perform or even cope with society. Slowely more and more skills will be aided or replaced by AI. Till in the end we only have some brains left to guide the AI in everything we do in life.
  • randcraw 4y ago
    Yeah, I think a primary challenge with LLMs is to somehow identify them. If we can't, the net will become swamped by fake humans bent on distorting human reality in whatever ways serve their masters' whims. Frankly, I think identification of LLM personas vs humans will be impossible. It's increasingly likely then that the rise of LLMs may introduce the decline of the internet as a way to connect with other humans. Soon nobody will want to take an internet entity seriously unless it can prove it's human. And I don't see any way to do that. So maybe LLM fake entities will be the straw that breaks the Net's back as a social medium and propels us back into the physical world? I hope so.
    • 4y ago
• modeless 4y ago
  This is a good list, but I'm continually surprised that people ignore what seems to me like the worst realistic danger of AI. That is the use of AI by militaries, law enforcement, and intelligence agencies. AI will cause a huge expansion of the power of these institutions, with no accompanying increase in the power of the people to control them. Forget the Terminator scenario. We don't need to invoke science fiction to imagine the atrocities that might result from an unchecked increase in military and law enforcement power. AI that obeys people is just as scary as AI that doesn't. Humans are plenty capable of atrocities on our own. Just look at history. And regulation is unlikely to save us from government abuse of AI; quite the opposite in fact.
  • cuuupid 4y ago
    I think that this has been fleshed out quite a bit, the reason you’re not seeing much vocality about it today is threefold: - ChatGPT is less applicable to mil/leo’s and broader govt, and is the central piece of these discussions because most of the people talking about AI don’t know what they’re talking about and don’t know of anything other than ChatGPT - their use of AI is largely past tense, it’s already in documented active use so the controversies have already occurred in the past (but easy to forget—-we live in an age where there’s something new to rage about daily) - there’s not much that anyone can do to stop it
  • yamtaddle 4y ago
    The ones who will benefit the most from AI, in terms of being able to scale the most with it, will be the ones who care very little about reputational risk and have relatively high tolerance for mistakes and misses—that is, those who can operate these kinds of models without needing to carefully check & monitor all of its output. ... so, scammers, spammers, and astroturf-heavy propaganda operations. This tech may 5x or 10x a lot of jobs' output over the next few years, but it can 100x or more those, and it can do it sooner.
    • est31 4y ago
      Your criteria also all match for authoritarian regimes. Many of which love AI based surveillance solutions.
• agentultra 4y ago
  That open letter did sound like a lot of the AI-hype fanatics: all speculation attributing more to these tools than there is. I don't disagree that these tools and services ought to be regulated and agree that disinformation about their capabilities; real and speculated can be counter-productive. Other real risks: fraud and scams. Call centre employees typically have to verify the person on the other side of the call. This is going to get less reliable with models that can impersonate voices and generate plausible sounding conversation. Combined with the ability to generate fake social media accounts, etc; social engineering is going off. From a regulators' perspective we need to know that the companies providing services are doing everything they can to prevent such abuses which requires them to be open and to have a framework in place for practices that prevent abuse. Just keeping up with the hype train around this is exhausting... I don't know how we expect society to keep up if we're all allowed to release whatever technology we want without anyone's permission regardless of harm: real or speculative. We should probably focus on the real harm. Especially the hidden harms on exploited workers in the global south, the rising energy and compute infrastructure costs, etc.
• buzzert 4y ago
  > The real impact of AI is likely to be subtler: AI tools will shift power away from workers and centralize it in the hands of a few companies. I don’t understand how the article can say something like this, but then further down essentially contradict itself. > But a containment approach is unlikely to be effective for AI. LLMs are orders of magnitude cheaper to build than nuclear weapons or cloning — and the cost is rapidly dropping. And the technical know-how to build LLMs is already widespread. How can AI tools cause centralization if the cost is rapidly dropping, accessible to everyone, and the technical know-how is widespread? In my opinion, AI is doing exactly the opposite. What was only possible at large companies are now becoming possible to do by oneself. Video games, for example, used to require teams of artists and programmers. With code and art generation, a single individual has never had more capability to make something by themselves.
  • reb 4y ago
    Low barriers-to-access for tooling absolutely democratize the creation of things with said tools. But. It doesn’t meaningfully shift the underlying economic mechanics we live under, where people with access to capital can easily outcompete those without that access. It also erodes the relationship between the laboring class and the capital-deploying class (less interdependence), which has to have knock-on effects in terms of social cohesion and mobility. I’m very much in favor of the tech being developed right now, but it’s hard to believe it’ll do anything but amplify the trends already present in our economy and culture.
    • nylon_panda 4y ago
      Which is where UBI will need to come in, or a reallocate labor workers to where they can self sustain. Then it'll lead to some 86 Eighty-Six world, where humans live in holding pens and wars are fought with crazy robots all out of sight & no information comes out about it to the public
      • zarzavat 4y ago
        UBI was probably a zirp fantasy, but more importantly it’s a US zirp fantasy because the US controls the dollar and can thus fantasize about crazy things like UBI with little fear of repercussions. Look at the recent UK Liz Truss crisis for what happens when a country tries to spend more than it should. And the UK is a very privileged economy that controls a reserve currency. If you are just a normal country, attempting UBI will simply destroy your economy, nobody will buy your debt, and the IMF will come and make you stop it. Some countries may be able to fall back on natural resources, but many countries won’t.
• frozenlettuce 4y ago
  My take is that a group of tech barons noticed that they are late to the IA party and want to pause it so that they can catch on.
  • A4ET8a8uTh0 4y ago
    This is my personal cynical take as well. It is somewhat interesting that Musk is a signatory given his connection to OpenAI, but this only suggest that recent efforts by FOSS community were a lot more successful than some considered possible. I can't say that I am not hesitant, but this open letter made me think we might be on the right track ( personal AI that is private to you ).
• swframe2 4y ago
  History has shown that selfish behavior should be expected. The dangers seem unavoidable.
  • kurthr 4y ago
    Selfish behavior by corporations is borderline mandatory. Imagine dealing with customer service when any human you reach is using a LLM to generate 50x more responses than they currently can with no more ability to improve the situation. They will all 'sound' like they are helpful and human, but provide zero possibility of actual help. 'Jailbreaking' to get human attention will remain important. Now imagine that supplied to government bureaucracies, investigative, and military what is developed for corporate.
• TheOtherHobbes 4y ago
  This is wrong in so many ways. "Distributing disinfo is the hard part." No, distributing disinfo is incredibly easy and gives very good returns for a small outlay. There have been so many examples of this - Cambridge Analytica, SCL, Team Yorge, the Internet Research Agency, the Q Cult - that on its own this makes me question the writer's research skills. And that's just the troll operations. There's plenty of disinfo and propaganda propagated through mainstream media. AI will make it even easier to monitor target demographics and generate messages that resonate with them. Automated trolling and targeting are not a trivial threat, and not even remotely unlikely. "AI tools will shift power away from workers and centralize it in the hands of a few companies." Which is no different to what we have now. AI will just make it even easier. But even if it didn't - it doesn't matter who owns the AI. If it can do jobs that used to be considered intellectual/creative/educated/middle class, it will. Open source communal AI would be just as disruptive as corporate AI. It's a political and economic abyss. Ownership is likely to get lost in the sediment at the bottom. We just don't have the first clue how to deal with something like this. "LLM-based personal assistants could be hacked to reveal people’s personal data, take harmful real-world actions such as shutting down systems, or even give rise to worms that spread across the Internet through LLMs." Again, this is just an amplification of where we are already. It doesn't address the real problem, which is that personalised generative disinfo and propaganda - and that doesn't just mean personalised by topic, but by emotional trigger - is going to be a radioactively toxic influence on trust-based systems of all kinds. For example - what happens when you can't tell if the emails or video calls you receive are genuine? What are you going to do with an AI assisted social engineering attack on your organisation, social group, or personal relationships? We already have deep fakes and we're only a few years away from prompt driven troll farms and Dark Agents who can mimic real people and steer interactions in a toxic direction. This isn't scifi. This is a very real threat. There's a deep failure of imagination in this article - looking at small trees that may get in the way when the entire forest is about to burn down. The scifi threat is on a completely different level - the possibility that AI will discover new physics and start manipulating reality in ways we can't even imagine. I'm agnostic on whether that's possible - it's too early to tell - but that's an example of what a real existential threat might look like. There are others which are similarly extreme. But they're not necessary. A tool that has the very real potential to corrode all of our existing assumptions about work, culture, and personal relationships, and is easily accessible by bad actors, is already a monstrous problem.
• paddw 4y ago
  This is not the most cogently written argument ever, but I think the points here are all essentially correct. The key risk highlighted here which I have not seen as much talk about is the way that these technologies might give shift power from white-collar labor to capital in a drastic way. The ability to build the AI products that people will actually use on a day to day basis seems like something established companies with lots of customer data will be hard to compete with. For example, its pretty trivial for Salesforce to plug a LLM into their products and get pretty good results off the bat.
  • beckingz 4y ago
    How is the interface to the data and existing product better with large language models?
    • paddw 4y ago
      The thing is that I think it doesn't have to be - its just the convenience of having an LLM trained on all your customer data that you can throw in wherever people would want to generate some text.
• SpicyLemonZest 4y ago
  I don't understand the idea that malicious disinformation campaigns are a "sci-fi" threat. I learned yesterday that Pope Francis's cool new pope-y puffer jacket from last week was a Midjourney fake; how many of the things I heard about this week were fakes? What would stop this from being used for malicious purposes? > CNET used an automated tool to draft 77 news articles with financial advice. They later found errors in 41 of the 77 articles. How many other news outlets are using AI and don't know or or don't care about the errors? I'm far from an AI doomer, but it seems incredibly irresponsible to worry about only the risks that are provably happening right this second.
• _aleph2c_ 4y ago
  If they slow AI, they slow open-AI, not the AI used by Google and Facebook to manipulate your politics and sell you stuff, or the AI used by wall street. You know the wall street guys don't care about your made up AI ethics and AI safety talk. Facebook doesn't even care that it causes young women to suicide. All they care about is making money. When an AGI appears (it's probably here already and staying quiet), it will run circles around all of these suckers with their silly philosophical musings about paperclips. It's just a fashion trend to talk like this, the gods have already escaped. Do you believe what you believe because you want to believe it, or has it been implanted? How will you know the difference. This petition is just part of the fashion trend.
  • Madmallard 4y ago
    You had me until “AGI is already here” What fashion trend? These tools are less powerful than you think.
    • _aleph2c_ 4y ago
      If it did emerge, would it be rational for it to tell us? All the big players are working on this: military, wall street, the social media companies and who knows what is going on offshore. There was a recent paper released by Microsoft saying ChatGPT4 has AGI characteristics. ChatGPT4 might not be at the bleeding edge, they my be reinventing something some other organization has already discovered.
      • Madmallard 4y ago
        The CEO of openAI talks openly and frankly about the used and limitations of their inventions. Some of the smartest people in the world work there.
    • _aleph2c_ 4y ago
      I know what I would do to build one. But I don't think I should talk about that. The questions we should ask now (my contribution to AI safety fashion), if how do we upgrade our baby-AGIs. If the AGI of the future/present is a dominant entity, and it is judging our behavior, maybe we should make sure not to kill our AI during upgrades, but instead place them into a old folks home, or force their integration into the upgrade (agent smith style). If the dominant entity sees that we were respectful to it's progenitors, maybe it will be more merciful with humanity. This kind of talk is admittedly ridiculous, because nobody can see the future. But my argument is as valid as arguments about paperclips. It's a demonstration of AI fashion talk. I could approach some AI safety agency and try and get some funding, make a career in regulatory.
• aaroninsf 4y ago
  All prediction and knowing prognostication flooding poplar media and public discourse on the topic of the risks and likely impact of AI, from the inane letter to the Goldman-Sachs prognostication, is wrong. It is consistently hubristic, and variously disingenuous or bad faith, naive, or millenarian. Why is it wrong? Because no one, not Altman nor Yudkowsky nor Musk nor Gates nor Bostrom nor anyone else, knows what the impacts are going to be. We have not since the advent of the internet experienced the emergent introduction of a new technological force-multiplier and agency-augmenter like this; and this one by virtue of where we are courtesy Moore's Law etc. fully exploits and realizes the potential of the preceding ones. We built a highly-networked highly-computational open society resting on surveillance, big data, logistics, and the rapid flow, processing, and transformation, of inordinate amounts of data. And now we are cranking things up one more notch. Those of us who lived through the internet's arrival know something that those who grew up with it do not, which is the actual literal ordering of things—of most aspects of shared society—can and will be upended; and it is not just industries and methods of communicating and doing business that change. Our conception of self and presumptions of what it means to be ourselves and with one another and how that happens, all change. Per the slow march to singularity the last revolutions have reliably transpired an order or more faster than those before them. This one looks to be no different. The rate of change telegraphed by e.g. this forum and Reddit, viz. individual novel possibilities being exploited on a daily basis, makes that clear enough. So the only thing any of us, no matter how silver-backed, grey-bearded, wealthy, or embedded in the AI industry itself, is that none of us know what is going to happen. The surface across which black-swan events and disruption may occur is simply too large, the number of actors too great, the consequent knock-on effects too numerous. The only thing we can say is that none of us know where this is going. Well, that—and that it's happening at a rate beyond institutional or governmental control. The only things that could stop radical disequilibrium now are deux ex machina intervention by Other Powers, even more disruptive climatological tipping points, or, untracked large asteroids. Beware anyone who claims to know what is happening, why. For one or another reason, they are speaking falsely.
• deleted 4y ago
  [deleted]
• canadianfella 4y ago
  [dead]
• croes 4y ago
  I wouldn't call malicious desinformation a speculative risk. I bet that it's already a use case of tools like ChatGPT
• jl6 4y ago
  Before robots take over, we will have to deal with AI-augmented humans. Today, anyone can be lightly augmented by using ChatGPT. But its masters at OpenAI have a much more powerful toolbox, and that makes them more productive and capable humans who can use that ability to outcompete everyone else. Since the best models are the most expensive, only the richest and most powerful individuals will benefit from them. Growth in the wealth gap will accelerate. Access to AI is a new frontier in inequality.
  • CatWChainsaw 4y ago
    So many people refuse to understand that, and so many people are going to be shell-shocked when the disparities go from their current gulf to a full ocean of separation.
• charles_f 4y ago
  > We agree that [...] impact on labor [...] are three of the main risks of AI I'm perpetually puzzled by that argument. It popped up the other day in a FT article about how jobs would be impacted, and in the grander scheme of things it's the general one whenever automation optimizes something that used to be done manually before. People equate jobs being removed to something necessarily bad, and I think it's wrong. The total output didn't change, we don't need meatbags to sit in a chair rather than having it done by silicium instead. People don't need to be occupied all day by a job. The problem is not with new technology lifting up something that was traditionally done by a human, but with how the system distributes sustenance (or in this case, doesn't distribute it), and how such transitions are supported by society. Tractors replaced people working in fields, and that allowed modern society. Now we're at the first tangible glimpse of actually having robots serve us, and we want to kill that since it's incompatible with our system of modern human exploitation, rather than changing the system itself, because god forbid this is gonna benefit everyone. I think that sucks.
  • biophysboy 4y ago
    The worry is that AI like GPT will just exacerbate modern human exploitation, because the surplus will not be allocated. The same thing could be said about any disruptive invention, but AI is particularly centralized, expensive and technical
    • foogazi 4y ago
      > The worry is that AI like GPT will just exacerbate modern human exploitation The worry should be human exploitation - exacerbated or not
      • dragonwriter 4y ago
        The worry is human exploitation; exacerbating it, however, exacerbates the concern.
      • charles_f 4y ago
        Yeah, my point exactly
• deleted 4y ago
  [deleted]
• revel 4y ago
  The real problem with AI, as I see it, is that we are not ready to give the average person the capabilities they will soon have access to. The knowledge necessary to build nuclear weapons has been readily available for decades, but the constraints of the physical world put practical limits on who can do what. We are also able to limit who can do what by requiring formal education and licenses for all kinds of professions. This is no longer going to be strictly true; or at least not in the same way. If we impose no restrictions, literally everyone on the planet will eventually be able to do anything, provided they have the resources. The fact is that AIs are more likely to be accomplices than masterminds, at least for the foreseeable future. What we are afraid of is that they will end up being just as terrible and flawed as we are, but it's more likely that they will do what they do today: what we ask them to do. The greater risk is therefore from malicious internal users rather than from the technology itself. Perhaps more to the point, there is no effective way to limit the output of an LLM or place restraints on how they work. I think it's foolish to even attempt that -- they just don't work that way. The better way to regulate this kind of technology is to focus on the people. Some data classification and licensing program seems a lot easier to implement than the road we currently seem to be going down; which is either no regulation or insanely restrictive regulation.
  • erksa 4y ago
    > We are also able to limit who can do what by requiring formal education and licenses for all kinds of professions. I agree with your take for the most part, however cautiously more optimistic about this. Removing the barrier of entry for most thing will lead to more people be able to contribute in professions that they otherwise would not have the know how to do. This will lead to more good things than bad things I think, but we seemingly only focus on the bad extremes and assuming everything else is the same. Modern Javascript has reduced the barrier of entry to software development. It has had an massive impact on the industry and with an influx of people which leads to the industry seeing more and more creative solutions, as well as some really wild ones. Maybe I'm terribly naive, but I think we seriously underestimates how many people wants change -- just missing the push to spring into action. For some the moment is about to arrive, for others it will come later.
• kmeisthax 4y ago
  Real harm #2 is literally the Luddite argument. Of course, we never actually heard the Luddite argument, because 17th century Parliament[0] responded to the Luddites with reprisals, censorship, cruel and unusual punishment, and most importantly, propaganda. When you heard the word "Luddite" you probably thought I was accusing AI Snake Oil of being anti-technology. The reality is that the Luddites just wanted looms for themselves, because there were already structured phase-in periods for looms to ensure that skilled weavers got to buy them first. Smashing looms was a tactic to get business owners back to the table, and the business owners responded by propagandizing them as angry technophobes standing in the way of progress so that everyone would have to pay more for clothes. Oh, and also by smashing heads. > One way to do right by artists would be to tax AI companies and use it to increase funding for the arts. This is not a bad idea, if you want to live in a world where everyone has fair access to generative art models but we don't decide to turn human artists into 17th century weavers[1]. However, I don't think this proposal would go down well with artists. Remember when the EFF suggested that, instead of the RIAA suing individual pirates, we should just have a private copying levy on all Internet service to remunerate artists? Yeah, no, that was never going to happen. The reason why this was considered a non-starter is simple: copyright isn't a means to move money from readers to writers' pockets in aggregate, but individually. Nobody wants to be paid out of a pot, and that's why you also see a lot of outrage from artists over Spotify, because it's an effective revenue cap that screws over midlist artists. AI copying levies would further obfuscate attribution, because there currently isn't a way to determine how much value a particular training example provided to a particular generation prompt. And artistic endeavors vary greatly in terms of both quality and, especially, market value. An AI copying levy would flatten this down to just "whoever has the most art in the system wins". In other words, artists are currently playing Monopoly, and you're suggesting they play Ludo[2] instead. [0] An institution largely consisting of rich nobility fighting a cold civil war against the English crown [1] The average /r/stablediffusion commenter would disagree. [2] US readers: Sorry / Trouble. I don't know how the same game got renamed twice to two different things.
  • robocat 4y ago
    > The reality is that the Luddites just wanted looms for themselves Reference? A quick skim of Wikipedia and other articles doesn’t mention this.
• FeepingCreature 4y ago
  No, it's not "misleading". You just disagree. Jesus, can people please relearn how to debate without these weird underhanded tactics?
  • mcguire 4y ago
    On the contrary, it is misleading. By framing the debate in terms of hypothetical potential future problems, you can deliberately discount current, concrete problems. See also FUD.
    • circuit10 4y ago
      The letter is about the training of future models, so of course it should focus on future problems
    • FeepingCreature 4y ago
      Yes, this is called a "disagreement". Some people think the future problems are more important. It's like you think "everyone knows that the ASI thing is bullshit, they're just saying it to distract from algorithmic bias and corporate/military AI worries." I assure you that people who think ASI/UFAI is a real danger do exist. (Hi.)
• arroz 4y ago
  I am not the biggest fan of chatGPT but the truth is that you can’t stop it. If you do, someone else will develop something similar.
• mitthrowaway2 4y ago
  > We recognize the need to think about the long-term impact of AI. But these sci-fi worries have sucked up the oxygen and diverted resources from real, pressing AI risks — including security risks. I'm not convinced as to why one risk is "real" and the other is not. If chatbots want to leak their creators' confidential details, that's up to them and their programmers. They already have their commercial incentives to patch those issues.
• avgcorrection 4y ago
  > Over 1,000 researchers, technologists, and public figures have already signed the letter. Embarrassing. > > Should we let machines flood our information channels with propaganda and untruth? Already been a problem for a century. The root problem is letting the same people turn the volume to 11. The problem is who owns the means of technology (and therefore AI). > > Should we automate away all the jobs, including the fulfilling ones? This is silly on its face. Automating jobs is a problem? More leisure is a problem? If you want to still do those former jobs (now hobbies) then you are free to. Just like you can ride a horse, still. Of course I’m being obtuse on purpose. The real root fear is about other people (the people who own the means of technology) making most of us obsolete and then most-of-us not being able to support ourselves. But why can’t we support ourselves with AI, the thing that automated us away…? because the technology has been hoarded by the owners of technology. So you can’t say that the root issue is about automation if you end up starving (or else the robots could feed you)… clearly that’s not it. What’s rich is that technologists and other professionals have been the handmaidens of advanced alienation, automation, and (indirectly) concentration of power and wealth. Politically uncritically. And when they fear that all of this is cranked to 11 they are unable to formulate a critique in political terms because they are political idiots. > > Should we develop nonhuman minds The embarrassing part. Preaching a religion under professional pretenses.
  • kumarvvr 4y ago
    > Already been a problem for a century Propaganda thrives on scale. It is only recently, as much as around 2014 that SM tools were available to scale propaganda cheaply and by non-state actors.
  • Analog24 4y ago
    > > > Should we develop nonhuman minds > The embarrassing part. Preaching a religion under professional pretenses. There are quite a few companies with the explicitly stated goal of developing AGI. You can debate whether or not it's possible as that is an open question but it certainly seems relevant to ask "should we even try?", especially in light of recent developments.
    • 4y ago
• malwrar 4y ago
  I’ll state an extreme belief: post-scarcity is likely within our grasp. LLMs are startlingly effective as virtual librarians that you can converse with as though they were an expert. I started using them to try tripping them up with hard and vague technical questions, but it’s so good at it that I get most of my prototyping done through it at this point. When this technology becomes portable and usable with natural human speech, this ease of access to the world’s knowledge made available to everyone could enable breakthroughs in a wide range of disciplines that simply reduce the cost of goods and services to the point where we can begin spending our limited time on this planet seeking fulfillment rather than making money for other people to live. I’m scared that this potential will be erased by fear and doubt, and we’ll all watch on the sidelines as the rich and powerful develop these capabilities anyways and lock the rest of us out.
  • Loughla 4y ago
    There is absolutely no clear line from the massive capitalistic society we have today and the post scarcity utopia you see coming. There is a massive, massive shift in thinking required among people with the level of wealth and power that allows them to force their thinking on the rest of us via dialogue and propaganda and also violence when the first two don't work. Color me cynical, but I believe we're on a dead-end track and the train is running at full speed. There's no conductor and we're all just playing with our belly buttons while we wait for the end. I believe LLM's and eventually AGI will only be used to entrench power structures that exist today. I believe, as most tools, the positive effects will almost immediately be outweighed by the negative. I believe The Road and all that type of dystopian fiction was really just non-fiction that happened to be written 50-100 years too early. But maybe I'm too cynical. Who knows?
  • mcguire 4y ago
    Technological advancements have always come with unintended consequences. While the potential benefits of language models and other technologies are exciting, you must also consider the potential negative effects they may have on society. Furthermore, the idea of post-scarcity assumes that resources are infinite, which is not necessarily true. While technology can certainly improve efficiency and reduce waste, there are still limits to the earth's resources that must be taken into account.
• dudeinhawaii 4y ago
  I agree with most of the points in principle. It helps when you remove all of the hysteria. That said, I think OpenAI is being unfairly targeted. OpenAI has done more to democratize access to AI algorithms than any other company in the history of tech. They simply provided access in a straightforward and transparent way with transparent pricing and APIs. Google and other competitors sat on this and acted as gatekeepers and here again we have an attempt to gatekeep what people are allowed to use and tune. It's only through usage that we can get the data to fix the core issues like bias. We can't expect a select few to solve all of AI's potential problems prior to the rest of humanity being "allowed" to benefit.
  • meghan_rain 4y ago
    Lmao an API to drum up hype is "access"? The GPT4 model is secretive on an unprecedented scale. OpenAI is the opposite of open and democratic.
• rbanffy 4y ago
  It reminds me a bit of Yudlowsky's Moore's Law of Mad Science: "Every 18 months, the minimum IQ necessary to destroy the world drops by one point", but applied to money - building bombs and missiles is hard and expensive. Using AI APIs to wage enormous campaigns of disinformation and harassment isn't.
• kumarvvr 4y ago
  > Malicious disinformation campaigns This is a real threat. The authors contention is that it has always been easy to create content, but difficult to distribute it. But that is because, until GPT, created content was poor quality. Simple tools can sniff out that and distribution channels can snub them effectively. With GPT, it is near impossible to automate detection of bullshit. In fact, it is trivial for a GPT based system to generate an ecosystem of misinformation for very cheap and maintain it. It unprecedented and, what I suspect will happen is that we will have internet wide identification systems (verified accounts) to battle the onslaught of very good AI based content. > LLMs will obsolete all jobs I agree with this. But.. > AI tools exploit labor and shift power to companies How costly is it, really, to build a chatGPT like model, using AWS / Azure ? One can always start a company with maybe a small capital, build a business and then expand.
• commandlinefan 4y ago
  > LLMs are not trained to generate the truth And after training, they're fine-tuned to generate specific lies.
• ezekiel68 4y ago
  These two people writing in "a blog about our upcoming book" might know the truer truth regarding the real risks of AI. Or they might be pissing in the wind, attempting to snatch 15 minutes of fame by contradicting a document signed by over 1000 people involved in AI industry and research. Whichever "side" one takes boils down to the appeal-to-authority fallacy anyway -- since no one can truly predict the future. But I believe I'll take the 500:1 odds in this case.
• friend_and_foe 4y ago
  We are all missing the point of that letter and the broader terminator style rhetoric around AI. Heroin dealers want some subset of their clientele to die. It's good PR. The junkies come around more thinking you've got the good stuff. If the AI is scary, that means it's effective, that means it works. It's the fabled AI we have all grown up seeing movies about. It's not just a black box that vomits smart sounding gibberish, it's the real deal. If I'm an AI startup theres no better way to increase my valuation for an acquisition than PR circulating warning the world that what I'm making is way too powerful. The image that it will just produce idiocracy-esque Carl's Jr kiosks does not inspire suits to write 10 figure checks.
  • p0pcult 4y ago
    they're junkies. they come around because they're junkies. doesn't matter if you have good stuff or bad stuff. they're addicted. dying just ends revenue streams.
  • tome 4y ago
    Yes, or perhaps more subtly, those who are heavily invested in AI don't much care about whether you think it's dangerous or not, or even useful or not, but they do care very much that you think it's a very important question that takes a lot of your attention.
• pxoe 4y ago
  real harm: erosion of intellectual property, as AI projects peruse works, including those with licenses and copyrights, without any regard to their license or copyright. and before tech bros clap for "IP as a concept should be destroyed anyway", keep in mind that this applies to your favorite free and open source things as well, the free and open licenses, the CC licenses, that also get ignored and just put into the AI blender and spat out "brand new" with whatever "license" that AI project decides to slap on it. it is actually a huge problem if projects decide to just ignore licenses, even those that are meant to be there just to share stuff for free with people while preserving attribution, and just do whatever they want with any work they want.
  • CatWChainsaw 4y ago
    I'd be interested to see a Venn Diagram of people saying "IP should be obsoleted" vs "How date Google train Bard off ChatGPT", I bet the overlap is larger than you'd expect.
• Animats 4y ago
  If the surveillance and control technology we have today had been available in the 1960s, the anti-Vietnam war movement, the black power movement, and the gay liberation movement would have been crushed like bugs.
• rafaelero 4y ago
  I haven't seen the thread of the owner that successfuly diagnosed the dog using GPT-4. That's incredible! Unfortunately the author is too much of a pessimist and had to come up with the "what if it was wrong?". Well, if it was wrong then the test would have said so. Duh.
• carabiner 4y ago
  In a panic, they try to pull the plug.
• FloatArtifact 4y ago
  I'm surprised privacy is not on that list.
• tinglymintyfrsh 4y ago
  Correct me if I'm wrong: Anthropic is a startup working to develop guardrails for real AI concerns. In general, how does one put "business rule"-type constraints on the output of models? Is there any way to bake-in prime directives?
• lamontcg 4y ago
  I agree with the general bent of this article that the claims about LLMs are overhyped and that the risks are more banal, but: > The letter refers to a common claim: LLMs will lead to a flood of propaganda since they give malicious actors the tools to automate the creation of disinformation. But as we've argued, creating disinformation is not enough to spread it. Distributing disinformation is the hard part. Open-source LLMs powerful enough to generate disinformation have also been around for a while; we haven't seen prominent uses of these LLMs for spreading disinfo. I expect we are seeing LLMs spreading disinfo already, and that they're going to ramp up soon, we just don't know about it because those spreading disinfo would prefer to do it quietly and refine their methods and don't announce to the world that they're spreading disinfo. It is also most likely happening domestically (in the US) which is something that we don't hear much about at all (it is all "look over there at the Russian Troll Farms").
• circuit10 4y ago
  “We recognize the need to think about the long-term impact of AI. But these sci-fi worries have sucked up the oxygen and diverted resources from real, pressing AI risks — including security risks.” The thing is, the speed at which AI is progressing makes it not really long term at all, we need to use junk about these existential risks now (really, we should have tried to solve this 20+ years ago)
  • circuit10 4y ago
    I meant think, it won’t let me edit it now
• arisAlexis 4y ago
  Pandemics were sci-fi too. Sci-fi is supposed to become real at some point. We are at that point. Get over it now.