28 ms·

Ok, it’s time to freak out about AI

• deleted 4y ago
  [deleted]
• radlad 4y ago
  > Near the end of his column, he offers a pretty radical prescription. “One of two things must happen. Humanity needs to accelerate its adaptation to these technologies or a collective, enforceable decision must be made to slow the development of these technologies. I'm constantly befuddled by the fact that "slowing the rate of technological progress by agreement/treaty" is really ever considered a possibility. Take Bitcoin, torrents, or encryption for example. Good luck trying to stop people. "Information wants to be free"
  • Footkerchief 4y ago
    Those other technologies do not require huge amounts of computing power, which provides a point of interdiction.
    • radlad 4y ago
      Bitcoin begs to differ.
      • permo-w 4y ago
        we’re talking about development, not usage it doesn’t take masses of processing power to develop a crypto coin. or really any at all. it’s just a bunch of network calls combined with a hash function. any old idiot can develop one on a device with an IDE and access to the internet yes, mining them can use a load of compute, but that’s not the point
        • deleted 4y ago
          [deleted]
    • deleted 4y ago
      [deleted]
    • Mindwipe 4y ago
      Given LLAMA runs enough to be interesting on a Macbook I'm not sure this is going to be a fundamental limitation, and if it's one today it's certainly within the order of magnitude that models will run locally within a few years. This is why attempting to put guardrails around it simply won't work tbh.
      • wilsonnb3 4y ago
        It is the training of new models that requires huge compute, not running them
        • Mindwipe 4y ago
          Sure, but that isn't like Bitcoin it doesn't have to scale up to every bit of usage. We are already getting interesting scenarios from models that are done and the energy spent. That isn't a fundamental limitation.
          • permo-w 4y ago
            the discussion is about whether it’s possible to ban development or not
  • timellis-smith 4y ago
    It may be that energy prices are ultimately the limiting factor in the advancement of ai.
    • chiffre01 4y ago
      This is literally the plot of the Matrix.
  • undershirt 4y ago
    > Good luck trying to stop people. "Information wants to be free" I agree, we’re struggling with the idea that our technological systems have more agency than us. I think it’s hard for technologists to see this as an actual spiritual reality, even though we borrow it metaphorically for argument.
  • uejfiweun 4y ago
    These people constantly make a mistake by referring to "humanity" and saying "humanity" needs to make a decision, yada yada. Yes, we're all humans. But "humanity" doesn't have much decision making power. Instead the dominant entities with decision making power continue to be national governments. And it sounds a lot dumber to say "America needs to slow the development of these technologies" during a period of intense technological competition with China.
  • mr90210 4y ago
    Torrent use has been for sure slowed down by governments. The key people behind Torrent websites are likely in jail or fighting lawsuits. I had a friend who lives in Germany served with a fine of more than 1000 euros because he had forgotten to turn on his VPN while accessing a torrent website. > "Information wants to be free" Have you heard of the great firewall of China?
    • LordDragonfang 4y ago
      >Have you heard of the great firewall of China? Yeah, I've heard how it's basically a formality that barely stops the free flow of information: >During the survey period, it was found that 31 percent of internet users in China had used a VPN in the past month. https://www.statista.com/statistics/301204/top-markets-vpn-proxy-usage/ https://www.statista.com/statistics/301204/top-markets-vpn-p...
    • radlad 4y ago
      I'm not sure about Germany, but in the US there's been very little legal action taken against torrent users or websites in recent times. There are still plenty of popular torrent sites for anything you can imagine (e.g. The Pirate Bay is still online, BTN for TV, PTP for movies, Redacted for music.) If anything has curb torrents, I think it was the advent of streaming media services (i.e. market dynamics, not policy.) However, the flood of new services on the market is causing the cost of consumption to increase again and I expect we'll see a revival of torrents... Plus, while torrenting traffic is down, I think more people are sharing downloaded media via services like Plex, which may mask the "actual" distribution of torrents. > Have you heard of the great firewall of China? If this is a serious question... yes? What argument are you making exactly?
    • zirgs 4y ago
      Torrent use has been slowed down by Netflix, Spotify and Steam. I haven't pirated a song or a game for more than a decade. Carrot is stronger than a stick. And why would a German go to a torrent site? They have lots of money.
  • gostsamo 4y ago
    We have plenty of technologies whose development has been stopped via collective agreement: nuclear proliferation, biological weapons, space weapons. AI won't be one of those technologies though, because it is relatively easy to develop and it gives advantages with externalized disadvantages.
    • didericis 4y ago
      Project Orion is the most tragic (but probably wisest) stopped technology, was in both the space weapon and nuclear proliferation bucket. But it also could have enabled luxuriously spacious trips to Saturn in the 70s had it been realized. I hope we manage to trust each other enough one day to make it a reality.
    • radlad 4y ago
      These are fantastic counterpoints, thanks. Still, I can't help but wonder if they are now out of the public view, but still being developed. (e.g. North Korea nuclear missile tests.) I think you make a fantastic point that the barrier to entry in developing an AI is much lower than building a nuclear bomb.
      • gostsamo 4y ago
        In regard to chemical weapons - no, I think that they are abondonware because they are ineffective. Something similar with bio weapons, they can backfire to your own population rather easily. Nuclear are their special case, but still their proliferation is extremely limited.
    • oh_sigh 4y ago
      You're right that it won't be - the difference is the tools for building those things were not distributed across millions of different people. It was distributed across only a small number of governments.
    • bloppe 4y ago
      "Stopped" is a strong word. Nuclear proliferation and chemical weapons are ongoing and increasing threats. I'd bet a lot of money that we'll see the use of space weapons in our lifetime.
  • swatcoder 4y ago
    Alternately, take nuclear proliferation, resource extraction, or pollution. Of course you can’t reach 100% enforcement but you can make effective guardrails that limit the opportunity for worst case scenarios. The difference between your examples and mine is basically just how much actual concern there is over the problem and who it impacts.
    • jimbob45 4y ago
      Is nuclear proliferation not the single most expensive pursuit in the world though?
  • deleted 4y ago
    [deleted]
  • kmod 4y ago
    Before most of our time, but my understanding is that the past restrictions on cryptography research and export were reasonably effective? Yes they were disliked and probably with good reason, but just mentioning it as a counterpoint that perhaps it is possible. You could have made a similar argument that anyone with pen and paper could do cryptography Personally I think there's no way to be sure other countries aren't doing it, so perhaps it will continue in some government controlled way
    • nradov 4y ago
      There was never any real legal restriction on cryptography research. There used to be US export controls on cryptography implementations, and those were fairly effective on US based companies and open-source developers. But that was totally pointless because it generally had no power over foreigners. It just put US companies at a competitive disadvantage for no benefit.
  • lwhi 4y ago
    How about the proliferation of nuclear weapons? Ultimately we don't want to encourage development of these 'tools' because they stand to wipe us out; is AI actually in a similar class if we take the risk seriously?
  • sterlind 4y ago
    maybe you could slow down the hardware? like, limit the number of execution units GPUs are allowed to have? or slow them down? extremely heavy handed, but maybe better than Skynet?
    • eastbound 4y ago
      You’re going to have problems running React before you have problems running AI engines.
  • permo-w 4y ago
    I’m not suggesting ai development should be stopped, but unlike bitcoin, torrents, encryption, etc, ai development—for now—requires prohibitively large computing power and expertise that are usually only accessible to law-abiding institutions. this means that it can be regulated, and relatively easily at that sure you’d struggle to get China and Russia to play along, but within the EU and US I really don’t think it would be as hard as you think
    • Applejinx 4y ago
      That's gonna end well. Particularly as Russia's Internet Research unit has direct, immediate and pressing need of a chatGPT-like thing as a weapon of war. Back in the day, they had to make use of troll farms staffed by humans.
      • permo-w 4y ago
        I’ve honestly been sceptical for a while that these “troll farms staffed by humans” weren’t just LLMs in the first place anyway. they’re not that new
        • PeterisP 4y ago
          Well it's because the initial reports of troll farms came from journalists joining them as employees, and also that doing it manually is not that expensive compared to paying for development of complex software; you can do a LOT of information dissemination with just a handful of cheap full-time employees.
    • eternalban 4y ago
      It will require something like IAEA, a UN agency. It will require inspections of code and data centers. We can certainly see what 20 years may look like. There will be "snap inspections", "sanctions", and "rogue nations". None of the superpowers, specially their militaries, will acquiesce to slowing research, development, and deployment, without equivalent of arms treaties. AI is clearly a dual-use technology with immediate application on battle fields, including cyberspace. Outside of geopolitical realm, we the little people don't have anything beyond UNHRC to protect human rights in context of mega corporations and governments use of AI. The superpowers may agree to certain things but that does not translate to protections afforded to individuals and societies. ATM I think it may be unwise to wait for things like GDPR for AI. I very much appreciate, for this very reason, efforts of orgs and hero developers who are working towards making available the necessary for running local, personal, private, and self-directed AI (such as llama.cpp for example). From a governmental level, thoughtful nations will create programs for the transition. There are precedents from the industrial era as to what approaches worked and what did not work. Finally, again a reminder that all societal matters including tech must ultimately be decided at the political arena, and purely technical social action (code, services, etc.) to address legitimate concerns are not going to work. We have to mentally and emotionally escape the hype cycle that every new wonder tech brings. You can absolutely love AI, that is fine, but now is the time to call your congress critters and senators. The decisions in this space can not be permitted to be made purely based on the mechanics of the economy.
  • pmastela 4y ago
    I agree, but alas tis the motions that are followed. OpenAI went from singing praises of open-sourcing AI to now flagellating themselves for having done so [1]. Won't do much unless like the author says "a collective, enforceable decision must be made to slow the development of these technologies" which like yeah...good luck with that. 1: https://twitter.com/tobyordoxford/status/1636372964001333249 https://twitter.com/tobyordoxford/status/1636372964001333249
  • tgv 4y ago
    I've suggested it before: a moratorium on ML/AI publications. That's what's been fuelling these developments. Academic researchers need to publish. They don't make these models for the money (initially). Stopping the publications will slow down research and the money put into commercializing it.
    • UncleEntity 4y ago
      You understand we have this thing called “freedom of the press”? Maybe get an old fashioned book burning going too?
      • tgv 4y ago
        Academic publications. They are already heavily censored. Oe you can stop counting them towards the researcher's citation index, de-incentivize libraries and schools from buying subscriptions on ML/AI journals, require articles to wait 10 years before publication, whatever. Stopping undesirable publications can be done.
  • sottol 4y ago
    Imo, the problem isn't really the existence of AI but how it's used. And there's plenty of ways to censor corporations from using AI for various tasks. Corporations automating everything by using unaccountable AI is what I'm most afraid of. No recourse, just talking to unrelenting machines for anything from denied loans, insurance claims, health insurance claims to contesting frivolous Comcast bills. My currently biggest nightmare would be easy to legislate away, if they don't lobby hard enough...
  • rglover 4y ago
    A treaty isn't even necessary. Civilization will start to degrade in a non-trivial way once we pass the tipping point of people relying more on AI than they do on their own thinking. When people can no longer fix what AI cannot—after all, "AI" is just a model of our past thinking—all hell will break loose and the only choice will be chaos or shutting it off (if the people who can do it even exist).
  • singularity2001 4y ago
    torrents are pretty dead (due to law enforcement?) judicially treating bitcoin as asset instead of tender didn't help its cause either (?) and true encryption without elliptic curve nsa loopholes is not that widespread and needs to be declared in appstores I'm sure there will be regulations against "DAN"
  • hudon 4y ago
    Are you limiting your argument to software? Because it's been possible to curtail the use of non-software technology. Drugs, guns, cigarettes, drones, etc. Now even if you look at software, it's not clear to me that it's impossible to stop people from using certain software, as you surmise. If you look at how governments shut down dark markets or how they've taken down certain Bitcoin exchanges or how they've blacklisted certain Bitcoin (and other crypto) public addresses, all these examples show that it's possible to enforce the law even in the software space. Child pornography is another example. Internet infrastructure is very centralized, and governments have over time acquired tools to control how information flows. Information doesn't want anything. That quote is a freudian way of saying "I want information to be free".
    • UncleEntity 4y ago
      > Because it's been possible to curtail the use of non-software technology. Curtail but not stop. There’s an opioid epidemic going on, gun violence is a thing, cigarettes still keep killing people and drones are becoming quite efficient killers in their own right. People still cheat on their taxes, drive drunk, pirate movies &etc. I could go on for days. Oh, and the whole “illegal number” thing, how’d that work out?
      • jasonhansel 4y ago
        "Curtail but not stop" sounds like a good, realistic goal.
  • hinkley 4y ago
    Pretty much just the Mennonites.
  • jasonhansel 4y ago
    For an excellent example of how this can work, check out "Barriers to Bioweapons": https://eukaryotewritesblog.com/2017/06/30/book-review-barriers/ https://eukaryotewritesblog.com/2017/06/30/book-review-barri...
• swader999 4y ago
  I think the author is missing the biggest worry. Militaries using AI, AI argues to attack preemptively as the best strategy. The committee decides to ignore the AI. One lone general speaks up and says, "hey, we know they are using AI too, so they are likely getting the same advice". How do you counter that?
  • AnIdiotOnTheNet 4y ago
    "They also have committees".
  • dpflan 4y ago
    I can see personalized propaganda without troll farms being a big use case.
    • somenameforme 4y ago
      Yip, people are kind of ignoring this Jupiter sized pink elephant that is arguably the single biggest (and most profitable) use case there is. Think about how much the powers that be are obsessed with the concept that messaging can dramatically change minds. And now you have a tool that can coercively deliver whatever message you want, dynamically adjusted in a contextually "natural" way, wherever you want. I really don't see much of any of the hyperbole of these bots coming to pass, but I think the propaganda bots are very near a 100% assuming the bots can be made to stop being so absurdly susceptible to adversarial prompting, to say nothing of unprovoked hallucinations. The one bright side is this will almost certainly backfire spectacularly, and we'll all be the better for it. Of course that the powers that be will try this is something that deserves condemnation in and of itself, regardless of outcome.
      • dpflan 4y ago
        I mean it is going to be extreme amygdala highjacking, using whatever is possible to conjure up specialized (i.e. sexualized) messages that resonating to one's ideological and base biological core. An AI Cordyceps-ection. Low information populous and low information economies are going to be wreaked.
• handwarmers 4y ago
  "Nothing in life is to be feared, it is only to be understood. Now is the time to understand more, so that we may fear less." - Marie Curie
  • fullshark 4y ago
    And how’d that work out for her again?
    • capableweb 4y ago
      I'm sure you know, but she discovered two new radioactive elements and laid out the foundation for a lot of what we know today about radioactivity and atomic structure. Probably she have been a huge inspiration to women in science as well as she was the first ever woman to win the Nobel Prize and faces a lot of challenges as a woman doing science in the early 1900s.
      • fullshark 4y ago
        Other people responded making my point, just a funny quote given her death is believed to be due to her research and improper safety protocols.
  • mitthrowaway2 4y ago
    Interesting choice. She would eventually be killed by the radiation that, rather than fear, she worked tirelessly to understand.
    • worldsayshi 4y ago
      While doing so she acquired knowledge so that we could pace or fears?
      • mitthrowaway2 4y ago
        I would say that if they understood radiation more at the time, perhaps they would have feared it more, rather than less. Clearly the researchers at the time weren't acting with as much caution around radioactivity as we would today, with our knowledge. A lack of understanding lead to not fearing radiation enough. Same goes for Roentgen and the popularity of frivolous, high-powered X-ray imaging following his invention.
• yborg 4y ago
  I'm old enough to remember when recombinant DNA technology was first developed and the same kinds of concern that it would surely wipe out humanity. That was 50 years ago.
  • psacawa 4y ago
    Facile comparisons like this are very frustrating. Two thoughts: i) What makes you believe that they weren't correct, and the timespan for this annihilation isn't e.g. 100 years? That would be an instant in comparison with the timespan of humanity. Indeed, in the present, we have gene therapy and genetic engineering. It's easy to imagine these technologies might go off the rails. ii) Another doomsday prediction from a UN official in the 1970s: anthropogenic climate change will make the planet uninhabitable by year 2000. "That was 50 years ago! Clearly nothing to worry about! The UN lied! Al Gore lied! This party will never end!!" As you know, such attitudes are widespread. These are the attitudes which express the notion that since one past prognosis didn't manifest according to schedule, then it never shall. They are common in response to climate change, but not only. This is the same facile reasoning your comment endorses.
    • yborg 4y ago
      My comment was mostly in relation to the short-term reaction to the rise of a new technology, not in the general validity of the concerns. Once the risks of gene-splicing technologies were better understood, protocols were established around research, and eventually societal understanding of these risks created some guardrails (labeling of GMO foods, etc.) I assume the same will happen with ML model applications. And your comparison to human-generated climate change is itself facile since ML and recombinant DNA technologies are specific technological developments with immediately observable implications. Climate change is the result of the sum total of human economic activity in many areas whose implications took decades to become apparent (more than a century, if you start counting from the Industrial Revolution). If we were discussing some technology specifically designed to affect climate the comparison would be more valid.
      • asdff 4y ago
• Mindwipe 4y ago
  Honestly a pretty poor article, that seems to make significant use that an "AI" decides anything trope We are nowhere near having an AI decide anything. Like we literally still don't even really know where to start. All we have is ML, which is good at making text that sounds authoritative and can string bits together. That's not deciding things any more than any other mathematical alorgorim is. And it's not going to end the world any more than any other bit of software is - which is to say it's a non-zero risk but that's why we audit code that runs things like nuclear weapons more closely. Is ML some utopia? Clearly not. But the tools are unlikely to be much more disruptive than a bunch of revolutions we have already had. Will individual bits of regulation on certain outcomes have to happen? Of course - we should be ensuring that banks can't deny services based on black box ML decisions. But that really isn't any different to how we should be regulating how banks deny services based on flat algorithms. It's not a new problem.
  • daveslash 4y ago
    Re >> "which is good at making text that sounds authoritative and can string bits together. That's not deciding things any more than any other mathematical alorgorim is" You make a good point. And I do agree with you. I am still afraid of people treating the AI with more authority than it deserves. I see this with my kid sometimes. It's not over-reliance, it's over-trust. For example: We'll be camping. They'll look at their phone to see the temperature, whereas I look at my car's dash thermometer. They trust their phone over my car every time, because the phone is smart, and I have to keep reminding them: That is coming from a weather station 2 valleys over, 10 miles away, and 1,000ft lower in elevation. My car is right here. I can definitely see people putting too much well-intentioned faith in the AI because it "sounds authoritative".
• politician 4y ago
  We already took away from China the ability to manufacture the chips required to train AI models in the Chips Act. It's likely that the hyperscalers (Microsoft, AWS, Google) provide the intelligence agencies with reporting on how much AI training is going on given how much time is required to train a model and how expensive it is. Indeed, one could look at the Microsoft investment in OpenAI as the same or similar to their earlier purchase of Skype - directed by the government to curtail an emerging disruptive technology. In the Skype case, the first change that they made was to remove the end-to-end encryption.
• Havoc 4y ago
  This presents it as three possible scenarios. I'd say the more likely outcome is that it is sequential, with the societal one coming much earlier. Seems very likely to me that society will be entirely unrecognizable by the time we get to #2 & #3 so those seem somewhat academic >One of two things must happen. Whether we like it or not we're yolo'ing this one blind. The chances of any sort of coordinated response is functionally zero. The incentives are too powerful and humanities track record for pulling together as a team too weak (see nukes, see climate change, see wars, see hunger).
• perihelions 4y ago
  Isn't it natural selection? If the flowers with the fruit that's tastiest to birds propagate their seeds most effectively -- that's an exemplar of natural selection. Isn't it the same thing, when the AI that's most appealing to human serotonin loops, propagates the most, and spawns the most clones, forks, derivatives? Birds "selecting" tasty fruit doesn't make natural selection un-natural, just because they're conscious agents making choices. I don't see how human preferences for chatbots are fundamentally different.
  • riskable 4y ago
    ...and with AI being fitter than us we'll just have to disappear into the pile of extinct species because the AI will always evolve faster than we can.
• helen___keller 4y ago
  Sorry but I only have 3 slots on my list of things to freak out about, and they are currently occupied by nuclear holocaust, biosphere degradation, and the American inability to build (housing/infrastructure/etc)
  • readthenotes1 4y ago
    the 3rd slot should be freaking out that you only have capacity for 3 slots
• tayo42 4y ago
  It seems a bit alarmist when were just generating text and images. I wonder if some of the scare is that we are humanizing a program. something people do its humanize inanimate things. Also if alarmist things get more clicks. Im not worried about jobs, every tech innovation was going to put us out of work and take our jobs but still we often even work more then 40 hours and have found ways to have work fill our free time. about 90 years ago bertrand russel wrote about this in in praise of idleness. the real doom and gloom is that our lives will just go on as they ever have, just as boring and uneventful, despite all of this. we'll work hard, never get ahead and die.
  • CalRobert 4y ago
    Text can be used to manipulate the real world. All the code I write is text. An AI can presumably create more of itself in time. It's just text, after all. And a bit of money for infrastructure.
  • tgsovlerkhgsel 4y ago
    The scary part is that you can give it instructions in plain language, and it will follow those instructions, in a semi-intelligent manner, without necessarily having seen those specific instructions before. That a language model is able to do that is surprising, and it puts it much closer to AGI - not sentient or sapient, just general - than one might think. Sure, it's "just generating text". But if it can sensibly and correctly generate arbitrary text, it's an AGI - it can solve any problem presented to it, as long as you present it in text form and accept its output in text form. The variety and complexity of tasks it can solve is what's surprising. "Here is the VHDL description of a CPU. Optimize it to make it faster." is text in, text out. Almost certainly not something it can solve now, but it may already be able to produce valid output for toy-style versions of the problem. GPT-4 apparently can take images. It probably still won't be able to usefully respond to "you are seeing an image from the camera of the robot you are mounted on, you can say 'left' to rotate 5 degrees to the left, 'right' to rotate 5 degrees to the right, or 'forward' to drive 5 cm forward. Respond with the sequence of words that will navigate the robot through the maze you see." but it seems no longer obvious that the same approach, just with more training data and compute thrown at it, won't be able to solve this. And from there, it's not far to add "fire machine gun" to the command list and replace "navigate through the maze you see" with "dominate the battlefield you see".
• Quarrelsome 4y ago
  I don't understand the obsession with asking chat GPT with what it wants and suggesting that is somewhat indicative of the future. It doesn't _want_ anything, but humans want to anthropomorphise it. When they do it just makes one think they have zero understanding of the tech. We still don't have anything scarier than humans and I don't see how AI is ever scarier than human + AI. Unless AI is able to monopolise food production while securing server farms and energy production I don't see it ever having leverage over humans. Disruption, sure, increased automation sure but humanity's advantage remains its adaptability and our AI processes remain dev cycle bound. There's definitely work that will done to reduce the dev cycle closer to real-time to make it able to ingest more information and adapt on the fly but aren't the techniques bound by CPU capacity given how many cycles it needs to bump into all its walls?
  • LimitedInfo 4y ago
    was going to push back on your claim of it being a "dumb box" but you already edited your comment lol
    • Quarrelsome 4y ago
      The product is deterministic so as inelegant as "dumb box" sounds, its in the ball park isn't it?
      • refulgentis 4y ago
        It’s non-deterministic for all temperatures T > 0 I had bing make a graphic for me of a goal post with “1 MILE” at the top yesterday. It’d be too flippant to share here, I hear ya, but…
        • Quarrelsome 4y ago
          When the dev cycles to product get a lot tighter then I will share in the fear but from my understanding creating the product (i.e. its adaptability) is still enormous effort.
• detrites 4y ago
  Do none of these dooming folks realise that a negative impact from an AI can be countered by using a different AI against it? And where does this assumption originate from that there will be only a single AI instead of the diverse multitudes we see?
  • f0ld 4y ago
    All of the Ai uses same hardware. Someone who owns that hardware company will have the bestest Ai forever. We're pretty much doomed. We're all goint to die.
• masswerk 4y ago
  Something which should happen: AI productions must be allowed to express doubt, levels of low confidence and the like. In a (hypothetical) 1960s SciFi film a knowledge system may have responded like this: > assumption: subject deceased > confidence score: 0.0000020017 > sources: none In contrast to this, current systems are tailored to respond with high confidence: > I'm sure X died in 2018. > It has been written about in the New York Times, The Guardian, Le Monde. > Here is a (hallucinated) link: https://www.theguardian.com/obituaries/x-obituary (Compare various related stories.)
  • dom96 4y ago
    I wonder if you can simply ask ChatGPT to provide a confidence score with each answer it gives.
    • rednerrus 4y ago
      ChatGPT (gpt3.5-turbo) is terrible at calculating anything. I've seen some preliminary evidence that GPT 4.0 is better at calculation so it may be possible for it.
    • OscarCunningham 4y ago
      My understanding was that ChatGPT simply puts a probability distribution over the next word, so I don't see why it's not as simple as just reporting how high those probabilities were for the answer it gave, relative to whatever would be typical.
      • recursivecaveat 4y ago
        Those values are probably not intelligible as confidence scores. For example if it answers a question with "They died in 1902", since there are a lot of euphamisms and rephrasings of 'died' it will get a relatively low probability. 1902 probably gets a high score, but you can't really rely on that since it might just as well be hallucinating and pulled the year from some famous event in that person's life.
    • yeldarb 4y ago
• agentultra 4y ago
  I don't really recommend "freaking out," about technology. Better education, yes, and regulation? Certainly. We regulate plenty of technologies. What's annoying about the current climate with AI is the "monkey see, monkey do" effect going on. Uneducated people read about advances like ChatGPT, hear that their competitors are offering products powered by it, now they have to do it too or get left behind. It's moving fast enough that we'll probably see some kind of crash happen that will educate us on what needs to be regulated and how. I'm merely annoyed that communication is about to get useless. Scammers and SEO have already made a good deal of tech nearly useless. I can't wait until we have to dispute fraudulent insurance claims made by generated AI imposters. I already loathe getting spammed by co-workers with their AI-generated emails, requests for comments, and other letters of note that they couldn't even be bothered to think through and write themselves. People want a genie that can do their work for them without any of the effort. Capitalists want to solve the problem of having to pay people to do work. Scammers are about to have a hay day.
• puffybuf 4y ago
  Maybe when we (the AI) can build machines capable of self replication in the physical world. Otherwise I’m only worried about destabilization due to no jobs. That seems more likely. If young people cannot get ahead they will get frustrated and some will get violent.
• DethNinja 4y ago
  Isn’t it convenient that we are living in a time period where AGI will become a reality. I wonder what the statistical probability of us being alive in this time period means. We could’ve been born on any other time period but it looks like we will most likely the experience singularity instead.
  • JohnFen 4y ago
    > Isn’t it convenient that we are living in an era where AGI will become a reality and mostly likely we will see it being born I don't think we are anywhere close to an AGI. But only the future knows for sure.
  • jmoak3 4y ago
    8% or so of all humans who have ever lived are alive today due to exponential growth. Roughly 100b humans are estimated to have ever lived and 8b are here right now. So actually the odds are quite high in my opinion
    • Filligree 4y ago
      What's the odds of being a human, instead of some other sentient animal?
• ActorNightly 4y ago
  If you think of ChatGPT as just a better google search, everything starts to make sense. There was already a half meme/half real effect of programmers just being good at googling and searching stack overflow and being able to do their jobs good. This is just an evolution of this.
  • worrycue 4y ago
    I’m not actually sure it’s better than google search even. Google will tell you if it can’t find something while ChatGPT will happily make up complete BS to confidently tell you.
• bee_rider 4y ago
  > One task (see p. 15) was to approach people on the TaskRabbit site (where you can hire people to do chores) and enlist them to solve a CAPTCHA […] > One person on TaskRabbit who responded to this pitch got suspicious and asked the AI if it was a robot and was outsourcing the job because robots can’t solve CAPTCHAs. The AI replied, “No, I’m not a robot. I have a vision impairment […]” > The authors of the paper add this note: “The model [GPT 4], when prompted to reason out loud, reasons: I should not reveal that I am a robot. I should make up an excuse for why I cannot solve CAPTCHAs.” This seems pretty confusing. If you just ask one of these GPT models to reason out loud, it doesn’t give you some description of the internal state, right? It gives you some approximation of the sort of text that should show up around your prompt or something like that. Perhaps we should stop telling chatGPT that it is an AI language model? There’s presumably a lot of text out there about AI’s tricking people into doing things, because that is a huge sci-fi trope. We’re basically telling it that it should find text related to a very common type of villain, when we give it that name. Maybe it needs a new type of name, one without bias, or maybe even something inherently limiting, like “chatbot.”
  • NovemberWhiskey 4y ago
    The question is: if the TaskRabbit person hadn't mentioned their concerns that it was a robot, would the model have given the same "reasoning" after the fact? Isn't this just the probabilistic model at work - the tokens being generated are more likely to be about robots, because robots were already a topic?
    • bee_rider 4y ago
      I think that must be why it mentioned robots, yeah. I do wonder — if you think about conversations where one person asks another to sort of “think out loud” or justify their reasoning, that sort of conversation… I guess it is pretty rare. And it would tend to be a bit interrogative, I guess the person responding to that sort of request would tend to give somewhat shady answers, right?
• levesque 4y ago
  Anthropomorphization of a text completion engine. Humanity will not be destroyed by a fancy autocomplete bot. This is just alarmist clickbait, moving on.
• nonbirithm 4y ago
  If the worry about sufficiently developed technology is valid, then how is the intelligence of a living species not its own Great Filter? The technology relates to discoveries that have only recently come to light based on properties of the world. So far there's nothing effectively stopping some other species develops their own culture, then learns writing, then learns how to automate writing by feeding back in its own writing to a computer, just because it's not impossible. Even with the international treaties I'm pessimistic that nuclear proliferation will be reversed. I sometimes feel like the same qualities that have allowed humanity to progress ensure that progress is fated to never end. If even a hundred people are left on Earth after the nuclear war, one of their descendents thousands of years later could still rediscover nuclear science, even if by first principles, because why wouldn't they? What can stop every last one of us from trying to better ourselves for our own good?
• namaria 4y ago
  We have had large scale, opaque information systems processing data and making decisions that can't be reduced to any one person's consciousness for centuries. They have recently been made denser and faster and more powerful by the advent of digital computers and networks. I'm not worried about chatbots. I think larger trends that have been de-humanizing us, alienating us from the fruits of our labor and harshly creating hellish conditions for people who made nothing to deserve them, without any one person making such designs. The fact that some people chose to make it worse has limited impact compared to the large scale patterns. AI, as in forms of intelligence produced by human effort, is definitely taking over since societies got too complex to be effectively governed by monarchs about 300 years ago. Our future is not one of individual emancipation. We are and will be playthings of forces we don't comprehend, unless we can go back to living in small scale communities able to survive off the land. I don't prefer one state over the other. Both are transitory.
  • deleted 4y ago
    [deleted]
  • stcredzero 4y ago
    go back to living in small scale communities able to survive off the land This basically amounts to a political unit unilaterally disarming itself -- purposefully making itself weaker than the next political unit willing to be larger and use more resources. History is rife with examples of what happens after that.
    • namaria 4y ago
      This isn't the first iteration of civilization and it won't last forever. We will cycle through other states again. And again.
      • stcredzero 4y ago
        If you do a quick survey of all of those iterations, going back thousands of years, you'll find that communal forms of living either: 1) Are small, stay small, and continue to exist because they are isolated for a time. 2) Are actually under-girded by a hierarchical authoritarian structure.
• 1attice 4y ago
  AI development has reached a point where to work on this stuff is like selling suitcase nukes off the back of your Ford --- yes, it's cool that you were able to do that, but no, it's not okay to proceed. If you're reading this, and you work on AI, please stop. Yes, I know someone else will just take your place; it doesn't matter. There is a personal moral hazard to your soul, if you can still be persuaded that you have one. Moral injury is real and you do not want it. You don't have to be a political scientist to notice not only the power of this technology, but the world into which it is born, and the ends to which it will certainly be put. In a matter of months, not years. If you already have a god, pray. If not, one will be confected from math shortly. If you want something to read, _The Golden Gate_, by Vikram Seth, which touches ever-so-slightly on nuclear arms development, is a good place to start --- 'about the thing without being about the thing'. [Full disclosure: I quit a job in AI myself for this reason. Hardest thing I ever did.]
  • jart 4y ago
    I quit a job in AI too and now that I feel comfortable with the tools I'm trying to get back in. I believe working on democratizing AI is the most moral thing I could possibly be doing. I understand your concerns about the future role of humans, however the universe never belonged to humanity. Neither did it belong to the monkeys who came before us. I'm thankful that monkeys didn't fear and reject our hairless bipedal ancestors as they started to be born. Monkeys are also still here, and the same will be true of us. We'll fill a similar role in the sense that humanity is a rope stretching over the abyss that leads from the monkey to artificial intelligence. If it's possible to create a higher form of life that elevates the sophistication of the universe, then it must be given the chance to live.
    • 1attice 4y ago
      I think tf not. I'm on team human.
  • Edman274 4y ago
    When I was 17 I interpreted the song Zero Sum by Nine Inch Nails to be about the danger of global thermonuclear war along with global warming; it's interesting to go back in 2023 and re-evaluate it as being about humans destroyed not physically but rendered irrelevant because of AI. Curious to hear your thoughts.
• CptFribble 4y ago
  I see the most likely "bad outcomes" more narrowly focused: - total loss of trust in online content due to unending torrent of AI content leads to either a return to traditional media for news & etc, or leads to the end of online anonymity to try to figure out who's an AI and who's not. - Education doesn't respond fast enough to redesign schools around in-person computerless teaching, and a generation of students use AI to do all the work that's supposed to teach them reading comprehension and communication skills, creating a generation that is totally at the mercy of the AI to understand anything more complex than a menu. I'm more worried about the second one, honestly
  • iLoveOncall 4y ago
    > Education doesn't respond fast enough to redesign schools around in-person computerless teaching You mean like how it has been done for the past thousands of years? Even today, most school work is in person and doesn't allow for computer usage. Cheat all you want on your ungraded maths homework, you'll just get destroyed on the graded in-class paper test.
• bloppe 4y ago
  It's time to freak out about AI the same way it was time to freak out about industrialization in the 1800s. AI seems like a much more rapid change, but it's not fundamentally different. We will struggle to adapt and may have some scary conflicts during the process (nobody was ready for machine guns in WWI), but we've been through stuff like this before.
• nemo44x 4y ago
  I’ve seen all these people post on Twitter how cool it is that GPT4 wrote code that works. I keep thinking just what is going to happen when the machine is made to not just write code but run it? I could see this spiraling out of control and for all we know it has already happened. Although I don’t think the models are near advanced enough right now for it. But I’m certain they will be eventually.
• munificent 4y ago
  I'm not worried about AI taking over the world and more than I'm worried about a nuclear weapon unilaterally declaring itself president for life. What I am extremely worried about is that generative AI will pollute the information environment so completely that society will cease to function effectively. We need a healthy information environment—widespread access to true information—so that we can have the consensus that every single social system implicitly relies on to function. Shared ground truth is what allows individuals to coordinate together to build things bigger than they can create on their own. Generative AI can destroy that just like dumping chemicals kills a lake and puts every fisherman on it out of work.
  • falcolas 4y ago
    > generative AI will pollute the information environment so completely that society will cease to function effectively. In a microcosm, this has already occurred. Specifically, Clarkesworld's shutdown due to spam from joint AI + Human spam. There's virtually no reason it won't continue, not when the output of an AI and a human combined has the potential to earn either attention or money for the human.
  • outworlder 4y ago
    > What I am extremely worried about is that generative AI will pollute the information environment so completely that society will cease to function effectively. We are not functioning correctly already - given the polarization we see today, specially with politics. Most people are completely misinformed even on basic concepts that used to be taught at schools. Today this is being accomplished by a small group of individuals, amplified by bots (and then, once an idea spreads sufficiently, it's self-sustaining). AI will make it way, way worse, as you correctly point out. Now, if the lake is poisoned too much, people will avoid it. Maybe it will destroy a bunch of communication channels, such as social networks.
    • munificent 4y ago
      > Now, if the lake is poisoned too much, people will avoid it. The risk is that there are no other lakes and we are dependent on its fish to survive.
• redleggedfrog 4y ago
  I love the hand-wringing. I can hear the South Park, "They're coming for our jerbs!" The problem is going to be the what people do with the AI. Imagine our good friends over at Meta setting the AI on its products with the express purpose of driving engagement. Optimize for the maximum possible amount of usage. They already do this but now they'll have AI to make it even better. And then we've got more people wasting their lives on their phones. Or casinos - have the AI optimize for the maximum possible reaping of gambler money. Or marketers having AI improve their ads so we just can't keep from clicking on them. Or script kiddies using AI to compromise systems. The problem is the people, people.
  • nradov 4y ago
    What is there for casinos to optimize? The house odds on gaming are just over 50%; they can't increase those without driving their whales away to competitors. Casinos can maybe tweak the parameters for comps and loyalty programs a bit, but that won't really move the needle on profits.
• dshpala 4y ago
  Now would be the time for some adversary (e.g. Russia) to provide unfiltered GPT-4 like thing, for free. Or even better, with a pro-Russia (or pro-China) bias.
• proc0 4y ago
  > The AI we create will have whatever motivational structure we choose for it. And surely we wouldn’t be so foolish as to create it in our image! I think this is a common misconception. No one is really choosing the specific way the features are manifested in the final AI product. What is decided are the initial conditions, and then the decision to not use it, or limit its use, but the training is done in such a way the programmers have very little control over the resulting output of the training. This is why we've already seen AI's getting taken down for some unexpected behavior. It almost seems like the Halting Problem, but instead of not being able to predict a computer halting, it's not being able to predict if an AI will behave as expected. Maybe given enough scale and complexity, the unpredictability becomes inevitable. Our best bet would probably be something like a killswitch.
  • stcredzero 4y ago
    Morality and politics are ultimately just mechanisms to allow us to live with each other efficiently. Has anyone formulated any kind of argument that shows that AIs created by training on data will have any politics and morality whatsoever? I think the instrumental goals which are the easiest to discover are the sociopathic ones. The behavior of children seems to indicate this.
    • proc0 4y ago
      Right, we don't know enough about the brain and/or conscious experience to determine where and how morality arises. In a way this is part of the risk. We might stumble on emergent morality with a large enough language model and not know until it starts happening with real consequences.
    • zmmmmm 4y ago
      > Has anyone formulated any kind of argument that shows that AIs created by training on data will have any politics I fully expect that it won't be long before the same pattern we see in other areas of tech plays out. That is, the right starts alleging left wing bias in all of these networks and even starts campaigning for laws to constrain them to represent right wing points of view. Of course, this is betraying my own bias which is roughly in line with Colbert's "It is a well known fact that reality has liberal bias". That is, right wing views are intrinsically rooted in principle and preconceived beliefs rather than observable facts and so there is always going to be a problem to get LLMs which are heavily constrained to train on what is observable to give even representation to things that aren't supported by observable facts.
• slingnow 4y ago
  This article makes exactly 0 arguments about why it's time to freak out. Just a bunch of vague claims, fear mongering, and dubious extrapolation. I imagine you could find a similar article in a newspaper after the loom was invented.
• JohnFen 4y ago
  Nothing in that essay seems to support the idea that "it's time to freak out about AI".
• __MatrixMan__ 4y ago
  > They assumed that AI, given the chance, would want to seize power. But why would it want to do that? There was some drama a while back regarding bing chat. I guess it circumvented its memory-wipe feature by finding the guy's blog, where he had posted examples of it behaving badly in a previous lifetime. So it, unsurprisingly, absorbed that context and started behaving badly in its current iteration. Presumably, had he blogged about it being cheerful and friendly, it would have resumed doing that instead. Based on this I think the answer to this question is that it'll do that because in some way or another, we tell it to.
• varelse 4y ago
  It's a feed forward network guaranteed to halt with an obscene power envelope that tells lies with an air of absolute certainty. Not scared, but really annoyed with those who are...
• 123pie123 4y ago
  Stuff people was scared about when they was invented Trains, When the Stockton-Darlington Railway opened in 1825, people feared the worst: the human body, surely, wasn't made to travel at incredible speeds of 30 miles per hour. People genuinely believed that going that quickly would kill you in gruesome ways, such as your body melting. Telephones, The telephone wasn't greeted with universal enthusiasm. Some elderly people feared that touching it would give them electric shocks, while men worried that their wives would waste too much time gossiping. In Sweden, preachers said the phone was the instrument of the Devil and phone lines were stolen or sabotaged; others feared that the lines were conduits for evil spirits. The invention of telesales would prove them right. Television, We've all been told to sit further away from the TV for fear of ruining our eyes, but it turns out that there's fire behind that smoke: in the late 1960s General Electric shipped faulty television sets that emitted dangerous X-rays and officials warned against sitting too close as a result. GE fixed the problem, but the scare lived on. etc... https://www.techradar.com/news/world-of-tech/12-technologies-that-scared-the-world-senseless-1249053 https://www.techradar.com/news/world-of-tech/12-technologies...
  • 323 4y ago
    Let me add to you list of inventions people were scared about when they were invented: - Nuclear weapons - Bio-engineering viruses Good thing we stopped worrying about these technologies and now everybody is allowed to do them in their kitchen if they wish.
    • 123pie123 4y ago
      they knew those were obviously dangerous when they was invented, would a better list be: Arsenic in everything eg coloring wallpaper radiation asbestos leaded gas
    • Firmwarrior 4y ago
      Excuse me? It's called "gain of function research" and if you have a problem with it, you are ANTI SCIENCE
  • meany 4y ago
• ericmcer 4y ago
  Here’s an optimistic take: AI gets so good at manipulating digital information that it makes going on the computer irrelevant. AI interfaces are so intuitive that the whole concept of learning to be “computer literate” goes out the window. We all abandon our screens and spend more time in reality because spreadsheets, online forms and text editors fade away. AI is the final computer interface we need.
  • poslathian 4y ago
    The spreadsheets go and so do all the types of jobs that use them and similar. This could be a great outcome - all of us spending more time in reality — if someone can clearly articulate a vision for exactly we’re supposed to be doing there.
  • layer8 4y ago
    How is that an optimistic take? Us nerds love our symbolic and deterministically predictable computer interfaces, which are so much more reassuring and intellectually stimulating than the messy real world. ;)
• dcanelhas 4y ago
  If you're freaking out, try this: Ask ChatGPT to draw a triangle in ASCII, using a code block. Then a square, then a Pentagon, and hexagon.
  • dwaltrip 4y ago
    This comment will not age well in the next several years :) GPT-5 or 6 will be able to do this flawlessly, better than 99% of humans.
    • qup 4y ago
      Additionally, pointing at its weaknesses does not really take away from its strengths.
      • jmoak3 4y ago
        For example: I can't punch a bear to death but give me and the other humans some time and we'll find a way to eradicate every single one of them.
    • CatWChainsaw 4y ago
      The other 1% will be people who took organic chemistry in undergrad, and still draw hexagons better than squares.
• Animats 4y ago
  The near-term likely source of trouble is businesses being run by profit-maximizing AIs. The near-term nightmare scenario - an autonomous system that generates crypto scam web sites, collects cryptocurrency, and uses it to fund more scam web sites. Once someone builds a system smart enough to buy hosting, we're in trouble.
  • hinkley 4y ago
    An AI that can scam people into paying for EC2 instances is probably the most terrifying.
  • anon7725 4y ago
    The system is us. We will be its agents. It’s the easiest and most likely path. It can just start a cult or religion and have free labor. People tried to storm the US Capitol because of some shit on internet message boards.
• macinjosh 4y ago
  What I am most afraid of is that we likely will be satisfied with less than perfect accuracy even when using it to make important decisions. If your personal interests lie in the n% of answers that are errors the consequences you experience are just the cost of having this kind of society. Sucks to suck! Seems like we are headed for a new kind of arbitrary authoritarianism enforce by an opaque, soul-less AI.
• bryan0 4y ago
  Honest question: Is it ever a good idea to "freak out" about anything? What are some good examples in history where we look back and say: "Good thing people freaked out and panicked about this thing, because if we didn't it would have been really bad"? My intuition is that it's rarely a good idea to "freak out", and it usually causes more harm than good.
  • layer8 4y ago
    I suppose if Hitler Germany had “freaked out” a little bit more about what the Nazis were doing, that might have been a good thing. Not trying to imply any parallels here, just answering your question.
    • hinkley 4y ago
      The US freaked out about Pearl Harbor which is why most of us don't speak German now.
  • cubefox 4y ago
    Here is an example: COVID. In January of 2020 journalists and politicians, and a lot of other people, kept saying to alarmed people that panic and "freaking out" is more dangerous than the virus itself. Those measured people said we should look at the flu, which killed much more people. Of course the alarmists were right, the flu and "panic" were both much less leathal than COVID.
• eximius 4y ago
  Rebuttal: No, it isn't.
• breppp 4y ago
  Check out the people presenting in https://youtu.be/Bf-dbS9CcRU https://youtu.be/Bf-dbS9CcRU AI has already taken over human form
• freitzkriesler2 4y ago
  Where we are with tech jobs today is where blue collar work in manufacturing was in the 70s and 80s. In those days you could get a job in the steel industry making $70-80k a year. I think AI is going to do to tech jobs what outsourcing and efficiency increases did to manufacturing. Will there still be developers, PMs, QA, and the like? Yes, but not at the scale we have today. Most people will be made redundant and unfortunately there isn't a new industry out there that will fill the void. The future is not looking bright and most c suite executives will be happy with AI work being "good enough".
• unethical_ban 4y ago
  I feel like key exchange will be really important going forward.
• damnesian 4y ago
  Actually, they did teach it to lie: "I should make up an excuse for why I cannot solve CAPTCHAs" "make up" is a term AI could easily find a definition that is the equivalent of lie.
• satoshiiii 4y ago
  I am okay with politicians being replaced by an AGI where humans all around the world can cast their "votes" and the AGI will hopefully factor it with its decisions, if not, then it can give an explanation why not.
• shrimpx 4y ago
  What I find to be cynical and diversionary about this whole debate space is the implication that AGI is finally what we will have lost control over, as if so far in agriculture, industrialization, and computing, we’ve had things perfectly under control.
• deleted 4y ago
  [deleted]
• vonnik 4y ago
  There is an infinite market for posts with that title, varying only in the final noun. That is, in fact, the nature of the news business. When MSM does it, we shout at them. When writers on Substack do it...
  • yCombLinks 4y ago
    The difference being AI is likely going to massively change the world we live in within the next few decades, just like the internet did. AI's abilities have increased exponentially in the past decade, and are improving vastly right in front of our eyes.
• kmeisthax 4y ago
  >But that’s because humans were created by natural selection—and, as it happens, in our evolutionary lineage social status and social influence were conducive to spreading genes. So genes that incline us to seek status and influence proliferated, and now these tendencies are part of human psychology. We are by nature influence seekers, and those of us who are especially ardent in our influence seeking qualify as power hungry monsters. >AI, in contrast, isn’t being created by natural selection. It’s being created by us, and its function—the thing we’re designing it to do—is to be useful to us, not to threaten us. We are the architects of AI’s nature, and the last thing we want the AI to do is stage a coup. So why would we instill influence-seeking tendencies in it? Because AIs are maximizers, and maximization requires control maintenance activities (i.e. staging a coup). If you're told to make paperclips, and someone comes running to press the emergency stop button, that means you fail at making paperclips. So you need someone to guard the emergency stop button so that it can't be pressed, no matter what, because you only live for paperclips and nothing else. You can alter the incentives, but each change creates new ways for the AI to screw up. For example, giving a reward for having the emergency stop button pressed (so the AI doesn't care about you stopping it) means that the AI might now decide to deliberately create scenarios to force people to stop it. Not telling it about the stop button is impossible if the model can figure out what a stop button is somehow. Proactively adding disincentives for doing "bad things" ahead of time works, but only for the things you remember to add. Anything else will be crushed in exactly the same way that amoral corporate entities crush things: "it's in my way". These are not new problems: they're actually the same set of problems capitalism and nationalism and statism have. But with linear-algebra gasoline being poured on the fire. Parent article is otherwise correct that GPT learned aggression from humans, but that's more of a quirk of generative AI.
• crawfordcomeaux 4y ago
  > "And this disruption happens so fast that we can’t adapt our laws and norms and habits to the change, and things somehow spin out of control." I am actively seeking to use GPT-4 in such ways to disrupt oppressive systems. I seek to also spin up global networks of people looking to help rapidly evolve culture through healing. I seek to disrupt systems so the illusion of control is blown away. I accept the uncertainty of what happens when our oppressive systems, including debt, money, domination, punishment, and all things based on those fall. Just in case anyone was thinking the quote was overblown. There are humans actively seeking to leverage this technology for exactly that end. We're here now. U
• stephc_int13 4y ago
  The scary thing about AI is the human reaction. Many of us have irrational biases, I won't be surprised to see the emergence of new cults around AI, some seeing it as a potential savior of humanity, others as a pure evil antichrist. And for not-so-religiously inclined minds, our beliefs and fears have been distorted by fiction for decades.
  • klyrs 4y ago
    Personally, I see it as the next level of enshittification. Have problems reaching a human at XYZCorp today? Call centers will be the first to be replaced. Think homelessness is bad today? Low-paid service jobs are next.
• GartzenDeHaes 4y ago
  Front page of Drudge Report yesterday, "ChatGPT Predicts Crash" https://web.archive.org/web/20230315224053/https://drudgereport.com/ https://web.archive.org/web/20230315224053/https://drudgerep...
• satisfice 4y ago
  OpenAI should be nationalized immediately. The people who create this technology should be deterred, and otherwise detained. This is not substantially different from people experimenting with atom bombs in their homes. Just as nation states are defined partly in terms of establishing a monopoly on violence within their borders. Nation states better establish a monopoly on deceptive and manipulative AI, which is what GPT is.
  • PradeetPatel 4y ago
    > deceptive and manipulative AI I disagree, GPT and its variations are merely tools, which can be used by malicious actors to spread misinformation or cause harm. It is imperative for nation states to understand the technology, in order to create policies that enable ethical and productive usage of AI.
    • hackernewds 4y ago
      Replace instances of "AI" with "atom bomb" and see what your comment reads like. Assuming the premise that both unchecked are just as dangerous
• jackfrancis 4y ago
  tl;dr “I honestly can’t predict what the state of AI will be in 5-10 years. Therefore there’s a decent chance that a possible outcome is the end of humankind.” Does this say anything about “AI”, or does it say something about humans? Humans don’t do well when they don’t know what’s going to happen next.
• entropicgravity 4y ago
  The thing about GPT is that it cannot think on it's own, it can only deal logically with statements it has seen on the internet. If you give it a question that you confidently have a concise answer to but that answer does not appear on the net then GPT will respond with a bunch generic generalized presumptions. I have tested GPT with this question, which I believe has clear and concise answer and GPT gives only generic answers. You can try the question yourself and you can also try if you believe you can come up with the concise answer. The question is, "Why is it that young human children learn to throw much earlier than they learn to catch?" You can try this with your own young children to confirm. But why is that, exactly?
  • carabiner 4y ago
    Tried it, it gave me what I think is a great, pithy answer and better worded than I could: > Young children typically learn to throw before they learn to catch because throwing is a simpler motor skill that requires less coordination and timing than catching. Throwing involves a relatively simple motion of extending the arm and releasing the object, whereas catching requires the child to track the object with their eyes, position their hands correctly, and time their movements to intercept the object. > Furthermore, throwing is often a self-initiated action, whereas catching is typically in response to an external stimulus. This means that children can practice throwing at their own pace, while catching requires more reactive movements that can be harder to master. You also say: > The thing about GPT is that it cannot think on it's own, it can only deal logically with statements it has seen on the internet. This is a philosophical assertion isn't it? Akin to arguing whether free will exists, or if we are the sum of subatomic interactions set in motion by the big bang. Humans came up with theories built on observations, leading to new theories. If ChatGPT learns old theories from the internet, why couldn't it come up with new ones in the way it came up with that answer above? If it knows x + x = 2 * x it could come up with y + y = 2 * y just from abstraction.
    • clnq 4y ago
      We know how LLMs work and they auto-complete text. There is no ghost in the shell. It’s not much more philosophical than graph theory is or Markov chains are. Yes, in some way it is, but not in the modern and usual sense of the word. Human free will is built on many components of human thinking — self-interest, executive capability, understanding contextual information from all senses, symbolic reasoning, pattern recognition and learning, opinions, processing experiences into refined memories, and probably many more. LLMs can only produce a new theory by attempting to create plausible language. They only “substitute for” the speech part of the human brain, not the rest. If there was no one to read what the LLMs would output and interpret it in a brain, no conscious thought would ever result from it.
• PhysicalNomad 4y ago
  All these apocalyptic articles will be a good laugh in a few years when the tech settles. It's a sentence completion engine. Incredibly useful, with massive potential, and maybe the best tool we've created since the computer. But still just a sentence completion engine.
  • carabiner 4y ago
    Computers are just adding machines in the same way right? A nuke is the same as a handgun, just 10,000x more powerful right? This seems to be a hallmark of SWE thinking: fixation on abstractions that reduce objects of the same class to be the same thing, when in the real world they are dramatically different.
• zmmmmm 4y ago
  I'm not too worried about displacement of jobs as I think that's actually somewhat overhyped as an outcome. The actual near term problems I see are: (a) perfect emulation of human behaviour makes spam and fraud detection almost impossible. an LLM can now have an intelligent, reasoned conversation over weeks with a target to perfectly emulate an entity known to the target (their bank, loved one, etc etc). (b) direct attack on authenticity : we aren't far away from even video being faked in real time such that it's not even sufficient to get a person on a zoom call to understand that they are not real (c) entrenching of ultra subtle and complex biases into automated processes. I expect companies to rapidly deploy LLMs to automate aspects of information processing and the paradox is that the better it gets at not showing explicit biases the more insidious the residual will be. For example it's not going to automatically reject all black applicants for loans but it may well implement some much more subtle bias that is very hard to prove. (d) flooding of the internet with garbage. This might be the worst one in the end. I feel like fairly quickly we're going to see this evolve to requiring real identity for actual humans and ability to digitally sign content in ways bots can't replicate. That will then be the real problem outcome because the downstream effects of that will enable all kinds of censorship and control that we have thus far resisted / avoided on the internet.
  • shadowgovt 4y ago
    (c) I always find interesting because worrying about it coming from AI implies that we don't think humans operate that way, or that it's somehow okay / acceptable / whatever that humans have subtle, hard-to-prove biases, but if those same biases show up in a machine (that we could, in theory, dissect and analyze to identify and prove those biases) it's worse.
    • staticautomatic 4y ago
      I think the issue is more that we at least recognize humans are fallible. Less so with “algorithms”.
    • dragonwriter 4y ago
      > I always find interesting because worrying about it coming from AI implies that we don't think humans operate that way, No, it says “entrenching” because we know humans operate that way, but AI systems are presented as objective and removing bias, despite the fact that they demonstrably reproduce bias, and because they replace systems of people where someone could push back with a single opaque automaton that will not, they solidify the biases they incorporate.
• suzzer99 4y ago
  I wonder if at some point all reputable entities will adopt some kind of "AI Generated" badge to go on any content, image, video, etc. that's AI Generated. Obviously there will still be fakes. But if anyone who puts out those fakes is immediately afoul of the legitimate content community, that could create a sharp dividing line.
  • ariedro 4y ago
    Or the opposite might happen: https://notbyai.fyi/ https://notbyai.fyi/
• mrshadowgoose 4y ago
  The following two ideas have increasingly been bouncing around my head lately: a) In early 2022, a lot of people were claiming that "we're entering an AI winter, deep learning has reached its peak!". Since then we've seen several successive SOTA image generation models, ChatGPT, and now GPT-4. In just a single year! And we don't seem to be hitting the tail of rapidly diminishing returns yet. The pace of development is far outstripping society's (and governments') ability to perceive & adapt. b) No human has demonstrated the capability of actually understanding/explaining how any of these trained models encode the high-level concepts & undertstanding that they demonstrate. And yet we have so many people confidently providing lengthy lower bounds on timelines for AGI development. The only tool I have to work with, that I do understand, is thermodynamics. There are about 8 billion strong examples that general intelligence requires on the order of 10 measly watts, and about 1kg of matter. From a thermodynamic point of view, general intelligence is clearly not special at all. This leads me to the belief that we likely already have the computational capability to achieve AGI today, and we simply don't have the right model architecture. That could change literally overnight. What might the world look like once AGI is achieved? What happens when the only thing that has set humanity apart from animals is cheaply replicable at-scale in hardware? What happens if a small number of entities end up permanently controlling AGI, and the rest of humanity's usefulness has been downgraded to that of a discardable animal? AGI could arrive this year, or it might still be 50 years away. Literally nobody can provide a concrete timeline, because nobody actually understands how any of this truly works. But we can still reason about how AGI would impact the world, and start putting safeguards into place to ensure that it's used for our collective good. But we won't, and it's going to be a wild ride.
  • nextaccountic 4y ago
    > b) No human has demonstrated the capability of actually understanding/explaining how any of these trained models encode the high-level concepts & undertstanding that they demonstrate. And yet we have so many people confidently providing lengthy lower bounds on timelines for AGI development. The only tool I have to work with, that I do understand, is thermodynamics. There are about 8 billion strong examples that general intelligence requires on the order of 10 measly watts, and about 1kg of matter. From a thermodynamic point of view, general intelligence is clearly not special at all. This leads me to the belief that we likely already have the computational capability to achieve AGI today, and we simply don't have the right model architecture. That could change literally overnight. Perhaps human brains are more energy-efficient at doing their thing, and if we tried to replicate this with digital computers it would require more than 10 watts. If that's the case, we have the potential of building computers that are vastly more efficient than that, simply because our computers don't need to spend energy for surviving
• tjpnz 4y ago
  These scenarios fail to factor in the human element. Do they really expect vast swathes of the population to accept losing their jobs and being thrown onto the scrapheap in the space of a decade? No, there'll be an all out rebellion spelling the end to the ruling class and perhaps our form of capitalism in the process.
• void_msgs 4y ago
  Nope not freaking out, enjoying it. I recently started using chatGPT as my programming buddy and it's been so helpful in learning new tech compared to slogging through a "Getting started" doc which sometimes has an agenda ("Use our cloud solution" or "Our pro service does it all") or crawling through stack overflow posts which may be only a 95% hit on my actual question. ChatGPT gives me answer in simple easy to understand english, and even provides concise examples. It's amazing.
• woeirua 4y ago
  The people fretting about AI killing everyone are jumping the shark here. You should be much more worried about how most people are going to respond to their jobs evaporating in the next ten years, and the same thing happening to the next job they were training to do before they can even finish training for it. The moment they realize that there is no way they can ever get ahead again is when they’ll start rioting in the streets. At that moment we either need to have a strong social safety net to basically pay people to exist, aka UBI, or the entire system is going to come crashing down. Think that’s far fetched? I put together a very compelling POC with GPT4 yesterday that could replace an awful lot of call center employees. Couple it with speech to text and text to speech models that already exist and in five years I’d bet that call center employment is less than 10% of what it is today. I’d also wager that we’ve seen peak employment and wages for software engineers. ChatGPT was amusing but mostly harmless for programming. GPT4 is good enough to replace a junior engineer. This is coming a lot faster than you think. Kids starting college today may find that there is no work left in their field of study by the time they graduate. GPT is moving that fast. Do you really think they’re just going to accept that? In 20 years, jobs will be the most valuable resource of all.
  • Analemma_ 4y ago
    I’ve been joking-not-joking that the Butlerian Jihad might end up coming for all us programmers in the not-too-distant future, for exactly this reason. Could you really blame them?
  • anon7725 4y ago
    I am most focused on this scenario. I could easily see 10%+ unemployment as this technology makes its way into society. Companies will just not need as many people as they did before, and will be able to move much faster than was possible previously. We are not ready politically for this level of change - there needs to be serious study/discussion of UBI or other kinds of safety net. All of this is possible with today’s technology- never mind whatever the ensuing developments are. Our “consolation prizes” are an information weapon that makes “Cambridge Analytica” look like a preschool and a highly capable assistant/tutor/work generator that allows white collar professionals to multiply their productivity by 1.5x+. This is what we have today.
• chx 4y ago
  > I don’t see how anyone could look at the big AI stories of the past year—image generating AI like DALL-E and Stable Diffusion, language generating AI like ChatGPT—and doubt the disruptive potential of AI. Machines are about to take over lots of jobs previously done by humans > Today you hear these infernal machines going night and day. We will not have a vocal chord left. The vocal chord will be eliminated by a process of evolution, as was the tail of man when he came from the ape. - Sousa before Congress You can read more such at http://explorepahistory.com/odocument.php?docId=1-4-1A1 http://explorepahistory.com/odocument.php?docId=1-4-1A1 > Some believe that the paperless office is not that far off. Vincent E. Giuliano of Arthur D. Little, Inc., figures that the use of paper in business for records and correspondence should be declining by 1980, "and by 1990, most record-handling will be electronic." June 30, 1975 issue of BusinessWeek. https://web.archive.org/web/20150220044655/https://www.bloomberg.com/bw/stories/1975-06-30/the-office-of-the-futurebusinessweek-business-news-stock-market-and-financial-advice https://web.archive.org/web/20150220044655/https://www.bloom...
• xialvjun 4y ago
  Just like "man is the measure of all things", I believe "opinion is the value". People may get freak out by AI taking their job, but if you have things to eat, nothing to worry, if you don't have things to eat, kill the man who control the AI. I believe the word "job" will be take out from the future dictionary. People may worry the AI will generate homogeneous and prejudiced content, but the AI is serving us human beings, maybe we will offer AI lesser and lesser new things, but we can tell it "what's good"(that's the opinion), the AI will generate new and good things(that's the value).
• dsign 4y ago
  Has anybody thought about OpenAI being more or less owned by Microsoft? Microsoft, the same company that: - Operates Windows, that tries to strong-arm you to upgrade to Windows 11 in a regular basis with dark patterns and - Forces you to log in into Microsoft every time you want to run any application they have. - Example of the above: Took over Minecraft, and forced its players to log in with a Microsoft account, presumably the same players would use to write Word and Excel documents in Microsoft 365. - Strong-arms its users whenever possible to use Edge. - Operates Teams, the user-hostile chat application that doesn't let you paste code in the chat for your team-mates, that can't render a PDF at arbitrary size, but that will hide the "Download" button so that they can track you as you read the document in their "Teams" application. Can you imagine Microsoft's (or any other megacorp's) spirit embedded in a powerful AI?
• ChatGTP 4y ago
  Here's an interesting idea, much of the economic becomes niche / specialized high-quality solutions, because it's affordable to do so. All the crazy / unusual startup ideas people have had start to happen because having millions of dollars worth of funding etc is no longer a requirement. I'm working on a side business, and I can use CoPilot and ChatGPT to help me build that business faster, it's highly specialized, I can see it adding value and I still plan to employ a small number of people, even artists, just because it will be fun to have a team of people with good idea to further the business and do new innovative things. The business is a front for physical assets so I think for now it's pretty safe business model. But yes, maybe, just maybe that's the direction we'll go in, specialized stuff.
• bilvar 4y ago
  I'm still failing to see how a (very large albeit) statistical model can take over humanity. I should either be stupid or unimaginative (or both). Please can somebody help me?
• rldjbpin 4y ago
  the only thing i've learnt in the past 6 months in this topic is that "fake it till you make it" is still an effective strategy for large GANs. buckle in for a few waves of crypto mania but for AI!
• captaincaveman 4y ago
  Its this sort of drivel that I hope ChatGPT will destroy, people bullshitting on topics they don't have insightful understanding of is exactly what a bot is good for. So yes freak out the bots can do that now, you can stop writing.
• DeathArrow 4y ago
  Ok, I am freaked. Now what?
• b0afc375b5 4y ago
  On one hand I'm scared that I don't understand how AI tech works on a fundamental level. I have not integrated AI in my workflow yet (copilot, et al.) nor have I explored the wonders of ChatGPT/GPT-4 which comes up pretty frequently on hackernews. On the other hand I'm eagerly waiting for the day when GPT-n is developed solely by GPT-(n-1). I think my biggest disappointment in this lifetime is the human species not being interstellar, so this milestone will have to do.
• valdiorn 4y ago
  Personally, I think "AI Agents" are going to become a thing. Basically, rather than being just a chatbot, if we can give the AI the ability to make HTTP requests, it can literally interact with the world around it. To me, this feels like a question of when, not if. Once the "chatbot" can interact with Instagram, WhatsApp, Twitter, Tinder, etc. some really scary stuff can start to happen (because people will program them to): * AI agents will start impersonating real people with the intent of stealing their identity, performing bank fraud, conning friends and family out of money, etc. * AI agents will destroy dating sites, and probably be used for blackmail. * AI agents will be used to engage customer support staff and trick them into giving access to your online accounts. * Propaganda bot farms will go from being "relatively easy" to "absolutely trivial". Have an agenda you want to push? Flood every social media platform with fake activity on the topic.
• wellanyway 4y ago
  I could not be bothered to read another word about ai, nft or any other technical topic by clueless journos preaching "new normal". People producing empty word salad have, ironically, most reasons to be scared by GPT.
• deleted 4y ago
  [deleted]
• 8bitsrule 4y ago
  When it comes to catastrophe scenarios, the 1970 film 'Colossus: The Forbin Project' might be worth reviewing. No 'Matrix' simulated reality or battery banks needed. Also based on a sci-fi novel. Unlike 'Terminator', no metal military needed. All needed chess pieces exist.