7 ms·

Text continuation poses "the greatest of dangers"?

by Terretta 2y ago

hn_go_brrrrr 2y ago
While we're being overly reductionist: it's just moving bits around, what harm could that possibly do?
- brigadier132 2y ago
  [flagged]
- Dalewyn 2y ago
  "AI" is one of two things depending on whether you think it's actually intelligent: * If "AI" is actually intelligent, then it's no worse a threat than any living being known to man. * If "AI" is not actually intelligent, then it's no worse a threat than any other computer program. Both threat models are very thoroughly known and understood. I second parent commenter's sentiment that calling "AI" the "gravest of dangers" is a gross misrepresentation.
  - comp_throw7 2y ago
    > * If "AI" is actually intelligent, then it's no worse a threat than any living being known to man. ...does not follow? "If an atom bomb is actually a bomb, then it's no worse a threat than any other existing bomb known to man."
    - Dalewyn 2y ago
      An atomic bomb explodes. A bomb explodes. Both are very destructive. Is there a difference? I say no, and I say that as a Japanese-American who visited Hiroshima Peace Memorial numerous times and never felt anything particular from the exhibits. People die when they are killed. Likewise, if "AI" is actually intelligent then it is no worse a threat than a bumbling baby or a Vladimir Putin or a dog, among other living beings.
      - mlyle 2y ago
        > Likewise, if "AI" is actually intelligent then it is no worse a threat than a bumbling baby or a Vladimir Putin or a dog, among other living beings. I think a dog and a dictator might both be able to have negative impacts, but treating them as equal just shows a desire to troll. A dog can only bite so many people.
      - comp_throw7 2y ago
        > People die when they are killed. Yes, indeed, which is why we should avoid creating something that will kill everyone... ...so I guess I'm glad to hear there's no difference in the amount of damage that can be inflicted by a baby, and by Vladimir Putin, and superintelligent AI really can't be any worse than either.
      - BuyMyBitcoins 2y ago
        >”An atomic bomb explodes. A bomb explodes. Both are very destructive. Is there a difference? I say no” Atomic bombs and hydrogen bombs are orders of magnitude more destructive and also produce fallout. How in the world can that difference be dismissed?
        jiggawatts 2y ago
        Not the original commenter, but one interpretation is that “dead is dead”. You can’t be “more killed”, so past a point, all dangerous things are equally dangerous.
        Dalewyn 2y ago
        This. I don't see nuclear weapons as any worse than conventional weapons because ultimately people die. Is it comforting or otherwise somehow better for people to be killed by conventional weapons instead of nuclear weapons? I find such a proposition preposterous. All bombs are bad, whether they're conventional or nuclear is irrelevant. For that matter, all wars are bad and should be avoided unless no other options remain. Likewise, "AI" is just as much a threat as any other intelligent being or computer program depending on which line of thought you subscribe to. "AI" is not a "gravest of dangers".
  - Jedd 2y ago
    I don't believe asserting two parallel and comparably false equivalences meaningfully progresses the discussion around this technology and risk. If this tech exceeds human capabilities to 'think' (plan, reason, manipulate, etc) then it is obviously a bigger threat than any less capable entity ('any living being', as you refer). Because no humans will be able to out-plan/reason/manipulate it. If the tech doesn't achieve intelligence - a tricky goal, and I eschew mislabelling LLMs as AIs for that reason - that doesn't mean its risk is comparable to the worst of any existing technology. There's just no way to extrapolate from one to the other, there. It might be orders of magnitude worse than any other 'computer program' - you can't say for sure, from where we're standing now.
    - Aexian 2y ago
      > you can't say for sure, from where we're standing now. We can already see how LLMs can be substantially worse - they can cosplay human thoughts and sentiment in a way that wasn’t previously possible. So much of this debate seems focused on this conception of Skynet-style murderous AI - or at least manipulative and scheming HAL 9000 types. But an order of magnitude greater risk has already arrived just from scaling existing harms. Phishing schemes and pig butchering scams have destroyed countless lives. They’re now easier and more scalable than ever. As are fake news and disinformation campaigns. Several companies are productising AI girlfriends with predatory pricing models, capitalising the human desire for connection and intimacy in the way slot machines and sports betting monetise our desire for a better life. That’s new. It may not be an order of magnitude worse for everyone yet - but for certain vulnerable groups, that future has arrived already.
  - 0xDEAFBEAD 2y ago
    >* If "AI" is actually intelligent, then it's no worse a threat than any living being known to man. Humans are quite a threat to other species, e.g. gorillas, largely due to our intelligence advantage. If a new species arrives on Earth that's smarter than we are, there's a good chance we'll be displaced.
    - blksv 2y ago
      But AI (as we have it today and in the foreseeable future) is nowhere near the definition of a species. It is an enormous _server farm_ doing series of matrix multiplications followed by nonlinear transformations, with us, humans, supplying input data (as well as hardware, electricity, and maintenance) and assigning meaning to the outputs. And we have completely no ideas of doing "AI" any other way. I agree, that AI may be dangerous if used for destructive purposes, or if used for some critical tasks with too much trust (and the hype that "we're so dangerously near a superintelligence" makes the latter much more likely, in my opinion). But that Humanity will be displaced by autonomous server farms? No way. (As to the original comment, I think that a bunch of nearly(?) demented elders holding nuclear buttons is a much worse (and immediate!) threat than a server farm which we finally conclude to be intelligent.)
      - 0xDEAFBEAD 2y ago
        >But AI (as we have it today and in the foreseeable future) is nowhere near the definition of a species. It is an enormous _server farm_ doing series of matrix multiplications followed by nonlinear transformations, with us, humans, supplying input data (as well as hardware, electricity, and maintenance) and assigning meaning to the outputs. And we have completely no ideas of doing "AI" any other way. One gorilla says to the other: "Those human brains are just synapses firing. They depend on nature to survive. Not a problem" Another way of thinking about it... Suppose we create a server emulation of a highly intelligent, manipulative serial killer, and speed up the emulation so it thinks 1000x as fast as a human. How do you feel about this? Is the fact that it's "just a server farm" reassuring?
        blksv 2y ago
  - EGreg 2y ago
    If "AI" is actually intelligent, then it's no worse a threat than any living being known to man How does the conclusion follow?
  - edanm 2y ago
    > * If "AI" is actually intelligent, then it's no worse a threat than any living being known to man. That's not true, because you don't address the degree of intelligence or the scale that computers can achieve. Even if all we were able to do is to create human-intelligence-equivalent AIs, but we were able to create, say, a billion on command, I'd say we have a major threat.
    - d1sxeyes 2y ago
      One point to recall on this as well is that if we do develop an intelligent AI, it may have taken us millions of years, but we did it. An intelligent AI would potentially be able to iterate and self develop much faster than we have been able to. It probably won’t take long to go from ‘human equivalent’ to being beyond that.
      - edanm 2y ago
        That's true, but I long ago got convinced that this particular worry isn't the most important one (the recursive-self-improvement one). I'm fairly sure that Yudkowsky and others are also pushing that less, because there is a reasonable chance that it's just not a necessary part of the argument; even without self-improving, it's possible that just something like GPT-8 or something is harmful enough on its own that there are significant risks.
  - ben_w 2y ago
    The most dangerous living beings in history have killed billions between them. Mostly that's diseases, but we can also easily add tens to hundreds of millions from humans fighting and starving each other over the last 200 years alone. Currently around a million a year killed in road traffic accidents, too, but I don't see stats for the total worldwide deaths since their invention. The greatest threats from other computer programs have been that during the Cold War the NATO early warning radar triggered an alert for a massive incoming fleet of Soviet aircraft because it hadn't been programmed to know that the moon wasn't supposed to respond to IFF pings, while the Soviet early warning radar triggered an alert an incoming ICBM because it hadn't been programmed to correctly handle the sun reflecting off clouds. It was human intervention which stopped that turning into a pointless self-annihilation, and that's specifically what gets lost as automation increases. I also do not believe that either of these threat models "are very thoroughly known and understood", because if the former was then we would've seen an end to genocides, while if the latter was we would've seen an end to zero-day vulnerabilities (especially in critical infrastructure).
- yehabut 2y ago
  That’s the thing though; reducing the risk rhetorically reduces existential threat. If it is propagated as “just electrons” no one gets anxious If it’s propagated as “end is nigh” everyone panics Both are reasonable in their grammatical correctness but only one seems a reasonable philosophy
tbrownaw 2y ago
Yes. The pen is mightier than the sword after all.
- dylan604 2y ago
  Yes, we have already been shown this is true in Indiana Jones Last Crusade
wokwokwok 2y ago
Can we move beyond the “man made climate change isn’t real” discourse where you somehow pretend that AI “Is no big deal”. Let’s have a closer look at the actual parent comment hm? > this organization is increasingly looking like a circus, and arguably worse, one that has taken its eye off the ball.
- lolinder 2y ago
  This isn't like climate change. "AI" is a successful marketing term for a loosely related collection of technologies that push the boundaries of what computation has hitherto been capable of. There are very real disagreements between very real experts on just how dangerous these new technologies are and just how far they'll actually be able to push the boundaries of computation. Comparing skepticism about AI alarmism to climate change denial is just lazy internet rhetoric. (One major difference between the two is that billion-dollar corporations stand to gain quite a bit if they can persuade governments that AI is so dangerous that it needs to be tightly regulated, so they're incentivized to play up the dangers. With climate change it's the opposite: the billion-dollar corporations are incentivized to downplay the risks so they can continue business as usual.)
  - wokwokwok 2y ago
    I didn’t say it was like climate change, I said it was like claiming climate change is not real. > There are very real disagreements between very real experts on just how dangerous these new technologies are …but they do not dispute that the technology is impactful. In fact, anyone who claims it is just “Text continuation”(quote) is being deliberately disingenuous and deliberately downplaying it. > Comparing skepticism about AI alarmism to climate change denial is just lazy internet rhetoric Unlike calling AI “Text continuation”? It’s awfully convenient it’s only lazy internet rhetoric when it’s coming from the other side. Hm? Why is only one side allowed to do that? Would you care to call out the other side for being reductionist? That’s a very high horse you’re sitting on.
    - deleted 2y ago
      [deleted]
tummler 2y ago
Hoo boy... if you think chatbots are all this is... wait til you find out what's next.
- ta988 2y ago
  What's next?
  - deleted 2y ago
    [deleted]
  - deleted 2y ago
    [deleted]
  - deleted 2y ago
    [deleted]
  - optomas 2y ago
    Are you capable of being manipulated through text? How about an AI trained to generate text that will extract exactly the desired response from you? An advertiser's wet dream.
    - EGreg 2y ago
      That’s thinking too small. Most people are nudged or coerced into taking action or supporting an action by the people who surround them. Their friends. Their coworkers. Their community. Like why people were on facebook 20 years ago, or forwarding chain emails, for instance. Unleashing an army of AI bots to infiltrate an online community and shift its discourse can be done at scale. While at the same time those humans who resist can be endlessly distracted by either arguing with bots or their own friends who have been affected by bots. All that is possible to do with CURRENT TECHNOLOGY.
conception 2y ago
https://www.theguardian.com/technology/article/2024/may/17/ai-weapons-palantir-war-technology https://www.theguardian.com/technology/article/2024/may/17/a... "...I also went to a panel in Palantir’s booth titled Civilian Harm Mitigation. It was led by two “privacy and civil liberties engineers” – a young man and woman who spoke exclusively in monotone. They also used countless euphemisms for bombing and death. The woman described how Palantir’s Gaia map tool lets users “nominate targets of interest” for “the target nomination process”. She meant it helps people choose which places get bombed. After she clicked a few options on an interactive map, a targeted landmass lit up with bright blue blobs. These blobs, she said, were civilian areas like hospitals and schools. The civilian locations could also be described in text, she said, but it can take a long time to read. So, Gaia uses a large language model (something like ChatGPT) to sift through this information and simplify it. Essentially, people choosing bomb targets get a dumbed-down version of information about where children sleep and families get medical treatment." Definitely not dangerous at all.
- badestrand 2y ago
  That rather sounds like Palantir is dangerous and not OpenAI.
- Skgqie1 2y ago
  Yes, having eyes is incredibly dangerous. It allows people to select a target from anything they can see.
  - mnyary 2y ago
    ”Another Lavender user questioned whether humans’ role in the selection process was meaningful. “I would invest 20 seconds for each target at this stage, and do dozens of them every day. I had zero added-value as a human, apart from being a stamp of approval. It saved a lot of time.” This article is more than 1 month old ‘The machine did it coldly’: Israel used AI to identify 37,000 Hamas targets; Guardian, 2 Apr 2024
- mr_mitm 2y ago
  But the claim was "th gravest of dangers". While your scenario is a nightmare, I'm not sure it qualifies as "the gravest".
tracerbulletx 2y ago
Well as you surely must know transformers are general sequence continuation machines and have been proven to work well with almost any type of input and output sequence. You also must be aware that people are trying to plug these general sequence continuation machines in to all kinds of different sequences and giving them all kinds of power to take action. These sequence continuation machines are also getting more complex and multi layered every day and also being wired together in novel ways. This is a dangerous thing to do depending on what actions you allow them to take because of the impact on society of having agents taking actions without oversight.
dheera 2y ago
If it's used to write code that runs on an ICBM without verification, yes. If it is used to write the firmware that runs inside 1 million lithium battery BMS systems that causes them all to catch on fire after 65536 seconds of runtime due to an integer overflow and code crashing and a FET getting left in the on state, then yes. I'm not an AI safety nut and don't think AI itself is a threat, but what is a threat is lazy humans using shitty AI to take over things that humans should be doing.
EGreg 2y ago
I will assume you’re asking in good faith. Text continuation to match what people around the internet are saying requires the model internally to “program itself”. It is a non-von-neumann architecture but it encodes durable structures in BILLIONS of bytes, that’s like a codebase of gigabytes. It can pass the Bar exam, various pre-med exams, knows more than any particular human on every subject, and it’s only been a couple years. The text continuation was the first interface, now they are doing multimodal. As it learns to predict the latent space for those things, it will be like predicting things in the real world. Training physical robots will become faster and easier too. In short, this is basically a system to search through the space of all possible alhorithms in order to “reason about” a huge model of the world, enough to understand it and predict it with small loss. AlphaZero used MCTS and can beat every human in chess now, and even every program written by a human, with beautiful gameplay. And even without AI, just the fact that the best performing algorithms are downloaded onto millions of computers is already multiplying the power far more than the 1% of humans who might do better in some area. So it can be better than 95% of humans at most tasks !
tempodox 2y ago
You're right, on the face of it it sounds absurd and by rights it should be. However, absurdity has never stopped mankind from doing horrible things. I for one don't see the gravest dangers in the tech itself, but in the stuff that people will (ab)use it for. And, of course, should AGI (which won't be text continuation, by the way) ever actually happen, then all bets are off. I just don't believe we're in any immediate danger of that.
bmitc 2y ago
Yes, exactly because it is just text continuation. It is being integrated into systems and used as a tool as if it is much, much more than text continuation. That is the danger. I've been using Copilot frequently lately, and it's startling just how often it is completely wrong despite being confidently presented as completely right.
wk_end 2y ago
I’m a bit of an AI skeptic and find the way people talk vaguely about the “dangers” of AI weird and maybe just more hype (or efforts for regulatory capture) rather than actual cause for concern. Still, to me, it seems like the two big scary things that might fall out of recent developments in generative AI are: 1. It might vacuum up an enormous number of jobs very quickly; enough jobs and quickly enough to cause serious social unrest. 2. The ability to generate convincing fake videos of basically anything might catastrophically erode our sense of truth or reality. Both of these strike me as plausible concerns in the near future (arguably they’re already happening). Not really sure what we could do about them, though - the cat is kind of out of the bag.
- verisimi 2y ago
  Re 1 - aren't people mostly doing bullshit jobs already? Re 2 - fakery in videos and images has been a part since their inception. So, don't worry, I guess? The only thing is the scale of bs that could be put out, and whether the wider population lose trust in what they see on screens. Which would be a good thing, imo.
  - shinycode 2y ago
    Bullshit jobs yes but still they are paid. What happens in a capitalist society when thousands of people loose their job and can’t produce value and contribute to society ? It’s tearing apart the fabric of capitalism is some way. What happen when all those people without jobs won’t be able to train for better jobs (because they would have done it already and won’t have a bullshit job in the first place) and either fall poor or move to less paid jobs ?
    - wordpad 2y ago
      Technology and automation tends to create more jobs. As operational costs decrease, your end service or product becomes cheaper, so all your b2b or b2c customers can afford more of it which leads to more opportunities, doing things which were cost prohibitive before. Some of those things may too require manual labor.
ignoramous 2y ago
> the greatest of dangers The danger comes from humans themselves wielding this tech, under the assumption it will keep improving quadratically. Some truly unhinged already thriving there: https://news.ycombinator.com/item?id=39124666 https://news.ycombinator.com/item?id=39124666
Nevermark 2y ago
Text continuation defines a task. It does not limit the sophistication of the solution. The difference between problems and solutions seems to trip up a lot of people. —- It gets even more nuanced. At its basic level all types of digital AI are “just” arithmetic, or “just” Boolean logic. But arithmetic’s and Boolean logic’s simplicity don’t limit what systems can be built with them, as they are Turing Complete. Likewise, the training algorithm for deep learning models is “just” gradient descent (or a variation), it’s “just” a dumb optimizer. But again, gradient descent places no limits on what a system with enough resources can learn. The whole point of learning systems is the basic resources of the model are provided, and within those resource limits (parameters, computational speed and time) the “dumb” learning algorithm learns the patterns in the data. The data patterns define the complexity of the solution. And the amount of information in a large collection of human correspondence includes patterns, meta patterns and abstractions for things like science, philosophy, psychology, law, art, on and on, that incidentally have a real bearing on sentence completion in that context. So a model will actually have to learn those things to perform its task well. New Law of Algorithm Level Confusion: Whenever someone says some learning system is “just” doing some simple task, they are confusing a simple task definition with a simple solution, or a simple learning algorithm with simple learned relationships. In both cases, the simplicity of the former places no limits on the sophistication of the latter.
- Terretta 2y ago
  > ... they are Turing Complete. OK, so all software must be regulated? Arguing that simple tools can be used for dangerous results and thus must be regulated or made “safe” is like saying since more murders in America are committed with hammers than long guns, you must show an ID to buy a hammer at Home Depot, and demanding "safety" training of text continuation models is like requiring hammers be redesigned to have pillows for hammerheads. We already have laws against bad acts. Hold wrongdoers accountable. Leave hammers alone.
  - Nevermark 2y ago
    I was responding directly to your comment: > Text continuation poses "the greatest of dangers"? The task of text continuation is no guarantee of safety. A language model trained on pure military information, communication and actions could be quite dangerous if given an ability to direct, or contribute, to actual action. Not arguing anything about regulation.
VladimirGolovin 2y ago
Text continuation doesn't. AGI does.
hpeter 2y ago
Yes because people are stupid and vulnerable and will believe things a computer tells them. They can make life decisions on chatGPT advice, instead of asking friends or family. Heck maybe they don't have family, vulnerable people like that can be exploited with convincing chatbots easily.
gosub100 2y ago
I haven't read all the replies to see if this has been posted yet, but one of the consequences is erosion of trust in reality itself: we're starting to see the comments "are you a bot?/is this real? I can't tell if you're serious or if this was from ChatGPT". It's quite possible that soon the majority of comments online will be with bots, or people will just avoid discourse more and more because they don't know/believe they are talking to a human.

Text continuation poses "the greatest of dangers"?

hn_go_brrrrr 2y ago

While we're being overly reductionist: it's just moving bits around, what harm could that possibly do?

brigadier132 2y ago
[flagged]
Dalewyn 2y ago
"AI" is one of two things depending on whether you think it's actually intelligent: * If "AI" is actually intelligent, then it's no worse a threat than any living being known to man. * If "AI" is not actually intelligent, then it's no worse a threat than any other computer program. Both threat models are very thoroughly known and understood. I second parent commenter's sentiment that calling "AI" the "gravest of dangers" is a gross misrepresentation.
- comp_throw7 2y ago
  > * If "AI" is actually intelligent, then it's no worse a threat than any living being known to man. ...does not follow? "If an atom bomb is actually a bomb, then it's no worse a threat than any other existing bomb known to man."
  - Dalewyn 2y ago
    An atomic bomb explodes. A bomb explodes. Both are very destructive. Is there a difference? I say no, and I say that as a Japanese-American who visited Hiroshima Peace Memorial numerous times and never felt anything particular from the exhibits. People die when they are killed. Likewise, if "AI" is actually intelligent then it is no worse a threat than a bumbling baby or a Vladimir Putin or a dog, among other living beings.
    - mlyle 2y ago
      > Likewise, if "AI" is actually intelligent then it is no worse a threat than a bumbling baby or a Vladimir Putin or a dog, among other living beings. I think a dog and a dictator might both be able to have negative impacts, but treating them as equal just shows a desire to troll. A dog can only bite so many people.
    - comp_throw7 2y ago
      > People die when they are killed. Yes, indeed, which is why we should avoid creating something that will kill everyone... ...so I guess I'm glad to hear there's no difference in the amount of damage that can be inflicted by a baby, and by Vladimir Putin, and superintelligent AI really can't be any worse than either.
    - BuyMyBitcoins 2y ago
      >”An atomic bomb explodes. A bomb explodes. Both are very destructive. Is there a difference? I say no” Atomic bombs and hydrogen bombs are orders of magnitude more destructive and also produce fallout. How in the world can that difference be dismissed?
      - jiggawatts 2y ago
        Not the original commenter, but one interpretation is that “dead is dead”. You can’t be “more killed”, so past a point, all dangerous things are equally dangerous.
        Dalewyn 2y ago
        This. I don't see nuclear weapons as any worse than conventional weapons because ultimately people die. Is it comforting or otherwise somehow better for people to be killed by conventional weapons instead of nuclear weapons? I find such a proposition preposterous. All bombs are bad, whether they're conventional or nuclear is irrelevant. For that matter, all wars are bad and should be avoided unless no other options remain. Likewise, "AI" is just as much a threat as any other intelligent being or computer program depending on which line of thought you subscribe to. "AI" is not a "gravest of dangers".
- Jedd 2y ago
  I don't believe asserting two parallel and comparably false equivalences meaningfully progresses the discussion around this technology and risk. If this tech exceeds human capabilities to 'think' (plan, reason, manipulate, etc) then it is obviously a bigger threat than any less capable entity ('any living being', as you refer). Because no humans will be able to out-plan/reason/manipulate it. If the tech doesn't achieve intelligence - a tricky goal, and I eschew mislabelling LLMs as AIs for that reason - that doesn't mean its risk is comparable to the worst of any existing technology. There's just no way to extrapolate from one to the other, there. It might be orders of magnitude worse than any other 'computer program' - you can't say for sure, from where we're standing now.
  - Aexian 2y ago
    > you can't say for sure, from where we're standing now. We can already see how LLMs can be substantially worse - they can cosplay human thoughts and sentiment in a way that wasn’t previously possible. So much of this debate seems focused on this conception of Skynet-style murderous AI - or at least manipulative and scheming HAL 9000 types. But an order of magnitude greater risk has already arrived just from scaling existing harms. Phishing schemes and pig butchering scams have destroyed countless lives. They’re now easier and more scalable than ever. As are fake news and disinformation campaigns. Several companies are productising AI girlfriends with predatory pricing models, capitalising the human desire for connection and intimacy in the way slot machines and sports betting monetise our desire for a better life. That’s new. It may not be an order of magnitude worse for everyone yet - but for certain vulnerable groups, that future has arrived already.
- 0xDEAFBEAD 2y ago
  >* If "AI" is actually intelligent, then it's no worse a threat than any living being known to man. Humans are quite a threat to other species, e.g. gorillas, largely due to our intelligence advantage. If a new species arrives on Earth that's smarter than we are, there's a good chance we'll be displaced.
  - blksv 2y ago
    But AI (as we have it today and in the foreseeable future) is nowhere near the definition of a species. It is an enormous _server farm_ doing series of matrix multiplications followed by nonlinear transformations, with us, humans, supplying input data (as well as hardware, electricity, and maintenance) and assigning meaning to the outputs. And we have completely no ideas of doing "AI" any other way. I agree, that AI may be dangerous if used for destructive purposes, or if used for some critical tasks with too much trust (and the hype that "we're so dangerously near a superintelligence" makes the latter much more likely, in my opinion). But that Humanity will be displaced by autonomous server farms? No way. (As to the original comment, I think that a bunch of nearly(?) demented elders holding nuclear buttons is a much worse (and immediate!) threat than a server farm which we finally conclude to be intelligent.)
    - 0xDEAFBEAD 2y ago
      >But AI (as we have it today and in the foreseeable future) is nowhere near the definition of a species. It is an enormous _server farm_ doing series of matrix multiplications followed by nonlinear transformations, with us, humans, supplying input data (as well as hardware, electricity, and maintenance) and assigning meaning to the outputs. And we have completely no ideas of doing "AI" any other way. One gorilla says to the other: "Those human brains are just synapses firing. They depend on nature to survive. Not a problem" Another way of thinking about it... Suppose we create a server emulation of a highly intelligent, manipulative serial killer, and speed up the emulation so it thinks 1000x as fast as a human. How do you feel about this? Is the fact that it's "just a server farm" reassuring?
      - blksv 2y ago
- EGreg 2y ago
  If "AI" is actually intelligent, then it's no worse a threat than any living being known to man How does the conclusion follow?
- edanm 2y ago
  > * If "AI" is actually intelligent, then it's no worse a threat than any living being known to man. That's not true, because you don't address the degree of intelligence or the scale that computers can achieve. Even if all we were able to do is to create human-intelligence-equivalent AIs, but we were able to create, say, a billion on command, I'd say we have a major threat.
  - d1sxeyes 2y ago
    One point to recall on this as well is that if we do develop an intelligent AI, it may have taken us millions of years, but we did it. An intelligent AI would potentially be able to iterate and self develop much faster than we have been able to. It probably won’t take long to go from ‘human equivalent’ to being beyond that.
    - edanm 2y ago
      That's true, but I long ago got convinced that this particular worry isn't the most important one (the recursive-self-improvement one). I'm fairly sure that Yudkowsky and others are also pushing that less, because there is a reasonable chance that it's just not a necessary part of the argument; even without self-improving, it's possible that just something like GPT-8 or something is harmful enough on its own that there are significant risks.
- ben_w 2y ago
  The most dangerous living beings in history have killed billions between them. Mostly that's diseases, but we can also easily add tens to hundreds of millions from humans fighting and starving each other over the last 200 years alone. Currently around a million a year killed in road traffic accidents, too, but I don't see stats for the total worldwide deaths since their invention. The greatest threats from other computer programs have been that during the Cold War the NATO early warning radar triggered an alert for a massive incoming fleet of Soviet aircraft because it hadn't been programmed to know that the moon wasn't supposed to respond to IFF pings, while the Soviet early warning radar triggered an alert an incoming ICBM because it hadn't been programmed to correctly handle the sun reflecting off clouds. It was human intervention which stopped that turning into a pointless self-annihilation, and that's specifically what gets lost as automation increases. I also do not believe that either of these threat models "are very thoroughly known and understood", because if the former was then we would've seen an end to genocides, while if the latter was we would've seen an end to zero-day vulnerabilities (especially in critical infrastructure).
yehabut 2y ago
That’s the thing though; reducing the risk rhetorically reduces existential threat. If it is propagated as “just electrons” no one gets anxious If it’s propagated as “end is nigh” everyone panics Both are reasonable in their grammatical correctness but only one seems a reasonable philosophy

tbrownaw 2y ago

Yes. The pen is mightier than the sword after all.

dylan604 2y ago
Yes, we have already been shown this is true in Indiana Jones Last Crusade

wokwokwok 2y ago

Can we move beyond the “man made climate change isn’t real” discourse where you somehow pretend that AI “Is no big deal”. Let’s have a closer look at the actual parent comment hm? > this organization is increasingly looking like a circus, and arguably worse, one that has taken its eye off the ball.

lolinder 2y ago
This isn't like climate change. "AI" is a successful marketing term for a loosely related collection of technologies that push the boundaries of what computation has hitherto been capable of. There are very real disagreements between very real experts on just how dangerous these new technologies are and just how far they'll actually be able to push the boundaries of computation. Comparing skepticism about AI alarmism to climate change denial is just lazy internet rhetoric. (One major difference between the two is that billion-dollar corporations stand to gain quite a bit if they can persuade governments that AI is so dangerous that it needs to be tightly regulated, so they're incentivized to play up the dangers. With climate change it's the opposite: the billion-dollar corporations are incentivized to downplay the risks so they can continue business as usual.)
- wokwokwok 2y ago
  I didn’t say it was like climate change, I said it was like claiming climate change is not real. > There are very real disagreements between very real experts on just how dangerous these new technologies are …but they do not dispute that the technology is impactful. In fact, anyone who claims it is just “Text continuation”(quote) is being deliberately disingenuous and deliberately downplaying it. > Comparing skepticism about AI alarmism to climate change denial is just lazy internet rhetoric Unlike calling AI “Text continuation”? It’s awfully convenient it’s only lazy internet rhetoric when it’s coming from the other side. Hm? Why is only one side allowed to do that? Would you care to call out the other side for being reductionist? That’s a very high horse you’re sitting on.
  - deleted 2y ago
    [deleted]

tummler 2y ago

Hoo boy... if you think chatbots are all this is... wait til you find out what's next.

ta988 2y ago
What's next?
- deleted 2y ago
  [deleted]
- deleted 2y ago
  [deleted]
- deleted 2y ago
  [deleted]
- optomas 2y ago
  Are you capable of being manipulated through text? How about an AI trained to generate text that will extract exactly the desired response from you? An advertiser's wet dream.
  - EGreg 2y ago
    That’s thinking too small. Most people are nudged or coerced into taking action or supporting an action by the people who surround them. Their friends. Their coworkers. Their community. Like why people were on facebook 20 years ago, or forwarding chain emails, for instance. Unleashing an army of AI bots to infiltrate an online community and shift its discourse can be done at scale. While at the same time those humans who resist can be endlessly distracted by either arguing with bots or their own friends who have been affected by bots. All that is possible to do with CURRENT TECHNOLOGY.

conception 2y ago

https://www.theguardian.com/technology/article/2024/may/17/ai-weapons-palantir-war-technology https://www.theguardian.com/technology/article/2024/may/17/a... "...I also went to a panel in Palantir’s booth titled Civilian Harm Mitigation. It was led by two “privacy and civil liberties engineers” – a young man and woman who spoke exclusively in monotone. They also used countless euphemisms for bombing and death. The woman described how Palantir’s Gaia map tool lets users “nominate targets of interest” for “the target nomination process”. She meant it helps people choose which places get bombed. After she clicked a few options on an interactive map, a targeted landmass lit up with bright blue blobs. These blobs, she said, were civilian areas like hospitals and schools. The civilian locations could also be described in text, she said, but it can take a long time to read. So, Gaia uses a large language model (something like ChatGPT) to sift through this information and simplify it. Essentially, people choosing bomb targets get a dumbed-down version of information about where children sleep and families get medical treatment." Definitely not dangerous at all.

badestrand 2y ago
That rather sounds like Palantir is dangerous and not OpenAI.
Skgqie1 2y ago
Yes, having eyes is incredibly dangerous. It allows people to select a target from anything they can see.
- mnyary 2y ago
  ”Another Lavender user questioned whether humans’ role in the selection process was meaningful. “I would invest 20 seconds for each target at this stage, and do dozens of them every day. I had zero added-value as a human, apart from being a stamp of approval. It saved a lot of time.” This article is more than 1 month old ‘The machine did it coldly’: Israel used AI to identify 37,000 Hamas targets; Guardian, 2 Apr 2024
mr_mitm 2y ago
But the claim was "th gravest of dangers". While your scenario is a nightmare, I'm not sure it qualifies as "the gravest".

tracerbulletx 2y ago

Well as you surely must know transformers are general sequence continuation machines and have been proven to work well with almost any type of input and output sequence. You also must be aware that people are trying to plug these general sequence continuation machines in to all kinds of different sequences and giving them all kinds of power to take action. These sequence continuation machines are also getting more complex and multi layered every day and also being wired together in novel ways. This is a dangerous thing to do depending on what actions you allow them to take because of the impact on society of having agents taking actions without oversight.

dheera 2y ago

If it's used to write code that runs on an ICBM without verification, yes. If it is used to write the firmware that runs inside 1 million lithium battery BMS systems that causes them all to catch on fire after 65536 seconds of runtime due to an integer overflow and code crashing and a FET getting left in the on state, then yes. I'm not an AI safety nut and don't think AI itself is a threat, but what is a threat is lazy humans using shitty AI to take over things that humans should be doing.

EGreg 2y ago

I will assume you’re asking in good faith. Text continuation to match what people around the internet are saying requires the model internally to “program itself”. It is a non-von-neumann architecture but it encodes durable structures in BILLIONS of bytes, that’s like a codebase of gigabytes. It can pass the Bar exam, various pre-med exams, knows more than any particular human on every subject, and it’s only been a couple years. The text continuation was the first interface, now they are doing multimodal. As it learns to predict the latent space for those things, it will be like predicting things in the real world. Training physical robots will become faster and easier too. In short, this is basically a system to search through the space of all possible alhorithms in order to “reason about” a huge model of the world, enough to understand it and predict it with small loss. AlphaZero used MCTS and can beat every human in chess now, and even every program written by a human, with beautiful gameplay. And even without AI, just the fact that the best performing algorithms are downloaded onto millions of computers is already multiplying the power far more than the 1% of humans who might do better in some area. So it can be better than 95% of humans at most tasks !

tempodox 2y ago

You're right, on the face of it it sounds absurd and by rights it should be. However, absurdity has never stopped mankind from doing horrible things. I for one don't see the gravest dangers in the tech itself, but in the stuff that people will (ab)use it for. And, of course, should AGI (which won't be text continuation, by the way) ever actually happen, then all bets are off. I just don't believe we're in any immediate danger of that.

bmitc 2y ago

Yes, exactly because it is just text continuation. It is being integrated into systems and used as a tool as if it is much, much more than text continuation. That is the danger. I've been using Copilot frequently lately, and it's startling just how often it is completely wrong despite being confidently presented as completely right.

wk_end 2y ago

I’m a bit of an AI skeptic and find the way people talk vaguely about the “dangers” of AI weird and maybe just more hype (or efforts for regulatory capture) rather than actual cause for concern. Still, to me, it seems like the two big scary things that might fall out of recent developments in generative AI are: 1. It might vacuum up an enormous number of jobs very quickly; enough jobs and quickly enough to cause serious social unrest. 2. The ability to generate convincing fake videos of basically anything might catastrophically erode our sense of truth or reality. Both of these strike me as plausible concerns in the near future (arguably they’re already happening). Not really sure what we could do about them, though - the cat is kind of out of the bag.

verisimi 2y ago
Re 1 - aren't people mostly doing bullshit jobs already? Re 2 - fakery in videos and images has been a part since their inception. So, don't worry, I guess? The only thing is the scale of bs that could be put out, and whether the wider population lose trust in what they see on screens. Which would be a good thing, imo.
- shinycode 2y ago
  Bullshit jobs yes but still they are paid. What happens in a capitalist society when thousands of people loose their job and can’t produce value and contribute to society ? It’s tearing apart the fabric of capitalism is some way. What happen when all those people without jobs won’t be able to train for better jobs (because they would have done it already and won’t have a bullshit job in the first place) and either fall poor or move to less paid jobs ?
  - wordpad 2y ago
    Technology and automation tends to create more jobs. As operational costs decrease, your end service or product becomes cheaper, so all your b2b or b2c customers can afford more of it which leads to more opportunities, doing things which were cost prohibitive before. Some of those things may too require manual labor.

ignoramous 2y ago

> the greatest of dangers The danger comes from humans themselves wielding this tech, under the assumption it will keep improving quadratically. Some truly unhinged already thriving there: https://news.ycombinator.com/item?id=39124666 https://news.ycombinator.com/item?id=39124666

Nevermark 2y ago

Text continuation defines a task. It does not limit the sophistication of the solution. The difference between problems and solutions seems to trip up a lot of people. —- It gets even more nuanced. At its basic level all types of digital AI are “just” arithmetic, or “just” Boolean logic. But arithmetic’s and Boolean logic’s simplicity don’t limit what systems can be built with them, as they are Turing Complete. Likewise, the training algorithm for deep learning models is “just” gradient descent (or a variation), it’s “just” a dumb optimizer. But again, gradient descent places no limits on what a system with enough resources can learn. The whole point of learning systems is the basic resources of the model are provided, and within those resource limits (parameters, computational speed and time) the “dumb” learning algorithm learns the patterns in the data. The data patterns define the complexity of the solution. And the amount of information in a large collection of human correspondence includes patterns, meta patterns and abstractions for things like science, philosophy, psychology, law, art, on and on, that incidentally have a real bearing on sentence completion in that context. So a model will actually have to learn those things to perform its task well. New Law of Algorithm Level Confusion: Whenever someone says some learning system is “just” doing some simple task, they are confusing a simple task definition with a simple solution, or a simple learning algorithm with simple learned relationships. In both cases, the simplicity of the former places no limits on the sophistication of the latter.

Terretta 2y ago
> ... they are Turing Complete. OK, so all software must be regulated? Arguing that simple tools can be used for dangerous results and thus must be regulated or made “safe” is like saying since more murders in America are committed with hammers than long guns, you must show an ID to buy a hammer at Home Depot, and demanding "safety" training of text continuation models is like requiring hammers be redesigned to have pillows for hammerheads. We already have laws against bad acts. Hold wrongdoers accountable. Leave hammers alone.
- Nevermark 2y ago
  I was responding directly to your comment: > Text continuation poses "the greatest of dangers"? The task of text continuation is no guarantee of safety. A language model trained on pure military information, communication and actions could be quite dangerous if given an ability to direct, or contribute, to actual action. Not arguing anything about regulation.

VladimirGolovin 2y ago

Text continuation doesn't. AGI does.

hpeter 2y ago

Yes because people are stupid and vulnerable and will believe things a computer tells them. They can make life decisions on chatGPT advice, instead of asking friends or family. Heck maybe they don't have family, vulnerable people like that can be exploited with convincing chatbots easily.

gosub100 2y ago

I haven't read all the replies to see if this has been posted yet, but one of the consequences is erosion of trust in reality itself: we're starting to see the comments "are you a bot?/is this real? I can't tell if you're serious or if this was from ChatGPT". It's quite possible that soon the majority of comments online will be with bots, or people will just avoid discourse more and more because they don't know/believe they are talking to a human.