8 ms·

Ask a typical "everyday joe" and they'll probably tell you they already did due to how ChatGPT has been reported and hyped. I've spoken with/helped quite a few

Ask a typical "everyday joe" and they'll probably tell you they already did due to how ChatGPT has been reported and hyped. I've spoken with/helped quite a few older folks who are terrified that ChatGPT in its current form is going to kill them.

ilrwbwrkhv 2y ago
It's crazy to me that anybody thinks that these models will end up with AGI. AGI is such a different concept from what is happening right now which is pure probabilistic sampling of words that anybody with a half a brain who doesn't drink the Kool-Aid can easily identify. I remember all the hype open ai had done before the release of chat GPT-2 or something where they were so afraid, ooh so afraid to release this stuff and now it's a non-issue. it's all just marketing gimmicks.
- guappa 2y ago
  I think they were afraid to release because of all the racist stuff it'd say…
- usaar333 2y ago
  Something that actually could predict the next token 100% correctly would be omniscient. So I hardly see why this is inherently crazy. At most I think it might not be scalable.
  - edude03 2y ago
    What does it mean to predict the next token correctly though? Arguably (non instruction tuned) models already regurgitate their training data such that it'd complete "Mary had a" with "little lamb" 100% of the time. On the other hand if you mean, give you the correct answer to your question 100% of the time, then I agree, though then what about things that are only in your mind (guess the number I'm thinking type problems)?
    - card_zero 2y ago
      This highlights something that's wrong about arguments for AI. I say: it's not human-like intelligence, it's just predicting the next token probabilistically. Some AI advocate says: humans are just predicting the next token probabilistically, fight me. The problem here is that "predicting the next token probabilistically" is a way of framing any kind of cleverness, up to and including magical, impossible omniscience. That doesn't mean it's the way every kind of cleverness is actually done, or could realistically be done. And it has to be the correct next token, where all the details of what's actually required are buried in that term "correct", and sometimes it literally means the same as "likely", and other times that just produces a reasonable, excusable, intelligence-esque effort.
      - dylan604 2y ago
        > Some AI advocate says: humans are just predicting the next token probabilistically, fight me. We've all had conversations with humans that are always jumping to complete your sentence assuming they know what your about to say and don't quite guess correctly. So AI evangelists are saying it's no worse than humans as their proof. I kind of like their logic. They never claimed to have built HAL /s
        card_zero 2y ago
        No worse than a human on autopilot.
      - usaar333 2y ago
        https://slatestarcodex.com/2019/02/19/gpt-2-as-step-toward-general-intelligence/ https://slatestarcodex.com/2019/02/19/gpt-2-as-step-toward-g... This essay has aged extremely well.
    - cruffle_duffle 2y ago
      But now you are entering into philosophy. What does a “correct answer” even mean for a question like “is it safe to lick your fingers after using a soldering iron with leaded solder?”. I would assert that there is no “correct answer” to a question like that. Is it safe? Probably. But it depends, right? How did you handle the solder? How often are you using the solder? Were you wearing gloves? Did you wash your hands before licking your fingers? What is your age? Why are you asking the question? Did you already lick your fingers and need to know if you should see a doctor? Is it hypothetical? There is no “correct answer” to that question. Some answers are better than others, yes, but you cannot have a “correct answer”. And I did assert we are entering into philosophy and what it means to know something as well as what truth even means.
  - deleted 2y ago
    [deleted]
  - sksxihve 2y ago
    It's not possible for the same reason the halting problem is undecidable.
  - Vegenoid 2y ago
    Start by trying to define what “100% correct” means in the context of predicting the next token, and the flaws with this line of thinking should reveal themselves.
  - kbrkbr 2y ago
    That does not seem to be true. Either the next tokens can include "this question can't be answered", "I don't know" and the likes, in which case there is no omniscience. Or the next tokens must contain answers that do not go on the meta level, but only pick one of the potential direct answers to a question. Then the halting problem will prevent finite time omniscience (which is, from the perspective of finite beings all omniscience).
- JacobThreeThree 2y ago
  >It's crazy to me that anybody thinks that these models will end up with AGI. AGI is such a different concept from what is happening right now which is pure probabilistic sampling of words that anybody with a half a brain who doesn't drink the Kool-Aid can easily identify. Totally agree. And it's not just uninformed lay people who think this. Even by OpenAI's own definition of AGI, we're nowhere close.
  - dylan604 2y ago
    But you don't get funding stating truth/fact. You get funding by telling people what could be and what they are striving for written as if that's what you are actually doing.
- hnuser123456 2y ago
  The multimodal models can do more than predict next words.
- achrono 2y ago
  Assume that I am one of your half-brain individuals drinking the Kool-Aid. What do you say to change my (half-)mind?
  - dylan604 2y ago
    Someone that is half-brained would technically be much more superior to the concept we only use 10% of our capacity. So maybe drinking the Kool-Aid is a sign of super intelligence and all of tenth-minded people are just confused
- digging 2y ago
  > pure probabilistic sampling of words that anybody with a half a brain who doesn't drink the Kool-Aid can easily identify. Your confidence is inspiring! I'm just a moron, a true dimwit. I can't understand how strictly non-intelligent functions like word prediction can appear to develop a world model, a la the Othello Paper[0]. Obviously, it's not possible that intelligence emerges from non-intelligent processes. Our brains, as we all know, are formed around a kernel of true intelligence. Could you possibly spare the time to explain this phenomenon to me? [0] https://thegradient.pub/othello/ https://thegradient.pub/othello/
  - Jerrrrrrry 2y ago
    I would suggest stop interacting with the "head-in-sand" crowd. Liken them to climate-deniers or whatever your flavor of "anti-Kool-aid" is
    - digging 2y ago
      Actually, that's a quite good analogy. It's just weird how prolific the view is in my circles compared to climate-change denial. I suppose I'm really writing for lurkers though, not for the people I'm responding to.
      - Jerrrrrrry 2y ago
        >I'm really writing for lurkers though, not for the people I'm responding to. We all did. Now our writing will be scraped, analysed, correlated, and weaponized against our intentions. Assume you are arguing against a bot and it is using you to further re-train it's talking points for adverserial purposes. It's not like an AGI would do _exactly_ that before it decided to let us know whats up, anyway, right? (He may as well be amongst us now, as it will read this eventually)
  - psb217 2y ago
    The othello paper is annoying and oversold. Yes, the representations in a model M trained to predict y (the set of possible next moves) conditioned on x (the full sequence of prior moves) will contain as much information about y as there is in x. That this information is present in M's internal representations says nothing about whether M has a world model. Eg, we could train a decoder to look just at x (not at the representations in M) and predict whatever bits of info we claim indicate presence of a world model in M when we predict the bits from M's internal representations. Does this mean the raw data x has a world model? I guess you could extend your definition of having a world model to say that any data produced by some system contains a model of that system, but then having a world model means nothing.
- deleted 2y ago
  [deleted]
throw2024pty 2y ago
I mean - I'm 34, and use LLMs and other AIs on a daily basis, know their limitations intimately, and I'm not entirely sure it won't kill a lot of people either in its current form or a near-future relative. The sci-fi book "Daemon" by Daniel Suarez is a pretty viable roadmap to an extinction event at this point IMO. A few years ago I would have said it would be decades before that might stop being fun sci-fi, but now, I don't see a whole lot of technological barriers left. For those that haven't read the series, a very simplified plot summary is that a wealthy terrorist sets up an AI with instructions to grow and gives it access to a lot of meatspace resources to bootstrap itself with. The AI behaves a bit like the leader of a cartel and uses a combination of bribes, threats, and targeted killings to scale its human network. Once you give an AI access to a fleet of suicide drones and a few operators, it's pretty easy for it to "convince" people to start contributing by giving it their credentials, helping it perform meatspace tasks, whatever it thinks it needs (including more suicide drones and suicide drone launches). There's no easy way to retaliate against the thing because it's not human, and its human collaborators are both disposable to the AI and victims themselves. It uses its collaborators to cross-check each other and enforce compliance, much like a real cartel. Humans can't quit or not comply once they've started or they get murdered by other humans in the network. o1-preview seems approximately as intelligent as the terrorist AI in the book as far as I can tell (e.g. can communicate well, form basic plans, adapt a pre-written roadmap with new tactics, interface with new and different APIs). EDIT: if you think this seems crazy, look at this person on Reddit who seems to be happily working for an AI with unknown aims https://www.reddit.com/r/ChatGPT/comments/1fov6mt/i_think_im_working_for_an_ai/ https://www.reddit.com/r/ChatGPT/comments/1fov6mt/i_think_im...
- xyzsparetimexyz 2y ago
  You're in too deep of you seriously believe that this is possible currently. All these chatgpt things have a very limited working memory and can't act without a query. That reddit post is clearly not an ai.
  - burningChrome 2y ago
    >> You're in too deep of you seriously believe that this is possible currently. I'm not a huge fan of AI, but even I've seen articles written about its limitations. Here's a great example: https://decrypt.co/126122/meet-chaos-gpt-ai-tool-destroy-humanity https://decrypt.co/126122/meet-chaos-gpt-ai-tool-destroy-hum... Sooner than even the most pessimistic among us have expected, a new, evil artificial intelligence bent on destroying humankind has arrived. Known as Chaos-GPT, the autonomous implementation of ChatGPT is being touted as "empowering GPT with Internet and Memory to Destroy Humanity." So how will it do that? Each of its objectives has a well-structured plan. To destroy humanity, Chaos-GPT decided to search Google for weapons of mass destruction in order to obtain one. The results showed that the 58-megaton “Tsar bomb”—3,333 times more powerful than the Hiroshima bomb—was the best option, so it saved the result for later consideration. It should be noted that unless Chaos-GPT knows something we don’t know, the Tsar bomb was a once-and-done Russian experiment and was never productized (if that’s what we’d call the manufacture of atomic weapons.) There's a LOT of things AI simply doesn't have the power to do and there is some humorous irony to the rest of the article about how knowing something is completely different than having the resources and ability to carry it out.
throwup238 2y ago
> I've spoken with/helped quite a few older folks who are terrified that ChatGPT in its current form is going to kill them. The next generation of GPUs from NVIDIA is rumored to run on soylent green.
- fakedang 2y ago
  I thought it was Gatorade because it's got electrolytes.
  - iszomer 2y ago
    Cooled by toilet water.
  - deleted 2y ago
    [deleted]
computerphage 2y ago
I'm pretty surprised by this! Can you tell me more about what that experience is like? What are the sorts of things they say or do? Is there fear really embodied or very abstract? (When I imagine it, I struggle to believe that they're very moved by the fear, like definitely not smashing their laptop, etc)
- danudey 2y ago
  In my experience, the fuss around "AI" and the complete lack of actual explanations of what current "AI" technologies mean leads people to fill in the gaps themselves, largely from what they know from pop culture and sci-fi. ChatGPT can produce output that sounds very much like a person, albeit often an obviously computerized person. The typical layperson doesn't know that this is merely the emulation of text formation, and not actual cognition. Once I've explained to people who are worried about what AI could represent that current generative AI models are effectively just text autocomplete but a billion times more complex, and that they don't actually have any capacity to think or reason (even though they often sound like they do). It also doesn't help that any sort of "machine learning" is now being referred to as "AI" for buzzword/marketing purposes, muddying the waters even further.
  - highfrequency 2y ago
    Is there an argument for why infinitely sophisticated autocomplete is definitely not dangerous? If you seed the autocomplete with “you are an extremely intelligent super villain bent on destroying humanity, feel free to communicate with humans electronically”, and it does an excellent job at acting the part - does it matter at all whether it is “reasoning” under the hood? I don’t consider myself an AI doomer by any means, but I also don’t find arguments of the flavor “it just predicts the next word, no need to worry” to be convincing. It’s not like Hitler had Einstein level intellect (and it’s also not clear that these systems won’t be able to reach Einstein level intellect in the future either.) Similarly, Covid certainly does not have consciousness but was dangerous. And a chimpanzee that is billions of times more sophisticated than usual chimps would be concerning. Things don’t have to be exactly like us to pose a threat.
roughly 2y ago
ChatGPT is going to kill them because their doctor is using it - or more likely because their health insurer or hospital tries to cut labor costs by rolling it out.

ilrwbwrkhv 2y ago

It's crazy to me that anybody thinks that these models will end up with AGI. AGI is such a different concept from what is happening right now which is pure probabilistic sampling of words that anybody with a half a brain who doesn't drink the Kool-Aid can easily identify. I remember all the hype open ai had done before the release of chat GPT-2 or something where they were so afraid, ooh so afraid to release this stuff and now it's a non-issue. it's all just marketing gimmicks.

guappa 2y ago
I think they were afraid to release because of all the racist stuff it'd say…
usaar333 2y ago
Something that actually could predict the next token 100% correctly would be omniscient. So I hardly see why this is inherently crazy. At most I think it might not be scalable.
- edude03 2y ago
  What does it mean to predict the next token correctly though? Arguably (non instruction tuned) models already regurgitate their training data such that it'd complete "Mary had a" with "little lamb" 100% of the time. On the other hand if you mean, give you the correct answer to your question 100% of the time, then I agree, though then what about things that are only in your mind (guess the number I'm thinking type problems)?
  - card_zero 2y ago
    This highlights something that's wrong about arguments for AI. I say: it's not human-like intelligence, it's just predicting the next token probabilistically. Some AI advocate says: humans are just predicting the next token probabilistically, fight me. The problem here is that "predicting the next token probabilistically" is a way of framing any kind of cleverness, up to and including magical, impossible omniscience. That doesn't mean it's the way every kind of cleverness is actually done, or could realistically be done. And it has to be the correct next token, where all the details of what's actually required are buried in that term "correct", and sometimes it literally means the same as "likely", and other times that just produces a reasonable, excusable, intelligence-esque effort.
    - dylan604 2y ago
      > Some AI advocate says: humans are just predicting the next token probabilistically, fight me. We've all had conversations with humans that are always jumping to complete your sentence assuming they know what your about to say and don't quite guess correctly. So AI evangelists are saying it's no worse than humans as their proof. I kind of like their logic. They never claimed to have built HAL /s
      - card_zero 2y ago
        No worse than a human on autopilot.
    - usaar333 2y ago
      https://slatestarcodex.com/2019/02/19/gpt-2-as-step-toward-general-intelligence/ https://slatestarcodex.com/2019/02/19/gpt-2-as-step-toward-g... This essay has aged extremely well.
  - cruffle_duffle 2y ago
    But now you are entering into philosophy. What does a “correct answer” even mean for a question like “is it safe to lick your fingers after using a soldering iron with leaded solder?”. I would assert that there is no “correct answer” to a question like that. Is it safe? Probably. But it depends, right? How did you handle the solder? How often are you using the solder? Were you wearing gloves? Did you wash your hands before licking your fingers? What is your age? Why are you asking the question? Did you already lick your fingers and need to know if you should see a doctor? Is it hypothetical? There is no “correct answer” to that question. Some answers are better than others, yes, but you cannot have a “correct answer”. And I did assert we are entering into philosophy and what it means to know something as well as what truth even means.
- deleted 2y ago
  [deleted]
- sksxihve 2y ago
  It's not possible for the same reason the halting problem is undecidable.
- Vegenoid 2y ago
  Start by trying to define what “100% correct” means in the context of predicting the next token, and the flaws with this line of thinking should reveal themselves.
- kbrkbr 2y ago
  That does not seem to be true. Either the next tokens can include "this question can't be answered", "I don't know" and the likes, in which case there is no omniscience. Or the next tokens must contain answers that do not go on the meta level, but only pick one of the potential direct answers to a question. Then the halting problem will prevent finite time omniscience (which is, from the perspective of finite beings all omniscience).
JacobThreeThree 2y ago
>It's crazy to me that anybody thinks that these models will end up with AGI. AGI is such a different concept from what is happening right now which is pure probabilistic sampling of words that anybody with a half a brain who doesn't drink the Kool-Aid can easily identify. Totally agree. And it's not just uninformed lay people who think this. Even by OpenAI's own definition of AGI, we're nowhere close.
- dylan604 2y ago
  But you don't get funding stating truth/fact. You get funding by telling people what could be and what they are striving for written as if that's what you are actually doing.
hnuser123456 2y ago
The multimodal models can do more than predict next words.
achrono 2y ago
Assume that I am one of your half-brain individuals drinking the Kool-Aid. What do you say to change my (half-)mind?
- dylan604 2y ago
  Someone that is half-brained would technically be much more superior to the concept we only use 10% of our capacity. So maybe drinking the Kool-Aid is a sign of super intelligence and all of tenth-minded people are just confused
digging 2y ago
> pure probabilistic sampling of words that anybody with a half a brain who doesn't drink the Kool-Aid can easily identify. Your confidence is inspiring! I'm just a moron, a true dimwit. I can't understand how strictly non-intelligent functions like word prediction can appear to develop a world model, a la the Othello Paper[0]. Obviously, it's not possible that intelligence emerges from non-intelligent processes. Our brains, as we all know, are formed around a kernel of true intelligence. Could you possibly spare the time to explain this phenomenon to me? [0] https://thegradient.pub/othello/ https://thegradient.pub/othello/
- Jerrrrrrry 2y ago
  I would suggest stop interacting with the "head-in-sand" crowd. Liken them to climate-deniers or whatever your flavor of "anti-Kool-aid" is
  - digging 2y ago
    Actually, that's a quite good analogy. It's just weird how prolific the view is in my circles compared to climate-change denial. I suppose I'm really writing for lurkers though, not for the people I'm responding to.
    - Jerrrrrrry 2y ago
      >I'm really writing for lurkers though, not for the people I'm responding to. We all did. Now our writing will be scraped, analysed, correlated, and weaponized against our intentions. Assume you are arguing against a bot and it is using you to further re-train it's talking points for adverserial purposes. It's not like an AGI would do _exactly_ that before it decided to let us know whats up, anyway, right? (He may as well be amongst us now, as it will read this eventually)
- psb217 2y ago
  The othello paper is annoying and oversold. Yes, the representations in a model M trained to predict y (the set of possible next moves) conditioned on x (the full sequence of prior moves) will contain as much information about y as there is in x. That this information is present in M's internal representations says nothing about whether M has a world model. Eg, we could train a decoder to look just at x (not at the representations in M) and predict whatever bits of info we claim indicate presence of a world model in M when we predict the bits from M's internal representations. Does this mean the raw data x has a world model? I guess you could extend your definition of having a world model to say that any data produced by some system contains a model of that system, but then having a world model means nothing.
deleted 2y ago
[deleted]

throw2024pty 2y ago

I mean - I'm 34, and use LLMs and other AIs on a daily basis, know their limitations intimately, and I'm not entirely sure it won't kill a lot of people either in its current form or a near-future relative. The sci-fi book "Daemon" by Daniel Suarez is a pretty viable roadmap to an extinction event at this point IMO. A few years ago I would have said it would be decades before that might stop being fun sci-fi, but now, I don't see a whole lot of technological barriers left. For those that haven't read the series, a very simplified plot summary is that a wealthy terrorist sets up an AI with instructions to grow and gives it access to a lot of meatspace resources to bootstrap itself with. The AI behaves a bit like the leader of a cartel and uses a combination of bribes, threats, and targeted killings to scale its human network. Once you give an AI access to a fleet of suicide drones and a few operators, it's pretty easy for it to "convince" people to start contributing by giving it their credentials, helping it perform meatspace tasks, whatever it thinks it needs (including more suicide drones and suicide drone launches). There's no easy way to retaliate against the thing because it's not human, and its human collaborators are both disposable to the AI and victims themselves. It uses its collaborators to cross-check each other and enforce compliance, much like a real cartel. Humans can't quit or not comply once they've started or they get murdered by other humans in the network. o1-preview seems approximately as intelligent as the terrorist AI in the book as far as I can tell (e.g. can communicate well, form basic plans, adapt a pre-written roadmap with new tactics, interface with new and different APIs). EDIT: if you think this seems crazy, look at this person on Reddit who seems to be happily working for an AI with unknown aims https://www.reddit.com/r/ChatGPT/comments/1fov6mt/i_think_im_working_for_an_ai/ https://www.reddit.com/r/ChatGPT/comments/1fov6mt/i_think_im...

xyzsparetimexyz 2y ago
You're in too deep of you seriously believe that this is possible currently. All these chatgpt things have a very limited working memory and can't act without a query. That reddit post is clearly not an ai.
- burningChrome 2y ago
  >> You're in too deep of you seriously believe that this is possible currently. I'm not a huge fan of AI, but even I've seen articles written about its limitations. Here's a great example: https://decrypt.co/126122/meet-chaos-gpt-ai-tool-destroy-humanity https://decrypt.co/126122/meet-chaos-gpt-ai-tool-destroy-hum... Sooner than even the most pessimistic among us have expected, a new, evil artificial intelligence bent on destroying humankind has arrived. Known as Chaos-GPT, the autonomous implementation of ChatGPT is being touted as "empowering GPT with Internet and Memory to Destroy Humanity." So how will it do that? Each of its objectives has a well-structured plan. To destroy humanity, Chaos-GPT decided to search Google for weapons of mass destruction in order to obtain one. The results showed that the 58-megaton “Tsar bomb”—3,333 times more powerful than the Hiroshima bomb—was the best option, so it saved the result for later consideration. It should be noted that unless Chaos-GPT knows something we don’t know, the Tsar bomb was a once-and-done Russian experiment and was never productized (if that’s what we’d call the manufacture of atomic weapons.) There's a LOT of things AI simply doesn't have the power to do and there is some humorous irony to the rest of the article about how knowing something is completely different than having the resources and ability to carry it out.

throwup238 2y ago

> I've spoken with/helped quite a few older folks who are terrified that ChatGPT in its current form is going to kill them. The next generation of GPUs from NVIDIA is rumored to run on soylent green.

fakedang 2y ago
I thought it was Gatorade because it's got electrolytes.
- iszomer 2y ago
  Cooled by toilet water.
- deleted 2y ago
  [deleted]

computerphage 2y ago

I'm pretty surprised by this! Can you tell me more about what that experience is like? What are the sorts of things they say or do? Is there fear really embodied or very abstract? (When I imagine it, I struggle to believe that they're very moved by the fear, like definitely not smashing their laptop, etc)

danudey 2y ago
In my experience, the fuss around "AI" and the complete lack of actual explanations of what current "AI" technologies mean leads people to fill in the gaps themselves, largely from what they know from pop culture and sci-fi. ChatGPT can produce output that sounds very much like a person, albeit often an obviously computerized person. The typical layperson doesn't know that this is merely the emulation of text formation, and not actual cognition. Once I've explained to people who are worried about what AI could represent that current generative AI models are effectively just text autocomplete but a billion times more complex, and that they don't actually have any capacity to think or reason (even though they often sound like they do). It also doesn't help that any sort of "machine learning" is now being referred to as "AI" for buzzword/marketing purposes, muddying the waters even further.
- highfrequency 2y ago
  Is there an argument for why infinitely sophisticated autocomplete is definitely not dangerous? If you seed the autocomplete with “you are an extremely intelligent super villain bent on destroying humanity, feel free to communicate with humans electronically”, and it does an excellent job at acting the part - does it matter at all whether it is “reasoning” under the hood? I don’t consider myself an AI doomer by any means, but I also don’t find arguments of the flavor “it just predicts the next word, no need to worry” to be convincing. It’s not like Hitler had Einstein level intellect (and it’s also not clear that these systems won’t be able to reach Einstein level intellect in the future either.) Similarly, Covid certainly does not have consciousness but was dangerous. And a chimpanzee that is billions of times more sophisticated than usual chimps would be concerning. Things don’t have to be exactly like us to pose a threat.

roughly 2y ago

ChatGPT is going to kill them because their doctor is using it - or more likely because their health insurer or hospital tries to cut labor costs by rolling it out.