4 ms·

Do we really lack a good understanding of LLM's and deep nets that we need to be afraid of them? I would love to see this disproved with some open source work o

Do we really lack a good understanding of LLM's and deep nets that we need to be afraid of them? I would love to see this disproved with some open source work on the internals of these models and how they do inference and exactly reason. And why they could possibly never realize an AGI with the current stream of models. Being able to display human level intelligence and creative in confined spaces (be it Chess or Go based models) is something we have been progressing on for a bit - now that the same is applied to writing, image or audio / speech generation we suddenly start developing a fear of AGI. Is there a phrase for the fear of AI now building up?

ethanbond 3y ago
No one has ever been convinced to fly an airplane into a building by a chess board. People have gotten convinced of that by words though. The “attack surface” on human sensibility is just enormous if you’re able to use believably human language.
- jurgenaut23 3y ago
  > The “attack surface” on human sensibility is just enormous if you’re able to use believably human language. That's a fantastic quote ;-)
- EA-3167 3y ago
  Well... words and decades of circumstances. If you removed the circumstances (the religion, the conflict, the money, geography, etc) then the words would be absolutely hollow. I think we tend to credit words where often circumstances are doing the heavy lifting. For example try to start a riot with words in Rodeo Drive. Now try to do it in Nanterre. Or better yet, try to start a riot in Nanterre before a 17 year was shot by police, vs. after. You'll get a sense of just how valuable your words really are.
  - mistermann 3y ago
    > Well... words and decades of circumstances. That also can be modified with words though (but for both good and bad). Unfortunately, those with expertise in this domain may not have all of our best interests at heart. > If you removed the circumstances (the religion, the conflict, the money, geography, etc) then the words would be absolutely hollow. There's also the problem of non-religious faith based belief.
  - chairhairair 3y ago
    Many people are in desperate situations. Most of them are not convinced to do much about it. Words can aim discontent. Were the economic conditions in Weimar Germany that much worse than many places today?
    - submeta 3y ago
      Reminds me of the idea of a "tipping point." When we hit this point, words can really get people moving. This has been true for big changes like revolutions and movements, like Black Lives Matter, #MeToo, or Fridays for Future. Words might not do much without the right situation, like the parent mentioned with Rodeo Drive and Nanterre. But they're still important. They can guide people's anger and unhappiness. In the case of Weimar Germany, the severe economic instability and social discontent following World War I created a fertile ground for radical ideologies to take root. When these conditions coincided with persuasive rhetoric, it catalyzed significant societal change. So, while words can indeed be powerful, they're often most effective when spoken into pre-existing circumstances of tension or dissatisfaction. They can then direct this latent energy towards a specific course of action or change.
  - ethanbond 3y ago
    There were plenty of well-off people who flew to Syria to go behead other people. Anyway this doesn't matter that much. Sure, you can imagine a world totally different from ours where there would be zero differential risk between a chess-playing computer and a language-speaking computer. But we live in this world, and the risk profile is not the same.
  - Zigurd 3y ago
    Quite so, which is why retrospective analysis like "The CIA helped start The Paris Review and that made literature friendly to neoliberal ideology" are confections of confirmation bias. Nothing is ever that pat. But tidy little conspiracies are also never the goal. A nudge is both all that is realistic to aim for and a few successes are all you need to shift public perception. Arming every ambitious cult leader wannabe from some retrograde backwater with an information war WMD deserves some caution.
  - TheOtherHobbes 3y ago
    The circumstances are very active, and AI has real potential to act as a force multiplier for them. We've already seen this with troll farms and campaigns of seeded insanity like the Q Cult. Existing AI tools can make similar efforts cheaper and more effective, and future AI tools are likely to be even more powerful. There's a huge difference between how an experienced technical researcher sees AI and how a politician, war lord, or dark media baron sees it.
- jrflowers 3y ago
  Nobody has been convinced to fly an airplane into a building by a language model either.
  - ethanbond 3y ago
    "This might be a risk" "No it's not, it hasn't materialized" Risk: Possibility of loss or injury Risks, definitionally, are things that have not happened yet.
    - hackinthebochs 3y ago
      Seeing how confidently and obtusely people dismiss the risks of AI makes me so much more fearful of what's to come. Human hubris knows no limits.
      - jrflowers 3y ago
        It’s interesting how confidently and obtusely people will proclaim categorical knowledge of the future. It is a little disconcerting that there is a fight between two somewhat cultish sects when it comes to language models. Both sides call them “artificial intelligence”, one side says they’ll save the world, the other side says they’ll end it. There is very little room to even question “Is this actually AI that we’re looking at?” when loudest voices on the subject are VC tech bros and a Harry Potter fan fiction author that’s convinced people that he is prescient.
        hackinthebochs 3y ago
        Trying to impute the motives of ones interlocutor is dumb and boring. How about we discuss the first-order issue instead. Here's my argument for why x-risk is a real possibility: The issue is that small misalignments in objectives can have outsized real-world effects. Optimizers are constrained by rules and computational resources. General intelligence allows an optimizer to find efficient solutions to computational problems, thus maximizing the utility of available computational resources. The rules constrain its behavior such that on net it ideally provides sufficient value to us above what it destroys. But misalignment in objectives provides an avenue by which the AGI can on net destroy value despite our best efforts. Can you be sure you can provide loophole-free objectives that ensures only value-producing behavior from the human perspective? Can you prove that the ratio of value created to value lost due to misalignment is always above some suitable threshold? Can you prove that the range of value destruction is bounded so that if it does go off the rails, its damage is limited? Until we do, x-risk should be the default assumption. What say you?
tempodox 3y ago
I'm not afraid of LLMs, they are stupid machines. I'm afraid of what humans will use them for, with serious, irreversible real-life consequences.

ethanbond 3y ago

No one has ever been convinced to fly an airplane into a building by a chess board. People have gotten convinced of that by words though. The “attack surface” on human sensibility is just enormous if you’re able to use believably human language.

jurgenaut23 3y ago
> The “attack surface” on human sensibility is just enormous if you’re able to use believably human language. That's a fantastic quote ;-)
EA-3167 3y ago
Well... words and decades of circumstances. If you removed the circumstances (the religion, the conflict, the money, geography, etc) then the words would be absolutely hollow. I think we tend to credit words where often circumstances are doing the heavy lifting. For example try to start a riot with words in Rodeo Drive. Now try to do it in Nanterre. Or better yet, try to start a riot in Nanterre before a 17 year was shot by police, vs. after. You'll get a sense of just how valuable your words really are.
- mistermann 3y ago
  > Well... words and decades of circumstances. That also can be modified with words though (but for both good and bad). Unfortunately, those with expertise in this domain may not have all of our best interests at heart. > If you removed the circumstances (the religion, the conflict, the money, geography, etc) then the words would be absolutely hollow. There's also the problem of non-religious faith based belief.
- chairhairair 3y ago
  Many people are in desperate situations. Most of them are not convinced to do much about it. Words can aim discontent. Were the economic conditions in Weimar Germany that much worse than many places today?
  - submeta 3y ago
    Reminds me of the idea of a "tipping point." When we hit this point, words can really get people moving. This has been true for big changes like revolutions and movements, like Black Lives Matter, #MeToo, or Fridays for Future. Words might not do much without the right situation, like the parent mentioned with Rodeo Drive and Nanterre. But they're still important. They can guide people's anger and unhappiness. In the case of Weimar Germany, the severe economic instability and social discontent following World War I created a fertile ground for radical ideologies to take root. When these conditions coincided with persuasive rhetoric, it catalyzed significant societal change. So, while words can indeed be powerful, they're often most effective when spoken into pre-existing circumstances of tension or dissatisfaction. They can then direct this latent energy towards a specific course of action or change.
- ethanbond 3y ago
  There were plenty of well-off people who flew to Syria to go behead other people. Anyway this doesn't matter that much. Sure, you can imagine a world totally different from ours where there would be zero differential risk between a chess-playing computer and a language-speaking computer. But we live in this world, and the risk profile is not the same.
- Zigurd 3y ago
  Quite so, which is why retrospective analysis like "The CIA helped start The Paris Review and that made literature friendly to neoliberal ideology" are confections of confirmation bias. Nothing is ever that pat. But tidy little conspiracies are also never the goal. A nudge is both all that is realistic to aim for and a few successes are all you need to shift public perception. Arming every ambitious cult leader wannabe from some retrograde backwater with an information war WMD deserves some caution.
- TheOtherHobbes 3y ago
  The circumstances are very active, and AI has real potential to act as a force multiplier for them. We've already seen this with troll farms and campaigns of seeded insanity like the Q Cult. Existing AI tools can make similar efforts cheaper and more effective, and future AI tools are likely to be even more powerful. There's a huge difference between how an experienced technical researcher sees AI and how a politician, war lord, or dark media baron sees it.
jrflowers 3y ago
Nobody has been convinced to fly an airplane into a building by a language model either.
- ethanbond 3y ago
  "This might be a risk" "No it's not, it hasn't materialized" Risk: Possibility of loss or injury Risks, definitionally, are things that have not happened yet.
  - hackinthebochs 3y ago
    Seeing how confidently and obtusely people dismiss the risks of AI makes me so much more fearful of what's to come. Human hubris knows no limits.
    - jrflowers 3y ago
      It’s interesting how confidently and obtusely people will proclaim categorical knowledge of the future. It is a little disconcerting that there is a fight between two somewhat cultish sects when it comes to language models. Both sides call them “artificial intelligence”, one side says they’ll save the world, the other side says they’ll end it. There is very little room to even question “Is this actually AI that we’re looking at?” when loudest voices on the subject are VC tech bros and a Harry Potter fan fiction author that’s convinced people that he is prescient.
      - hackinthebochs 3y ago
        Trying to impute the motives of ones interlocutor is dumb and boring. How about we discuss the first-order issue instead. Here's my argument for why x-risk is a real possibility: The issue is that small misalignments in objectives can have outsized real-world effects. Optimizers are constrained by rules and computational resources. General intelligence allows an optimizer to find efficient solutions to computational problems, thus maximizing the utility of available computational resources. The rules constrain its behavior such that on net it ideally provides sufficient value to us above what it destroys. But misalignment in objectives provides an avenue by which the AGI can on net destroy value despite our best efforts. Can you be sure you can provide loophole-free objectives that ensures only value-producing behavior from the human perspective? Can you prove that the ratio of value created to value lost due to misalignment is always above some suitable threshold? Can you prove that the range of value destruction is bounded so that if it does go off the rails, its damage is limited? Until we do, x-risk should be the default assumption. What say you?

tempodox 3y ago

I'm not afraid of LLMs, they are stupid machines. I'm afraid of what humans will use them for, with serious, irreversible real-life consequences.