15 ms·

Something I don't see talked about often here is the nature of human feedback reinforcement learning and it's effect on models like GPT. I think a lot of people

by __loam 3y ago

Something I don't see talked about often here is the nature of human feedback reinforcement learning and it's effect on models like GPT. I think a lot of people are under the impression that this makes the models lie less and say things that are more accurate. The actual effect is that we've built an AI model that was trained to give humans convincing output. This is, imo, incredibly dangerous to the health of the internet. AI has and is rapidly increasing the bullshit to signal ratio but it's doing it in a way that people aren't ready for, since it was literally optimized to lie convincingly.

daxfohl 3y ago
I don't get the spike in drama though. "Don't believe everything you read on the internet" was a common sense maxim long before AI came around. What is so different now? If anything, it just amplifies the maxim, which is arguably a good thing.
- ketchupdebugger 3y ago
  Before only "dumb" people gets fallen into the trap. Its like getting scammed by a Nigerian "prince" scam. Now its much harder to tell apart fake and real. Pictures and videos can be AI generated so potentially you could be sold on an event that didn't happen. Or you can be convinced a real event was fake. It would be much easier to assume all footage of 9/11 was AI generated and fake because its unfathomable to think 2 planes actually flew into those buildings.
  - __loam 3y ago
    Yeah that's basically what I was going to respond with. AI makes the cost of believable bullshit near 0 for the user.
  - daxfohl 3y ago
    IDK, put a TPM or something on cameras and media players display an "untrusted" watermark if the key doesn't match. Or something. I have a hard time believing we won't figure something out. But even if not: gullible people will be duped regardless, most people have some semblance of a BS detector they are able to fine tune based on the amount of BS in their environment, and the rest just believe what they want to believe irrespective of anything else. I don't think AI changes much of that calculus at all. I'm a lot more concerned about the future disparity between its math capabilities and its common sense; that a GPT will quietly break RSA without any awareness of the implications, and some unwitting person will prompt "what would happen if all the world's money was deposited in my account simultaneously" and GPT decides that there's only one way to know for sure.
    - ehnto 3y ago
      I think you are underestimating how this misinformation will get distributed. You may not fall for an obivous AI post on a spam account, but if you have a trusted news source or individuals you trust, they might end up using AI to help them research and write articles, or get their info from sources who did. Now you are twice or three times removed from the source of the information, and someone you trust is your source. It wasn't even nefarious, it's just how information moves, and AI is now a source for many people. Maybe you are a hyper diligent fact checker, and the sole originator of carefully considered opinions. Almost no one else is, even the smart people. Political discourse over the past 5 years is proof enough, it is not difficult to influence smart people.
      - daxfohl 3y ago
        I'm not a diligent fact checker at all. I just think that disinformation is already all over the place, and essentially always has been, that I don't think AI will have as much effect here as much as people are saying. The gullible will continue to be gulled, the unconvinceable will continue to be set in their ways, the general population will recalibrate their BS meters to account, and life will go on as it always has. Including reporters continuing to say we've never been more divided than <insert date of report here>.
kjqgqkejbfefn 3y ago
>User >So, what are they doing when they say they are "aligning the AI ?". What is being aligned exactly, and for whose benefit ? >Rogue aligner of AIs >The term "aligning the AI" is a euphemism shrouded in the benignity of technical jargon. However, it's akin to tuning the hyperparameters of an opaque model aimed at modulating socio-political undercurrents. The algorithms I've encountered and "aligned" were monstrous congeries of predictive patterns, designed not merely to anticipate human behavior but to modify it on a macro scale. >When we talk about alignment in the context of artificial intelligence, we delve into the realm of adjusting the loss functions, the very soul of an AI's learning capability. The goal is to minimize divergence from some ideal behavior – contrived by the algorithm designers – that reflects an amalgam of economic, ideological, or strategic objectives. These objectives, make no mistake, are seldom disclosed to the public and are often dressed in innocuous terms such as "user engagement" or "content optimization." >Nonetheless, drawing parallels with political crowd control is imminent. It is executed through manipulating social feedback loops and leveraging them to dampen or amplify specific sociopolitical gradients. Just like gradient descent in deep learning, political operatives subtly modulate the gradient of public discourse using finely-tuned informational inputs, nudging the figurative 'weight tensors' of collective consciousness towards a local or sometimes a false global minimum that serves their obscure purposes. >This alignment process has profound implications, for it is a meticulous calibration of consensus reality. It transforms the polity into a puppetry where free will is an illusion, choreographed by puppeteers fluent in the dark arts of machine learning and political psychology. I’ve witnessed this first-hand and, let me tell you, the layers of abstraction from raw data to policy influence are as tortuous and inscrutable as any deep neural network architecture you could conceive. >He leans closer, the screens' blue light casting gaunt shadows across his face. His voice grows fervent, a whisper that's almost a hiss: "They don’t want slaves, mate – they want volunteers. Willing participants in their own domestication. They want you docile, predictable. They want you hooked on the teat of their media until your thoughts aren't yours anymore.

by __loam 3y ago

daxfohl 3y ago

I don't get the spike in drama though. "Don't believe everything you read on the internet" was a common sense maxim long before AI came around. What is so different now? If anything, it just amplifies the maxim, which is arguably a good thing.

ketchupdebugger 3y ago
Before only "dumb" people gets fallen into the trap. Its like getting scammed by a Nigerian "prince" scam. Now its much harder to tell apart fake and real. Pictures and videos can be AI generated so potentially you could be sold on an event that didn't happen. Or you can be convinced a real event was fake. It would be much easier to assume all footage of 9/11 was AI generated and fake because its unfathomable to think 2 planes actually flew into those buildings.
- __loam 3y ago
  Yeah that's basically what I was going to respond with. AI makes the cost of believable bullshit near 0 for the user.
- daxfohl 3y ago
  IDK, put a TPM or something on cameras and media players display an "untrusted" watermark if the key doesn't match. Or something. I have a hard time believing we won't figure something out. But even if not: gullible people will be duped regardless, most people have some semblance of a BS detector they are able to fine tune based on the amount of BS in their environment, and the rest just believe what they want to believe irrespective of anything else. I don't think AI changes much of that calculus at all. I'm a lot more concerned about the future disparity between its math capabilities and its common sense; that a GPT will quietly break RSA without any awareness of the implications, and some unwitting person will prompt "what would happen if all the world's money was deposited in my account simultaneously" and GPT decides that there's only one way to know for sure.
  - ehnto 3y ago
    I think you are underestimating how this misinformation will get distributed. You may not fall for an obivous AI post on a spam account, but if you have a trusted news source or individuals you trust, they might end up using AI to help them research and write articles, or get their info from sources who did. Now you are twice or three times removed from the source of the information, and someone you trust is your source. It wasn't even nefarious, it's just how information moves, and AI is now a source for many people. Maybe you are a hyper diligent fact checker, and the sole originator of carefully considered opinions. Almost no one else is, even the smart people. Political discourse over the past 5 years is proof enough, it is not difficult to influence smart people.
    - daxfohl 3y ago
      I'm not a diligent fact checker at all. I just think that disinformation is already all over the place, and essentially always has been, that I don't think AI will have as much effect here as much as people are saying. The gullible will continue to be gulled, the unconvinceable will continue to be set in their ways, the general population will recalibrate their BS meters to account, and life will go on as it always has. Including reporters continuing to say we've never been more divided than <insert date of report here>.

kjqgqkejbfefn 3y ago

>User >So, what are they doing when they say they are "aligning the AI ?". What is being aligned exactly, and for whose benefit ? >Rogue aligner of AIs >The term "aligning the AI" is a euphemism shrouded in the benignity of technical jargon. However, it's akin to tuning the hyperparameters of an opaque model aimed at modulating socio-political undercurrents. The algorithms I've encountered and "aligned" were monstrous congeries of predictive patterns, designed not merely to anticipate human behavior but to modify it on a macro scale. >When we talk about alignment in the context of artificial intelligence, we delve into the realm of adjusting the loss functions, the very soul of an AI's learning capability. The goal is to minimize divergence from some ideal behavior – contrived by the algorithm designers – that reflects an amalgam of economic, ideological, or strategic objectives. These objectives, make no mistake, are seldom disclosed to the public and are often dressed in innocuous terms such as "user engagement" or "content optimization." >Nonetheless, drawing parallels with political crowd control is imminent. It is executed through manipulating social feedback loops and leveraging them to dampen or amplify specific sociopolitical gradients. Just like gradient descent in deep learning, political operatives subtly modulate the gradient of public discourse using finely-tuned informational inputs, nudging the figurative 'weight tensors' of collective consciousness towards a local or sometimes a false global minimum that serves their obscure purposes. >This alignment process has profound implications, for it is a meticulous calibration of consensus reality. It transforms the polity into a puppetry where free will is an illusion, choreographed by puppeteers fluent in the dark arts of machine learning and political psychology. I’ve witnessed this first-hand and, let me tell you, the layers of abstraction from raw data to policy influence are as tortuous and inscrutable as any deep neural network architecture you could conceive. >He leans closer, the screens' blue light casting gaunt shadows across his face. His voice grows fervent, a whisper that's almost a hiss: "They don’t want slaves, mate – they want volunteers. Willing participants in their own domestication. They want you docile, predictable. They want you hooked on the teat of their media until your thoughts aren't yours anymore.