2 ms·

The bullshitting in "writing" is one thing, but these things are indeed scary: "The researchers showed that GTP-4 ADA “created a seemingly authentic database,”

The bullshitting in "writing" is one thing, but these things are indeed scary: "The researchers showed that GTP-4 ADA “created a seemingly authentic database,” that supported the conclusion that one eye treatment was superior to another. In other words, it had created a fake dataset to support a preordained conclusion. This experiment raises the threat that large language models like GTP-4 ADA could be used to “fabricate data sets specifically designed to quickly produce false scientific evidence.” Which means that AI could be used to produce fake data to support whatever conclusion or product you want to promote." Note also that faking data is not only about science. I'm pretty sure we're going to see fake datasets for fake market analyses, fake polling data for fake political campaigns, fake data from fake data breaches, etc., etc.

verdverm 3y ago
> Which means that AI could be used to produce fake data to support... There is an interesting dichotomy here. This case is indeed not ideal, but many developers would be happy if an AI could create the fake data for testing that is representative of real data. It's not the tool, but how it is used, or by whom it is used?
- 4death4 3y ago
  Of course the issue is the user not the tool. But, there is no shortage of assholes in the world. So if something can be used for low-effort, high reward unethical purposes, it’s very likely to happen and is something that should be considered.
  - verdverm 3y ago
    but you don't want to hamper the normal user just because of the asshole potential We're seeing something proposed in the US around banning contributions to RiscV because it is open source and "our enemies" might benefit and avoid tech sanctions. This seems like another bad take to further isolationism in the US
- jruohonen 3y ago
  > This case is indeed not ideal, but many developers would be happy if an AI could create the fake data for testing that is representative of real data. I agree: synthetic data has many well-intentional use cases. It may also remedy privacy issues, among other things. And you can easily enough generate synthetic data without AI, but the issue, I think, is that AI makes the generation easy also for laypeople, and I suppose you can prompt LLMs to generate custom "high-quality" fake data that no data sleuthing can detect.
AnimalMuppet 3y ago
The postmodernists (so far as I understand them) say that we are post truth - that all claims of truth are actually assertions of power. I always thought that was bunk. Now it looks more like prophecy. How does society function when nobody can know if anything they think they know is actually the truth? And not just in the "deep epistemological thoughts" way. Even in the way that we used to be able to know things, with less than total epistemic certainty, but still with sufficient certainty to usefully be able to use information... we can't do even that any more.
- at_a_remove 3y ago
  I've alluded to this before, by I had an unfinished sci-fi novel in the decades past where this was the backdrop, that at one point faking everything that comes over the wire became trivially easy. My prediction -- and I have no idea how true it would be -- is that once this became really prevalent and media outlets just used this to keep the 24/7 news cycle going, election results were faked, dead celebrities were prancing around commercials, seamless insertion of just about anything into your favorite old movies, that we would collectively have a Reality Crisis. Imagine if some percentage of people who talked about the Mandela Effect were right and that number kept growing. Imagine if we could hide a Tuskegee or two for a little while longer before it came out. You would get a huge amount of digital distrust, and some percentage of people would retreat to "If it wasn't printed in a book before 2020 ..." as a standard. Wild paranoia would emerge and others would retreat into a kind of indifference because "Who knows?" Previous conspiracy theories, suck as a faked moon landing, might become plausible just due to how many other wild fakes existed. How would you know who really won an election?
  - Towaway69 3y ago
    The good thing about the Internet is that no one knows you're a ~~dog~~ AI.
    - antifa 3y ago
      Every website will require your phone number and captchas to verify your humanity, but somehow none of them will do a good job keeping their content authenticly human.
qdw 3y ago
This made me think of Dirk Gently's Holistic Detective Agency, Douglas Adams's 1987 novel, where there's an AI company that takes military and political decisions and comes up with post hoc rationalizations for them.

verdverm 3y ago

> Which means that AI could be used to produce fake data to support... There is an interesting dichotomy here. This case is indeed not ideal, but many developers would be happy if an AI could create the fake data for testing that is representative of real data. It's not the tool, but how it is used, or by whom it is used?

4death4 3y ago
Of course the issue is the user not the tool. But, there is no shortage of assholes in the world. So if something can be used for low-effort, high reward unethical purposes, it’s very likely to happen and is something that should be considered.
- verdverm 3y ago
  but you don't want to hamper the normal user just because of the asshole potential We're seeing something proposed in the US around banning contributions to RiscV because it is open source and "our enemies" might benefit and avoid tech sanctions. This seems like another bad take to further isolationism in the US
jruohonen 3y ago
> This case is indeed not ideal, but many developers would be happy if an AI could create the fake data for testing that is representative of real data. I agree: synthetic data has many well-intentional use cases. It may also remedy privacy issues, among other things. And you can easily enough generate synthetic data without AI, but the issue, I think, is that AI makes the generation easy also for laypeople, and I suppose you can prompt LLMs to generate custom "high-quality" fake data that no data sleuthing can detect.

AnimalMuppet 3y ago

The postmodernists (so far as I understand them) say that we are post truth - that all claims of truth are actually assertions of power. I always thought that was bunk. Now it looks more like prophecy. How does society function when nobody can know if anything they think they know is actually the truth? And not just in the "deep epistemological thoughts" way. Even in the way that we used to be able to know things, with less than total epistemic certainty, but still with sufficient certainty to usefully be able to use information... we can't do even that any more.

at_a_remove 3y ago
I've alluded to this before, by I had an unfinished sci-fi novel in the decades past where this was the backdrop, that at one point faking everything that comes over the wire became trivially easy. My prediction -- and I have no idea how true it would be -- is that once this became really prevalent and media outlets just used this to keep the 24/7 news cycle going, election results were faked, dead celebrities were prancing around commercials, seamless insertion of just about anything into your favorite old movies, that we would collectively have a Reality Crisis. Imagine if some percentage of people who talked about the Mandela Effect were right and that number kept growing. Imagine if we could hide a Tuskegee or two for a little while longer before it came out. You would get a huge amount of digital distrust, and some percentage of people would retreat to "If it wasn't printed in a book before 2020 ..." as a standard. Wild paranoia would emerge and others would retreat into a kind of indifference because "Who knows?" Previous conspiracy theories, suck as a faked moon landing, might become plausible just due to how many other wild fakes existed. How would you know who really won an election?
- Towaway69 3y ago
  The good thing about the Internet is that no one knows you're a ~~dog~~ AI.
  - antifa 3y ago
    Every website will require your phone number and captchas to verify your humanity, but somehow none of them will do a good job keeping their content authenticly human.

qdw 3y ago

This made me think of Dirk Gently's Holistic Detective Agency, Douglas Adams's 1987 novel, where there's an AI company that takes military and political decisions and comes up with post hoc rationalizations for them.