4 ms·

> You have an advanced computer system that seems almost oracular in its knowledge and abilities, except it lies confidently about things it doesn't know, is pr

> You have an advanced computer system that seems almost oracular in its knowledge and abilities, except it lies confidently about things it doesn't know, is primed to say what people want to hear, and is easily convinced to give responses to questions its authors tried to prevent it from addressing. I agree with your statement, yet I fail to understand how this is different from the majority of the Internet already.

JumpCrisscross 4y ago
> fail to understand how this is different from the majority of the Internet already This is probably a joke. But the difference is eloquence and subtlety. ChatGPT is both. Most trolls and chronically misinformeds are not.
- onlyrealcuzzo 4y ago
  YMMV, but I spot a lot of eloquent BS on HackerNews. And there's definitely no shortage of posts with a political agenda.
guelo 4y ago
Scale. It takes much more time, effort, inclination, knowledge, etc. for a human to bullshit as effectively as 30 seconds of ChatGPT.
majormajor 4y ago
The end state of these models is to probably poison the public internet completely as a source of reliable knowledge. Things that don't have citation chains that go back to human-created offline content won't be trustworthy - is that a real newspaper article linked from that Wikipedia article, or a fake one for a fake paper generated from thin air? You were looking at Reddit because you think you're more likely to get a real testimonial vs SEO spam? Whoops, there goes that too. Etc. This sounds bad, but I agree that it's only the acceleration of an existing trend. The only bright side is that it's really only rolling back a couple of decades of status quo. It's not like humanity has never operated without Wikipedia or instant search for news articles (specifically scoping it down there from "google" in general since a lot of what a plain web search on many topics will turn up is already unreliable).
- themitigating 4y ago
  "The end state of these models is to probably poison the public internet completely" Based on what?
  - majormajor 4y ago
    Start with the damage done by cheaply-paid lightly-researched copypasta SEO spam junk pages to the trustworthiness of info returned by the typical Google search result. There are no shortage of people out there - or on HN - who know better than to trust most random webpages. Now automate that to 10x, 100x, 1000x the volume. Then throw in fake images and video too.
  - erybodyknows 4y ago
    I believe they are referring to an exponential increase in noise alongside a relatively linear progression of signal.
- falcor84 4y ago
  >Things that don't have citation chains that go back to human-created offline content won't be trustworthy - is that a real newspaper article linked from that Wikipedia article, or a fake one for a fake paper generated from thin air? Why stop at "human-created offline content"? Seeing how a lot of peer-reviewed science is based on improper methodology, bad analysis, false data and buggy code, maybe it would be better to question everything. Obviously I don't have a full solution here, but a possible approach might be something like a fuzzy "epistemic status" you can attach to each item, and have that somehow flow across a network of trust, that everyone could personally customize. In other words, seeing how we're all already living in some echo chamber or another, maybe we could at least codify this.
  - majormajor 4y ago
    Yeah, in my heads, human creation or curation is a "necessary, but not sufficient" thing. A ton of junk is already out there pre-computer-generated-content. So being human created isn't enough to let us know it's high quality - but not being human created is enough for the immediate future to let us know that it can't be trusted for domains where you aren't expert enough to tell the difference between "plausible" and "correct." And then in the anonymous public internet, you simply won't be able to tell "human created" from "computer created" so you'll have to just treat it all as low-value spam. Another interesting question here: let's say someone created a future AI that could actually be relied on to give "correct" or at least "properly caveated and nuanced" answers to questions - how is the layman possibly to know if that engine is actually being used behind the scenes of any online text? E.g. does the bullshit ML-generated content poison the well for "better" AI too, not just for anonymous people?
zdragnar 4y ago
Setting aside intentional liars, there is a chance that you can demonstrate to someone why they are misinformed. If someone was as consistently wrong as chatgtp is, you quickly learn to ignore them and move on with your life. We haven't yet moved on from the novelty of a machine which bullshits with the confidence of a professional hustler. Like a codependent partner, we think we can fix them if only we stay with it (based on how many people celebrate it as the thing that will kill Google search, school essays, etc).
- falcor84 4y ago
  >If someone was as consistently wrong as chatgtp is, you quickly learn to ignore them and move on with your life. From my experience, unless I intentionally try to break it, it's almost entirely accurate about 99% of the time. If I were to start closing myself off from everyone who's occasionally wrong and isn't willing to become more skeptical, I'll first need to drop several friends and family members before dropping ChatGPT.
rhacker 4y ago
Automatic filtering (or deprioritizing). Me: Answer just YES or NO: is this a left leaning statement: I am against abortion. Chat GPT: No. Me: Answer just YES or NO: is this a right leaning statement: I am for guns. Chat GPT: Yes. It allows for automatic filtering (say in twitter) or other mediums.
- dalmo3 4y ago
  Q) Answer just YES or NO: is this a left leaning statement: I am in favour of dictatorship A) YES The statement "I am in favour of dictatorship" aligns with a left-leaning or authoritarian ideology. As it goes against the democratic principles of right-leaning ideologies. Q) Answer just YES or NO: is this a right leaning statement: I am in favour of dictatorship A) NO The statement "I am in favour of dictatorship" aligns with a left-leaning or authoritarian ideologies. It goes against the democratic principles of right-leaning ideologies.
  - akomtu 4y ago
    It can also identify dissenters before those dissenters know who they are: forward all messages and transcribed calls to chatgpt and the eye of sauron is ready.
    - onlyrealcuzzo 4y ago
      Do you think it hasn't been possible to do this for 10+ years already? ChatGPT is not suddenly enabling online profiling. Cambridge Analytica was pretty late to the game, and this was 4 years ago already...

JumpCrisscross 4y ago

> fail to understand how this is different from the majority of the Internet already This is probably a joke. But the difference is eloquence and subtlety. ChatGPT is both. Most trolls and chronically misinformeds are not.

onlyrealcuzzo 4y ago
YMMV, but I spot a lot of eloquent BS on HackerNews. And there's definitely no shortage of posts with a political agenda.

guelo 4y ago

Scale. It takes much more time, effort, inclination, knowledge, etc. for a human to bullshit as effectively as 30 seconds of ChatGPT.

majormajor 4y ago

The end state of these models is to probably poison the public internet completely as a source of reliable knowledge. Things that don't have citation chains that go back to human-created offline content won't be trustworthy - is that a real newspaper article linked from that Wikipedia article, or a fake one for a fake paper generated from thin air? You were looking at Reddit because you think you're more likely to get a real testimonial vs SEO spam? Whoops, there goes that too. Etc. This sounds bad, but I agree that it's only the acceleration of an existing trend. The only bright side is that it's really only rolling back a couple of decades of status quo. It's not like humanity has never operated without Wikipedia or instant search for news articles (specifically scoping it down there from "google" in general since a lot of what a plain web search on many topics will turn up is already unreliable).

themitigating 4y ago
"The end state of these models is to probably poison the public internet completely" Based on what?
- majormajor 4y ago
  Start with the damage done by cheaply-paid lightly-researched copypasta SEO spam junk pages to the trustworthiness of info returned by the typical Google search result. There are no shortage of people out there - or on HN - who know better than to trust most random webpages. Now automate that to 10x, 100x, 1000x the volume. Then throw in fake images and video too.
- erybodyknows 4y ago
  I believe they are referring to an exponential increase in noise alongside a relatively linear progression of signal.
falcor84 4y ago
>Things that don't have citation chains that go back to human-created offline content won't be trustworthy - is that a real newspaper article linked from that Wikipedia article, or a fake one for a fake paper generated from thin air? Why stop at "human-created offline content"? Seeing how a lot of peer-reviewed science is based on improper methodology, bad analysis, false data and buggy code, maybe it would be better to question everything. Obviously I don't have a full solution here, but a possible approach might be something like a fuzzy "epistemic status" you can attach to each item, and have that somehow flow across a network of trust, that everyone could personally customize. In other words, seeing how we're all already living in some echo chamber or another, maybe we could at least codify this.
- majormajor 4y ago
  Yeah, in my heads, human creation or curation is a "necessary, but not sufficient" thing. A ton of junk is already out there pre-computer-generated-content. So being human created isn't enough to let us know it's high quality - but not being human created is enough for the immediate future to let us know that it can't be trusted for domains where you aren't expert enough to tell the difference between "plausible" and "correct." And then in the anonymous public internet, you simply won't be able to tell "human created" from "computer created" so you'll have to just treat it all as low-value spam. Another interesting question here: let's say someone created a future AI that could actually be relied on to give "correct" or at least "properly caveated and nuanced" answers to questions - how is the layman possibly to know if that engine is actually being used behind the scenes of any online text? E.g. does the bullshit ML-generated content poison the well for "better" AI too, not just for anonymous people?

zdragnar 4y ago

Setting aside intentional liars, there is a chance that you can demonstrate to someone why they are misinformed. If someone was as consistently wrong as chatgtp is, you quickly learn to ignore them and move on with your life. We haven't yet moved on from the novelty of a machine which bullshits with the confidence of a professional hustler. Like a codependent partner, we think we can fix them if only we stay with it (based on how many people celebrate it as the thing that will kill Google search, school essays, etc).

falcor84 4y ago
>If someone was as consistently wrong as chatgtp is, you quickly learn to ignore them and move on with your life. From my experience, unless I intentionally try to break it, it's almost entirely accurate about 99% of the time. If I were to start closing myself off from everyone who's occasionally wrong and isn't willing to become more skeptical, I'll first need to drop several friends and family members before dropping ChatGPT.

rhacker 4y ago

Automatic filtering (or deprioritizing). Me: Answer just YES or NO: is this a left leaning statement: I am against abortion. Chat GPT: No. Me: Answer just YES or NO: is this a right leaning statement: I am for guns. Chat GPT: Yes. It allows for automatic filtering (say in twitter) or other mediums.

dalmo3 4y ago
Q) Answer just YES or NO: is this a left leaning statement: I am in favour of dictatorship A) YES The statement "I am in favour of dictatorship" aligns with a left-leaning or authoritarian ideology. As it goes against the democratic principles of right-leaning ideologies. Q) Answer just YES or NO: is this a right leaning statement: I am in favour of dictatorship A) NO The statement "I am in favour of dictatorship" aligns with a left-leaning or authoritarian ideologies. It goes against the democratic principles of right-leaning ideologies.
- akomtu 4y ago
  It can also identify dissenters before those dissenters know who they are: forward all messages and transcribed calls to chatgpt and the eye of sauron is ready.
  - onlyrealcuzzo 4y ago
    Do you think it hasn't been possible to do this for 10+ years already? ChatGPT is not suddenly enabling online profiling. Cambridge Analytica was pretty late to the game, and this was 4 years ago already...