5 ms·

Incredible! Isn't this going to completely destroy trust in the shared Internet experience when scammers start using this technology to create fake profiles gal

Incredible! Isn't this going to completely destroy trust in the shared Internet experience when scammers start using this technology to create fake profiles galore? Entire fake conversations with potential "friends" and "dates". You're going to lose trust in everyone other than people you added on Facebook / Twitter / HN before 2022. You will never know, even after a long conversation, whether the person was real or not! The entirety of the web experience relies on the assumption that when you see a bunch of sentences, they were written by a person. This is so ubiquitous, so tacit, so universal a foundation of the experience of the shared Internet that a 90%-good-enough Turning-test-passing automated conversationalist is like a drop of poison that renders the whole lake of water unusable. We're in the early stages of this infinite-spam world.

azinman2 4y ago
It’s interesting you say incredible when I’m coming away with the opposite. It is not only overly polite, but constantly wrong yet so confident in its answers. And even when you iterate it still doesn’t get there, yet each time saying “oh you’re right” as if it finally got it.
- Digit-Al 4y ago
  I think we're all familiar with people who say something completely wrong, with full and absolute confidence that they are right :-)
- exit 4y ago
  > It is not only overly polite this is a deliberate goal of OpenAI's training, not an emergent feature of the underlying technology. > constantly wrong how close to right do you think it gets with the "Three Pints of Guinness" explanation? i would have given full marks for and additional comment like "it's considered funny because the man himself claims to have quit drinking but he actually continues to do so via the tradition" nevertheless, it does a remarkable job of parsing the structure of the joke and conveying this parsing via natural language. to me, my "additional comment" looks like just a further iteration on presenting this parsing. i would say it gets very close to "completely right".
  - matt_kantor 4y ago
    Interesting. My first thought upon reading that one was that "he says he quit drinking but is obviously currently drinking" is the first thing I'd explain about the punchline (then moving on to explain how that connects to the tradition).
  - deleted 4y ago
    [deleted]
- vijucat 4y ago
  Consumers lost $5.2 billion to phishing last year. This technology may not fool you, but there are enough folks who will be fooled by the basic conversational skills it has. Scammers can be 10x "productive" with this tech.
- madiator 4y ago
  And I am completely surprised by people who think this tech today is not good, and fail to account that it can get way better in the future.
  - azinman2 4y ago
    I’m guessing by people you mean me? I’m only talking about the state of ChatGPT as per the examples given. I’m not talking about the wider implications into the future or its other amazing capabilities.
- 13of40 4y ago
  Being right and sounding right are two important but separate skills in natural language communication. It sounds like they've nailed one of them and the other is somewhat hit and miss right now. To me it's obvious that there's no reasoning going on behind the curtain, but in a lot of domains that doesn't matter. You don't need your answering machine to opine on philosophy questions just like you don't need your mechanic to invent and machine a custom part every time you bring your car in. (Edit: Point being if you can back the chatty part with something that's actually an expert in the domain you need it to talk about, then you've got cold fusion.) I don't think it's ready for prime time yet, but the way voice recognition and synthesis, ML, and scalable computing power have been going in the past several years, we're not far off. And that's incredible.
  - azinman2 4y ago
    As you point out what’s missing is the philosophy of mind bit. It’s all just probabilistic NL models that will output _something_, whether it has any concept of rationale or not. What is absolutely astonishing is that ChatGPT can accomplish what it does with just these markov-chain-like methods. I never would have predicted it.
mhb 4y ago
No, it's great. No one should trust the "shared Internet experience". Instead of blindly believing what some random entity spews out, this makes it clear that the underpinning of a community is trust and a reader should explicitly take the source into account when evaluating information.
- lukaesch 4y ago
  I guess Elon sees this and that‘s why he is betting high on verification badges at Twitter
  - FabHK 4y ago
    ... when will the first AI get a credit card?
- spacemadness 4y ago
  I would think the constant astroturfing on sites like Reddit would have driven this concept home for people, especially on a site like HN, but reading this thread apparently it hasn’t.
alienalp 4y ago
In that case you can also replace yourself with yours and it may let you know if it thinks there may be something you may be interested in.
dclowd9901 4y ago
I’m thinking of the Silicon Valley episode where Gilfoyle writes an AI to talk to Dinesh so he’ll stop bothering him while he works and how useful that would be for my job.
tarkin2 4y ago
I’m not sure only trusting people you know in real life would be a bad thing. The internet has disconnected most from their surroundings and community. If this new AI makes people seek out and trust only those they know it may reconnect people into their physical lives. …And create a deeper distrust of technology.
visarga 4y ago
> You're going to lose trust in everyone other than people you added on Facebook / Twitter / HN before 2022 You don't know who is rewriting their replies with chatGPT.
brk 4y ago
Or you could just be ‘friends’ or whatever on social media with people you’ve actually met in person. This could be great for killing the whole ‘influencer’ economy and people posturing online for someone they’ve never met.
p-e-w 4y ago
> You will never know, even after a long conversation, whether the person was real or not! If you can't tell, why does it matter?

azinman2 4y ago

It’s interesting you say incredible when I’m coming away with the opposite. It is not only overly polite, but constantly wrong yet so confident in its answers. And even when you iterate it still doesn’t get there, yet each time saying “oh you’re right” as if it finally got it.

Digit-Al 4y ago
I think we're all familiar with people who say something completely wrong, with full and absolute confidence that they are right :-)
exit 4y ago
> It is not only overly polite this is a deliberate goal of OpenAI's training, not an emergent feature of the underlying technology. > constantly wrong how close to right do you think it gets with the "Three Pints of Guinness" explanation? i would have given full marks for and additional comment like "it's considered funny because the man himself claims to have quit drinking but he actually continues to do so via the tradition" nevertheless, it does a remarkable job of parsing the structure of the joke and conveying this parsing via natural language. to me, my "additional comment" looks like just a further iteration on presenting this parsing. i would say it gets very close to "completely right".
- matt_kantor 4y ago
  Interesting. My first thought upon reading that one was that "he says he quit drinking but is obviously currently drinking" is the first thing I'd explain about the punchline (then moving on to explain how that connects to the tradition).
- deleted 4y ago
  [deleted]
vijucat 4y ago
Consumers lost $5.2 billion to phishing last year. This technology may not fool you, but there are enough folks who will be fooled by the basic conversational skills it has. Scammers can be 10x "productive" with this tech.
madiator 4y ago
And I am completely surprised by people who think this tech today is not good, and fail to account that it can get way better in the future.
- azinman2 4y ago
  I’m guessing by people you mean me? I’m only talking about the state of ChatGPT as per the examples given. I’m not talking about the wider implications into the future or its other amazing capabilities.
13of40 4y ago
Being right and sounding right are two important but separate skills in natural language communication. It sounds like they've nailed one of them and the other is somewhat hit and miss right now. To me it's obvious that there's no reasoning going on behind the curtain, but in a lot of domains that doesn't matter. You don't need your answering machine to opine on philosophy questions just like you don't need your mechanic to invent and machine a custom part every time you bring your car in. (Edit: Point being if you can back the chatty part with something that's actually an expert in the domain you need it to talk about, then you've got cold fusion.) I don't think it's ready for prime time yet, but the way voice recognition and synthesis, ML, and scalable computing power have been going in the past several years, we're not far off. And that's incredible.
- azinman2 4y ago
  As you point out what’s missing is the philosophy of mind bit. It’s all just probabilistic NL models that will output _something_, whether it has any concept of rationale or not. What is absolutely astonishing is that ChatGPT can accomplish what it does with just these markov-chain-like methods. I never would have predicted it.

mhb 4y ago

No, it's great. No one should trust the "shared Internet experience". Instead of blindly believing what some random entity spews out, this makes it clear that the underpinning of a community is trust and a reader should explicitly take the source into account when evaluating information.

lukaesch 4y ago
I guess Elon sees this and that‘s why he is betting high on verification badges at Twitter
- FabHK 4y ago
  ... when will the first AI get a credit card?
spacemadness 4y ago
I would think the constant astroturfing on sites like Reddit would have driven this concept home for people, especially on a site like HN, but reading this thread apparently it hasn’t.

alienalp 4y ago

In that case you can also replace yourself with yours and it may let you know if it thinks there may be something you may be interested in.

dclowd9901 4y ago

I’m thinking of the Silicon Valley episode where Gilfoyle writes an AI to talk to Dinesh so he’ll stop bothering him while he works and how useful that would be for my job.

tarkin2 4y ago

I’m not sure only trusting people you know in real life would be a bad thing. The internet has disconnected most from their surroundings and community. If this new AI makes people seek out and trust only those they know it may reconnect people into their physical lives. …And create a deeper distrust of technology.

visarga 4y ago

> You're going to lose trust in everyone other than people you added on Facebook / Twitter / HN before 2022 You don't know who is rewriting their replies with chatGPT.

brk 4y ago

Or you could just be ‘friends’ or whatever on social media with people you’ve actually met in person. This could be great for killing the whole ‘influencer’ economy and people posturing online for someone they’ve never met.

p-e-w 4y ago

> You will never know, even after a long conversation, whether the person was real or not! If you can't tell, why does it matter?