3 ms·

>speech produced without regard for truth People are not any different from transformer models in this regard even if they are well-intentioned. Every time the

>speech produced without regard for truth People are not any different from transformer models in this regard even if they are well-intentioned. Every time there is a physics or finance thread on HN, a lot of well meaning software engineers transformer out absolute nonsense with all of the same "confident, language that signals an intelligent author, completely wrong" qualities that we see in these AIs.

wpietri 4y ago
I get your point -- people also produce bullishit -- but people are significantly different in that they are not only capable of finding the truth but mostly do so with some regularity. I expect that even the most "confident but wrong" software engineer has areas of their life where are perfectly able to recognize the truth and correct their statements for it. Indeed, observing children, I think the feedback loop of "confidently say wrong thing and self-correct over time" is how a lot of actual learning happens. Transformer models concern me because they are entirely missing that loop.
- whatshisface 4y ago
  ChatGPT also has areas where it can recognize the truth. I agree that some people can hedge their confidence in some areas, but it's not a universal trait that everyone exhibits all the time. I think this shows we're only sometimes generally intelligent.
  - ljw1001 4y ago
    The real difference is in scale. Automations can be coordinated to produce self-affirming bullshit at a scale that drives real discussion out of view. You already see this on twitter with troll farms and primitive bots. Now it will be a tidal wave
    - whatshisface 4y ago
      It already is a tidal wave just with people.
  - wpietri 4y ago
    > ChatGPT also has areas where it can recognize the truth. Do you have some examples here?
    - whatshisface 4y ago
      ChatGPT will provide correct answers to a lot of questions, especially ones where you'd expect to find mostly correct answers in the first few Google results.

wpietri 4y ago

I get your point -- people also produce bullishit -- but people are significantly different in that they are not only capable of finding the truth but mostly do so with some regularity. I expect that even the most "confident but wrong" software engineer has areas of their life where are perfectly able to recognize the truth and correct their statements for it. Indeed, observing children, I think the feedback loop of "confidently say wrong thing and self-correct over time" is how a lot of actual learning happens. Transformer models concern me because they are entirely missing that loop.

whatshisface 4y ago
ChatGPT also has areas where it can recognize the truth. I agree that some people can hedge their confidence in some areas, but it's not a universal trait that everyone exhibits all the time. I think this shows we're only sometimes generally intelligent.
- ljw1001 4y ago
  The real difference is in scale. Automations can be coordinated to produce self-affirming bullshit at a scale that drives real discussion out of view. You already see this on twitter with troll farms and primitive bots. Now it will be a tidal wave
  - whatshisface 4y ago
    It already is a tidal wave just with people.
- wpietri 4y ago
  > ChatGPT also has areas where it can recognize the truth. Do you have some examples here?
  - whatshisface 4y ago
    ChatGPT will provide correct answers to a lot of questions, especially ones where you'd expect to find mostly correct answers in the first few Google results.