2 ms·

I help answer questions on a few IRC channels. ChatGPT has made our life considerably and noticeably harder, as the knowledge that comes out of it is often blat

I help answer questions on a few IRC channels. ChatGPT has made our life considerably and noticeably harder, as the knowledge that comes out of it is often blatantly wrong, yet extremely confident. We spend a lot of time untangling it. I would not be sad one bit if all of it died tomorrow. In my view, it seems like it's more important for AI to look coherent and seem impressive than be correct. It's extremely possible it will provide net negative results if we don't get this under control. There's no clock to race against. Let's take our time and get it right.

wpietri 3y ago
Well put. I think this is especially important: > There's no clock to race against. Nothing bad is going to happen if society doesn't rush LLMs out. It's not like we're facing a shocking deficit of text.
- 8n4vidtmkvmk 3y ago
  no clock for society. some clock for companies to get ahead and make money while they can
BoorishBears 3y ago
I've used IRC for years and people being confidently wrong has always been an issue. LLMs are definitely a new concept, but not every facet of them has 0 precedent. Someone surfacing a bad source repeatedly isn't new, you label the channel so people know it's banned, and if the same person keeps surfacing bad info you mute them, then ban them if it continues. If you have a sudden influx of low quality new users it's also not new, you require authentication and that alone cuts down on a ton of spam
- nullc 3y ago
  IRC? Try HN. ... or really just about anywhere. People rarely post "I don't know", if they know they don't know they don't post. ... and you don't post you don't exist. So in any venue online the level of confidence is dramatically overstated and probably the amount of wrong answers too. Not that things would be much better for LLMs if that weren't true. LLMs that learned their "don't know" practices from text-- they'd only be right about it on things that everyone didn't know and would probably give erroneous don't know answers or things they actually do know, so it would be just another way of being incorrect. :) Suppressing out of control making stuff up is hard for human minds too. Go talk to a four or five year old. :) I've wondered if part of the reason for it is that it's something much easier learned through interaction than from observation.

wpietri 3y ago

Well put. I think this is especially important: > There's no clock to race against. Nothing bad is going to happen if society doesn't rush LLMs out. It's not like we're facing a shocking deficit of text.

8n4vidtmkvmk 3y ago
no clock for society. some clock for companies to get ahead and make money while they can

BoorishBears 3y ago

I've used IRC for years and people being confidently wrong has always been an issue. LLMs are definitely a new concept, but not every facet of them has 0 precedent. Someone surfacing a bad source repeatedly isn't new, you label the channel so people know it's banned, and if the same person keeps surfacing bad info you mute them, then ban them if it continues. If you have a sudden influx of low quality new users it's also not new, you require authentication and that alone cuts down on a ton of spam

nullc 3y ago
IRC? Try HN. ... or really just about anywhere. People rarely post "I don't know", if they know they don't know they don't post. ... and you don't post you don't exist. So in any venue online the level of confidence is dramatically overstated and probably the amount of wrong answers too. Not that things would be much better for LLMs if that weren't true. LLMs that learned their "don't know" practices from text-- they'd only be right about it on things that everyone didn't know and would probably give erroneous don't know answers or things they actually do know, so it would be just another way of being incorrect. :) Suppressing out of control making stuff up is hard for human minds too. Go talk to a four or five year old. :) I've wondered if part of the reason for it is that it's something much easier learned through interaction than from observation.