5 ms·

> As with everything GPT-3, this answer sounds very convincing! But is this actually true? I would expect the SQL query optimizer to be smart enough to optimize

> As with everything GPT-3, this answer sounds very convincing! But is this actually true? I would expect the SQL query optimizer to be smart enough to optimize the LEFT JOIN to the point where it would run efficiently. I think GPT-3’s answer to that question is actually very misleading. I really enjoy how well it gives wrong answers. The internet being polluted with GPT3-like text models is going to send us back to the time when verifying anything online as truthful is difficult, if it hasn't already.

mwint 4y ago
Maybe my next move should be to start a consulting shop: “We Fix GPT-3 Code”.
- abrax3141 4y ago
  Good luck with that. I don't think that these things are fixable. They're basically giant grammatical confabulation engines. When you want grammatically-correct random noise, great. Don't fly a plane with them, please...or at least not one I'm on!
  - trention 4y ago
    He meant fixing the output(s) not the model that produced them.
    - atq2119 4y ago
      I'm pretty sure abrax3141 understood this. Some things are just unfixable because they're "not even wrong". You basically have to start from scratch.
  - goodside 4y ago
    No method is fool-proof for GPT-3, but there are tricks that suppress its tendency to hallucinate, such as demonstrating in a k-shot prompt the possibility of false assumptions in a question. I demonstrate this here: https://mobile.twitter.com/goodside/status/1545793388871651330 https://mobile.twitter.com/goodside/status/15457933888716513... Models more advanced that GPT-3, such as LaMDA, have entire subsystems specifically dedicated to “grounding” output in truthful information. Hallucination is at least partially a solved problem, but the methods haven’t disseminated broadly yet.
    - abrax3141 4y ago
      Unless it’s reading and (in some sense) understand FDA and CDC drug approvals, ct.gov, pubmed, and NCCN and other treatment guidelines, and realizing when drugs are suddenly deprecated as well as suddenly approved, it’s going to commit mal practice. You could do all that, but then you don’t have a language model, you have a domain specific medical AI.
      - goodside 4y ago
        Not sure where medicine even entered the discussion. Obviously GPT-3 is not a doctor and shouldn’t be used as one. That’s a much harder problem than suppressing the tendency of GPT-3 to confabulate/hallucinate fictional memories.
        abrax3141 4y ago
        It’s not obvious. Medical advice is among the top searches on google - possibly the top. And is, by that very same virtue, among the first things people are going to ask what they hysterically believe to be an AI.
        goodside 4y ago
        There’s a big leap from “some layperson might misuse this for medical advice” to “code generated by GPT-3, for any purpose, cannot be reviewed and fixed by expert human programmers,” which is where this thread started. I’m not sure what you’re arguing, or if you’re even trying to support that original point.
        abrax3141 4y ago
        We should probably cut this off, but let me try to explain what I had in mind, which might be none sense, but was certainly not well explained. Too briefly, presumably we can agree that it’s even hard for experts to check and fix code created by other humans effectively. Now put this in a dangerous domain. I originally said aviation, but (confusingly) switched to medicine for reasons not worth going into, but we can replace “program” with “treatment regimen” and “expert programmer” with “specialist physician”. The fail I was reaching for is where what is hallucinated by the LLM looks really convincing (has great grammar), and is close enough to correct that it isn’t glaringly wrong - and these models are good at doing those two things! - but by virtue of the fact that they can’t reason, they aren’t actually “thinking things through” - which is why, of course, you need the programmer/specialist. But the hallucination is so good (yet wrong, per unreasoned) that users (patients) are taking the word of the model. You think that can’t happen, but we do it all the time with results from “Dr Google”. A friend who is a doctor says that patients regularly come in with inch think binders of stuff they printed off the web that he has to talk them down from. We’re headed for a near future where instead of being talked down from stuff you merely printed off, you’ll have to be talked down from an extremely convincing wrong treatment hallucinated by Dr HAL. Anyway…never mind.

mwint 4y ago

Maybe my next move should be to start a consulting shop: “We Fix GPT-3 Code”.

abrax3141 4y ago
Good luck with that. I don't think that these things are fixable. They're basically giant grammatical confabulation engines. When you want grammatically-correct random noise, great. Don't fly a plane with them, please...or at least not one I'm on!
- trention 4y ago
  He meant fixing the output(s) not the model that produced them.
  - atq2119 4y ago
    I'm pretty sure abrax3141 understood this. Some things are just unfixable because they're "not even wrong". You basically have to start from scratch.
- goodside 4y ago
  No method is fool-proof for GPT-3, but there are tricks that suppress its tendency to hallucinate, such as demonstrating in a k-shot prompt the possibility of false assumptions in a question. I demonstrate this here: https://mobile.twitter.com/goodside/status/1545793388871651330 https://mobile.twitter.com/goodside/status/15457933888716513... Models more advanced that GPT-3, such as LaMDA, have entire subsystems specifically dedicated to “grounding” output in truthful information. Hallucination is at least partially a solved problem, but the methods haven’t disseminated broadly yet.
  - abrax3141 4y ago
    Unless it’s reading and (in some sense) understand FDA and CDC drug approvals, ct.gov, pubmed, and NCCN and other treatment guidelines, and realizing when drugs are suddenly deprecated as well as suddenly approved, it’s going to commit mal practice. You could do all that, but then you don’t have a language model, you have a domain specific medical AI.
    - goodside 4y ago
      Not sure where medicine even entered the discussion. Obviously GPT-3 is not a doctor and shouldn’t be used as one. That’s a much harder problem than suppressing the tendency of GPT-3 to confabulate/hallucinate fictional memories.
      - abrax3141 4y ago
        It’s not obvious. Medical advice is among the top searches on google - possibly the top. And is, by that very same virtue, among the first things people are going to ask what they hysterically believe to be an AI.
        goodside 4y ago
        There’s a big leap from “some layperson might misuse this for medical advice” to “code generated by GPT-3, for any purpose, cannot be reviewed and fixed by expert human programmers,” which is where this thread started. I’m not sure what you’re arguing, or if you’re even trying to support that original point.
        abrax3141 4y ago
        We should probably cut this off, but let me try to explain what I had in mind, which might be none sense, but was certainly not well explained. Too briefly, presumably we can agree that it’s even hard for experts to check and fix code created by other humans effectively. Now put this in a dangerous domain. I originally said aviation, but (confusingly) switched to medicine for reasons not worth going into, but we can replace “program” with “treatment regimen” and “expert programmer” with “specialist physician”. The fail I was reaching for is where what is hallucinated by the LLM looks really convincing (has great grammar), and is close enough to correct that it isn’t glaringly wrong - and these models are good at doing those two things! - but by virtue of the fact that they can’t reason, they aren’t actually “thinking things through” - which is why, of course, you need the programmer/specialist. But the hallucination is so good (yet wrong, per unreasoned) that users (patients) are taking the word of the model. You think that can’t happen, but we do it all the time with results from “Dr Google”. A friend who is a doctor says that patients regularly come in with inch think binders of stuff they printed off the web that he has to talk them down from. We’re headed for a near future where instead of being talked down from stuff you merely printed off, you’ll have to be talked down from an extremely convincing wrong treatment hallucinated by Dr HAL. Anyway…never mind.