4 ms·

> If it produces plausible articles or computer code it means the inevitable hallucinations are becoming harder to spot. How functional does the code have to b

> If it produces plausible articles or computer code it means the inevitable hallucinations are becoming harder to spot. How functional does the code have to be before it's no longer a hallucination? What if we are all not much more than only drawing "on the (admittedly vast) proportion of [our experiences] ingested at training time?

eropple 4y ago
> How functional does the code have to be before it's no longer a hallucination? At all, preferably. Hallucinating `the_hard_bit()` from a library that doesn't exist isn't particularly useful. (That said, I do use GitHub Copilot, because when it's looking at actually related stuff, that's pretty good. Should we just hand an unbounded search and ingest to ChatGPT? Probably not!)
- WalterSear 4y ago
  It doesn't always do that: it frequently outputs working code.
scarecrowbob 4y ago
"How functional does the code have to be before it's no longer a hallucination?" Well, at the point when it works but I can't understand it well enough to verify that it is correct, we may have some issues. I know someone who has been using it to write complex regexs. To mutate the joke, to me it just sounds like now they have three problems.
- WalterSear 4y ago
  So either write tests, or ask GPT to rewrite the code sp that you can understand it, then write tests.
deleted 4y ago
[deleted]
wpietri 4y ago
> How functional does the code have to be before it's no longer a hallucination? From LLMs, at least, I expect it will always be a hallucination. Code is never the point. Code is the working medium by which people solve problems for other people. One way to see this is to realize that code bases on their own are generally worthless in the sense that people rarely pay much money for them. They pay for users. They pay for teams. But they don't generally pay for raw code. Another way to look at it: imagine that a manager looks at ChatGPT and says, "At last, I can fire all the programmers. Fuck those guys." They set out to build an app. How long do you think it will take before they are forced to admit defeat and hire somebody who can read and edit the code? Even if you think they make it all the way to a revenue-generating product, the manager will have become a prompt engineer, creating a large mass of interrelated prompts that are used to make code that makes the app. We have not eliminated the programmer; we have turned a manager into a programmer who has been forced to discover a new programming language. One that is clearly more English-like, but it also lacking in precision and, on current evidence, is much harder to use. But I think the more likely outcome is that they will need actual programmers pretty quickly. That at best they will have speeded the creation of some more or less generic code. Which is exactly what we saw with the code-generation wizards of earlier eras: you got a fast initial result as long as it was pretty standard. But then you were generally worse off, because you had a bunch of only semi-coherent code that somebody had to understand before they could do novel or difficult things.
- WalterSear 4y ago
  > Another way to look at it: imagine that a manager looks at ChatGPT and says, "At last, I can fire all the programmers. Fuck those guys." They set out to build an app. How long do you think it will take before they are forced to admit defeat and hire somebody who can read and edit the code? How is this different from "At last, I can fire all [those senior programmers and hire junior ones, and keep a team lead around, to keep them on track]. Fuck those guys."?
  - wpietri 4y ago
    Maybe I'm not getting the point of your question. It's different in that junior programmers are still people, capable of much more than LLMs are. But yes, it's the same shortsighted executive intent.
snickerbockers 4y ago
>How functional does the code have to be before it's no longer a hallucination? IDK, I haven't tried copilot, but does the code it generates work on the first try with no human intervention? >What if we are all not much more than only drawing "on the (admittedly vast) proportion of [our experiences] ingested at training time? ugh, this argument again. Despite the machine learning community using words such as "training" and "learning" to describe the way they tune parameters, it has never been proven that any existing AI resembles human cognition. This is something that needs to be demonstrated empirically.
- WalterSear 4y ago
  > does the code it generates work on the first try with no human intervention? Sometimes, it certainly does. Does your code always work on the first try with no intervention?
  - snickerbockers 4y ago
    Not always on the first time, but when there's a problem I don't need somebody else to intervene and make it work for me.
    - WalterSear 4y ago
      I've seen at least one demonstration where the user re-fed problematic generated text into chat-GPT and successfully asked it to find the problem.
      - snickerbockers 4y ago
        I'm sorry, who did you say re-fed problematic generated text into chat-GPT and successfully asked it to find the problem?
somenameforme 4y ago
Imagine yourself, for a minute, 50,000 years ago in the past. By many estimates this was just before humans had developed the ability for normal speech. The bleeding edge of technology is "poke him with the pointy end" and the epitome of human speech and literature is *angry grunting sound*. Even if somehow one was able to insert the entire quantifiable knowledge of humanity into an e.g. ChatGPT style system you're not really going to get anything more out of it than what you put into it. Ask the computer for how to develop nuclear energy and you're going to get back *confused grunting sound*. You're not really going to get from there (as a machine) to man on the moon, Shakespeare, and nuclear energy through anything like a normal recombination of what's already known. Yet, somehow, humanity did. And extremely fast. The time frame from then to now is but 1000-2000 human generations. And that with endless war, fallible memory, dark ages, knowledge being lost (or burnt), and so on endlessly. A "intelligent" computer system, without such flaws, ought be able to replicate our progress in a negligible amount of time. But whatever technology this may be that I'm appealing to, I don't really see it on our current path with natural language search/recombination.
grogenaut 4y ago
I burned an hour with chat gpt insisting an AddOrUpdate function existed in microsoft entity framework. When I called bullshit it hallucinated that another library contained it. Then it hallucinated versions... Then I gave up... A week later I noticed that Update in the new version does an Upsert... by reading the f*cking docs... google also didn't know this answer nor did SO. Thius has been my experience with ChatGPT and code,it hallucinates a lot of stuff.

eropple 4y ago

> How functional does the code have to be before it's no longer a hallucination? At all, preferably. Hallucinating `the_hard_bit()` from a library that doesn't exist isn't particularly useful. (That said, I do use GitHub Copilot, because when it's looking at actually related stuff, that's pretty good. Should we just hand an unbounded search and ingest to ChatGPT? Probably not!)

WalterSear 4y ago
It doesn't always do that: it frequently outputs working code.

scarecrowbob 4y ago

"How functional does the code have to be before it's no longer a hallucination?" Well, at the point when it works but I can't understand it well enough to verify that it is correct, we may have some issues. I know someone who has been using it to write complex regexs. To mutate the joke, to me it just sounds like now they have three problems.

WalterSear 4y ago
So either write tests, or ask GPT to rewrite the code sp that you can understand it, then write tests.

deleted 4y ago

[deleted]

wpietri 4y ago

> How functional does the code have to be before it's no longer a hallucination? From LLMs, at least, I expect it will always be a hallucination. Code is never the point. Code is the working medium by which people solve problems for other people. One way to see this is to realize that code bases on their own are generally worthless in the sense that people rarely pay much money for them. They pay for users. They pay for teams. But they don't generally pay for raw code. Another way to look at it: imagine that a manager looks at ChatGPT and says, "At last, I can fire all the programmers. Fuck those guys." They set out to build an app. How long do you think it will take before they are forced to admit defeat and hire somebody who can read and edit the code? Even if you think they make it all the way to a revenue-generating product, the manager will have become a prompt engineer, creating a large mass of interrelated prompts that are used to make code that makes the app. We have not eliminated the programmer; we have turned a manager into a programmer who has been forced to discover a new programming language. One that is clearly more English-like, but it also lacking in precision and, on current evidence, is much harder to use. But I think the more likely outcome is that they will need actual programmers pretty quickly. That at best they will have speeded the creation of some more or less generic code. Which is exactly what we saw with the code-generation wizards of earlier eras: you got a fast initial result as long as it was pretty standard. But then you were generally worse off, because you had a bunch of only semi-coherent code that somebody had to understand before they could do novel or difficult things.

WalterSear 4y ago
> Another way to look at it: imagine that a manager looks at ChatGPT and says, "At last, I can fire all the programmers. Fuck those guys." They set out to build an app. How long do you think it will take before they are forced to admit defeat and hire somebody who can read and edit the code? How is this different from "At last, I can fire all [those senior programmers and hire junior ones, and keep a team lead around, to keep them on track]. Fuck those guys."?
- wpietri 4y ago
  Maybe I'm not getting the point of your question. It's different in that junior programmers are still people, capable of much more than LLMs are. But yes, it's the same shortsighted executive intent.

snickerbockers 4y ago

>How functional does the code have to be before it's no longer a hallucination? IDK, I haven't tried copilot, but does the code it generates work on the first try with no human intervention? >What if we are all not much more than only drawing "on the (admittedly vast) proportion of [our experiences] ingested at training time? ugh, this argument again. Despite the machine learning community using words such as "training" and "learning" to describe the way they tune parameters, it has never been proven that any existing AI resembles human cognition. This is something that needs to be demonstrated empirically.

WalterSear 4y ago
> does the code it generates work on the first try with no human intervention? Sometimes, it certainly does. Does your code always work on the first try with no intervention?
- snickerbockers 4y ago
  Not always on the first time, but when there's a problem I don't need somebody else to intervene and make it work for me.
  - WalterSear 4y ago
    I've seen at least one demonstration where the user re-fed problematic generated text into chat-GPT and successfully asked it to find the problem.
    - snickerbockers 4y ago
      I'm sorry, who did you say re-fed problematic generated text into chat-GPT and successfully asked it to find the problem?

somenameforme 4y ago

Imagine yourself, for a minute, 50,000 years ago in the past. By many estimates this was just before humans had developed the ability for normal speech. The bleeding edge of technology is "poke him with the pointy end" and the epitome of human speech and literature is *angry grunting sound*. Even if somehow one was able to insert the entire quantifiable knowledge of humanity into an e.g. ChatGPT style system you're not really going to get anything more out of it than what you put into it. Ask the computer for how to develop nuclear energy and you're going to get back *confused grunting sound*. You're not really going to get from there (as a machine) to man on the moon, Shakespeare, and nuclear energy through anything like a normal recombination of what's already known. Yet, somehow, humanity did. And extremely fast. The time frame from then to now is but 1000-2000 human generations. And that with endless war, fallible memory, dark ages, knowledge being lost (or burnt), and so on endlessly. A "intelligent" computer system, without such flaws, ought be able to replicate our progress in a negligible amount of time. But whatever technology this may be that I'm appealing to, I don't really see it on our current path with natural language search/recombination.

grogenaut 4y ago

I burned an hour with chat gpt insisting an AddOrUpdate function existed in microsoft entity framework. When I called bullshit it hallucinated that another library contained it. Then it hallucinated versions... Then I gave up... A week later I noticed that Update in the new version does an Upsert... by reading the f*cking docs... google also didn't know this answer nor did SO. Thius has been my experience with ChatGPT and code,it hallucinates a lot of stuff.