4 ms·

I mention this all the time. I wanted GPT to give me an NGINX config for Active Collab as we're previously using Apache, and thus an htaccess file. I fed it al

I mention this all the time. I wanted GPT to give me an NGINX config for Active Collab as we're previously using Apache, and thus an htaccess file. I fed it all the documentation from Active Collab, I couldn't get anything valid out of it. Hallucinated all kinds of things that weren't there. I then gave it the URL rewrites that would be required, line-by-line, spent a long time trying to correct it. No bueno, even worse hallucination. I spent days on trying to get it to output a valid NGINX config that incorporated these URL rewrites. It can't reason, it's doing exactly what LLMs do, which is next word prediction. I can't imagine what people are using it for in terms of a valuable addition to their workflow with how much it hallucinates. If it can't even do an NGINX config, what use is it for anything else? People saying it's helping them learn programming languages. Fuck me, they don't know when it's wrong, and it will be wrong at some point, it's an LLM.

stevenhuang 4y ago
There are flaws but also consider PEBCAK error. For one next time when it starts hallucinating and a gentle course correction doesn't do it, just start a new chat with a different prompt approach. Having the error in its context reinforces the same mistake and sometimes it can't get out of this loop.
- hammyhavoc 4y ago
  PEBKAC in what context? The prompts themselves or the documentation? Because I got Active Collab running on NGINX myself. I already did this in terms of starting new chats, I spent days on it, and consulted with half a dozen devs supposedly using it in their workflows. It's very easy to make it hallucinate.
  - stevenhuang 4y ago
    In terms of using gpt optimally. But fair enough. If you tried it in multiple sessions trying to convert Apache rewrites to nginx and it wasn't able to do it, I guess this is another failure mode. I just found that curious because chatgpt is usually very very good at regex. Side note is Google extra terrible lately or is there really no docs on this almost anywhere? All I could find about it is this and the rules looked very simple, from my experience chatgpt should have got this https://activecollab.com/help/books/self-hosted-activecollab/requirements https://activecollab.com/help/books/self-hosted-activecollab...
    - hammyhavoc 4y ago
      That's the documentation on the URL. And yeah, I was surprised too given how people had hyped it so heavily in much more complex scenarios for dev work (well, claimed to anyway, they haven't documented it in most cases of claiming to x).
- smoldesu 4y ago
  > but also consider PEBCAK error. No. It's an AI error, the person between the chair and keyboard just hit enter. If this "error" goes away when I hit the enter key a few more times and get lucky, it's not my fault.
  - hammyhavoc 4y ago
    Well, the error doesn't go away, it's a "fail state", but that still doesn't mean PEBCAK either.
circuit10 4y ago
It’s true that their training goal sometimes rewards them hallucinations, encouraging them do that, but that doesn’t mean it’s not reasoning. If anything hallucinations involve more reasoning because they have to make up something new that wasn’t there
- hammyhavoc 4y ago
  It's doing what LLMs do. Next word prediction. It didn't "make it up", it incorrectly predicted the relationship of words. It isn't reasoning about the solution to a problem either, it's running as expected in relating words to each other, that doesn't mean it has any form of understanding of the words or even what it's rendering as an output.
  - circuit10 4y ago
    It does just predict the next word, yes, but it turns out that predicting the next word is a complex problem that involves solving many types of subproblems that involve relatively complex reasoning, or at least something that looks very, very much like reasoning
    - hammyhavoc 4y ago
      Is it any more reasoning than Bayes' theorem is reasoning? It's statistics and derived probability from said statistics.
      - circuit10 4y ago
        > Bayesian inference is a method of statistical inference in which Bayes' theorem is used to update the probability for a hypothesis as more evidence or information becomes available. That is very clearly reasoning
        hammyhavoc 4y ago
        No, it's probability entered into an algorithm as a variable. This is not reasoning, this is probability, and there is no reasoning involved. Go and program HomeAssistant (FOSS) with an automation using Bayes' theorem. Then proceed to laugh about all the fail states you encounter, thus realizing it is not reasoning, but probability. https://community.home-assistant.io/t/how-bayes-sensors-work-from-a-statistics-professor-with-working-google-sheets/143177 https://community.home-assistant.io/t/how-bayes-sensors-work... It's quite literally known as Bayesian probability.
Dudeman112 4y ago
>If it can't even do an NGINX config, what use is it for anything else? Turn it around. I'm sure there are lots of people around who couldn't do it either, even with a ridiculously high amount of time to do it
- mr_mitm 4y ago
  Yes, but we are discussing the claim that LLMs might exhibit superhuman smartness.
  - Aeolos 4y ago
    They are fallible, but they quite clearly exhibit superhuman smartness when compared to the average human. As a thought experiment, assume the average human may be able to translate text between two human languages, or write code in two-three programming languages. GPT4 can perform those tasks on a much more diverse set of human _and_ programming languages. Is that not superhuman? Yes, it makes mistakes. But take a hundred humans off the street and ask them to write an NGINX configuration or translate between Indian and French - how many would be able to do that? How many would be able to do that without any mistakes?
rileyphone 4y ago
The vast majority of humans would fail your reasoning test.
- hammyhavoc 4y ago
  And? We are talking about AI, not humans. People on HN claim they're using it for XYZ in development, yet it can't even generate the necessary NGINX config, despite being given the URL rewrites it'll need to incorporate. The point is that it hallucinates. It isn't that it failed, it's that despite giving it everything it needs to know, it hallucinated all kinds of things not in the documentation, not in my prompts et al. Why? Because it's an LLM. It isn't fit for purpose in this context. A next word prediction AI (an LLM) isn't appropriate for these kinds of problems.
  - Aeolos 4y ago
    Is your argument that AI needs to be held to a different standard than humans? Because humans confidently hallucinate answers all the time. To me, it feels quite intuitive that an AI trained on human knowledge would automatically learn to do the same.
    - hammyhavoc 4y ago
      Neither. The point: it isn't that it failed. It isn't even that it "hallucinates answers", it's that it infers relationships between words that don't exist because it's an LLM. It predicts the next word. That's what it does. Something that predicts the next word isn't an appropriate method of doing x in y of z cases, because its reliability in providing the designated function is important. Ergo, yes, LLMs may well have applications, but most of the problems that people are throwing it at are inappropriate, just like blockchain fetishism versus a database. For the overwhelming majority of problems, AI is not the answer, neither is a blockchain, neither is an NFT. Call hallucination what it is: a fail state. It got it wrong. It didn't "hallucinate". With standard conditional logic, x yields y result. That's very useful where you want consistency and reliability, ergo, those problems are best not handled via an LLM. Why not use the appropriate tool for the job? Deductive versus inductive versus abductive reasoning.

stevenhuang 4y ago

There are flaws but also consider PEBCAK error. For one next time when it starts hallucinating and a gentle course correction doesn't do it, just start a new chat with a different prompt approach. Having the error in its context reinforces the same mistake and sometimes it can't get out of this loop.

hammyhavoc 4y ago
PEBKAC in what context? The prompts themselves or the documentation? Because I got Active Collab running on NGINX myself. I already did this in terms of starting new chats, I spent days on it, and consulted with half a dozen devs supposedly using it in their workflows. It's very easy to make it hallucinate.
- stevenhuang 4y ago
  In terms of using gpt optimally. But fair enough. If you tried it in multiple sessions trying to convert Apache rewrites to nginx and it wasn't able to do it, I guess this is another failure mode. I just found that curious because chatgpt is usually very very good at regex. Side note is Google extra terrible lately or is there really no docs on this almost anywhere? All I could find about it is this and the rules looked very simple, from my experience chatgpt should have got this https://activecollab.com/help/books/self-hosted-activecollab/requirements https://activecollab.com/help/books/self-hosted-activecollab...
  - hammyhavoc 4y ago
    That's the documentation on the URL. And yeah, I was surprised too given how people had hyped it so heavily in much more complex scenarios for dev work (well, claimed to anyway, they haven't documented it in most cases of claiming to x).
smoldesu 4y ago
> but also consider PEBCAK error. No. It's an AI error, the person between the chair and keyboard just hit enter. If this "error" goes away when I hit the enter key a few more times and get lucky, it's not my fault.
- hammyhavoc 4y ago
  Well, the error doesn't go away, it's a "fail state", but that still doesn't mean PEBCAK either.

circuit10 4y ago

It’s true that their training goal sometimes rewards them hallucinations, encouraging them do that, but that doesn’t mean it’s not reasoning. If anything hallucinations involve more reasoning because they have to make up something new that wasn’t there

hammyhavoc 4y ago
It's doing what LLMs do. Next word prediction. It didn't "make it up", it incorrectly predicted the relationship of words. It isn't reasoning about the solution to a problem either, it's running as expected in relating words to each other, that doesn't mean it has any form of understanding of the words or even what it's rendering as an output.
- circuit10 4y ago
  It does just predict the next word, yes, but it turns out that predicting the next word is a complex problem that involves solving many types of subproblems that involve relatively complex reasoning, or at least something that looks very, very much like reasoning
  - hammyhavoc 4y ago
    Is it any more reasoning than Bayes' theorem is reasoning? It's statistics and derived probability from said statistics.
    - circuit10 4y ago
      > Bayesian inference is a method of statistical inference in which Bayes' theorem is used to update the probability for a hypothesis as more evidence or information becomes available. That is very clearly reasoning
      - hammyhavoc 4y ago
        No, it's probability entered into an algorithm as a variable. This is not reasoning, this is probability, and there is no reasoning involved. Go and program HomeAssistant (FOSS) with an automation using Bayes' theorem. Then proceed to laugh about all the fail states you encounter, thus realizing it is not reasoning, but probability. https://community.home-assistant.io/t/how-bayes-sensors-work-from-a-statistics-professor-with-working-google-sheets/143177 https://community.home-assistant.io/t/how-bayes-sensors-work... It's quite literally known as Bayesian probability.

Dudeman112 4y ago

>If it can't even do an NGINX config, what use is it for anything else? Turn it around. I'm sure there are lots of people around who couldn't do it either, even with a ridiculously high amount of time to do it

mr_mitm 4y ago
Yes, but we are discussing the claim that LLMs might exhibit superhuman smartness.
- Aeolos 4y ago
  They are fallible, but they quite clearly exhibit superhuman smartness when compared to the average human. As a thought experiment, assume the average human may be able to translate text between two human languages, or write code in two-three programming languages. GPT4 can perform those tasks on a much more diverse set of human _and_ programming languages. Is that not superhuman? Yes, it makes mistakes. But take a hundred humans off the street and ask them to write an NGINX configuration or translate between Indian and French - how many would be able to do that? How many would be able to do that without any mistakes?

rileyphone 4y ago

The vast majority of humans would fail your reasoning test.

hammyhavoc 4y ago
And? We are talking about AI, not humans. People on HN claim they're using it for XYZ in development, yet it can't even generate the necessary NGINX config, despite being given the URL rewrites it'll need to incorporate. The point is that it hallucinates. It isn't that it failed, it's that despite giving it everything it needs to know, it hallucinated all kinds of things not in the documentation, not in my prompts et al. Why? Because it's an LLM. It isn't fit for purpose in this context. A next word prediction AI (an LLM) isn't appropriate for these kinds of problems.
- Aeolos 4y ago
  Is your argument that AI needs to be held to a different standard than humans? Because humans confidently hallucinate answers all the time. To me, it feels quite intuitive that an AI trained on human knowledge would automatically learn to do the same.
  - hammyhavoc 4y ago
    Neither. The point: it isn't that it failed. It isn't even that it "hallucinates answers", it's that it infers relationships between words that don't exist because it's an LLM. It predicts the next word. That's what it does. Something that predicts the next word isn't an appropriate method of doing x in y of z cases, because its reliability in providing the designated function is important. Ergo, yes, LLMs may well have applications, but most of the problems that people are throwing it at are inappropriate, just like blockchain fetishism versus a database. For the overwhelming majority of problems, AI is not the answer, neither is a blockchain, neither is an NFT. Call hallucination what it is: a fail state. It got it wrong. It didn't "hallucinate". With standard conditional logic, x yields y result. That's very useful where you want consistency and reliability, ergo, those problems are best not handled via an LLM. Why not use the appropriate tool for the job? Deductive versus inductive versus abductive reasoning.