4 ms·

Someone smart once said: If it is good, we call it "creativity." If it is bad, we call it "hallucination." This isn't a bug (or limitation, as the authors sa

Someone smart once said: If it is good, we call it "creativity." If it is bad, we call it "hallucination." This isn't a bug (or limitation, as the authors say). It's a feature.

tgv 3y ago
Asking it to write code for you is basically asking it to hallucinate.
- gardenhedge 3y ago
  I don't think so. I think it's asking it to repeat code it has been trained on
  - tgv 3y ago
    Search for a piece of code you wrote. If it's more than 15 lines and not boilerplate, chances are you won't find it anywhere on the net.
    - gardenhedge 3y ago
      To be honest, I imagine I would. Variables named could be different but it would largely be the same as code others have written. For example, I am creating an app in React Native at the moment. My app will not be unique in terms of code but instead in business domain.
    - Jensson 3y ago
      But there are many pieces of code that I've written that you can find in many places on the net, having a tool that can adapt that to your codebase in seconds is useful. It doesn't have to be smart, just pasting in an function and fitting that to your code is useful.
      - tgv 3y ago
        Sure, but the point is it will have to adapt it to your code, if only in naming. So it has to make up things, i.e. hallucinate. It can't just reproduce the best match in memory.
        Jensson 3y ago
        Yeah, these models are very good at making up names, that is what they are trained to do after all. Their ability to do logic isn't that impressive though and seems to be on the level of a human that doesn't understand the topic but has seen many examples.
    - gessha 3y ago
      This reminds me of a bit from a Hugh and Laurie sketch: > Imagine a piano keyboard, eighty-eight keys, only eighty-eight and yet, and yet, new tunes, melodies, harmonies are being composed upon hundreds of keyboards every day in Dorset alone. Our language, Tiger, our language, hundreds of thousands of available words, frillions of possible legitimate new ideas, so that I can say this sentence and be confident it has never been uttered before in the history of human communication: "Hold the newsreader's nose squarely, waiter, or friendly milk will countermand my trousers." One sentence, common words, but never before placed in that order. And yet, oh and yet, all of us spend our days saying the same things to each other, time after weary time, living by clichaic, learned response: "I love you", "Don't go in there", "You have no right to say that", "shut up", "I'm hungry", "that hurt", "why should I?", "it's not my fault", "help", "Marjorie is dead". You see? That surely is a thought to take out for a cream tea on a rainy Sunday afternoon. https://abitoffryandlaurie.co.uk/sketches/language_conversation https://abitoffryandlaurie.co.uk/sketches/language_conversat...
  - intended 3y ago
    The term “Hallucinate” is a misnomer. Humans can hallucinate, we can get sick and perceive a world which is incongruous with reality. LLMs are just generating tokens. Hallucination perpetuates an unhelpful anthropomorphization of LLMs.
    - gessha 3y ago
      I don’t think the term is that bad here because I haven’t seen a lot of comparisons with human hallucinations. Users see it as a machine artifact.
      - intended 3y ago
        It’s like the term “god particle” - it invites comparisons and allusions that do not match reality.
LightBug1 3y ago
I imagine the gold is in knowing whether the LLM understands when it's doing either? Isn't this the difference between a human and an LLM? A human knows it's making an educated guess and (should) say so. Or it knows when it's being creative, and can say so. If it doesn't know which is which, then it really does bring it home that LLM's are not that much more than (very sophisticated) mechanical input-output machines.
- vladms 3y ago
  You mean "some humans know and could say so". And this reflection process is not captured in the data we fed to LLM-s (like let's say a lesson in which teacher asks "do you know X?", and students first answer "I don't know", etc.) Also, LLM-s could report more statistical measures for each answer and external tools could interpret them.
- devjab 3y ago
  Isn’t it always hallucinating though? We just don’t care when it gets it “right”. As I understand it, it’s still just probability based on what is likely to be a good set of words to answer the prompt tasking it. It doesn’t actually know anything, it’s just extremely good at making stuff up. Which is still very useful for a lot of things. Just maybe not things to which value is assigned based on how efficient and correct the answer is. Like you can have GPT make a marketing campaign for you, or you can have it design all the icons you need for your application UI, but you can’t reliably make it wrote high performance back-end code without having humans judge the results. Similarly you can’t use it to teach anyone anything, not really, because unless you’re already an expert on the subject being taught, you aren’t likely to spot when it gets things wrong. I guess you can argue that a lot of teaching is flawed like that, and you wouldn’t be wrong. Like, I was taught that the pyramids was build by slave labour, even after the archeological evidence had shown this to be likely false. But our text books were a decade old because our school didn’t really renew them very often… in such a case GPT might have been a more correct teacher, but the trick is that you won’t really know. Which is made even more complicated by the fact that it might teach different things to different students. Like, I just asked ChatGPT 3.5 who build the pyramids in 3 different prompts, in one it told me it was ordinary people. In the others it told me it was mostly skilled labour under guidance of “architects” and “engineers”. Still better than teaching us it was done by slave labour like my old book, but the book was still consistent in what was considered to be the truth at the time.
audunw 3y ago
True, but I think we can fall into the trap of expecting too much of LLMs. Their knowledge can seem perfect. They can answer almost anything, so it’s easy to get the illusion that they can answer anything truthfully. In terms of what we can expect of future improvements, I think it’s overly optimistic to expect any kind of super intelligence beyond what we see today (that is, having access to all the worlds publicly available information, or rapidly generating texts/images/videos that fall into existing creative patterns). I suspect that more creative intelligence requires an extremely fine balance to not “go crazy”.. that is, producing output we’d consider creative rather than hallucinations. I think getting this balance right will get exponentially harder as we create feedback loops within the AI that let its intelligence evolve. And it’s entirely possible that humans have already optimised this creative intelligence feedback loop as much as the universe allows. Having a huge amount of knowledge can obviously benefit from more neurons/storage. But we simply don’t know if that’s true for creative intelligence yet
- badgersnake 3y ago
  > True, but I think we can fall into the trap of expecting too much of LLMs. We’re already well past that point. Why? Because saying incredible things about AI attracts VC money.
Seb-C 3y ago
That is correct, it's always hallucinating and making things up. Just because those hallucinations sometimes randomly happens to be right, people concluded that being wrong is the exception, while being right is somehow the rule. It's like when people read [insert millenias old text here], finds a part that happens to illustrate something in their life today and conclude that it is a prophecy that predicted the future. The meaning/truth in those is nothing more than a cognitive bias from the mind of the reader, not an inherent quality of the text.
- somewhereoutth 3y ago
  For heavy LLM users, there is probably a dopamine hit when it does something right, much as gamblers get a hit when the fruit machine pays out. Perhaps LLM use is no more productive than gambling, and perhaps can be abused in a similar way.
  - HKH2 3y ago
    You might not have found any uses that suit your niche, but that doesn't mean those of us who have are just making up stories about productivity.
    - somewhereoutth 3y ago
      "You might not have found any games that suit your skill set, but that doesn't mean those of us who have are just making up stories about making money" Sorry, somewhat trite and unfair, but, if there is a gambling-like dopamine reward cycle occurring, then the users would have a hard time being truly objective about any productivity boost in total. They may instead focus on the 'wins', without taking into account any overheads or 'losses', much as a gambler would do.
      - HKH2 3y ago
        Sure, confirmation bias exists, but you can compare with the alternatives. E.g. a search engine can give you zero useful results, and you can fine tune your query and still get nothing after scrolling through pages of results (Do people really take the losses into account when using search engines?) I find prompt engineering with LLMs more useful because you get nudged in interesting directions, and even if you come away with no direct results, you have more of an idea of what you are looking for. Maybe lateral thinking is overrated.
zer00eyz 3y ago
> This isn't a bug If it isn't a bug, it dam well isn't a hallucination, or creativity. This is a deeply integrated design defect. One that highlights what we're doing (statistically modeling lots of human language)... Throwing more data against this path isnt going to magically make it wake up and be an AGI. And this problem is NOT going to go away. The ML community need to back off the hype train. The first step is them not anthropomorphizing their projects.

Someone smart once said: If it is good, we call it "creativity." If it is bad, we call it "hallucination." This isn't a bug (or limitation, as the authors say). It's a feature.

tgv 3y ago

Asking it to write code for you is basically asking it to hallucinate.

gardenhedge 3y ago
I don't think so. I think it's asking it to repeat code it has been trained on
- tgv 3y ago
  Search for a piece of code you wrote. If it's more than 15 lines and not boilerplate, chances are you won't find it anywhere on the net.
  - gardenhedge 3y ago
    To be honest, I imagine I would. Variables named could be different but it would largely be the same as code others have written. For example, I am creating an app in React Native at the moment. My app will not be unique in terms of code but instead in business domain.
  - Jensson 3y ago
    But there are many pieces of code that I've written that you can find in many places on the net, having a tool that can adapt that to your codebase in seconds is useful. It doesn't have to be smart, just pasting in an function and fitting that to your code is useful.
    - tgv 3y ago
      Sure, but the point is it will have to adapt it to your code, if only in naming. So it has to make up things, i.e. hallucinate. It can't just reproduce the best match in memory.
      - Jensson 3y ago
        Yeah, these models are very good at making up names, that is what they are trained to do after all. Their ability to do logic isn't that impressive though and seems to be on the level of a human that doesn't understand the topic but has seen many examples.
  - gessha 3y ago
    This reminds me of a bit from a Hugh and Laurie sketch: > Imagine a piano keyboard, eighty-eight keys, only eighty-eight and yet, and yet, new tunes, melodies, harmonies are being composed upon hundreds of keyboards every day in Dorset alone. Our language, Tiger, our language, hundreds of thousands of available words, frillions of possible legitimate new ideas, so that I can say this sentence and be confident it has never been uttered before in the history of human communication: "Hold the newsreader's nose squarely, waiter, or friendly milk will countermand my trousers." One sentence, common words, but never before placed in that order. And yet, oh and yet, all of us spend our days saying the same things to each other, time after weary time, living by clichaic, learned response: "I love you", "Don't go in there", "You have no right to say that", "shut up", "I'm hungry", "that hurt", "why should I?", "it's not my fault", "help", "Marjorie is dead". You see? That surely is a thought to take out for a cream tea on a rainy Sunday afternoon. https://abitoffryandlaurie.co.uk/sketches/language_conversation https://abitoffryandlaurie.co.uk/sketches/language_conversat...
- intended 3y ago
  The term “Hallucinate” is a misnomer. Humans can hallucinate, we can get sick and perceive a world which is incongruous with reality. LLMs are just generating tokens. Hallucination perpetuates an unhelpful anthropomorphization of LLMs.
  - gessha 3y ago
    I don’t think the term is that bad here because I haven’t seen a lot of comparisons with human hallucinations. Users see it as a machine artifact.
    - intended 3y ago
      It’s like the term “god particle” - it invites comparisons and allusions that do not match reality.

LightBug1 3y ago

I imagine the gold is in knowing whether the LLM understands when it's doing either? Isn't this the difference between a human and an LLM? A human knows it's making an educated guess and (should) say so. Or it knows when it's being creative, and can say so. If it doesn't know which is which, then it really does bring it home that LLM's are not that much more than (very sophisticated) mechanical input-output machines.

vladms 3y ago
You mean "some humans know and could say so". And this reflection process is not captured in the data we fed to LLM-s (like let's say a lesson in which teacher asks "do you know X?", and students first answer "I don't know", etc.) Also, LLM-s could report more statistical measures for each answer and external tools could interpret them.
devjab 3y ago
Isn’t it always hallucinating though? We just don’t care when it gets it “right”. As I understand it, it’s still just probability based on what is likely to be a good set of words to answer the prompt tasking it. It doesn’t actually know anything, it’s just extremely good at making stuff up. Which is still very useful for a lot of things. Just maybe not things to which value is assigned based on how efficient and correct the answer is. Like you can have GPT make a marketing campaign for you, or you can have it design all the icons you need for your application UI, but you can’t reliably make it wrote high performance back-end code without having humans judge the results. Similarly you can’t use it to teach anyone anything, not really, because unless you’re already an expert on the subject being taught, you aren’t likely to spot when it gets things wrong. I guess you can argue that a lot of teaching is flawed like that, and you wouldn’t be wrong. Like, I was taught that the pyramids was build by slave labour, even after the archeological evidence had shown this to be likely false. But our text books were a decade old because our school didn’t really renew them very often… in such a case GPT might have been a more correct teacher, but the trick is that you won’t really know. Which is made even more complicated by the fact that it might teach different things to different students. Like, I just asked ChatGPT 3.5 who build the pyramids in 3 different prompts, in one it told me it was ordinary people. In the others it told me it was mostly skilled labour under guidance of “architects” and “engineers”. Still better than teaching us it was done by slave labour like my old book, but the book was still consistent in what was considered to be the truth at the time.

audunw 3y ago

True, but I think we can fall into the trap of expecting too much of LLMs. Their knowledge can seem perfect. They can answer almost anything, so it’s easy to get the illusion that they can answer anything truthfully. In terms of what we can expect of future improvements, I think it’s overly optimistic to expect any kind of super intelligence beyond what we see today (that is, having access to all the worlds publicly available information, or rapidly generating texts/images/videos that fall into existing creative patterns). I suspect that more creative intelligence requires an extremely fine balance to not “go crazy”.. that is, producing output we’d consider creative rather than hallucinations. I think getting this balance right will get exponentially harder as we create feedback loops within the AI that let its intelligence evolve. And it’s entirely possible that humans have already optimised this creative intelligence feedback loop as much as the universe allows. Having a huge amount of knowledge can obviously benefit from more neurons/storage. But we simply don’t know if that’s true for creative intelligence yet

badgersnake 3y ago
> True, but I think we can fall into the trap of expecting too much of LLMs. We’re already well past that point. Why? Because saying incredible things about AI attracts VC money.

Seb-C 3y ago

That is correct, it's always hallucinating and making things up. Just because those hallucinations sometimes randomly happens to be right, people concluded that being wrong is the exception, while being right is somehow the rule. It's like when people read [insert millenias old text here], finds a part that happens to illustrate something in their life today and conclude that it is a prophecy that predicted the future. The meaning/truth in those is nothing more than a cognitive bias from the mind of the reader, not an inherent quality of the text.

somewhereoutth 3y ago
For heavy LLM users, there is probably a dopamine hit when it does something right, much as gamblers get a hit when the fruit machine pays out. Perhaps LLM use is no more productive than gambling, and perhaps can be abused in a similar way.
- HKH2 3y ago
  You might not have found any uses that suit your niche, but that doesn't mean those of us who have are just making up stories about productivity.
  - somewhereoutth 3y ago
    "You might not have found any games that suit your skill set, but that doesn't mean those of us who have are just making up stories about making money" Sorry, somewhat trite and unfair, but, if there is a gambling-like dopamine reward cycle occurring, then the users would have a hard time being truly objective about any productivity boost in total. They may instead focus on the 'wins', without taking into account any overheads or 'losses', much as a gambler would do.
    - HKH2 3y ago
      Sure, confirmation bias exists, but you can compare with the alternatives. E.g. a search engine can give you zero useful results, and you can fine tune your query and still get nothing after scrolling through pages of results (Do people really take the losses into account when using search engines?) I find prompt engineering with LLMs more useful because you get nudged in interesting directions, and even if you come away with no direct results, you have more of an idea of what you are looking for. Maybe lateral thinking is overrated.

zer00eyz 3y ago

> This isn't a bug If it isn't a bug, it dam well isn't a hallucination, or creativity. This is a deeply integrated design defect. One that highlights what we're doing (statistically modeling lots of human language)... Throwing more data against this path isnt going to magically make it wake up and be an AGI. And this problem is NOT going to go away. The ML community need to back off the hype train. The first step is them not anthropomorphizing their projects.