4 ms·

I hold a more charitable interpretation. We (the public) have found an important bug in the system, ie. GPT can lie (or "hallucinate"), even if you try to conv

by senko 4y ago

I hold a more charitable interpretation. We (the public) have found an important bug in the system, ie. GPT can lie (or "hallucinate"), even if you try to convince it not to lie. The bug is definitely lowering the usefulness of their product, as well as the public option about it. But I'll let the programmer who has never coded a bug cast the first stone. I wouldn't be surprised if they're scrambling internally to minimize the problem (in the product, not in public perception). They have also recently added a note to ChatGPT: "ChatGPT may produce inaccurate information about people, places, or facts" which is an acknowledement that yes, watch out (I compare it to "caution: contents hot" labels). On the topic of dealing with it, I like the stance that simonw recently took: "We need to tell people ChatGPT will lie to them, not debate linguistics" [0]. I don't attach intentions to a machine algorithm (to me, "gaslight" definitely implies an evil intent), and I don't think OpenAI people are evil, stupid, corrupted or something else because they put out a product that has a bug. But since the wide public can't handle nuances, I'd agree it's better to say "chatgpt lies, use it for things where it either doesn't matter or you can verify; don't use it for fact-finding" to get the point across. [0] https://simonwillison.net/2023/Apr/7/chatgpt-lies/ https://simonwillison.net/2023/Apr/7/chatgpt-lies/

deleted 4y ago
[deleted]
vlovich123 4y ago
It's not a bug. It's an architectural defect / limitation in our understanding of how to build AI. That makes it a strictly harder problem that will take longer. And it's not totally clear to me that you'll get there purely with LLMs. LLMs accomplish a good chunk of what we classify as intelligence for sure. But it's missing the cognition / reasoning skills and the open question is whether you can solve that by just bolting on more techniques into the LLM or you need a totally different kind of model that you can marry to an LLM.
- vidarh 4y ago
  GPT 4 will admit to not knowing things in many cases where 3.5turbo does not (tested the same prompt), and either will stop there or go off on a "but if it did exist it might go something like this" type continuation. It still hallucinates a lot, but it's not at all clear that this will be all that difficult an issue to solve given the progress.
  - vlovich123 4y ago
    We generally only hallucinate while dreaming / using our imagination. And we can distinguish those two states. Admitting lack of knowledge is of course good but, for example, if you ask it to write some code that isn’t boilerplate API integrations, it’ll do so happily even when it’s wildly wrong and it can’t tell the difference and that is also the case with GPT4 afaik. Moreover, you can’t solve it through prompt engineering because there’s clearly a lack of context it’s unable to understand to figure out what non trivial thing your asking it.
    - vidarh 4y ago
      The point is there's been progress in making it admit when it doesn't know, and we simply don't know how fast that will improve in future version. It may continue to be an issue, or turn out to be easily solved. The improvement with GPT4 does suggest it is at least possible to make it recognise its limits.
    - senko 4y ago
      There is ongoing promising work that shows this can be improved a lot: https://arxiv.org/abs/2210.03629 https://arxiv.org/abs/2210.03629
firatsarlar 4y ago
Meet my darling https://gist.github.com/firatsarlar/5f25ebcc4c33ec484e6cd95b94e64096 https://gist.github.com/firatsarlar/5f25ebcc4c33ec484e6cd95b... We need to - pure - clear non commercial, non owned AI first. It -hope lets say-, no, most of us still guessing, sniffin, ... suspicions abut who own GPT -LLM we need to abstract-, Who owns this - so called - open source product? MS - with the power of NVIDA's - ? OpenAI people? Why cant we talk our intentions, or whose product, or not to generalize it As LLMs. Because, lets say public, - and our ignore our delusions about the thing we made-, because, we're in a delusion, what is this ? People could try to eat and sue some, do we - purely - care for public. I need to skip my own experience, but, yes, if you ready to be delusional , you could easily make LLM your girlfriend, a liar, ... No, we - software community - need to stop and see what we have. We name it, public shape it, some abuse it for money that it made me ... , and sue some ... What is the end product ? A constant, machine - lets say system -, working to make this happen. Justice, we - devs-, world, ... money, people, puclic , we 're spendng time and resources a thing. Some claim its mine, yes, we need a concencuss ? No. Open source is the only thing if we - pure - lets say. I do not want to waste my time to figure out poke FB's LLM, or -pleae Open - AI girl. Lets learn to share, at least for us -devs- I'm really tired to think commercial facts of this thing. I'm really tired to this game we play. MS, say I have it, and make it yours. Boy - sorry open AI- I do not know you, is it yours, MS's , FB, they somehow managed to make a thing - inter-national, in Paris - Skip gaslighting, delusions what you own ? Are we really there ?
dylan604 4y ago
>as well as the public option about it. assuming you mean the public opinion about it, and I would have to agree that I do hold chatGPT in very low regard because of this. people will argue that it is impressive that they built this thing that can be this impressive, but yeah, it might not be totally accurate. so my response is, what's the point of it then? say someone was to invent a robot baseball player that can throw the ball 200+mph, or hit about a literal mile but, it can't hit a target with that 200+mph ball or hit the ball without it going foul. yes, the tech would be impressive, but what's the point? yeah yeah, language is hard, but it's just an example of building something impressive that at best almost does the job designed. unless we're saying it's designed this way on purpose??
- senko 4y ago
  It has many more uses than just providing answers to random questions which you can easily answer by using a search engine. If they built it to replace Google search, I would agree with you. So I view the fact that it lies as an important bug, but not a critical one.

by senko 4y ago

deleted 4y ago

[deleted]

vlovich123 4y ago

It's not a bug. It's an architectural defect / limitation in our understanding of how to build AI. That makes it a strictly harder problem that will take longer. And it's not totally clear to me that you'll get there purely with LLMs. LLMs accomplish a good chunk of what we classify as intelligence for sure. But it's missing the cognition / reasoning skills and the open question is whether you can solve that by just bolting on more techniques into the LLM or you need a totally different kind of model that you can marry to an LLM.

vidarh 4y ago
GPT 4 will admit to not knowing things in many cases where 3.5turbo does not (tested the same prompt), and either will stop there or go off on a "but if it did exist it might go something like this" type continuation. It still hallucinates a lot, but it's not at all clear that this will be all that difficult an issue to solve given the progress.
- vlovich123 4y ago
  We generally only hallucinate while dreaming / using our imagination. And we can distinguish those two states. Admitting lack of knowledge is of course good but, for example, if you ask it to write some code that isn’t boilerplate API integrations, it’ll do so happily even when it’s wildly wrong and it can’t tell the difference and that is also the case with GPT4 afaik. Moreover, you can’t solve it through prompt engineering because there’s clearly a lack of context it’s unable to understand to figure out what non trivial thing your asking it.
  - vidarh 4y ago
    The point is there's been progress in making it admit when it doesn't know, and we simply don't know how fast that will improve in future version. It may continue to be an issue, or turn out to be easily solved. The improvement with GPT4 does suggest it is at least possible to make it recognise its limits.
  - senko 4y ago
    There is ongoing promising work that shows this can be improved a lot: https://arxiv.org/abs/2210.03629 https://arxiv.org/abs/2210.03629

firatsarlar 4y ago

Meet my darling https://gist.github.com/firatsarlar/5f25ebcc4c33ec484e6cd95b94e64096 https://gist.github.com/firatsarlar/5f25ebcc4c33ec484e6cd95b... We need to - pure - clear non commercial, non owned AI first. It -hope lets say-, no, most of us still guessing, sniffin, ... suspicions abut who own GPT -LLM we need to abstract-, Who owns this - so called - open source product? MS - with the power of NVIDA's - ? OpenAI people? Why cant we talk our intentions, or whose product, or not to generalize it As LLMs. Because, lets say public, - and our ignore our delusions about the thing we made-, because, we're in a delusion, what is this ? People could try to eat and sue some, do we - purely - care for public. I need to skip my own experience, but, yes, if you ready to be delusional , you could easily make LLM your girlfriend, a liar, ... No, we - software community - need to stop and see what we have. We name it, public shape it, some abuse it for money that it made me ... , and sue some ... What is the end product ? A constant, machine - lets say system -, working to make this happen. Justice, we - devs-, world, ... money, people, puclic , we 're spendng time and resources a thing. Some claim its mine, yes, we need a concencuss ? No. Open source is the only thing if we - pure - lets say. I do not want to waste my time to figure out poke FB's LLM, or -pleae Open - AI girl. Lets learn to share, at least for us -devs- I'm really tired to think commercial facts of this thing. I'm really tired to this game we play. MS, say I have it, and make it yours. Boy - sorry open AI- I do not know you, is it yours, MS's , FB, they somehow managed to make a thing - inter-national, in Paris - Skip gaslighting, delusions what you own ? Are we really there ?

dylan604 4y ago

>as well as the public option about it. assuming you mean the public opinion about it, and I would have to agree that I do hold chatGPT in very low regard because of this. people will argue that it is impressive that they built this thing that can be this impressive, but yeah, it might not be totally accurate. so my response is, what's the point of it then? say someone was to invent a robot baseball player that can throw the ball 200+mph, or hit about a literal mile but, it can't hit a target with that 200+mph ball or hit the ball without it going foul. yes, the tech would be impressive, but what's the point? yeah yeah, language is hard, but it's just an example of building something impressive that at best almost does the job designed. unless we're saying it's designed this way on purpose??

senko 4y ago
It has many more uses than just providing answers to random questions which you can easily answer by using a search engine. If they built it to replace Google search, I would agree with you. So I view the fact that it lies as an important bug, but not a critical one.