6 ms·
The biggest drawback of LLM is that it never answers with "I don't know" (unless it is some quote) and it just brings bullshit hallucinations which human has to
by imhoguy 4y ago
The biggest drawback of LLM is that it never answers with "I don't know" (unless it is some quote) and it just brings bullshit hallucinations which human has to reject as wrong. Thus it is mostly useless for anything serious.
Personally I use it to beautify some text, but still have to do a bit of correction to fix b/s or missed context.
- Nevermark 4y agoThis seems like a great opportunity for some good old fashioned adversarial training. Post train one LLM to please another LLM, that rates the quality of the first model's responses and calls it on any bullshit! (And vice versa.)
- IIAOPSW 4y agoStable diffusion (of bullshit).
- not2b 4y agoI expect that this will be tried, and I worry about some negative consequences if it works well. It could be a way of generating very effective propaganda, that defeats the efforts of the opposing LLM to call bullshit. On the other hand, diffusion models seem to have replaced GANs for image synthesis, so perhaps there's something I'm missing, or perhaps there's a way to combine both techniques.
- Nevermark 4y agoI am not sure calling bullshit on images works the same way, given we generally want to invite creativity to image generation. Although an adversarial extra-finger detector seems in order! Extra fingers are not creativity. They are taboo!! But for text, there is a lot of structure around bullshitting, and fortunately, the internet is full of examples of people calling bullshit. As long as the bullshit adversary has to give a strong critique to back up any bullshit call, it should work. And bullshit judgments can be de-bullshitted too. This would be the same as what our social circles, work colleagues, families, etc. do for us, so a circular firing squad of bullshit judges, who also give gold stars for quality responses, seems like a natural solution for models too.
- not2b 4y agoThe first really good image generators for faces (see for example https://this-person-does-not-exist.com https://this-person-does-not-exist.com ) used GANs, which were trained by having one model generate faces and another model saying bullshit, that isn't a face. I'm not worried about using approaches like this to converge on truth (model A generates an answer to a question, model B attempts to refute model A). Perhaps we could train model A to generate lies that model B can't detect.
- jug 4y agoBing AI often tells me that something is not known and when it hasn’t found any sources to confirm something. I think ChatGPT is worse off though.
- icelancer 4y ago> The biggest drawback of LLM is that it never answers with "I don't know" I see this a lot on social media, but it simply isn't true in my experience as someone who uses multiple APIs from OpenAI.
- pfortuny 4y agoAsk it to “prove how a martingale converges to the zariski measure on a commutative scheme.” Totally absurd question, totally absurd answer.
- icelancer 4y agoI asked ChatGPT Legacy: "Hello! Can you prove how a martingale converges to the zariski measure on a commutative scheme?" It replied: "Hello! The notion of a martingale converging to the Zariski measure on a commutative scheme is a highly advanced and abstract mathematical concept. It is not possible to provide a simple proof for this statement, as it requires a deep understanding of several advanced mathematical concepts such as probability theory, algebraic geometry, and commutative algebra." It then listed definitions for martingale, Zariski measure, and a commutative scheme, and then concluded with: "However, to fully understand this concept, you would need to study advanced mathematical topics such as algebraic geometry, commutative algebra, and probability theory." That seems like a useful enough answer with definitions that effectively converges to "I don't know."
- pfortuny 4y agoNotice that martingales do not converge AND the zariski measure is not a thing at all. Sorry saw your reply too late.
- deleted 4y ago[deleted]
- adamddev1 4y agoYes, I see a lot of people talking about how it revolutionizes learning, but it only provides decent answers for extremely well-known and popular topics on the internet. Ask it a question about a less-documented human language for instance and it will mislead you with the most hilariously wrong and confidently made-up garbage answers. This could end up being another example of knowledge quality going down as people keep going for the easiest was to get answers, like searching for info on topics on Instagram and TikTok.
- taneq 4y agoI’m not an expert, but I’ve always felt the use of softmax everywhere is a contributing factor. It’s basically saying “you HAVE to pick one” rather than “which one would you pick?”
- jll29 4y agoThere is the notion of the _rejection class_, which addresses the problem of 'You have to pick one' - it is the class you select when you mean to say 'none of the above'.
- deleted 4y ago[deleted]
- photochemsyn 4y agoIt seems to respond better if you include a 'in the context of <subject matter>' at the end of the prompt. Then keep refining context over a series of questions.
- dymax78 4y agoI haven't experienced chatgpt explicitly tell me "I don't know", but it has provided me with factually incorrect answers pertaining to the functionality of an appliance and associated software. Further probing and I was advised to contact IBM support.... lol. My usage has been confined to specific technical inquiries and chatgpts benefit has been colossal in saving me the time of sifting through decade(s) of documentation, forum posts, bug reports, kb articles, etc.
- andrepd 4y agoHonestly, how can that be? If it's bullshitting me half of the time... If it's something I already know I can see it's bullshitting, if it's something I don't know (i.e. I'm using it for a practical purpose) then how can I tell!
- bob1029 4y agoCan't you address this with adversarial training items that build upon each other? I will be doing some testing of this against the base davinci model over the next few weeks.
- ryloric 4y agoIt has no way of judging the quality of it's own answer, I see this trait as it's biggest hurdle to most practical uses.
- dclowd9901 4y agoYeah I don’t understand why “I don’t know, but…” or “I’m not too sure, but…” isn’t part of the generative text. Does everyone on the web talk like they have the answers to everything? I’m genuinely asking.
- Bost 4y agoSorry but that's not true. I play with ChatGPT only for about a week or two and I have seen such a response a couple of times. However it's not easy to get such an answer. I'm not denying that. (Unfortunately I can't provide you an example. The ChatGPT history is not available right now. Wed Mar 8 02:02:14 PM CET 2023)
- namrog84 4y agoI just saw some post 9n the subteddit talking about methods to get it to provide exactly that about hallucination and confidence. With various techniques. I think in the future it'll become less problematic or more transparent about hallucinations