2 ms·

But sometimes it makes up answers that are wrong but sound plausible to you. You have no way to tell when it does this, and neither does it.

arresin 2y ago
Or when you tell it you think the answer is something else it agrees with you and apologies that yes, now it can see, 2+2 does equal 5.
neuralnetes-COO 2y ago
That is the entire purpose of LLMOps. Provide guardrails to prevent hallucination and ensure precise control of GenAI output.
- tayo42 2y ago
  How can you tell what's true or not?
  - neuralnetes-COO 2y ago
    You have to develop your own QA methods to ensure output is exactly what you want.
falcor84 2y ago
Neither can you easily notice that when a human teacher/tutor does that. At least with math, you're able to try it yourself and see if it works. I've had many cases where following through on a misleading explanation by a teacher/book actually ended up leading to me better retaining the topic.
xscott 2y ago
That's absolutely a bug that needs to be fixed, but I think it's possible. Maybe have the network which generates the answer be moderated by another network that assesses the truthiness of it. It's just a matter of priorities for the company designing the models.
- ben_w 2y ago
  It might be possible, but nobody knows for sure, because these models are rather more mysterious than their architecture suggests. > Maybe have the network which generates the answer be moderated by another network that assesses the truthiness of it. Like a GAN? Sometimes you can do that, but it seems not always. If this was simple and obvious, they'd have done it as soon as the first one was interesting-but-wrong.
- jobs_throwaway 2y ago
  Especially in a limited domain like grade school math, it seems entirely plausible that we can have models in very short order that ~never hallucinate. There's no external dependencies and the problem space is extremely well-defined and constrained. Much, much, much easier than making something like Chat-GPT never hallucinate

arresin 2y ago

Or when you tell it you think the answer is something else it agrees with you and apologies that yes, now it can see, 2+2 does equal 5.

neuralnetes-COO 2y ago

That is the entire purpose of LLMOps. Provide guardrails to prevent hallucination and ensure precise control of GenAI output.

tayo42 2y ago
How can you tell what's true or not?
- neuralnetes-COO 2y ago
  You have to develop your own QA methods to ensure output is exactly what you want.

falcor84 2y ago

Neither can you easily notice that when a human teacher/tutor does that. At least with math, you're able to try it yourself and see if it works. I've had many cases where following through on a misleading explanation by a teacher/book actually ended up leading to me better retaining the topic.

xscott 2y ago

That's absolutely a bug that needs to be fixed, but I think it's possible. Maybe have the network which generates the answer be moderated by another network that assesses the truthiness of it. It's just a matter of priorities for the company designing the models.

ben_w 2y ago
It might be possible, but nobody knows for sure, because these models are rather more mysterious than their architecture suggests. > Maybe have the network which generates the answer be moderated by another network that assesses the truthiness of it. Like a GAN? Sometimes you can do that, but it seems not always. If this was simple and obvious, they'd have done it as soon as the first one was interesting-but-wrong.
jobs_throwaway 2y ago
Especially in a limited domain like grade school math, it seems entirely plausible that we can have models in very short order that ~never hallucinate. There's no external dependencies and the problem space is extremely well-defined and constrained. Much, much, much easier than making something like Chat-GPT never hallucinate