4 ms·

You're not thinking about this probabilistically. This isn't about "correct" and "incorrect". There is an entire universe between those two poles with regards t

You're not thinking about this probabilistically. This isn't about "correct" and "incorrect". There is an entire universe between those two poles with regards to most kinds of knowledge. Facts are a specific kind of knowledge, and frankly, not that important. It's more important to know why the Civil War was fought than to know the exact dates and locations of each battle. I've heard people say they wouldn't use ChatGPT even if there was "only a 1 in a billion chance that it made their bank account details public"... May I introduce you to some very real probabilities: https://oddsofdying.com https://oddsofdying.com If you are living in America there is a 1 in 615 chance that your cause of death will be the result of an automobile accident. So yes, it is unlikely that we will ever create a tool that can answer with 100% confidence. It is also unlikely that a manufacturing process will result in a widget that conforms to allowed tolerances 100% of the time. However, in manufacturing this is understood. A defect of 3.4 per million widgets is considered an incredibly high process capability. These tools are being made more reliable everyday. Please have a realistic goal in mind. Edit: Well I've learned this much. Many of you are not Beyesians!

sarchertech 4y ago
I asked it 3 questions about distributed systems, 2 of the answers were so fundamentally wrong that they would lead to a complete misunderstanding of the subjects, and if you tried to implement systems based on that understanding they would be broken in hard to detect ways. The problem is that an LLM can only be as reliable as its training material. For example I wrote a blog post on the 2 generals problem. In the comments there is more text from inexperienced people asserting that a solution exists than there is text from my original article. An LLM trained on that article and comments will always be wrong. LLMs can be trained on smaller vetted training sets sure, but they also currently require massive amounts of data and there’s no guarantee that just waiting a few months or years will improve reliability enough to fix these issues.
- williamcotton 4y ago
  That's a very small sample size. Also, the scope of that question is rather broad. There are a number of techniques you could try to improve your results. But I get the distinct feeling that you are not interested in improving your results as you have a motivation to "prove" something about LLMs.
  - sarchertech 4y ago
    I have no strong interest either way. I played with chatgpt with no expectations going in, and I’ve played with it for many hours since. I have found it severely lacking for many purposes which people have said it’s suited. That’s it.
  - goatlover 4y ago
    Yes, but you have to know to improve the results, and the techniques don't alway work. I asked for a table of sports results from a specific date in the past that's easy to lookup, and it gave me wrong results. But they looked plausible. When I asked it about specific result, it gave me a completely misleading answer, but one that would be plausible to someone who doesn't know the sport very well. After informing it of the wrong answer, it corrected itself by giving a different wrong answer. I tried that a couple times with the same result. This becomes a problem when people start to treat LLMs as authoritative on facts.
- unyttigfjelltol 4y ago
  Isn't part of the problem that an LLM will complete a statement with words that form a kind of statistical midpoint of the things that might come next. But when we're dealing with specific things in the real world, the true way to complete a sentence will not be that kind of statistical function, in fact it only reliably will follow from actually knowing the answer. So, it's sort of like driving a car based not on a complete map of the area, but a partial map where the missing pieces are filled in from statistical averages of things that appear generally in the world-- but not necessarily in that location. We all may be impressed at how close the predictions come to reality, but the only truly reliable thing is that those predictions, measured with precision, will be wrong. And if you rely on them without verifying them yourself, you might have a serious problem.
  - williamcotton 4y ago
    I’m going to refrain from epistemological instruction but I would recommend you start with the Greeks and at least cover Kant and the analytical philosophers that followed. The “true” way to answer a proposition is not even remotely as simple as you’ve described.
    - unyttigfjelltol 4y ago
      Well, yes, but ... the fact remains an LLM is not equipped to grapple with truths.
    - goatlover 4y ago
      Or we could just deflate "truth" for this particular discussion, since what matters is whether predicting the next word works well enough without knowledge of the world. Pragmatically speaking, that's what matters.
d0mine 4y ago
Consider why chatgpt is banned on Stackoverflow despite the median quality already being poor there (because its answers/questions are even worse than that; the more answers from chatgpt, the worse the site)
- williamcotton 4y ago
  The irony here is that I get more help from ChatGPT than Stack Overflow. I suspect I'm better at using the tool than most people. So if most people are not good at using the tool and then submitting their poor results to Stack Overflow, I can see the problem.

sarchertech 4y ago

I asked it 3 questions about distributed systems, 2 of the answers were so fundamentally wrong that they would lead to a complete misunderstanding of the subjects, and if you tried to implement systems based on that understanding they would be broken in hard to detect ways. The problem is that an LLM can only be as reliable as its training material. For example I wrote a blog post on the 2 generals problem. In the comments there is more text from inexperienced people asserting that a solution exists than there is text from my original article. An LLM trained on that article and comments will always be wrong. LLMs can be trained on smaller vetted training sets sure, but they also currently require massive amounts of data and there’s no guarantee that just waiting a few months or years will improve reliability enough to fix these issues.

williamcotton 4y ago
That's a very small sample size. Also, the scope of that question is rather broad. There are a number of techniques you could try to improve your results. But I get the distinct feeling that you are not interested in improving your results as you have a motivation to "prove" something about LLMs.
- sarchertech 4y ago
  I have no strong interest either way. I played with chatgpt with no expectations going in, and I’ve played with it for many hours since. I have found it severely lacking for many purposes which people have said it’s suited. That’s it.
- goatlover 4y ago
  Yes, but you have to know to improve the results, and the techniques don't alway work. I asked for a table of sports results from a specific date in the past that's easy to lookup, and it gave me wrong results. But they looked plausible. When I asked it about specific result, it gave me a completely misleading answer, but one that would be plausible to someone who doesn't know the sport very well. After informing it of the wrong answer, it corrected itself by giving a different wrong answer. I tried that a couple times with the same result. This becomes a problem when people start to treat LLMs as authoritative on facts.
unyttigfjelltol 4y ago
Isn't part of the problem that an LLM will complete a statement with words that form a kind of statistical midpoint of the things that might come next. But when we're dealing with specific things in the real world, the true way to complete a sentence will not be that kind of statistical function, in fact it only reliably will follow from actually knowing the answer. So, it's sort of like driving a car based not on a complete map of the area, but a partial map where the missing pieces are filled in from statistical averages of things that appear generally in the world-- but not necessarily in that location. We all may be impressed at how close the predictions come to reality, but the only truly reliable thing is that those predictions, measured with precision, will be wrong. And if you rely on them without verifying them yourself, you might have a serious problem.
- williamcotton 4y ago
  I’m going to refrain from epistemological instruction but I would recommend you start with the Greeks and at least cover Kant and the analytical philosophers that followed. The “true” way to answer a proposition is not even remotely as simple as you’ve described.
  - unyttigfjelltol 4y ago
    Well, yes, but ... the fact remains an LLM is not equipped to grapple with truths.
  - goatlover 4y ago
    Or we could just deflate "truth" for this particular discussion, since what matters is whether predicting the next word works well enough without knowledge of the world. Pragmatically speaking, that's what matters.

d0mine 4y ago

Consider why chatgpt is banned on Stackoverflow despite the median quality already being poor there (because its answers/questions are even worse than that; the more answers from chatgpt, the worse the site)

williamcotton 4y ago
The irony here is that I get more help from ChatGPT than Stack Overflow. I suspect I'm better at using the tool than most people. So if most people are not good at using the tool and then submitting their poor results to Stack Overflow, I can see the problem.