4 ms·

If they could do that they already would have. Frankly, I think you're massively underestimating the difficulty in what you're describing (assuming what you're

If they could do that they already would have. Frankly, I think you're massively underestimating the difficulty in what you're describing (assuming what you're describing would actually work, and I'm not sure it would). To do what you're suggesting, you'd need a human to evaluate and feed into one of these LLMs every known fact. Good luck with that. And that's ignoring the issue of maintaining that model as those facts change.

CuriouslyC 4y ago
That's not true at all. Right now there's a tension in ML between hoovering up as much data as possible (since more data, even if it's bad, tends to improve the model in general even if it poisons certain parts of it) and curating/annotating data sets to improve answer quality and consistency. Data set annotation is expensive and time consuming, and it's a hard sell to shareholders to spend a billion dollars on data set annotation when there's still more data that can be cheaply obtained, and the liability or cost for producing incorrect answers is intangible or low. After Google's massive share hit from Bard's exoplanet flub though, I don't think the costs are quite as intangible as they were, and I expect accuracy will become a new push in models over the next few years.
- deleted 4y ago
  [deleted]
- BaseballPhysics 4y ago
  > That's not true at all. I actually made three statements in my comment and you only disputed the first one. Yours is a good counter-argument to my first statement, and it is a valid rebuttal, I concede the point! But I maintain the scale of the problem, and the difficulty of maintenance, will make manual curation impossible, while automated "fact generation" is a technology that doesn't exist today.
  - CuriouslyC 4y ago
    Eh, when we have trillion dollar companies whose core product is AI commonly used for question/answer tasks, and query accuracy directly impacts the bottom line, I don't think spending 10 billion a year on annotation is a big ask. It might not be perfect, but as long as your query responses are significantly more accurate than the competition you'll end up as Google to their Altavista. Additionally, you don't have to manually curate the entire data set, only a large enough chunk to predict accuracy annotations for the stuff you haven't curated yet. Facts that change over time shouldn't be hard to identify if you periodically do cross validation on existing annotations to pick out ones where the predicted accuracy differs significantly from the annotation. There are a lot of process and engineering questions still to be answered in terms of building high quality AI products, but I don't think there are hidden dragons that are going to lead to another AI winter. There will be hiccups and snafus, but AI is the real deal.

CuriouslyC 4y ago

That's not true at all. Right now there's a tension in ML between hoovering up as much data as possible (since more data, even if it's bad, tends to improve the model in general even if it poisons certain parts of it) and curating/annotating data sets to improve answer quality and consistency. Data set annotation is expensive and time consuming, and it's a hard sell to shareholders to spend a billion dollars on data set annotation when there's still more data that can be cheaply obtained, and the liability or cost for producing incorrect answers is intangible or low. After Google's massive share hit from Bard's exoplanet flub though, I don't think the costs are quite as intangible as they were, and I expect accuracy will become a new push in models over the next few years.

deleted 4y ago
[deleted]
BaseballPhysics 4y ago
> That's not true at all. I actually made three statements in my comment and you only disputed the first one. Yours is a good counter-argument to my first statement, and it is a valid rebuttal, I concede the point! But I maintain the scale of the problem, and the difficulty of maintenance, will make manual curation impossible, while automated "fact generation" is a technology that doesn't exist today.
- CuriouslyC 4y ago
  Eh, when we have trillion dollar companies whose core product is AI commonly used for question/answer tasks, and query accuracy directly impacts the bottom line, I don't think spending 10 billion a year on annotation is a big ask. It might not be perfect, but as long as your query responses are significantly more accurate than the competition you'll end up as Google to their Altavista. Additionally, you don't have to manually curate the entire data set, only a large enough chunk to predict accuracy annotations for the stuff you haven't curated yet. Facts that change over time shouldn't be hard to identify if you periodically do cross validation on existing annotations to pick out ones where the predicted accuracy differs significantly from the annotation. There are a lot of process and engineering questions still to be answered in terms of building high quality AI products, but I don't think there are hidden dragons that are going to lead to another AI winter. There will be hiccups and snafus, but AI is the real deal.