3 ms·
Have any of the leading models been trained on that kind of corpus? Could a corpus like that even be constructed without vociferous and immediate dispute about
by swatcoder 2y ago
Have any of the leading models been trained on that kind of corpus? Could a corpus like that even be constructed without vociferous and immediate dispute about what amounts to "largely true"? Would a corpus like that have enough suitable data to generate conversational output that feels satisfying?
I thought that most/all of these big models are relying on a much much larger and more diverse body of content. Is that wrong?
- stale2002 2y ago> Would a corpus like that have enough suitable data to generate conversational output that feels satisfying? Yes? The point is that we can use LLMs right now. They mostly work. That is the empirical evidence that is immediately verifiable right now. This is not a hypothetical. LLMs mostly work.
- swatcoder 2y agoHuh? I don't understand how this engages with what I wrote or what I was responding to. Yes, it's very very obvious that the conversational LLM's we engagw with mostly work for conversational UX. The discussion is about the above commenter's reference to "largely true" data. I'm not aware of any leading LLM built on something that could be characterized that way, but I'm genuinely curious to hear otherwise.
- deleted 2y ago[deleted]
- rsynnott 2y agoSee Tim Bray’s example; the LLM very much did not mostly work once it was talking about something that he knew about; most of what it said was nonsense. I suspect a _huge_ driver of the hype is that people are not usually experts on the things that they ask these about, so they see a magical oracle rather than a nonsense generator.
- CuriouslyC 2y agoPhi2 was trained on a very highly controlled corpus.
- kergonath 2y ago> Have any of the leading models been trained on that kind of corpus? Does such a corpus even exist? Maybe a bunch of Mathematics papers, and even then... To build a corpus like that would mean trusting the human gatekeepers or curators, which just shifts the problem, because humans are also prone to this sort of mistake.