3 ms·
Given that "The Pile" includes PubMed and NIH as data sources it would be unlikely to have GPT4 not use them at all. Even GPT3 uses Wikipedia which does have (m
by jprafael 3y ago
Given that "The Pile" includes PubMed and NIH as data sources it would be unlikely to have GPT4 not use them at all. Even GPT3 uses Wikipedia which does have (mostly) factual data with cited sources.
- ekianjo 3y agoProblem is half of the publications or even more is pure garbage that cannot be reproduced so there is that.
- tazjin 3y ago> Even GPT3 uses Wikipedia which does have (mostly) factual data with cited sources. There's a LOT of stuff on Wikipedia where the source is a link to some random, long article and it's unclear where exactly the referred to information is coming from. Gets significantly worse for any "hot" topic.
- kaliqt 3y agoIt's lossy though, it tries to remember everything and relate that to everything else. Performance would increase significantly if it were tuned for that use case.