4 ms·
What they're citing isn't what the LLM "scraped", it's what the retrieval model fed to the LLM. You're not guaranteed that it's what it actually used to give yo
by make3 3y ago
What they're citing isn't what the LLM "scraped", it's what the retrieval model fed to the LLM. You're not guaranteed that it's what it actually used to give you the output, and it's also definitely not all the text that it used to get appropriate knowledge to generate the answer, as this is split over whatever millions of examples for the language and for human language in a non human-understandable way
- pbhjpbhj 3y agoA couple of times I've had the reference not include the detail being mentioned in the foregoing paragraph; the citations are still highly relevant, but it wasn't quite what I expected.
- slowhadoken 3y agoI've heard this coldtake before but OpenAI's source code isn't open to academic scrutiny. So I don't understand why some people are so confident about how it works. It's certainly not magic and Phind seems to be capable of it citation.
- deleted 3y ago[deleted]
- make3 3y agoIt's transformer based language-modelling 101, not really a take, just stating facts. It's highly unlikely Phind has completely fundamentally changed all the exact same problems that the whole field is working on simultaneously, single-handedly, in a purely novel way. It's just how transformers work.
- slowhadoken 3y agoPhind appears to be doing it though. LLMs are stochastic parrots, I don’t see a radical difference. Input goes in, output comes out. Neural network aren’t magic, they’re a complex function. 1 node or a billion we can track the data that’s changing the weights inside the network.
- make3 3y agoI do this for a living
- gnaritas99 3y ago[dead]