3 ms·
The current training strategies for LLMs do not also simultaneously build knowledge databases for reference by some external system. It would have to take place
by andrewmcwatters 2y ago
The current training strategies for LLMs do not also simultaneously build knowledge databases for reference by some external system. It would have to take place outside of inference. The "knowledge" itself is just the connections between the tokens.
There is no way to tell you whether or not a trained model knows something, and not a single organization publishing this work is formally verifying falsifiable, objective training data.
It doesn't exist. Anything you're otherwise told is just another stage of inference on some first phase of output. This is also the basic architecture for reasoning models. They're just applying inference recursively on output.
- zby 2y agoWell - it does not need to 'know' anything - it just needs to generate the string "I don't know" when it does not have better connections.
- hatefulmoron 2y agoWhat does better connections in this context mean? To begin ranking the quality of connections, or "betterness", don't you need something approximating knowledge?