3 ms·
I am glad that you think it an excellent point! I think that this might be a nice way of getting round my objection, but there is one worry, which is that X is
by eynsham 2y ago
I am glad that you think it an excellent point!
I think that this might be a nice way of getting round my objection, but there is one worry, which is that X is relative to a distribution on the questions we ask when we aren’t dealing with Encyclopedia Eric but with an LLM. I don’t actually use LLMs very much myself, partly out of arrogance and Luddite tendencies. But I suspect that the value of X for some sorts of questions (simple quiz questions, maybe?) and some LLMs (maybe not Google’s) will be high enough to end up in the murky case.
Of course, both you and I agree that there /is/ clearly a difference. I can see the attraction of appealing to the intuitive or pretheoretic notion of knowledge, since it’s a fairly straightforward way of stating the difference and it’s not obvious how else one might put it (I suppose ‘LLMs don’t explicitly think through the facts stored when deciding what to say’ is one way of putting it.)
I remember some time ago rather sleepily watching John Hawthorne talk about conditionals; my sole memory was of his banging on about ‘the little logician in the brain’ (I think the point was something like: some conditionals seem [in]felicitous in virtue of form because the little logician in the brain is reading them; others seem infelicitous because we examine them more closely and look e.g. at the referents involved, in which case e.g. Gricean considerations apply). One difference at least in the case of LLMs that makes sense to me is that there is no ‘little logician’ in LLMs.
- TeMPOraL 2y ago> X is relative to a distribution on the questions we ask when we aren’t dealing with Encyclopedia Eric but with an LLM. Assuming I understand what you mean here correctly, this should be the case for both LLMs and Encyclopedia Eric - there are topics Eric knows by heart (or thinks they know); there are specific phrases seared into his mind through sheer exposure during his life prior to becoming a living Encyclopedia. There are words he's used to, and exact synonyms he barely recognizes. All that means your chance of getting correct answer to your query depends, in complex and unknown to you way, on how you state it.
- eynsham 2y agoI think the thought experiment can be set up both ways. In one case Eric has a fixed probability of getting /any/ query right. (This might leave boosting open so we might want to gerrymander repeated queries out.) In the other this is relative to the distribution of queries.
- lukev 2y agoA relevant point to this is the notion of "System-1" vs "System-2" thinking. Somewhat dubious when applied to actual human psychology but I think a valid metaphor for how LLMs work: they are only capable of System-1 thinking; a single forward pass through the weights of intuition In my actual life, I don't trust my own System-1 thoughts: for anything important, I'm always going to engage System-2. And LLMs don't have a System-2. (I also agree that when dealing with LLMs the value of X is not a single value but a highly complex space depending on the nature of the question and the training data of the model. In my mind it does not change the epistemological equation, it just means that even the value of X itself is harder to "know", so this ambiguity can only ever make LLMs a less viable source of knowledge.)
- eynsham 2y agoI am not sure that the ambiguity has to work that way. The suggestion I am making is that if we fix the distribution on questions and the training data, we might (a) know the value of X in this specific case, (b) be able to ensure that it is fairly high. I’d say this is the murky case because on the fixed training data and query distribution X ≈ 1 and we know that even though we don’t know the value of X on other training data and other query distributions. I think that might be where the disagreement lies.