4 ms·
How can Claude "know" whether something "is unlikely to be found more than once or twice on then internet"? Unless there are other sources that explicitly say "
by armchairhacker 2y ago
How can Claude "know" whether something "is unlikely to be found more than once or twice on then internet"? Unless there are other sources that explicitly say "[that thing] is obscure". I don't think LLMs can report if something was encountered more/less often in their training data, there are too many weights and neither us nor them know exactly what each of them represents.
- brulard 2y agoI believe Claude is aware if information close to the one retrieved from the vector space is scarce. I'm no expert, but i imagine it makes a query to the vector database and get the data close enough to places pointed out by the prompt. And it may see that part of the space is quite empty. If this is far off, someone please explain.
- th0ma5 2y agoI think good, true, but rare information would also fit that definition so it'd be a shame if it discovered something that could save humanity but then discounted it as probably not accurate.
- deleted 2y ago[deleted]
- rf15 2y agoI wonder if that's the case - the prompt text (like all text interaction with LLMs) is seen from "within" the vector space, while sparcity is only observable from the "outside"
- deleted 2y ago[deleted]
- furyofantares 2y agoI think it could be fine tuned to give it an intuition, like how you or I have an intuition about what might be found on the internet. That said I've never seen it give the response suggested in this prompt and I've tried loads of prompts just like this in my own workflows and they never do anything.
- GaggiX 2y agoI thought the same thing, but when I test the model on like titles of new mangas and stuff that were not present in the training dataset, the model seems to know of not knowing. I wonder if it's a behavior learned during fine-tuning.
- viraptor 2y agoLLMs encode their certainty enough to output it again. They don't need to be specifically trained for this. https://ar5iv.labs.arxiv.org/html/2308.16175 https://ar5iv.labs.arxiv.org/html/2308.16175
- acchow 2y agoYes, this is achieved through a meta process > By sampling multiple responses from the LLM and considering the one with the highest confidence score, we can additionally obtain more accurate responses from the same LLM, without any extra training steps > Our proposed LLM uncertainty quantification technique, BSDetector, calls the LLM API multiple times with varying prompts and sampling temperature values (see Figure 1). We expend extra computation in order to quantify how trustworthy the original LLM response is The data is there, but not directly accessible to the transformer. The meta process enables us to extract it
- fnordpiglet 2y agoI’d also note this isn’t confidence in the answer but in the token prediction. As LLMs have no conceptual “understanding,” they likewise have no computable confidence in accuracy of the correctness of their answers as we understand correctness. While certainly token confidence can be a proxy it’s not a substitute.
- adastra22 2y agoGood luck defining “understanding” in a way that lets you say LLMs don’t understand but humans do. At the end of the day we’re just a weighted neural net making seat of the pants confidence predictions too.
- acchow 2y ago> At the end of the day we’re just a weighted neural net making seat of the pants confidence predictions too. We might be. Or we might be something else entirely. Who knows?
- halJordan 2y agoit doesn’t know. it also doesn’t actually “think things through” when presented with “math questions” or even know what math is.
- dr_dshiv 2y agoHere, check it out— Claude sharing things that are only “once or twice on the internet” https://claude.site/artifacts/605e9525-630e-4782-a178-020e15fa6f00 https://claude.site/artifacts/605e9525-630e-4782-a178-020e15... It is funny, because it says things like “yak milk cheese making tutorials” and “ancient Sumerian pottery catalogs”. But that’s only the extremely rare. The things for “only once or twice” are “the location of jimmy Hoffa’s remains” and “banksy’s true identity.”
- ethbr1 2y agoI'm sure somewhere it crawled a forum of Googlewhack terms. https://en.m.wikipedia.org/wiki/Googlewhack https://en.m.wikipedia.org/wiki/Googlewhack
- ks2048 2y agoThis list of things that "only appear once or twice on the internet" makes no sense to me. Many are things that don't exist at all, depending on how you define it. A guess the best defense of Claude is that the question is a bit ill-defined.