3 ms·
I don’t think the public o1 can search the internet yet, unlike 4o. In principle it could know that something is more commonly cited based on its training data.
by sd9 2y ago
I don’t think the public o1 can search the internet yet, unlike 4o. In principle it could know that something is more commonly cited based on its training data. But it could also just be hallucinating.
- diggan 2y ago> In principle it could know that something is more commonly cited based on its training data Could it? Without explicit training for that, how would it be expected to know it has to be able to count occurrences of something?
- sd9 2y agoI think it would be more vibes based - commonly occurring things would be reinforced more in the weights. Rather than it explicitly counting the number of occurrences.
- diggan 2y agoSo the probabilities would be skewed towards something, but unless the model could somehow count/infer its own weights, I don't see how it could "introspect" to see if something is more common than something else.
- freehorse 2y ago> it could know that something is more commonly cited based on its training data No there is no such concept or way to do something like that. LLMs do not have such kind of meta-knowledge over their training data or weights. But there could be explicit mentions about this on their training data and they could pick on that and that is probably the simplest explanation.
- famouswaffles 2y ago>LLMs do not have such kind of meta-knowledge over their training data or weights. Not sure this is a claim that can be confidently made. https://arxiv.org/abs/2309.00667 https://arxiv.org/abs/2309.00667 https://x.com/flowersslop/status/1873115669568311727?t=eBMbK-FOvEn1ghqkaqZrHg&s=19 https://x.com/flowersslop/status/1873115669568311727?t=eBMbK...