3 ms·
One is their state-of-the-art model, the other one's the best model they can run at scale and speed people expect from a search engine.
by input_sh 1y ago
One is their state-of-the-art model, the other one's the best model they can run at scale and speed people expect from a search engine.
- cwalv 1y agoThat sounds reasonable, and I don't doubt that it's part of the reason. Still, iiuc the solution to hallucination is that they can essentially train the model to recognize when it "doesn't know", and to say so in that case rather than just puke back the highest probability BS. I.e. it's a training time factor, not inference time, so it's not a fundamental cost issue, but more about priorities.