3 ms·
Anything you can do in Jev can be done with an LLM at much greater cost and latency.
by zwily 15d ago
Anything you can do in Jev can be done with an LLM at much greater cost and latency.
- mmnfrdmcx 15d agoAgree, except the probabilities for outcomes in the structured output. I don't think you can get those for most frontier LLMs (logprobas). You can get it for open source models but not frontier LLMs.
- esafak 15d agoThat number is a big deal, assuming it is well calibrated. Did they talk about calibration?
- Matticus_Rex 15d agoI've seen them talk about it a bit on Twitter -- it seems to be fairly well-calibrated in general, but obviously you need to test it on your use case and dial it in comparison with known data for best results.