34 ms·
Can't I just parallelize my LLM calls myself for each question?
by satvikpendem 8d ago
Can't I just parallelize my LLM calls myself for each question?
- orbital-decay 8d agoYou can. It will be expensive, slow, and less reliable than a specialized model.
- zwily 8d agoAnything you can do in Jev can be done with an LLM at much greater cost and latency.
- mmnfrdmcx 8d agoAgree, except the probabilities for outcomes in the structured output. I don't think you can get those for most frontier LLMs (logprobas). You can get it for open source models but not frontier LLMs.
- esafak 8d agoThat number is a big deal, assuming it is well calibrated. Did they talk about calibration?
- Matticus_Rex 8d agoI've seen them talk about it a bit on Twitter -- it seems to be fairly well-calibrated in general, but obviously you need to test it on your use case and dial it in comparison with known data for best results.
- shados 6d agoYup. LLMs can do almost anything. Can they do it at the speed, cost and confidence of a model like Jev is a different story. In Jev output tokens are straight up free because it's not doing text token generations.