28 ms·

What's the cost (broadly speaking, not in your specific case) of doing the same work an LLM would have done without the LLM though?

• robrenaud 7d ago
  In the Jev use case, LLMs are horribly uncalibrated. In general, they will not produce good probability estimates. Their generality also comes with a latency/computation costs.
  • jmalicki 7d ago
    For the Jev use case for LLMs, do you mean having the LLM produce a probability as text?