3 ms·
For production systems the considerations are: - how often does it error? - how bad are the errors? - how tolerable are the errors? - how detectable are the
by somewhereoutth 3y ago
For production systems the considerations are:
- how often does it error?
- how bad are the errors?
- how tolerable are the errors?
- how detectable are the errors?
- how recoverable are the errors?
For example, a pocket calculator that occasionally was out at the 3rd decimal place might do much more damage than one that quite regularly returned NaN.
LLMs error both regularly and badly, so detectability and recoverability are going to be crucial for useful deployment in systems outside of those that have a high tolerance for errors (e.g. algorithmic feeds).
- intended 3y agoYes. If you want to make something that works, your life is figuring out evaluation and LLM ops. At that point, you know its not thinking, its doing token prediction.