5 ms·

I mean you don't have to. With 0 temp, the only source of non-determinism is really the associative math chaning on accumulation of the floating point multiplic

I mean you don't have to. With 0 temp, the only source of non-determinism is really the associative math chaning on accumulation of the floating point multiplication. That could be eleminated if we wanted to buy restricting the ordering.

bandrami 8mo ago
True but at that point you're not really using "an LLM" in the usual sense, you're using a very resource-hungry database.