4 ms·

> why shouldn't LLMs Because they're non-deterministic.

• sega_sai 1y ago
  It is one thing that you are getting results that are samples from the distribution ( and you can always set the temperature to zero and get there mode of the distribution), but completely another when the distribution changes from day to day.
• NiloCK 1y ago
  What? No they aren't. You get different results each time because of variation in seed values + non-zero 'temperatures' - eg, configured randomness. Pedantic point: different virtualized implementations can produce different results because of differences in floating point implementation, but fundamentally they are just big chains of multiplication.
  • plaguuuuuu 1y ago
    On the other hand, responses can be kind of chaotic. Adding in a token somewhere can sometimes flip things unpredictably.
  • int_19h 1y ago
    But experience shows that you do need non-zero temperature for them to be useful in most cases.