3 ms·

LLMs are deterministic. We just sample the results, no? Also, no reason AI needs not be deterministic.

• TeMPOraL 2y ago
  > LLMs are deterministic. In theory, yes. In practice, parallelism combined with floating point math make current implementations fundamentally non-deterministic.
  • Tycho 2y ago
    Can you elaborate more on the parallelism aspect?
  • deleted 2y ago
    [deleted]
  • mensetmanusman 2y ago
    Not fundamentally, but extremely costly to keep track of everything. Still deterministic though.
• joquarky 2y ago
  Temperature cannot ever reach 0 (this causes a division error), so they are not deterministic.