4 ms·

Yes, this happens by design.

• oblio 2y ago
  Interesting, why? Is there no theoretical way to have stable models? Or some kind of executive decision?
  • H8crilA 2y ago
    It's just how this sort of a model works in inference. You can set the temperature to zero, then you'll get mostly constant outputs (but not quite, since as far as I can tell nobody is working on determinization of such systems, and it is not really deterministic).
    • oblio 2y ago
      I'm going to have to dust off some ancient math... I think LLMs are based on neuronal networks which are based on regression models which are based on systems of differential equations for which last time I studied this stuff, we only had probabilistic solutions? It's been a bunch of decades since I did anything even remotely related to these areas (and I was never good at math).
      • H8crilA 2y ago
        Check out 3blue1brown, they have a series on transformers. Very approacheable. The randomness is very explicit in inference.