4 ms·
LLMs specifically are fine with random bits flipped for the results to be 'creative'.
by fooker 9mo ago
LLMs specifically are fine with random bits flipped for the results to be 'creative'.
- jedberg 9mo agoThat's not exactly how LLM temperature works. :). Also that's on inference, not training. Presumably these would be used for training, the latency would be too high for inference.
- fooker 9mo agoIt doesn't work like that, but it can. Latency would be fine for inference, this is low earth orbit, that is about 25ms optimistically. Well within what we expect from our current crop of non local LLMs.