3 ms·
The prompt may be the same but the seed is different every time.
by koakuma-chan 1y ago
The prompt may be the same but the seed is different every time.
- biophysboy 1y agoCould you not cache the top k outputs given a provided input token set? I thought the randomness was applied at the end by sampling the output distribution.