4 ms·
I'm not sure what you mean by random sampling. If I sample a random SVG, I wouldn't expect to look like anything, let alone roughly like a unicorn.
by rcme 3y ago
I'm not sure what you mean by random sampling. If I sample a random SVG, I wouldn't expect to look like anything, let alone roughly like a unicorn.
- sigmoid10 3y agoI mean random sampling in the sense how autoregressive language models like GPT generate sequences using token probabilities. It's not a random svg, but the text sequence that is used to draw it suffers from inherent non-determinism in the underlying model.
- rcme 3y agoRelying on token probabilities seems like the exact opposite of random
- esafak 3y ago"Random" is a loose term. We're discussing a technical subject. Talk about the distribution the random numbers are sampled from.
- sigmoid10 3y agoThe neural network just generates a set of probabilities for all tokens. The actual next token is then sampled from this set, which is always random for T>0 (and in the case of GPT4 even for T=0, because of the way the model itself works).
- deleted 3y ago[deleted]