2 ms·
Couldn't you just up the temperature/change some other parameter to get it to be more random/"creative"? It wouldn't be active/intentional randomness/novelty li
by diggan 2y ago
Couldn't you just up the temperature/change some other parameter to get it to be more random/"creative"? It wouldn't be active/intentional randomness/novelty like what a human would do, but at least it shouldn't generate exactly the same naming.
- gwern 2y agoNo. The collapse is not as simple as simply shifting down most of the logits, so ramping up the temperature does little until outputs start degenerating.