3 ms·
Purely probabilistically, trying things off the beaten path is just a matter of higher LLM temperature. Turning up GPT-4's temperature is basically an expensive
by adtac 3y ago
Purely probabilistically, trying things off the beaten path is just a matter of higher LLM temperature. Turning up GPT-4's temperature is basically an expensive /dev/urandom pipe today, but I don't see any fundamental reason why LLMs can't catch up. Maybe all it takes is tinkering with how temperature is calculated.
- ben_w 3y agoYou've missed the metaphor, I think. Higher temperatures will make it "more creative" for lack of a better term, but there's a lot of specialist knowledge it doesn't have and which you can't give it just by twiddling a dial. It has a massive speed advantage that lets it read the whole internet, but it's dumb enough that it also needs to, and when even that doesn't give it enough examples, it's like asking me to improvise Latin based on what I recognise of the loanwords in English.
- jwilber 3y agoNo - temperature is a sampling measurement from a distribution. If you adjust temperature, you still ultimately pull from the same distribution. Parent is saying the advantage is in pulling ideas from outside that encoded distribution.