3 ms·
>but it is to make them choose more non default pathways Imagine you as a human are working on writing some code, but at the end of every hour, you lose memory
by ActorNightly 2y ago
>but it is to make them choose more non default pathways
Imagine you as a human are working on writing some code, but at the end of every hour, you lose memory of what happened in the first 10 minutes of the current hour, as well as any work that you have done. Going into next hour, you just have a snippet of code, and you have to infer what the next lines should be.
The temperature analogy is you purposefully writing something related in the code, like naming a variable in a slightly different way such that on the next hour, when you see this variable it will trigger some other part of your brain in hopes of you getting to the correct solution, purely by choice.
Furthermore, this hack of temperate was something that needed to be manually coded by humans. A model that could reason would not need those types of hacks.
- mewpmewp2 2y agoI don't understand how it relates to temperature? Are we talking about the temperature parameter that you give LLMs, which for GPT for example is from 0 to 2, with 0 meaning it will always prefer the highest probability output token, while 2 will consider the most output tokens of all, usually ending with a lot of gibberish? E.g. if I write "I have a cat and a " It would have highest probability of picking a word "dog" next, so temperature 0 means it will pretty much always pick dog. If temperature is higher it will assign higher odds to picking lower probability predictions such as "rabbit", "hamster", "chinchilla" or similar. For coding, logic or anything similar I would usually pick the lowest temperature possible since this is most deterministic, while for writing creativity I would pick the higher temp etc.
- ActorNightly 2y agoIm saying temperature is a hack to make the models actually produce real answers.
- mewpmewp2 2y agoBut they can get also real answers even if you have the temperature setting as 0, where it will always pick the highest scoring token?
- buneskamin 2y agoI think he's saying if you set temp to 0 and answers become deterministic, it will appear that the model is just memorising and reciting. The randomness is a hack that 'forces' the model to generalise by deliberately knocking it off the track of the most probable next token