5 ms·

Can you explain what temperature is, in this context? I don't know the terminology

by kldx 4y ago

drexlspivey 4y ago
High temperature picks more safe options when generating the next word while low temperature makes it more “creative”
- LeoPanthera 4y ago
  This isn't true. It has nothing to do with "safe" options. It controls output randomness, and you actually want HIGH temps for "creative" work.
LeoPanthera 4y ago
It controls how predictable the output is. For a low "temperature", input A always, or nearly always, results in output B. For a high temperature, the output can vary every run.
kenjackson 4y ago
Temperature indicates how probabilistic the next word/term will be. If temperature is high, then given the same input, it will output the same words. If the temperature is low, it will more likely output different words. When you query the model, you can specify what temperature you want for your responses.
- kgwgk 4y ago
  > If temperature is high, then given the same input, it will output the same words. If the temperature is low, it will more likely output different words. The other way around. Think of low temperatures as freezing the output while high temperatures induce movement.
  - kenjackson 4y ago
    Thank you. I wrote the complete opposite of what was in my head.
  - samstave 4y ago
    So many better terms could have been used ; Adjacency, Velocity, Adhesion, etc But! if temp denotes a graphing in a non-linear function (heat map) then it also implies topological, because temperature is affected by adjacency - where a topological/toroidal graph is more indicative of the selection set?
    - andrepd 4y ago
      Temperature here is used in the sense of statistical physics: a parameter that affects the probability of each microstate.
    - machina_ex_deus 4y ago
      The term temperature is used because they are literally using Boltzmann's distribution from statistical mechanics: e^(-H/T) where H is energy (Hamiltonian), T is temperature. The probability they give to something of score H is just like in statistical mechanics, e^(-H/T) and they divide by the partition function (sum) similarly to normalize. (You might recognize it with beta=1/T there)
    - skygazer 4y ago
      Right, it’s not termed to be understood from the user perspective, a common trait in naming. It increases the jumble within the pool of next possible word choices, as heat increases Brownian motion. That’s the way I think of it, at least.
- leereeves 4y ago
  > If temperature is high, then given the same input, it will output the same words. It's the other way 'round - higher temperature means more randomness. If temperature is zero the model always outputs the most likely token.
ayewo 4y ago
This highly upvoted article [1][2] explained temperature: But, OK, at each step it gets a list of words with probabilities. But which one should it actually pick to add to the essay (or whatever) that it’s writing? One might think it should be the “highest-ranked” word (i.e. the one to which the highest “probability” was assigned). But this is where a bit of voodoo begins to creep in. Because for some reason—that maybe one day we’ll have a scientific-style understanding of—if we always pick the highest-ranked word, we’ll typically get a very “flat” essay, that never seems to “show any creativity” (and even sometimes repeats word for word). But if sometimes (at random) we pick lower-ranked words, we get a “more interesting” essay. The fact that there’s randomness here means that if we use the same prompt multiple times, we’re likely to get different essays each time. And, in keeping with the idea of voodoo, there’s a particular so-called “temperature” parameter that determines how often lower-ranked words will be used, and for essay generation, it turns out that a “temperature” of 0.8 seems best. (It’s worth emphasizing that there’s no “theory” being used here; it’s just a matter of what’s been found to work in practice. 1: https://news.ycombinator.com/item?id=34796611 https://news.ycombinator.com/item?id=34796611 2: https://writings.stephenwolfram.com/2023/02/what-is-chatgpt-doing-and-why-does-it-work/ https://writings.stephenwolfram.com/2023/02/what-is-chatgpt-...

Can you explain what temperature is, in this context? I don't know the terminology

by kldx 4y ago

Can you explain what temperature is, in this context? I don't know the terminology

drexlspivey 4y ago

High temperature picks more safe options when generating the next word while low temperature makes it more “creative”

LeoPanthera 4y ago
This isn't true. It has nothing to do with "safe" options. It controls output randomness, and you actually want HIGH temps for "creative" work.

LeoPanthera 4y ago

It controls how predictable the output is. For a low "temperature", input A always, or nearly always, results in output B. For a high temperature, the output can vary every run.

kenjackson 4y ago

Temperature indicates how probabilistic the next word/term will be. If temperature is high, then given the same input, it will output the same words. If the temperature is low, it will more likely output different words. When you query the model, you can specify what temperature you want for your responses.

kgwgk 4y ago
> If temperature is high, then given the same input, it will output the same words. If the temperature is low, it will more likely output different words. The other way around. Think of low temperatures as freezing the output while high temperatures induce movement.
- kenjackson 4y ago
  Thank you. I wrote the complete opposite of what was in my head.
- samstave 4y ago
  So many better terms could have been used ; Adjacency, Velocity, Adhesion, etc But! if temp denotes a graphing in a non-linear function (heat map) then it also implies topological, because temperature is affected by adjacency - where a topological/toroidal graph is more indicative of the selection set?
  - andrepd 4y ago
    Temperature here is used in the sense of statistical physics: a parameter that affects the probability of each microstate.
  - machina_ex_deus 4y ago
    The term temperature is used because they are literally using Boltzmann's distribution from statistical mechanics: e^(-H/T) where H is energy (Hamiltonian), T is temperature. The probability they give to something of score H is just like in statistical mechanics, e^(-H/T) and they divide by the partition function (sum) similarly to normalize. (You might recognize it with beta=1/T there)
  - skygazer 4y ago
    Right, it’s not termed to be understood from the user perspective, a common trait in naming. It increases the jumble within the pool of next possible word choices, as heat increases Brownian motion. That’s the way I think of it, at least.
leereeves 4y ago
> If temperature is high, then given the same input, it will output the same words. It's the other way 'round - higher temperature means more randomness. If temperature is zero the model always outputs the most likely token.

ayewo 4y ago

This highly upvoted article [1][2] explained temperature: But, OK, at each step it gets a list of words with probabilities. But which one should it actually pick to add to the essay (or whatever) that it’s writing? One might think it should be the “highest-ranked” word (i.e. the one to which the highest “probability” was assigned). But this is where a bit of voodoo begins to creep in. Because for some reason—that maybe one day we’ll have a scientific-style understanding of—if we always pick the highest-ranked word, we’ll typically get a very “flat” essay, that never seems to “show any creativity” (and even sometimes repeats word for word). But if sometimes (at random) we pick lower-ranked words, we get a “more interesting” essay. The fact that there’s randomness here means that if we use the same prompt multiple times, we’re likely to get different essays each time. And, in keeping with the idea of voodoo, there’s a particular so-called “temperature” parameter that determines how often lower-ranked words will be used, and for essay generation, it turns out that a “temperature” of 0.8 seems best. (It’s worth emphasizing that there’s no “theory” being used here; it’s just a matter of what’s been found to work in practice. 1: https://news.ycombinator.com/item?id=34796611 https://news.ycombinator.com/item?id=34796611 2: https://writings.stephenwolfram.com/2023/02/what-is-chatgpt-doing-and-why-does-it-work/ https://writings.stephenwolfram.com/2023/02/what-is-chatgpt-...