9 ms·
What is Temperature in NLP?
- ivxvm 4y agoThat's really a great explanation. Short but detailed, rigorous and visual. Really hits the spot.
- ec109685 4y agoIs this part of why ChatGPT hallucinates? Lower probability responses sometimes "win" which forces the model to take the wrong path?
- albystein 4y agoI think hallucination is akin to encountering a problem on a test to which you don’t fully remember the right concept to solve it. You might attempt the problem with whatever little knowledge you can recall, but the answer is ultimately incorrect. Likewise, these LLMs exhibit such behavior when they hallucinate. I think what distinguishes the LLM’s hallucinations from humans is the confidence level. A human who is unsure of their answer might tell you that they’re not certain or they don’t know. LLMs like chatgpt, however, will just confidently make stuff up.
- geraneum 4y agoHigher probably responses are not necessarily “truthful” or “logical” in any given context. Here* you can see an example of what happens when you always choose the highest probability. [*] https://writings.stephenwolfram.com/2023/02/what-is-chatgpt-doing-and-why-does-it-work/ https://writings.stephenwolfram.com/2023/02/what-is-chatgpt-...
- muzani 4y agoIt can halluciate at low temp too. e.g. mice don't necessarily eat cheese; it's just a common stereotype of what mice eat. High temp is more likely to lead it into ranting on an unrelated topic. You ask it a question on what the mouse eats. It answers the question and then starts saying what lizards eat, or goes on a rant about food chains.
- recuter 4y agoMy favorite prompt to try so far to play with this effect is: Picture yourself in a boat on a river With tangerine trees and marmalade skies It might veer off into a scenario where you're on a romantic boat trip or talk about how it is its favorite Beatles song and how it remembers them going on Ed Sullivan. :) The temp affects the "color" of the next word which sends it on a tangent just like a person -- especially somebody with ADHD or playing an improve game as opposed to lip reading, for example.
- wodenokoto 4y agoNo. I'd say everything the model does is a hallucination. Some writers tend to split ChatGPT oputput into "correct answers" and "Hallucinated answers". If this is what you are thinking when saying "Is this part of why ChatGPT hallucinates?", then no. The model never knows if what it is saying is correct. It has a probability about how likely the words are to follow eachother, and that is about it. The temperature changes how these probabilities are calculated, making otherwise less probable words more probable or vice versa. However, the model doesn't choose the most likely word every time. Instead it rolls a dice to see where it wants to go next. If Cheese is 80% and Cookie is 20% it won't necessarily choose Cheese. It rolls a die, and maybe the outcome lands on the less likely Cookie. The roll of the dice happen _after_ temperature has affected the calculation of probabilities. Maybe it is this roll you are thinking of as hallucination?
- logicallee 4y agoYou say it never knows if what it is saying is correct, but in my experience it tells you when it does and doesn't know about a concept. For example in one thread I had it give me some random words from linear algebra, then in another thread I combined these into the nonsense phrase "decomposed orthogonal eigenvector division" and asked it if it was familiar with this, and if so to define it. It said it wasn't familiar with it and asked for more information. I'd say that shows it knows what it doesn't know.
- detrites 4y agoWhile it can be customary to trivialise this, the reality is that no "intelligence" is sure if what it's saying may be correct or constitute any kind of "knowledge". At this level a human can't claim to arbiter fact versus fiction. Fundamentally, no one can be sure what they experience is not some kind of simulation, or a dreamlike state beneath some more real "reality". As such, assigning hierarchy to such a foundation lacking in tangible proof is arbitrary. The point being, unless it's possible to conclusively and infallibly prove you are not hallucinating, you can't lay claim to having knowledge an AI is hallucinating, or not.
- jnwatson 4y ago
- WithinReason 4y agoI think it hallucinates in scenarios where it either doesn't "remember" the answer or there is no answer. For example: since the text generation is sequential, based on context the highest probability continuation in a situation might be: "A famous paper that discussed this phenomenon is" However, when the network gets to this point it doesn't know the name of such a paper. Maybe it already knew that it doesn't know the paper's name but the above sentence was generated despite that, since everything else was lower probability or it got sampled due to higher temperature. In any case, it can't go back and delete the sentence so it must make up something. So it ends up going with the most plausible sounding (highest sample probability) continuation, which ends up being a "hallucination". Basically hallucinations are produced when the network "writes itself into a corner" due to the sequential nature of text generation.
- golol 4y agodefinetly not.
- kzrdude 4y agoMice in nature don't eat cheese. So even the most favoured response is a hallucination.
- qwertox 4y agoIn nature they also don't eat cookies then.
- denton-scratch 4y agoThey favour grains over cheese. Cookies and baguettes (even for non-French mice) work better than cheese in mousetraps. Peanut butter is widely recommended as the best mousetrap bait. I have had success with peanut butter.
- triyambakam 4y agoIs there any centralized vernacular? I know what a hallucination is, but I frequently come across many terms I need to dig into with surrounding context to figure out. Which works but is not efficient.
- karmasimida 4y agoSimple: low temp -> make large logits larger, more confident output, but diversity suffers high temp -> vice versa, diverse outputs, quality could suffer
- Zondartul 4y agoFrom what I've seen, low temp "only the most probable" output tends to be more repetetive and incoherent than medium-tempeature output, which is a bit of a paradox. I wonder why that is.
- geocar 4y agoLook up at the stars: You can only see a few of them. These functions are like really twisted/complex lenses and mirrors you might imagine we have hoisted into view so that now the sky is completely white (to our eyes) and we can see every star in the sky. Now without knowing the precise shape and position of that contraption, because all those stars really flicker at different rates, you can with a very fast camera recognise them by their flicker and note that the same star is actually visible in multiple "places" in the sky. You might if it is useful to you consider a star being "close" to many different stars in the sky -- as distorted by those lenses and mirrors -- simply by how long you watch the flicker. It is in this way, that I imagine "low temperatures" producing incoherence like the fast camera, not waiting long enough to see how many better (closer!) options there might be, seeing only the brightest (closest to us! cookie!) stars, and not waiting for all the other options (cheese, baguette, etc) to make into a longer exposure. Or the kinds of things you can meaningfully say after studying the Internet for (any random) week is probably going to be more repetitive and less coherent than the kinds of things you can meaningfully say after studying the Internet for 20 years.
- p1esk 4y agoI did not understand your explanation.
- f_devd 4y agoThe reason is that actual human-written text generally doesn't follow the most probable next word rule, but rather there are occasional 'decisions' made which make the text unique and therefore more interesting/coherent. There are other sampling methods which try to avoid this issue like "Locally Typical" and "Nucleus" sampling.
- taylorius 4y agoRoughly analogous process to the temperature parameter in simulated annealing, I think. Low temperature equates to a greedy algorithm (i.e. always take the most likely next word fragment) - higher temps. increase the probability of taking a lower probability fragment.
- denysvitali 4y agoBasically exploration vs exploitation
- albystein 4y agoI’m not sure if comparing it to the greedy algorithm is the correct way to think about it. Even after applying the temperature, if you use greedy decoding, you’ll still pick the output that has the highest probability. Where the concept of temperature is effective though is when you use a sampling method like sampling from a multinomial distribution, nucleus sampling, etc.
- windsignaling 4y agoThe analogy is apt. In SA, the temperature is used to make the distribution sharper or fatter. In this article, the temperature is used to make the distribution sharper or fatter.
- deleted 4y ago[deleted]
- lxe 4y agoFrom what I understand top_k is the number of words it generates to pick from, and the top_p is the percentage of the top words it will use as the next word, correct?
- albystein 4y agoI think it depends on the sampling/decoding method. Also, perhaps you could elaborate more on what you mean by top_p
- clbrmbr 4y ago> A higher temperature θ “excites” previously low probability outputs. I found this misleading, because the low probability outputs were already excited. Consider that the first softmax formula has no theta in the denominator, so it’s equivalent to theta=1 (max temperature). The way I explained temperature to a colleague last week was: temperature of one results in drawing from the posterior distribution, while temperature of zero gives the argmax that distribution. Intermediate temperatures get some blend between them. But now I see that that hand-wavy description doesn’t explain what happens to the runner ups to the argmax, which this site beautifully does.
- LudwigNagasena 4y agoThe metaphor is straightforward: higher temperature ~ higher entropy ~ less information ~ probabilities are further from certainty (0 and 1). I think the connection originally comes from Boltzmann machines and simulated annealing that were based off statistical physics.
- qwertox 4y agoAn explanation on what `logit` is, how it is obtained, would have been nice. Is it the count of occurrences of the word following "the mouse ate the"?
- drexlspivey 4y agoLogits are the raw predictions of the model that you then need to normalize to turn into probabilities (that sum to 1). You can do that is with the softmax function.
- spywaregorilla 4y agoMachine learning models don't do well with math that has hard bounds at 1 and 0 (probability). So we use the log of the odds ratio instead. This function goes from negative infinity to positive infinity. And is mappable back to 0 and 1. So as you get closer to infinity log odds, you get closer to 100% certainty that your answer is A. It's just ln(p/(1-p)) under the hood. https://en.wikipedia.org/wiki/Logit https://en.wikipedia.org/wiki/Logit
- abecedarius 4y agoYeah -- the choice of temperature corresponds to the base of the logarithm. When you think of probabilities in the log-odds representation, a bayesian update means adding an increment. https://arbital.com/p/bayes_log_odds/ https://arbital.com/p/bayes_log_odds/
- albystein 4y agothis is not what language models typically output as logits though. perhaps the term now is being misused and it’s causing a lot of confusion
- albystein 4y agoBasically, logits are the raw outputs of the last linear layer of a neural network just before the softmax(for multi-class classification tasks) or the sigmoid(for binary classification tasks) is applied to the outputs.
- mlazos 4y agoI get that temperature comes from the connection to physics but this seems like an overly complex explanation to a relatively simple formula. If you have a softmax and you multiply the input by some factor, if that factor forces the exponent to approach zero you’re approaching the uniform distribution, and if you make the exponent approach infinity you get the arg max. Curious if others feel the same way.
- rickdeckard 4y agoAre you suggesting that your explanation is more simple? (genuine question) I find "temperature" quite intuitive as a metaphor in that context. It doesn't even require me to think of the formula to understand "the higher the temperature, the more it will create a stew of the results"
- albystein 4y agoThis is exactly how I understand it. Although, I feel the thermodynamics analogy might be more intuitive to most people
- Havoc 4y agoAh the make it more psychopathic button
- deleted 4y ago[deleted]
- spywaregorilla 4y agoI feel like it would be have been a lot more intuitive calling it something like "Capriciousness" It's not "randomness". It's more like "willingness to choose less likely answers".
- malborodog 4y agoUnderrated comment; and I'm using this term to describe the effect of temperature in future.
- scottmsul 4y agoEveryone here seems totally lost on the physics connection. Suppose you have a box of atoms, each atom can be in one of two states, a low energy E1 and a high energy E2. If the box has a temperature T, then the probability that any atom is in state E1 is e^(-E1/kT) / [ e^(-E1/kT) + e^(-E2/kT) ], and similar for E2. As you lower the temperature most of the atoms gravitate towards the lower energy state E1, and as you raise the temperature they gravitate towards a 50/50 mix of E1 and E2.
- kgwgk 4y agoFun fact: not only you get closer to a 50/50 mix of E1 and E2 (increasing temperature) but if you continue to add energy to the box you will find yourself with more atoms in the E2 state than in the E1 state. The temperature will go from infinity to -infinity and as you keep adding energy you will approach zero temperature from the left (increasing temperature). The zero value is reached when the energy of the system can no longer be increased and all the atoms are in the E2 state.
- jbay808 4y agoThe most fun part of learning statistical mechanics was un-learning everything I had learned about temperature in high school!
- bobbylarrybobby 4y agoHow are you defining temperature? I assume it's not average kinetic energy of the particles. Is it that definition I learned once upon a time where T = d entropy /d energy? Is this a useful definition of temperature if it leads to this scenario?
- jbay808 4y agoYes, this is in accordance with the definition of temperature as d entropy / d energy, which is the more generally-applicable definition. It's easy to see from this that temperature becomes negative if adding more energy into the system causes the entropy to decrease. In most systems that doesn't happen because there are always an increasing number of higher-energy states that open up, but in a very carefully constrained system, the added energy forces the system into a smaller number of excited states away from a higher-entropy ground state. A real-world example of this is a laser's population inversion. A system at finite negative temperature is actually considered "hotter" than a normal system at any positive finite temperature; if you put them in thermal contact, heat will flow in the direction that increases entropy, which you get by taking energy out of the negative-temperature inverted system and adding it into the ordinary positive-temperature system. This increases the entropy of both systems. The definition where temperature is the "average kinetic energy of the particles" is a special case, and it only really works when that energy is evenly distributed over all degrees of freedom. For example, you wouldn't consider an icy comet to be at a high temperature just because it's moving quickly, even though its particles have a great deal of kinetic energy on average!
- a_c 4y agoTo me, ML is tending towards religion. Religion in the way of blackbox processing. Seems no one can reason what happens within the network, the number of parameters, the number of layer, the choice of temperature, etc. Or it is just my ignorance. While the enlightenment nudges mankind toward science, could it be that the scientific breakthrough of ML is to knock us back into pre-enlightened age? Per stephen wolfram [1] > It’s worth emphasizing that there’s no “theory” being used here; it’s just a matter of what’s been found to work in practice. https://writings.stephenwolfram.com/2023/02/what-is-chatgpt-doing-and-why-does-it-work/ https://writings.stephenwolfram.com/2023/02/what-is-chatgpt-...
- H8crilA 4y agoThe model knows not where it is, but it knows where it isn't. By subtracting where it is from where it isn't it obtains a difference, or gradient. Gradient is used to generate corrective commands to drive the model from a position where it was to a position where it wasn't.
- smartmic 4y agoYou might be interested in the writings of "Better Without AI", your comment immediately makes me think of this: https://betterwithout.ai/gradient-dissent#backpropaganda https://betterwithout.ai/gradient-dissent#backpropaganda
- pixl97 4y agoHow different is this from medicine? When problems begin to contain irreducible complexity human reasoning starts to break. We can't test every possible state to determine which state is the best (in human reasonable energy use and timeframes). When this occurs we use heuristics and throw something at the wall to see what works. Sometimes there just may not be a simpler theory of why something works. Or the 'simple' theory is still massively complex. Reality doesn't owe us an easy explanation.
- bflesch 4y agoHow can the author work for a company like TikTok with obvious chinese censorship?