3 ms·

Anyone know somewhere someone dumb like me can "Ask an AI expert"? I want to ask, for example, how is it that an LLM when given the same prompt does not respon

Anyone know somewhere someone dumb like me can "Ask an AI expert"? I want to ask, for example, how is it that an LLM when given the same prompt does not respond in the same deterministic way? I guess I want to learn this stuff and should maybe follow one of those "write an LLM in an hour" type videos on YouTube.

rahimnathwani 3y ago
For this particular question, ask chatgpt how temperature affects llm softmax sampling. For other things, study using Karpathy's videos.
8note 3y ago
For that answer, you can refer to the 3blue1brown videos The llm model outputs a vector of probabilities for tokens, and the llm user picks a token from the most likely list using a random number
zozbot234 3y ago
> I want to ask, for example, how is it that an LLM when given the same prompt does not respond in the same deterministic way? You can control that in most systems with an inference-set parameter called "temperature". But setting the temperature as low as possible tends to lead to very low-quality answers - the system can't crawl out of some local optimum and ends up repeating itself over and over. Such answers may be "deterministic" but they're also not good.
zipfcharge 3y ago
It's because an LLM is essentially a probability matrix. You type a prompt, then it calculates what's the probability of getting a next word and so on, eventually forming a sentence. The probability learned is based on the training data. Because of the underlying probability model, it's not going to be 100% deterministic. Plus a model like ChatGPT purposefully have "temperature" parameter that will further add randomisation to the whole process. My answer is based on this paper if you're interested to read more: The Matrix: A Bayesian learning model for LLMs, https://arxiv.org/abs/2402.03175 https://arxiv.org/abs/2402.03175
- flopriore 3y ago
  Are there any ways to show the source of the information retrieved by the model? For instance, the LLM forms a sentence and it points to a stackoverflow answer with the same or similar content.
  - JKCalhoun 3y ago
    As I understand it, pretty sure that is impossible. When it is input a single datum, sure, trivial. As soon as it is fed a second one though the weights are already a kind of blend of the two tokens (so to speak).
    - spmurrayzzz 3y ago
      Its not impossible, but its definitely difficult. There is some overlap in the methods used to detect benchmark data contamination, though its not entirely the same thing. For the detection use case, you already know the text you're looking for and you are just trying to demonstrate that the model has "seen" the data in its training set. The challenge is proving that it is statistically improbable that the model could stochastically generate the same tokens without having seen them during training. Some great research exists in this area [1] and I expect much of it may be repurposed for black box attribution in the future (in addition to all the work being done in the mechanistic interpretability field) [1] https://arxiv.org/abs/2311.04850 https://arxiv.org/abs/2311.04850
throwawaymaths 3y ago
> how is it that an LLM when given the same prompt does not respond in the same deterministic way? In software (not in the model) here's literally a random number generator that picks from a weighted set of "next-token" choices that the model spits out. The selection process can have a series of knobs to manipulate the responses. If you want it to be deterministic (if you have direct access to the software) you can tell it to set "top-k = 1" or "temperature = 0.0" (depending on your software) and it will be deterministic. Usually the default settings are not for determinism, because for whatever reason the quality of the results tends to not be that good when you go fully d.
int_19h 3y ago
I found this to be a good start that explains things fairly methodically, but without losing the high-level perspective. https://writings.stephenwolfram.com/2023/02/what-is-chatgpt-doing-and-why-does-it-work/ https://writings.stephenwolfram.com/2023/02/what-is-chatgpt-...

rahimnathwani 3y ago

For this particular question, ask chatgpt how temperature affects llm softmax sampling. For other things, study using Karpathy's videos.

8note 3y ago

For that answer, you can refer to the 3blue1brown videos The llm model outputs a vector of probabilities for tokens, and the llm user picks a token from the most likely list using a random number

zozbot234 3y ago

> I want to ask, for example, how is it that an LLM when given the same prompt does not respond in the same deterministic way? You can control that in most systems with an inference-set parameter called "temperature". But setting the temperature as low as possible tends to lead to very low-quality answers - the system can't crawl out of some local optimum and ends up repeating itself over and over. Such answers may be "deterministic" but they're also not good.

zipfcharge 3y ago

It's because an LLM is essentially a probability matrix. You type a prompt, then it calculates what's the probability of getting a next word and so on, eventually forming a sentence. The probability learned is based on the training data. Because of the underlying probability model, it's not going to be 100% deterministic. Plus a model like ChatGPT purposefully have "temperature" parameter that will further add randomisation to the whole process. My answer is based on this paper if you're interested to read more: The Matrix: A Bayesian learning model for LLMs, https://arxiv.org/abs/2402.03175 https://arxiv.org/abs/2402.03175

flopriore 3y ago
Are there any ways to show the source of the information retrieved by the model? For instance, the LLM forms a sentence and it points to a stackoverflow answer with the same or similar content.
- JKCalhoun 3y ago
  As I understand it, pretty sure that is impossible. When it is input a single datum, sure, trivial. As soon as it is fed a second one though the weights are already a kind of blend of the two tokens (so to speak).
  - spmurrayzzz 3y ago
    Its not impossible, but its definitely difficult. There is some overlap in the methods used to detect benchmark data contamination, though its not entirely the same thing. For the detection use case, you already know the text you're looking for and you are just trying to demonstrate that the model has "seen" the data in its training set. The challenge is proving that it is statistically improbable that the model could stochastically generate the same tokens without having seen them during training. Some great research exists in this area [1] and I expect much of it may be repurposed for black box attribution in the future (in addition to all the work being done in the mechanistic interpretability field) [1] https://arxiv.org/abs/2311.04850 https://arxiv.org/abs/2311.04850

throwawaymaths 3y ago

> how is it that an LLM when given the same prompt does not respond in the same deterministic way? In software (not in the model) here's literally a random number generator that picks from a weighted set of "next-token" choices that the model spits out. The selection process can have a series of knobs to manipulate the responses. If you want it to be deterministic (if you have direct access to the software) you can tell it to set "top-k = 1" or "temperature = 0.0" (depending on your software) and it will be deterministic. Usually the default settings are not for determinism, because for whatever reason the quality of the results tends to not be that good when you go fully d.

int_19h 3y ago

I found this to be a good start that explains things fairly methodically, but without losing the high-level perspective. https://writings.stephenwolfram.com/2023/02/what-is-chatgpt-doing-and-why-does-it-work/ https://writings.stephenwolfram.com/2023/02/what-is-chatgpt-...