5 ms·

Pardon my ignorance, but I am a dilettante. I was just wondering about this earlier today. How much unlocking of a more active intelligence from LLMs, will far

Pardon my ignorance, but I am a dilettante. I was just wondering about this earlier today. How much unlocking of a more active intelligence from LLMs, will far larger context length provide? Would virtually unlimited context move us much closer to an LLM where training is continuous? Is that a bit of a holy grail? I assume not, but would love to know why.

mritchie712 3y ago
context length doesn't unlock intelligence, just more information. e.g. adding info that wasn't part of training (or wasn't heavily weighted in training).
- consumer451 3y ago
  Right, I should have been more clear than the words "active intelligence." But as one use of this... would unlimited, say 1 to 10 billion tokens of context, used as a system prompt, with "just" 32k left for the user, allow a model to be updated every day, in between actual training? (This is in the future, where training only takes a month, or much less.) I guess part of what I really don't understand is how context tokens compare to training weights, as far as value to the final response. Would a giant context window muddle the value of weights? (Maybe what I am missing is the human-feedback on the training weights? If the giant system prompt I am imagining is garbage, then that would be bad.)
  - nl 3y ago
    In-context learning (ICL) is a thing. People aren't entirely sure how it works[1]. LLMs are very effective at few-shot learning via so yes, for all practical purposes yes, large context windows do allow for continuous learning. Note that the context needs to be loaded and processed on every request to the LLM though - so all that additional information has to be "retaught" each time. [1] https://openreview.net/pdf?id=992eLydH8G https://openreview.net/pdf?id=992eLydH8G "These results indicate that the equivalence between ICL (ed: In-context-learning) and GD (ed: Gradient Descent) is an open hypothesis, requires nuanced considerations, and calls for further studies.
    - consumer451 3y ago
      Thank you so much for your response. It's amazing what typing a little acronym like "ICL" can do as far as sharing knowledge. This is so cool! Also, your link appears to exactly address my question. It's late here, but I am very excited to do my best at understanding that paper in the morning.
- farts_mckensy 3y ago
  I mean, working memory is an aspect of intelligence.
  - deleted 3y ago
    [deleted]
- nl 3y ago
  > context length doesn't unlock intelligence, just more information This isn't correct (for most definitions of "unlock intelligence"). In-Context Learning (ICL) can do everything off-line training can do. > We find that both Reinforced and Unsupervised ICL can be quite effective in the many-shot regime, particularly on complex reasoning tasks. Finally, we demonstrate that, unlike few-shot learning, many-shot learning is effective at overriding pretraining biases and can learn high-dimensional functions with numerical inputs. https://arxiv.org/abs/2404.11018 https://arxiv.org/abs/2404.11018
a_wild_dandan 3y ago
> How much unlocking of a more active intelligence from LLMs, will far larger context length provide? Remarkably, foundation models can learn new tasks via a few examples (called few-shot learning). LLM answers also significantly improve when given relevant supplemental information. Boosting context length: grows its "working memory"; provides richer knowledge to inform its reasoning; and expands its capacity for new tasks, given germane examples. > Would virtually unlimited context move us much closer to an LLM where training is continuous? No. You can already continually train a context-limited LLM. Virtually unlimited context window schemes also exist. Training is separate concept from context length. In pre-training, we work backward from a model's incorrect answer, tweaking its parameters to more likely say the correct thing next time. Fine-tuning is the same, but focusing on specific tasks important to the user. After training, when running the model (called inference), you can change the context length to suit your needs and tradeoffs.
- consumer451 3y ago
  Thank you. If you don't mind... > tweaking its parameters to more likely say the correct thing next time. Is this entirely, or just partially done via human feedback on models like GPT-4 and LLama-3, for example?
  - hirako2000 3y ago
    Look into agent training.
  - a_wild_dandan 3y ago
    Training is done by partially hiding (masking) text and making a (decoder-only) model predict what's missing. Training text which humans prefer as model responses is called RLHF. It's an expensive and small text dataset.
phillipcarter 3y ago
The other thing to consider is that, at least today, more context also means more chances at hallucinations or otherwise imperfect recall that goes into giving an answer. This has certainly been my experience with larger context windows.
- consumer451 3y ago
  Thanks, I was wondering about this. Has this been your experience across many models, universally? Or, are some worse than others at what I just learned is called In-Context Learning?
  - danielbln 3y ago
    Recall ability varies quite a bit. GPT-4-Turbo's recall becomes quite spotty as you reach 30-50k tokens, whereas Claude-3 has really good recall over the entire context window.
  - phillipcarter 3y ago
    This has been my experience across all of them, yes. Especially when I ask it to select a decently-sized subset of the text I pass in, as opposed to just doing a needle-in-the-haystack type thing.

mritchie712 3y ago

context length doesn't unlock intelligence, just more information. e.g. adding info that wasn't part of training (or wasn't heavily weighted in training).

consumer451 3y ago
Right, I should have been more clear than the words "active intelligence." But as one use of this... would unlimited, say 1 to 10 billion tokens of context, used as a system prompt, with "just" 32k left for the user, allow a model to be updated every day, in between actual training? (This is in the future, where training only takes a month, or much less.) I guess part of what I really don't understand is how context tokens compare to training weights, as far as value to the final response. Would a giant context window muddle the value of weights? (Maybe what I am missing is the human-feedback on the training weights? If the giant system prompt I am imagining is garbage, then that would be bad.)
- nl 3y ago
  In-context learning (ICL) is a thing. People aren't entirely sure how it works[1]. LLMs are very effective at few-shot learning via so yes, for all practical purposes yes, large context windows do allow for continuous learning. Note that the context needs to be loaded and processed on every request to the LLM though - so all that additional information has to be "retaught" each time. [1] https://openreview.net/pdf?id=992eLydH8G https://openreview.net/pdf?id=992eLydH8G "These results indicate that the equivalence between ICL (ed: In-context-learning) and GD (ed: Gradient Descent) is an open hypothesis, requires nuanced considerations, and calls for further studies.
  - consumer451 3y ago
    Thank you so much for your response. It's amazing what typing a little acronym like "ICL" can do as far as sharing knowledge. This is so cool! Also, your link appears to exactly address my question. It's late here, but I am very excited to do my best at understanding that paper in the morning.
farts_mckensy 3y ago
I mean, working memory is an aspect of intelligence.
- deleted 3y ago
  [deleted]
nl 3y ago
> context length doesn't unlock intelligence, just more information This isn't correct (for most definitions of "unlock intelligence"). In-Context Learning (ICL) can do everything off-line training can do. > We find that both Reinforced and Unsupervised ICL can be quite effective in the many-shot regime, particularly on complex reasoning tasks. Finally, we demonstrate that, unlike few-shot learning, many-shot learning is effective at overriding pretraining biases and can learn high-dimensional functions with numerical inputs. https://arxiv.org/abs/2404.11018 https://arxiv.org/abs/2404.11018

a_wild_dandan 3y ago

> How much unlocking of a more active intelligence from LLMs, will far larger context length provide? Remarkably, foundation models can learn new tasks via a few examples (called few-shot learning). LLM answers also significantly improve when given relevant supplemental information. Boosting context length: grows its "working memory"; provides richer knowledge to inform its reasoning; and expands its capacity for new tasks, given germane examples. > Would virtually unlimited context move us much closer to an LLM where training is continuous? No. You can already continually train a context-limited LLM. Virtually unlimited context window schemes also exist. Training is separate concept from context length. In pre-training, we work backward from a model's incorrect answer, tweaking its parameters to more likely say the correct thing next time. Fine-tuning is the same, but focusing on specific tasks important to the user. After training, when running the model (called inference), you can change the context length to suit your needs and tradeoffs.

consumer451 3y ago
Thank you. If you don't mind... > tweaking its parameters to more likely say the correct thing next time. Is this entirely, or just partially done via human feedback on models like GPT-4 and LLama-3, for example?
- hirako2000 3y ago
  Look into agent training.
- a_wild_dandan 3y ago
  Training is done by partially hiding (masking) text and making a (decoder-only) model predict what's missing. Training text which humans prefer as model responses is called RLHF. It's an expensive and small text dataset.

phillipcarter 3y ago

The other thing to consider is that, at least today, more context also means more chances at hallucinations or otherwise imperfect recall that goes into giving an answer. This has certainly been my experience with larger context windows.

consumer451 3y ago
Thanks, I was wondering about this. Has this been your experience across many models, universally? Or, are some worse than others at what I just learned is called In-Context Learning?
- danielbln 3y ago
  Recall ability varies quite a bit. GPT-4-Turbo's recall becomes quite spotty as you reach 30-50k tokens, whereas Claude-3 has really good recall over the entire context window.
- phillipcarter 3y ago
  This has been my experience across all of them, yes. Especially when I ask it to select a decently-sized subset of the text I pass in, as opposed to just doing a needle-in-the-haystack type thing.