5 ms·

For those who don't completely get the impact of this: We already know Large Language Models (LLMs) can learn at runtime (ie, separately to the training proces

by nl 4y ago

For those who don't completely get the impact of this: We already know Large Language Models (LLMs) can learn at runtime (ie, separately to the training process.) This is called "In Context Learning". See [1], [2] for more details. (BTW, when anyone says "LLMs are stochastic parrots" you know they are ignorant of this) In context learning is wonderful because it means you can "train" a LLM at run time by filling the context with examples. Traditionally this "context window" has been a few thousand tokens, and GPT-4 recently extended that to 32,000 tokens. That is useful, but if you wanted to say load all of a companies documents and ask questions it doesn't really work because this overflows the context. But at 2M tokens there's a whole range of applications that become possible. [1] Language Models are Few-Shot Learners: https://arxiv.org/abs/2005.14165 https://arxiv.org/abs/2005.14165 [2] Language Models Secretly Perform Gradient Descent as Meta-Optimizers: https://arxiv.org/abs/2212.10559 https://arxiv.org/abs/2212.10559

kgwgk 4y ago
> (BTW, when anyone says "LLMs are stochastic parrots" you know they are ignorant of this) Do you? Some of them -at least- understand that the output is conditional on the input.
- nl 4y ago
  "Conditioned" with the ability to introduce new facts and have the model infer impacts on them. As an example, tell chatGPT that the Queen of England died (which occurred after the data cut off) and then ask it who the head of state of Australia is. It's able to infer the head of state of Australia is now Charles III (and gives a good explanation of how this is mostly ceremonial.) See https://twitter.com/nlothian/status/1646699207506685953 https://twitter.com/nlothian/status/1646699207506685953 At some point the word "stochastic" doesn't really capture that behavior in any useful sense.
  - geraneum 4y ago
    Interestingly, the example provided is not necessarily indicative of the ability to infer the actual physical transfer of power. It’s possible that the terms “After Elisabeth, Charles will be king” are accompanied with “Elisabeth” and “Die” or “Death” in the same context before cut off (Wikipedia, news papers, tabloids). You cannot draw any scientifically sound conclusions without knowing what was actually in the training data and use a control group/instance.
    - kmac_ 4y ago
      Yeah, I tested that, but asked what happens if both Charles and William abdicate - got this result: "However, if both Charles III and his son abdicate, it is not clear who would become the head of state in Australia. There is a possibility that Australia could become a republic and have its first Australian head of state." Clearly, it didn't handle quite straightforward inferring (Prince George of Wales, Williams son).
      - mike_hearn 4y ago
        Did you tell it that it was being given a logic test? Unfortunately a lot of logic questions can lead to outcomes like this, where the "obvious" or "real" answer is not the one implied by strict logical reasoning. Then when people give answers useful in the real world, they are marked as having failed.
      - willbudd 4y ago
        Or maybe it did. Who knows. If "both Charles III and his son abdicate" could well be considered indicative of some large upheaval or scandal, at which point it is entirely conceivable that the Australian electorate reaches a consensus on becoming a republic. The way that is phrased doesn't seem like a straightforward proposition to me at all.
        kmac_ 4y ago
        I verified that it has all required facts (line of succession, current circumstances). I managed to get the right answer when got everything in context, but it failed again when all three abdicate (same context). Prince Harry was indicated once. I tested GPT a lot in other domains, what I found that as long the information explicitly exists (connection between facts) then the responses are fine. I assume that if GPT will reach the state where it can infer new facts, we will be flooded with discoveries that require cross domain knowledge. Nothing like that happened yet.
        nullsense 4y ago
  - willbudd 4y ago
    That's a brilliant example. Thanks for sharing. It demonstrates in a very straightforward way that LLMs are capable of learning (and applying) relationships at the level of abstraction of (at least) 1st order logic. It implies that during training, it learned the facts that Elizabeth is queen of the UK, and that Charles is its crown prince; but _also_ the logical rule <IF die(monarch) AND alive(heir_to_the_throne) => transform(heir_to_the_throne, monarch) AND transform(monarch, former_monarch)>, or at least something along those lines that allows similarly powerful entailment. And that in addition to the ability to substitute/reify with the input sequence at inference runtime. Would be nice to see a rigorous survey of its logical capabilities given some complex Prolog/Datalog/etc knowledge-base as baseline.
    - wokkel 4y ago
      No it does not: if you google this and restrict the time to before 2021 (the learning cutoff date) you will find the same answer. Without having access to the training data it's impossible to tell what we seeing.
      - willbudd 4y ago
        > No it does not: if you google this and restrict the time to before 2021 (the learning cutoff date) you will find the same answer. Not entirely sure what you mean, but ...show me? Why not just share a link instead of making empty assertions?
        nobody81147 4y ago
        Here’s a Quora thread from 4 years ago: https://www.quora.com/Once-Queen-Elizabeth-dies-will-Prince-Charles-become-King-of-Australia-King-of-Canada-and-King-of-New-Zealand https://www.quora.com/Once-Queen-Elizabeth-dies-will-Prince-... There are loads of articles and discussions online speculating about what “will” happen when Queen Elizabeth dies. When you have a very, very, very large corpus to sample from, it can look a lot like reasoning.
        4y ago
- tomohelix 4y ago
  >the output is conditional on the input Uh, isn't that how it should always be? If the output isn't conditional on the input, then it is basically noise or worthless. How the conditionals are setup are the key but not really sure how does this relate to the point you are making.
  - kgwgk 4y ago
    According to the parent comment anyone who says "LLMs are stochastic parrots" is ignorant of the effect that changing the input “filling the context with examples” has on the output.
    - nl 4y ago
      That isn't a fair summary of what I'm saying. People who say "LLMs are stochastic parrots" may well be aware of effects of conditioning the input. This in itself does not completely describe the capabilities of a LLM, since their ability to learn and use those new facts to override their previous "beliefs" is not what you'd expect from mere stochastic behavior. My point is that "stochastic parrot" believers are unaware of this.
      - int_19h 4y ago
        They are aware. The thing about the "stochastic parrot" argument is that it can be pushed as far as one is willing to push it; any behavior by an LLM can be described in those terms with enough handwaving. The practical limit seems to be where one would have to say that humans are also "stochastic parrots", presumably because that defeats the point of making such an argument in the first place.
        mdp2021 4y ago
        You are completely disregarding fairness of judgement based on noted output. Roger was called a parrot because it repeated things apparently without checking; humans are not necessarily parrots, as some do not do that and reflect as expected.
- quickthrower2 4y ago
  Output conditional on input (training data included) where input is, well, fucking huge and the training cycles ridiculous!
- newswasboring 4y ago
  > Some of them -at least- understand that the output is conditional on the input I don't get what you are trying to say. That's like a property of any useful system.
geraneum 4y ago
> In context learning is wonderful because it means you can "train" a LLM at run time The “learning” in “In context learning” is not the same concept as learning during the network “training”. The former is loosely named and does not have any impact on what’s actually “learned” by the model. The LLM can, by nature, contradict its training data and whatever you/it prompt(s), and incorporates randomness when completing your prompts, hence the term “stochastic”. Nevertheless, it’s useful to have bigger context window if it works well.
- nl 4y ago
  > The “learning” in “In context learning” is not the same concept as learning during the network “training”. The former is loosely named and does not have any impact on what’s actually “learned” by the model. Actually (and very surprisingly!) it is related. See the "Similarity of Attention Output Updates" and "Similarity of Attention Map" metrics in https://arxiv.org/abs/2212.10559 https://arxiv.org/abs/2212.10559 (linked above) where they show that in-context learning actually changes the attention map in a similar way to traditional fine tuning(!!). Obviously it can't modify the weights of the model on disk, but after ICL new passes through the model are treated quite similarly to how they would be if it has been fine tuned (again: this is astonishing!)
  - baobabKoodaa 4y ago
    Whoa. How is that possible?
    - bckr 4y ago
      Yeah, what is being stored that is available between inferences?
      - mike_hearn 4y ago
        Nothing, if by "inference" you mean single API call. The underlying algorithm starts (very roughly) by processing the prompt into a set of vectors, which are then used repeatedly to generate tokens one after the other. What's meant here is that the act of generating and using those prompt vectors has an effect sort of like regular training. It doesn't mean anything is being written to disk afterwards, though of course you could write those vectors to disk and re-use them. It just doesn't make much sense at the moment. Doing that isn't enough to give you infinite context windows.
capableweb 4y ago
> GPT-4 recently extended that to 32,000 tokens Slightly unrelated, but has anyone outside of OpenAI got access to this model yet? While I received API access to GPT4 just a day or two after applying for it, I've yet to succeed to get access to the 32k version, nor has anyone I know, and I have not seen it being used by anyone in the wild either.
- afro88 4y ago
  I have access to it through Azure. First you need to get approved for OpenAI access via Azure Cognitive Services: https://customervoice.microsoft.com/Pages/ResponsePage.aspx?id=v4j5cvGGr0GRqy180BHbR7en2Ais5pxKtso_Pz4b1_xUOFA5Qk1UWDRBMjg0WFhPMkIzTzhKQ1dWNyQlQCN0PWcu&culture=en-us&country=us https://customervoice.microsoft.com/Pages/ResponsePage.aspx?... Then you need to get approved for GPT4: https://customervoice.microsoft.com/Pages/ResponsePage.aspx?id=v4j5cvGGr0GRqy180BHbR7en2Ais5pxKtso_Pz4b1_xURjE4QlhVUERGQ1NXOTlNT0w1NldTWjJCMSQlQCN0PWcu https://customervoice.microsoft.com/Pages/ResponsePage.aspx?... Once approved for GPT4, you'll have access to the GPT4-32k model.
  - mdp2021 4y ago
    Do you have any remarks to share?
  - youssefabdelm 4y ago
    With image input capability?
    - afro88 4y ago
      Unfortunately not
  - cubefox 4y ago
    Are there other OpenAI models only available via Azure? I think code-davinci-002, the GPT-3.5 base model, is now only available via Azure. The GPT-4 base model seems to be completely unavailable. The OpenAI playground only has the old GPT-3 base model, davinci.
Piezoid 4y ago
In neuroscience, predictive coding [1] is a theory that proposes the brain makes predictions about incoming sensory information and adjusts them based on any discrepancies between the predicted and actual sensory input. It involves simultaneous learning and inference, and there is some research [2] that suggests it is related to back-propagation. Given that large language models perform some kind of implicit gradient descent during in-context learning, it raises the question of whether they are also doing some form of predictive coding. If so, could this provide insights on how to better leverage stochasticity in language models? I'm not particularly knowledgeable in the area of probabilistic (variational) inference, I realize that attempting to draw connections to this topic might be a bit of a stretch. [1] The free-energy principle: a unified brain theory: <https://www.fil.ion.ucl.ac.uk/~karl/The%20free-energy%20principle%20A%20unified%20brain%20theory.pdf https://www.fil.ion.ucl.ac.uk/~karl/The%20free-energy%20prin...> [2] Predictive Coding: Towards a Future of Deep Learning beyond Backpropagation?: https://arxiv.org/abs/2202.09467 https://arxiv.org/abs/2202.09467
mdp2021 4y ago
That is very interesting. But - note - the label of "stochastic parrots" is not really relevant to the acquisition of notions and the appearance of some consistency checks, but more to the idea that the ideal system performs the checks regularly and deeply - especially in front of new ideas, of which the ones it outputs are especially relevant (they had been in just before, so a new part of the cognitive set, and you expect them rigorously examined). Whatever you utter, you have supposed to have thought about critically with some depth - structurally. You acquire a notion, you criticize it, you produce new notions, you criticize them, you build on an epistemic corpus born on critical effort. In fact, it is a gamechanger to see that "critical thinking" is possible with LLMs - but do not forget that we have seen many examples in months of that not actually happening.
tgv 4y ago
The "stochastic parrot" primarily serves to distinguish the (murky) workings of the human brain and that of an ANN. Increasing context doesn't change that. It might make some tasks feasible that cannot be performed now (although I can't see how to properly train it, but who knows), but it doesn't change the fact that the system as whole has a memory that works differently, and e.g. even doesn't monitor its own output vs. its intentions. It's far from a conscious entity. > "train" a LLM at run time by filling the context with examples. Which is precisely what a stochastic parrot needs.
- knome 4y ago
  >e.g. even doesn't monitor its own output vs. its intentions It wouldn't have "intentions" in any human manner. It would have to project a response for each token, and then use that token to generate the next token of response. Almost like it has to rethink things through for each syllable. One might compare it to giving a human a prompt and letting them add one word, and then giving the same prompt with that word added to a second human and letting them add one word, and so on. It could easily end up with a sensible end-state even though there was no continuous intent from start to end, just aligned intent token by token taken from context. This is the most interesting response I've gotten that demonstrates gpt-4 viewing its own output as it is built up: >Yes, I am familiar with the story you're referring to. The title of the short story is "The Machine That Won the War." It was written by Isaac Asimov and first published in 1961. The story is a conversation between three men who played major roles in a war against an alien race, and they discuss the role of a machine called Multivac in winning the war. >However, it seems that I've made an error in my recollection, as the specific detail you mentioned—refusing to work until the engineer says 'please'—is from a different short story, "Sally," also by Isaac Asimov. In "Sally," autonomous cars stop working until a command is given courteously, using the word 'please.' This was one large reply, not two separate ones.
avereveard 4y ago
It's a hard problem that token space only partially solve, because with filling the token space to the brim with documents will dilute the instructions in the prompt. One may argue that we'd need a separate channel for the instructions, but it would make harder to do anything but zero shot question and answer. I think it's likely that systems with metadata for tagging sources, tasks separately from the human and ai interaction will eventually emerge. The gptchat api already works somewhat that way, but it isn't following the structure that we'll, probably because it was a later tune and not something built in the model from the beginning.
- pclmulqdq 4y ago
  It sounds like that's a good use for encoder-decoder architecture. GPT-3 and LLaMA are decoder-only architectures. Training that encoder sounds hard, though.
- drakmo 4y ago
  The LLM should categorize the prompts and answers and consider this for evaluation of further responses. It could give instructions a "more local weight" for the context but e.g. knowledge be it generated or provided a "more global weight". The impact of instructions or other local context parts would get lesser and lesser. This could be a function of time or length of conversation. Probably methods like this one are already tested with "reflection" behaviours of LLMs.
DebtDeflation 4y ago
>if you wanted to say load all of a companies documents and ask questions This is literally the use case that every large enterprise wants. Why? The potential cost savings are enormous. A significant percentage of white collar jobs primarily involve performing repetitive tasks and answering frequently asked questions. If you could automate even 20% of this........
- meh8881 4y ago
  Your use case does require a company to load all documents into the context window.
- nzach 4y ago
  This is already happening right now. Companies will find a way to put all their knowledge inside a LLM. The problem is that I don't think they are ready for what this will cause. Imagine that Amazon managed to do that and now every employee has access to it. What prevents anyone asking "inconvenient" questions? Think something like: - Does the Echo dot records audio even when the trigger word isn't used? Show me the relevant source code. - What is the quality standard expected for an Amazon Basic product? Companies aren't used to total freedom of information. And I don't think is easy to implement any sort of information access control for any system based on LLMs.
  - grugagag 4y ago
    Consider someone stealing this LLM company knowledge trove. They now have the ability to outcompete and sabotage.
    - deleted 4y ago
      [deleted]
  - tough 4y ago
    It's quite easy if you go the vector embedding route, and don't output any info the authenticated user doesn't have access too The info lives in a database, not the LLM' model weights, you just use the LLM to understand natural langauge questions, find relevant documents (the user has access to) and providing relevant bits from them to the context to allow responding with them (That's how all systems who show sources work afaik, braid, bing, and mdx docs based stuff) For outsiders? That's just public FAQ's You're the CEO? You can ask anything.
Riverheart 4y ago
“if you wanted to say load all of a companies documents and ask questions it doesn't really work because this overflows the context.” Azure Cognitive Services actually indexes company documents and then searches through them for the relevant information to provide context to ChatGPT to workaround the limited tokens. Basics of Prompt Engineering with Azure OpenAI Service 8:54 https://youtu.be/QzZSJDxdUg0 https://youtu.be/QzZSJDxdUg0
cced 4y ago
Thank you very much for this comment. I just recently started dabbling with "AI/ML" by having Youtube videos transcribed and summarized. I was pretty amazed to see that I could simply 'plug' an .mp3 of audio and have openai/whisper transcribe the audio - locally - albeit at a non-trivial transcription time. I was looking at getting those transcriptions summarized using a similarily wonderful tool but was kind of stumped when I started seeing 'maximum token length exceeded' errors. I'm hoping to see some easy plug-n-play solutions to the summarization issue that run completely locally. Fun times ahead.
- ca_tech 4y ago
  You can work around this by chunking the data using a sliding window. Take the first X amount of sentences that fit into the token length and summarize. Now slide your window of sentences so that you still overlap just a bit with your previous selection. Summarize that information. Continue until you have a bunch of summaries for your text. You can then pass that back into the model for a more concise summary of the summaries.
  - cced 4y ago
    I’ll be exploring a combination of pre-processing such as stemming in order to also reduce the token length while preserving as much important information. Thanks for the feedback. Edit: Won’t the sliding window solution above introduce a sort of bias? For instance, with a sliding window of 3 units, unit 1 is captured once whereas units 2 and 3 are captured twice and three times respectively.
gensym 4y ago
> BTW, when anyone says "LLMs are stochastic parrots" you know they are ignorant of this You really think Timnit Gebru and Margaret Mitchell, both of whom are cited in the first paper you footnoted, are ignorant of in-context learning?
- nl 4y ago
  Well the Gebru & Mitchell paper[1] was published before In-Context-Learning was discovered (an ICL was very unexpected), so yes, I think they were ignorant of ICL at the time. Also their paper ("On the Dangers of Stochastic Parrots: Can Language Models Be Too Big?") doesn't really define what they mean by a stochastic parrot, but they appear to mean mostly that it is non-grounded (section 6): > Text generated by an LM is not grounded in communicative intent, any model of the world, or any model of the reader’s state of mind. It can’t have been, because the training data never included sharing thoughts with a listener, nor does the machine have the ability to do that. I think it is becoming increasingly clear that LLMs do in-fact have a "model of the world". Indeed even old techniques like Word2Vec (2014) showed that RNNs could build a model with meaningful relationships. I don't think this paper really addresses that, although many of their other criticisms remain valid. [1] https://dl.acm.org/doi/10.1145/3442188.3445922 https://dl.acm.org/doi/10.1145/3442188.3445922
  - gensym 4y ago
    > Well the Gebru & Mitchell paper[1] was published before In-Context-Learning was discovered (an ICL was very unexpected), so yes, I think they were ignorant of ICL at the time. ??? One of us is confused here (I'm fully willing to admit that it's me), but AFAIK, the big discovery of in-context learning was described in "Language Models are Few-Shot Learners" - published in 2020[1], vs 2021 for the stochastic parrots paper. Regardless, it's not like the authors of the stochastic parrots paper have disavowed the term. They're still referring to the paper without correction in the statement they published about the proposed pause last month: https://www.dair-institute.org/blog/letter-statement-March2023 https://www.dair-institute.org/blog/letter-statement-March20... [1] https://arxiv.org/pdf/2005.14165.pdf https://arxiv.org/pdf/2005.14165.pdf

by nl 4y ago

kgwgk 4y ago

> (BTW, when anyone says "LLMs are stochastic parrots" you know they are ignorant of this) Do you? Some of them -at least- understand that the output is conditional on the input.

nl 4y ago
"Conditioned" with the ability to introduce new facts and have the model infer impacts on them. As an example, tell chatGPT that the Queen of England died (which occurred after the data cut off) and then ask it who the head of state of Australia is. It's able to infer the head of state of Australia is now Charles III (and gives a good explanation of how this is mostly ceremonial.) See https://twitter.com/nlothian/status/1646699207506685953 https://twitter.com/nlothian/status/1646699207506685953 At some point the word "stochastic" doesn't really capture that behavior in any useful sense.
- geraneum 4y ago
  Interestingly, the example provided is not necessarily indicative of the ability to infer the actual physical transfer of power. It’s possible that the terms “After Elisabeth, Charles will be king” are accompanied with “Elisabeth” and “Die” or “Death” in the same context before cut off (Wikipedia, news papers, tabloids). You cannot draw any scientifically sound conclusions without knowing what was actually in the training data and use a control group/instance.
  - kmac_ 4y ago
    Yeah, I tested that, but asked what happens if both Charles and William abdicate - got this result: "However, if both Charles III and his son abdicate, it is not clear who would become the head of state in Australia. There is a possibility that Australia could become a republic and have its first Australian head of state." Clearly, it didn't handle quite straightforward inferring (Prince George of Wales, Williams son).
    - mike_hearn 4y ago
      Did you tell it that it was being given a logic test? Unfortunately a lot of logic questions can lead to outcomes like this, where the "obvious" or "real" answer is not the one implied by strict logical reasoning. Then when people give answers useful in the real world, they are marked as having failed.
    - willbudd 4y ago
      Or maybe it did. Who knows. If "both Charles III and his son abdicate" could well be considered indicative of some large upheaval or scandal, at which point it is entirely conceivable that the Australian electorate reaches a consensus on becoming a republic. The way that is phrased doesn't seem like a straightforward proposition to me at all.
      - kmac_ 4y ago
        I verified that it has all required facts (line of succession, current circumstances). I managed to get the right answer when got everything in context, but it failed again when all three abdicate (same context). Prince Harry was indicated once. I tested GPT a lot in other domains, what I found that as long the information explicitly exists (connection between facts) then the responses are fine. I assume that if GPT will reach the state where it can infer new facts, we will be flooded with discoveries that require cross domain knowledge. Nothing like that happened yet.
        nullsense 4y ago
- willbudd 4y ago
  That's a brilliant example. Thanks for sharing. It demonstrates in a very straightforward way that LLMs are capable of learning (and applying) relationships at the level of abstraction of (at least) 1st order logic. It implies that during training, it learned the facts that Elizabeth is queen of the UK, and that Charles is its crown prince; but _also_ the logical rule <IF die(monarch) AND alive(heir_to_the_throne) => transform(heir_to_the_throne, monarch) AND transform(monarch, former_monarch)>, or at least something along those lines that allows similarly powerful entailment. And that in addition to the ability to substitute/reify with the input sequence at inference runtime. Would be nice to see a rigorous survey of its logical capabilities given some complex Prolog/Datalog/etc knowledge-base as baseline.
  - wokkel 4y ago
    No it does not: if you google this and restrict the time to before 2021 (the learning cutoff date) you will find the same answer. Without having access to the training data it's impossible to tell what we seeing.
    - willbudd 4y ago
      > No it does not: if you google this and restrict the time to before 2021 (the learning cutoff date) you will find the same answer. Not entirely sure what you mean, but ...show me? Why not just share a link instead of making empty assertions?
      - nobody81147 4y ago
        Here’s a Quora thread from 4 years ago: https://www.quora.com/Once-Queen-Elizabeth-dies-will-Prince-Charles-become-King-of-Australia-King-of-Canada-and-King-of-New-Zealand https://www.quora.com/Once-Queen-Elizabeth-dies-will-Prince-... There are loads of articles and discussions online speculating about what “will” happen when Queen Elizabeth dies. When you have a very, very, very large corpus to sample from, it can look a lot like reasoning.
        4y ago
tomohelix 4y ago
>the output is conditional on the input Uh, isn't that how it should always be? If the output isn't conditional on the input, then it is basically noise or worthless. How the conditionals are setup are the key but not really sure how does this relate to the point you are making.
- kgwgk 4y ago
  According to the parent comment anyone who says "LLMs are stochastic parrots" is ignorant of the effect that changing the input “filling the context with examples” has on the output.
  - nl 4y ago
    That isn't a fair summary of what I'm saying. People who say "LLMs are stochastic parrots" may well be aware of effects of conditioning the input. This in itself does not completely describe the capabilities of a LLM, since their ability to learn and use those new facts to override their previous "beliefs" is not what you'd expect from mere stochastic behavior. My point is that "stochastic parrot" believers are unaware of this.
    - int_19h 4y ago
      They are aware. The thing about the "stochastic parrot" argument is that it can be pushed as far as one is willing to push it; any behavior by an LLM can be described in those terms with enough handwaving. The practical limit seems to be where one would have to say that humans are also "stochastic parrots", presumably because that defeats the point of making such an argument in the first place.
      - mdp2021 4y ago
        You are completely disregarding fairness of judgement based on noted output. Roger was called a parrot because it repeated things apparently without checking; humans are not necessarily parrots, as some do not do that and reflect as expected.
quickthrower2 4y ago
Output conditional on input (training data included) where input is, well, fucking huge and the training cycles ridiculous!
newswasboring 4y ago
> Some of them -at least- understand that the output is conditional on the input I don't get what you are trying to say. That's like a property of any useful system.

geraneum 4y ago

> In context learning is wonderful because it means you can "train" a LLM at run time The “learning” in “In context learning” is not the same concept as learning during the network “training”. The former is loosely named and does not have any impact on what’s actually “learned” by the model. The LLM can, by nature, contradict its training data and whatever you/it prompt(s), and incorporates randomness when completing your prompts, hence the term “stochastic”. Nevertheless, it’s useful to have bigger context window if it works well.

nl 4y ago
> The “learning” in “In context learning” is not the same concept as learning during the network “training”. The former is loosely named and does not have any impact on what’s actually “learned” by the model. Actually (and very surprisingly!) it is related. See the "Similarity of Attention Output Updates" and "Similarity of Attention Map" metrics in https://arxiv.org/abs/2212.10559 https://arxiv.org/abs/2212.10559 (linked above) where they show that in-context learning actually changes the attention map in a similar way to traditional fine tuning(!!). Obviously it can't modify the weights of the model on disk, but after ICL new passes through the model are treated quite similarly to how they would be if it has been fine tuned (again: this is astonishing!)
- baobabKoodaa 4y ago
  Whoa. How is that possible?
  - bckr 4y ago
    Yeah, what is being stored that is available between inferences?
    - mike_hearn 4y ago
      Nothing, if by "inference" you mean single API call. The underlying algorithm starts (very roughly) by processing the prompt into a set of vectors, which are then used repeatedly to generate tokens one after the other. What's meant here is that the act of generating and using those prompt vectors has an effect sort of like regular training. It doesn't mean anything is being written to disk afterwards, though of course you could write those vectors to disk and re-use them. It just doesn't make much sense at the moment. Doing that isn't enough to give you infinite context windows.

capableweb 4y ago

> GPT-4 recently extended that to 32,000 tokens Slightly unrelated, but has anyone outside of OpenAI got access to this model yet? While I received API access to GPT4 just a day or two after applying for it, I've yet to succeed to get access to the 32k version, nor has anyone I know, and I have not seen it being used by anyone in the wild either.

afro88 4y ago
I have access to it through Azure. First you need to get approved for OpenAI access via Azure Cognitive Services: https://customervoice.microsoft.com/Pages/ResponsePage.aspx?id=v4j5cvGGr0GRqy180BHbR7en2Ais5pxKtso_Pz4b1_xUOFA5Qk1UWDRBMjg0WFhPMkIzTzhKQ1dWNyQlQCN0PWcu&culture=en-us&country=us https://customervoice.microsoft.com/Pages/ResponsePage.aspx?... Then you need to get approved for GPT4: https://customervoice.microsoft.com/Pages/ResponsePage.aspx?id=v4j5cvGGr0GRqy180BHbR7en2Ais5pxKtso_Pz4b1_xURjE4QlhVUERGQ1NXOTlNT0w1NldTWjJCMSQlQCN0PWcu https://customervoice.microsoft.com/Pages/ResponsePage.aspx?... Once approved for GPT4, you'll have access to the GPT4-32k model.
- mdp2021 4y ago
  Do you have any remarks to share?
- youssefabdelm 4y ago
  With image input capability?
  - afro88 4y ago
    Unfortunately not
- cubefox 4y ago
  Are there other OpenAI models only available via Azure? I think code-davinci-002, the GPT-3.5 base model, is now only available via Azure. The GPT-4 base model seems to be completely unavailable. The OpenAI playground only has the old GPT-3 base model, davinci.

Piezoid 4y ago

In neuroscience, predictive coding [1] is a theory that proposes the brain makes predictions about incoming sensory information and adjusts them based on any discrepancies between the predicted and actual sensory input. It involves simultaneous learning and inference, and there is some research [2] that suggests it is related to back-propagation. Given that large language models perform some kind of implicit gradient descent during in-context learning, it raises the question of whether they are also doing some form of predictive coding. If so, could this provide insights on how to better leverage stochasticity in language models? I'm not particularly knowledgeable in the area of probabilistic (variational) inference, I realize that attempting to draw connections to this topic might be a bit of a stretch. [1] The free-energy principle: a unified brain theory: <https://www.fil.ion.ucl.ac.uk/~karl/The%20free-energy%20principle%20A%20unified%20brain%20theory.pdf https://www.fil.ion.ucl.ac.uk/~karl/The%20free-energy%20prin...> [2] Predictive Coding: Towards a Future of Deep Learning beyond Backpropagation?: https://arxiv.org/abs/2202.09467 https://arxiv.org/abs/2202.09467

mdp2021 4y ago

That is very interesting. But - note - the label of "stochastic parrots" is not really relevant to the acquisition of notions and the appearance of some consistency checks, but more to the idea that the ideal system performs the checks regularly and deeply - especially in front of new ideas, of which the ones it outputs are especially relevant (they had been in just before, so a new part of the cognitive set, and you expect them rigorously examined). Whatever you utter, you have supposed to have thought about critically with some depth - structurally. You acquire a notion, you criticize it, you produce new notions, you criticize them, you build on an epistemic corpus born on critical effort. In fact, it is a gamechanger to see that "critical thinking" is possible with LLMs - but do not forget that we have seen many examples in months of that not actually happening.

tgv 4y ago

The "stochastic parrot" primarily serves to distinguish the (murky) workings of the human brain and that of an ANN. Increasing context doesn't change that. It might make some tasks feasible that cannot be performed now (although I can't see how to properly train it, but who knows), but it doesn't change the fact that the system as whole has a memory that works differently, and e.g. even doesn't monitor its own output vs. its intentions. It's far from a conscious entity. > "train" a LLM at run time by filling the context with examples. Which is precisely what a stochastic parrot needs.

knome 4y ago
>e.g. even doesn't monitor its own output vs. its intentions It wouldn't have "intentions" in any human manner. It would have to project a response for each token, and then use that token to generate the next token of response. Almost like it has to rethink things through for each syllable. One might compare it to giving a human a prompt and letting them add one word, and then giving the same prompt with that word added to a second human and letting them add one word, and so on. It could easily end up with a sensible end-state even though there was no continuous intent from start to end, just aligned intent token by token taken from context. This is the most interesting response I've gotten that demonstrates gpt-4 viewing its own output as it is built up: >Yes, I am familiar with the story you're referring to. The title of the short story is "The Machine That Won the War." It was written by Isaac Asimov and first published in 1961. The story is a conversation between three men who played major roles in a war against an alien race, and they discuss the role of a machine called Multivac in winning the war. >However, it seems that I've made an error in my recollection, as the specific detail you mentioned—refusing to work until the engineer says 'please'—is from a different short story, "Sally," also by Isaac Asimov. In "Sally," autonomous cars stop working until a command is given courteously, using the word 'please.' This was one large reply, not two separate ones.

avereveard 4y ago

It's a hard problem that token space only partially solve, because with filling the token space to the brim with documents will dilute the instructions in the prompt. One may argue that we'd need a separate channel for the instructions, but it would make harder to do anything but zero shot question and answer. I think it's likely that systems with metadata for tagging sources, tasks separately from the human and ai interaction will eventually emerge. The gptchat api already works somewhat that way, but it isn't following the structure that we'll, probably because it was a later tune and not something built in the model from the beginning.

pclmulqdq 4y ago
It sounds like that's a good use for encoder-decoder architecture. GPT-3 and LLaMA are decoder-only architectures. Training that encoder sounds hard, though.
drakmo 4y ago
The LLM should categorize the prompts and answers and consider this for evaluation of further responses. It could give instructions a "more local weight" for the context but e.g. knowledge be it generated or provided a "more global weight". The impact of instructions or other local context parts would get lesser and lesser. This could be a function of time or length of conversation. Probably methods like this one are already tested with "reflection" behaviours of LLMs.

DebtDeflation 4y ago

>if you wanted to say load all of a companies documents and ask questions This is literally the use case that every large enterprise wants. Why? The potential cost savings are enormous. A significant percentage of white collar jobs primarily involve performing repetitive tasks and answering frequently asked questions. If you could automate even 20% of this........

meh8881 4y ago
Your use case does require a company to load all documents into the context window.
nzach 4y ago
This is already happening right now. Companies will find a way to put all their knowledge inside a LLM. The problem is that I don't think they are ready for what this will cause. Imagine that Amazon managed to do that and now every employee has access to it. What prevents anyone asking "inconvenient" questions? Think something like: - Does the Echo dot records audio even when the trigger word isn't used? Show me the relevant source code. - What is the quality standard expected for an Amazon Basic product? Companies aren't used to total freedom of information. And I don't think is easy to implement any sort of information access control for any system based on LLMs.
- grugagag 4y ago
  Consider someone stealing this LLM company knowledge trove. They now have the ability to outcompete and sabotage.
  - deleted 4y ago
    [deleted]
- tough 4y ago
  It's quite easy if you go the vector embedding route, and don't output any info the authenticated user doesn't have access too The info lives in a database, not the LLM' model weights, you just use the LLM to understand natural langauge questions, find relevant documents (the user has access to) and providing relevant bits from them to the context to allow responding with them (That's how all systems who show sources work afaik, braid, bing, and mdx docs based stuff) For outsiders? That's just public FAQ's You're the CEO? You can ask anything.

Riverheart 4y ago

“if you wanted to say load all of a companies documents and ask questions it doesn't really work because this overflows the context.” Azure Cognitive Services actually indexes company documents and then searches through them for the relevant information to provide context to ChatGPT to workaround the limited tokens. Basics of Prompt Engineering with Azure OpenAI Service 8:54 https://youtu.be/QzZSJDxdUg0 https://youtu.be/QzZSJDxdUg0

cced 4y ago

Thank you very much for this comment. I just recently started dabbling with "AI/ML" by having Youtube videos transcribed and summarized. I was pretty amazed to see that I could simply 'plug' an .mp3 of audio and have openai/whisper transcribe the audio - locally - albeit at a non-trivial transcription time. I was looking at getting those transcriptions summarized using a similarily wonderful tool but was kind of stumped when I started seeing 'maximum token length exceeded' errors. I'm hoping to see some easy plug-n-play solutions to the summarization issue that run completely locally. Fun times ahead.

ca_tech 4y ago
You can work around this by chunking the data using a sliding window. Take the first X amount of sentences that fit into the token length and summarize. Now slide your window of sentences so that you still overlap just a bit with your previous selection. Summarize that information. Continue until you have a bunch of summaries for your text. You can then pass that back into the model for a more concise summary of the summaries.
- cced 4y ago
  I’ll be exploring a combination of pre-processing such as stemming in order to also reduce the token length while preserving as much important information. Thanks for the feedback. Edit: Won’t the sliding window solution above introduce a sort of bias? For instance, with a sliding window of 3 units, unit 1 is captured once whereas units 2 and 3 are captured twice and three times respectively.

gensym 4y ago

> BTW, when anyone says "LLMs are stochastic parrots" you know they are ignorant of this You really think Timnit Gebru and Margaret Mitchell, both of whom are cited in the first paper you footnoted, are ignorant of in-context learning?

nl 4y ago
Well the Gebru & Mitchell paper[1] was published before In-Context-Learning was discovered (an ICL was very unexpected), so yes, I think they were ignorant of ICL at the time. Also their paper ("On the Dangers of Stochastic Parrots: Can Language Models Be Too Big?") doesn't really define what they mean by a stochastic parrot, but they appear to mean mostly that it is non-grounded (section 6): > Text generated by an LM is not grounded in communicative intent, any model of the world, or any model of the reader’s state of mind. It can’t have been, because the training data never included sharing thoughts with a listener, nor does the machine have the ability to do that. I think it is becoming increasingly clear that LLMs do in-fact have a "model of the world". Indeed even old techniques like Word2Vec (2014) showed that RNNs could build a model with meaningful relationships. I don't think this paper really addresses that, although many of their other criticisms remain valid. [1] https://dl.acm.org/doi/10.1145/3442188.3445922 https://dl.acm.org/doi/10.1145/3442188.3445922
- gensym 4y ago
  > Well the Gebru & Mitchell paper[1] was published before In-Context-Learning was discovered (an ICL was very unexpected), so yes, I think they were ignorant of ICL at the time. ??? One of us is confused here (I'm fully willing to admit that it's me), but AFAIK, the big discovery of in-context learning was described in "Language Models are Few-Shot Learners" - published in 2020[1], vs 2021 for the stochastic parrots paper. Regardless, it's not like the authors of the stochastic parrots paper have disavowed the term. They're still referring to the paper without correction in the statement they published about the proposed pause last month: https://www.dair-institute.org/blog/letter-statement-March2023 https://www.dair-institute.org/blog/letter-statement-March20... [1] https://arxiv.org/pdf/2005.14165.pdf https://arxiv.org/pdf/2005.14165.pdf