4 ms·

Like the earlier responses said, if it is retrieving information from an external data source, that would be a much easier way to update things. REALM (https://

Like the earlier responses said, if it is retrieving information from an external data source, that would be a much easier way to update things. REALM (https://arxiv.org/abs/2002.08909 https://arxiv.org/abs/2002.08909) comes to mind as an example of active work in this area. Updating a monolithic LLM seems like a harder problem at the moment, and probably would fall under the umbrella of "continual learning", though that sub-field would have their own subdivisions of different methods. As for how they would potentially perform, those results would be mostly empirically measured, but neither approach would completely remove the possibility of (1) missing results -- which is probably acceptable given Google doesn't always return everything relevant, and (2) hallucinating non-factual responses -- which can be more dangerous, if not from directly instigating / causing harm, then from the incessant worry that it might.

impalallama 4y ago
Wouldn’t retrieving information from an external source effectively be just recreating a search engine with a fancy natural language interface
MuffinFlavored 4y ago
> Like the earlier responses said, if it is retrieving information from an external data source I was under the impression it went through like a multi-month long ultra expensive training process involving many GPUs on terabytes of data "snapshotted" point in time. Is it possible to do this at a smaller scale once a day at the end of the day with "all new content scraped from the Internet nightly"?
- nicoburns 4y ago
  I think the idea might be to effectively provide it with the result of a google search (or other datasource) as part of the prompt.
- visarga 4y ago
  Not effectively, because you probably can't return to language pre-training after multi-task fine-tuning and RLHF. Stage 1 has to go before stages 2 and 3. So they would need to fine-tune a stage 1 model and re-apply stages 2 and 3.
- dragonwriter 4y ago
  > Is it possible to do this at a smaller scale once a day at the end of the day with "all new content scraped from the Internet nightly"? If the model has the ability to browse the web (the tech behind ChatGPT by design does, but it is disabled in ChatGPT proper, which is, in some respects, a conservative public demo) and incorporate data in responses, this can “cover” a bit for staleness in the base model. Even if you can't retrain the model fully daily, with enough resources [0] you can have multiple training sessions running concurrently and just swap in new backend models behind the interface as they arr ready, which, combined with browsing ability. acheives something very similar. This obviously, barring an enormous advantage in underlying tech or access to relevant training data for one player, works most in favor of whoever can subsidize the biggest hardware commitment. (Given the different training stages, there may be cost efficiency advantages to, say, running less-frequent iterations of the lowest-level training stage but more frequent iterations of the higher-level ones.) [0] probably an utterly ludicrous investment for all but a handful of firms, but for a ~$trillion firm where this hits very close to their core business? Not so ludicrous.
- shawntan 4y ago
  How I imagine what you're thinking of is a kind of small update to an existing model by doing backpropagation training on 'new' data. There are existing problems with that, one keyword to lookup is 'catastrophic forgetting', where you update on recent stuff so much that you overfit on that and forget all the prior stuff you learned before. There are methods to mitigate that, but I would call that an 'active area of research'.
- 1024core 4y ago
  The paper about PaLM mentioned that they used 6144 TPU chips to train it over several weeks. That's a metric shit ton of processing power (pardon my language). If Google could dedicate 2 racks of TPUs to just continuously training PaLM (note: I'm not sure if LAMDA is PaLM, but let's assume they're similar), they could do this retraining. Catch is: consider these statements: Donald Trump is the President. Joe Biden is the President. Barack Obama is the President. George W Bush is the President. George H. W. Bush is the President. Bill Clinton is the President. All of them were true at some point of time. How do you train a model to disambiguate these?
  - htrp 4y ago
    Lamda is about 20% of the size of palm IIRC
nestorD 4y ago
Retrieval-based models have a number of strengths that make them particularly important from my point of view: - as said they can be updated with fresh information on the fly, - they can give you sources for their results (meaning that you can fact-check the output!), - they achieve similar performances for significantly smaller models (as long as you have a large dataset to retrieve from) meaning that they could run locally on consumer hardware (coupled with a large dataset on disk or the ability to read information from internet). If I were to spend a significant amount of time building my own language model, it would be retrieval-based and try to preserve those properties (in particular the ability to run on consumer hardware, enabling developers to run models locally has proved to be a game changer when stable diffusion came out).
- iandanforth 4y ago
  If anyone is tempted to do this, here is a good blog post on how to build your own retrieval-based LLM powered conversational agent. https://dagster.io/blog/chatgpt-langchain https://dagster.io/blog/chatgpt-langchain

impalallama 4y ago

Wouldn’t retrieving information from an external source effectively be just recreating a search engine with a fancy natural language interface

MuffinFlavored 4y ago

> Like the earlier responses said, if it is retrieving information from an external data source I was under the impression it went through like a multi-month long ultra expensive training process involving many GPUs on terabytes of data "snapshotted" point in time. Is it possible to do this at a smaller scale once a day at the end of the day with "all new content scraped from the Internet nightly"?

nicoburns 4y ago
I think the idea might be to effectively provide it with the result of a google search (or other datasource) as part of the prompt.
visarga 4y ago
Not effectively, because you probably can't return to language pre-training after multi-task fine-tuning and RLHF. Stage 1 has to go before stages 2 and 3. So they would need to fine-tune a stage 1 model and re-apply stages 2 and 3.
dragonwriter 4y ago
> Is it possible to do this at a smaller scale once a day at the end of the day with "all new content scraped from the Internet nightly"? If the model has the ability to browse the web (the tech behind ChatGPT by design does, but it is disabled in ChatGPT proper, which is, in some respects, a conservative public demo) and incorporate data in responses, this can “cover” a bit for staleness in the base model. Even if you can't retrain the model fully daily, with enough resources [0] you can have multiple training sessions running concurrently and just swap in new backend models behind the interface as they arr ready, which, combined with browsing ability. acheives something very similar. This obviously, barring an enormous advantage in underlying tech or access to relevant training data for one player, works most in favor of whoever can subsidize the biggest hardware commitment. (Given the different training stages, there may be cost efficiency advantages to, say, running less-frequent iterations of the lowest-level training stage but more frequent iterations of the higher-level ones.) [0] probably an utterly ludicrous investment for all but a handful of firms, but for a ~$trillion firm where this hits very close to their core business? Not so ludicrous.
shawntan 4y ago
How I imagine what you're thinking of is a kind of small update to an existing model by doing backpropagation training on 'new' data. There are existing problems with that, one keyword to lookup is 'catastrophic forgetting', where you update on recent stuff so much that you overfit on that and forget all the prior stuff you learned before. There are methods to mitigate that, but I would call that an 'active area of research'.
1024core 4y ago
The paper about PaLM mentioned that they used 6144 TPU chips to train it over several weeks. That's a metric shit ton of processing power (pardon my language). If Google could dedicate 2 racks of TPUs to just continuously training PaLM (note: I'm not sure if LAMDA is PaLM, but let's assume they're similar), they could do this retraining. Catch is: consider these statements: Donald Trump is the President. Joe Biden is the President. Barack Obama is the President. George W Bush is the President. George H. W. Bush is the President. Bill Clinton is the President. All of them were true at some point of time. How do you train a model to disambiguate these?
- htrp 4y ago
  Lamda is about 20% of the size of palm IIRC

nestorD 4y ago

Retrieval-based models have a number of strengths that make them particularly important from my point of view: - as said they can be updated with fresh information on the fly, - they can give you sources for their results (meaning that you can fact-check the output!), - they achieve similar performances for significantly smaller models (as long as you have a large dataset to retrieve from) meaning that they could run locally on consumer hardware (coupled with a large dataset on disk or the ability to read information from internet). If I were to spend a significant amount of time building my own language model, it would be retrieval-based and try to preserve those properties (in particular the ability to run on consumer hardware, enabling developers to run models locally has proved to be a game changer when stable diffusion came out).

iandanforth 4y ago
If anyone is tempted to do this, here is a good blog post on how to build your own retrieval-based LLM powered conversational agent. https://dagster.io/blog/chatgpt-langchain https://dagster.io/blog/chatgpt-langchain