4 ms·

I’d point out that this is the “missing link” to a next-gen LLM. When the scientists figure out how to enable a remembered collaboration that continually progr

I’d point out that this is the “missing link” to a next-gen LLM. When the scientists figure out how to enable a remembered collaboration that continually progresses results, then we’ll have something truly remarkable.

danielmarkbruce 2y ago
You can do this with a sophisticated RAG system.
- ChicagoDave 2y ago
  To some degree, sure, but I'm suggesting a productized client. I envision an IDE with separate windows for collaborative discussions. One window is the dialog, and another is any resulting code as generated files. The files get versioned with remarks on how and why they've changed in relation to the conversation. The dev can bring any of those files into their project and the IDE will keep a link to which version of the LLM generated file is in place or if the dev has made changes. If the dev makes changes, the file is automatically pushed to the LLM files window and the LLM sees those changes and adds them to the "conversation". The collaboration continually has "why" answers for every change. And all of this without anyone digging into models, python, or RAG integrations.
  - danielmarkbruce 2y ago
    You could build this now, using gpt-4 (or any of the top few models) and a whole bunch of surrounding systems. It would be difficult to pull off, but there is not technology change required. If you build it you'll need to be digging into models, python and various pieces of "RAG" stuff, but your users wouldn't need to know anything about it.
gwern 2y ago
You can do this with dynamic evaluation (https://gwern.net/doc/ai/nn/dynamic-evaluation/index https://gwern.net/doc/ai/nn/dynamic-evaluation/index): literally just train on the data as you are fed new data to predict, in the most obvious possible way. It works well, and most SOTAs set in text prediction 2010-2018 would use dynamic evaluation for the boost. (And it has equivalents for other tasks, like test-time adaptation for CNN classifiers.) In fact, dynamic evaluation is so old and obvious that the first paper to demonstrate RNNs really worked better than n-grams as LLMs did dynamic evaluation: https://gwern.net/doc/ai/nn/rnn/2010-mikolov.pdf#page=2 https://gwern.net/doc/ai/nn/rnn/2010-mikolov.pdf#page=2 Dynamic evaluation offers large performance boosts in Transformers just as well as RNNs. The fact that it's not standard these days has more to do with cloud providers strongly preferring models to not change weights to simplify deployment and enable greater batching, and preferring to crudely approximate it with ever greater context windows. (Also probably the fact that no one but oldtimers knows it is a thing.) I think it might come back for local LLMs, where there is only one set of model weights running persistently, users want personalization and the best possible performance, and where ultra-large context windows are unacceptably memory-hungry & high-latency.
- spmurrayzzz 2y ago
  I've done some testing on this idea of real-time training for memory purposes using prompt tuning, prefix tuning, and LoRAs. Its hard to pull off in many cases but I've seen it work in surprising situations. I got the original idea when I was testing out the concept of using LoRA swapping for loading different personalities/output styles per batched user prompts. Those were pre-trained, but it occurred to me that soft prompts could be trained much faster to potentially remember just a handful of facts. The basic idea is that you summarize chunks of the convo over time and create tiny ephemeral datasets that are built for retrieval tasks. You can do this by asking another model to create Q&A pairs for you about the summarized convo context. Each sample in the dataset is an instruction-tuned format with convo context plus the Q&A pair. The training piece is straightforward but is really where the hair is. Its simple to, in the soft prompt use case, to train a small tensor on that data and just concatenate it to future inputs. But you presumably will have some loss cutoff, and in my experience you very frequently don't hit that loss cutoff in a meaningfully short period of time. Even when you do, the recall/retrieval may still not work as expected (though I was surprised how often it does). The biggest issue is obviously recall performance given that the weights remain frozen, but also the latency introduced in the real-time training can end up being a major bottleneck, even when you lazily do this work in the background.

danielmarkbruce 2y ago

You can do this with a sophisticated RAG system.

ChicagoDave 2y ago
To some degree, sure, but I'm suggesting a productized client. I envision an IDE with separate windows for collaborative discussions. One window is the dialog, and another is any resulting code as generated files. The files get versioned with remarks on how and why they've changed in relation to the conversation. The dev can bring any of those files into their project and the IDE will keep a link to which version of the LLM generated file is in place or if the dev has made changes. If the dev makes changes, the file is automatically pushed to the LLM files window and the LLM sees those changes and adds them to the "conversation". The collaboration continually has "why" answers for every change. And all of this without anyone digging into models, python, or RAG integrations.
- danielmarkbruce 2y ago
  You could build this now, using gpt-4 (or any of the top few models) and a whole bunch of surrounding systems. It would be difficult to pull off, but there is not technology change required. If you build it you'll need to be digging into models, python and various pieces of "RAG" stuff, but your users wouldn't need to know anything about it.

gwern 2y ago

You can do this with dynamic evaluation (https://gwern.net/doc/ai/nn/dynamic-evaluation/index https://gwern.net/doc/ai/nn/dynamic-evaluation/index): literally just train on the data as you are fed new data to predict, in the most obvious possible way. It works well, and most SOTAs set in text prediction 2010-2018 would use dynamic evaluation for the boost. (And it has equivalents for other tasks, like test-time adaptation for CNN classifiers.) In fact, dynamic evaluation is so old and obvious that the first paper to demonstrate RNNs really worked better than n-grams as LLMs did dynamic evaluation: https://gwern.net/doc/ai/nn/rnn/2010-mikolov.pdf#page=2 https://gwern.net/doc/ai/nn/rnn/2010-mikolov.pdf#page=2 Dynamic evaluation offers large performance boosts in Transformers just as well as RNNs. The fact that it's not standard these days has more to do with cloud providers strongly preferring models to not change weights to simplify deployment and enable greater batching, and preferring to crudely approximate it with ever greater context windows. (Also probably the fact that no one but oldtimers knows it is a thing.) I think it might come back for local LLMs, where there is only one set of model weights running persistently, users want personalization and the best possible performance, and where ultra-large context windows are unacceptably memory-hungry & high-latency.

spmurrayzzz 2y ago
I've done some testing on this idea of real-time training for memory purposes using prompt tuning, prefix tuning, and LoRAs. Its hard to pull off in many cases but I've seen it work in surprising situations. I got the original idea when I was testing out the concept of using LoRA swapping for loading different personalities/output styles per batched user prompts. Those were pre-trained, but it occurred to me that soft prompts could be trained much faster to potentially remember just a handful of facts. The basic idea is that you summarize chunks of the convo over time and create tiny ephemeral datasets that are built for retrieval tasks. You can do this by asking another model to create Q&A pairs for you about the summarized convo context. Each sample in the dataset is an instruction-tuned format with convo context plus the Q&A pair. The training piece is straightforward but is really where the hair is. Its simple to, in the soft prompt use case, to train a small tensor on that data and just concatenate it to future inputs. But you presumably will have some loss cutoff, and in my experience you very frequently don't hit that loss cutoff in a meaningfully short period of time. Even when you do, the recall/retrieval may still not work as expected (though I was surprised how often it does). The biggest issue is obviously recall performance given that the weights remain frozen, but also the latency introduced in the real-time training can end up being a major bottleneck, even when you lazily do this work in the background.