4 ms·
In the past, this might have been called an "expert system". Heh, it comes full circle... After ~50 years of Expert Systems winter, we're training our new AGI
by bbor 2y ago
In the past, this might have been called an "expert system".
Heh, it comes full circle... After ~50 years of Expert Systems winter, we're training our new AGIs to become more specialized! This is a memorable lesson that binaries must always be deconstructed, at least to some extent -- kinda like the endless dance we're doing between monoliths and microservices as each new generation of tools runs into the problems inherent in each.
I am naive about LLM technology so I'm not sure about the details or the nomenclature
You've got all the details right though, so that's pretty impressive :). AFAICT from a quick glance at the code (https://github.com/qhjqhj00/MemoRAG/blob/main/memorag/memorag.py#L83-L86 https://github.com/qhjqhj00/MemoRAG/blob/main/memorag/memora...), it is indeed "fine tuning" (jargon!) a model on your chosen book, presumably in the most basic/direct sense: asking it reproduce sections of text at random from the book given their surrounding context, and rewarding/penalizing the neural network based on how well it did. The comment mentions GPU memory in the Colab Notebook merely because this process is expensive -- "fine tuning" is the same thing as "training", just with a nearly-complete starting point. Thus the call to `AutoModelForCausalLM.from_pretrained()`.
To answer your question explicitly: the fine-tuning step creates a modified version of the base model as an "offline" step, so the memory requirements during inference (aka "online" operation) are unaffected. Both in terms of storage and in terms of GPU VRAM. I'm not the dev tho so obv apologies if I'm off base!
I would passionately argue that that step is more of a small addition to the overall pipeline than a core necessity, though. Fine-tuning is really good for teaching a model to recreate style, tone, structure, and other linguistic details, but it's not a very feasible way to teach it facts. That's what "RAG" is for: making up for this deficiency in fine-tuning.
In other words, this repo is basically like that post from a few weeks back that was advocating for "modular monoliths" that employ both strategies (monolith vs. microservices) in a deeply collaborative way. And my reaction is the same: I'm not convinced the details of this meshing will be very revolutionary, but the idea itself is deceptively clever!
- spmurrayzzz 2y ago> AFAICT from a quick glance at the code (https://github.com/qhjqhj00/MemoRAG/blob/main/memorag/memora https://github.com/qhjqhj00/MemoRAG/blob/main/memorag/memora...), it is indeed "fine tuning" (jargon!) a model on your chosen book, presumably in the most basic/direct sense: asking it reproduce sections of text at random from the book given their surrounding context, and rewarding/penalizing the neural network based on how well it did. Maybe your use of quotes is intentional here, but for posterity's sake there is no actual fine-tuning happening using user input in the code you linked, insofar as the weights of the model aren't being touched at all, nor are they modifying anything else that could impact the original weights (like a LoRA adapter). You touch on this, I think (?), in some of your subsequent language but it read as a little confusing to me at first glance. Or maybe I've been too deep in the ML weeds for too many years at this point. The paper details the actual process, but the TL;DR is that the memory module they use, basically a draft model, does go through a pretraining phase using the redpajama dataset, and then an SFT phase with a different objective. This all happens before and irrespective of the inference-time task (i.e. asking questions about a given text). Also, as has been pointed out in other comments, the draft model could really be any model that supports long context and has decent retrieval performance. So the actual training phases here may be non-essential depending on your infra/cost constraints.
- bbor 2y agoThanks for the corrections! I’m very much not an expert on LLM usage in the real world. But I’m a bit confused: does go through a pretraining phase using the redpajama dataset, and then an SFT phase with a different objective Isn’t that equivalent to what I said, since “SFT” seems to stand for “supervised fine-tuning”? That it starts with a pre trained model, and then modifies that model according to your corpus? Perhaps the confusion here is my ambiguity with “model”; I now see that there’s really two models-one for generating a draft + clues and one for constructing the final output—and this library only concerns/modifies the former. Maybe?
- spmurrayzzz 2y agoI should have quoted you more specifically, my apologies. I was responding to the comment that there was some training of the "model on your chosen book". There is no fine-tuning done specific to the corpus you own. I noted this in a sibling comment, but both the pretraining and fine-tuning objective uses a generic dataset (redpajama) which "aims to maximize the generation probability of the next token given the KV cache of the previous memory tokens" (quote from section 2.2 of the paper). This is why I noted you could really use any long-context model that also has good retrieval performance. They're training their own draft model in lieu of using an existing model, but you could get similar/better outcomes using something like claude sonnet 3.5.
- bbor 2y agoThanks for taking the time, that makes sense. This is not the first time I've misunderstood something by having opinions about what it should be doing in my opinion, haha. I absolutely agree with your last point, too.