3 ms·
Dumb question: What does it mean to "train" an LLM on something? I've used Llama Index to create an index based on a large dataset which I can then query. Is th
by jamestimmins 4y ago
Dumb question: What does it mean to "train" an LLM on something? I've used Llama Index to create an index based on a large dataset which I can then query. Is that what it means? Or is it something else?
- Barrin92 4y agoIn this case they trained a model from scratch, which means they built their own LLM, which is what's called a transformer, on their own financial data. How transformers work in detail is a little bit more involved. That's not the same as indexing which is a way to tune an existing model for a particular context.
- jamestimmins 4y agoOk that makes a lot of sense, thanks for explaining!
- iKlsR 4y agoThe current crop of services and pdf chatbots that you see going around using "train" on your own data aren't doing anything of the sort. They are essentially finding relevant sections of your docs and stuffing that into the prompt context window. Think of llama as the trained base model (not particularly useful for any one thing by itself) and alpaca is a fine tuned variant of that to give it chatgpt like "powers".