3 ms·
As we get better with miniaturizing LLMs this might become a good approach. Right now LLMs with enough world knowledge and language understanding to do these ta
by wongarsu 3y ago
As we get better with miniaturizing LLMs this might become a good approach. Right now LLMs with enough world knowledge and language understanding to do these tasks are still so big that stacking models like this leads to significant latency. That's acceptable for some use cases, but a major problem for most use cases.
Of course it becomes more viable if each "layer" is not a whole LLM with its own input and output but a modification you can slot into the original LLM. That's basically what LoRAs are.