4 ms·
Is this like, the AI equivalent of “another layer will fix it” that crypto fans used? “It’s ok bro, another model will fix, just please, one more ~layer~ ~agen
by FridgeSeal 3y ago
Is this like, the AI equivalent of “another layer will fix it” that crypto fans used?
“It’s ok bro, another model will fix, just please, one more ~layer~ ~agent~ model”
It’s all fun and games until you can’t reliably generate your base models anymore, because all your _base_ data is too polluted.
Let’s not forget MS has a $10bn stake in the current crop of LLM’s turning out to be as magic as they claim, so I’m sure they will do anything to ensure that happens.
- eightysixfour 3y agoI mean, you can use Phi now and it outclasses anything else in its size. This isn’t some “it could happen” situation.
- FridgeSeal 3y agoOh I’m sure it works wonderfully for now. My point is about the inevitable future when _those_ models start to struggle. The phi approach doesn’t seem like breaking the ouroboros, it just feels like inserting another model/snake into the loop.
- eightysixfour 3y ago“Struggle” at what? Struggle to have enough data to get smarter? Struggle to perform RAG and find legitimate sources? I don’t think that we are going to get big improvements in LLMs without architecture improvements that need less data, and the current generation of models appears to be good enough at creating content from data/knowledge to train any future architectures we have with better synthetic datasets. Fortunately we have already seen examples of both of these “in the lab” and will probably see commercially sized models using some of the techniques in the coming months.