3 ms·
I actually think we are mostly on the right path, with the greatest exception being the current cost and overhead required to train the very best language model
by kortex 3y ago
I actually think we are mostly on the right path, with the greatest exception being the current cost and overhead required to train the very best language models. But we are already seeing smaller, fine-tunable, user-deployable models.
I think (and hope) it'll track a similar trajectory to 3d printing:
1. Expensive systems only approachable to large corporations
2. Lower-cost but affordable as a business expense to cottage industry
3. Affordable to hobbyists, but requiring domain expertise to operate
4. COTS plug and play solutions for the wide audience
Right now we are at (1) with the huge scale LLMs, but scaled models are runnable by hobbyists (2-3) and we are at (4) when it comes to things like image generation and music splitting.