3 ms·
This was a huge concern in 2022, before open models with useful IQ levels were released. Today, it's hard to imagine prohibitive prices at the end-user level, b
by worstspotgain 2y ago
This was a huge concern in 2022, before open models with useful IQ levels were released. Today, it's hard to imagine prohibitive prices at the end-user level, because it's hard to envision an application that won't eventually have a "good enough" open/free on-device inference implementation.
Two caveats: application-specific patents are still possible (and many torpedo patents are undoubtedly en route right now) and this argument might not hold at the middleman level (where wrapper apps might get their margins squeezed out.)
- antupis 2y agoI would say RAG + llama3 is already in that good enough level. Claude and gpt4-o are best but when you need steady performance or a cheap interface on a scale is it very hard to compete against llama3.