3 ms·
Is this not an incredibly expensive/unsustainable method? I agree with the sentiment that MoE is the way to go as the newer models will probably see diminishing
by trash_cat 3y ago
Is this not an incredibly expensive/unsustainable method? I agree with the sentiment that MoE is the way to go as the newer models will probably see diminishing returns. But the compute for a single prompt will suddenly increase 7-15 fold?
- bigEnotation 3y agoYeah, see GPT 3.5 vs GPT 4 pricing.
- bearjaws 3y ago"all you need is a six figure OpenAI bill"
- Garlef 3y agoif it replaces 2-3 experts that might be a good deal
- ukuina 3y agoIf GPT4 is 20x the price of GPT3.5, but it only takes 10x GPT3.5 runs to get similar quality of response (and likely faster), you'll still come out ahead.
- dimask 3y agoI doubt that 10xGPT3.5 > GPT4. There are a lot of tasks that GPT4 can do and GPT3.5 just cannot. Also, in such cases I find that GPT3.5's hallucinations are quite consistent, so such a method is probably not gonna help.
- imtringued 3y agoSo what? It's not like GPUs are compute starved.
- ZachSaucier 3y agoAlso the amount of non-renewable resources used and emissions goes up 7-15x