4 ms·
So far OpenAI's modus operandi was "does it get better if we make it bigger". But now we've reached "good enough" (arguably already with GPT3.5, as evidenced by
by wongarsu 3y ago
So far OpenAI's modus operandi was "does it get better if we make it bigger". But now we've reached "good enough" (arguably already with GPT3.5, as evidenced by free ChatGPT running on GPT3.5-turbo).
There has been promising work about making models smaller without losing performance (e.g. by training them longer), or quantizing the weights (llama.cpp etc) to make the model cheaper to run. I believe we will see a lot more focus on this over the next couple years, and given how there's been comparatively little work done in this direction I wouldn't be surprised by 10x or 100x efficiency gains.
- SubiculumCode 3y agoefficiency efficiency efficiency will be the call. How do we reduce model size without losing ability. Can hardware be designed to run a particular set of weights efficiently?
- theRealMe 3y agoI don’t want to make unsubstantiated guesses, but we should remember that GPT3.5 was originally just GPT3.5 and when they came out with turbo it not only sped it up but also reduced the cost (to us) by like 10x of what it used to be if I remember right. So they are (or at the very least “were”) working on and succeeding on reducing its cost to run.
- sharemywin 3y agolet's not forget about good old fashion moore's law. which I believe in GPUs is even faster. and I'm not sure if some of these exotics Hardware architectures will pane out or not either.
- flangola7 3y agoFaster is an understatement. On password hashing the margin of safety dropped by 30x since 2020 due to GPU improvements.