4 ms·
You can't outspend the industry labs given the compute inflation in transformer architectures (unless you are ridiculously well connected in the venture/soverei
by htrp 2y ago
You can't outspend the industry labs given the compute inflation in transformer architectures (unless you are ridiculously well connected in the venture/sovereign funding communities).
And realistically, do we need another GPT4 evaluation paper?
- ein0p 2y agoThat is by far not the only thing industry labs are working on currently. I work in one. My group might be unusual, but I can’t name a single currently active project here that is not a departure from Transformers one way or another. I expect a ton of such efficiency oriented work in the next 4-5 years. We can’t be burning money as inefficiently as we do it right now.