3 ms·
I think one big difference might be that we are limited by hardware. For this to be true, the models and algorithms will need to become exponentially more effic
by icapybara 4y ago
I think one big difference might be that we are limited by hardware. For this to be true, the models and algorithms will need to become exponentially more efficient over time, because the hardware won't.
- credit_guy 4y agoWell, right now we think performance comes from software plus hardware. I think going forward, performance will come from software, hardware and training. Chances are that ChatGPT in the current configuration (i.e. same NN architecture and number of parameters) run on the current hardware would be much more powerful if you could continue training it with 10 times more data than you used so far. Separate from that, I think you could also increase tremendously ChatGPT's power if you could pair it with some regular old-fashioned computer. For example, if you ask ChatGPT to write some python code, it produces code that is surprisingly close to correct. But why not give ChatGPT a separate module or sandbox where it can actually run and test that code?