2 ms·
It’s odd you’re invoking moore’s law here, which actually isn’t a law, and is well understood and accepted over at least the last decade across business and aca
by JohnMakin 2y ago
It’s odd you’re invoking moore’s law here, which actually isn’t a law, and is well understood and accepted over at least the last decade across business and academia that it is slowing and is not limitless. As per what “other” advances you’re referring to, I can’t speculate - maybe quantum computing? We are quite a long way away from that being a commercial reality.
- Herring 2y agoIt's mainly hardware upgrades (new TSMC nodes), training data quantity/quality improvements, and model architecture (and broader software) changes. Here's a fun project I've had my eye on recently: https://github.com/KellerJordan/modded-nanogpt/ https://github.com/KellerJordan/modded-nanogpt/