3 ms·
I don't think Moore's law is the most important factor, instead algorithmic improvements will enable to have smaller models that are as capable as these humonge
by Tepix 4y ago
I don't think Moore's law is the most important factor, instead algorithmic improvements will enable to have smaller models that are as capable as these humongeous models.
Llama 65b is hopefully just the beginning of this trend. It outperforms OPT-175b.
- buildbot 4y agoThat holds true if scaling laws don’t hold true, otherwise a hypothetical Llama 175b would be even better. So the high end will always be on big clusters.
- deleted 4y ago[deleted]
- 2OEH8eoCRo0 4y ago'Moore's Law' was in quotes for a reason.