2 ms·
I think you are right, the next 33 years are likely to be very different. I would lean to them being even more dramatic, due to the opportunity to advance algo
by Nevermark 3y ago
I think you are right, the next 33 years are likely to be very different.
I would lean to them being even more dramatic, due to the opportunity to advance algorithms, not just resources.
On the more obvious side, most libraries are not yet taking full advantage of many known gradient optimization techniques. It’s been so much easier to just add data & processing that there is an overhangs of tools to still apply.
And large successful models are telling us important things.
For instance, it is clear that language models are learning a kind of logic of language similar to how we process thoughts, allowing highly disparate types information to be woven together sensibly.
At some point, identifying the nature of that processing could radically simplify language processing.
That is just one opportunity for radical architecture and algorithm advances, and it would be revolutionary.