4 ms·
> I mean we could be far overfeeding our data models too. As far as I know evidence we have suggests the opposite, improvements are still mostly coming from mo
by morelisp 4y ago
> I mean we could be far overfeeding our data models too.
As far as I know evidence we have suggests the opposite, improvements are still mostly coming from more parameters and more data.
> data models don't take years in real time to train either.
Given the incommensurate architectures a fairer calculation of learning rate might be Wh - a human brain needs about 500Wh a day, GPT-3 was suspected to take about 1GWh to train.