2 ms·
I mean we could be far overfeeding our data models too. Of course data models don't take years in real time to train either.
by pixl97 4y ago
I mean we could be far overfeeding our data models too. Of course data models don't take years in real time to train either.
- morelisp 4y ago> I mean we could be far overfeeding our data models too. As far as I know evidence we have suggests the opposite, improvements are still mostly coming from more parameters and more data. > data models don't take years in real time to train either. Given the incommensurate architectures a fairer calculation of learning rate might be Wh - a human brain needs about 500Wh a day, GPT-3 was suspected to take about 1GWh to train.