3 ms·
Someone pointed out that the energy cost of training GPT is roughly on par with a single transcontinental flight. If so, I don't think this is a limiting factor
by int_19h 4y ago
Someone pointed out that the energy cost of training GPT is roughly on par with a single transcontinental flight. If so, I don't think this is a limiting factor in any meaningful sense - you could spend that much energy daily, and it would still be a drop in the bucket overall for any moderately large business.
- bamboozled 4y agoI would imagine trying new datasets on daily basis wouldn’t be trivial ?
- Xelynega 4y agoThe bottleneck would be the number of workers on sites like mechanical turk available to create the datasets. Might take a few more years before amazon and facebook get enough third world countries to the point they can exploit their labour online to create daily training sets.