5 ms·

Sure, but eventually we’re going to hit on environmental and cost-effective power limits of training, and it’s not worth the cost to train the model. AFAIUI, t

Sure, but eventually we’re going to hit on environmental and cost-effective power limits of training, and it’s not worth the cost to train the model. AFAIUI, that’s part of the point that Gebru was trying to make before she was fired.

visarga 4y ago
Gebru doesn't have anything good to say about AI, it's only downsides. She's biased being an activist and all.
- clankyclanker 4y ago
  Her position doesn’t invalidate her arguments. How good is a product that can’t stand up to criticism? Edit: to tie it back to the original criticism: what’s the maximum training cost we’re willing to accept for the model? How can we guarantee return greater than the increased training cost?
rpdillon 4y ago
This makes me wonder what the costs of training an actual human artist are. Are AIs less efficient?
- sinenomine 4y ago
  You don't have to retrain an AI to spin up another instance - just download a 4gb weight file via a magnet link floating around the internet and run some python code in terminal on your old PC. This kills the comparison. And training a real living breathing person in a rich OECD country is going to be costly - no offense meant, I'm actually not from OECD.
- notahacker 4y ago
  You can create a lot more copies of the AI. But then again, when it comes to commercial needs, a human doesn't need "retraining" every time you ask them to draw something they weren't familiar with when they went through art school...
sinenomine 4y ago
For now I can run my stable diffusion on a vintage laptop from a decade ago, on CPU (!), and it doesn't even utilize most of my RAM. And training this model was still cheap compared to, say, a google senior engineer yearly salary. The limits of scaling are further than laymen may want to believe. With an order of magnitude more parameters it won't just do hands, it will do quite a bit more.

visarga 4y ago

Gebru doesn't have anything good to say about AI, it's only downsides. She's biased being an activist and all.

clankyclanker 4y ago
Her position doesn’t invalidate her arguments. How good is a product that can’t stand up to criticism? Edit: to tie it back to the original criticism: what’s the maximum training cost we’re willing to accept for the model? How can we guarantee return greater than the increased training cost?

rpdillon 4y ago

This makes me wonder what the costs of training an actual human artist are. Are AIs less efficient?

sinenomine 4y ago
You don't have to retrain an AI to spin up another instance - just download a 4gb weight file via a magnet link floating around the internet and run some python code in terminal on your old PC. This kills the comparison. And training a real living breathing person in a rich OECD country is going to be costly - no offense meant, I'm actually not from OECD.
notahacker 4y ago
You can create a lot more copies of the AI. But then again, when it comes to commercial needs, a human doesn't need "retraining" every time you ask them to draw something they weren't familiar with when they went through art school...

sinenomine 4y ago

For now I can run my stable diffusion on a vintage laptop from a decade ago, on CPU (!), and it doesn't even utilize most of my RAM. And training this model was still cheap compared to, say, a google senior engineer yearly salary. The limits of scaling are further than laymen may want to believe. With an order of magnitude more parameters it won't just do hands, it will do quite a bit more.