5 ms·
Well, the number of parameters is directly related to the size of the machine to keep the model around. And so while you can host GPT-2 on a single consumer GPU
by fock 4y ago
Well, the number of parameters is directly related to the size of the machine to keep the model around. And so while you can host GPT-2 on a single consumer GPU, for ChatGPT you allegedly need 4-8-10k-a-piece GPUs. Which also consume 8-16 the power of your consumer GPU. So if you continue that, the cost to society will be too high to bear for this kind of games. (and in my eyes already is, as the outputs of large generative models are 100% used to generate more stuff to consume for the same amount of already information-overloaded people)
- ryan93 4y agoA single international flight literally consumes a swimming pools worth of fuel. a few GPUs is nothing especially as we transition to wind and solar.
- fock 4y agowell, then stop flying and the useless content-creation and let's get back to our "few GPUs" for consumption, when living, food and flying have transitioned to wind and solar. Then people can start setting up their text-based Holodeck-precursor consuming 1kW during usage. Just be clear: the alternative to letting a future GPT-derivative write your novel is to go to the library and pick a book. That works now and has negligible emissions. A future GPT (currently you won't get a consistent long form text) might do the same - optimistically it will do it on only 2-3kWh of energy (which amounts to the cost of creating a physical book). Granted, you only can sustain a human on that for a day, but then the humans don't go away because of AI. Conclusion to me: the things are fascinating and so are H-bombs... Question: which kind of time do you expect people to spend with these models? Which human needs will they satisfy? How will their energy consumption then still be negligible?
- tomrod 4y ago> Well, the number of parameters is directly related to the size of the machine to keep the model around. I bet this has some potential.