2 ms·
> if we can figure out what the heck "major" trained-from-scratch models are and if we can figure out some objective source for tracking Hmm. Yeah, we'll need
by margalabargala 3y ago
> if we can figure out what the heck "major" trained-from-scratch models are and if we can figure out some objective source for tracking
Hmm. Yeah, we'll need to hammer out a solid definition. Further complicating things are models that may not be publicly available and are internally used by companies, though those may not be trained from scratch.
I would be fine with your suggestion to frame it in terms of percent power generation, though it might be hard to disentangle training costs from usage costs from that number. I would argue that including usage energy cleanliness is in the "spirit" of the bet but I'm happy to try to disentangle it as I originally proposed training-only.
> Something else to consider is the algorithmic improvements between now and 2030. From Yann LeCunn: Training LLaMA 13B emits 24 times less greenhouse gases than training GPT-3 175B yet performs better on benchmarks.
This is an excellent point, and definitely works in your favor. Really I'm on the good side of this bet. I win either way :) either my charity makes money, or I'm pleasantly surprised by climate impacts.
I've also not used longbets before. I would think we want to hammer out exact terms here before we set something up there?