4 ms·

Probably the best time to start train 100T model is never

• samstave 3y ago
  What if you could train an AI with a desired outcome to their answers? I.E. ; "answer this question where the outcome is the most beneficial to quality of life"
  • moffkalast 3y ago
    > life A strange game. The only winning move is not to play. unplugs self
  • chromoblob 3y ago
    A properly designed AI agent would do exactly that.
  • EvgeniyZh 3y ago
    I'll take your question further: what if we have unlimited data (say some crazy rich RL environment or way to produce high quality and diverse synthetic data)? You still have to get these 1e30 FLOPs. Lets say you can connect 100 of these bad boys together with 40% utilization, with total 4e19 FLOPs/s. Assume also Moore's law keeps working indefinitely. When should we start training 100T model on it to get is as early as possible? We wait x years and the start training on machine with 4e19*2^(x/2) FLOPs/s. Turns out the answers is ~16 years, after which we'll have 1e22 FLOPs/s and 1e30 FLOPs will take another 3 years.