3 ms·

You did not count the gathering of data and then training the LLM.

• lambdaba 2y ago
  Inference consumes far more total compute than training the model.
  • MrMcCall 2y ago
    By 'inference', do you mean using the model to compute a specific series of results?
    • lambdaba 2y ago
      Yes, training is a tiny amount of usage, this is well-known.
      • MrMcCall 2y ago
        Well, if that's the case, then the situation is worse than I imagined. I thought that training the model would be far more expensive computationally than using it to compute an 'answer'. Thanks.
        • MF-DOOM 2y ago
          Training is much more computationally intensive than computing a _single_ answer. But not compared to running the model over time at scale for millions of users.
          • MrMcCall 2y ago
            Thanks. That is precisely how I intuited the situation. 40+ years of programming FTW!
  • deleted 2y ago
    [deleted]