4 ms·

It's still millions of parameters. What kind if hardware is needed to train a 10 million parameter model?

• solarmist 5y ago
  Like could I train it in a week with a few GPU's? Or would this still require a cluster to train in a reasonable amount of time?
• greens 5y ago
  A single consumer GPU for ~hours
  • binarymax 5y ago
    Yes, verbatim from the paper: "Moreover, training with PET can be performed in several hours on a single GPU without requiring expensive hyperparameter optimization."
    • solarmist 5y ago
      Nice. Still reading it.
  • solarmist 5y ago
    Oh! Nice! The a much more accessible than I was expecting.