3 ms·

And if you add in heavy sparsification it should fit and run on a raspberry pi.

• rvz 1y ago
  So in around 6 months, we will see that the person who bought this H200 in the listing just got scammed for $250k and will realize that you just needed specific quantizations to the model and a few optimizations to run locally. Unless they want to train their own model, buying this for inference for $250k is unnecessary and still isn't enough for a full production deployment.
• vFunct 1y ago
  It's already sparsified from the 150T parameter model..
  • yorwba 1y ago
    It took me several hours to realize that 150 trillion parameters is a reference to the number of synapses in a human brain.