In principal you can run it on just about any hardware with enough storage space. It's just a question of how fast it will run. This readme has some benchmarks

In principal you can run it on just about any hardware with enough storage space. It's just a question of how fast it will run. This readme has some benchmarks with a similar set of models (and the code has support for even swapping data out to disk if needed): https://github.com/FMInference/FlexGen https://github.com/FMInference/FlexGen And here are some benchmarks running OPT-175B purely on (a very beefy) CPU machine. Note that the biggest llama model is only 65.2B: https://github.com/FMInference/FlexGen/issues/24 https://github.com/FMInference/FlexGen/issues/24