3 ms·

My $3000 (custom) consumer desktop runs llama2 70B at ~3-6 t/s iirc. Boston Dynamics can absolutely get LLMs to run on these things.

• gs17 3y ago
  That desktop would add a lot to the power requirements, though.