4 ms·

DumpsterCluster: From Dumpster Diving to Serving Llama-70B on $60 GPUs

• tacomagick 2mo ago
  It appears badwidth is the limiting factor for these projects, because the Tokens per second appears really low even for the compared "proper" setup