4 ms·
Serve OPT-175B with Alpa using commodity GPUs: https://alpa-projects.github.io/tutorials/opt_serving.html https://alpa-projects.github.io/tutorials/opt_serving.
by zhisbug 4y ago
Serve OPT-175B with Alpa using commodity GPUs: https://alpa-projects.github.io/tutorials/opt_serving.html https://alpa-projects.github.io/tutorials/opt_serving.html
If you have difficulties accessing 8x 80GB A100 (or AWS P4) this might be a good use case.