4 ms·
llama can run on an m1. T5 still needs a specialized gpu
by itake 4y ago
llama can run on an m1. T5 still needs a specialized gpu
- DoctorOetker 4y agoWhat is the reason T5 needs a specialized GPU and Llama doesn't? In the end they are mathematical models, so what would prevent someone from loading T5 into a machine with plenty of RAM (like a server)? Would the codebase truly require that much refactoring? How difficult would it be to rewrite the model arhitecture as a set of mathematical equations (Einstein summation) and reimplement inference for CPU?
- itake 4y agoI'm far from an expert in this area. But llama has been updated so anyone can hack with it on their m1 macbook (which many developers have). If someone updated T5 to be as easy to dev against, then I am sure they would see similar community interest. Most people don't have the hardware or budget to access these specialized high vram GPUs.