3 ms·

Is there a reason Llama is getting so much attention compared to say T5 11B? Not sure how neutral or what benchmarks are used on the following link, but T5 see

Is there a reason Llama is getting so much attention compared to say T5 11B? Not sure how neutral or what benchmarks are used on the following link, but T5 seems to sit a lot higher on this leaderboard? https://accubits.com/large-language-models-leaderboard/ https://accubits.com/large-language-models-leaderboard/

itake 4y ago
llama can run on an m1. T5 still needs a specialized gpu
- DoctorOetker 4y ago
  What is the reason T5 needs a specialized GPU and Llama doesn't? In the end they are mathematical models, so what would prevent someone from loading T5 into a machine with plenty of RAM (like a server)? Would the codebase truly require that much refactoring? How difficult would it be to rewrite the model arhitecture as a set of mathematical equations (Einstein summation) and reimplement inference for CPU?
  - itake 4y ago
    I'm far from an expert in this area. But llama has been updated so anyone can hack with it on their m1 macbook (which many developers have). If someone updated T5 to be as easy to dev against, then I am sure they would see similar community interest. Most people don't have the hardware or budget to access these specialized high vram GPUs.
barbariangrunge 4y ago
Is llama open source? I heard it was pirated from Facebook
- DoctorOetker 4y ago
  I did not claim llama was open source, but I see the url I posted insinuates that (probably for a contorted meaning of open source, as in source available for approved academics). Anyway, T5 being available for download from Huggingface only makes my question more pertinent...
  - w4ffl35 4y ago
    I made an app for running t5 locally - compiled version allows you to run without installing anything. https://capsizegames.itch.io/chat-ai https://capsizegames.itch.io/chat-ai https://github.com/Capsize-Games/chatai https://github.com/Capsize-Games/chatai
    - DoctorOetker 4y ago
      interesting, what are the hardware requirements? does it happen to run on CPU on a server with 96GB RAM?
      - w4ffl35 4y ago
        the compiled app is meant for people to install and use with their GPU and runs on as low as a GTX 1080. I haven't tested against CPU only builds. You can take a look at the source code and see if it would be useful to you.

itake 4y ago

llama can run on an m1. T5 still needs a specialized gpu

DoctorOetker 4y ago
What is the reason T5 needs a specialized GPU and Llama doesn't? In the end they are mathematical models, so what would prevent someone from loading T5 into a machine with plenty of RAM (like a server)? Would the codebase truly require that much refactoring? How difficult would it be to rewrite the model arhitecture as a set of mathematical equations (Einstein summation) and reimplement inference for CPU?
- itake 4y ago
  I'm far from an expert in this area. But llama has been updated so anyone can hack with it on their m1 macbook (which many developers have). If someone updated T5 to be as easy to dev against, then I am sure they would see similar community interest. Most people don't have the hardware or budget to access these specialized high vram GPUs.

barbariangrunge 4y ago

Is llama open source? I heard it was pirated from Facebook

DoctorOetker 4y ago
I did not claim llama was open source, but I see the url I posted insinuates that (probably for a contorted meaning of open source, as in source available for approved academics). Anyway, T5 being available for download from Huggingface only makes my question more pertinent...
- w4ffl35 4y ago
  I made an app for running t5 locally - compiled version allows you to run without installing anything. https://capsizegames.itch.io/chat-ai https://capsizegames.itch.io/chat-ai https://github.com/Capsize-Games/chatai https://github.com/Capsize-Games/chatai
  - DoctorOetker 4y ago
    interesting, what are the hardware requirements? does it happen to run on CPU on a server with 96GB RAM?
    - w4ffl35 4y ago
      the compiled app is meant for people to install and use with their GPU and runs on as low as a GTX 1080. I haven't tested against CPU only builds. You can take a look at the source code and see if it would be useful to you.