4 ms·

Tomorrow my desktop computer hopefully

• mattwad 3y ago
  i run a home media server, can't wait to be able to add my own LLM service. it's just a matter of time for it to be something i can install over the weekend with proper hardware
  • proaralyst 3y ago
    Have you tried https://ollama.com/ https://ollama.com/ ? You may find you already can
  • gryfft 3y ago
    git clone https://github.com/ggerganov/llama.cpp cd llama.cpp make ./server -m models/7B/ggml-model.gguf -c 2048 I don't think it'll take you the whole weekend :)
• PeterisP 3y ago
  I really doubt it. "Tomorrow" your desktop computer might be twice powerful but at the same time the "good model of tomorrow" will be four or ten times larger - I'd expect that the gap between what can be done locally versus what is offered as a service will grow, not shrink.
  • Ukv 3y ago
    The diminishing returns from model scale means that if your personal computer improves twofold in the same time a datacenter improves fivefold, you may still have lessened the gap in terms of quality. That doesn't mean you'll be able to run the best model, but I'm relatively optimistic about the gap not growing out of control.
    • PeterisP 3y ago
      Well, sure, one thing is that the absolute numbers do increase, so for any given notion of "good enough", every device at some point will reach the level where it will be able to run it.