buyucu

23 ms·

541.
by buyucu 2y ago
I did not assume the worst when submitting the post, but that is also my suspicion. The whole thing is very dodgy.
542.
by buyucu 2y ago
I totally agree that ollama needs competition. They have been doing very sketchy things lately. I wish llama.cpp had an alternative wrapper client like ollama.
543.
by buyucu 2y ago
I use ollama because llama.cpp dropped support for vlms. I would happily switch back if llama.cpp starts supporting vlms again.
544.
by buyucu 2y ago
yes, it's a convenience wrapper around llama.cpp
545.
by buyucu 2y ago
I was not aware of this context, thanks!
546.
Llama.cpp supports Vulkan. why doesn't Ollama?(github.com)
217 points by buyucu 2y ago|228 comments
547.
by buyucu 2y ago
llama.cpp has supported vulkan for more than a year now. For more than 6 months now there has been an open PR to add vulkan backend support for Ollama. However, Ollama team has not even looked at it or commented on it. Vulkan backends ar
548.
by buyucu 2y ago
I'm not suprised
549.
by buyucu 2y ago
This probably means that FBI has done way worse stuff than this. It just wasn't exposed yet.
550.
by buyucu 2y ago
not the whole model obviously since it just came out. but people have been successful in replicating the core RL principle behind it.
551.
by buyucu 2y ago
looks like azure supports deepseek now: https://azure.microsoft.com/en-us/blog/deepseek-r1-is-now-av...
552.
by buyucu 2y ago
give deepseek a few weeks to catch up. they probably got waaay more customers than they originally anticipated.
553.
by buyucu 2y ago
I hope this moves the masses from closed models to open models.
554.
by buyucu 2y ago
both llama.cpp and vllm support inference with rocm or vulkan. inference is the easiest thing to decouple from nvidia.
555.
by buyucu 2y ago
llm inference is fine on rocm. llama.cpp and vllm both have very good rocm support. llm training is also mostly fine. I have not encountered any issues yet. most of the cuda moat comes from people who are repeating what they heard 5-10 ye
556.
by buyucu 2y ago
that will have no effect. best frontier models are now chinese.
557.
by buyucu 2y ago
My company recently switched from A100s to MI300s. I can confidently say that in my line of work, there is no CUDA moat. Onboarding took about month, but afterwards everything was fine.
558.
by buyucu 2y ago
I have a Xiaomi phone, a Huawei Matebook and a BYD car. I guess I am pivoting to China after all :)
559.
by buyucu 2y ago
And yet, Huawei is still doing fine.
560.
by buyucu 2y ago
It's too late for that. That ship sailed a long time ago. The best language model right now is open source. Let that sink in.
561.
by buyucu 2y ago
I'm willing to bet ''ban DeepSeek'' voices will start soon. Why compete, when you can just ban?
562.
by buyucu 2y ago
I have no sympathy for OpenAI here. They are (allegedly) a non-profit with open in the title that refuse to open-source their models. They are now upset at a startup who is more loyal to OpenAI's original mission that OpenAI is today.
563.
by buyucu 2y ago
this is how science works.
564.
by buyucu 2y ago
It's hilarious how the discussion in the PR is not about the PR itself, but about the implications of this.
565.
by buyucu 2y ago
scarecity is a great motivator for innovation. abundance breeds complacency. I meant big in terms of innovation, not budget.
566.
by buyucu 2y ago
there are smaller distillations all the way down to 1.5b parameters. I'm running 7b on my laptop. https://ollama.com/library/deepseek-r1
567.
by buyucu 2y ago
deepseek fine-tuned qwen32b with data generated by deepseek671b
568.
by buyucu 2y ago
There are two things. First, deepseek v3 and r1 are both amazing models. Second, the fact that deepseek was able to pull this off with such modest resources is an indication that there is no moat, and you might wake up tomorrow and find an
569.
by buyucu 2y ago
DeepSeek-R1-Distill-Qwen-32B is my new default model on my home server. previously it was aya-32b.
570.
by buyucu 2y ago
this is hilarious. hubris is a terrible thing. there is always bigger fish.

More ›