buyucu

22 ms·

511.
by buyucu 2y ago
Anthropic CEO is just mad that there is a new player in the business. DeepSeek models are not just better, they are also open weight. I'm sure Anthropic's valuation got hit badly by the DeepSeek saga, and the CEO is lashing out i
512.
by buyucu 2y ago
Deepseek is much more trustworthy than OpenAI. Deepseek released the weights of their top language model. I can host and run it myself. Does OpenAI do the same? Thanks, but no thanks! I won't be using ClosedAI.
513.
by buyucu 2y ago
why should anyone use this when deepseek is free/cheaper? openai is no longer relevant.
514.
by buyucu 2y ago
I'm willing to bet that Vulkan will outperform OpenVINO. Vulkan is the API right now in the graphics world. It's very well supported and actively being improved on. Everyone is pouring resources into making Vulkan better. OpenVI
515.
by buyucu 2y ago
It depends on your GPU. Vulkan is well-supported by essentially all GPUs. AMD support ROCm well for their datacenter GPUs, but support for consumer hardware has not been as good.
516.
by buyucu 2y ago
ollama is giving people a lot of reasons to switch: https://news.ycombinator.com/item?id=42886680
517.
by buyucu 2y ago
the pr was working 6 months ago and it has been rebased multiple times as the ollama team kept ignoring it and mainline moved. I'm using it right now.
518.
by buyucu 2y ago
deepseek is huge with 671b parameters. they keep it in hard disk, and load it piece by piece to the ram. the innovation is that they kick out everything other than the kv cache from the ram.
519.
by buyucu 2y ago
Thanks for this context, I will give RamaLlama a try!
520.
by buyucu 2y ago
I applaud how hardcore this is. Swapping the model from disk and just keeping the KV cache on the CPU ram.
521.
by buyucu 2y ago
llama.cpp already supports Vulkan. This is where all the hard work is at. Ollama hardly does anything on top of it to support Vulkan. You just check if the libraries are available, and get the available VRAM. That is all. It is very si
522.
by buyucu 2y ago
I literally start the sentence with 'I disagree with this'
523.
by buyucu 2y ago
this is great! I think pufferfish is taking PRs to his fork as well.
524.
by buyucu 2y ago
I didn't say it was a good point. I said I disagree, but it's a respectable opinion I could imagine someone having.
525.
by buyucu 2y ago
This is incorrect. The effort it took to enable Vulkan was relatively minor. The PR is short and to be honest it doesn't do much, because it doesn't need to.
526.
by buyucu 2y ago
the owner of that PR has already forked ollama. try it out. I did and it works great.
527.
by buyucu 2y ago
it's a grey zone but vlms are effectively not being developed anymore.
528.
by buyucu 2y ago
If you support Vulkan, you support almost every GPU out there in the consumer market across all hardware vendors. It's an amazing fallback option. I agree they should also support OpenVINO, but compared to Vulkan OpenVINO is a tiny ma
529.
by buyucu 2y ago
supporting vulkan will help ollama reach the masses who don't have dedicated gpus from nvidia. this is such a low hanging fruit that it's silly how they are acting.
530.
by buyucu 2y ago
this pr was reviewable multiple times, rebased multiple times. all because ollama team kept ignoring it. it has been open for almost 7 months now without a single comment from the ollama folks.
531.
by buyucu 2y ago
the work involved is tiny compared to the work llama.cpp did to get vulkan up and running. this is not rocket science.
532.
by buyucu 2y ago
ollama has been advertising llama.cpp features as their own, which I find very dishonest.
533.
by buyucu 2y ago
with vulkan it runs much much faster on consumer hardware, especially opn igpus like intel or amd.
534.
by buyucu 2y ago
check the PR, it's a very short one. It's not more complicated than setting a compile time flag. I have no idea why they have been ignoring it. Ollama is just a friendly front end for llama.cpp. It doesn't have to do any of
535.
by buyucu 2y ago
isn't lm stuido closed source?
536.
by buyucu 2y ago
Llama.cpp dropped support for multimodal vlms. That is why I am using ollama. I would happily switch back if I could.
537.
by buyucu 2y ago
I disagree with this, but it's a reasonable argument. The problem is that the Ollama team has basically ignored the PR, instead of engaging the community. The least they can do is to explain their reasoning. This PR is #1 on their re
538.
by buyucu 2y ago
I agree they should rename them. But defaulting to a 671b model is also evil.
539.
by buyucu 2y ago
ollama advertising llama.cpp features as their own is very dishonest in my opinion.
540.
by buyucu 2y ago
ollama is not doing anything. llama cpp does all that work. ollama is just a small wrapper on top.

More ›