DiabloD3

23 ms·

391.
by DiabloD3 1y ago
I mean, if I thought I had a plan to be the guy who saves Intel from it's own mistakes, I'd be optimistic too. Also, I looked into the claim when he had said it, apparently he was being intentionally misleading about it, and the p
392.
by DiabloD3 1y ago
I love how they don't actually explain why I (or anyone else) would ever implement their API. Given how disastrous the AI 'industry' has been, between misappropriating data from customers, performing actions on behalf of cust
393.
by DiabloD3 1y ago
Yep, Dov Frohman's contribution to tech is well known and very appreciated. Edit: Look, to whoever is out there on a downvote spree, I don't care if I get downvoted, man, but wild you'd just downvote people talking about a gu
394.
by DiabloD3 1y ago
AIPAC was founded in 1954.
395.
by DiabloD3 1y ago
Yes. I can give you a non-technical answer, since HN is ostensibly business as well. Intel fired the one CEO that spoke both engineer and business, and Gelsinger could have been their Lisa Su. They fired the only talented CEO they've h
396.
by DiabloD3 1y ago
What a strange way for Microsoft to announce they're ready for round two with the DoJ.
397.
by DiabloD3 1y ago
Your pet theory needs work. 1) I have no issues throwing ROCm (which still uses its own path in the driver) at my Radeon at the same time I'm hammering it with "normal" path APIs (Legacy D3D, D3D12, OpenGL, Vulkan, etc), they
398.
by DiabloD3 1y ago
What about them? RDNA4 is officially supported on ROCm (release for that came out shortly after the drivers shipped), and PyTorch officially supports ROCm and AMD officially supports PyTorch's ROCm target.
399.
by DiabloD3 1y ago
For a great article about typefaces, the size of text on that website sure is small.
400.
by DiabloD3 1y ago
Ollama isn't an inference engine, its a GUI slapped onto a perpetually out-of-date vendored copy of Llama.cpp underneath. So, if you're trying to actually count LLama.cpp downloads, you'd combine those two. Also, I imagine mo
401.
by DiabloD3 1y ago
Ollama themselves upload models to the mirror, and often mislabel them. When R1 first came out, for example, their official copy of it was one of the distills labeled as "R1" instead of something like "R1-qwen-distill".
402.
by DiabloD3 1y ago
You can disagree all you want, but Ollama does not keep their llama.cpp vendored copy up to date, and also ships, via their mirror, completely random badly labeled models claiming to be the upstream real ones, often misappropriated from m
403.
by DiabloD3 1y ago
Recommending Ollama isn't useful for end users, its just a trap in a nice looking wrapper.
404.
by DiabloD3 1y ago
Wow, that is you.
405.
by DiabloD3 1y ago
I'm not going to open an issue on this, but you should consider expanding on the self-hosting part of the handbook and explicitly recommend llama.cpp for local self-hosted inference.
406.
by DiabloD3 1y ago
That's not entirely wrong. https://gpuopen.com/download/RDNA_Architecture_public.pdf I've been showing this one to people for a few years as a good introduction on how RDNA diverged from GCN->CDNA. The mai
407.
by DiabloD3 1y ago
There isn't an RDNA5 on the roadmap, though. It's been confirmed 4 is the last (and was really meant to be 3.5, but grew into what is assumed to be the PS5/XSX mid-gen refresh architecture). Next is UDNA1, a converged archite
408.
by DiabloD3 1y ago
Yep, it is me.
409.
by DiabloD3 1y ago
This is the most tsoding thing possible.
410.
by DiabloD3 1y ago
I wouldn't recommend Ubuntu, though. One of the biggest faults of Linux is we don't have an easy, user friendly, idiot proof distro for normies, but Ubuntu is just broken corporate slop. When I was wearing the various "save u
411.
by DiabloD3 1y ago
Shame this is written in Python, looks very interesting, but I'm no expert in this field. If there is anything here worth using, it's entirely possible that the llama.cpp crew can save it from vanishing into obscurity.
412.
by DiabloD3 1y ago
I have not read this for like 20 years. It's still as good and as absolutely batshit as I remember. A lot of commenters here talk about how its "problematic" or "squick", but that's mostly the point: this isn&#
413.
by DiabloD3 1y ago
I'm aware. One of the biggest things in fixing the Linux desktop was no longer needing drivers in the Xserver and needing it to be suid root. What was linked is written in Python. Nothing in Python is ever going to be a userland driver
414.
by DiabloD3 1y ago
Don't use Ollama, use llama.cpp instead.
415.
by DiabloD3 1y ago
They were doing this before, the difference with this is, the version of ROCm you use is locked to the driver versions that are supported, which is a very narrow range. With this new thing, the backend API is now formalized and easier to su
416.
by DiabloD3 1y ago
AMD is extremely bad at communications. The driver already contains everything ROCm requires to talk to the GPU, and ROCm itself is only a SDK that contains runtimes, libraries, and compilers. This part of TinyGrad is not a driver, however
417.
by DiabloD3 1y ago
You're conflating two different things. ROCm isn't part of AMD drivers, its a software library that helps you support legacy compute APIs and stuff in the BLAS/GEMM/LAPACK end of things. The part of ROCm you're inte
418.
by DiabloD3 1y ago
Q4_K_M is the "default" for a lot of models on HF, and they generally require ~20GB of VRAM to run. It will not fit entirely on a 16GB card. You want to be about 3-4GB VRAM on top of what the model requires. A back of the envelope
419.
by DiabloD3 1y ago
I'd suggest buying a better GPU, only because all the models you want need a 24GB card. Nvidia... more or less robbed you. That said, Unsloth's version of Qwen3 30B, running via llama.cpp (don't waste your time with any other
420.
by DiabloD3 1y ago
No, I do absolutely remember them integrating dotnet-script-like functionality into the runtime, and I remember experimenting with it. Was it only in a beta version and removed to re-appear now? It used the Rosyln compiler to compile on dem

More ›