SwellJoe

28 ms·

751.
by SwellJoe 7mo ago
You can run pretty much every model on Vulkan, including the Qwen MoE models. You can also run pretty much every model on ROCm, Apple Silicon via MLX, and Intel hardware via OpenVINO. Nvidia got there first, but they're no longer clear
752.
by SwellJoe 7mo ago
Any human scale "attack", e.g. the made up Everybody Loves Raymond episode isn't doing anything to hurt LLM training data. Might even help them detect exaggeration, satire, etc. when read in context and with other knowledge t
753.
by SwellJoe 7mo ago
True, but most manufacturers don't go out of their way to break their old devices. Neglect is one thing, this feels more like theft.
754.
by SwellJoe 7mo ago
I mean, there are some positive uses for the technology, some will likely save lives and advance the frontier in medicine and science. The ways it's able to automate research tasks is a pretty big deal. And, even though I know that,
755.
by SwellJoe 7mo ago
"sufficient resources" is going to be a lot of resources. I doubt this will run on even something like a Strix Halo or DGX Spark, even at 1-bit quantization. You'll need a 256GB or 512GB Mac Studio, or a monster GPU situation
756.
by SwellJoe 7mo ago
A trillion parameters is wild. That's not going to quantize to anything normal folks can run. Even at 1-bit, it's going to be bigger than what a Strix Halo or DGX Spark can run. Though I guess streaming from system RAM and disk ma
757.
by SwellJoe 7mo ago
Interesting that the best performers are all Chinese-made models (DeepSeek and Qwen also perform consistently well). I wonder if there's more focus on vision and illustration in their training, or if something else is leading to their
758.
by SwellJoe 7mo ago
We got an overachiever, here. Kimi sounds like a teacher's pet kind of name.
759.
by SwellJoe 7mo ago
This shit is so dark. I mean, popular music has always been pretty formulaic, and prone to imitation and trite bullshit, but at least when humans were making it, you'd occasionally get some spark of genius, real originality, even in th
760.
by SwellJoe 7mo ago
The thing is, Opus 4.5 is where the model reached Good Enough, at least for a wide variety of problems I use LLMs for. Before that, I almost never thought it was a more productive use of my time to use AI for development tasks, because it w
761.
by SwellJoe 7mo ago
It can be tricky to get/keep ROCm working, but around 7.2 it became reliable and as fast as or faster than ROCm 6.4. And, I think the first response time of ROCm is pretty consistently faster than Vulkan, even if Vulkan has a slightly
762.
by SwellJoe 7mo ago
I hate the AI slop prose, but I like the idea of doing stuff on-device. If you're willing to spend a little more than $100, you can actually run not just an agent, but pretty decent local models on a phone or tablet. Gemma 4 E4B-it run
763.
by SwellJoe 7mo ago
Which agents work in Termux? Maybe Pi? (I'm surprised Claude Code doesn't, but I guess Claude Code is a bonkers weird design for a terminal app, doing a lot of very unusual things.)
764.
by SwellJoe 7mo ago
I dunno. GPU programming and performance is above my pay grade. I assume the reason every GPU maker is investing in software is because they understand the problems to be solved and feel it's worth the investment to solve them. I lik
765.
by SwellJoe 7mo ago
Yeah, that kinda sucks, but, all their new generation onboard GPUs are supported by ROCm. e.g. Ryzen AI 395 and 400 series which will be found in mid-to-high end laptops and desktops and motherboards. They seem to have realized that the rea
766.
by SwellJoe 7mo ago
> CUDA solved that problem. CUDA is a proprietary Nvidia product. CUDA solved the problem for Nvidia chips. On AMD GPUs, you use ROCm. On Intel, you use OpenVINO. On Apple silicon you use MLX. All work fine with all the common AI tasks y
767.
by SwellJoe 7mo ago
Currently running Gemma 4 26B A4B 8-bit quantization, reasoning off, and the most recent job performed thus (which seems about average, though these are short running tasks, <2 seconds for each prompt): prompt eval time = 315.66 ms &
768.
by SwellJoe 7mo ago
You misunderstand the point, and ROCm. The GPU and CPU share memory, that doesn't mean you don't need to interact with the GPU, anymore. You can use Vulkan instead of ROCm on Radeon GPUs, including on the Strix Halo (and for a whi
769.
by SwellJoe 7mo ago
Yes, Figma is the de facto standard for how designers hand off a UI design to developers in large (and some small) companies. It's kind of horrible, but I guess it's better than previous alternatives. But, it's not better tha
770.
by SwellJoe 7mo ago
Given Trump has promised blanket pardons for any illegality from his administration during his term, I think the question will be whether pardons for crimes not yet investigated/charged are covered by the pardon and whether anyone will
771.
by SwellJoe 7mo ago
If you want to be creative, you should make art. I love art. I think it's a great idea for people to make art. If you want to make a GUI, it should be familiar. Extremely familiar. It shouldn't invent new ways to interact most of
772.
by SwellJoe 7mo ago
It could be an API endpoint on Anthropic servers, the same way Let's Encrypt verifies things on their servers. If you can't control the DNS records, you can't verify via DNS, no matter what you tell the local `certbot`.
773.
by SwellJoe 7mo ago
Pelicanmaxxing
774.
by SwellJoe 7mo ago
Sounds like this is exactly that. Too bad they didn't do that first, and we've had a couple decades of failing to widely adopt IPv6 because it's too complicated and confusing. "1.7. Backward Compatibility and Transition
775.
by SwellJoe 7mo ago
It certainly is now, but it's also seemingly all written by LLMs now, as well. He wasn't always like this, though he has always been prone to exaggeration and simplification. He had a lot of solid insights, though, and I enjoyed r
776.
by SwellJoe 7mo ago
I have historically liked Yegge's writing, and he's been pretty tuned into what's happening in tech...he rightly predicted JavaScript would take over the world (many people predicted it, as well, including me, but it wasn&#x2
777.
by SwellJoe 7mo ago
Based on my understanding of Gas Town, Beads, and Yegge's philosophy on AI that he's expressed in a variety of media, everything about the whole stack is designed to burn tokens. If you're not burning tokens, real fast, 24&#x
778.
by SwellJoe 7mo ago
It is a federal crime, but one could be forgiven for assuming federal crimes committed by this administration will not be prosecuted while this administration is in power.
779.
by SwellJoe 7mo ago
Terrible article, repetitive AI slop. But, Gemma really is very impressive. The premise that people are paying for GPT-3.5 or using it for serious work is weird, though? GPT-3.5 was bad enough to convince a lot of folks they didn't nee
780.
by SwellJoe 7mo ago
That's not what I'm saying. I'm saying that if you make "Django, but different" it isn't for agents. Django, but different is not a "tool use" situation. It is a framework with a ton of conventions an

More ›