mips_avatar

5 ms·

61.
by mips_avatar 3mo ago
I benchmarked mine for a deep research workload I was running. Concurrency 1 is the speed you'd get if you're chatting with one agent, 2x3090 (has an nvlink bridge though it didn't seem to matter hugely for inference) Qwen 3.
62.
by mips_avatar 3mo ago
You need the 128gb ram config to get the 614 GB/s bandwidth (which is $6999), you could skip out on upgrading the storage to save money but at that point I think most people upgrade the storage too at which point it's $8-10k + tax
63.
by mips_avatar 3mo ago
The cool thing about the 3090s is the RAM bandwidth. Token generation is mostly bottlenecked on memory bandwidth. Dual 3090s have 1.87 TB/s memory bandwidth (0.936 TB/s each), vs the M5 Macbook pro with only 0.3 TB/s (max ch
64.
by mips_avatar 3mo ago
Here's my build! https://jonready.com/blog/posts/local-llm-rig.html I love it because the watercooled 3090s are completely silent even under load. Facebook marketplace is definitely the move for a lot of the
65.
by mips_avatar 3mo ago
I think the sweet spot right now is 2x 3090s and a pcie 4 motherboard with 64-128 gb of ddr4 ram, you can build this right now for $3k and it runs qwen 27b/35b stupid fast at int4.
66.
by mips_avatar 4mo ago
I think it's going to be like DMCA, like hard to convict you for having the files but distributing them might be illegal
67.
by mips_avatar 4mo ago
The problem is they've convinced mainstream people that a model that can find a bug in Microsoft Windows is a bigger problem than Microsoft not caring about fixing it.
68.
by mips_avatar 4mo ago
I think it's more that they're abandoning simpler AI tasks to chinese models. Qwen 35b and deepseek flash are better than gp5 mini on my tasks and way cheaper.
69.
by mips_avatar 4mo ago
This is what OpenAI/Anthropic want, it's better marketing than they can pay for -- and it creates a precedent for permanently banning the next generation of open weights models
70.
by mips_avatar 4mo ago
OpenAI/Anthropic are begging to be restricted because it's great marketing and it creates a precedent to permanently ban open weights models. The problem is nobody in government believes in/cares about commodity pricing of tr
71.
by mips_avatar 4mo ago
My GPUs at home produce 60°C water. I just wish I had a good use for them.
72.
by mips_avatar 4mo ago
It wasn't a slight provocation that injured me, it was a car hitting me
73.
by mips_avatar 4mo ago
As someone who bike commutes my highest risk of dying this year is from biking. I've been hit twice biking in Seattle.
74.
by mips_avatar 4mo ago
Everyone I know who bike commutes also owns a car and pays taxes on it.
75.
by mips_avatar 4mo ago
I think we're about to be overwhelmed with good software that isn't great.
76.
by mips_avatar 4mo ago
Well Anthropic would love some regulatory capture.
77.
by mips_avatar 4mo ago
Makes sense, I keep hoping some startup will be able to crack the open licensing without limiting their business problem. Would love it if we could do a bit better than the rugpulling dynamics that are kind of common now without just givin
78.
by mips_avatar 4mo ago
This isn't about how to build an AI-native startup, it's how to use Anthropic tools to automate 2019 style app building. An AI native startup would have AI infused into the product, and Anthropic doesn't want anyone but them
79.
by mips_avatar 4mo ago
Cool that you licensed it with GPL, what was your thinking on the license?
80.
by mips_avatar 4mo ago
They’re not safety guardrails they’re anthropic doesn’t like anyone who isn’t anthropic working on AI rails
81.
by mips_avatar 4mo ago
Anthropic specifically said that those notifications are temporary and fable5 will only pretend to help you if it’s ml classifier gets tripped
82.
by mips_avatar 4mo ago
Anthropic is trying to hide bad behavior by being vague, it's important to not be vague when calling it out.
83.
by mips_avatar 4mo ago
From the model card: "the safeguards will limit effectiveness through methods such as prompt modification, steering vectors, or parameter-efficient fine-tuning" aka they will take your ML research code and inject bugs into it unti
84.
by mips_avatar 4mo ago
Yeah we can probably figure out how to run it on xiaomi gpus
85.
by mips_avatar 4mo ago
They've said that they'll stop notifying developers when this gets triggered, instead they'll load in basically like a LORA that's designed to inject bugs into your code.
86.
by mips_avatar 4mo ago
I guess it’s better they’re open about doing bad things. But now it’s a problem that they think this sets a precedent. They are one step away from feeling justified in using claude code running on a deepseek engineers laptop to hack deepsee
87.
by mips_avatar 4mo ago
Given that Anthropic has never released anything open weights I wouldn’t count on the fact that they view finetuning Gemma 4 as something allowable. I think they think nobody other than Anthropic should have AI
88.
by mips_avatar 4mo ago
Margin compression is terrifying
89.
by mips_avatar 4mo ago
Thats exactly it
90.
by mips_avatar 4mo ago
At least it gave an error! This whole silent nerfing idea is so wrong

More ›