c7b

8 ms·

91.
by c7b 3mo ago
I'm not sure what you're trying to say. Is that a good or a bad thing? Model distillation is presumably part of the reason why Qwen is so good, yes. As a consumer, that's a good thing I would say. It's a natural counterb
92.
by c7b 3mo ago
Unified memory feels like the future of consumer hardware, agreed! Do check out r/StrixHalo
93.
by c7b 3mo ago
Cool! Anything you want to share? I haven't looked much into my system prompt yet, do you have any tips?
94.
by c7b 3mo ago
Web search, MTP (speeds up generation), uncensored models. Lots more things on my bucket list (eg various things related to image generation). Not gonna lie, if you're coming from ChatGPT/Claude Code, you'll mostly be adding
95.
by c7b 3mo ago
Personally, I would always max out the RAM you can fit into your budget. You might get lower bandwidth (= slower generation) than you do on a Mac if you choose a Strix Halo or DGX Spark, but there are always new tweaks being discovered to s
96.
by c7b 3mo ago
Gotta say, I've lost all interest in cloud-based AI products. Too many cool features and workflows that I was once excited about that I can't or don't use anymore for a variety of reasons (price hikes, subjectively nerfed, di
97.
by c7b 3mo ago
I'm not sure I understand the Weights policy. This site says the weights are Apache-licensed, suggesting it's open weights. But I can't find a download link. Their Huggingface profile seems to only provide an earlier snapshot
98.
by c7b 3mo ago
Desire to learn is deeply ingrained in humans and hard to root out. The education system does a pretty good job at it, unfortunately, but nothing ever is 100% effective.
99.
by c7b 3mo ago
Moral panic. We all have private tutors on every subject at our disposal now. It takes some special kind of mental gymnastics to conclude from this that no one will learn anything anymore henceforth. The article is just a long-winded way of
100.
by c7b 3mo ago
This. Do consider local LLMs, but set aside a dedicated machine for it. Connect via VPN or reverse proxy. If it's not a Mac them I'd also put a server distro on it. No need for a desktop environment, save your RAM.
101.
by c7b 3mo ago
You could fit a Q4 GLM5.2 in 512GB and still have some space for context (372-475GB for the model): https://unsloth.ai/docs/models/glm-5.2 But yeah, there's a bit of a dearth of models that could fully utiliz
102.
by c7b 3mo ago
My 2c: you don't need the Strix Halo desktop, the chip comes in many rigs, most of them cheaper, the performance difference isn't worth it. It used to be half the price of a DGX Spark or a Mac with 128GB RAM. If you can still find
103.
by c7b 3mo ago
The question is also what game they're playing. Deepseek came out of a hedge fund. I think it's no coincidence that their publications tend to have a large impact on AI stock prices. Destroying the growth story of overvalued stock
104.
by c7b 3mo ago
Almost anything can be made sense of from an evolutionary perspective. Often even the opposite of what's being observed. Can be a fun game to play. The corollary is it's not useful for vetting theories for plausibility.
105.
by c7b 4mo ago
Can someone explain the math to me? Why is 1-bit only ten percent less memory than 2-bit?
106.
by c7b 4mo ago
You can get a 128GB Strix Halo for under $3k. Used to be under $2k. Even if you believe it'll be completely obsolete for AI in two years, it'll still be good for many other things. Games for at least several more years, a great ho
107.
by c7b 4mo ago
What's your hardware stack?
108.
by c7b 4mo ago
Why? Afaik, apprentices are employees, with all the benefits that that entails.
109.
by c7b 4mo ago
Sounds like the opposite of what the OP and GP were advocating for.
110.
by c7b 4mo ago
I make an important distinction between cloud services and local AI. My lifetime spending on cloud AI is probably less than $500, and I don't intend to spend any more. But I've already dropped $2.5k on new hardware for local infer
111.
by c7b 4mo ago
Congrats! I like the code-based CAD paradigm, but one question about that: why did you choose OpenSCAD instead of more powerful alternatives like CadQuery?
112.
by c7b 4mo ago
If you don't mind trusting another company with forwarding your emails, it's definitely less hassle to set up an equivalent service for yourself.
113.
by c7b 4mo ago
And if you ask a bit more, you'll find that those same people are very likely to daily-drive AI models, desktop and phone operating systems and various other software critical to their professional and personal lives from US companies.
114.
by c7b 4mo ago
Probably a stupid question, but could this archive embedded videos as well?
115.
by c7b 4mo ago
Since it's not mentioned in the article, the distinction between open source and open weights is important. Open weights models are almost like a 'first shot is free' entry drug. Without at least the original training data yo
116.
by c7b 4mo ago
Could you put some numbers and examples behind the efficiency gap between data center and consumer-grade AI hardware? Did you include examples like the RTX Spark on the consumer side? I was always amazed at the low power consumption of unif
117.
by c7b 4mo ago
Wouldn't downloading conversations be more useful? If you've input something into Claude that you don't want a future trillion-dollar US company (a bicorn?) to know, this script won't help I'm afraid. Free reminder
118.
by c7b 4mo ago
Interesting that this doesn't seem to use blockchains. Arguably it would have been a good use case. OP, could you elaborate on the reasons for the choice (if it was a conscious one at all)?
119.
by c7b 4mo ago
I'm old enough to remember that capitalism used to always feel like this. Even worse. Predatory subscriptions, targeting minors in some cases (remember phone ringtones?), toll numbers, unclear fees, surprise bills (my favorite - roamin
120.
by c7b 4mo ago
1,5k. For two months of that spend you could buy a machine that can self-host decent models, plus a year's worth of electricity. It's not up there in terms of quality, but with a bit more effort it works pretty decently. I'm

More ›