andy99

11 ms·

121.
by andy99 3mo ago
The open weights models would be even more effective by incentivizing people and companies to switch jurisdictions. It might even break SFs monopoly on AI, or at least weaken it, everybody isn’t there exclusively to funnel money to openAI a
122.
by andy99 3mo ago
I’m in a bubble so I may not see the big picture, but I feel like these guys are completely destroying any kind of credibility they had. Anthropic especially has this arrogant, “we’re smarter than you”, communication style, and acts like th
123.
by andy99 3mo ago
Yeah this makes it unreadable for me, immediately distracted by how stupid it is and lose interest in the article. As someone else noted, it’s people who think they are writing for an “executive” audience that have never been within 100 fe
124.
by andy99 3mo ago
Rumored M5 ultra bandwidth is apparently 1100 GB/S (M3 Ultra is something like 880). This is 276 GB/S so they’re not really in the same league unfortunately. This will be considerably cheaper though.
125.
by andy99 3mo ago
> RAM prices weren't so f*cked I think we'd be seeing 256GB and even 512GB unified memory systems becoming quite common I agree but unfortunately that speaks to the depth of demand right now. Even as more capacity comes online
126.
by andy99 3mo ago
I use it exclusively as a server for LLM inference - first more for research but have been using it for coding now that there are sufficiently capable models that run on. For that I’m very happy.
127.
by andy99 3mo ago
Try poolside that came out yesterday https://news.ycombinator.com/item?id=49004937 or Qwen 3.5 122B A10B, both use more memory and still have experts sized for decent speed at the 395’s memory bandwidth at 4bit quantization
128.
by andy99 3mo ago
I have a framework desktop w/ 128GB that I bought last Christmas and if I’m looking at it right it costs $2000 (CAD) more now because of the RAM shortage (and in any event is apparently out of stock). Would love to have 192 GB but I’m
129.
by andy99 3mo ago
I don’t understand any of them. Normally even when I’m not really familiar with a tool, I have enough background knowledge to understand why it’s funny e.g. Scheme and Haskell jokes or something. I do use basic git regularly and all of thi
130.
by andy99 3mo ago
If an AI researcher was going to pelicanmaxx, they would almost certainly apply the augmentations mentioned in the article during training, e.g. randomly selecting animals and conveyances. You’d want a model that generalizes well, just sfti
131.
by andy99 3mo ago
This is more a statement of how awful Canadian banks are than anything else. For anyone unaware we have an oligopoly of five identical banks all of which treat their customers like shit and effectively extract tax from the Canadian populati
132.
by andy99 3mo ago
This AI written article seems to be substituting ethics for “taste” and making similar arguments to those from the past. Choosing what to do is more important than doing it is a taste problem, of which ethics is an aspect but one of many.
133.
by andy99 3mo ago
Real morality doesn’t pay, shallow “ethics” for business, tech, etc. has a whole industry that pays very all.
134.
by andy99 3mo ago
To go off topic a bit further, I recently went into Walgreens to buy some bottled water, and other than Evian, it was all advertised as alkaline. Is that just a trend, was it already alkaline and now that’s just in fashion? Personally I see
135.
by andy99 3mo ago
It’s the Strix halo (AMD) with 128 GB shared memory. The 4bit quant is ~75GB. Unfortunately I don’t know about the best way of running on an Nvidia gpu, you could try llama.cpp and offloading as many layers as possible into the gpu and usin
136.
by andy99 3mo ago
Replying to myself, seems this PR was merged into main and it the model does work with a Vulkan backend on my Framework desktop, I’m getting about 220 tok/s prompt processing and 21 tok/s output on the 4-bit quant. This is really
137.
by andy99 3mo ago
These articles are propaganda, it’s not an independent journalist writing it, they’re doing it at the behest of Anthropic or someone like them that’s pushing for regulatory capture.
138.
by andy99 3mo ago
This appears to be mostly due to fringe / activist views about copyright, rather than anything to do with quality or principle. If it was the latter I could get on board, as in instituting some standards against slop. But in reality it
139.
by andy99 3mo ago
Seems it’s not fully supported in mainline llama.cpp yet https://github.com/ggml-org/llama.cpp/pull/25165 In the huggingface link they mention building for CPU and for CUDA, does anyone know if that means it
140.
by andy99 3mo ago
This was posted earlier but didn’t get traction, and I made the following comment: Id want to know if “AI” makes a material difference vs just having access to the wrong answer. Like someone could be given search access that successfully re
141.
by andy99 3mo ago
Right, and there are two parallel tracks. First is the “every crack and crevice” part - “ summarize with AI”, “re write with AI”, “help me write”, “analyze with AI”, basically useless features being splattered everywhere in the name of inco
142.
by andy99 3mo ago
Id want to know if “AI” makes a material difference vs just having access to the wrong answer. Like someone could be given search access that successfully retrieved wrong answers to questions, would that give the same results. How much do u
143.
by andy99 3mo ago
Also, eating say a Big Mac and fries isn’t that unhealthy when done occasionally, there’s a lot of salt and fat but nothing horrendous. Compared to the load on your liver and pancreas etc of consuming literally about 1/2 pound of fruct
144.
by andy99 3mo ago
I get about 12 tok/s with 27B 8 bit, 50 with 35B A3B 8 bit, and 12 with 3.5 122B A10B 4 bit. The latter is about 80 GB iirc. it feels like the best balance between using as much memory as I can and still having a smaller expert model f
145.
by andy99 3mo ago
Qwen 3.5 to 3.6 was a big jump for the same size, e.g. 29 to 32 on artificial analysis intelligence for the 35BA3B models. Although I don’t think anyone has released a better model of that size since. I would love to see something like a 90
146.
by andy99 3mo ago
2022, and pretty clear why. Here is the discussion from the time: https://news.ycombinator.com/item?id=30977147
147.
by andy99 3mo ago
They did it interactively with Claude, it’s possible that it played up the significance and humor of the findings in a way that the interaction left the user feeling like they were really on to something.
148.
by andy99 3mo ago
People ask the same question of why YC funds yet another Uber for dogs or a button on the Touch Bar that cost $10/mo to help join a meeting faster. They invest in people more than ideas, so you’ve got, at least in many cases, people wi
149.
by andy99 3mo ago
Interesting, I didn’t know this existed, do you think it’s competitive with Qwen 3.6 35B A3B which seems to be the closest comparator? It’s 20 vs 32 in favor of Qwen on artificial analysis intelligence index (cohere isn’t benchmarked on the
150.
by andy99 3mo ago
I’ve done some of what I think this is, working on prem with customers, and I find it funny when I see jobs for FDEs that are somehow all in-office in San Francisco. The whole idea of being forward deployed I take to mean actually deployed.

More ›