segmondy

7 ms·

181.
by segmondy 4mo ago
Completely worth it. At 6tk a second. If I can get 2 hrs of token generation. That's 2hrs * 3600secs * 6tk = 43200 tokens, at about 10tk to a line of code, that's about 4320 lines. Let's even trim it more and slice it b
182.
by segmondy 4mo ago
You can run it today with that 12gb vram 3060, but I would suggest getting 2 3090s. Use cmoe option. This will keep the attention/route tensors on the GPU and offload the rest to system memory. Try it now and see the performance.
183.
by segmondy 4mo ago
P40 was release 2016 and still selling like hotcakes!
184.
by segmondy 4mo ago
I run Q4_K_XL. All it takes to run to get about 6tk/sec is 512gb of ram and 2 3090 GPUs with llama.cpp -cmoe. I also have crappy DDR4, 2400mhz, 3200mhz will bring that speed up to about 9tk/sec. I also have ok 32core epyc CPU
185.
by segmondy 4mo ago
One shot prompt means you give the model and input, you get an output done. This was not a one shot prompt, but an agentic task as shown by the tool calls.
186.
by segmondy 4mo ago
What are you yapping about? This was not one shot prompting, but a long run horizon task. But GLM and Opus invoked at least 120+ tools across the runs.
187.
by segmondy 4mo ago
Local LLMs have been useful since 2024. If you don't know this then you are just far behind. Catch up!
188.
by segmondy 4mo ago
What I find sad is how much Anthropic goes to hide your data, yet they are happy to slurp up all yours and most of you are happy to hand it over. ... then they turn around and compete with you by building your products that eat into your
189.
by segmondy 4mo ago
The author is correct, the model architecture is now much more complicated. You can see this if you use llama.cpp and follow the project. The earlier models were always fully implemented. Yet with more contributors, as of today tons of l
190.
by segmondy 4mo ago
Will they? I don't know. Should they? Yes. If AI is so great, we should use it to optimize. Rewrite all the bloat into lean efficient systems...
191.
by segmondy 4mo ago
Who is "government" Is it not the people? The people don't want this crap. It's a few zealots with their motives that are pushing this down society.
192.
by segmondy 4mo ago
We already have done so multiple times :-( We are living on borrowed credit/reputation from the past, but it's fast eroding.
193.
by segmondy 4mo ago
You can do that with smaller models at home. Gemma-4-E4B will run on a 12gb GPU, and supports audio, image, video input
194.
by segmondy 4mo ago
They are not playing pissing fest. They have revolutionary research on Vision if you read their white papers, they just take their time. Every major release from them has brought something really new to the field, V3, R1, OCR, V3.2, V4.
195.
by segmondy 4mo ago
there is no debacle, ". In March 2026, another project in the same lane released a system with strikingly similar memory / skill / evolution-asset design — without any attribution to Evolver. " there is nothing new abo
196.
by segmondy 4mo ago
I run 27B at Q8 with fp16 KV cache at 50tk/sec on 2 3090s. Not 4090, Not 5090. 6 years old GPUs.
197.
by segmondy 4mo ago
DeepSeekv4+ will have image capability, they said so in their paper. GLM whenever they decide to. Both companies have they tech and for whatever reason haven't decide to prioritize it. Both of their OCR are SOTA among all OCR models
198.
by segmondy 4mo ago
You are obviously lying because it shows you have no experience with. GLM since 4.5 have been crushing it. all their models since then haven't skipped a beat. 4.5/4.5-air, 4.6, 4.7, 4.8, 5, 5.1. That aside, MiMoV2.5, MiniMax
199.
by segmondy 4mo ago
Probably not. Qwen3.(5|6)-27B seems like an "accidental freak". I'm not even sure they know what they did to create that. A decent amount of the team members left after that, so unfortunately, we might not be seeing an
200.
by segmondy 4mo ago
Why wait for the next few months? There are plenty of better models that you can run today locally. Qwen3.5-397B beats Qwen3.6-27B. MiniMax2.7 is a longrun horizon monster. (I haven't given 3 much of a try yet). KimiK2.6/2
201.
by segmondy 4mo ago
The entire point of this post is that it's open weights, you can run it yourself and don't have to deal with the API issues. You really do have that choice.
202.
by segmondy 4mo ago
You would think that Mythos and Fable would have fixed it all ...
203.
by segmondy 4mo ago
hundreds of thousands
204.
by segmondy 4mo ago
It's more than good. As of today, it's great. Those models listed in the blog are horrible compared to what you can run today, There's absolutely no reason to run those, you have Qwen3.6, Gemma4, and plenty other sized compa
205.
by segmondy 4mo ago
In the last few days, Chinese labs have given us MiniMaxM3, KimiK2.7 and now GLM5.2. Meanwhile US is censoring models. Reads like fiction.
206.
by segmondy 4mo ago
We are not missing the big picture, this is what Anthropic wanted. They made this bed, let them lay comfortable in it.
207.
by segmondy 4mo ago
perhaps that's the grift to handle lack of compute, they just switch you to a lesser model and gaslight you into thinking you triggered a filter, but the reality is they don't have the compute for it.
208.
by segmondy 4mo ago
Only because the people don't fight back. If they know that folks would fight back, they would behave themselves in the most polite and proper ways you won't believe.
209.
by segmondy 4mo ago
lot's of idiots are going to lose their money
210.
by segmondy 4mo ago
Mythos, Fable, are they trolling us?

More ›