wolttam

7 ms·

121.
by wolttam 4mo ago
If you have no need for Anthropic/OpenAI's frontier model capability, you may be better served with an open-weight model that can't be taken away. Edit: > GPT-5 does the job. I bring up DeepSeek V4 Flash a lot on HN, but
122.
by wolttam 4mo ago
I am one of those ecstatic folk :)
123.
by wolttam 4mo ago
Folks... read the actual tweet. They literally didn't vibe code it - they copy-pasted another project.
124.
by wolttam 4mo ago
This doesn't appear to be AI posturing, did you read the tweet? It is about one product blatantly, directly ripping off another.
125.
by wolttam 4mo ago
Baseline for “modern” apps, what? We’re talking about a terminal application here, there is definitely, most-assuredly ways to write something that does exactly what Claude Code does with a teeny fraction of the resource requirements. The
126.
by wolttam 4mo ago
They’re valid things to be concerned about IMO. I think you’re looking for an answer you’re not going to get unfortunately. I think there actually is a higher than average risk of data leakage with the insane optimizations that go into mo
127.
by wolttam 4mo ago
In other words: controlling for that kind of potential data-mixing is the same as in any other application where customer data is co-located within the same running process/memory/storage space.
128.
by wolttam 4mo ago
And chances are those 3-5 LLMs are more alike than they are different, because there is only one internet to pre-train on.
129.
by wolttam 4mo ago
I am 100% for fully agentic loops... for tasks other than engineering. I'm not willing to outsource the understanding how things work part of myself. That part of myself is what got me into computing in the first place. If this work
130.
by wolttam 4mo ago
> A better solution would be to root the damn TV and neuter its spyware/adware crap. That sounds like a lot of work. I don't want to sign up to this much work for every product I own that I want an iota of control over. So I wo
131.
by wolttam 4mo ago
I think the key to making "useful" things is to sandbox the agent and give it read/write access to strictly the data needed for the function. The agent can only talk to preordained services and its input to those services wil
132.
by wolttam 4mo ago
Would have run it with GLM on max/xhigh effort. Just for fun.
133.
by wolttam 4mo ago
https://developer.nvidia.com/blog/an-introduction-to-specula... You draft n tokens, and you verify them in a single forward pass. Here's the vLLM flag: --speculative-config '{{"method":"m
134.
by wolttam 4mo ago
> MTP in Inference. Our MTP strategy mainly aims to improve the performance of the main model, so during inference, we can directly discard the MTP modules and the main model can function independently and normally. *Additionally, we can
135.
by wolttam 4mo ago
Yep, those are the numbers I'm getting with DSv4 Flash on vLLM across 2 sparks.
136.
by wolttam 4mo ago
I suspect DwarfStar could probably squeeze more performance out of the single spark, maybe up closer to 20tok/s. Moving to 2 sparks meant switching to vLLM with 2-way tensor parallelism and working multi-token prediction. The paralleli
137.
by wolttam 4mo ago
I started with antirez' DwarfStar[1] on one spark and that (~11-14tok/s generation, ~300-400 tok/s prompt processing) was enough of a taste for me to jump into 2 sparks, running the native quant of DSv4 Flash. Now at 40-50tok
138.
by wolttam 4mo ago
> it is clear that actual intelligence has plateaued significantly. > Moving forward, the industry cannot continue to train bigger and bigger models since their intelligence not only plateaus but often will get worse These are wild cl
139.
by wolttam 4mo ago
Humans will become individually and independently less skilled while having access to tools that allow them to do far more than even the most skilled human could, before having access to these tools. I'm not sure if we'll become l
140.
by wolttam 4mo ago
I see those too and I think of it as the "thinking" in action. If you could replace their actual thinking trace with gibberish and get improved performance that scaled with the amount of gibberish you injected, that's what we
141.
by wolttam 4mo ago
The parent is off, you’re right. They may reason in any language, typically whatever the user’s language is, and you’ll see the reasoning directly with an open model like Deepseek. Research only showed that thinking might be disconnected
142.
by wolttam 4mo ago
If you know Seth you know he kind of talks like that comment reads, and has for quite a while
143.
by wolttam 4mo ago
I think it's highly likely that there will remain one or two companies on the very bleeding edge of AI development for the foreseeable future. But what I think a lot of people miss is that the market for the truly bleeding edge (develo
144.
by wolttam 4mo ago
One use I'd have for this is company wikis that you want to give folks easy offline access to (maybe the wiki has documentation that's useful at sites that don't have cellular coverage). Cool! It would be especially cool to
145.
by wolttam 4mo ago
This is typical for GLM releases.
146.
lmcli v0.7.0 – TUI harness with smooth performance up to 1M context(codeberg.org)
4 points by wolttam 4mo ago|1 comments
147.
by wolttam 4mo ago
lmcli v0.7.0 just released with support for sub-agents, custom tools, and smooth performance up to 1M context length. lmcli is a minimal, general-purpose LLM harness. It comes with some sample configuration for a coding agent I've foun
148.
by wolttam 4mo ago
To answer this and my sibling, it's DeepSeek V4 Flash at native FP4 quantization, on two Nvidia DGX Sparks. Which is a bit of kit but still paltry relative to the data centre. ~40 TPS generation, ~2000 TPS prompt processing, which make
149.
by wolttam 4mo ago
The point is not to be as good as the multi-trillion parameter model you can host in across 72 GPUs (or whatever). I'm running a 248B model on a paltry amount of hardware and getting plenty of good use out of it. Sure, the most deman
150.
by wolttam 4mo ago
Cute but I won first time on all 3 sizes

More ›