girvo

8 ms·

121.
by girvo 4mo ago
Yes, but that was because I could see the writing on the wall with respect to hardware prices being cooked by AI demand, so I built the best computer possible at the time knowing it'd probably need to last me the next 5+ years So not r
122.
by girvo 4mo ago
> Is it worth the claim that they are making in a press release? Definitely, yes, because being vague about it like they have been lets investors fill-in-the-blanks with whatever they want it to mean.
123.
by girvo 4mo ago
...I had no idea Blackmagic was australian! Thats wild, but maybe explains why their tech is so prevalent here haha. Well that and its all pretty excellent and great value (at least in the indie scene I was a part of)
124.
by girvo 4mo ago
> And postmen never stop to chat I can say that at least where I live (Brisbane Australia's inner-ring suburbs) that's still the case: my postie is super friendly, loves my dog, and always has a couple of minutes to say hi! And
125.
by girvo 4mo ago
Not very well, if you're using the latest version of Codex. They broke a lot of stuff that made that possible.
126.
by girvo 4mo ago
I’ve had a surprising amount of success by emailing one of the authors of various papers and asking those exact kind of questions (though more specific: I need to show that I have put effort in!)
127.
by girvo 4mo ago
GPUs? Though that’s kind of cheating considering it’s basically a monopoly at this point
128.
by girvo 4mo ago
Do we have data on that either way? I’m genuinely asking, not snarking: I tried to look it up a while ago but couldn’t find as much as I hoped
129.
by girvo 4mo ago
They are one of the few tech segments that is still doing so, and that’s mainly down to “smart” TVs being able to get revenue in other ways.
130.
by girvo 4mo ago
Now play games on it and show me the benchmarks. I mean that for real: I’ve been impressed by the performance of the M4 Mini I own, but a gaming machine it is not
131.
by girvo 4mo ago
> was it all for naught? I accepted a long time ago that it is all for naught :) Enjoy our time on this earth, do what we can, focus on people and it'll be alright
132.
by girvo 4mo ago
> The only thing moving to digital real id does is prevent criminal activity. What utter nonsense.
133.
by girvo 4mo ago
Though because they don't show you, they could be lying about it. Very unlikely, I think, would be too dangerous IMO. But technically possible
134.
by girvo 4mo ago
Amusing :) but I specifically mean in a SWE context. It’s used constantly by LLMs, Claude especially. Genuinely so curious to see certain phrases come up so often, beyond what I would’ve expected for how often they’re used by people
135.
by girvo 4mo ago
I’m not sure if it is written by an LLM, but anything being called “load-bearing” (formatted that way and all) sets off my alarm bells
136.
by girvo 4mo ago
> And when corporate needs to lay off 30% they’ll think nothing of it. Having seen this literally play out this year: it was all of the C’s that were first to be cut shrug
137.
by girvo 4mo ago
It’s a lot harder to get right than it sounds. I’ve been trying to as a Pi extension, but models are biased to think they’re better than they actually are. So far the best results I’ve got have been using a much smaller local model as a sim
138.
by girvo 4mo ago
Still helps, and Step 3.5/3.7 were specifically trained for MTP (in a weird triple layer/triple head fashion with a kind of unique architecture) With the currently-in-PR implementation it doubles decode performance for all the tas
139.
by girvo 4mo ago
The other benefit is that speculative decoding literally trades compute to make up for low bandwidth, so MTP/EAGLE/DFlash are unreasonably effective on the GB10 IMO, as long as your use case fits it. I’m getting 40tk/s decode
140.
by girvo 4mo ago
Because they're fun to play with :) But also because it's likely to hold most of its value into the next few years, based on the looks of things, too
141.
by girvo 4mo ago
27B seems surprisingly resiliant to quantisation. Though my evals showed there was some impact to coding ability from 8 bit to 4 bit, it was less than I would've expected: and it was on task types that you've said above that you d
142.
by girvo 4mo ago
> Currently, I have a very lightweight harness - the pi agent with everything stripped (`pi -nc --offline`) and a short system prompt [1] to align it a bit with my style This really is the secret to getting the most out of these models I
143.
by girvo 4mo ago
I love my Spark-alike, but they really aren't inference boxes IMO. They're experimentation boxes. A couple of 3080 20GB's for cheap from China, a 5090, an RTX Pro 6000 if you can swing the horrible cost: those are better choi
144.
by girvo 4mo ago
And rising. It's depressing.
145.
by girvo 4mo ago
> DGX Spark-alike is really just asking for trouble. Prefill kills perf. You're right that prefill kills perf, but shrug the GB10 has far more compute than it has memory bandwidth, so prefill isn't it's bottleneck.
146.
by girvo 4mo ago
To be fair: if you're happy with ds4 then IMO stick with it! Step 3.7 is notably better than 3.5 1. Use the official StepFun GGUF, IQ4_XS - theirs is better tuned in my experience than the other quants 2. Temp 1.0 top_p 0.95 sampling p
147.
by girvo 4mo ago
>What's your evaluation setup like? It sounds like maybe the best thing to do is have a realistic evaluation that resembles your actual intended workload and workflow, and then just try everything. That is quite literally what I hav
148.
by girvo 4mo ago
[flagged]
149.
by girvo 4mo ago
It isn’t though, I’ve run both through a bunch of coding evals. You nearly certainly didn’t have the right sampling parameters or quantised the KV cache? Ds4 is impressive for what it is, but it loops and over thinks even more, burning mass
150.
by girvo 4mo ago
You could even argue that it’s the load-bearing point!

More ›