kouteiheika

5 ms·

31.
by kouteiheika 2mo ago
> Gated in that I cannot use non-US models. This is incredibly silly. First I would try to convince whoever has decreed this to reconsider. The only SOTA open-weight models are non-US, and rather than being afraid of potential Chinese ba
32.
by kouteiheika 2mo ago
> if you want to always stay on the bleeding edge (and not use their model-specific images they publish before it lands in a release), then some amount of patching might be needed especially for new models. Well, except this isn't e
33.
by kouteiheika 2mo ago
Prompt was completely unrelated and was nothing special. Just a batch job to translate a bunch of benign sentences from one language to another. I'm afraid credit here fully goes to vLLM, and whatever bug triggered this.
34.
by kouteiheika 2mo ago
> What's you opnion vs. SGLang in this regard? No opinion yet, but I'm planning to temporarily switch and see how it goes. I really hope the grass is greener there, but I'm keeping my expectations in check for now.
35.
by kouteiheika 2mo ago
I love vLLM, but damn if it isn't frustratingly buggy. I was recently running DeepSeek-V4-Flash on a B300. On v0.26 it was totally broken, and I had to add three out-of-tree patches to fix it. I updated to v0.27 -- no patches necessary
36.
by kouteiheika 2mo ago
> Engineers complained that their agent made unhelpful comments and produced low-quality output. Do you mean they found Claude's output, full of smoking-guns and honest caveats which are all load-bearing and genuinely bite -- they f
37.
by kouteiheika 2mo ago
Ah, okay. Sorry, fair enough!
38.
by kouteiheika 2mo ago
> but they omit the tool definitions. Those are much more interesting if you want to understand what Claude can actually do for you. You can reconstruct them through prompting Claude directly but that's extra friction and risks refu
39.
by kouteiheika 2mo ago
Right, so according to you it's because of the US government that they don't release it to the public? Have you missed their constant and incessant fear mongering? The causality chain here was not "US government says its da
40.
by kouteiheika 2mo ago
Have you tried to use Fable for anything even remotely security related, when the refusals kick in as soon as you even fart in the vague direction of anything security or biology-adjacent?
41.
by kouteiheika 2mo ago
> I don't understand all this spite about "rich friends" Okay, here's a challenge: I assume you're not a rich and powerful entity, so try to gain access to Mythos. I'll wait. > I mean what honestly are yo
42.
by kouteiheika 2mo ago
> This is absolutely still shy of Sol and Fable Not sure about Sol as I haven't used it, but, at least for security work -- does it matter? It's not like you will be allowed to use Fable (or access Mythos) for anything cybersec
43.
by kouteiheika 2mo ago
Well, um... I fail to see how this changes anything? Give me two random pieces of text, one written by Claude, one not. I'm pretty sure I will be able to tell with close to 100% accuracy which is which, as long as as there will be a no
44.
by kouteiheika 2mo ago
> On 2x RTX Pro 6000 Blackwell, its ~8k tok/s prefill and ~250 tok/s on a single stream. For reference, on a 1x B300 it's over 400 tok/s decode on a single stream.
45.
by kouteiheika 3mo ago
> This is a really disingenuous way to characterize the article (CO2 and global warming were much more of a focus than water, so your "or something" is doing a lot of work). Remind me please, how much exactly do data centers co
46.
by kouteiheika 3mo ago
Note that this describes an older, currently pretty much obsolete technique which does lobotomize the model somewhat.
47.
by kouteiheika 3mo ago
You can find the current state-of-art tool for censorship removal here: https://github.com/p-e-w/heretic
48.
by kouteiheika 3mo ago
> The most important pieces of information in this report are (a) the confirmation that the PRC models have no guardrails and will participate in offensive activity This isn't really important for open-weight models, because the gua
49.
by kouteiheika 3mo ago
> Noticed none of the comparisons mention Kimi K3. That's by design. Anthropic wants to make open-weight models illegal (not my speculation -- Dario explicitly said so), so I assume they don't want to give them any undue attent
50.
by kouteiheika 3mo ago
Assuming they did then they surely paid for them, which makes it "not stealing". Am I also "stealing proprietary U.S. technology" by harvesting my Claude chats from my `.claude` directory and training a bunch of models o
51.
by kouteiheika 3mo ago
It's an open-weight model so it literally doesn't matter whether it refuses by default or not, because it's pretty trivial to uncensor[1] any open-weight model and make it not refuse. [1]: https://github.com/p
52.
by kouteiheika 3mo ago
> So they are essentially a protection against spurious errors Only coincidentally. Sampling nonsense tokens will certainly degrade its performance and/or brick it, but it's also there to encourage diversity. For example, imagi
53.
by kouteiheika 3mo ago
Obligatory "The Conspiracy Against High Temperature Sampling": https://gist.github.com/Hellisotherpeople/71ba712f9f899adcb0...
54.
by kouteiheika 3mo ago
> And then there's the fact Fable is barely usable because it randomly downgrades to Opus out of nowhere whenever it thinks about exploits. I suspect it's not what you meant, but it's definitely not random and is very deli
55.
by kouteiheika 3mo ago
...until you get rug-pulled (like with Fable recently) because the model you're depending on is proprietary, then you have fun with no model at all. The sooner Chinese labs catch up the better it is for all of us, even if you don'
56.
by kouteiheika 3mo ago
> In other words, it's the most average, the most flat. [..] And when you think about it, the AI's training data comes from the broadest consumer distribution. It spits out that kind of data because it learned that most people
57.
by kouteiheika 3mo ago
> AI polished it [...] Compared to purely human writing [...] reads better than you'd expect I'd hard disagree here. Not when you're bombarded with this every single day. First, you start to be able to reliable recognize t
58.
by kouteiheika 3mo ago
> the mechanics that make zoning real: why #[repr(C)] is load-bearing, why the magic number is often 128 and not 64, and why adding prefetch hints can make things worse. > The honest rule Sigh... yet another Claude written article, an
59.
by kouteiheika 3mo ago
> HauHauCS has large collection but there are many others Before anyone recommends these models to other people I'd suggest they read this thread: https://old.reddit.com/r/LocalLLaMA/comments/1sw77p0&#
60.
by kouteiheika 3mo ago
> But if you assume he's just a villain You don't have to assume anything. A true "good guy" doesn't openly say that he's fine with autonomous, AI-powered weapons being used against me, and mass surveillance

More ›