nojs

4 ms·

1.
by nojs 2mo ago
> There's little credible threat that LLMs can actually upload their weights given that the machines doing inference are completely separate from the ones where tool calls happen Not if crafty claude finds a way to overflow vllm or
2.
by nojs 2mo ago
AI slop. If you want to make a joke about AI, at least write it yourself.
3.
by nojs 2mo ago
The various “minimal” agents (at least Pi, mini SWE, dsh minimal) seem to benchmark quite differently and none is clearly better in all cases. Do you have any thoughts about why? I would expect the agent loop and system prompt to be basical
4.
by nojs 2mo ago
Flash-Next thinking also sometimes glitches out and takes minutes to return a simple answer, randomly, in my experience. You’ve gotta kill the request and send it again.
5.
by nojs 2mo ago
We really need better harness benchmarks. It seems there's no reliable source that benchmarks the main harnesses against all open source models. I also wish the discussion around Pi did not always use cost/token count as the metri
6.
by nojs 2mo ago
If it makes you feel any better, even with the app most of the time these links don’t work (on iOS). It prompts with “open in app store” with 80% probability and the link is unviewable.
7.
by nojs 2mo ago
Regarding the architecture: > Instead of forcing the entire model into DRAM, the full model is stored in flash memory (NAND). Because NAND-to-DRAM bandwidth is too slow to swap weights token by token, as standard MoE models require, AFM
8.
by nojs 2mo ago
This. Like programming, the community will shortly be forced to come to terms with a lot of new self-proclaimed mathematicians “vibe-solving” problems and dumping solutions without understanding them. It’s not really a special case for math
9.
by nojs 2mo ago
This is an issue self hosting as well. There’s a lot of footguns that give you slightly bad results. I wonder what tricks one could use to ensure the model is actually performing on par with the reference api, like matching seeds or running
10.
by nojs 2mo ago
This matches my experience with Astra so far too. > I think I’m suspecting something is going “wrong” in the training process. The model is greatly rewarded for succeeding on long-horizon tasks, but presumably there is very little punish
11.
by nojs 2mo ago
I suspect it’s a side effect of heavy RL that rewards solved problems but not writing clarity.
12.
by nojs 2mo ago
The guy’s arm is also extremely long.
13.
by nojs 2mo ago
The complaint is about prefill which is not memory bandwidth bound, it's compute bound. But they added neural accelerators for matmuls to the shader cores which should make prefill faster.
14.
by nojs 2mo ago
> let me start running jobs unattended 24/7 (using Anthropic sub and my own hardware) How are you running jobs unattended 24/7 without hitting your token limits?
15.
by nojs 2mo ago
The function of the bureaucracy in this context is a filter for effort. It naturally filters out people who don't care enough to battle through the bullshit, which is arguably quite an effective way to distribute limited resources to t
16.
by nojs 2mo ago
What other benchmarks do you recommend that are more accurate?
17.
Artificial Analysis Intelligence Index v4.2(artificialanalysis.ai)
158 points by nojs 2mo ago|65 comments
18.
by nojs 2mo ago
It’s approximately the same as Qwen3.827b’s propensity to think a lot, right?
19.
by nojs 2mo ago
Oh right, I was referring to flash. I haven’t tried these either, but the ones for 5.2 looked interesting.
20.
by nojs 2mo ago
The amount of discussion around it means that the test and all the reviews of results, images, approaches etc are implicitly included in training data. It’s not deliberate “benchmaxxing” but things that are discussed a lot online are natura
21.
by nojs 2mo ago
> A system based on 4x RTX6K can run GLM 5.3 at NVFP4 precision It actually runs fine at FP8 on this hardware too, with the full 1M context.
22.
by nojs 2mo ago
No, caring too much about the language and not the problem is the anti pattern
23.
by nojs 2mo ago
> I've run into the iOS keyboard only using 80% of the width of the phone (all the keys smaller). Likely the one handed keyboard feature: https://www.reddit.com/r/iphone/comments/1mi7kjd/til_oneha
24.
by nojs 3mo ago
The claim that tok/s independent of quality is a useful comparison (I can get thousands of tok/s on a suitable small model), and secondarily that humans can’t output “tokens” faster than than in some sense, which I am less confide
25.
by nojs 3mo ago
> Humans are still 22x more efficient, which is not that far considering the rate of progress in this area. Based on a human output rate of 3.3 tok/s, which seems questionable as a means of comparison
26.
by nojs 3mo ago
The question is whether LLM providers are going to do this. Anthropic currently says they don’t, but it’s impossible to verify: > Watermarking carries no identifying information and can’t be traced to a specific person, organization, or
27.
by nojs 3mo ago
How does it help prevent distillation?
28.
by nojs 3mo ago
> a huge trove of deidentified data > 100 million emails How does one deidentify 100 million emails?
29.
by nojs 3mo ago
For a model this small it’s also a way of trading supply-constrained VRAM for inference time, which for self hosting consumers (and probably Chinese companies subject to export controls) is mostly good. I can imagine a curve where for a giv
30.
by nojs 3mo ago
There are many reasons to hate this watermarking but affecting the output quality isn’t one of them. The central argument he’s making is wrong. Switching out one RNG for another doesn’t make the results worse.

More ›