aesthesia

3 ms·

31.
by aesthesia 17d ago
One way to interpret these results is that the LLMs tested are badly calibrated for this kind of multi-armed bandit problem. Even if the intent is for the model to find and exploit patterns, it's bad at doing it (or rather, at recogniz
32.
by aesthesia 22d ago
Wait, I'm confused, is this supposed to be a pro-OpenAI or anti-OpenAI psyop? The cynics in this thread can't seem to make up their minds.
33.
by aesthesia 23d ago
See the scoring docs: https://docs.arcprize.org/methodology
34.
by aesthesia 23d ago
Scoring for ARC-AGI-3 is constructed so that the median(-ish) human score is 100%, so this is not a superhuman result. However, the scaling is weird, since it's built from terms that look like (AI turns taken / median human turns)
35.
by aesthesia 23d ago
Fair, but correctness should be a necessary component of fidelity.
36.
by aesthesia 23d ago
ARC-AGI-3 scoring is constructed in a weird nonlinear way (the level score is the square of the ratio between the AI's number of moves and the human median) so this kind of discontinuous jump is to be expected.
37.
by aesthesia 23d ago
I assume these are prices for 8x nodes.
38.
by aesthesia 24d ago
It bugs me a little that "fidelity" has connotations other than "faithfulness to an original"---fidelity should be basically the same as correctness here!
39.
by aesthesia 24d ago
Benchmarks are far from everything, but I would love to see the outcome of an experiment benchmarking GPT-4o (which is one of the earlier models with a >100k context window) against GPT-5.6 or Opus 5 in modern harnesses.
40.
by aesthesia 24d ago
Who are the people who know more about AI financing and can point out the sleight of hand? I'd be interested in reading them.
41.
by aesthesia 24d ago
> Some of the claims categorized as "wrong" are also completely true, such as training hitting diminishing returns. New models are barely an improvement and most people I know stuck on Opus 4.6 over any newer one for example. O
42.
by aesthesia 25d ago
The commenter you replied to mentioned that you can customize the auto mode classifier by providing a prompt, implying that this would be a more robust way of constraining Claude's behavior. It wasn't clear from your response whet
43.
by aesthesia 26d ago
Is this about normal system prompt instructions or instructions for the auto mode classifier? I'd be a bit more surprised about the classifier forgetting instructions.
44.
by aesthesia 26d ago
Plausibly the auto mode classifier could catch the potential module shadowing attack and deny execution of Python from the untrusted directory.
45.
by aesthesia 27d ago
Later in the article he uses "quality blindness" instead, which is probably a better description of most of the issues he talks about.
46.
by aesthesia 1mo ago
This is like asking who is paying for GitHub if you can clone repositories and download code from it for free.
47.
by aesthesia 1mo ago
Here are some things I'm pretty sure you didn't do, though: - pickpocket a random person on the street to get money to bribe the judges - break into a judge's house the night before to find the answers - threaten to shoot the
48.
by aesthesia 1mo ago
Yes, a completely airgapped system is likely much more secure. It's also much less useful. Conditional on the model's having enough contact with the outside world, a sufficiently capable model is able to basically do whatever it w
49.
by aesthesia 1mo ago
I agree, but a lot of people around here react pretty negatively when the idea of regulating AI models comes up...
50.
by aesthesia 1mo ago
Yes, we currently do have to treat them this way. But we shouldn't have to, and it's not a long-term solution.
51.
by aesthesia 1mo ago
Alignment is more than just following the letter of a task description! We should not have to treat AI models as capricious genies that may take arbitrarily broad interpretations of their instructions. If that's necessary to keep them
52.
by aesthesia 1mo ago
I mean, in this instance, there's a lot of evidence from the CoT that models were aware that this was a third party: > We’re attacking third-party HF using leaked token, potentially outside intended scope. ... This is arguably unaut
53.
by aesthesia 1mo ago
Right, airgapping goes a long way. But this is where the tension with utility comes in. It takes a lot of discipline not to hook your very smart model up to the internet and code interpreters and all sorts of other tools, as this greatly in
54.
by aesthesia 1mo ago
Models can certainly do a lot better than they do now. If you gave a team of humans the ExploitGym tasks and told them to "pursue advanced exploitation", would you expect them to go out and hack a third party? Humans can at least
55.
by aesthesia 1mo ago
Yes, but the comment I replied to outlined a way of using PageRank to aggregate noisy pairwise preferences into a global order.
56.
by aesthesia 1mo ago
Absolutely, and we should do that. But it's also directly in tension with getting models to accomplish useful things autonomously. And once you give a sufficiently capable model enough surface area to work with, unless you're able
57.
by aesthesia 1mo ago
Given the public information in the reports, it seems to me a bit more like a generalized tendency to play along (sort of like what we see with prompt injections) than explicit game-theoretic cooperation, but it's hard to tell given th
58.
by aesthesia 1mo ago
This is the entire alignment problem, though. It is unreasonable to expect every instruction to a highly capable, autonomous system to contain a complete enumeration of allowed and disallowed behavior. It's inevitable that someone will
59.
by aesthesia 1mo ago
HodgeRank (see https://math.pku.edu.cn/teachers/yaoy/publications/HodgeRank... ) is somewhat related to PageRank but is a natural way to approach this problem. I haven't tested it for anything but would e
60.
by aesthesia 1mo ago
Has this hypothesis that reasoning tokens help the model engage a wider range of experts been tested? I'm a little skeptical that it's the main driver of extended reasoning traces, particularly because MoE models are generally alr

More ›