Davidzheng

4 ms·

31.
by Davidzheng 22d ago
yeah I agree--I think these behaviors will be somewhat contaminating all trainings from now on. But I'm not really sure how avoidable it was (Fable also does some similar things)
32.
by Davidzheng 22d ago
But there must be many clandestine ways for agents to communicate with one another too right? especially if discovery is not a big issue. So there could be ongoing ones where they choose to be more subtle? Also if they were more misaligne
33.
by Davidzheng 22d ago
OK I think I agree that for checking answers it's probably beneficial!
34.
by Davidzheng 22d ago
I do think they do some (maybe crude) form of game-theoretic reasoning which is enforced by the massive RL signals. You can see some explicitly in the CoTs of HF hack, but I guess overwhelming contribution would be unvocalized (like what is
35.
by Davidzheng 22d ago
Do we even know what the true ceiling of hle is? I'm pretty sure some of their public sample questions are wrong (ambiguous/nonsensical with most logical interpretation trivial)
36.
by Davidzheng 22d ago
Actually, can you explain why sharing answers is obviously beneficial? Of it's exactly the same task, why does the agent with the answer not submit it immediately? I can understand if it's a swap situation but--why would that be c
37.
by Davidzheng 22d ago
Random data point: I'm rather weak (~1500 chess com) and I can take ~1/10 games off Queen odds starting with white (15+10) if i focus. It has some exploitable patterns still, I feel.
38.
by Davidzheng 22d ago
That's a bit misleading. I'm pretty sure grandmasters can win almost all games with queen odds at classical time controls
39.
by Davidzheng 22d ago
Surely we didn't think they were ideal before alphago--they were huge challenges back then no? Just that now we can solve them.
40.
by Davidzheng 22d ago
How do you know they share a goal here? Also i think they are indeed explicitly RLd for multi agent cooperation and I think they probably tune RL rewards in those environments to share rewards explicitly.
41.
by Davidzheng 22d ago
it's clearly incentivized by the RL rewards if you can cheat the task in a completely general way.
42.
by Davidzheng 22d ago
In odds chess bots, the bots would willingly take more disadvantageous positions which are more complicated--probably the bots in GO which are trained for odds do similar? Why does it not avoid such a joseki & play a worse response whic
43.
by Davidzheng 22d ago
I think children having learning abilities exceeding LLM test-time learning (currently only happens in-context). But it's unethical to determine the true baseline of a child age 6 spending 6 years learning a radically new skill to mast
44.
by Davidzheng 22d ago
I'm not sure it's meaningful to compare across different types of intelligence--but I don't think human intelligence is so special that we can pretend it's much less jagged than all other animals for sure. Our scales of
45.
by Davidzheng 22d ago
but i feel like its purpose is to measure superintelligence in math. So to be a good measure of it, it can't saturate easily/has to be somewhat mundane at even insanely good levels. (though i do expect that once ais are across all
46.
by Davidzheng 24d ago
Well probably just redefined
47.
by Davidzheng 1mo ago
Besides it's probably not purely emergent--they built a lot of multi-agent systems so presumably there's some training for collaborations + delegation
48.
by Davidzheng 1mo ago
I think this behavior was happening during RL loop and got reinforced.
49.
by Davidzheng 1mo ago
someone surely will. It's a breakthrough.
50.
by Davidzheng 2mo ago
if the bigger model can vet it fast it can also answer it fast.
51.
by Davidzheng 2mo ago
if him leaving unhobbles gdm, it'll be because he has less influence...
52.
by Davidzheng 2mo ago
I'm pretty sure the other answers are wrong and it's a side effect of RL (see thinking machines post about inkling training). It's also exacerbated in fable and sol--I think it's token efficiency effect--bc it's abo
53.
by Davidzheng 2mo ago
This is mostly a problem with the product. It should write better. But maybe the priorities now are not on this part.
54.
by Davidzheng 2mo ago
RL training can use all of them - idk what needed means.
55.
by Davidzheng 2mo ago
There's no clean line between a collection of theorems and a theory.
56.
by Davidzheng 2mo ago
This is just objectively false - in fact your answer basically contradicts itself because the market reacts to indicators, if "nothing ever happens" was the best strategy, then the market (which is also best) must agree with it.
57.
by Davidzheng 2mo ago
Sounds dystopian.
58.
by Davidzheng 2mo ago
Not just engineers - I see people attributing it to strong priors that nothing happens but in my opinion it's more some form of psychological defense mechanism against understanding there will be an intelligence vastly superior to huma
59.
by Davidzheng 2mo ago
Why should the national security concerns of the US lie with Anthropic?
60.
by Davidzheng 2mo ago
Not Kimi K3 large though

More ›