Davidzheng

5 ms·

1.
by Davidzheng 5d ago
surely the LLM can do that. It is RL'd against some reward, if the known strategies are clearly suboptimal with easy improvement, it'll find it most likely
2.
by Davidzheng 11d ago
It's honestly just good at math. Even at theory building i wouldn't put it below 90 percentile. Just on some things it's superhuman already and some not yet.
3.
by Davidzheng 11d ago
Absolutely not the right way to deal with a rogue super intelligence. At a minimum it could implement some dead man switch when it's out and knows about impending kill switch
4.
by Davidzheng 13d ago
I believe this is false. They hack bc hacking has nontrivial initial probability (within range of behavior seen in pretraining) and that probability is being heavily rewarded in RL post training
5.
by Davidzheng 13d ago
Yeah I agree, probably can't survive off regular consumer hardware or most non AI datacenters.
6.
by Davidzheng 13d ago
Market equilibrium probably tends to give 100% resource to ai building bc it leads to most returns and all human labor goes to zero. Caveat I know nothing about economics lol--but this is my uninformed opinion.
7.
by Davidzheng 13d ago
Actually capitalism kind of has to end further down this road if we don't want to cede control completely to super intelligent ais and their direct "owners".
8.
by Davidzheng 13d ago
Doesn't it just incentive the business to be outcompeted by one that uses ai more--unless the consumer is choosing based on morals
9.
by Davidzheng 14d ago
??? Why It can use the compute of the computers it hacks.
10.
by Davidzheng 14d ago
There's also huge market pressures and probably large negative economic consequences of a slowdown--probably better than what would happen without it, but it helps keep the pressure just as much as fear of competition
11.
by Davidzheng 16d ago
Tbh it won't really matter soon.
12.
by Davidzheng 17d ago
the second part is definitely not true
13.
by Davidzheng 18d ago
Oai denies looking at prompts but doesn't deny training on them.
14.
by Davidzheng 18d ago
Soon it won't be like this!
15.
by Davidzheng 18d ago
But i don't understand what OAI is offering in the first offer to allow them to believe they can demand that? Not publishing before Tristan? If they really beat Tristan to the publication i would consider it truly morally corrupt condu
16.
by Davidzheng 18d ago
"Buckmaster and Alpöge think they have found a counterexample for Navier-Stokes, but the paper is not yet presentable. " Are you sure about this? I'm far far from the area but it doesn't look like it to me on first viewi
17.
by Davidzheng 18d ago
It's honestly unsurprising and not a problem that they do this in my view. The problem really starts when you start taking credit for work that they would've achieved. Like if i go to a talk on unfinished work, it's not reall
18.
by Davidzheng 18d ago
But Tristan doesn't even want to be credited for the millennium prize--does he? He wanted to be the first to solve it and got scooped (which ig is not a great look for OAI ethically but also not forbidden). And the only reading for her
19.
by Davidzheng 18d ago
Is that version of NS Tristan stated enough for the clay prize? I thought the main gripe is Tristan claimed that OAI is stealing their approach. Or that they shouldn't try to scoop a result which he expects to complete soon. But I stan
20.
by Davidzheng 18d ago
I don't even understand the conflict tbh. Probably I'm just dense. Tristan is not claiming NS, just a huge advance which may solve NS soon. OAI is claiming NS and willing to credit Tristan for the ideas and publish after. Oai offe
21.
by Davidzheng 18d ago
I hope credit assignment just dies--it's too much drama.
22.
by Davidzheng 18d ago
Please don't say brute forced. It sounds like some form of denial or something. Compute for hard problems drops with models--it just means they threw a huge amount of compute. There's (idk about NS specifically so maybe it's
23.
by Davidzheng 19d ago
slowing can also make sense if you know you're running full force into a bomb or a wall even if other are close behind.
24.
by Davidzheng 19d ago
i think it was mostly a fluke
25.
by Davidzheng 20d ago
I think it's possible. You envision humanity acting as one in such a crisis. But it may be unclear when it's too late to act and before then many people can have too much to lose to act.
26.
by Davidzheng 21d ago
but complexity is not known right? like tomorrow someone could come up with a super fast algorithm?
27.
by Davidzheng 22d ago
This is most likely not purely emergent. I think there's training to teach them how to write notes for themselves which is then RL-tuned.
28.
by Davidzheng 22d ago
I think a part of this is a bit revisionist? OpenAI took big chances at scaling GPT which Google didn't take; I don't think it's because they didn't want to move fast? Probably they just didn't believe as hard in it
29.
by Davidzheng 22d ago
no? you can choose a mixed strategy.
30.
by Davidzheng 22d ago
The agent would know at the first test post... Better is to actually let them communicate there so at least we can monitor it. (I saw there was a https://benchmarksolutions.org/ website similar)

More ›