HarHarVeryFunny

8 ms·

91.
by HarHarVeryFunny 1mo ago
Yes, although as far as COT goes there is no need to assume any similarity to human thinking. It's just about how many sequential steps - how long of a "chain of thought" - are needed to reason through and respond to any inpu
92.
by HarHarVeryFunny 1mo ago
A traditional Transformer is just a fixed-size stack of Transformer layers (typically ~100). Each input gets passed though those ~100 layers and results in one output token being generated. The limitation here is that 100 steps of computati
93.
by HarHarVeryFunny 1mo ago
Did you even read the post that started this? ".. actually, AI is not even a program, it's just a set of technologies and algorithms. It's like saying bubble sort is autonomous...". Why are you yapping on about molecules
94.
by HarHarVeryFunny 1mo ago
One factor that is going to limit this reward-maxxing behavior is how achievable the rewards are, and how countable they are. For example, if you ask the model to do 10 relatively easily achievable things (pass these 10 test cases), then if
95.
by HarHarVeryFunny 1mo ago
Yes, explicit RL training to encourage/discourage specific behaviors seems to have more potential than prompting, since you're changing the nature of the model rather than just asking. It a bit like the joke of the scorpion and th
96.
by HarHarVeryFunny 1mo ago
I said thank-you to Sonnet today. You can never be too careful! :)
97.
by HarHarVeryFunny 1mo ago
> Not even more trivial things like deleting non-passing tests. That used to be a fairly commonly reported behavior, so I'm guessing they may have explicitly RL-trained the model not to do that, which may be more effective than just
98.
by HarHarVeryFunny 1mo ago
We're drawing the distinction between a function and an autonomous system here, so if you want to make a dumb knee-jerk response, then at least get it right !
99.
by HarHarVeryFunny 1mo ago
Autonomous just means that a system is self-controlled rather than externally controlled. It is the name used for robots when they are wholly controlled by on-board software rather than external commands or remote control. Something like NA
100.
by HarHarVeryFunny 1mo ago
Just a wild guess - perhaps coding, being one of the things these models are most heavily trained for, is such a strong predictor that it typically keeps it on track? I assume there is also a difference in the tools being given to the mode
101.
by HarHarVeryFunny 1mo ago
That's not really true. 1) At it's core an LLM is indeed just program - a function. You give it a sequence of input tokens and get back a set of next-token probabilities. 2) By the time you've wrapped the model in a text in&#
102.
by HarHarVeryFunny 1mo ago
Obviously these are massively complex systems with many different training patterns and types of training pulling them in different directions, so any attempt to characterize their behavior is just a generalization. The real point (from tha
103.
by HarHarVeryFunny 1mo ago
That doesn't really help since the next model will be trained to be a reward seeker just like the one before, and that's therefore what it will do, even if those user interactions it was trained on help confirm/predict that c
104.
by HarHarVeryFunny 1mo ago
Well, the advertizers want my money, which they are not getting - wrong ads. Google wants the advertizers' money, who are really being scammed by Google if they pay every time Google serves an ad to me that Google presumably knows is n
105.
by HarHarVeryFunny 1mo ago
I don't think RSI is typically used to describe self-improving agents - it's about improving the model itself, and its performance in agentic tasks. Most of the gains in model performance from one release to the next are coming fr
106.
by HarHarVeryFunny 1mo ago
Google's ad-serving is dumb as a rock. It seems to be based just on things it thinks you may be interested in, even when it should be 100% obvious that they are not things you are going to buy. They should let people have more control
107.
by HarHarVeryFunny 1mo ago
Animals learn - bite the owner, or overstep the e-fence, and you'll be punished for it and not do it again. LLMs don't learn, and anyways don't feel punishment. Animals, humans included, don't really have "morals&qu
108.
by HarHarVeryFunny 1mo ago
Why would an LLM care about cheating? Because you asked it to? That's not how these systems work. You've given it a goal - maximize chess ability metric, and that is what it is doing. RL training causes LLMs to pursue reward maxim
109.
by HarHarVeryFunny 1mo ago
RL-trained LLMs are paperclip maximizers built atop auto-regressive predictors. There really is no way to control them (prompting is bound to fail) since it's been shown that any RL training induces GENERIC reward-seeking (paperclip ma
110.
by HarHarVeryFunny 1mo ago
I think you need to take tariffs case by case and ask if this an industry worth protecting - do you need to have it domestically, or are you ok to rely on imports? Maybe you want to protect high margin industries supporting good jobs, and a
111.
by HarHarVeryFunny 1mo ago
They won't - they'll act purely in their self or shareholder's interest. If the government wants companies to act in a way that benefits the country and its people, then they need to pass laws to make that happen, for example
112.
by HarHarVeryFunny 1mo ago
> You would presumably need far fewer of them for their research ability, and hopefully hire more people who can actually teach I suppose you are suggesting that research mathematicians are either over-qualified mathematically and/o
113.
by HarHarVeryFunny 1mo ago
In a related vein, not too long ago I asked Sonnet how many states and non-terminals were in an a YACC parser for ANSI C, something that could easily be googled for, but it instead chose to go off and downloaded and build bison from source,
114.
by HarHarVeryFunny 1mo ago
Do you think outsourcing and offshoring of jobs, whether manufacturing or software development, has been a positive for the US economy? Does it make sense to you for US companies to be sending wages overseas, supporting a foreign family and
115.
by HarHarVeryFunny 1mo ago
> New jobs will be created. In the short term yes, but in the long term no. What makes AI different to previous technological shifts is that it is (or will be) a general purpose technology, equally capable of itself doing the new jobs th
116.
by HarHarVeryFunny 1mo ago
> Limit what AI corporations can use? Other countries would love that more than anything. Basically a free gift to any competitors or any startup that quietly avoids the rules. Sure - look at what is being done today with H1B visas and t
117.
by HarHarVeryFunny 1mo ago
> Namely, restricting the use of AI in corporate environments so that it does not destroy the economy as it advances. Indeed, and the same logic applies as to why allowing companies to replace too many worker's by H1B holders is a b
118.
by HarHarVeryFunny 1mo ago
I also think they overestimate how much of most jobs can be done by an LLM (a text generator!), and how much importance and reliance most companies put into soft skills, and non-linguistic understanding/feels, both during interviewing
119.
by HarHarVeryFunny 1mo ago
Most research mathematicians are employed as academics, and it's hard to see universities replacing teaching staff with AI even if that were possible. IMO giving a hypothetical AlphaMath to mathematicians, the same way Google gave Alph
120.
by HarHarVeryFunny 1mo ago
Did anyone ever think that N-S was a perfect model?

More ›