4 ms·
My eye glazed over a bit during the opening paragraphs, but once you get to the meat of the article about how OpenAI's own researchers are using their tools it
by simonw 1mo ago
My eye glazed over a bit during the opening paragraphs, but once you get to the meat of the article about how OpenAI's own researchers are using their tools it gets a lot more interesting.
I noted that they use the acronym RSI (for Recursive Self-Improvement) without defining it. I think that's a little out of touch - I don't think RSI is a well-known acronym outside of OpenAI's bubble yet.
- vatsachak 1mo agoRSI started when humans discovered tool use. I mean one could argue that RSI always begins in any physical environment. The book "What is intelligence?" by Blaise Aguera is great
- lokar 1mo agoAre you sure that was not iterative improvement?
- password54321 1mo agoUsing tools to build tools is recursive.
- HarHarVeryFunny 1mo agoIt's not recursive when it's done iteratively, or are you imagining GPT Astra designing GPT Galactia, which starts designing GPT Oh-My-God-ica before it has finished being created itself?
- 0x63_Problems 1mo agoI think it's only recursive from the perspective of the humans, i.e. they design Astra, which itself as part of its deployment designs Galactica, etc. So humans develop things one after the other, but when the thing itself starts developing new things, those are happening 'recursively' in its scope.
- itishappy 1mo agoThat sounds more iterative than recursive. Recursion requires feeding the output back into the input, so creating version 4 requires results from version 3. You cannot recur in parallel. Iteration does not. You can iterate in parallel.
- HarHarVeryFunny 1mo agoYou can search in parallel, but a depth N search can only become a depth N+1 search after the depth N is done (i.e. sequentially). In any case the name RSI has stuck - the idea doesn't change or make any more sense by giving it a different name.
- itishappy 1mo agoBecause "depth" is recursive. You can search twice without waiting for the results of your first search: iteration. You can't if the thing you need to search for is the results of your first search: recursion.
- HarHarVeryFunny 1mo agoHere's the concept. Version 1 -> Version 2 -> Version 3 -> ... You can call it krispy kreme donuts if you want to.
- josh-sematic 1mo agoThe “recursive” part comes from the fact that you have an AI which was developed by an AI (that was developed by an AI (that was developed by an AI (…)))
- HarHarVeryFunny 1mo agoSounds like "recursively" walking to the grocery store by putting one foot in front of the other (that put itself in front of the other (that put itself in front of the other (...)))
- adastra22 1mo agoWhat is the difference between?
- topaz0 1mo agoIteration and recursion are famously equivalent
- dgacmu 1mo agoIndeed, many programmers might pattern match to repetitive stress injury and think of their brushes with carpal tunnel syndrome. :)
- andrewingram 1mo agoYeah, I kept looking for the first place it was defined in the article and... nothing
- iamflimflam1 1mo agoThey must have picked that habit up from Claude...
- rossant 1mo agoSame. Defining acronyms should become a habit when writing.
- Schlagbohrer 1mo agoRe-become a habit. It has long been standard good writing to always define an acronym on first use.
- HarHarVeryFunny 1mo agoRSI is a fetishistic term among the singularity crowd, who imagine AI "recursively" improving itself in some exponential fashion until there is a bright flash of white light and it reveals itself in the form of god. Or something like that. I don't know why whoever coined the term chose "recursive" rather than "iterative" - just sounds more likely to lead to infinite regress I suppose. This notion of recursive/iterative self-improvement, whereby generation #1 AI improves itself to create generation #2, then generation #2 further improves itself to create generation #3, etc, seems to conflict with the reality that what we have with LLMs is models whose performance/capability is defined by data, not code, so the most you can do is have your LLM design synthetic data, or just do Karpathy-style "auto research" where all you are doing is using the LLM to automate your experiments. At the end of the day, each experiment, designed by a person and/or LLM, then needs to compete with all your other ideas for compute to be tested at scale, and no amount of recursion or self-improvement will materialize an infinite amount of compute out of thin air, so your recursively synthetic-data gobbling LLM will continue to improve at the same pace it ever did.
- jazzyjackson 1mo agoYes the exponential self improvement folks have never heard of an eigenvalue I guess. You can loop forever using output as input but at some point the result will stop changing (depending on the function)
- fuzzfactor 1mo ago>AI "recursively" improving itself in some exponential fashion until there is a bright flash of white light Sounds like repetitive stress to me. >loop forever using output as input but at some point the result will stop changing Running in place will eventually wear you out too. Plus with some things it can be difficult to know for sure if that's where you are at the time. Even worse may be if you were almost running in place, it could be orders of magnitude more difficult to discern, especially if the scale was massive to an unprecedented degree.
- ajkjk 1mo agothat's not really how eigenvalues work... they specifically also model the case where the result keeps changing exponentially.
- sho_hn 1mo agoI actually think a goal of the current crop of OpenAI posts is expressely to reset the spectrum by normalizing the concept of RSI as something normal and safe to pursue. The message is running through all of them. It's a mix of marketing and pacifying the intelligentia. It's timed this way because the term is not yet well known outside the safety debate circles, so they get to frame it now. Instead of something to fear, it will be accepted as the next step. In approximately two days the groupie crowd will write LinkedIn posts about how Sam is winning because they have the better RSI, and this will become the new standard wisdom. In a month an AI expert will try to sell you a webinar on how to enable "RSI" in your org and your inbox will ask you if your team is doing the "RSI" yet.
- dgellow 1mo agoYep, it’s exactly this
- NitpickLawyer 1mo ago> It's timed this way because the term is not yet well known The basic concept has been here since llama3, in the open models. Likely earlier in closed labs. You use the previous gen models to curate and prepare data for the next gen. Now with the added benefit of actual arch/algo improvements (also public since gemini 2.5 gaining 1% efficiency on training next gen). This has been known for at least 2 years, in the open.
- sho_hn 1mo ago
- deleted 1mo ago[deleted]