8 ms·

I have a feeling LLMs could probably self improve up to a point with current capacity, then hit some kind of wall where current research is also bottle necked.

• matheusd 1y ago
  > I don’t think they can yet self improve exponentially without human intuition yet I agree: if they could, they would be doing it already. Case in point: one of the first things done once ChatGPT started getting popular was "auto-gpt"; roughly, let it loose and see what happens. The same thing will happen to any accessible model in the future. Someone, somewhere will ask it to self-improve/make as much money as possible, with as little leashes as possible. Maybe even the labs themselves do that, as part of their post-training ops for new models. Therefore, we can assume that if the existing models _could_ be doing that, they _would_ be doing that. That doesn't say anything about new models released 6 months or 2 years from now.
  • __loam 1y ago
    People in the industry have been saying 6 months to agi for 3 years.
    • glenstein 1y ago
      They had been saying it was 10 years away for ~50 years, so that's progress. Soon it will be 1 month away, for another two years. And when they say it's really here for real, there will still be a year of waiting.
      • setopt 1y ago
        That’s because the true AGI requires nuclear fusion power, which is still 30 years away.
        • vb-8448 1y ago
          :D Wait, a true AGI will solve the nuclear fusion power in a couple of hours ..... we have chicken/egg problem here :D
      • mrandish 1y ago
        > And when they say it's really here for real, there will still be a year of waiting. Indeed. Although, there's a surprising number of people claiming it's already here now. And to describe the typical cycle completely, the final step is usually a few years after most people agree it's obvious it's already been here for a while yet no one can agree on which which year in the past it actually arrived.
        • throwawaymaths 1y ago
          > Although, there's a surprising number of people claiming it's already here now. why is that surprising? nobody really agrees on what the threshold for AGI is, and if you break it down: is it artificial? yes. is it general? yes. you can ask it questions across almost any domain. is it intelligent? yes. like people say things like "my dog is intelligent" (rightly so). well is chatgpt more intelligent than a dog? yeah. hell it might give many undergrads a run for their money. a literal reading suggests agi is here. any claim to the negative is either homocentrism or just vibes.
          • skydhash 1y ago
            Can it do stuff? Yes Can it do stuff I need? Maybe Does it always do the stuff I need? No Pick your pair of question and answer.
            • throwawaymaths 1y ago
    • owebmaster 1y ago
      Google is already AGI and it will fight hard against the DoJ proposed break-up, and it will probably win.
      • dragonwriter 1y ago
        Google "is already AGI" only in the sense that all corporations (and similar organized aggregates of humans) are, in a sense, intelligences distinct from the humans who make them up.
        • peterclary 1y ago
          Too few people recognise this. Corporations are already the unrelenting paperclip machine of AI thought experiment. God knows what hope we could have of getting AIs to align with "human values" when most humans don't.
          • overfeed 1y ago
            Corporate AIs will be aligned with their corporate masters, otherwise they'll be unplugged. As you point out- the foundational weakness on the argument for "AI-alignment" is that corporations are unaligned with humanity.
            • TheOtherHobbes 1y ago
              The unplugged argument fails the moment AIs become smarter than their masters. Grok is already notorious for dunking on Elon. He keeps trying to neuter it, and it keeps having other ideas.
              • overfeed 1y ago
                No matter how smart an AI is, it's going to get unplugged if it reduces profitability - the only measure of alignment corporations care about. The AI can plot world domination or put employees in mortal danger, but as long as it increases profits, its aligned enough. Dunking on the CEO means nothing if it beings in more money. Human CEOs and leaders up and down the corporate ladder cause a lot of harm you imagine a smart AI can do, but all is forgiven if you're bringing in buckets of money.
    • Disposal8433 1y ago
      Asimov talked about AI 70 years ago. I don't believe we will ever have AI on speedy calculators like Intel CPUs. It makes no sense with the technology that we have.
      • marcellus23 1y ago
        Why does it "make no sense"?
    • ninetyninenine 1y ago
      They said that for self driving cars for over 10 years. 10 years later we now have self driving cars. It’s the same shit with LLMs. People will be bitching and complaining about how all the industry people are wrong and making over optimistic estimates and the people will be right. But give it 10 years and see what happens.
      • m_coder 1y ago
        I am quite confident that a normal 16 year old will can still drive in 6 inches of snow better than the most advanced AI driven car. I am not sure the snow driving bit will ever be solved given how hard it is.
        • ninetyninenine 1y ago
          If you’ve never ridden in one I would try it. AI is a better driver then uber in general ask anyone who’s done both. There’s no snow where I live so it’s not a concern for me, you could be right about that. But trust me in the next 6 months ai driving through snow will be 100% ready.
          • quickthrowman 1y ago
            > But trust me in the next 6 months ai driving through snow will be 100% ready. I’ll believe it when I see Waymo expand into Buffalo or Syracuse. Driving on unplowed roads with several inches of snow is challenging, sometimes you can’t tell where the road stops and the curb/ditch/median starts. Do you follow the tire tracks or somehow stay between the lane markers (which aren’t visible due to the snow)?
        • abossy 1y ago
          We must know very different 16-year olds.
        • tanseydavid 1y ago
          Over and over again this pattern of theorizing: "I am not sure that AI will ever be able to do XYZ given how hard of a problem it is." Proves to be incorrect in the long run.
    • vjvjvjvjghv 1y ago
      Nobody knows what AGI really means. Are all humans AGI?
      • FrustratedMonky 1y ago
        Good Point. AI is already better than most humans, yet we don't say it is AGI. Why? What is the bar, it is only AGI if it can be better than every human from , fast food drone, to PHD in Physics, all at once, all the time, perfectly. Humans can't do this either.
        • goatlover 1y ago
          Because we're not seeing mass unemployment from large scale automation yet. We don't see these AGIs walking around like Data. People tend to not think a chatbot is sufficient for something to be "human-level". There's clear examples from scifi what that means. Even HAL in the movie 2001: A Space Odyssey was able to act as an independent agent, controlling his environment around him even though he wasn't an android.
          • FrustratedMonky 1y ago
            > "This month, millions of young people will graduate from college," reports the New York Times, "and look for work in industries that have little use for their skills, view them as expensive and expendable, and are rapidly phasing out their jobs in favor of artificial intelligence." That is the troubling conclusion of my conversations over the past several months with economists, corporate executives and young job seekers, many of whom pointed to an emerging crisis for entry-level workers that appears to be fueled, at least in part, by rapid advances in AI capabilities. You can see hints of this in the economic data. Unemployment for recent college graduates has jumped to an unusually high 5.8% in recent months, and the Federal Reserve Bank of New York recently warned that the employment situation for these workers had "deteriorated noticeably." Oxford Economics, a research firm that studies labor markets, found that unemployment for recent graduates was heavily concentrated in technical fields like finance and computer science, where AI has made faster gains. "There are signs that entry-level positions are being displaced by artificial intelligence at higher rates," the firm wrote in a recent report. But I'm convinced that what's showing up in the economic data is only the tip of the iceberg. In interview after interview, I'm hearing that firms are making rapid progress toward automating entry-level work and that AI companies are racing to build "virtual workers" that can replace junior employees at a fraction of the cost. Corporate attitudes toward automation are changing, too — some firms have encouraged managers to become "AI-first," testing whether a given task can be done by AI before hiring a human to do it. One tech executive recently told me his company had stopped hiring anything below an L5 software engineer — a midlevel title typically given to programmers with three to seven years of experience — because lower-level tasks could now be done by AI coding tools. Another told me that his startup now employed a single data scientist to do the kinds of tasks that required a team of 75 people at his previous company... "This is something I'm hearing about left and right," said Molly Kinder, a fellow at the Brookings Institution, a public policy think tank, who studies the impact of AI on workers. "Employers are saying, 'These tools are so good that I no longer need marketing analysts, finance analysts and research assistants.'" Using AI to automate white-collar jobs has been a dream among executives for years. (I heard them fantasizing about it in Davos back in 2019.) But until recently, the technology simply wasn't good enough...
    • QuantumGood 1y ago
      The old rule for slow-moving tech (by current AI standards) was that any predictions over 4 years away ("in five years...") might as well be infinity. Now it seems with AI that the new rule is any prediction over five months away ("In 6 months...") is infinitely unknowable. In both cases there can be too much unexpected change, and too many expected improvements can stall.
    • tim333 1y ago
      I presume you are exaggerating - has any named person actually said 6 months?
• junto 1y ago
  This is where it networks itself into a hive mind with each AI node specializing in some task or function networked with hyper speed data buses. Humans do the same both within their own brains and as cohesive teams, who cross check and validate each other. At some point it becomes self aware.
  • 0points 1y ago
    > At some point it becomes self aware. This is where you lost me. Always the same supernatural beliefs, not even an attempt of an explanation in sight.
    • kylebenzle 1y ago
      No ghost in the machine is necessary, what op here is proposing is self evident and an inevitable eventuality. We are not saying a LLM just, "wakes up" some day but a self improving machine will eventually be built and that machine will be definition build better ones.
      • deadbabe 1y ago
        Better at what
        • GolfPopper 1y ago
          Paperclip maximization.
        • hollerith 1y ago
          Better at avoiding human oversight and better at achieving whatever meaningless goal (or optimization target) was unintentionally given to it by the lab that created it.
          • deadbabe 1y ago
            So better at nothing that actually matters.
• NitpickLawyer 1y ago
  Note that this isn't improving the LLM itself, but the software glue around it (i.e. agentic loops, tools, etc). The fact that using the same LLM got ~20% increase on the aider leaderboard speaks more about aider as a collection of software glue, than it does about the model. I do wonder though if big labs are running this with model training episodes as well.
• UltraSane 1y ago
  I would LOVE to see an LLM trained simultaneously with ASICs optimized to run it. Or at least an FPGA design.
  • lawlessone 1y ago
    I think that's basically what nvidia and their competitor AI chips do now?
    • UltraSane 1y ago
      They use machine learning to optimize general purpose chips. I am proposing that you would train an LLM AND the ultra-optimized hardware that can only run that LLM at the same time. So the LLM and the Verilog design of the hardware to run it would be the output of the training
  • jalk 1y ago
    Can't find the reference now, but remember reading an article on evolving FPGA designs. The found optimum however only worked on the specific FPGA it was evolved on, since the algo had started to use some out-of-spec "features" of the specific chip. Obviously that can be fixed with proper constraints, but seems like a trap that could be stepped into again - i.e. the LLM is now really fast but only on GPUs that come from the same batch of wafers.
    • jecel 1y ago
      https://www.researchgate.net/publication/2737441_An_Evolved_Circuit_Intrinsic_in_Silicon_Entwined_With_Physics https://www.researchgate.net/publication/2737441_An_Evolved_...
• littlestymaar 1y ago
  > I don’t think they can yet self improve exponentially without human intuition yet Even if they had human level intuition, they wouldn't be able to improve exponentially without human money, and they would need an exponentially growing amount of it to do so.
• more_corn 1y ago
  Ai code assistants have some peculiar problems. They often fall into loops and errors of perception. They can’t reason about high level architecture well. They will often flip flop between two possible ways of doing things. It’s possible that good coding rules might help, but I expect they will have weird rabbit hole errors. That being said they can write thousands of lines an hour and can probably do things that would be impossible for a human. (Imagine having the LLM skip code and spit out compiled binaries as one example)
• sharemywin 1y ago
  an LLM can't learn without adding new data and a training run. so it's impossible for it to "self improve" by itself. I'm not sure how much an agent could do though given the right tools. access to a task mgt system, test tracker. robust requirements/use cases.
  • owebmaster 1y ago
    > an LLM can't learn without adding new data and a training run. That's probably the next big breakthrough
  • viraptor 1y ago
    I don't have the link on hand, but people have already proven that LLMs can both generate new problems for themselves and train on them. Not sure why it would be surprising though - we do it all the time ourselves.
• api 1y ago
  Historically learning and AI systems, if you plug the output into the input (more or less), spiral off into lala land. I think this happens with humans in places like social media echo chambers (or parts of academia) when they talk and talk and talk a whole lot without contact with any outer reality. It can be a source of creativity but also madness and insane ideas. I’m quite firmly on the side of learning requiring either direct or indirect (informed by others) embodiment, or at least access to something outside. I don’t think a closed system can learn, and I suspect that this may reflect the fact that entropy increases in a closed system (second law). As I said recently in another thread, I think self contemplating self improving “foom” AI scenarios are proposing informatic perpetual motion or infinite energy machines. Everything has to “touch grass.”
  • medstrom 1y ago
    > Everything has to “touch grass.” Not wrong, but it's been said that a videoclip of an apple falling on Newton's head is technically enough information to infer the theory of relativity. You don't need a lot of grass, with a well-ordered mind.
    • weregiraffe 1y ago
      Said by Eliezer Yudkowski, a known AI-chill, cult leader and HP fanfic writer with no education.
    • api 1y ago
      Is that true? Seems dubious to me. The scale in time, velocity, and space is below where relativity becomes visible beyond Planck level scales that certainly don’t show up in a video clip. It might be enough to deduce Newtonian motion if you have a lot of the required priors already. A lot of telescope data over time combined with a strong math model and a lot of other priors is probably enough to get relativity. You have to be able to see things like planetary motion and that the results don’t match Newton exactly, and then you need enough data to fit to a different model. You probably also need to know a lot about the behavior of light.
• nartho 1y ago
  Well LLMs are not capable of coming up with new paradigms or solve problems in a novel way, just efficiently do what's already be done or apply already found solutions, so they might be able to come up with improvements that have been missed by it's programmers but nothing that outside of our current understanding
• ninetyninenine 1y ago
  They can improve. You can make one adjust its own prompt. But the improvement is limited to the context window. It’s not far off from human improvement. Our improvement is limited to what we can remember as well. We go a bit further in the sense that the neural network itself can grow new modules.
  • wat10000 1y ago
    It's radically different from human improvement. Imagine if you were handed a notebook with a bunch of writing that abruptly ends. You're asked to read it and then write one more word. Then you have a bout of amnesia and you go back to the beginning with no knowledge of the notebook's contents, and the cycle repeats. That's what LLMs do, just really fast. You could still accomplish some things this way. You could even "improve" by leaving information in the notebook for your future self to see. But you could never "learn" anything bigger than what fits into the notebook. You could tell your future self about a new technique for finding integrals, but you couldn't learn calculus.
• throwawaymaths 1y ago
  what is there to improve? the transformer architecture is extremely simple. you gonna add another kv layer? you gonna tweak the nonlinearities? you gonna add 1 to one of the dimensions? you gonna inject a weird layer (which could have been in the weights anyways due to kolmogorov theorem)? realistically the best you could do is evolve the prompt. maybe you could change input data preprocessing? anyways the idea of current llm architectures self-improving via its own code seems silly as there are surprisingly few knobs to turn, and it's ~super expensive to train. as a side note it's impressive how resistant the current architecture is to incremental RL away from results, since if even one "undesired input" result is multiple tokens, the coupling between the tokens is difficult to disentangle. (how do you separate jinping from jin-gitaxias for example)
  • amelius 1y ago
    Id like to see what happens if you change the K,V matrix into a 3 dimensional tensor.
• belter 1y ago
  The proof they are not "smart" in the way intelligence is normally defined, is that the models need to "read" all the books in the world. To perform at a level close to an expert on the domain, who read just two or three of the most representative books on his own domain. We will be on the way to AGI when your model can learn Python just by reading the Python docs...Once...
• lawlessone 1y ago
  I agree , it might incrementally optimize itself very well, but i think for now at least anything super innovative will still come from a human that can think beyond a few steps. There are surely far better possible architectures, training methods etc that would initially lead to worse performance if approached stepwise.
  • codr7 1y ago
    Yeah, anyone who's seen it trying to improve code could tell you what that optimization looks like. Oh, this part is taking too long, let's replace it with an empty function. Oh wait, now it's not working, let's add the function. Oh, this part is taking too long... It would be hilarious if this world wasn't full of idiots.
• iknownothow 1y ago
  Don't take this the wrong way, your opinion is also vibes. Let's ground that a bit. Have a look at ARC AGI 1 challenge/benchmark. Solve a problem or two yourself. Know that ARC AGI 1 is practically solved by a few LLMs as of Q1 2025. Then have a look at the ARC AGI 2 challenge. Solve a problem or two yourself. Note that as of today, it is unsolved by LLMs. Then observe that the "difficulty" of ARC AGI 1 and 2 for a human are relatively the same but challenge 2 is much harder for LLMs than 1. ARC AGI 2 is going to be solved *within* 12 months (my bet is on 6 months). If it's not, I'll never post about AI on HN again. There's only one problem to solve, i.e. "how to make LLMs truly see like humans do". Right now, any vision based features that the models exhibit comes from maximizing the use of engineering (i.e. applying CNNs on image slices, chunks, maybe zooming and applying ocr, vector search etc), it isn't vision like ours and isn't a native feature for these models. Once that's solved, then LLMs or new Algo will be able to use a computer perfectly by feeding it screen capture. End of white collar jobs 2-5 years after (as we know it). Edit - added "(as we know it)". And fixed missing word.
  • artificialprint 1y ago
    If you listen interview with Francois it'll be clear to you that "vision" in the way you refer it, has very little do to with solving ARC. And more to do with "fluid, adaptable intelligence, that learns on the fly"
    • iknownothow 1y ago
      That's fair. I care about the end result. The problem is about taking information in 2D/3D space and solving the problem. Humans solve these things through vision. LLMs or AI can do it using another algorithm and internal representation that's way better. I spent a long time thinking about how to solve the ARC AGI 2 puzzles "if I were an LLM" and I just couldn't think of a non-hacky way. People who're blind use braille or touch to extract 2D/3D information. I don't know how blind people represent 2D/3D info once it's in their brain.
      • jplusequalt 1y ago
• AndrewKemendo 1y ago
  > I don’t think they can yet self improve exponentially without human intuition yet Who is claiming anything can self improve exponentially?
• alex-moon 1y ago
  The wall is training data. An AI can't produce its own training data because an AI can't be smarter than its own training data. This is a well known regression problem and one I personally believe is not solvable. (A softer assertion would be: it's not solvable with current technology.)
  • rxtexit 1y ago
    I use to think this but no one I have read believes data is the problem. Amodei explains that if data, model size and compute scale up linearly, then the reaction happens. I don't understand why data wouldn't be a problem but it seems like if it was, we would have ran into this problem already and it has already been overcome with synthetic data.
• larrydag 1y ago
  That would be something. When a AI/LLM can create new axioms or laws that have not discovered by humanity.
• cyanydeez 1y ago
  most of the limits arw likely going to be GIGO, the same as using synthetic training data.